Introduction for Thinking Through Data

Introduction

THINKING THROUGH DATA

· · · A politician brings a chart to Parliament, which shows that the economy is growing slower than expected—we need to cut costs. On another screen, another graph shows the kind of growth that is cause for concern: global CO₂ emissions are not slowing down—we need to act now. Statistics show that the number of refugees from North Africa and the Middle East is in the millions—where can they go? Data and numerical representations are ubiquitous in modern society, and we meet them in a range of platforms and in many contexts: in financial sector predictions, in political debates, in insurance and risk management, in immigration policies, biometrics, medicine, and more. Statistical entities help us make sense of connections that would otherwise seem random and see patterns that would otherwise be invisible. Yet, we often forget to ask: What does it mean to see the world through a curve? How do data shape our thinking about the world around us?

Statistical entities such as averages, aggregates, distributions, outliers, and patterns seem to hover between several domains: they are neither invented nor discovered, they are neither real nor metaphysical. On the one hand, they are “creatures of classification and calculation, of conventions of coding, modeling, and sampling. It is the artifice of definition that makes them cohere—or unravel,” as science historian Lorraine Daston states.¹ They acquire significance and strength in specific historical circumstances and pass away in others. On the other hand, statistical entities are also robust. The search for averages, as the statistician Adolphe Quetelet led from around the 1830s and onwards, was not only influential for the natural sciences, but also marked the beginning of an extensive interest in regularities in statistics—in demography, criminology, and medicine—from the perspective of large populations and the law of large numbers.² Mathematician and physicist Carl Friedrich Gauss’s famous distribution, the one sometimes referred to as “the normal distribution,” would turn out to be not just one of the most important and frequently occurring probability distributions in statistics, but also a completely novel way of approaching the world through data, where measurements from different contexts distribute in the same way, as a bell curve around an estimated mean value.³ The search for orderly shapes in a chaotic world had begun.

The foregoing questions are addressed through a discussion of the status of knowledge production as a fundamentally mediated and aesthetic phenomenon. Knowledge is mediated, as mathematical and statistical thinking rely on abstract models and symbolic systems that decisively affect and shape analytical outcomes. When the bell shape of a curve frames analyses and interpretations of complex phenomena such as economics or demographics, most measurements are expected to distribute around a mean, with rare exceptions on the tail ends to either side. This expectation ultimately affects what is gathered, what is measured, and what is sought: it includes those aspects that conform to the bell shape, while excluding those that do not. As such, this modeled view of the measured input should be understood as more than a mere vehicle of information transfer, along the lines of classical theories of communication. As media theorist Liv Hausken argues, it is necessary to move beyond the idea of a medium as something rather fixed, like an apparatus, and toward conceptions of mediation as a process, as the performance of a task. Such a “shift from medium to mediation does not only represent a shift in focus,” she argues, but also “represents a shift in perspective from medium as an object of study and media as collections of artifacts and technologies, to medium and media as concepts, ideas, models for understanding practices, articulations and experiences.”⁵ By emphasizing such a focus on mediation, this book explores the contexts, discourses, and cultural systems that situate the knowledge production of statistical data processing. It emphasizes that today’s so-called data-driven and empiricist epistemologies cannot be reduced to, or deduced from, the techniques used and the technologies of which they are a part.⁶ In addition to this book’s focus on processes of mediation, it presents a discussion of the status of knowledge production as a fundamentally aesthetic phenomenon. By applying the fundamental instantiations of these abstractions as visualizations and diagrams, it investigates the concrete processes by which knowledge is produced and made generalizable.⁷ In this book, aesthetics is understood as broader than just something pertaining to the realm of art and art criticism.⁸ Instead, it is linked to knowledge production and science more generally and describes a certain way of knowing, a particular attitude,⁹ or disponibility for a critical, aesthetic way of obtaining knowledge.¹⁰ Put differently, this book studies a range of concrete phenomena—graphs, scientific diagrams, mathematical visualizations, and artistic investigations and artworks that address such mathematical and statistical entities—by examining their particular ways of operating aesthetically and uses these insights to investigate the more general mode of operation by which knowledge comes into being. Thus, mathematical and statistical entities present certain insights through complex processes of mediation, and in doing so, they allow scholars to approach the subjects they study. Crystallizing in a particular “line of sight,” as A. S. Aurora Hoel poetically expresses it,¹¹ a graph, diagram, or visualization does not only transmit information, but also expressively conveys meaning, making it visible, and thereby understandable.¹²

The digital object suggested here is not an object in the usual sense, but a theoretical concept set up to investigate a heterogeneous digital field that could not be coherently described in other ways. It may be understood as a conceptual intervention: a (con)cept (from the Greek kope, meaning “oar” or “handle,” or the Latin capere, meaning “to grasp, lay hold, or comprehend”),¹⁵ that is, something to hold on to or grasp. It is analogous to Karl Marx’s conception of political economy as a theoretical object expressed through the interactions of commodities, production, and labor. In this sense, a digital object is not a concrete object such as a compiler, a desktop icon, or a fiberoptic cable, but the thing that digital entities such as outliers, aggregates, and patterns are expressions of, in the same way that for Marx, commodities are expressions of political economy.¹⁶ Similarly to other theoretical objects such as la langue in Ferdinand de Saussure’s framework,¹⁷ which do not exist “out there” in the physical, concrete sense—and seem to elude direct measurement and empirical observation—a digital object also needs to be applied analytically for the purpose of seeing something that would otherwise remain obscure, or to assemble something that would remain fragmented. Just as Marx’s famous critique of political economy identifies important political questions, such as exploitation and alienation, by analyzing the relations between commodities, production, and labor, in this book, analyses of a digital object’s manifestations as outliers, aggregates, and patterns are used to reveal important epistemological questions. Although it is difficult, if not impossible, to study abstract-formal modes of operation directly—political economy as such or the fundamental nature of language—they may be approached by analyzing the ways in which they are realized concretely. Therefore, each of the three main chapters of this book is dedicated to a specific mode of the digital object’s operation—the outlier, the aggregate, and the pattern, respectively—with the purpose of understanding how the digital object influences and shapes modern knowledge production.

The definition of the digital object presented in this book diverges from previous definitions of digital objects by referencing a theoretical concept, rather than a list of things “out there.” It is not a collection or compilation of a range of machines or apparatuses creating meaning through their physical components or material composition as in, for example, Jussi Parikka’s material studies of digital “machines” or Wolfgang Ernst’s media archaeological works.¹⁸ It does not describe a selection of semi-physical formations, composed of data and metadata structured within a computational assemblage, which assume the semblance of objects, as theorized by Yuk Hui in his exploration of digital objects, influenced by the work of Gilbert Simondon and Bernard Stiegler.¹⁹ And it does not equate (big) data to big objects akin to Timothy Morton’s hyperobjects such as global warming, climate, or oil. Instead, the digital object presented in this book transcends ontological delineations, existing foremost as an epistemological construct—a conceptual tool employed to reveal a range of processes or structural connections that take on the form of an object that permeates modern processes of knowledge production in analytical work, theoretical traditions, and concrete applications. It serves as a unit of understanding, facilitating both theoretical analysis and practical application—the ability to transform an experience of the world into an unambiguous digital representation, and to share these digital data with various communities of knowledge, or apply them to politics or economics. The digital object changes our way of seeing the world, whether by transforming the dimensions of a house into a number of standardized feet, the people of a country into a distribution of demographic data, or a person’s face into a robust biometric model.

[C]ultural techniques are (a) operative processes that enable work with things and symbols; (b) they are based on a separation between an implied “know how” and an explicit “know that”; (c) they can be understood as skills that habituate and regularize the body’s movements and that express themselves in everyday fluid practices; (d) at the same time, such techniques can provide the aesthetic and material-technical foundation for scientific innovation and new theoretical objects.²³

Having read Krämer and Bredekamp, one might argue that cultural techniques are situated between process and thing, that is, between established and regulated patterns, shaped and sedimented over time, and more established thing-like formations that, via scientific processes, for example, acquire an aesthetic, or sometimes material, character. This technical-structural interest in cultural phenomena inherent in theories of cultural techniques—particularly in Krämer and Bredekamp’s work—aligns with this book’s aim of developing a digital version of the better-known theoretical object encountered in theories such as Ferdinand de Saussure’s abstract language system or Karl Marx’s political economy. Krämer and Bredekamp state that this is an approach that seeks an understanding of the fundamental “physiognomy of a culture” by investigating the structural operations that regulate and organize things: “The history of culture always already is the history of its cultural techniques,” they write, “just as the history of science cannot be decoupled from the changes in the everyday techniques of perception, communication, representation, archiving, counting, measuring.”²⁴ Cultural techniques define the agency of media, or, in media theorist Cornelia Vismann’s words, “If media theory were, or had, a grammar, that agency would find its expression in objects claiming the grammatical subject position and cultural techniques standing in for verbs.”²⁵

The persistent interest in the technologically mediated grammar of culture is particularly interesting in the context of this book, because it supplements a (historically Anglo-American) focus on the content or meaning of media or things by suggesting that we take a closer look at what media theorist Liam Cole Young calls technical-scientific “ways of doing”: counting, measuring, collecting, observing, playing, confessing, listing.²⁶ What we call media (gramophones, telegraphs, and computers, to use Young’s examples) communicate and order things around them by encoding non-sense into sense (and vice versa). For instance, by translating data into alphanumerical characters, a computer does not only transmit, it also transforms. Cultural techniques—and the concept of the digital object proposed here—assume the position of the parasitic third,²⁷ as media theorist Bernhard Siegert—via philosopher and mathematician Michel Serres—states; that is, a kind of middle or mediating position that operates through a communicative work that disassociates itself from any dichotomic form. In Serres’s model of communication, the fundamental relationship is not between sender and receiver—as it so often is in classical communication models—but between communication and noise.²⁸ For Siegert (and Serres), communication is not primarily information exchange, but an act of ordering that introduces distinctions and differentiations. Siegert’s understanding of cultural techniques is one of interruption, disturbance, and deviation,²⁹ that is, a way of approaching media that aims to create “an awareness of the plenitude of a world of as-yet-undistinguished things that, as an inexhaustible reservoir of possibilities, remains the basic point of reference for every type of culture,” as he writes in his 2015 book, Cultural Techniques: Grids, Filters, Doors, and Other Articulations of the Real.³⁰

Although this book is deeply inspired by theories of cultural techniques, it also differs from them in decisive ways. An example of a technical mode of operation that displays this difference could be listing. As an artifact, the list is found as far back on Ancient Sumerian clay tablets, but it is also present in contemporary computer code. If one addresses listing as a cultural technique, one becomes aware of how it not only distributes data, but also defines certain items, inscribes order, and helps to decide what to include and exclude.³¹ Listing transforms people, words, or things into dynamic units that may be processed, stored, or transmitted. Listing determines which words are significant or redundant when included in searches, for example, when using Google’s search engine.³² Listing encodes things into a symbolic order and thereby subjects them to manipulation, revision, erasure, or reversibility.³³ In many cases, listing is more political³⁴ than one might think: as Young states, this form of protocol “determines how computation unfolds; how a person is listed can determine his or her fate.”³⁵ However, listing, as a technical mode of operation, or a cultural technique in the digital age, also implies a theoretical object that—so to speak—“takes” this technique as its thing. What kind of object enables this particular alphanumerical encoding of the symbolic order? What kind of object translates people, words, or things into units that may be distributed, transformed, or deleted? What kind of thinking goes into a protocol or a list? Listing, as phenomenon, may be studied in several ways: it may be examined through the lens of media archaeology—as a concrete machine or technique that, through material appliances and historical sediments, provides a range of varied, overlooked potential;³⁶it may be investigated in light of systemic or computational operations, such as arrays, queues, and stacks, and in terms of the databases that organize it;³⁷ it may explored in terms of its poetic relation to language, literature, and imagination, as Umberto Eco³⁸ and Jorge Luis Borges³⁹ did; or it may, as mentioned above, be studied as a cultural technique that has a certain kind of medial agency in its fundamental communicative potential. The list as the concrete thing that manipulates, revises, erases, or reverses is decisive for the manifestation of the listing logic (the theoretical object) behind it, which produces its epistemological, symbolic, or political effects. Here, again, it is not so much about the physical configuration of the list—the clay tablet or the code fragments—as it is about the list as grammatical subject that acts on behalf of the more general listing operation, that is, the necessary, yet ephemeral basis for a digital way of thinking, or the appearance of a list as a concrete, aesthetic phenomenon that contributes to knowledge production. This epistemological object is the recurring figure in this book. Conceptually, it should not be understood as a comprehensive categorization, or a typological designation that seeks to conform or sort a range of phenomena. As a contribution, its form is way more abductive; more of an experimental, theoretical, and analytical synthesis than an exhaustive description of an emerging digital field.

Etymologically, the word digital comes from the Latin word, digitus, which may be traced back to the first century BCE, when it meant “finger” or “toe,” or simply the act of counting (as in “counting on your fingers,” because numbers under ten were counted on the fingers). In a letter to his friend Atticus, written between 68 and 44 BCE, the famous lawyer and scholar Cicero complains about how Brutus (Atticus’s friend) wants to charge him 48 percent interest, instead of the usual 12 percent. Offended, Cicero says, “What a wide difference this implies you will certainly be able to reckon, if I know your fingers.” This phrase, “if I know your fingers” (si tuos digitos novi),⁴¹ implies that the use of one’s fingers to count when carrying out more advanced operations, such as calculating of interest and debt, dates to at least 68 to 44 BCE. The fingers were used as concrete, physical units of measurement—to count on and point with—and also facilitated symbolic operations that enabled the negotiation of prices, deals, loans, and so on. The phrase, “if I know your fingers” may be understood to reference a more general, common method known and used by all: that is, a sort of universal system where operators and functions were rather fixed.

The term digital may be traced to the proto-Indo-European root form, deik, which signifies “to show.” This root, highlighted in the American Heritage Dictionary of Indo-European Roots, is connected to various words, including the Sanskrit dic (meaning “to show” or “to point out”), Greek terms such as deiknynai (“to show” or “to prove”) and dike (“custom” or “usage”), Latin words such as dicere (“to speak, tell, say”) and digitus (“finger”), and Old High German zeigon and German zeigen (“to show”).⁴² This connection, related to a word extensively used in societal, economic, juridical, cultural, and aesthetic contexts, is significant. Beyond its common usage, the deik root conveys an alternate interpretation of digital—one that transcends mere registration or storage of binary-encoded information.⁴³ Instead, it reveals a perspective that emphasizes the contextual representation of the world: it indicates the contextual presentation of the world, that is, to the concrete subjects and objects whose stories are told, and to the aesthetic and cultural situation in which they take part. Conversely, the deuk root, also of proto-Indo-European origin, has a different focus. It indicates the potential for “pulling,” “dragging,” or “leading something away.” This root underpins essential scientific concepts centered on the notion of “leading away” or “deriving.” Concepts such as reduction, induction, deduction, and abduction, integral to computer science and information-oriented research, gain prominence here. Notably, induction, deduction, and abduction have achieved distinction through their application in the work of philosopher and mathematician Charles Sanders Peirce.

In his Lowell lectures of 1866, “The Logic of Science; or Induction and Hypothesis,” Peirce describes the complicated relations between objects and signs. He believes that logic, in the broadest sense of the word, is closely related to the ways in which things are represented, that is, with the general ways in which signs operate.⁴⁴ In his lectures, he connects two of his best-known triadic models: the three trichotomies of the sign—iconic (similar), indexical (causal), and symbol (conventional) representations—and his classification of reasoning in a schematic diagram that describes the fundamental relations between representamen and object. Abduction (which Peirce calls hypothesis at this point)⁴⁵ is linked to icons (through likeness), induction to indices, and deduction to symbols. He writes:

We come to [ . . . ] the argument. [—] It will therefore be divided into three species according as this representation is a likeness, index, or symbol. These three species are the same as Hypothesis, Induction, and Deduction. Hypothesis brings up to the mind an image of the true qualities of a thing—it therefore informs us as to comprehension but not as to Extension, that is it represents a representation which has Comprehension without Extension; in other words it represents a likeness.⁴⁶

The representational relations between signs and things are fundamentally connected to the three modalities of the deuk root—according to Peirce, to abduction, deduction, and induction—and may be expressed as shown in table 1.

TABLE 1. The three modalities of the deuk root, as they relate to C. S. Peirce’s three trichotomies of the sign and triadic classification of reasoning.

These modalities share the fact that their semiotic and representational qualities relate them to acts of derivation, etymologically marked by the deuk root. In other words, they illustrate an approach that derives by either explaining a new connection (abducing), generalizing from observations (inducing), or inferring from theory (deducing). When linked to the three trichotomies of the sign, these derivations may be expressed a little differently, yet still focus on their inferring role in reasoning: here, the interest is directed by relations of likeness, for instance, between a thing and an image (icons), relations of causality between something already known and its traces (indices), or relations of convention between something observed many times and its possible meanings (symbols). All these modalities derive something from the things they describe, and may again be connected to the deuk root form, and the technical or informational interest in derivation, that is, in the relations between representation and thing in counting.

An approach based on the alternative etymological connotation of the deik root form disrupts the meaning of digital. Here, the focus is less on “leading away,” or “deriving,” and more on “showing,” for instance, what happens, the ways in which it is experienced, and the position from which something is seen. This difference may be tentatively visualized as shown in table 2.

TABLE 2.Schematic visualization of deuk and deik forms as they relate to C. S. Peirce’s three trichotomies of the sign and triadic classification of reasoning and the etymology of the word digital.

Although the deuk forms are built on derivation, that is, an approach that extracts and “leads away”—operating on representations of the subjects studied—the deik forms are built on presentations, that is, on an approach that focuses on the processes and contexts that include the subjects. Translated into a digital vocabulary, a shift from representation to presentation marks a shift in interest from a focus on formalizing or abstracting the meaning of digital phenomena, that is, from their symbolic value, to a focus on displaying, presenting, or contextualizing how they operate more generally, for example, culturally or aesthetically.

Although a focus on the abstract, informational level of representation is both relevant to and necessary for many professional disciplines, a more process-oriented concept may help to provide new analytical insights. Instead of focusing exclusively on digital phenomena as logical-mathematical and informational, derived from the surrounding world as sets of data, they may also be approached as something to be perceived with the senses, as something to be experienced in a certain way in a specific setting. When Cicero counts on his fingers, finds that the interest is too high, and complains to his friend Atticus, it is precisely this choice of approach that is evident: the symbolic universal system, which they both use, may presumably be analyzed on the basis of its logical-mathematical qualities as a method for calculating interest, yet it may also be discussed in relation to the situation in which Cicero finds himself, as he looks at his fingers and does not understand why they—these fingers that usually provide an accurate method of counting, that usually provide him with a way to calculate, discuss, and visualize all kinds of mathematical matters—suddenly fail him. The universal system is a method that, like so many other things, is restricted to the context in which it operates, a context that makes some calculations possible while excluding others, that deceives one part while letting the other win.

This book aspires to follow a more deictic approach by describing the actual experiences and contexts that form the basis of cultural techniques of counting and sorting. It dives into the particularities of digital culture: into the concrete practices of data processing as they manifest in census-taking, punched-card operations, statistical summaries and models, face recognition aggregates, and more. Just as Peirce speaks of a “thirdness” or middle of communication, this book aspires to begin from the “middle of things,”⁵⁰ rather than “from above.”⁵¹ Therefore, this book’s descriptions of the central term digital are made with concrete digital operations and their relational (social, cultural, and aesthetic) influence in mind. Thus, digital has two meanings: it identifies the specific contexts and situations in which digital calculations are actualized and—at the same time—synthesizes the disparate array of ways in which this actualization is carried out. It covers a range of ways in which discontinuous data are displayed and described by statistical concepts and visualizations, and the particular, theoretical mindset or way of thinking that underpins these expressions. Whether they are mathematical, statistical, or philosophical, theoretical maneuvers always presuppose an object of thought. The central object of thought in this book—the digital object—is not the pixelated representation on the screen or the lithium battery that supports its appearance; instead, it is a new type of object that finds its way into everyday life, into political decision-making, into research politics, into social spaces, and into aesthetics. It is not a new class of objects that needs to be covered or schematized, but a reaction to a need to reflect on the digital operations that renegotiate so many things around us, from the largest, general structures of global politics to the smallest, most intimate relationships.

The first chapter focuses on the statistical outlier and the data set values that are intentionally excluded and marginalized. The outlier, as figure, is addressed on two levels: first, on a concrete level, where the historical application and understanding of data processing is studied, and secondly, on a more general level, where exclusion is understood culturally, politically, and aesthetically, based on omissions of context and metadata, or of certain historical events or individuals, for example. In this chapter, a central argument is that outliers play a decisive role in the history of data processing and also in the history of the computer and the introduction of digital systems into modern society. Accordingly, this chapter explores, first, how concrete strategies for homogenizing data sets moderate and mark the study output and secondly, how exclusions, more generally and literally, make some forms of knowledge visible and others invisible. The central artistic exploration included in this chapter is the artist Rossella Biscotti’s Other project (2014–2015), in which she focuses on the institutional dimensions of outliers (aesthetic, social, economic) and on the technological structures that facilitate their operations (machines, buildings, and techniques). Several large textile pieces produced on automatic Jacquard looms study and present the individuals excluded from comprehensive demographic surveys undertaken in Belgium in the first decade of the 2000s. By employing the textile’s medium-specific character, Biscotti visualizes not only the embedded statistical outliers, but also the actual process that yields the censuses’ specific results.

The second chapter focuses on the theoretical and historical idea of statistical aggregation as a mode of reasoning that selects, combines, and merges to make sense of data sets. In the nineteenth century, the word aggregation denoted the combination of observations, a definition that conveys the idea that there is a gain in information to be had, beyond what is revealed by the individual values of a data set, by combining them into a statistical summary. A central argument in this chapter is that combination, as a central mode of operation, contributes to understanding how the aesthetic processes inherent in presentations of data not only make it possible to manage the data sets, but also to make them visible in the first place. As such, the “data presentations” created by aggregation are described as a special kind of operative image.⁵⁵ Through an aesthetic close reading of Adam Broomberg and Oliver Chanarin’s artistic project, Spirit Is a Bone (2014), this chapter focuses on the human face as it is translated into computational data. It thematizes the limitations of what a face is, as it challenges fundamental ideas of subjectivity: When does a face cease to look like itself? What are the most important characteristics of a face? What is noise, and how is one to identify the most essential parts of a face?⁵⁶

The third chapter focuses on the philosophical, technical, and aesthetic idea of the pattern and with it, on more general discussions of so-called data-driven knowledge production⁵⁷ and techniques and methods, which convey the process of pattern recognition and abstract comparison as such.⁵⁸ This chapter argues that knowledge does not just emerge from analytically applied concepts or models,⁵⁹ or from the data set “itself,”⁶⁰ but emerges from a mediated interplay between context and conventions: concrete visualizations and diagrammatic abstractions.⁶¹ With new methods for data processing that include machine learning and other types of “intelligent” systems, new statistical objects emerge, new classifications become possible, and new virtual connections are created. The third chapter’s point of departure is Stéphanie Solinas’s art project Dominique Lambert (2004–2016) and the numerous metatheoretical and methodological questions of pattern recognition, comparison, and comparability to which it gives rise. In this chapter, comparison is broadly understood in terms of discussions of the challenges and possibilities attached to ideas of comparatism as a methodological strategy in art and literature analyses and as a broader theoretical and scientific approach. The Dominique Lambert project questions not only how and when different things may be compared, but also how patterns and comparisons may be understood more generally, and how they are legitimatized in relation to humanistic knowledge production.

Notes

1. Lorraine Daston, “Why Statistics Tend Not Only to Describe the World But to Change It,” London Review of Books 22, no. 8 (2000), 35–36.

2. The law of large numbers, first propounded by Italian mathematician Jacob Bernoulli in 1713, and further described by French mathematician and physicist Siméon Denis Poisson in 1837, suggests that as a sample size grows, its mean moves closer to the average of the whole population. This idea has had a profound influence on statistics, particularly in relation to its applications to sociology and political economy in the nineteenth century, a discussion I go into in chapter 2.

3. The Gaussian distribution, also known as “the normal distribution,” is a widely used model for the distribution of continuous variables, e.g., Christopher M. Bishop, Pattern Recognition and Machine Learning, Information Science and Statistics (New York: Springer, 2006), 78–79. Although some credit the mathematician de Moivre with formulating the normal distribution, it is most often associated with the German mathematician and scientist Carl Friedrich Gauss, who introduced this important statistical concept in his 1823 monograph, Theoria combinationis observationum erroribus minimis obnoxiae (“Theory of the Combination of Observations Least Subject to Error”), where, among other things, the normal distribution is formulated. See Carl Friedrich Gauss, Theory of the Combination of Observations Least Subject to Errors, trans. G. W. Stewart, Classics in Applied Mathematics (Philadelphia: SIAM, 1995).

4. In his 2014 article, Rob Kitchin examines what he identifies as the new “empiricist epistemologies” of modern science, which “declare ‘the end of theory,’ the creation of data-driven rather than knowledge-driven science, and the development of digital humanities and computational social sciences that propose radically different ways to make sense of culture, history, economy and society” (my emphasis). See Rob Kitchin, “Big Data, New Epistemologies and Paradigm Shifts,” Big Data & Society 1, no. 1 (2014): 1. He argues that the emergence of Big Data and new data analytics are disruptive innovations, which reconfigure how research is conducted, and he calls for broader critical reflection on the part of the academic disciplines working with data processing concerning the epistemological implications of the developing data revolution. This book is intended to contribute to such a critical endeavor.

5. Liv Hausken, Thinking Media Aesthetics: Media Studies, Film Studies and the Arts (Frankfurt am Main: Peter Lang International Academic Publishing Group, 2013), 31.

6. As Lotte Philipsen and Rikke Schmidt Kjærgaard state, “Representation, scientific data representation, too, is inevitably a matter of aesthetics since all representations are created and shaped under human influence. Even a graph on a monitor has at least shape, color, position, and size, and however advanced devices we may use in the process, these representational features are culturally and technically constructed by humans”; see Lotte Philipsen and Rikke Schmidt Kjærgaard, The Aesthetics of Scientific Data Representation: More Than Pretty Pictures (New York: Routledge, 2018), xii.

7. A. S. Aurora Hoel [formerly Aud Sissel Hoel], “Lines of Sight: Peirce on Diagrammatic Abstraction,” in Das Bildnerische Denken: Charles S. Peirce, ed. Franz Engel, Moritz Queisner and Tullio Viola (Berlin/Boston: De Gruyter, 2012).

8. This book draws on an understanding of aesthetics that originates in the concept introduced in 1735, when German philosopher Alexander Gottlieb Baumgarten defined it in his master’s thesis, Meditationes philosophicae de nonnullis ad poema pertinentibus [Philosophische Betrachtungen über einige Bedingungen des Gedichtes], to mean epistêmê aisthetikê, or the science of what is sensed and imagined; see Alexander Gottlieb Baumgarten, Reflections on Poetry: Alexander Gottlieb Baumgarten’s Meditationes philosophicae de nonnullis ad poema pertinentibus, ed. Karl Aschenbrenner and William B. Holther (Berkeley: University of California Press, 1954), 86–87.

9. Although the idea of an aesthetic “attitude” has been hotly debated, particularly as it is understood in aesthetic attitude theories (e.g., George Dickie, “The Myth of the Aesthetic Attitude,” American Philosophical Quarterly (Oxford) 1, no. 1 [1964]; for a response to this critique, see G. Kemp, “The Aesthetic Attitude,” British Journal of Aesthetics 39, no. 4 [1999]). It is relevant to this topic to discuss how an observer addresses and experiences aesthetic phenomena. Therefore, the attitude adopted in this book is one of searching for aesthetic features in many kinds of objects, and not just art works or traditional aesthetic objects (or even beautiful objects). By focusing on aesthetic features of a scientific diagram or census-taking model, for example, I seek to understand how knowledge is produced along the lines of particular instantiations of abstract thinking. For an in-depth discussion of the term attitude and its problems as philosophical term, see Alexandra King, “The Aesthetic Attitude,” in The Internet Encyclopedia of Philosophy, https://iep.utm.edu/aesthetic-attitude/.

10. Jørn Erslev Andersen, Sansning og erkendelse: Æstetikhistoriske grundtekster fra Baumgarten til Kant (Aarhus: Aarhus Universitetsforlag, 2012), 237.

11. Hoel, “Lines of Sight.”

12. Hoel, 268.

13. To paraphrase Michel Foucault’s definition of the apparatus (dispositif), as quoted by Giorgio Agamben, “[t]he apparatus is thus always inscribed into a play of power, but it is also always linked to certain limits of knowledge that arise from it and, to an equal degree, condition it. The apparatus is precisely this: a set of strategies of the relations of forces supporting, and supported by, certain types of knowledge.” Giorgio Agamben, What Is an Apparatus? And Other Essays (Stanford, CA: Stanford University Press, 2009), 2.

14. Agamben, What Is an Apparatus? And Other Essays, 2.

15. Douglas R. Harper, “concept (n.),” in Online Etymology Dictionary (Tupelo, Mississippi, 2021), https://www.etymonline.com/word/concept.

16. See Karl Marx and Frederick Engels, Collected Works of Karl Marx and Frederick Engels, vol. 28 (London: Lawrence & Wishart Electric Books, 2010), 49.

17. See Ferdinand de Saussure, Course in General Linguistics, ed. Charles Bally, Albert Sechehaye, and Albert Reidlinger, trans. Wade Baskin (New York: Philosophical Library, 1959), 7–8.

18. For a great introduction to Wolfgang Ernst’s materialist approach to media theory and history, see Wolfgang Ernst and Jussi Parikka, Digital Memory and the Archive (Minneapolis: University of Minnesota Press, 2012). For an example of Jussi Parikka’s materialist approach, see Jussi Parikka, A Geology of Media, Electronic Mediations (Minneapolis: University of Minnesota Press, 2015).

19. See Yuk Hui, On the Existence of Digital Objects, Electronic Mediations (Minneapolis: University of Minnesota Press, 2016).

20. Kulturtechniken has been translated into English in many ways over the years: “cultural technologies,” “cultural techniques,” and “culture technics” (with and without a hyphen). As Geoffrey Winthrop-Young mentions in his translator’s note in Bernhard Siegert’s book, Cultural Techniques: Grids, Filters, Doors, and Other Articulations of the Real, the greatest dilemma in this translation is the word Technik, as its connotations range from “gadgets, artifacts, and infrastructures all the way to skills, routines, and procedures—it is thus wide enough to be translated as technology, technique, or technics.” Bernhard Siegert, Cultural Techniques: Grids, Filters, Doors, and Other Articulations of the Real, trans. Geoffrey Winthrop-Young (New York: Fordham University Press, 2015), xv. Winthrop-Young opts for “techniques,” and manages to encompass “drills, routines, skills, habituations, and techniques as well as tools, gadgets, artifacts, and technologies,” as possible associations, and, arguably, “cultural techniques” is the most appropriate term. It is the one I use in this book.

21. This particular inflection of the term references its etymological roots in culture, and derives from the Latin cultura, which in turn derives from colere (meaning “to tend, guard, cultivate, till”).

22. Geoffrey Winthrop-Young, Ilinca Iurascu, and Jussi Parikka, “Cultural Techniques,” Theory, Culture & Society 30, no. 6 (2013): 5.

23. The quote is from a translated and revised edition of the original 2003 article published as “Kultur, Technik, Kulturtechnik: Wider die Diskursivierung der Kultur,” in Sybille Krämer and Horst Bredekamp, eds. Bild, Schrift, Zahl (Munich: Fink, 2003): 11–22. The English translation, “Culture, Technology, Cultural Techniques—Moving Beyond Text,” is provided by Michael Wutz and appeared in Theory, Culture & Society 30, no. 6 (2013), quote on page 27.

24. Krämer and Bredekamp, “Culture, Technology, Cultural Techniques—Moving Beyond Text,” 25.

25. Cornelia Vismann, “Cultural Techniques and Sovereignty,” Theory, Culture & Society 30, no. 6 (2013): 83.

26. Liam Cole Young, “Cultural Techniques and Logistical Media: Tuning German and Anglo-American Media Studies,” M/C journal 18, no. 2 (2015).

27. Bernhard Siegert, “Cultural Techniques: Or the End of the Intellectual Postwar Era in German Media Theory,” Theory, Culture & Society 30, no. 6 (2013): 61–62.

28. Michel Serres, The Parasite, trans. Lawrence R. Schehr (Baltimore: Johns Hopkins University Press, 1982), 13.

29. Siegert, Cultural Techniques, 23.

30. See Siegert, 23.

31. See Young, “Cultural Techniques and Logistical Media.” See Cornelia Vismann, Files: Law and Media Technology, trans. Geoffrey Winthrop-Young, Meridian, Crossing Aesthetics (Stanford, CA: Stanford University Press, 2008), 5–6.

32. See Daniel Rosenberg, “Stop, Words,” Representations 127, no. 1 (2014).

33. See Young, “Cultural Techniques and Logistical Media.” See Sybille Krämer, “The Cultural Techniques of Time Axis Manipulation: On Friedrich Kittler’s Conception of Media,” Theory, Culture & Society 23, no. 7/8 (2006).

34. Here, “politics” is closely related to the sensorial, that is, to the realm of aesthetics. Politics as an aesthetically mediated phenomenon may be understood in relation to the definition proposed by Jacques Rancière: “This aesthetics should not be understood as the perverse commandeering of politics by a will to art, by a consideration of the people qua work of art. . . . [A]esthetics can be understood in a Kantian sense re-examined perhaps by Foucault—as the system of a priori forms determining what presents itself to sense experience. It is a delimitation of spaces and times, of the visible and the invisible, of speech and noise, that simultaneously determines the place and the stakes of politics as a form of experience. Politics revolves around what is seen and what can be said about it, around who has the ability to see and the talent to speak, around the properties of spaces and the possibilities of time.” See Jacques Rancière, The Politics of Aesthetics: The Distribution of the Sensible (London: Continuum, 2004), 13.

35. Young, “Cultural Techniques and Logistical Media,” np.

36. E.g., Lisa Gitelman, Paper Knowledge: Toward a Media History of Documents, Sign, Storage, Transmission, (Durham, NC: Duke University Press, 2014). See Jussi Parikka, What Is Media Archaeology? (Cambridge: Polity Press, 2012).

37. E.g., Alison Adam, “Lists,” in Software Studies: A Lexicon, ed. Matthew Fuller (Cambridge: MIT Press, 2008), 174–78. Another source is Vismann, Files, 163.

38. Umberto Eco, The Infinity of Lists, trans. Alastair McEwen (New York: Rizzoli, 2009), 7.

39. Jorge Luis Borges, Other Inquisitions, 1937–1952, trans. Ruth L. C. Simms (University of Texas Press, 1964), 103.

40. When representing things other than numbers, these must first be encoded, i.e., transformed into a numerical representation that may be decoded into the original input with the correct approach. For example, the letters of the alphabet have a default order and may be encoded by counting a as 1, b as 10 (2), c as 11 (3) and so on. More complex objects, such as pictures, are often encoded recursively, e.g., with each pixel encoded separately as a series of three numbers (red, blue, green), and the entire picture is encoded as a series of pixels with added metadata that outlines the dimensions of the picture to be used when decoding.

41. Marcus Tullius Cicero, Letters to Atticus, trans. E. O. Winstedt (London: William Heinemann, 1913), 412.

42. See Calvert Watkins, The American Heritage Dictionary of Indo-European Roots (Boston: Houghton Mifflin, 1985), and Douglas R. Harper, “digit (n.),” in Online Etymology Dictionary (Tupelo, Mississippi, 2021).

43. The recent explosion in companies that deliver data-driven products has made data comparable in value to oil or even the very rays of the sun. See “The World’s Most Valuable Resource Is No Longer Oil, but Data,” The Economist (London), May 6, 2017, or Ludwig Siegele, “Are Data More Like Oil or Sunlight?” The Economist (London), February 20, 2020.

44. Robert Burch, “Charles Sanders Peirce,” in The Stanford Encyclopedia of Philosophy, ed. Edward N. Zalta (Stanford, CA: Metaphysics Research Lab, Stanford University, 2022).

45. Igor Douven, “Abduction,” in The Stanford Encyclopedia of Philosophy.

46. Charles S. Peirce, “Lowell Lectures on The Logic of Science; or Induction and Hypothesis: Lecture IX,” in Writings of Charles S. Peirce. Volume 1, 1857–1866: A Chronological Edition, ed. Max H. Fisch (Bloomington: Indiana University Press, 1982).

47. Keith Green, “Deixis and Anaphora: Pragmatic Approaches,” in Encyclopedia of Language and Linguistics, 2nd ed., ed. Keith Brown, 415–17 (Oxford: Elsevier, 2006), https://doi.org/10.1016/B0-08-044854-2/00328-X.

48. Maja Bak Herrie, “Unddragelsens kunst. Hermetiske objekter og deiksis uden kontekst i Gertrude Steins ‘Tender Buttons,’” Passage 32, no. 77 (2017).

49. When speaking of a certain “deictic orientation” and even applying it as a sort of counterconcept to mathematical or technical understandings of digital phenomena, an author must have certain reservations. When I transfer this concept from its traditional use in the field of linguistics, I risk undermining its cogency and being imprecise. In linguistics, deixis is primarily used as a technical concept to describe certain terms, whereas the way I use it is metaphorical. I suggest using it to see broader or more general characteristics ascribed to deictic terms, that is, subjective experience on the one hand—a cognitive center of orientation, where language operates and assigns meaning to things and people—and, on the other hand, a “situational ‘co-ordination’” of these people (I/you, us/them), places (here/there, this/that), and times (now/then, yesterday/today); e.g., Chris Baldick, “deixis,” The Oxford Dictionary of Literary Terms (Oxford University Press, 2015); Green, “Deixis and Anaphora.” Although this transfer from one discipline to another may cause certain problems, I nonetheless choose to draw on this theoretical resource, because in the same way that linguistic organizations of things, people, and places in time and space require mutual attention to contextual and situational conditions, and to the experimental orientation according to which the linguistic inquiry is received, digital representations and presentations also need to address how data sets and models relate to the surrounding world, and how its abstractions are received and interpreted.

50. Particularly in his later work, Maurice Merleau-Ponty criticizes the Cartesian legacy, which is also supplemented by a criticism of Jean-Paul Sartre’s thinking. Sartre is wrong to separate subjects and objects according to a sharp distinction between the “in-itself” and “for-itself.” Merleau-Ponty maintains that Sartre conceives subjectivity as holding being before itself, as a spectacle, and, hence, as not operating “from the middle of things.” This contrasts with Merleau-Ponty’s project, which specifically explores the in-betweenness, that is, the lived relations in which humans are embedded. See Maurice Merleau-Ponty, “La Nature ou le monde du silence: Pages d’introduction,” in Maurice Merleau-Ponty, ed. Emmanuel de Saint Aubert (Paris: Hermann, 2008), 48. For an in-depth discussion of this idea of a middle, or “in-betweenness,” in Merleau-Ponty’s work, see A. S. Aurora Hoel and Annamaria Carusi, “Merleau-Ponty and the Measuring Body,” Theory, Culture & Society 35, no. 1 (2018).

51. A central concept in Alfred North Whitehead’s writings is the so-called bifurcation of nature, explained in The Concept of Nature, where he writes, “For natural philosophy everything perceived is in nature. We may not pick and choose. For us the red glow of the sunset should be as much part of nature as are the molecules and electric waves by which men of science would explain the phenomenon.” See Alfred North Whitehead, The Concept of Nature (Cambridge: Cambridge University Press, 1920), 29. In other words, this bifurcation of nature implies a way of thinking that divides the world in two: one that is composed of the fundamental constituents of the universe—invisible to the naked eye, but accessible with the right tools of measurement—and the other, which is constituted of what the mind must add to these basic building blocks of the world, for it to make sense; see Isabelle Stengers, Thinking with Whitehead: A Free and Wild Creation of Concepts (Cambridge, MA: Harvard University Press, 2011), xii. According to Whitehead, although the natural sciences start with the first—the molecules and the electrical waves, that is, from below—the social sciences and humanities start from above, from the redness of the sunset and its warmth on our bodies. However, the problem with this kind of thinking is that it challenges basic philosophical questions, e.g., about the status of the mind or the nature of subjective experience. “If nature really is bifurcated, no living organism would be possible, since being an organism means being the sort of thing whose primary and secondary qualities—if they exist—are endlessly blurred. Since we are organisms surrounded by other organisms, nature has not bifurcated,” Bruno Latour writes in his foreword to Stenger’s book, see Stengers, Thinking with Whitehead, xiii. Latour mentions this division again in his translation of Whitehead’s terms into his own pair of concepts, matters of fact and matters of concern. With these concepts, he argues that we should approach things in their double meaning, from the Old English and German Ding, as both an object out there and a concern: “Icelanders boast of having the oldest Parliament, which they call Althing, and you can still visit in many Scandinavian countries assembly places that are designated by the word Ding or Thing. Now, is this not extraordinary that the banal term we use for designating what is out there, unquestionably, a thing, what lies out of any dispute, out of language, is also the oldest word we all have used to designate the oldest of the sites in which our ancestors did their dealing and tried to settle their disputes? A thing is, in one sense, an object out there and, in another sense, an issue very much in there, at any rate, a gathering. [T]he same word thing designates matters of fact and matters of concern,” see Bruno Latour, “Why Has Critique Run Out of Steam? From Matters of Fact to Matters of Concern,” Critical Inquiry 30, no. 2 (2004): 233.

52. Mieke Bal, Travelling Concepts in the Humanities (University of Toronto Press, 2002), 45.

53. See Juliane Rebentisch, Theorien der Gegenwartskunst zur Einführung (Hamburg: Junius, 2013). For an in-depth discussion of the capability of artworks and exhibitions to produce “reflexive transformations” of otherwise nonartistic issues, see Jacob Lund, “Exhibition as Reflexive Transformation,” OBOE Journal 3, no. 1 (2022): i–x.

54. Although using the concept of knowledge might contribute to a problematic prioritization of discursive over aesthetic or visual approaches, Tom Holert in his 2020 book Knowledge Beside Itself: Contemporary Art’s Epistemic Politics (London: Sternberg Press) nonetheless argues that by engaging in the “the activation, reconstruction, resurrection, recomposition, and invention of ways of knowing and modes of thought that are irreducible to Western rationalism and cognitivism, the aesthetic might be regained as the reservoir and repertoire of a cognition that is based in bodily, sensations, in affect, in empathy” (61). In alignment with this perspective, I use a similar conception of “knowledge production” within this book, viewing knowledge as an epistemic activity encompassing language use, thinking, learning, and archiving, or “social organizing” (45).

55. For an in-depth discussion of this “operative” approach to images, see A. S. Aurora Hoel, “Operative Images. Inroads to a New Paradigm of Media Theory,” in Image–Action–Space: Situating the Screen in Visual Practice, ed. Luisa Feiersinger, Kathrin Friedrich, and Moritz Queisner, 11–28. (Berlin, Boston: De Gruyter, 2018). These ideas draw in part on Harun Farocki’s conception of active images, see Harun Farocki, “Phantom Images,” Public 29 (2004), https://public.journals.yorku.ca/index.php/public/article/view/30354. See also Sybille Krämer, “Operative Bildlichkeit. Von der ‘Grammatologie’ zu einer ‘Diagrammatologie’? Reflexionen über erkennendes Sehen” (Bielefeld: Transcript Verlag, 2015).

56. See Chiara Ambrosio, “Composite Photographs and the Quest for Generality: Themes from Peirce and Galton,” Critical inquiry 42, no. 3 (2016) and Lorraine Daston and Peter Galison, “The Image of Objectivity,” Representations, no. 40 (1992): 98–117.

57. For a discussion of the idea of “data-driven” epistemologies, see Kitchin, “Big Data, New Epistemologies and Paradigm Shifts.” This approach to knowledge production has received extensive criticism, e.g., danah boyd and Kate Crawford, “Six Provocations for Big Data” (paper presented at the A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society, 2011).

58. E.g., Luciana Parisi, “Critical Computation: Digital Automata and General Artificial Thinking,” Theory, Culture & Society 36, no. 2 (2019). See also Pedro Domingos, The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World (New York: Basic Books, 2015); David Bollier and Charles M. Firestone, The Promise and Peril of Big Data (Washington, DC: Aspen Institute, Communications and Society Program, 2010); and Luciano Floridi, “Big Data and Their Epistemological Challenge,” Philosophy & Technology 25, no. 4 (2012).

59. See Simon Aagaard Enni and Maja Bak Herrie, “Turning Biases into Hypotheses through Method: A Logic of Scientific Discovery for Machine Learning,” Big Data & Society 8, no. 1 (2021). For an in-depth discussion of the potential problems of using of machine learning in scientific knowledge production, see Simon Aagaard Enni, “Deliberation and Dissemination in Machine Learning” (Aarhus: Computer Science, Aarhus University, 2021).

60. See Lisa Gitelman, “Raw Data” Is an Oxymoron (Cambridge, MA: MIT Press, 2013).

61. Charles Sanders Peirce, The Collected Papers of Charles Sanders Peirce, electronic ed., ed. Charles Hartshorne, Paul Weiss, and Arthur W. Burks, vol. 5, Pragmatism and Pragmaticism (Cambridge, MA: Belknap Press of Harvard University Press, 1994). See also Hoel, “Lines of Sight.”