Dataffect: Numerical Epistemology and the Art of Data Sonification

Abstract

This article examines the history of sonification in sound art, focusing on the role that data play in influencing artistic creation and aesthetic experience. The author discusses sonified data artworks that go beyond the simple representation of information and that offer critiques of what Horkheimer and Adorno described as the dehumanizing notion of equivalence at the heart of the bureaucratic, capitalist economy. Concluding with a discussion of his installation Seismology as Metaphor for Empathy (2012), the author suggests that representing data through sound can engender powerful affective responses to the cold abstraction of information.

Supplemental materials such as audio files related to this article are available at <www.mitchellakiyama.com/filter/Projects/Seismology-as-Metaphor-for-Empathy>

Writing against a backdrop of world war, Max Horkheimer and Theodor Adorno looked back on the legacy of the Enlightenment and raged that reason had done little to bring about a better world. Instead, the rationalization of knowledge had led to a “disenchantment” whose instruments—the abstract equivalence manifest in capitalist exchange, data, technocracy—had expedited mass violence. “Bourgeois society,” they wrote, “is ruled by equivalence. It makes dissimilar things comparable by reducing them to abstract quantities” [1]. People and things, they lamented, had become little more than informational units whose meaning was given by tabulation and algorithmic analysis rather than by myth and narrative [2]. There was a monstrous end to this shift: Having become abstract, having become little more than abject points of data, European Jews could be indexed by the Nazis (with the assistance of IBM) and liquidated with an efficiency only possible thanks to computerized logistics [3].

Data, more than ever, are the matrix through which human experience is filtered. Since the 1960s, artists have increasingly turned to data as a medium, creating sound art, Internet-based works, installations, sculptures and performances that grapple with the reality of numerical encoding, wrestling with the fact that now most everyday operations and procedures are in some way digitized, stored, transmitted [4]. In the context of the heterogeneous array of sonic practices that constitute the field of sound art, “sonification” has become the catchall term to describe the transformations of the “relationships in data or information into sound(s) that exploit the auditory perceptual abilities of human beings such that the data relationships are comprehensible” [5]. Artists have transposed weather patterns into immersive installations, used DNA sequences as the basis of generative musical compositions and translated the physical features of a roadway into sonic feedback, to list but a few attempts to use sound to make the politics and concerns implicit in data resonate [6].

What is significant about sonic data art is its potential to affect a listener by engendering unexpected relations among events/sites/things, by modulating/intensifying/collapsing these relations in ways that are only possible through computational processes [7]. In the transduction of raw data into sonic events, there lies the potential for apparently unrelated bodies/concerns to be made proximate to each other, to resound in excess of their relations to the data that subtend their manifestation [8]. At their most vibrant, artworks produced from sonified data create conditions in which the perniciousness of capitalist equivalence might be subverted or undone, engendering empathy or horror by playing on sound’s ability to both “modulate mood” and function as an “intermediary” between varying regimes of sensory experience and information [9].

Fig. 1.

John Cage, Atlas Eclipticalis, 1961–1962. (Copyright © 1961 by Henmar Press, Inc. All rights reserved.)

Used by permission of the C.F. Peters corporation.

Precursors/Post-Cursor

While a long lineage of philosophers, composers and tinkerers stretching back at least as far as Pythagoras has keenly explored the relationships among numbers, physical phenomena and sound, it was not until the 1960s that artists began translating nonacoustic phenomena into audible experience with the objective of calling attention to the transformation itself. Works such as John Cage’s Atlas Eclipticalis (1961–1962) (Fig. 1)—which plotted star charts onto the conventional musical staff—and Alvin Lucier’s composition Music for Solo Performer (1965...