International Conference on Auditory Display 2004: Listening to the Mind Listening (review)

Gregory Kramer, in the 1994 book Auditory Display (Santa Fe Institute Studies in the Sciences of Complexity. Boulder, Colorado: Westview Press), pioneered the organization of the field into auditory icons (earcons), audification, and sonification. Most computer users routinely experience earcons while messaging online or emptying their (digital) trash; occasionally there are papers on the subject at the International Conference on Auditory Display (ICAD)—for example, the development of next-generation auditory interfaces for the navigation of cellphone menus ("Earcons in Motion," Sami Ronkainen, ICAD 2001). The last paper on audification, in which digital data whose characteristic frequencies fall outside of the range of human hearing are "played back" so as to be audible, was presented back in 2002 ("Auditory Seismology," F. Dombois, ICAD 2002).

Sonification, in which data is used to control some aspect of sound for the purpose of communicating information about that data, is a divergent field in which more questions are being asked than answered. The central question of sonification has been called "the mapping problem." If a sinusoidal oscillator tone is played at 440 Hz, what temperature is that? If one wishes to design a sonification scheme to monitor the temperature of a mixing tank in a processing plant, should the pitch be higher if the temperature increases? Would anyone want to listen to sine oscillator tones all day? Would rainforest sounds or crickets be more appropriate? If musical sounds are used, would that be too distracting, or would the listener be misled by unintended connotations?

The topic of sonification generates a lot of papers and discussion at ICAD conferences (2004 was no exception), and there is always an interesting mix of approaches to the problem. Toolkits for designing sonifications ("A Flexible Framework for Real-Time Sonification with SONART," Woon Seung Yeo, Jonathan Berger, and R. Scott Wilson, ICAD 2004), examples of specific sonifications ("sMax, A Multimodal Toolkit for Stock Market Data Sonification," Fabio Ciardi, ICAD 2004), novel synthesis techniques with intriguing mapping possibilities ("Physically-Based Models for Liquid Sounds," Kees van den Doel, ICAD 2004), sonification perception studies ("Individual Differences, Cognitive Abilities, and the Interpretation of Auditory Graphs," Bruce Walker and Lisa Mauney, ICAD 2004), and sound spatialization and spatial auditory perception ("An Orientation Experiment Using Auditory Artificial Horizon," Matti Gröhn, Tapio Lokki, and Tapio Takala, ICAD 2004) are some examples.

One tension that typically exists in sonification research, at least as evidenced in ICAD conferences, is between the artistic, the scientific, and the practical. Sonifications designed by composers or sound artists often follow in the practice of 20th-century composition (e.g., Iannis Xenakis's Achorripsis), in which data or numerical calculations play a significant role but are shaped by many aesthetic decisions and adjustments, and are realized using interesting orchestrations or timbres. The richness and complexity of the end result may, however, frustrate the experimental psychologist, who wishes to isolate a limited number of sonic attributes and measure the subject responses to them in a controlled environment. In a practical situation, it may also frustrate the listener who wishes to hear what the data is doing—the complex sounds may be too difficult to decode.

Two installations in the lobby of The Studio in the Sydney Opera House, "ASX Voices" by Fabio Ciardi and "PlantA" by Garth Paine, sought to sonify, respectively, "the Australian voices of the global economy" and "meteorological processes vital to plants." Both works were presented over an eight-channel sound system. The former used recorded data from the Australian stock market; the latter was driven in real-time by weather instruments at the Opera House collecting wind speed, wind direction, temperature, and solar radiation data. The composers provided no written explanations of the specific mappings and I found it difficult to discern even qualitative aspects of the data trends by sitting, listening, and guessing.

A sonification designed for practical use by financial traders ("Creating Functional and Livable Soundscapes for Peripheral Monitoring of Dynamic Data," Brad Mauney and Bruce Walker, ICAD 2004) provides a background rainforest sound...