- An Interactive Music Environment for Large Groups with Giveaway Wireless Motion Sensors
Few existing systems effectively enable a large number of participants to collaboratively control a real-time, centralized interaction. This is particularly true in the area of interactive dance. Wearable dance interfaces (e.g., Siegel and Jacobsen 1998; Paradiso et al. 2000; Aylward and Paradiso 2006) allow a single dancer or small group of dancers to control music with their actions, but these do not scale to allow for hundreds of participants to interact concurrently. Various vision-based tracking systems are able to extract considerable nuance from dance ensembles (e.g., Wechsler, Weiss, and Dowling 2004; Obermaier 2004; Camurri et al. 1999), but they generally exploit a highly structured and stable stage environment with tight lighting constraints. The problems of cost, data-communication bandwidth, and system responsiveness become increasingly difficult as the number of participants increases. A system that could effectively give control to a large number of dancers offers the possibility of environments with extremely responsive music and lighting, engaging users with a heightened sense of expressiveness.
To address these issues of large-group musical mapping, we have developed a scalable system first introduced in Feldmeier (2002); Feldmeier, Malinowski, and Paradiso (2002); and Feldmeier and Paradiso (2004) that can effectively gather data over an essentially unlimited audience size. The system consists of wireless sensors that are given to audience members to collect rhythm and activity information from the crowd that can be used to dynamically determine musical structure, sonic events, and/or lighting control. (A block diagram of the system is shown in Figure 1.) The sensors are small and lightweight, and they can therefore be either worn or held by a participant. To detect the participant's motion, they have radio-frequency (RF) transmitters that send a short pulse of RF energy whenever they encounter a dynamic acceleration greater than a predetermined level. Finally, they are inexpensive enough to be viable as disposable, "giveaway" items for large crowds.
The sensors' RF pulses are collected by receiver base stations that have limited sensitivity, enabling the development of zones of interaction around each one. In this manner, multiple base stations can be used in a venue to create distinct areas where the controller takes on new functions. This zoning information can also be used to direct the music and lighting to respond to the participants' actions in that area, localizing the response to a smaller group of proximate dancers. The pulses received by the base stations are then sent to the MIDI converter, which counts the number of pulses detected in each zone and transmits this information at regular intervals via MIDI serial communication. These MIDI signals are then received by a Macintosh G4 computer, where they are analyzed to detect activity levels and rhythmic features of the audience.
These parameters are then available to be mapped to musical content and/or lighting control information. For our applications, all data analysis and musical mapping is done in the Max programming environment, and sound generation is performed off-board with dedicated hardware music synthesizers. Lighting content is generated with an IBM-compatible personal computer, and control information is sent to the lighting instruments via DMX serial communication (Randall 2002). The sound and lighting changes are then realized, to which the audience in turn responds, allowing the experience to build upon itself and giving the users an increased connection to the music.
A summary of several projects exploring electronic musical interfaces that facilitate group interaction [End Page 50] can be found in Blaine and Fels (2003) and Weinberg (2005). Past projects such as the Brain Opera (Paradiso 1999), the SIGGRAPH 98 Interactive Dance Club (Ulyate and Bianciardi 2002), and the ADA installation at Expo 2003 in Neuchatel (Eng 2004) have explored the use of rooms with a vast sensor and interface infrastructure for facilitating large-group entertainment. Although a goal of research into Ubiquitous Computing (Weiser 1991) is to make such infrastructures economical and commonplace, these installations still tend to be extremely expensive, with their deployment requiring up to millions of dollars.
Click for larger view
View full resolution