Evaluation of Input Devices for Musical Expression: Borrowing Tools from HCI

The widespread availability of high-performance, affordable personal computers has brought a new wealth of possibilities regarding real-time control of musical parameters. In fact, real-time gestural control of computer music has become a major trend in recent years (e.g., Wanderley and Battier 2000).

Various input devices for musical expression—also called hardware interfaces, control surfaces, or (gestural) controllers—have been proposed (Pennycook 1985; Roads 1996; Paradiso 1997; Mulder 1998; Bongers 2000, Cook 2001; Piringer 2001). These devices can be roughly classified into several categories: instrument-like controllers that try to emulate the control interfaces of existing acoustic instruments; instrument-inspired controllers that are basically designed loosely following the characteristics of existing instruments (but that do not necessarily seek an emulation of their counterparts); extended instruments, that is, acoustic instruments augmented by the use of several sensors; and alternate controllers, whose designs do not follow that of any existing instrument.

A more careful examination reveals two main trends behind these categories: the tendency to design controllers to best fit some already developed motor control ability (the case of the first three categories), or an attempt to deliberately avoid any relationship to gestural vocabularies associated to existing instruments, therefore allowing the use of different movements and postures not traditionally used in music performance. Conversely, alternate controllers, instrument-inspired controllers, and, to a certain extent, extended instruments, have been designed to fit idiosyncratic needs of performers and composers, but as such they have usually remained inextricably tied to their creators.

This situation brings up a number of questions related to the possible use of these interfaces by different performers and musicians. In fact, many times these developments have only been used in very few circumstances, notably at conference demonstrations. Therefore, one needs to find ways to compare the several designs to make sense of the variety of developments. This presents a problem when deciding on which parameters or features of various input devices to use as bases for comparison, particularly when discussing music of varying aesthetic directions. For instance, how do we evaluate an input device without taking into account a specific aesthetic context? That is, if people have only heard one type of music played on the violin, how can they tell if the violin is generally a versatile instrument? What is part of the composition, and what is part of the technology? How can we rate the usability of an input device if the only available tests were done by few—possibly one—expert and motivated performers?

A possible solution to the problem of comparing devices is to turn our attention to existing research in related fields. In this article, we approach the evaluation of input devices for musical expression by drawing parallels to existing research in the field of Human-Computer Interaction (HCI). We extensively review the existing work on the evaluation of input devices in HCI and discuss possible applications of this knowledge to the development of new interfaces for musical expression. We finally suggest and discuss a set of musical tasks to allow the evaluation of existing input devices.

Human-Computer Interaction

The field of HCI has historically drawn from four complimentary domains—software engineering, software human factors, computer graphics, and cognitive science—that could be grouped into two main foci: methods and software (Carroll 2002). The methods focus became later known as usability engineering, while the software focus became known as user interface software and tools. In HCI, interaction is defined as a process of communication or information transfer from the user to the computer and from the computer to the user. The user starts an interactive process to achieve a given task (Dix et al. 1998). The task normally requires the user to monitor the system's status and to manually modify the system's parameters by respectively using output and input devices.

Therefore, the research on input device evaluation plays an important role in HCI, in particular on the definition of the interaction possibilities allowed to the users. These possibilities mainly depend on the interaction metaphor used in each application, the WIMP...