In lieu of an abstract, here is a brief excerpt of the content:

  • What Can the Language of Musicians Tell Us about Music Interaction Design?
  • Katie Wilkie, Simon Holland, and Paul Mulholland

It is difficult to create good interaction designs for music software or to substantially improve existing designs. One reason is that music involves diverse and complex concepts, entities, relationships, processes, terminologies, and notations. An open challenge for interaction designers is to find systematic ways of channeling the tacit, specialized knowledge of musicians into designs for intuitive user interfaces that can capably support musically skilled users, without excluding those with less technical musical knowledge.

One promising new approach to this challenge involves the application of research from the theory of image schemas and conceptual metaphors (Lakoff and Núñez 2000; Johnson 2005; Rohrer 2005, 2007). This theory posits that all human conceptual abilities are ultimately grounded in universal, prior, sensory-motor experiences of space, force, containment, and orientation, although the development of such groundings may be culturally influenced. The theory further posits that the nature of these groundings is amenable to empirical investigation, for example, by analyzing linguistic constructs in texts and discourse.

This theory has already been applied with some success to analyzing musical concepts (Saslaw 1996; Zbikowski 1997a, 1997b; Brower 2000; Eitan and Granot 2006; Eitan and Timmers 2010), mathematical concepts (Lakoff and Núñez 2000), general-purpose user-interface design (Hurtienne and Blessing 2007; Hurtienne and Israel 2007; Hurtienne, Israel, and Weber 2008), and sound-generation interaction designs involving simple musical parameters such as tempo, volume, and pitch (Antle, Droumeva, and Corness 2008; Antle, Corness, and Droumeva 2009). The present research appears to be the first to investigate the potential of conceptual metaphor theory for investigating music interaction designs for dealing with more complex musical concepts, such as harmonic progressions, modulation, and voice leading.

In the present study, we propose that by identifying the conceptual metaphors and image schemas used by musical experts when analyzing an excerpt of music, and then assessing the extent to which these conceptual metaphors are supported by existing music interaction designs, it is possible to identify areas where the designs do not match musicians' understanding of the domain concepts. This process provides a principled basis for identifying points at which designs could be improved to better support musicians' understanding and tasks, and for the provisional identification of possible improvements.

This article presents the results of an analysis of a short dialogue between three musicians as they discuss an excerpt of music. A methodology for the systematic identification of image schemas and conceptual metaphors is detailed. We report on the use of the results of the dialogue analysis to illuminate the designs of two contrasting examples of music software. Areas in which the designs might be made more intuitive are identified, and corresponding suggestions for improvements are outlined.

Embodied Cognition

Conceptual models are claimed to form the basis of our understanding of abstract domains (Johnson 2005). Research into the development of conceptual models has led to the hypothesis that our understanding of musical concepts, along with our understanding of all other abstract concepts, is grounded in our prior sensory-motor experiences (Saslaw 1996; Zbikowski 1997a, 1997b; Brower 2000; Lakoff and Núñez 2000; Johnson 2005; Rohrer 2005, 2007). It is useful to consider the detail of this hypothesis. Through the analysis of linguistic expressions, a number of constructs called image schemas have been identified. Image schemas represent perceptions of repeating patterns of bodily [End Page 34] experiences of space, forces, and interaction with other bodies in our environment (Lakoff and Núñez 2000; Johnson 2005; Rohrer 2005, 2007).


Click for larger view
View full resolution
Figure 1.

Representation of the relationship between OBJECTS and nested CONTAINERS.


Click for larger view
View full resolution
Figure 2.

Representation of the SOURCE-PATH-GOAL image schema.

For example, one common physical experience, often referred to in spoken and written language, is that of one object containing another. In image schematic terms, we refer to occurrences of the CONTAINER image schema. The CONTAINER image schema is used literally in phrases such as "put the toys in the box," where the box is a physical container into which objects can placed or from which they can be removed. More importantly...

pdf

Share