Designing Constraints: Composing and Performing with Digital Musical Systems

Currently, a musician working with digital technology is faced with a panoply of musical tools that can be roughly characterized by a split between ready-made music production software on the one hand, and audio-programming environments such as SuperCollider, CSound, Pure Data, Max/MSP, ChucK, or Audiomulch (to name but a few) on the other. Problems with the former lie in the conceptual and compositional constraints imposed upon users by software tools that clearly define the scope of available musical expressions. It is for this reason that many musicians, determined to fight the fossilization of music into stylistic boxes, often choose to work with programming environments that allow for more extensive experimentation. However, problems here include the practically infinite expressive scope of the environment, sometimes resulting in a creative paralysis or in the frequent symptom of a musician-turned-engineer. Consequently, a common strategy can be detected, defined here as that of designing constraints, where the instrument designer, the composer, or the performer (a distinction often irrelevant in these systems; see Drummond 2009) devises a relatively high-level system of constraints, encapsulating a defined space for potential expression, whether of compositional or gestural nature.

This article engages with this situation by exploring the Human-Computer Interaction (HCI) concepts of affordances, constraints, and mapping. We examine how computational systems of musical expression always involve the establishment of a particular stratum that provides certain affordances to the musician, while concurrently posing important constraints. The article studies how constraints can serve as sources for creative explorations. For this purpose, three systems are analyzed: the mLog, Phalanger, and the ixi lang. All these systems are instantiations that proscribe complexity in favor of a clear, explicit space of gestural trajectories and musical scope. Yet, in their nature as digital systems, we find that the named concretizations are always arbitrary, dynamic, and highly transient, posing problems for the instrument's identity and historical continuity.

Affordances and Constraints

The phenomenological method of philosophical inquiry, founded by philosophers such as Husserl, Heidegger, and Merleau-Ponty, has influenced much work in HCI (e.g., Winograd and Flores 1986; Dourish 2001). A related, yet narrower, focus of the relationship between the human and the world has been studied under the terms of ecological psychology. The ecological approach to cognition was developed by psychologist James J. Gibson, who studied human perception and the environment as a dynamic system (Gibson 1979). The field of HCI has incorporated many important concepts derived from ecological psychology, such as ecological affordance and constraint. The former is more commonly used within HCI (e.g., Norman 1988; Gaver 1991), but this article argues that in the context of musical interfaces, the latter might be more pertinent.

Affordances

Gibson (1979, p. 127) initially defined an environmental affordance as "what it offers the animal, what it provides or furnishes, either for good or ill." In this definition, affordances are properties of the relationship between the environment and the agent (human or animal). The relationship consists of a mapping between the properties of the environment to the potential actions of the agent. An instrument such as the violin affords certain actions to the human that it does not afford to a bee. Influenced by Gibson, Donald Norman, working in the field of HCI, introduced the idea of perceived affordances (Norman 1988): the properties that the agent perceives as possible actions upon an object. This is a narrower definition, as Gibson's original definition of affordances saw them as existing independently of the agent's perception—a view supported by Gaver (1991), who talks about perceptible, hidden, and false affordances. Norman's view has been influential in the field of HCI to the degree that most contemporary designers are aware of the importance of affordances in system design.

Affordances have also been defined as entirely subjective. Vera and Simon (1993) define affordances as "carefully and simply encoded internal representations of complex configurations of external objects, the encodings capturing the functional significance of the object." Costall (1995) extends this definition of affordance to also signify a social construction, something that people learn from each other in every...