Interacting with Symbol, Sound, and Feature Spaces in Orchidée, a Computer-Aided Orchestration Environment

Until recently, orchestration has remained a relatively unexplored domain of computer music. The problem of musical orchestration could be stated as the art of combining timbres and pitches to create particular sound textures. More generally, though, orchestration comes into play as soon as timbral issues are addressed in instrumental music, and it brings forward the problem of linking the sonic space (i.e., the realm of timbres) with the symbol space of the formal compositional processes (consisting of notes, for example). Since the beginning of the 20th century, composing with "sounds" rather than "notes" has become a widespread practice (Erickson 1975), hence emphasizing the relevance of this problem in contemporary music creation. However, contrary to other techniques of musical composition, orchestration requires difficult-to-formalize and difficult-to-verbalize knowledge to assemble data from these symbol and sound spaces. Hence, for years, computer music systems stayed away from the complexity of this problem, and orchestration was never much more than an empirical activity.

Since its early years, computer music research has evolved in two main directions. On one hand, the goal was to provide composers with the ability to manipulate symbolic musical objects (e.g., notes, chords, rhythms, and melodies). On the other hand, researchers concentrated their efforts on sound processing, analysis, and synthesis, leading to a deeper comprehension of many aspects of sound and its perception. In the signal-processing field, the growing interest in automatic feature extraction (Peeters 2004) and the relationship between empirical features and perceptual dimensions (McAdams et al. 1995) paved the way towards a symbolic organization of sound. It also encouraged the emergence of music information retrieval, a field that aims to extract information from audio signals for the purposes of automatic classification and latent structure discovery. In the meantime, recent advances in the field of computer-aided composition introduced significant openings to integrate generation of electronic sound material into symbolic musical practices (Bresson and Agon 2007), making sound synthesis understandable as an interactive, modular, and formalized process.

Though very promising, these trends did not yet converge to a "unified theory," and complex connections between audio and symbolic data remain an open issue. The computer music community still misses a unified comprehensive framework that could bring together various levels of sound and music representations, integrating orchestration processes among the pioneering fields of new music research.

In this article, we present an orchestration system called Orchidée, and we focus on how user interaction may be of great help in making symbol and sound worlds communicate together. We introduce a general orchestration scheme in which each step is either supervised by an interaction process or driven by search procedures that jointly optimize objectives in symbol and audio spaces. All core computation procedures embedded into the Orchidée orchestration kernel communicate with computer music environments through a simple client/server architecture. We introduce innovative interfaces in OpenMusic (Assayag et al. 1999), Max/MSP (Puckette 1991), and MATLAB, both for the specification of the orchestration problem and for the exploration of its potential solutions. These interfaces allow a fine comprehension of the links between symbolic and audio data through feedback loops and preferences-inference mechanisms.

This article is organized as follows. We first report previous research in the computer music community aimed at designing orchestration tools. We discuss their inner limitations and show how consequently they miss various aspects of orchestration complexity. We then suggest a multiple-viewpoint analysis of this complexity as a network of relations among sound, symbol, and feature spaces. From there, the orchestration problem is turned into a generic scenario for which the Orchidée client/server framework offers a solution. Each main step of this scenario (problem specification, search process, and exploration of solution space) is then separately detailed, and examples of graphical user interfaces (GUIs) are provided.

Related Work

Though computer-aided orchestration is quite a new topic in the computer music field, at least three attempts have been made in past research for designing orchestration tools. They all share the same paradigm: The goal is always to discover...