Supporting Music Composition with Interactive Paper

Music composition has been deeply influenced by the computational power introduced by computers, but despite the use of software to create new sounds or process symbolic music, composers still use paper in their creative process. Interactive paper creates new opportunities for combining expression on paper and computation. However, designing for highly individual creative practitioners who use personal musical representations is challenging. In this thesis, the author argues that composers need personal and adaptable structures on paper in which they can express and explore musical ideas.

Jérémie Garcia, exploring computer-based musical processes on paper.

(© Inria. Photo: H. Raguet.)

The author first presents three field studies with contemporary composers that examined the use of paper and the computer during the composition process and how linking the two media supports exploration of musical ideas. Based on participatory design studies, the author introduces Paper Substrates, interactive paper components that provide modular structures for interacting with personal representations of computer-based musical data. The author details tools to develop paper applications with the Paper Substrates approach.

Then the author describes a structured observation study with 12 composers who used Polyphony, a unified user interface that integrates interactive paper and electronic user interfaces, to compose a short electroacoustic piece. The study allowed the participants to systematically observe and compare their compositional processes.

Finally, the author reports on a project with Philippe Leroux during the composition of his piece “Quid Sit Musicus.” Several work sessions with the composer and a musical assistant led to the design of new paper-based interfaces for synthesizing sounds or controlling the spatialization from handwritten gestures over an old manuscript.