The Extraction of Expressive Shaping in Performance

In Mazzola and Zahorka (1994a), a general theory of performance transformations from a symbolic score space S to a corresponding physical space P was given. The transformation can be represented by performance vector fields that generalize the well-known hierarchies of tempo curves of a performance. This theory has been implemented as a module in the music research software RUBATO¯ (Mazzola and Zahorka 1994b) and has been successfully applied to the performance of classical compositions. Stange-Elbe (1999) has performed Contrapunctus III of J. S. Bach's Die Kunst der Fuge with RUBATO, and the results have been presented at the Diderot Conference on Mathematics and Music at IRCAM (Mazzola 2002c). The original theory dealt with the construction of a performance from a given score and a given performance field.

In this article, we address the question of inverse performance (Mazzola 1995) and how expressive shaping can be extracted from a given performance. The problem can be split into two parts: first, the problem of matching symbolic and performance events, and second, the calculation of the performance field after matching has been performed. In previous literature, the matching problem has been extensively discussed (e.g., Dannenberg 1984; Vercoe 1984; Puckette and Lippe 1992; Heijink et al. 2000). The approach to calculation of the performance field and the visualization of the extracted expressive shaping are new and thus receive main focus throughout this article.

EspressoRubette¯ is a software module that realizes the theory given in this article. The module is able to process a performance (e.g., a MIDI recording) and visualize the calculated vector field in real time. We will discuss the structure of Espresso Rubette and give the field interpolation and visualization principles. The article concludes with examples of visualization of performance fields.

Expressive Shaping as an Infinitesimal View on Expression

The precise description of a musical performance poses major difficulties. On the one hand, expression is a multi-layered semiotic phenomenon. That is, expression extends from surface parameters to more hidden structures that reveal the score's analytical depth structure, for example. We do not deal with this complex problem here, because on the other hand, the surface expressiveness is indecomposable in general, i.e., it is typically not possible to separate expressive shaping of onsets (agogics) from duration (articulation), loudness (dynamics), or pitch (intonation). For example, the "Chopin rubato" makes it impossible to recover a single tempo curve, because the agogical structure depends on pitch when chords are slightly arpeggiated, and the right-hand melody onsets are locally deformed against the left hand accompaniment. One therefore needs a language that copes with this intrinsic intertwining of parameters.

In traditional musicology, performance theory has never developed an adequate conceptualization beyond fuzzy common language descriptions, although Adorno (1963) promoted an infinitesimal view of performance, termed "micrologic" and based upon the insight that performance deals with the infinite interpolation of shaping parameters. This deficiency is typically reflected in the feuilletonistic music criticism and has to date prevented a truly scientific musicological performance theory. More specifically, inverse performance theory is far beyond musicological concepts, because the reconstruction of system parameters of a given performance would require a precise definition of the performance data and the system setup. Because not even a performance theory based upon score analysis is available, such a system description remains out of reach of traditional musicology. However, in computer-aided performance research, system descriptions, which would enable inverse performance theory, have been proposed (Sundberg 1991; Todd 1992).

We should remark on the concept of expression, because it is ambiguous in terms of content. If we attempt to analyze expression, this regards not the psychological perception of a performance by humans. This aspect is a legitimate one, but it touches a category that relates the performed music to human categorizations in terms of emotional response. Such a perspective is dealt with, for example, by Honing (1992) and Langner et al. (2000). In contrast, we regard expression as a rhetorically shaped transfer of structural score contents by means of the "deformation" mapping of symbolic data into a physical parameter space. The psychological implications...