- Spectral Tools for Dynamic Tonality and Audio Morphing
The Spectral Toolbox is a suite of analysis–resynthesis programs that locate relevant partials of a sound and allow them to be resynthesized at specified frequencies. This enables a variety of routines including spectral mappings (changing all partials of a source sound to fixed destination frequencies), spectral morphing (continuously interpolating between the partials of a source sound and those of a destination sound), and what we call Dynamic Tonality (a novel way of organizing the relationship between a family of tunings and a set of related spectra). A complete application called the TransFormSynth provides a concrete implementation of Dynamic Tonality.
Wendy Carlos looked forward to the day when it would be possible to perform any sound in any tuning: “[N]ot only can we have any possible timbre but these can be played in any possible tuning . . . that might tickle our ears” (Carlos 1987b). The Spectral Toolbox and TransFormSynth address two needs that have previously hindered the realization of this goal: (1) the ability to specify and implement detailed control over the sound’s spectrum and timbre and (2) a way to organize the presentation and physical interface of the infinitely many possible tunings.
The analysis–resynthesis process at the heart of the Spectral Toolbox is a descendent of the Phase Vocoder (PV) (Moorer 1976; Dolson 1986). But where the PV is generally useful for time stretching (and transposition after a resampling operation), the spectral resynthesis routine SpT.ReSynthesis allows arbitrarily specified manipulations of the spectrum. This is closely related to the fixed-destination spectral manipulations of Lyon (2004) and the audio effects in Laroche and Dolson (1999), but it includes more flexible mappings and an effective decomposition of tonal from noise material. It is also closely related to the spectral-mapping technique of Sethares (1998) but can function continuously (over time) rather than being restricted to a single slice of time. In the simplest application, SpT.Sieve, the partials of a sound (or all the partials in a performance) can be remapped to a fixed template; for example, the partials of a cymbal can be made harmonic, or all partials of a piano performance can be mapped to the scale steps of N-tone equal temperament (i.e., a division of the octave into N equal parts). By specifying the rate at which the partials may change, the spectrum of a source sound can be transformed over time into the spectrum of a chosen destination sound, as demonstrated in the routine SpT.MorphOnBang. Neither the source nor the destination need be fixed. The mapping can be dynamically specified so that [End Page 71] a source with partials at frequencies f0, f1, f2, . . . , fn is mapped to g0, g1, g2, . . . , gn. For example, the SpT.Ntet routine can be used to generate sounds with spectra that align with scale steps of the N-tone equal tempered scale.
Carlos (1987a) observed that “the timbre of an instrument strongly affects what tuning and scale sound best on that instrument.” The most complex of the routines, the TransFormSynth, allows the tuning to be changed dynamically over a broad continuum, and the tuning and timbre to be coupled (to an arbitrary degree). It does this by analyzing existing samples, and then resynthesizing the partials using the routines of Dynamic Tonality. Dynamic Tonality builds upon the concepts of transpositional invariance (Keislar 1987), tuning invariance (Milne, Sethares, and Plamondon 2007, 2008), and dynamic tuning (Milne, Sethares, and Plamondon 2007) by additionally including various just-intonation tunings and allowing the spectrum (overtones) of every note to be dynamically tempered toward a tuning that minimizes sensory dissonance (for intervals tuned to low integer ratios, or temperings thereof).
Transpositional invariance means that a pitch pattern, such as a chord or melody, has the same shape and the same fingering in all musical keys—it requires a controller with a two-dimensional lattice (array) of keys or buttons. Tuning invariance means that a pitch pattern has the same shape and the same fingering over a continuum of tunings, even though its...