Cell-utes and Flutter-Tongued Cats: Sound Morphing Using Loris and the Reassigned Bandwidth-Enhanced Model

The reassigned bandwidth-enhanced additive sound model is a high-fidelity sound representation that allows manipulations and transformations to be applied to a great variety of sounds, including noisy and inharmonic sounds. Combining sinusoidal and noise energy in a homogeneous representation, the reassigned bandwidth-enhanced model is ideally suited to sound morphing and is implemented in the open-source software library Loris. This article presents methods for using Loris and the reassigned bandwidth-enhanced additive model to achieve high-fidelity sound representations and manipulations, and it introduces software tools that allow programmers (in C/C ++ and various scripting languages) and non-programmers to use the sound modeling and manipulation capabilities of the Loris package.

The reassigned bandwidth-enhanced additive model is similar in spirit to traditional sinusoidal models (McAulay and Quatieri 1986; Serra and Smith 1990; Fitz and Haken 1996) in that a waveform is modeled as a collection of components, called partials, having time-varying amplitude and frequency envelopes. Our partials are not strictly sinusoidal, however. We employ a technique of bandwidth enhancement to combine sinusoidal energy and noise energy into a single partial having time-varying frequency, amplitude, and noisiness (or bandwidth) parameters (Fitz, Haken, and Christensen 2000a). The bandwidth envelope allows us to define a single component type that can be used to manipulate both sinusoidal and noisy parts of sound in an intuitive way. The encoding of noise associated with a bandwidth-enhanced partial is robust under time dilation and other model-domain transformations, and it is independent of other partials in the representation.

We use the method of reassignment (Auger and Flandrin 1995) to improve the time and frequency estimates used to define our partial parameter envelopes. The breakpoints for the partial parameter envelopes are obtained by following ridges on a reassigned time-frequency surface. Our algorithm shares with traditional sinusoidal methods the notion of temporally connected partial parameter estimates, but by contrast, our estimates are non-uniformly distributed in both time and frequency. This model yields greater resolution in time and frequency than is possible using conventional additive techniques and preserves the temporal envelope of transient signals, even in modified reconstruction (Fitz, Haken, and Christensen 2000b).

The combination of time-frequency reassignment and bandwidth enhancement yields a homogeneous model (i.e., a model having a single component type) that is capable of representing at high fidelity a wide variety of sounds, including inharmonic, polyphonic, impulsive, and noisy sounds. The homogeneity and robustness of the reassigned bandwidth-enhanced model make it particularly well-suited for such manipulations as cross synthesis and sound morphing.

Reassigned bandwidth-enhanced modeling and rendering and many kinds of manipulations, including sound morphing, have been implemented in an open-source software package called Loris. However, Loris offers only programmatic access to this functionality and is difficult for non-programmers to use. We begin this article with an introduction to the selection of analysis parameters to obtain high-fidelity, flexible representations using Loris, and we continue with a discussion of the sound morphing algorithm used in Loris. Finally, we present three new software tools that allow composers, sound designers, and non-programmers to take advantage of the sound modeling, manipulation, and morphing capabilities of Loris.

Reassigned Bandwidth-Enhanced Analysis Parameters

We have designed the reassigned bandwidth-enhanced analyzer in Loris to have parameters that are few and orthogonal. That is, we have minimized the number of parameters required and also minimized the interaction between parameters, so that changes in one parameter would not necessitate changes in other parameters. Moreover, we have made our parameters hierarchical, so that in most cases, a good representation can be obtained by adjusting only one or two parameters, and only rarely is it necessary to adjust more than three. Consequently, and in contrast to many other additive analyzers, the parameter space of the reassigned bandwidth-enhanced analyzer in Loris is smooth and monotonic, and it is easy to converge quickly on an optimal parameter set for a given sound.

The reassigned bandwidth-enhanced analyzer can be...