Skin Pattern Sonification as a New Timbral Expression
- Skin Pattern Sonification as a New Timbral Expression
The authors discuss two sonification projects that transform fingerprint and skin patterns into audio: (1) Digiti Sonus, an interactive installation performing fingerprint sonification and visualization and (2) skin pattern sonification, which converts pore networks into sound. The projects include novel techniques for representing user-intended fingerprint expression and skin pattern selection as audio parameters.
Supplemental materials such as audio files related to this article are available at <yoonchunghan.com/portfolio/DigitiSonus.html>.
Of all body patterns, fingerprints are the most unique visual patterns found on human and primate bodies [1]. In the current digital era, fingerprints are commonly used as secure keys to access personal information; however, the misuse of fingerprints has created critical privacy issues. The use of digitized personal body information has stimulated discussion about identification, privacy and the need to secure such data. Sonification of biometric data can open up new possibilities for resolving privacy issues through additional layers of data representation.
In this article, we describe two sonification projects utilizing interfaces for creating individual sonic signatures: one that transforms fingerprints into audio and another that transforms skin patterns into audio. We believe that these artistic and experimental developments also may yield practical uses for scientific applications.
Digiti Sonus—Interactive Fingerprint Sonification
Digiti Sonus is an interactive fingerprint sonification installation that transforms human fingerprints into musical sound. The installation uses algorithmic computing to enable audience members to explore their sonic identities through unique sounds generated by their fingerprint patterns. Users can interact with Digiti Sonus to loop and manipulate the captured sound, resulting in realtime experimental music representative of their sonic identities. Digiti Sonus investigates questions of identification and privacy through immersive audiovisual elements and the interaction of bodies and sensing interfaces. Given the auditory system’s ability to localize sound in three-dimensional space [2], we believe that fingerprint sonification can serve as an effective technique for the representation of complex biometric data. The three-dimensional visuals of Digiti Sonus also enhance the audience’s immersive experience. The work’s easy-to-use interface allows audience members to explore their sonic signatures quickly and intuitively in real time through simple physical interaction. The installation employs FM synthesis [3] based on various visual feature analyses such as minutiae, angle, area and the amount of applied fingerprinting pressure [4].
This work was exhibited at the ACM SIGGRAPH 2013 Art Gallery [5], as shown in Fig. 1. Approximately 200 people interacted with the installation, inputting their fingerprints and manipulating them to create diverse timbres of sound. Each person could hear his or her own unique sound, created by his or her own distinct fingerprint design. The resulting sounds were for the most part easy to distinguish from one another. However, occasionally the results were only subtly different when similar fingerprints were selected and created similar sounds.
Skin Pattern Sonification
Skin pattern sonification uses diverse visual features extracted from skin patterns to transform non-biometric pattern data into audio. The human body can be segmented into its various parts such as the head, neck, arms, legs, chest and pelvis. Each body part contains unique skin patterns and characteristics related to its function(s). The lines and wrinkles created by the folds on the palms of one’s hands, referred to in dermatoglyphics as ridge skin patterns [6], are one example; the unique occurrence and characteristics of chest hair, which depend on genetic disposition [7], are another. While skin patterns are not as clear as fingerprints due to their varied patterns and widespread area, the examination of the skin at a microscopic scale reveals complex patterns and visual features. The skin area that is scanned can be redefined, magnified and changed at different locations, times and scales.
One of the most challenging aspects of skin pattern sonification is that skin pattern images are difficult to distinguish through obvious visual features (as is possible with fingerprints). As a result, it was critically important to design a rotation/position-invariant feature extraction method. We found that fundamental visual features on the skin, such as pores, hairs and wrinkles, can be networked...
