Hearing the Music of the Hemispheres

An fMRI scan of performance artist Maria Chavez’s brain as she watches a stimulus film. (Courtesy of Zoran Josipovic and Dan Lloyd)

fMRI scans by Zoran Josipovic
Sonifications and visualizations of brain activity by Dan Lloyd
Portraits and Music of the Hemispheres clips by Elisa Da Prato and James Fideler
Music composed by Aaron Einbond and Alvin Lucier, performed by Einbond and Yarn/Wire
Article design by Alexei Taylor

“Hearing the Music of the Hemispheres” is a born-digital multimodal article incorporating film, video, and audio clips that are integrated in, and central to, the argument. It is best viewed using Google Chrome, but any browser on any computer or tablet device will work.

http://scalar.usc.edu/anvc/music-of-the-hemispheres/index

I am sitting on the right side of the brain, between the temporal and parietal lobes, on a folding metal chair facing the frontal lobes. Organ-like synthesizer music swells up and rolls away, echoes off to my left, then relaxes into silence. I hear the ping-ping-ping of a hammer on a small gong or block of wood. It sounds like rain.

I am listening to music that has been composed by sonifying a series of fMRI scans of performance artist Maria Chavez. These were taken as she watched a stimulus film instructing her to imagine the sound of rain, listen to a recording of rain, and listen to a musical composition incorporating recordings of rain by composer Aaron Einbond. These instructions were given onscreen, so I am hearing a sonification of Chavez’s brain’s activity as she engages in an act of spectatorship. You could say that I am hearing her spectating: I am listening to her listening-seeing. The audio-speakers — at the Issue Project Room, a performance space in Brooklyn, New York — are positioned so that I can spatially experience the blood flow in her brain (a way of measuring neural activity): the speakers in the front of the room (left and right) play music generated by activity in the frontal lobes of her brain; the next two are the temporal lobes; then the parietal lobes; and finally, at the back of the space, the occipital lobe. I am sitting in Maria Chavez’s head, listening to her brain play.

I am interested in Music of the Hemispheres for what we can learn about spectatorship. However, the concert can be understood in other ways as well. First, it can be seen “simply” as the performance of a musical composition in which Chavez’s brain is a performer improvising its response to the stimulus film, which is analogous to another player. Her brain’s improvisation is then “amplified” through the process of sonification so we can hear it, and shown onscreen so we can see it. Second, the sonifications can be seen as an aural portrait of Chavez — not unlike portraits composed by Virgil Thompson or Erik Satie, except that this is a self-portrait of Chavez’s brain activity composed by her own brain. Furthermore, the visualizations of her brain activity can be seen as a visual portrait. Additionally, neuroscientist Zoran Josipovic, who conducted the fMRI scans, notes that the data obtained from them can be used to analyze the way human beings respond to an image of the self as opposed to an image of another. Finally, neurophilosopher Dan Lloyd, who created the visualizations and sonifications, uses sonification as a way of understanding consciousness.

Analyses of spectatorship in theatre and performance studies have largely drawn from reception and reader-response theories in literary and cultural studies. But performance is a multimedia and multidisciplinary genre requiring multiple cognitive strategies for making meaning. Music of the Hemispheres is a concert, a film, a portrait, an improv, and a performed (neural) performance analysis of the way Chavez saw and heard the stimulus film, and offers an alternative, performance-driven model for understanding spectatorship.

Ed. note: TDR is a print journal, well almost. We — like everyone/everything else — are “migrating” (what a strange word in this context) to the...