- Film Theory and Synchronization
We never see the same thing when we also hear; we don't hear the same thing when we see as well.(Chion 1994, xxvii)
I look out of the window and observe how sonic and visual information come together in real life. In fact, the first thing to notice is that they do not come together in acts of appreciable synchronization terribly frequently. A noticeable exception is the rustling of some foliage on a balcony opposite, though admittedly, the synchronization is a bit fuzzy and general in nature—I cannot tell exactly which movement of which leaves creates any one grain of noise. A comparable situation arises when I see two older gentlemen speaking in Arabic, or at least I think I do. I am unable to see their lip movements and whether they correspond exactly to what I hear because they are too far away, but my sense of sound localization and that they are dressed [End Page 69] in a more traditional North African way makes me fairly certain that it is them that I hear. Some expressive hand gestures on the part of one man at what sounds like a heated moment in the conversation confirm the match. A rare point of punchy synchronization occurs when a little girl, running flat-footed down the street and making a resonant slapping sound on the pavement, suddenly drops the baguettes she is carrying and a dull thud is emitted. For the most part, though, sound and image either do not hook up at all, or at the very most, they barely graze each other. The sound of the cars that I see passing at the T-junction 150 meters down the street is almost entirely masked by cars that are nearer by. A man carrying his shopping seems to make almost no sound at all, except for one or two footfalls from his soft-soled shoes. The loud sound of close-by cars is ubiquitous, but at the angle I am sitting I cannot really see them, except for the brief moment when they are reflected in a mirrored door on the other side of the road. In fact, this universe seems to be full of things that I cannot see but I can hear—the slam of a door somewhere in my building, the braking of a car, and the tweeting of some birds—and of things that I can see but cannot hear—strangely silent people and things.
My experience of the real world seems like an avant-garde film in comparison to the carefully constructed arrangements of synchronization to be found in mainstream movies, and it is this gap between messy real life and highly crafted audio-visual renderings of it that interests me as an artist. The theory of synchronization is not only important to composers working with film and other visual media; I believe that an understanding of the alchemy that occurs when two things happen at the same time is valuable to all artists working in time-based media.
For Donnelly, acknowledging film sound's drive toward simplicity is the key to understanding the way that the brain responds to audio-visual input, and he uses aspects of Gestalt psychology to support his ideas. Gestalt psychology proposes a model of human perception where the different senses process their input in parallel, and perceived items are organized into coherent patterns to form whole percepts, which are somehow different to (or more than) the sum of their parts. The mind desires clarity over the complexity that constitutes the real world, and "we attempt to order the stimuli we perceive in as simple and regular form as possible" (Donnelly 2014, 20). Donnelly links this [End Page 70] tendency of perception as described by Gestalt psychology to the relative simplicity provided by the "limited repertoire of standardized shots in standardized relationships, with highly focused and structured sound" found in mainstream film (23). It is as if cinema of this kind hands us the "good gestalt" on a plate and explains the reason why "representational cinema seems like reality … we ask no further questions of it" (23). There is something...