By

Darrin Verhagen

Published on/by

[in]Transition

Accompanying text

Musicians, film-goers, and cinema theorists alike understand the emotional power of sound in film. Whilst the effects of the soundtrack may often be elicited subliminally, the strength of audio to add value to image and narrative is well accepted, if not fully understood. Part I: “Sound, Ears & Emotion” begins by outlining the functional mechanics of score and sound design. It draws on inherent elements of evolutionary biology and principles of systematic musicology to better understand how sound can make the audience feel. Importantly, these academic principles support an active practitioner’s perspective, and the essay explores through its form (music choices, vocal delivery, sound design, and visual choices) the very theories it is presenting.

Having established the means by which sound charges and changes the emotional experience of image and narrative, the essay then takes advantage of the opportunities afforded by eye tracking technology to consider more deeply the effects of sound beyond affect. Part II: “Sound, Eyes & Attention” draws on results from an ETMI Research Group study. It examines data collected from six participants watching Monsters Inc. under two conditions – sound on and sound off. By so doing, it considers how audio shifts the traditional balance between volitional (top down) and reactive (bottom up) attention. Through the exploration of the data collected, it demonstrates how contemporary sound film under silent conditions draws the eye to low-level, salient stimuli when a scene is assessed. These findings speak to sound’s capacity to immerse the audience in the world of the film, rather than being distracted in its absence by movement, light, and colour. Through audiovision, decoding the image is no longer a series of knee-jerk reactions. Viewers “view” differently.  Rather than the established understanding, where sound design is considered an instrument for generating the perception of image’s authenticity or music is used to synchronize group emotions, this research demonstrates sound’s capacity to actively shift our focus and change the methods by which we engage with what’s on screen.

For the first section, the video essay format provides an ideal opportunity to present the cinematic audiovisual relationships under discussion. Beyond mere outlines, these principles can be actively demonstrated. Examples are played with and without sound, radical recontextualisations and juxtapositions are constructed, and scenes from very different films are sutured together with audio as ideas are being explored. In the second section, examples of the footage from the eye tracking tests exploit a different advantage of the format, and data is presented as a temporal experience. In so doing, it allows for more dynamic and, at times, playful graphic analysis as a means to present what could otherwise be a series of dry stills.  What interests me most is the extent to which this format allows for a shift in tone from a standard journal article. Sadly, in print, there is limited capacity to augment academic writing with violence and stupidity. The possibility for academic rigor to coexist with devices from popular entertainment, to inform and engage beyond the limits of an academic cohort, is possibly the most exiting potential of the form.