A new discipline in the design of narrative environments [1], sound scenography [2] has its roots in the centuries-long exploration of the synergy of sound and space in the fields of art and music. Recent technological developments such as spatial audio, sound beaming, or interactive sound have made it easy to make sounds move freely through the listening space, and for the audiences to physically engage with a sound environment. Sound technology is opening up new creative realms for designers and artists, and calling for a reassessment of established scenographic concepts, as well as the development of new strategies for storytelling. In this article, I will present notable projects from the fields of music, sound art, live performance, and exhibitions, and examine the role of sound in these scenographic examples. This is an updated version of the original article which has been first published in PLOT, the German magazine about creative spaces, exhibition design, brand design, and set & stage design [3].

As the audience in a spatial media installation [4] or a scenographic staging, we always relate our experience to our prior knowledge of similar events. Sound-wise, our everyday life is what we know best: we are surrounded by sounds night and day, we are indeed a part of a living soundscape. The term soundscape has been coined by the Canadian composer and acoustic researcher R. Murray Schafer in the 1970s, defined as the „relationship of man and sonic environments of any kind“ [5]. Sounds of nature, of cities, of work, of machines, of spoken word, and of music are all potentially comprised in such a soundscape. Since the world we physically live in is not a flat screen, we experience the soundscape spatially. The way we localize sounds in space has been and still is a matter of a great deal of research. Simply put, there is a left, a right, a far, a close, a front, a back, an above, a below – and everything in between. Before we even realize that we hear a sound, our brain already knows the direction where the sound comes from [6]. Compared to our spatially limited visual sense of about 180° horizontally by 130° vertically, the sense of hearing is omnidirectional. In this way, sound connects us intimately to our environment. We use sound to guide ourselves through the crowded city, to react when someone calls out to us, to decide where to focus our attention, to discern the crescendo of an approaching truck, and to assess whether a place feels familiar or not.

As the audience in a spatial media installation or a scenographic staging, we always relate our experience to our prior knowledge of similar events. Sound-wise, our everyday life is what we know best: we are surrounded by sounds night and day, we are indeed a part of a living soundscape. The term soundscape has been coined by the Canadian composer and acoustic researcher R. Murray Schafer in the 1970s, defined as the „relationship of man and sonic environments of any kind“. Sounds of nature, of cities, of work, of machines, of spoken word, and of music are all potentially comprised in such a soundscape. Since the world we physically live in is not a flat screen, we experience the soundscape spatially. The way we localize sounds in space has been and still is a matter of a great deal of research. Simply put, there is a left, a right, a far, a close, a front, a back, an above, a below – and everything in between. Before we even realize that we hear a sound, our brain already knows the direction where the sound comes from. Compared to our spatially limited visual sense of about 180° horizontally by 130° vertically, the sense of hearing is omnidirectional. In this way, sound connects us intimately to our environment. We use sound to guide ourselves through the crowded city, to react when someone calls out to us, to decide where to focus our attention, to discern the crescendo of an approaching truck, and to assess whether a place feels familiar or not.

Sound and space are connected tightly in our perception. Likewise, the experience of our individual soundscape in everyday life determines the way we perceive spatial media installations. The acoustic environment of a spatial media installation unnoticeably establishes its atmosphere, even if it is slightly different for every person. It instantly sets us into relation to the staged exhibition space, to what is close at hand, and to its distant parts. While we are reading a text, considering an object, watching a movie, looking at other people, or becoming active ourselves, we hear everything around us. The acoustic environment complements the visual impression and therefore determines how deeply immersed we can feel in an exhibition, for example.

Sound Art and Space

Interesting works related to scenographic concepts can especially be found in the realm of sound art. Although the term sound art suggests a focus on the auditory part of an art work, many of the artworks actually include other media and materials such as video, narrative, or scenographic objects. Volker Straebel, a Berlin based musicologist, states: „[…] most of what we call sound installation today is actually a cross-media installation, and the implied primacy of sound directs and narrows down its reception significantly.“ [7]

An impressive audio work that addressed the themes of time and presence is the sound installation Forest (for a thousand years …) created by Janet Cardiff und George Bures Miller, and presented at dOCUMENTA (13). The work organically fitted into the scenographic environment, a little patch of woods in the midst of the Kassel’s Karlsaue park. The audio play was created specifically for this location and indeed used the actual geographic location of the installation and its history in the Second World War as one of the key narrative elements. The plot merged subtly with the visual and haptic impression of the surroundings and the actual sounds of the woods so well, that the audience was visibly disconcerted in their perception of certain sounds as being real, i.e. part of the immediate natural environment, as opposed to being part of the audio play. In fact, the outlines of both categories became effectively blurred.

Like so often in the works of Cardiff and Miller, visitors become part of the installation, in a sense, they become its actors. Similarly to Forest, The Murder of Crows is a scenographic sound installation, realized at the Hamburger Bahnhof – Museum für Gegenwart in Berlin in 2009. Around 100 loudspeakers were spread all over the hall, placed on chairs, stands, or hanging on the ceiling. „One soundscape moves into another with an electronic dreamscape composition shifting into sound effects such as factory noises, crashing waves or birds wings and then into a guitar and strings composition then into a choir sequence and marching band“, the artists state on their website. The visitors where encouraged to explore the scenery and to change their watching and listening perspective. For example, the voices of a Russian choir have been recorded one by one, and played back separately on different loudspeakers. This gave the visitor the chance to come very close to the voice – an astounding, intimate effect, which would not have been very likely to happen if the vocalist were physically present.

Music and Space

In music history, the Flemish composer Adrian Willaert is considered as the pioneer of spatial music. In the middle of the 16th century, during his time as the Kapellmeister at San Marco in Venice, he created compositions for several choirs positioned in different locations within a single space.

Although during the following eras composers experimented with spatial effects in music, it is the period after Second World War that brought the flourishing of spatial music. One of the most important composers who wrote music for space using electronic signal processing, as well as microphones and loudspeakers as instruments, was Karlheinz Stockhausen. His composition Mikrophonie, first performed at the Kugelauditorium of the German pavilion at the EXPO 1970 in Osaka, is one of the first pieces that has been dedicated to a three-dimensional loudspeaker array [8]. Stockhausen identified a multi-sensory relationship between sight and the hearing sense: „[…] in the past, the organ and the choir have been positioned in the gallery of a church. You could not see either one, but only hear them. Once, the choir has been the angels. The praying people only heard the chanting, and hence believed it would be the angel’s voices. At some point, somebody came up with the idea that everything should be visible. Now the choir stands around somewhere and has lost its magic. In order to free the people’s minds, I always encourage them to close their eyes during a concert, and only listen to the music. The sound experience is a completely different one when you’re not distracted by visual impressions“ [9]. In Music, this statement absolutely makes sense. But can this idea be applied to any listening situation, such as a spatial media installation, or a similar scenographic event?

Sound Scenograpy

Professionals creating the design of a space always make a decision about its sound, consciously or unconsciously. One should remember that it is practically impossible to not design the sound of a space, since there is no such thing as a space completely devoid of sound – thus the question what a space sounds like arises even for a primarily visual design. If the sound matter is not addressed, the realization of one’s artistic intention may prove to be unsuccessful. At the very least, a project might not unfold its full creative potential. This may sound as a potential problem, but luckily, sound also is a powerful design element that can really enrich the visitor’s experience. More and more people are involved in the field of designing sound in space – now often called sound scenography.

It has been shown that the simultaneous perception of visual stimuli affects one’s hearing experience, and vice versa[10]. In the art of cinematography, a lot of audio-visual effects are based on this fact. With regard to the scenography of spatial media installations we could venture to slightly modify the last sentence of Stockhausen’s quote: The visual experience is a completely different one when it is supported by sound in the best possible way. In order to explain what I'm up to, I would like to start from scratch, with silence.

First, the design of the sound for a spatial media installation starts with removing all sounds that could potentially distract from the creative intent – including bad room acoustics. I am talking about noisy air conditioning, humming lights, inappropriate room acoustics, squeezing floors, noisy machines, traffic noise from outside, and so on. But while silence enables us to concentrate, it also raises a problem that may be significant in shared experiences: silence takes away our acoustic privacy. When every distracting sound has been removed, a silent room could become unpleasant: visitors would feel inhibited to talk to others or to attract attention by making even a little noise. If this is not intended, that kind of silence should be avoided.

A space resonating in a way that supports a spatial media installation gives us back our acoustic privacy. While featuring the right sounds, the auditive experience will not distract our attention from the narrative at the same time. A sounding space frees the audiences to explore the space without this feeling of self-consciousness that often arises in silent spaces. Most often, people think about adding music, and, of course, that usually works.

But how is it possible to not only use background music covering silence and unpleasant noise, but to use sound in order to create an audiovisual narrative in space? How to interweave a visual and an auditive storyline into a complementary, single one? The cinematographic aspect is obvious when considering the spatial media installation Kingdom of the Shadows, realized at Rheinisches Landesmuseum Trier by TAMSCHICK MEDIA & SPACE in 2010. Elements of film, theater, animation, and audio play were merged into an immersive and entertaining spatial theatre. Video projections mapped onto exhibited antique reliefs and statues, infused by sound environments and narrated by actors, brought Ancient Rome back to life. The auditory intricacy combined with the sketchy imagery merged into a single narrative space, while enough was left out to leave space for the audience’s own fancy. Auditory storytelling in film and scenography are more related to each other than may be imagined. In films, the sound layer consists not only of dialogue, rather, most of the spatial context of the action, as well as the characters’ emotional states are described with sound. Sound atmospheres tell a lot about the environment of a visual scenery. Sound makes objects feel „real“ and lends them materiality, it gives any creature the semblance of life. Sound is even a fundamental part of the character’s temper. Accentuation of certain sounds – and their dulling – lends the dynamics to the story.

About the performer’s physical presence

I would like to come back to Stockhausen: „Once, the choir has been the angels. […] Now the choir stands around somewhere and has lost its magic.“ The listener can see how the sound is being produced. The German newspaper FAZ commented about Liszt’s Via Crucis, staged by Robert Wilson: „The pure sounding without visible musicians blends with the image-less light, thus creating a real space for mindfulness“ [11]. The oratorio has premiered at the Viehauktionshalle in Weimar, and did not include a single live performer, the concert featuring only light and spatial sound as interpreters. Since the light show extended in the whole space around the audience, it seemed obvious to stage the choreography of the music in the space as well. For example, the chanting of the choir has been pre-recorded, and then played back with loudspeakers placed above the audience during the show. The paradoxical effect was that the sound of the absent singers became almost physical, the voices rolled off one’s shoulders – it literally caused goose flesh. I reckon that the intensity of the experience would not have been the same had the musicians been physically present and visible in the show – their material presence and weight would have distracted from the etherial music itself and the spatial audio-visual staging.

In my experience, visibility is not only a question in the case of musicians, but also for audio equipment such as loudspeakers. As soon as the loudspeakers can be seen, it is hard for people not to be aware of the technical apparatus that enables the hearing experience. The Viehauktionshalle, where „Via Crucis“ has been performed, was a huge space where loudspeakers were concealed within the scenographic setup.

A different approach as to the role of musicians in a performance has been realized by the Berlin based artist network phase 7 with its walk-through opera Himmelsmechanik (Heaven Mechanics), performed at Deutsche Oper in Berlin. The opera took place in three different connecting spaces that the audience could wander about in, whereby in the last room, the musicians who could be guessed behind a veiled glass screen, were performing live from an acoustically separated room. The music was captured with microphones and distributed via a three-dimensional loudspeaker array all around and above the audience, which enabled the sound mixer to position and move the sounds of all the instruments independently in the space of the audience. In the case of Himmelsmechanik, the loudspeakers could be indeed seen by the audience – nevertheless the attention was seized by the performance of the solo vocalists, who walked among the audience. Hence, the audience was likewise not primarily aware of the audio technology.

Conclusion

Summing up, sound in films functions similarly to spatial media installations and other scenographic concepts. Sound designers can make use of a lot of filmic storytelling strategies. Yet there is one significant difference: the audience is free to physically move through the narrative space. In order to depict the content or story consistently, the space can be sonified consciously and hence made alive. Sound scenography, the art of staging sound in space, is now at its starting point in theatrical and scenographic concepts. Sound designers are adapting to new storytelling techniques with spatial audio formats, while the required technology becomes more affordable. All in all, advocating for considerate sound design concepts in scenographic productions is not a matter of putting more sound where there used to be only visuals, or addressing yet another sense for the sake of tickling more emotional response out of the audience. Both, visual and auditive design, should work together – at the very least. Skillfully handled, designing sound for narrative spaces can lead to more than just the sum of what we see and what we hear.


Footnotes

[1] The term narrative environments is not clearly defined as yet. However, my understanding goes along the lines of Wikipedia’s definition: „A narrative environment is a space, whether physical or virtual, in which stories can unfold. A virtual narrative environment might be the narrative framework in which game play can proceed. A physical narrative environment might be an exhibition area within a museum, or a foyer of a retail space, or the public spaces around a building - anywhere in short where stories can be told in space.“ https://en.wikipedia.org/wiki/Narrative_environment

[2] The term sound scenography is used to name the process of sound design similar to other scenographic uses of elements such as light, interior design, image projection, costume, etc., for the purpose of „manipulation and orchestration of the performance environment“ in relation to „the space in which the performance takes place and the placement of the audience“ (McKinney & Butterworth, 2009)

[3] Scherzer, J. (2014). Sound als Erzählebene in narrativen Räumen. Stuttgart: PLOT (PDF version in German available on http://files.taucher-sound.com/PLOT10_essay.pdf)

[4] The key characteristic that distinguishes spatial media installations from other media such as VR glasses is that the staged scenery surrounds the audience as a physical space, thus shifting the role of the spectators from recipients to participants, requiring them to spatially interact within the work to follow the story. Furthermore, as these spaces are usually conceived for multiple participants, spatial media installations are most often also shared experiences.

[5] Truax, B. (1974). Soundscape Studies: An Introduction to the World Soundscape Project. Numus West, no. 5.

[6] Blauert, J. (1974). Räumliches Hören. Stuttgart: Hirzel.

[7] Straebel, V. (2010). Vom Verschwinden der Klangkunst. In: Kiefer, P. (ed.). Klangräume der Kunst. Heidelberg: Kehrer Verlag

[8] A three-dimensional loudspeaker array consists of three or more loudspeakers horizontally arranged around the listener, plus loudspeakers above the listener.

[9] Stockhausen, K. (1999). Haben Sie Techno erfunden? Stockhausen: Ja! de-bug.de online magazine http://de-bug.de/mag/haben-sie-techno-erfunden-stockhausen-ja/comment-page-1/

[10] Dinh, H. Q., Walker, N., Hodges, L. F., Song, C., & Kobayashi, A. (1999, March). Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In Virtual Reality, 1999. Proceedings., IEEE (pp. 222-228). IEEE.

[11] www.robertwilson.com/via-crucis

Other references

Böhme, G. (2013). Atmosphäre, Essays zur neuen Ästhetik. Berlin: Suhrkamp Verlag.

Schafer, R. M. (1977). The tuning of the world (1st ed.). New York: Knopf.

Comment