Music and Sound Design in Video Games: A 2-Part Series
We previously talked about console innovation and how new technologies influenced the creation of game soundtracks. In this post, we’re diving into yet another aspect and will be discussing the role of music on the atmosphere and immersive capacity of games. We’re first going to get into both concepts by reviewing some of their intrinsic elements such as the narrative tone, the game’s perceived mood and the story themes. We will then discuss their relation to the games’ audio landscape by analyzing some titles in more detail.
Part 1 – Atmosphere and player immersion, a definition of the concepts
Atmosphere is a complex mix of story theme, narrative tone and perceived mood
An atmosphere can be viewed as a pervading tone or mood felt about a creative work. In games with a narrative, players are actively piecing together in-game elements to make sense of the story. A tone mostly emerges from a game’s writing, while a perceived mood is partly influenced by the game’s musical coat. Both narrative techniques are ultimately interpreted in a subjective manner by players but are still reliable means to encourage players to feel a certain way.
Games like Journey, Child of Light or Dead Space are such memorable experiences because they align their graphics and narrative tone with the soundtrack.
Along with a given tone and mood, the theme of a narrative also helps forge a game’s unique atmosphere. While a tone is the manner in which the game presents its theme, a theme is the main subject or idea behind the game.
For example: While both belong to a retrofuturistic genre and revolve around themes of self-determination and destiny, Bioshock and Paradigm have completely different tones. The first is grave while the second opts for a more humoristic approach.
The atmosphere of a game is another key factor in promoting its immersion
The player’s immersion is the deep mental and emotional involvement in a game. Highly immersive experiences are captivating enough to hook a player for extended gaming sessions and stay in their mind long after the game is finished.
Gaming strengthens the link between atmospheric engagement and immersion because it is highly interactive. Unlike movies or songs, the interactivity of games balances cognitive gaps.
When we watch a movie or listen to a song, those mediums control what we perceive. This isn’t the case for games—story and audio pacing, along with environmental storytelling and information giveaways, are essential for our imagination and engagement.
Cognitive gaps: the building blocks of the players’ imagination
Unsurprisingly, the use of cognitive gaps is linked to evolving technology. While it was easier in the ‘80s to leave space for cognitive gaps based on technical limits, recent gaming platforms allow developers to expand on the base content of their game for extended periods of time.
While adding content can be a way to genuinely prolong the story, some mechanics have a diminishing impact on atmosphere. A few that come to mind are kill and fetch quests, skin boxes, seasonal passes, loot boxes or cosmetics. These additions can make sense from a business point of view, as keeping an active player base offsets the development costs of new IPs. Furthermore, not all games need a strong atmosphere to have an impact on players.
For games which do rely on atmosphere though, like RPGs and survival games, these mechanics challenge the player’s suspension of disbelief. Like in design and art, imposing creative barriers makes for better results.
Part 2 – Music and Sound Effects, a Labor of Love
The main factors that go into shaping atmosphere and its immersiveness are various in-game elements such as narrative, audio, user interface, graphics, gameplay, difficulty or world design. In this part, we’ll look at two audio elements: music and sound effects.
Soundtracks and SFX are a way to enhance the story while complementing the gameplay. They should match the desired tone of the game while supporting its theme.
Music is all about pacing and integration
Most musical pieces in games are composed to be experienced in an extra-diegetic manner. This means they are heard by the player only, and not the in-game character. In this sense, music is there to ensure an adequate emotional response from the player.
So we see that soundtracks need to be considered with a certain pacing—just like comedy. If the build-up happens after the payoff, the joke is ruined. Depending on a game’s style, music composers integrate exploration and level discovery to their composition style. A rogue-like and an open world offer two distinct ways to present a soundtrack.
It can also be argued that music falls under the same umbrella as sound effects and ear fatigue. If the same track is looped too frequently, people get annoyed. Except for the cases of rhythm and racing games, having a soundtrack blaring from your speakers with no pauses between tracks will lessen its effect.
Walking on eggshells: Sound Effects
There’s a lot that needs to be considered when it comes to integrating sound effects to a game. The intensity of the volume, the repetition, the mix between extra- & intra-diegetic, etc. But it’s important not to overload the player. Good sound effects are designed to match the tone of the game while being coherent with its theme, but they still need to feel believable in the context of the in-game world.
A pitfall in sound design is failing to consider the dynamic range of the game’s audio. Dynamic range refers to “the ratio between the largest and smallest values that a certain quantity of sound can assume”. In other words, if the dynamic range is too compressed, it will result in a muddled mix, lacking the audio depth achieved by superimposing and distancing sounds from each other and from the music. There are many different methods used to balance the volume and reduce dynamic range compression. Some sound mastering processes include equalizing sound frequencies, stereo space management and volume disparity management.
Games that got it!
Theory is all well and good, but let’s have a look at games that made the most out of their soundtracks and sound effects: Metroid (1986), Shovel Knight (2014) and Dark Souls (2012). We’ve chosen these games in particular because they are related to some of the technological milestones we reviewed in our previous blog post.
METROID – A blur between soundtrack & sound effects
The first Metroid was published in 1986 on the NES, the legendary third generation 8-bit console created by Nintendo. Metroid was developed as a dark sci-fi game set on the planet Zebes. The story follows Samus, a human soldier attempting to retrieve stolen Metroid organisms from space pirates wanting to create biological weapons out of them. Metroid’s main themes are related to isolation and survival. They are supported by a grave, nearly Lovecraftian narrative tone. The level design and environments are heavily labyrinthic which adds to the strange feelings of quiet anxiety that the game emulates.
The creative work that went into making the Metroid soundtrack is fascinating. The music composer, Hirokazu Tanaka, approached composition and sound effects in a peculiar manner. Fed up with the overuse of up-beat melodies in games at the time, Tanaka recalls in a 2002 interview:
“I had a concept that the music for Metroid should be created not as game music, but as music making the players feel as if they were encountering a living creature. I wanted to create the sound without any distinctions between music and sound effects. The image I had was, anything that comes out of this game is the sound that the game makes.”
Even if Tanaka only worked on the first installment of the series, his footprint is still recognizable in the latest Metroid games. The NES was manufactured with 5 different synthesis patterns or voices, limiting the range of simultaneous notes the machine could play. In this sense, integrating the sound effects to the music was a smart move, and one that, looking back, hugely contributed to the soundtrack’s reputation. In addition, there’s no dialogue in the entire game—a clever way of playing with cognitive gaps by relying on environmental story-telling.
Arrival on planet Zebes – an example of Tanaka’s mix between musical theme and sound effects.
While there is no doubt that the atmosphere of Metroid was hugely shaped by its graphics, it is the soundtrack that puts the final nail in the coffin. Through the use of bass pedals and synths, Metroid’s musical pieces use lots of dissonant repeated chords. This type of composition helped further the eerie and oppressing feelings we sense while playing Metroid. Tanaka also chose not to use any melodic musical pieces until the final boss. His goal was to push players toward catharsis when finally finishing the game.
Metroid uses its sound effects to create a kind of unity in its soundtrack. This is thanks to environmental sounds such as doors opening between screens, so we already see a mix intra- and extra-diegetic sounds. In conclusion and based on the initial intent of Hirokazu Tanaka—atmospheric objective achieved.
Now let’s compare Metroid to a game which came out much later but was developed in the spirit of the NES, Shovel Knight: Shovel of Hope.
Metroid uses its sound effects to create a kind of unity in its soundtrack. This is thanks to environmental sounds such as doors opening between screens, so we already see a mix intra- and extra-diegetic sounds. In conclusion and based on the initial intent of Hirokazu Tanaka—atmospheric objective achieved.
Now let’s compare Metroid to a game which came out much later but was developed in the spirit of the NES, Shovel Knight: Shovel of Hope.
SHOVEL KNIGHT: Shovel of Hope – A modern take on games from the 8-bit era
While some liberties were taken by Yacht Club Games to stay coherent with the capacity of modern gaming platforms, Shovel Knight mostly resembles productions dating from the NES era.
The beginning of Shovel Knight shows our hero shattered by the separation from his partner, Shield Knight. They’ve both recently been defeated by the forces of chaos and The Order of No Quarter kidnapped Shield Knight. Our hero with a shovel now departs on a quest to save his lover! The whole narrative of the game revolves around themes of loss and resilience. To support this portrayal, the developers opted for a whimsical tone to write their dialogues.
The amount of work that the lead composer and sound artist Jake Kaufman put into the soundtrack is admirable. Manami Matsumae (Mega Man series) helped shape the soundtrack by composing two tracks for the game. In the previous entry of this series, we discussed how composing music for 8-bit consoles was everything but easy. Manual programming was required to input notes into the NES soundboard. Kaufman relied on the same method to create the soundtrack of Shovel Knight thanks to an NES emulator called Famitracker. Famitracker makes use of 3 additional audio channels added to the original NES systems to give more punchiness to the music, but no stereo-mixing or reverberation was used to keep the raw character of the sounds.
Jake Kaufman demoing the complexities of a couple Shovel Knight tracks.
Dissonant harmonies (notes that sound simultaneously) are superimposed over the up-beat melodies of the game. They are introduced by the main musical theme of the game and convey the state of hopelessness felt by our character. Due to the repetition of the pattern over the soundtrack, the music reminds the player that Shovel Knight is on a desperate quest. By the end, the track of the final boss is a clever usage of high keys and faster pacing which makes the boss fight genuinely exciting and is reminiscent of the cathartic effect of Metroid. The leitmotif disappears from the final musical piece as the boss is finally defeated.
Track: The Betrayer – Final boss encounter
Overall, the soundtrack serves and enhances the narrative. The end song was purposely used as a way to bring closure to the long separation of the two protagonists. The composition style of Kaufman is one of the bigger parts in making Shovel Knight so enjoyable – atmospheric objective achieved.
Transitioning from the 8-bit era, let’s have a look at a series of games which respectively incorporates interactive and adaptive music: Dark Souls.
Transitioning from the 8-bit era, let’s have a look at a series of games which respectively incorporates interactive and adaptive music: Dark Souls.
DARK SOULS – A clever use of interactive music
The greatest use of music is sometimes no music at all. In Dark Souls, the beautiful soundtrack reveals itself upon the player entering specific areas like boss fight arenas and bonfire hubs. While it is not a silent game, level exploration in Dark Souls is completely unscored. Room is left to sound effects to help the player focus on the gameplay while getting invested in the environments offered by the world of Lordran.
The From Software title is extolling story minimalism from all its pores, laying out its lore through item descriptions, NPC dialogs and environmental storytelling. Those who got interested in the lore of the series know that Dark Souls displays a genuinely captivating universe. In the game, you are a chosen undead whose goal is to preserve the Age of Fire and the reign of gods by renouncing your soul and linking a bonfire called the First Flame. The game’s main themes are related to fate, manipulation and power, and are supported by an austere and disillusioned tone. Composed by Motoi Sakuraba, the soundtrack is a mix of orchestral pieces accompanying fights against bosses. Yui Tanimura commented on the absence of music in level exploration in a 2013 interview:
"Because this game involves paying so much attention to your environment, to your surroundings, not just visually, but with sound, we felt that the implementation of music outside of the boss battles would get in the way of the actual strategizing throughout the game. I feel that being able to really immerse yourself and dive deep into the actual world as if you’re the character themselves, we thought the music would get in the way of that. But for the boss battles, the music obviously helps pump up the situation! So that’s basically the behind-the-scenes answer."
The main reason given by Tanimura is that sound effects have such an immersive power that they can be used as a replacement for music which would force players into cognitive overload. An example can be found in the use of the ringing sound of Andre of Astora’s hammer on his anvil at the bottom of the Cathedral of Undead Parish. The sound can be heard from the top of the cathedral and guides players in reaching a safe zone. It’s also a different kind of sensory stimulation which contrasts with graphics.
The series is overall known for integrating leitmotifs accompanying combat. A fairly recent example is the use of musical phrases matching the attack patterns of some of the bosses. The fight with the Nameless King in Dark Souls III is a great example of this technique.
Nameless King track in Dark Souls III – Extended version.
Through the use of monster screams, footsteps, weapon clashes and background noises, the game conveys a grim sense of realism. We can see players’ reception of the sound design of the game by browsing Reddit, here’s an example:
A typical Reddit thread concerning the soundtracks of Souls games.
In addition, light melodic touches were added to bonfire areas to enhance those awe-inspiring discoveries – atmospheric objective achieved.
Firelink Shrine theme in Dark Souls.
Some honorable mentions:
Subnautica - music and SFX: Isolation and survival themes are supported here by amazing underwater sound effects. Eerie, scary and definitely atmospheric.Little Nightmares - SFX: Here, the notion of fear and vulnerability is conveyed by various sensory techniques the sound effects which were directly inspired by the movie Eraserhead from David Lynch. Quite the claustrophobic atmosphere.
Hob – music: A mix of moog guitar, fretless bass and pedal steel creates a futuristic and rubbery, yet pretty lead instrument. It enhances the elusive and intriguing character of the narrative.
Doom 2016 - music: The work of Mick Gordon on the soundtrack of Doom has received a lot of praises from the industry during the BAFTA awards 2017. A great soundtrack which perfectly intertwines music and gameplay.
Part 3 – Voice-acting, the right mix of intention and delivery
When it comes to voice-acting and atmospheric engagement, there is no in-between – you either get it right, or you don’t.
On the “don’t” side, there are plenty examples of games which lacked funds, competencies, actors or time. It’s a shame due to the impact it has on immersion. Developers are not writers for the most part and it can show. Hiring professionals or at the very least people that have a sense for storytelling and acting is essential to achieve a coherent end product. That’s not to say bigger budget productions always get it right either. Part of the reason is that voice actors do not always have footage of the game to base themselves on during their performances. This sometimes makes for a harder time in getting dialogues right.
On the “do” side, while voice-overs should stay true to the tone and mood of the game, it doesn’t mean that dialogues should be predictable. It’s important to get off the beaten path to pleasantly surprise players. There are already hundreds of games belonging to specific genres such as sci-fi or fantasy. In this sense, dialogs in games will often be imbued with the lexicon and “feel” of the genre. To stand out, developers need to challenge players’ expectations. Wittiness of the script, variety of the voice actors and quality of the delivery all have a role to play.
Games that got it!
PYRE – When voice acting supports narrative
Here, we’ll take a closer look at Pyre which was made in 2017 by the team at Super Giant Games. Like most titles of the studio, Pyre features outstanding voice acting, supporting the deep and involving story.
Mixed feelings of compassion and regret emerge from the narrative as the game tells the story of a group of outcasts exiled from the Commonwealth civilization. They were cast into the landsat the bottom of an infinite pit called the Downside. At the head of a team of outcasts and in a similar fashion to Fire Emblem titles, the player’s goal will be to return the newly met exiles to society by winning Rites.
The gameplay consists of a revisited tournament of 3vs3 handball games with other sports mechanics added to the matches. Thing is, not all is rosy in the world of Pyre, and the player will have difficult decisions to make regarding the destiny of the protagonists.
Revolving around themes of redemption, transformation and justice, the game has a sometimes stern, sometimes compassionate tone. While not all characters speak a discernable human dialect, the quality of the delivery allows players to get attached to the characters. Greg Kasavin, the creative director of the game, recalls in a 2017 interview:
"This time we really wanted to make a game with an ensemble cast – a larger group of characters you could get closer to. We quickly gravitated toward the idea of these characters being trapped in some sort of mystical purgatory, and having to depend on one another to achieve both a shared goal – namely, their freedom – as well as their personal goals. And it all just flowed very naturally from there."
Pyre campaign tutorial. A narrator’s introduction to the game.
There is also an intradiegetic narrator voiced by Logan Cunningham. His role is to lay out the ground tone of the story by narrating the character’s journey during cut scenes. He also dives into a more proactive narration style when commenting gameplay phases. This has a huge impact on the immersive capacity of the interactions the player has with the game while offering insights into the lore of Pyre. To complement those narrated parts, Darren Korb composed an outstanding soundtrack which you can discover in the video below – atmospheric objective achieved.
Lone Minstrel Ballad in Pyre. A breath-taking example of the perfect mix between dialogues, singing and sound effects.
Some honorable mentions:
The Stanley Parable: The journey of Stanley would be a boring experience if the narrator wasn’t inputting some of his wit to the story. Thanks to the voice of Kevan Brighting, the game is a marvel of comedy.Divinity Original Sin: The game is fully voice-acted and the quality of the delivery is great for the most part. There is also a badass voice-acted rabbit, mate. 10/10.
Persona 5: The choice of casting in Persona games always makes for flaming debates. Thanks to over the top stories, the actors have room to express the full potential of their characters.
Achieving the right mix of music, sound effects and voice acting is a daunting task. One way to ease the process is to have the whole team behind the audio of the game work collaboratively. Music needs to be composed with full knowledge of the audio space that sound effects and dialogues will take. Though we hope to have demonstrated in this article that above technical factors, creating the audio of a game is a passionate craft of love.
References
Napolitano, J. 2013. Why Dark Souls doesn't have music outside of boss fights. Destructoid.com. Retrieved from Destructoid.
Wagner, P. 2017 An Interview on Ludomusicology: The Language of Music in Dark Souls. LanguageatPlay.de. Retrieved from Language at Play.
Dreyer, P. 2016 Supergiant’s Pyre burns bright with promise. RedBull.com. Retrieved from RedBull.
Salomaa, T. Music and Sound in Nintendo's Metroid. Retrieved from source.
Zhang, J., Fu, X. 2015 The Influence of Background Music of Video Games on Immersion. Journal of Psychology & Psychotherapy. Retrieved from Journal of Psychology & Psychotherapy.
Iyer, A. 2018 How Video Games Use Reduced Stimulation. Medium.com. Retrieved from Medium.
D'Angelo, D. 2014 Breaking the NES for Shovel Knight. Gamasutra.com. Retrieved from Gamasutra.
Wagner, P. 2017 An Interview on Ludomusicology: The Language of Music in Dark Souls. LanguageatPlay.de. Retrieved from Language at Play.
Dreyer, P. 2016 Supergiant’s Pyre burns bright with promise. RedBull.com. Retrieved from RedBull.
Salomaa, T. Music and Sound in Nintendo's Metroid. Retrieved from source.
Zhang, J., Fu, X. 2015 The Influence of Background Music of Video Games on Immersion. Journal of Psychology & Psychotherapy. Retrieved from Journal of Psychology & Psychotherapy.
Iyer, A. 2018 How Video Games Use Reduced Stimulation. Medium.com. Retrieved from Medium.
D'Angelo, D. 2014 Breaking the NES for Shovel Knight. Gamasutra.com. Retrieved from Gamasutra.
The importance of audio in creating a compelling, immersive gaming experience is paramount. At Altagram, we use our years of experience in the gaming industry to help game developers and localization managers get the results they want as efficiently as possible.
Check out our approach to the topic here.