What is Dynamic Music Composition?
Try watching a film with the sound switched off. Not only will you have no clue what anyone’s saying (unless you’re a talented lip-reader), but so much of the impactful emotional content is lost. You can watch the introduction to “Saving Private Ryan” and experience the visceral brutality of war with all of it’s bloody details and be completely immersed in the action. However, if you turn off the sound, a layer of realism and believability is lost. This isn’t just because the explosions are silent, it is because of the sense of weight a well designed sound effect can add. For example, in “Captain America: The Winter Soldier”, our shield-wielding hero, Captain America, is fighting the Winter Soldier with his enhanced mechanical arm. During the fight there is a moment where the Winter Soldier punches into Cap’s shield with all of his strength. This “Unstoppable force meets Immovable Object” style moment is brilliantly framed by the audio team who make all of the accompanying sound (music, shouting, ambient sound effects) duck quickly in volume while a huge impact effect plays, and the sub woofers in the cinema shake the room. The effect is spectacular and without it, the short one-second-long moment would be unnoticeable amongst the rest of the fight scene.
The beauty of designing audio and composing music for films and television shows is that everything is scripted. The composer knows the exact moment that a tear will roll down the cheek of a weeping widow and the sound designer can add a rolling thunder clap at the exact second the main character realises he was actually the bad guy all along. However, in interactive media like a videogame this simply isn’t possible. Very little in a video game can be scripted like it is in a film. This is because it is practically impossible to predict what the player is going to do at any given time while playing the game. Sometimes developers will force events that take the control away from the player to script events that can be “composed to” like a film. Unfortunately, this can cause the player to lose immersion. Imagine walking through a beautiful autumn forest accompanied by a lush string quartet playing sweet, yet melancholy music. You can walk about at your own pace, taking in the sights and smells, watching the leaves drift down from the canopy above. You approach a cliff, the rocks are worn and the mossy undergrowth can no longer be felt under your feet. Suddenly your head is yanked around 180˚ and your body automatically walks to the edge of the cliff. An owl flies gracefully past you and into the open air over the cliff; a gentle flute melody plays.
Imagine how much more impressive and immersive it would be if instead of having your control ripped away as you are forced to face a scripted event, the event only occurred if you approached the cliff edge by your own curiosity. Players that had no interest in looking over the cliff edge would never see the owl and his flute accompaniment but would retain control of their experience and be completely unaware they missed anything at all. This is the challenge of composing and designing sound for a game; your music and audio placement all has to dynamically react to how the player is exploring your world. It is a challenge that a lot of composers absolutely relish.
A common technique is to compose multiple ‘layers’ of music that work individually as well as with any other layer of the set. The composer will work with the audio programmer to create a system to judge the atmosphere the music needs to help illustrate. With this system the composer can create multiple variations of a piece that have different intensities and characteristics that support what is happening on screen. For example, lets go back to that lovely forest. You are walking along in peace while the string section plays its harmonies and gently rises and falls. This continues the entire time you are ambling along the path until you see a man standing on the path in front of you. A low bass tone and a drum begin to play as the strings shift into a suspenseful phrase. As you approach you are unsure what to expect, he might be an enemy or he might be a friend, do you want to take your chances or do you draw your sword? You decide that the risk is not worth taking and ready your weapon, he notices the shift in your stance and draws his own. The music shifts up a gear and a cymbal crashes as the horn section makes its entrance. In a deafening crescendo of sound, you clash swords with the stranger in a fight to the death.
The multiple layers in this example; slow strings, intense build, battle music, etc. are all technically playing at the same time. The clever system the programmer has put in place will look at the situation the player has found themselves in and adjust how loud the different layers are played. In it’s simplest form might work like this: Has the player just been hit in the chest with an arrow? Raise the volume of the intense battle layer and completely mute the nice relaxing layer. Is the player sat there doing absolutely nothing? Bring up the relaxing layer and make the atmospheric sound effects (wind in the trees, water in a babbling brook, etc.) louder. In short, the music reacts to what the player is doing in the game.
Currently Dynamic Music is not too widely used, or even known about. It is rather complex to implement and it can be a major challenge for composers and developers to work through. The main flaw with Dynamic Music is that it often won’t have as much thematic detail as a traditionally composed soundtrack. You will often only hear one or two different pieces that fit a scenario and they will have to be used in many places along the player’s journey. In traditional composition, the music can be tailored to each unique scenario or character in the story. The best time to use a Dynamic Music system is when you have a smaller budget and a less cinematic end goal. As we become more involved with technology and new media solutions such as Virtual Reality, I can see Dynamic Music finding it’s feet as a real player in the multimedia scene. Once we get better at producing diverse and reactive music systems, they may even be applicable to interactive films and animations. It’s an exciting idea and I hope that developers and producers in all forms of media take interest in this fascinating and unique style of music making.