If you've worked in the game industry, you know that each project presents a unique set of challenges. Some games, such as those for Xbox or for mobile devices, require audio tools that are specific to the platform on which you're working. Other times you're delivering straight WAV assets, and then collaborating with the development team to integrate them properly. Creating audio for games has always required myriad tools and special techniques; if anything, that's even truer today.
During the past several years, virtual synths and plug-ins have made it much easier to create audio and to switch between dozens of music cues on the fly. Hard-drive space is plentiful enough that you can rip all your sound-effects libraries to disk and implement a simple and streamlined database system. (For more information on this topic, see Nick Peck's “Finder's Keepers” article in the April 2003 issue of EM.) And game audio-specific tools such as Microsoft's XACT for the Xbox, Creative Lab's Interactive Spatial Audio Composition Technology (ISACT), and Beatnik's Mobile Sound Builder give audio professionals the ability to become much more involved in the mixing and mastering process.
Studio Gear
It used to take a rack full of Roland S-760s to generate a MIDI orchestral score worthy of at least the director's ear, if not the final product. Now you can accomplish the same thing with any number of orchestra and other instrument sample libraries. Multitrack software, a 2-track editor, some soft-instrument plug-ins, and a sound recorder with a mic still cover the basics of game audio production.
“It's getting to the point that almost everything can be created within the virtual environment, especially music,” says Aaron Marks, owner of On Your Mark Music Productions and author of The Complete Guide to Game Audio (CMP Books, 2001). “But the addition of live instruments can add a ‘real’ quality that can't be duplicated virtually. I always recommend that music composed [on the desktop] have at least one live instrument — something human to break what can often be a machine-sounding composition.”
FIG. 1: A multitrack editor such as Cakewalk Sonar, shown here, is a great platform for scoring game music and sound effects. The video window displays a frame from Ubisoft''s Myst IV.
Although a 2-track editor such as Sony Sound Forge or Bias Peak is indispensable and might be needed to deliver your sounds in their final format, the majority of sound development takes place in the multitrack environment. In addition to scoring cut scenes and assigning sound effects to video elements, multitrack software is great for creating original sound effects by themselves (see Fig. 1). You can easily import and manipulate pieces of canned material or use sounds that you've recorded in the field. (The new crop of noiseless, solid-state recorders, such as the Edirol R-1 and R-4 and the Marantz PMD670, is great for that purpose, though anyone who's ever strapped a mic to a race car at 190 mph or recorded an F-16 in flight probably has a different viewpoint on that.)
“One of the things that audio people suffer from is thinking that they need a ton of equipment to do good sound,” says composer Kemal Amarasingham, a cofounder of dSonic (Neverwinter Nights: The Shadows of Undrentide [2003], the Thief series, System Shock 2 [1999]). “I did all the ambient sounds for System Shock 2 [a famous PC game that won numerous sound-design awards] using the Sound Forge Synth tool.” Having the best mic preamps and samples recorded at 24 bits, 192 kHz helps but ultimately isn't necessary. That's especially true given that you'll be compressing the final files in order to fit your effects into a limited memory space.”
FIG. 2: Windows PCs are often the platform of choice for hardware emulators, such as Qualcomm''s BREW emulator pictured here. An emulator is used to test sounds before delivery to a client.
Even if you work primarily on a Mac, it helps to have a Windows computer around the studio so that you can test beta builds of games and make sure your assets play back properly. In addition, many platform-specific tools, such as hardware emulators for game testing, require a PC to run. For example, Qualcomm's BREW (Binary Runtime Environment for Wireless) is a Windows application that is often used for developing sound for cell phones (see Fig. 2).
Sound Effects
Once you have your source material, the next step is to develop sophisticated layered effects that are larger than life and jump out at the player. Layering gives you a unique final result, marking your sounds with their own sonic signature. It can also help to better integrate sounds into the game's audio landscape. For example, in Mumbo Jumbo's Chainz II (2005), I used a combination of chain-link sounds, taps I recorded on a metal pole, and gun and other weapons explosions to create the sound of two chains meeting and exploding on screen.
Light to moderate compression helps integrate the layers and give the sound more punch. Using EQ, shape the final sound to roll off gradually below 60 Hz, dip slightly in the lower midrange, and brighten slightly in the top end above 5 kHz. You can also use EQ to tame individual layers or to bring out certain sonic characteristics (for example, brightening the clink from a metal pole or adding punch to midrange gunshot sounds). Adding a touch of limiting is useful to tame any stray peaks.
Some sounds, such as fire, force-field magic-spell effects, mining, or chopping wood, have to loop. It's simple to do using the Pencil Edit (for removing clicks) and the Copy and Paste tools in a 2-track editor. (The game engine will take care of the correct number of loops; usually, you'll have to provide only one repetition of the sound from beginning to end, however short.) Always check your finished sounds on speakers and headphones to make sure there aren't any audible clicks or other artifacts.
Normalize all effects in the game to the same level (for example, 97 percent) and test them in sequence to make sure the overall volume, timbre, and EQ are consistent from effect to effect. Then get the delivery specs from your project lead. Mobile phones typically use 16-bit, 8 kHz mono; and game consoles normally use 16-bit, 22 kHz mono. Some developers prefer to use OGG format in their PC games so they don't have to pay MP3 licensing fees. Sound Forge can do all of those conversions without a problem. If you've got many dozens of effects to convert, programs such as the open source CDex (for importing) and Steinberg WaveLab (for exporting) can handle batch processing.
Composing Effective Music Cues
Placing music cues strategically has become preferable to running music in the background continuously in certain games. The original Tomb Raider, released by Eidos in 1996, was arguably the first popular example of that type of selective music placement. Adaptive music, which is music that changes in response to game play, has also increased in use recently. “For the project we're currently working on here [Gauntlet: Seven Sorrows, 2005],” said Alexander Brandon, audio manager at Midway Games, “we're using GigaStudio orchestral pieces because of the adaptive nature of the soundtrack. Sometimes the music needs to change two or three times within the design of a level.”
Deciding where to place music and ambient effects can be a collaborative process with the project lead. “Normally, I'll play the game to become familiar with the level I'm scoring for — especially to get a good sense of the pace of that level,” Amarasingham says. “Next, I'll go through and place all the sounds, ambience, and music, and then present the piece to the designers for review and discussion.” He adds, “Understanding the role of game designers and how they fit into the production process is extremely important. I'd say about 40 to 50 percent of making a game sound great is in the technical/implementation phase.”
“It's also helpful for sound designers to have at least a taste of programming experience, especially when dealing with programmers,” says Jason Kanter, the audio director at Super Ego Games, a NYC-based developer currently working on Rat Race for Atari. “Sometimes having the ability to explain a problem to a programmer in his or her own language can make the difference in whether or not that problem gets solved.”
FIG. 3: Darkworks/Ubisoft''s Cold Fear
provided composer Tom Salta with a creative sound-design and scoring challenge. This figure shows a scene from the game.
The game's genre, pace, and graphic style can dictate what types of sounds or instrumentation would be appropriate. Tom Salta is an independent composer and producer who jumped head first into the game industry in 2004. “In Cold Fear [Darkworks/Ubisoft, 2005], the game's environmental setting [a drifting Russian whaling ship in the middle of a howling storm on the Bering Sea] provided a great opportunity to incorporate elements such as metallic timbres, whale songs, gurgling water, a Red-Army-like choir, and lots of other unique elements into the soundtrack,” he says (see Fig. 3 and Web Clip 1).
“Bigger is not always better,” notes Ed Lima, audio director at Human Head Games and lead composer and sound designer for Id's Doom 3 (2004). “An orchestra wouldn't have worked for Doom 3,” he says. “It would have been too distracting, too in the foreground. By contrast, the orchestral score in Fable [2005] sounded fantastic. Peter Molyneux created a beautiful fantasy world of bright colors, and Russell Shaw's in-game score matched that perfectly.”
Another game that gets high marks for effective use of sound is Electronic Arts' 2004 release Need for Speed Underground 2 (see the sidebar “More Great Games” for other examples of great game sound). “Believable engine sounds are difficult to re-create in software because of memory constraints and the programming complexity of the software audio ‘engine’,” says Marks. “But this particular game manages to give a great performance. There are various vehicles and engine modifications available to the player; each one has a subtle difference in sound, which adds to the experience.”
Lima offers another tip: “A player will hear more elements in the game than what they see. It's not how much you show; it's how much you infer.”
Platform-Specific Tools
Game projects often require the use of special tools. One such example is XACT, Microsoft's audio-authoring technology for the Xbox (http://xbox.com/en-US/dev/contentproviders.htm). Equipped with an Xbox development kit, you can listen to and mix your sounds in real time in a playable build of the game. That means you can test and mix sound effects, music cues, and even adaptive music without having to go back and forth with the development team.
To say mobile-game audio development is a moving target is to significantly understate the problem. Whereas developing a title for Xbox, PS2, and GameCube requires three separate SKUs (at least in America), you might need 50 or more to cover all the kinds of mobile handsets available. Consequently, it takes a lot of work to test audio on all the appropriate handsets. Some models have only enough memory for MIDI effects, and older Nokia handsets require monophonic OTT files. (MIDI-to-OTT conversion software is available from Nokia.)
One possible solution to mobile-audio development comes from Beatnik (www.beatnik.com). Beatnik's software suite gives you a sophisticated interactive audio engine for mobile platforms — a rapidly growing market for games. Though you'll need to work with your game's development team to implement it, Beatnik's memory footprint is small and the advantages are numerous.
If you have a handle on the available tools for your target platform, it will make your life easier and enhance the quality of your work. “Knowing the capabilities and limitations of the product and the platform will help make the developer feel like you're a more integral part of the team — that you're someone who they don't have to waste their time babysitting,” says Marks. “But the music you're creating will sound that much better, because you know how to take advantage of a game's potential.”
You're Surrounded
Implementing surround sound requires a unique set of tools. The more the sound designer knows about the platform's capabilities, the more effectively they will be able to work with each game's audio programmers.
The Xbox is the only current-generation console platform that has Dolby Digital built into the hardware. That means you can implement 5.1 effects without having to code anything specific for the game. The other two consoles also support surround sound, but in software: the Playstation 2 has Dolby Digital, Pro-Logic II, and DTS implementations available; and the GameCube supports only Pro-Logic II.
“For a basic implementation, just having interactive sounds fly around the room, it would take good audio programmers an afternoon to upgrade their audio engine to Pro-Logic II,” says Jack Buser, manager of Game Developer Relations at Dolby Labs. “You can say that mixing a game becomes quite complex, because you don't have a static mix; you have a dynamic mix that changes in real time based on what you do with your character.”
Buser divides game audio into two categories: linear and interactive. Linear audio occurs in cut scenes and introductory movies. You score it like any other video. With interactive game audio, the mix is happening on the fly as you move your character around. “These are usually mono effects that are being panned by the engine in real time,” Buser says. “One person [who is responsible for] creating the game audio engine takes them and places them in real time.”
Creative Lab's ISACT (developer.creative.com) is a run-time mixing environment used for development on Windows machines and focuses on multichannel speaker playback systems. “We used ISACT to create gun sounds that took advantage of the system's spatial tools,” Amarasingham says. “For example, we created a minicannon that had the firing sounds in the front two speakers and the shells flying out the back two.”
Looking Ahead
The next generation in gaming is fast approaching, so getting a head start on appropriate technologies will increase your value as a sound designer. Expect to see an increasing focus on multichannel sound development, especially now that many gamers have home-theater speaker systems. Efforts to improve team-based audio development, improved audio testing environments, and game-sound mastering are other notable trends.
Microsoft, Sony, and Nintendo all revealed their next-generation console machines in May of this year, coinciding with the E3 Expo 2005 in Los Angeles. Games made for the Playstation 3 and the Xbox 360 will require high-definition, wide-screen support (at least 720p). The Playstation 3 is powered by a Power-PC based Cell processor clocked at 3.2 GHz, equaling the speed of the Xbox 360's Power-PC-based three-core CPU. The PS3 will use the Blu-ray disc format and will also play Super Audio CDs in addition to DVD-RW/+RW media.
The Xbox 360 will support Microsoft's XNA Studio, a team-based development environment. It will output multichannel surround at 16 bits, 48 kHz. Other Xbox 360 features include 320 independent decompression channels, 32-bit processing, and more than 256 audio channels. Technical details for Nintendo's new console (code-named Revolution) were not available at E3, but expect comparable capabilities. The clear trend is toward high-definition, wide-screen games with multichannel digital surround and powerful real-time mixing.
The next generation is already here in terms of portables. Nintendo released its dual-screen DS model at the end of 2004, and Sony's $249 PSP hit stores in March of this year. Both consoles work best with headphones, which naturally limits positional cues (but not the quantity and quality of content).
“The general trend is toward improvements in quality, quantity, adaptability, hyperrealism, and immersion,” says Chance Thomas, lead composer for Electronic Arts' Lord of the Rings series. Even on today's consoles and PC titles, just because you downsample your work to 22 kHz doesn't mean that the content itself has to suffer.
Effective game sound design doesn't always require esoteric tools. It's something on which any reasonably accomplished home recordist can get started. By practicing some of the tips and techniques outlined in this article, you can work on games using many of the tools that you already have in your arsenal. The equipment might not be very different, but there is a lot to learn in terms of skills and technical details. Still, game sound is something that's within the reach of the project studio owner.
Jamie Lendino has created audio for games such as Bethesda Softwork's The Elder Scrolls 4: Oblivion (2005) and Blue Fang Games' Zoo Tycoon 2: Endangered Species (2005). He is the owner of Sound For Games Interactive (www.soundforgames.com), a sound-production studio based in Astoria, New York.
MORE GREAT GAMES
When playing these games, pay careful attention to the sound effects, sound ambiences, multichannel sound, and music. Many of these titles also use interactive or adaptive music. Note how the music changes in dynamics and detail depending on the onscreen action.
Castlevania: Lament of Innocence (KCET/Konami, 2003). The Castlevania series is known for its top-quality, atmospheric music by Michiru Yamane. This latest title is no exception.
Halo 2 (Bungie Software/Microsoft Game Studios, 2004). Marty O'Donnell composed a grand sci-fi adventure score for this award-winning Xbox game.
Hitman: Contracts (Io Interactive/Eidos Interactive, 2004). Jesper Kyd composed the score for this game, which won the BAFTA award in 2005 and is getting a lot of attention.
Myst IV Revelation (Ubisoft Montreal/Ubisoft, 2004). This game features a beautifully composed score by Jack Wall.
Sly 2: Band of Thieves (Sucker Punch/Sony Computer Entertainment America, 2004). Pay particular attention to the smoothly integrated adaptive music by Peter McConnell, who was already famous for his incredible jazzy score to Lucas Arts' Grim Fandango (1998).
World of Warcraft (Blizzard Entertainment, 2004). The use of ambient sound design is notable in this online adventure installment of Blizzard's genre-defining real-time strategy game.