Master Class: Audio Alchemy

Learn how to use convolution to transform your sound files.BONUS MATERIALAdditional Resources: Click for a list of good programs that support convolution.Web Clips: Listen to audio examples that demonstrate the many things you can do with convolution.
Author:
Publish date:
Image placeholder title

Lurking inside your computer are dozens, perhaps even hundreds, of unique sound effects. No, I'm not referring to some random sample libraries that have mysteriously appeared on your desktop. Rather, I'm talking about all of the audio files you own, each of which can become a source for creative and one-of-a-kind effects processing if used as an impulse response with a convolution program.

Image placeholder title

FIG. 1: This image shows the spectra of a ride cymbal (left) and a cat''s meow (right). Note the overlapping frequencies.

Convolution is a powerful technique that multiplies the spectra of two files (one file is the source, or carrier, and the other is the impulse response) to produce a vast range of effects. If the impulse response (IR) is the “acoustic signature” of a real space — a large concert hall, for example — the result of the convolution process will be reverb, which many of the convolution programs currently on the market try to achieve (see “Trading Spaces” in the October 2004 issue of EM, available online at emusician.com). An endless number of filtering effects can also be created using very short, percussive IRs — such as a single clave or marimba hit — with the results sounding like lowpass, highpass, or even comb filtering. And you can create many types of exotic echoes and delays using different types of synthetic-waveform spectra.

You get many of the most unique and original sounds, however, when you use two standard audio files. This often results in a form of cross-synthesis, in which one file takes on some characteristic of the other: a sustained chord sung by a choir, for instance, could adopt the rhythmic pattern of a drum loop, or a stream of white noise could be molded into a 4-note seventh chord. Clearly, the sound-design potential is unlimited.

We've covered convolution in several previous articles (see “Sound Design Workshop: Convolution Reverb and Beyond” in the April 2005 issue and “Square One: Convolution Number Nine” in the June 1999 issue). But this time I'll go into a lot more depth about the creative uses of the technique. I'll discuss how to pick good files to produce interesting pairings and crossings, what tweaks you can make to your IRs to improve their potential, and how you might add convolution to your work flow. Many of the tips and tricks come from Virtuasonic's Alessandro Camnasio and Spirit Canyon Audio's Darrell Burgan, both of whom produce libraries of IRs intended specifically for sound-design purposes (see the sidebar “Dedicated Collections”). There are numerous programs that support convolution, so look around your desktop to find out what you might already own or see the online bonus material “Get in the Game” for a list of some of the best options.

Before discussing specific tips, both our experts stressed the importance of lowering the volume on your playback system as you conduct your convolution experiments. It's likely that you'll pair two files with strong peaks in the same region, which will produce resonances with very high amplitudes. At all times, be sure to moderate your listening level to avoid damage to your speakers and your ears (you have been warned!). Fortunately, many convolution programs offer gain adjustment in case you need to make a rapid move for the mouse.

Filter Fun

Getting started with convolution means finding IRs that you want to use and then deciding how to put the process into play. If you want to start simply and explore convolution's filtering potential, look for some short drum sounds, interesting sound effects (machines, clicks, gears, and such), or other broad-spectrum samples. On the PC, most convolution programs let you use any WAV file as an IR, so you should already have loads of material to work with.

On the Mac, different formats are supported by different programs: Apple Space Designer supports SD II, AIF, and WAV, among others, while Audio Ease Altiverb (also available on Windows) requires split-mono files. Check to see what the options are for loading IRs into the software you plan to use, then consider copying potential IRs to a dedicated folder so you won't waste time during a work session searching for them all over your drive.

Image placeholder title

FIG. 2: You can isolate any region of frequencies in Adobe Audition''s Spectral Frequency display. In this figure, middle regions of both channels (left) have been trimmed away from the rest of the file.

Burgan describes what some of convolution's potential for filtering is and how to get good results. “The output of convolving an IR with a signal is the spectral overlap of the two signals. Another way of saying this is to say that only the frequencies that are present in both signals will end up in the convolved signal. Musicians can use this to filter signals arbitrarily,” he says. “If they want to have as much of the spectrum of the original signal as possible, then the IR should contain as much of the audible spectrum as possible — white noise, for example.”

Burgan continues, “Much fun can be had by using IRs that have a much smaller spectral range, however. You could ensure that an IR causes a highpass-filter effect simply by ensuring that the IR itself has no frequencies below the cutoff. Taken to the logical extreme, if an IR has a discernible pitch — in other words, only one dominant frequency (say, A 440) — then anything convolved with that IR is going to sound like it has that pitch. If the pitch matches the pitch of the song, very interesting things can result.” According to Camnasio, using multiple convolution passes on the same source and filtering each differently (highpass, lowpass, bandpass, and so on) gives you even more possibilities. This would allow you, for instance, to keep only some desired range of frequencies in the final output.

For examples of convolution as a filtering effect, see Web Clips1, 2, 3, 4, 5, and 6. In these examples, I've used the same male vocal sample with different IRs, each of which produces a different result.

Moving On

To move past the filtering approach, there aren't too many hard-and-fast rules. Convolution is a technique that is well suited to experimentation — you can try convolving just about any two files and see what happens. You can also tweak the settings of many convolution programs to improve your initial results if they aren't totally satisfactory (see the section “Tweaking Time”). There are, however, a few guidelines to consider when matching up files for convolving.

For starters, the two files you choose to convolve should have some frequency components that overlap. That is because when the two spectra are multiplied, the regions that they have in common are emphasized, and the regions that have nothing in common produce only long strings of zeroes in the resulting audio file. Camnasio says, “When the spectra of the two files overlap, it is quite easy to obtain a well-balanced convolution, especially if your source has a wide spectrum. This makes drums and other forms of percussion sounds good candidates for fast and very easy convolution. You can often get away with little or no equalization, depending on your personal taste.” He also points out that vocal samples have a lot of potential for convolving.

You can probably determine whether there is any overlap just by listening carefully — no doubt you'll notice, for instance, that a cat's meow and a cymbal share spectral content (see Fig. 1 and Web Clip 7). But if you want a more accurate opinion, open the spectral-analysis window of your favorite audio application and compare the content of the two files. If you don't own a suitable program, check out Praat (www.praat.org) or Christoph Lauer's Sonogram, both of which are cross-platform. Or pick a more basic spectral-analysis tool from your favorite music-software download site.

If you want to use only a portion of the IR's spectrum (everything between, say, 100 and 1,500 Hz over the first 3 seconds, for example), then simply cut or trim away the part you don't need. You can easily do spectral-domain editing of this type with Mike Klingbeil's excellent cross-platform analysis-resynthesis tool, Spear (www.klingbeil.com/spear). Adobe Audition and Steinberg WaveLab 6 (among others) also have similar spectral-domain editing features (see Fig. 2).

Image placeholder title

FIG. 3: Unlike some other programs, Sonar''s Perfect Space imposes no limits on the length of the IR. The IR shown here lasts 45 seconds.

The duration of the files you use shouldn't be a limiting factor, nor do the IR and the carrier need to be the same length — it doesn't hurt to try files of different lengths to see what type of results you get. But keep in mind that many convolution programs limit the length of the IR. Sony Acoustic Mirror, for instance, found in the Effects menu of Sound Forge (all recent versions), has a 12-second limit, and Waves IR-1 limits the IR to 6 seconds. But Altiverb and Cakewalk Sonar Perfect Space impose no limit (see Fig. 3). As Burgan notes, the duration of the IR correlates directly with the amount of “smear” that happens when that IR is convolved with an audio signal. You can either shorten the IR directly by using your audio editor or use the convolution engine's envelope function to limit its duration. (Voxengo's Pristine Space, Christian Knufinke's SIR2, and others offer IR envelopes, which I'll cover in a moment.)

Choosing files with the same tempo can also produce good results. Burgan explains, “Let's say you have a song that is 100 bpm. Let's also say you have an IR that has some kind of modulation or motion in it that comes and goes in a tempo that matches 100 bpm. Also assume that the IR and the signal have the same sample rate. Now, if you convolve a 100 bpm signal with this 100 bpm IR, the beats will line up and very interesting things happen, particularly if you run something percussive through the IR, like a drum loop. Our IR collection Kaleidoskopy provides a large number of bpm-synced IRs for precisely this purpose. Run a 100 bpm loop through one of the 100 bpm IRs, and you can get some truly fun tempo-synced drum madness.” Web Clip 8 illustrates this approach. (You can find other examples on the Spirit Canyon Audio Web site at www.spiritcanyonaudio.com.)

Time and Again

Once you've identified a pair of files you want to convolve, there are many ways to put them to use. In most cases, using your convolution software as a plug-in effect on an audio track is the most efficient way to try out different IRs on a source. For instance, maybe you're doing vocal effects for a video game and have to create numerous variations on the voice of a character. Using the vocal sample on a track in your digital audio sequencer or editor, you could apply the convolution engine as an insert effect. This works well with programs like Altiverb and SIR2, both of which run as a VST (or other format) plug-in effect.

Another option, suggested by Camnasio, is to place the convolution plug-in on the audio output track of a soft synth. Set the wet to 100 percent and the dry to 0 percent, and you'll have endless new variations on your synth sound set. Consider using short IRs in this context so that you can switch IRs in real time without a glitch from overloading your computer's processor. And be sure to disable any effects processing on the synth patch — too much reverb, for example, can lead to a muddy sound very quickly (see Web Clip 9).

You can also use multiple instances of your convolution engine in a single session and process your source file, whether audio or the output of a synth, using serially arranged instances: the output of the first becomes the source for the second, the output of the second becomes the source for the third, and so on. Another option is to use the convolution engine as the input to a vocoder — try it as both carrier and modulator to see what happens — or dial in just a bit of convolution using a send slot to add a little spice to your audio track.

Burgan recommends using a different convolution instance in the left channel than in the right, each with a unique IR (see Web Clip 10). He says, “There is nothing in the world that says the L and R channels of an IR must have any relationship. Musicians can put different waveforms into the L and R channels, and the convolution engine will produce the corresponding different L and R output, which can be useful for some things. For example, let's say you convert an audio signal to mid-side (M-S) encoding. Then, when you convolve that signal with a stereo IR, effectively the convolution engine will convolve the M with the IR's L channel, and the S with the IR's R channel. When you convert the signal from M-S back to L/R after the convolution engine, you will have different apparent spectral signatures for the center of the panning zone versus the sides of the panning zone. Done carefully, you can apply convolution only to the mid or to the side, which can result in spectacular effects.”

He continues, “Imagine convolving a signal that has a lush saw-waveform pad in it, such that the mid channel remains untouched but the side channels are mostly convolved signal. You can get very unique evolving pads this way. Some downstream delay or reverb can give the pad even more motion.” To hear this effect, see Web Clip 11. (Note that SIR2 has a utility that will build a stereo IR file using two different mono files as input. It then adds the new file to its IR list automatically.)

Burgan also suggests freezing tracks if you plan to use multiple convolution instances simultaneously. As you'll soon discover, convolution is one of the more CPU-intensive effects. Finally, don't forget the option of processing a live audio signal. That can be a great approach in a live performance or just for experimenting with your voice or some other acoustic source. If your audio software doesn't support live processing, then consider a program like Tobybear's MiniHost (Win; www.tobybear.de/p_minihost.html), a standalone VST host that can load a convolution plug-in and process any incoming ASIO audio input signal.

Click to continue reading "Audio Alchemy."

Tweaking Time

If you aren't happy with the initial results you're getting, you can do lots of things to the IR to get extra mileage from it. But before looking at those options, remember that one of the great things about convolution is that the same IR can produce different results depending on what source you pair it with. Even if you don't get good results the first time around, there's no reason not to try the same IR again with a different audio file.

Image placeholder title

FIG. 4: Christian Knufinke''s SIR2 has a feature that allows you to create several envelope shapes automatically. The impact on the IR can be dramatic.

As Burgan notes, “The effect a particular IR will have cannot usually be fully predicted because it is dependent on both the IR and the signal it is being convolved with. IRs based on true white noise or IRs that contain only one frequency are an exception. This makes it difficult to have a favorite IR, because an IR that sounds good on one signal may produce useless effects upon another signal.”

One way to tweak your results is to explore the parameters of your convolution program. For instance, as mentioned previously, many programs have an envelope function that lets you determine how much of the IR you want to apply to your sound and how the IR's amplitude changes over time. Web Clip 12 illustrates the use of the envelope feature in SIR2. SIR2 even has a feature to generate a few repeating envelope shapes automatically (see Fig. 4). The 16-segment sawtooth, for example, will create a pulsing effect on any sustained sound.

According to Burgan, “The envelope of the IR is imposed on the carrier file when the two files are convolved. For example, if an IR has a very abrupt ending, the convolution ‘tail’ that you get when you convolve a signal with it will have the same abrupt ending. One of my favorite tricks is to create an IR that has a very long, slow fade-in over multiple seconds, then an abrupt ending — the convolution tail will have the same envelope. This can be useful for creating near-real-time ‘time reversal’ effects. Try putting a signal containing someone talking through such an IR, and the result will sound almost as if you reversed the signal that had the person talking — but you can do it while streaming live!” See Web Clip 13 for an example of this approach.

Many convolution programs also let you swap channels, reverse, or filter or EQ an IR. You should also expect to find a bypass switch and wet/dry mix feature. Programs designed primarily for reverb will also have predelay, room size, tail length, and other options, which offer even more resources for exploration. Altiverb's Decay parameter, for example, imposes an exponentially decaying envelope over the loaded IR. This can produce unexpected results, depending on how well the program can track the amplitude curve of the IR. Altiverb's Size (room) parameter is also useful for shifting the pitch of the carrier (see Fig. 5). Listen to Web Clips14 and 15 to hear the impact of using different room sizes on a single low piano note being convolved by a drum loop.

Modify the IR

Rather than adjusting an IR in your convolution program or using an unedited audio file as an IR, you can modify the IR file in a variety of ways using your audio editor to make even more unique sounds. For instance, you can cross-fade two IRs to impose a modulation between two keys, or you can reverse or time-stretch an IR (see Web Clips16 and 17). Lowering the bit depth of an IR also has an impact on the resulting file.

Image placeholder title

FIG. 5: Altiverb''s Size (room) parameter can be used to change the tuning of the IR.

According to Burgan, equalization really helps improve the results of the audio convolution. He notes, “It's very important to balance the spectrum, especially if there is too much energy (resonance) in a narrow range of frequencies. Moving a medium/narrow bell at +10 dB through the entire frequency range of the IR is an effective and fast way to search for undesired or disturbing frequencies. When you find the unwanted frequency, try to adjust the bell Q and lower the gain until that frequency disappears [see Web Clip 18 href="/emusician/web_clips/audio_alchemy_wc18.mp3"].” Burgan reminds readers to lower the main volume when starting this procedure, or you might burn your speakers (and ears) with unexpected resonances.

Many times, convolution results in a dull sound with little or no high end. Camnasio makes these suggestions: “When the resulting sound is too dull, I try to equalize the IR in the low range with a low shelf or a highpass filter. This comes from a method that is well known to mixing engineers, used when you wish to add a touch of reverb to the mix and don't want to lose definition in the low frequencies. You can also try to boost the high frequencies with a high shelf on the source material before using it for the convolution.”

Camnasio adds, “When your source sound has a narrow spectrum, the result of the convolution process will depend very much on the pitches it contains. For instance, take a piano melody and try to convolve it with different IRs. You should notice that some perform well, and others less so; some produce a balanced spectrum, while others generate strong resonances that you may want to remove with equalization. If you stretch the IR, the resonances are going to change. So this could be a very interesting method to refine the IR to suit your specific source material.”

Postprocessing

Even after you've performed the convolution, there are still some techniques that can make your files more useful. Burgan describes some ways to process your final convolved files to improve their sound. “Because of the nature of convolution,” he says, “resonances can occur very easily. For example, let's say a frequency of 600 Hz is strong in both the IR and in the signal being processed. The convolved result will have a very strong signal at 600 Hz, because the two signals resonate with each other at that frequency. There is no way to avoid this, nor can one predict in advance where resonances will occur, because it depends both on the IR and on the signal being processed. Therefore, it is a good idea to always follow a convolution engine with a good parametric EQ so that you can tame the resonances that will inevitably appear. In this case, I'd use a steep notch filter at 600 Hz to reduce the amplitude at that frequency, assuming you don't find the resonance useful.”

Burgan concludes, “People who have really mastered the art of convolution processing will create IRs specifically for the signal they wish to process with a goal in mind, applying the notions of spectrum, envelope, and duration to craft an IR that accomplishes the desired goal.”

No doubt making your own IRs is the “power user” approach, and if you have the time and inclination, it may be the best way to ensure that you'll get the results you're after. But more than likely, you already have many files on hand that you can use to explore convolution, and the collections from Spirit Canyon Audio, Virtuasonic, and others offer endless additional material for experimentation.

Convolution has vast potential for sound-design and compositional purposes and is one of the most underrated resources around. No matter what type of music you produce or which tools you use for that purpose, the files sitting on your hard drive could be just right for creating new, unique, and colorful sounds. Give convolution a try and see what types of happy accidents you come up with.

Associate Editor Dennis Miller uses convolution in the soundtracks of many of his mixed-media works. Listen to them atwww.dennismiller.neu.edu. He wishes to thank Alessandro Camnasio and Darrell Burgan for their help with this article.

Dedicated Collections

Though it's possible to match up any two files for convolving, there are several excellent collections that provide a massive number of IRs created specifically for sound-design purposes. These collections are organized in ways that make it easy to find an IR at a certain tempo or one that produces a desired quality in your sound. Note that none of these files come in split-mono format, so you'll need some software that can do the conversions for you if your host program requires it. On the Mac, Monkey Tools Sound Grinder (http://monkey-tools.com) is an inexpensive option;on the PC, several audio editors (Sound Forge and WaveLab, for example) include batch-processing features.

Spirit Canyon Audio
www.spiritcanyonaudio.com
Spirit Canyon's three collections are Kaleidoskopy ($34.99; 2,000-plus IRs), Spectral Relativity ($24.99; 850-plus IRs), and Sanitarium ($34.99; 500-plus IRs). Each collection comes both on CD-ROM and by download, and each is well organized in categories covering specific tempos and timbres (Colors, Cosmic, and Industrial, for instance). The IRs come only as WAV files and are 24-bit, 44.1 kHz (Spectral Relativity is 16-bit), with durations ranging from under a second to a few seconds. You'll find lots of audio examples and free IRs from the different collections at the manufacturer's Web site.

Virtuasonic
www.virtuasonic.com/synesthesia.htm
Virtuasonic produces a single IR library called Synaesthesis ($64.95) that consists of more than 1,000 24-bit, 44.1 kHz IRs in WAV or AIF format. The files are grouped into folders listed by letter (A, B, EF, GH, and so on), so it's a little tricky to track down a file if you have a specific purpose in mind. But given that experimentation is the name of the game with convolution, there's no reason not to start at the top and work your way down the alphabet. The manufacturer also includes a searchable HTML-based list of all the IRs, so clicking on categories such as Swirling or Metallic can point you in the right direction.

Emmanuel Deruty
www.1-1-1-1.net/pages/impulses/index.htm
Sound designer and composer Emmanuel Deruty has put together a unique set of IRs collected from his research activities and other sources. Some use Lorenz functions to generate spectra that are then synthesized as audio files for use as IRs, and others are sampled from acoustic sources such as musical instruments. The entire collection is offered as donationware.

Additional Resources

Convolutional empiricism: a comparative study of different IRs on source files

Convolution theory: more on convolution

BONUS MATERIAL
Additional Resources: Click for a list of good programs that support convolution.
Web Clips: Listen to audio examples that demonstrate the many things you can do with convolution.