Spectral Vistas

Synthesizing realistic instruments for fun and profit. You've just finished your latest symphony, but the New York Philharmonic isn't returning your calls,
Publish date:
Social count:
Synthesizing realistic instruments for fun and profit. You've just finished your latest symphony, but the New York Philharmonic isn't returning your calls,

Synthesizing realistic instruments for fun and profit.You've just finished your latest symphony, but the New York Philharmonic isn't returning your calls, and you don't have enough money to hire an orchestra. So you decide to use your General MIDI synth to orchestrate the piece. But wait a minute; that violin doesn't sound very real, and that flute - who are you kidding! What's a modern composer to do?

The problem is, your ears and brain are just too good and too smart to be tricked into believing that a GM flute is really a flute. For us to believe that a synthesized sound is real, the sound must behave according to the fundamental laws of acoustics. This is a principle I like to call "acoustic viability." Let's talk about what it means to create an acoustically viable sound and what you can do to make your sounds more realistic.

When you start to examine the makeup of a particular sound, you discover that timbre is not static. In fact, an instrument's timbre varies tremendously with respect to time, amplitude, and pitch. These variations are extremely important in that they enable us to identify instrumental sounds.

Unless synthesized instruments account for these timbral fluctuations, they will not sound acoustically viable. Like natural instruments, synthesized instruments must reflect the changes in timbre that are consistent with the principles of acoustics. Many common synthesizers don't take these subtleties into consideration, but there are ways to incorporate timbre changes into your sounds.

GETTING TO KNOW YOUSo what are the properties of an acoustically viable synthesized instrument? Three important principles of acoustics come to mind: initial transients, resonance, and envelope variations caused by amplitude and pitch. These are not the only issues to worry about, but they're very important, and addressing them will dramatically improve your sounds.

Initial transients. In the first 50 to 100 milliseconds of an instrumental sound, its spectrum is very unstable. This is due to inertia, the law of motion that states that things at rest tend to stay at rest unless acted upon by an external force. For example, the air column inside a saxophone has a certain amount of inertial resistance that must be overcome before it will vibrate properly. During the first 50 milliseconds of a note, that inertial battle produces wild spectral fluctuations called the initial transient. Pitch goes haywire, chaos ensues, and then you hear "saxophone."

Initial transients can vary dramatically at different pitches and amplitudes. For stringed instruments, higher pitches are generally produced by shorter lengths of string and thus exhibit less inertial resistance and shorter transients. Low pitches are produced by longer lengths of string and thus exhibit more inertia and longer transients. These variations in initial-transient behavior provide important cues as to what instrument is making the sound.

The impact of initial transients on an instrument's sound is twofold. In wind instruments, there is some pitch fluctuation caused by the instability of the air column. Second, the timbre of the sound is incomplete, nonharmonic (that is, the partials are not multiples of the fundamental frequency), or both. As the air column or string overcomes the inertial resistance, both pitch and spectrum stabilize.

Because most wavetable synthesizers are based on acoustic-instrument samples, the initial transient is preserved in the recorded samples. But if the synth doesn't use multiple wavetables (different samples for different pitches), the transient you hear could be acoustically wrong.

Resonance. It's well known that the timbre of acoustic instruments varies greatly across the pitch range. That is, the timbre of a given instrument is different depending on what pitch it's playing. As illustrated in Fig. 1, for example, the clarinet has three distinct registers: chalumeau (low), throat (written F superscript # to A superscript # above middle C), and clarion (upper). Each register has a different color and feel, but they are all part of a single, dynamic timbre.

Registers sound different because the musical instrument itself shapes the timbre through resonance. Have you ever noticed that when you sing in the shower, some notes boom out much louder than others, even though you aren't singing those notes any louder than others? This is because the shower functions as a resonator. It naturally amplifies certain frequencies based on its size and shape.

All musical instruments exhibit the same effect: The shape and size of a musical instrument creates natural regions of resonance called formants, and any frequency that falls within a formant region is naturally amplified, altering the resulting spectrum (see Fig. 2).

Remember that instrumental sounds contain many partials or harmonics. Consider an instrument with a formant region centered at 600 Hz. If you play a note with a fundamental frequency of 200 Hz, the third harmonic (600 Hz) is amplified, contributing significantly to the sound's spectrum. A note with a fundamental frequency of 250 Hz will have a different timbre because the second harmonic (500 Hz) and third harmonic (750 Hz) fall outside the formant's range.

How can you simulate formant resonance? With samplers, you can use multiple samples of an instrument spread out over the keyboard. A sample every three to five notes should give good results. But what if you don't have a lot of RAM in your sampler, or your wavetable synth doesn't use multiple samples? Another solution is to put a resonant bandpass filter in the signal path.

Bandpass filters can reasonably simulate the effect of an instrument's formants. The filter boosts frequencies inside the band, simulating the formant effect. However, if the ratio of the filter's center frequency to the filter's bandwidth (often called Q) is greater than 20, the filter begins to ring like a bell. This can be a cool effect, but it takes the formant a step too far.

Envelopes, amplitude, and pitch. The speed of an instrument's pitch envelope (during initial transients), spectral envelope, and amplitude envelope varies with both the pitch of a note and its dynamic level. This seems rather intuitive, and it's also related to inertia. However, there are some interesting interactions worth considering.

For example, pitch affects the amplitude envelope in various ways. Higher pitches have to move less mass, so the attack and release times of the amplitude envelope are shorter. This is because energy dissipates more quickly from small masses. Lower pitches have to move lots of mass, so attack times are longer. They also dissipate energy more slowly, so release times are longer as well.

Amplitude also affects its own envelope. When a note is played loudly, there is a lot of immediate energy to overcome inertia, so even low notes have quicker attack times with loud dynamics. In addition, the inertial resistance is overcome more quickly, resulting in shorter initial transients.

More important, loud notes have more partials than soft notes. In any wind instrument, the air column vibrates at several frequencies simul-taneously, each a multiple of the fundamental frequency. The louder a note is played, the more energy is pumped into the instrument, which stimulates higher-frequency vibrations (see Fig. 3). Each of these frequency components has its own amplitude envelope (these are called spectral envelopes to distinguish them from the main amplitude envelope). As amplitude increases, so does the speed of the spectral envelopes.

TRY THIS AT HOMETo incorporate some of the acoustic properties described so far, you'll need to adjust various parameters on your synthesizer. For example, a note's pitch and Velocity can be assigned to modify the attack and release times of the pitch and amplitude envelopes; this is called scaling. Look for features called something like envelope scaling, Velocity scaling, or amplitude scaling in your synthesizer's manual to find out how to accomplish this. Basically, you want to set it up so that higher pitches and Velocities shorten the pitch- and amplitude-envelope attack and release times.

Many samplers let you switch between soft, medium, and loud samples based on Velocity. You can also assign different samples to different keyboard zones, which sounds more realistic as you move up and down the keyboard. Of course, RAM requirements increase with either of these techniques.

If you own an FM synthesizer, you can assign Velocity and/or pitch to scale the modulation index, which affects the brightness of the sound and is an important factor in producing realistic sounds on these instruments. Finally, try assigning pitch and/or Velocity to control the cutoff frequency of a lowpass filter; the higher the pitch or Velocity, the higher the cutoff frequency of the lowpass filter and the brighter the sound.

The Roland JV series of synths provides a good model for experimentation. These instruments use multiple samples (Roland calls them Tones), each of which can be enveloped, filtered, chorused, and reverberated. Tones can be layered in multiple combinations, and you can activate different Tones based on note number or Velocity value. By scaling filter resonance and envelope attack and release times with Velocity and using different Tones for different pitch registers, you can put together compelling synthesized instruments.

Using pitch and Velocity to scale the timbre and envelopes is perhaps the most important technique you can use to create acoustically viable sounds. You'll be amazed at how powerful these techniques can be for suggesting realistic acoustic behaviors.

IS THAT ALL?In addition to the general principles described here, all acoustic instruments have unique properties that you should get to know when attempting to synthesize them. For example, unlike other woodwinds, the clarinet's spectrum contains primarily odd-numbered multiples of the fundamental frequency. In addition, the timbre changes very little during the sustained portion of a note. Starting with nothing more than a simple square wave, a filter, and an envelope generator, you can quickly create some convincing clarinet sounds.

On the other hand, brass instruments share certain acoustic properties. For example, notes at different amplitudes produce radically different spectra. In addition, higher partials generally take longer to sound than the lower partials. Certain brass instruments, such as the trumpet and trombone, have a cylindrical shape, while the French horn and tuba have a conical shape. These are just a few of the many factors that affect the "acoustic signature" of a musical instrument.

In addition, you need to understand how instruments are played, or more importantly, how they can't be played. A single piano note cannot crescendo, and you can't expect a woodwind or brass player to hold a note for more than 30 seconds. Knowing your acoustic instruments is as important as understanding your synthesizer.

GET VIABLEThe principle of acoustic viability is simple: If a sound behaves according to the laws of acoustics, people will believe the sound is real. Acoustic viability also helps convey those subtle changes in timbre that are necessary to create interesting and evocative music.

Incorporating the properties of acoustics in synthetic instruments is a powerful technique that will instantly improve your sounds. It does mean you'll be spending extra time programming your synth, but your music is worth the effort, right?