Customizing QuickTime MIDI

In 1991, after finishing my last tour with bluesman Joe Louis Walker and the BossTalkers, I sent rs to every music-software developer I could find, hoping
Author:
Publish date:

In 1991, after finishing my last tour with bluesman Joe LouisWalker and the BossTalkers, I sent résumés to everymusic-software developer I could find, hoping to get into a newline of work. Steve Hales had the foresight and generosity to offerthis burned-out road dog and blues-piano player a job as studiodirector of his new company, Halestorm.

While at Halestorm, I provided content for SoundMusicSys, asoftware sample-playback synthesizer developed by Hales and hispartner Jim Nitchals. They formed Halestorm to license theSoundMusicSys technology to game developers and other creators ofmultimedia products, and business was brisk. In fact, seed money toget the company growing came from the first big licensee, AppleComputer.

The idea behind SoundMusicSys was simple: use MIDI data to fireoff instrument samples at the correct times and pitches. The filescould be small, the code would be efficient, and the resultingmusic would sound as intended on any computer. Apple wanted toincorporate the technology into its QuickTime system extension torender Standard MIDI Files, and it licensed a GM sample libraryfrom Roland Corporation to provide the instrument sounds. Applereleased its version of the software synthesizer, QuickTime MusicArchitecture (QTMA), in 1995. (To find out more about theSoundMusicSys family tree, see the sidebar “A Rose by AnyOther Name.”)

QTMA has continued to evolve. With a bigger, better samplelibrary and a host of new features and audio-compressionalgorithms, PCs and Macs can playback Standard MIDI Files with anaudio quality rivaling that of hardware sound cards. Of course,QuickTime also has the added advantage of providing soundtracksthat are locked to picture. That is a common requirement in thefilm and TV worlds, but it is often difficult to manage in thechaotic multimedia environment.

HIDDEN TREASURE

The greatest strength of QTMA's software synthesizer is that itcan create custom instruments. General MIDI (GM) is, by definition,a finite palette of sounds, and multimedia musicians typicallyprefer to be limited only by their imaginations and engineeringskills. With QTMA, however, you can include your own samples andtrigger them at the appropriate times and pitches, which lets youwrite for instruments outside the GM specification. Vocals, guitarsolos, drum loops, horn-section riffs, explosions, bird calls— whatever kind of audio you want — can be bundled withthe MIDI data and transmitted over the Internet.

This feature has been hidden in QTMA for a long time, but thelatest version of QuickTime Player Pro (4.1.2 as of thiswriting) provides access to the Instrument parameters. I puttogether a QuickTime movie that consisted of a Standard MIDI Fileenhanced with custom samples. Putting all the pieces together was alittle tricky; it required careful planning and a lot of trial anderror.

PUZZLE PIECES

I began with a Pro Tools session (mainly drum loops and hornriffs) created by house-music producers Agent K and Division 6.Next I cut up the digital-audio tracks into short sections startingon the beat. I converted the audio into 16-bit, 22 kHz System 7format sounds with SoundApp, a versatile audio-conversionutility. That produced a series of small SFIL Mac files with theaudio data stored in an SND resource. The Mac OS can play thisnative format when you simply double-click on the audio file'sicon, so it's easily recognized and imported by QuickTime.

I then created a Standard MIDI File containing one track foreach sample, with a middle-C trigger note at each place I wantedthe sample to play. For example, the “beat1” sample isone bar long starting on the first beat, so I placed a C3 wholenote in each bar, starting at bar 10, to keep the beat in sync (seeFig. 1). Other tracks played the additional beats or firedoff the vocal and horn riffs at the correct times. I also includedthe GM piano, organ, and percussion tracks from the original ProTools session.

I imported the MIDI file into QuickTime Player Pro andsaved it as a movie (MOV) file. (You can purchase QuickTimePlayer Pro for $29.99 at www.apple.com/quicktime.) When Iselected the Get Info command from the Movie menu, a dialog boxappeared, offering access to the Music Track section and itsassociated Instruments list. The tracks I laid out in my sequencerwere displayed in the same order, so I knew which tracks went withwhich samples. I then dragged and dropped the appropriate System 7files onto the correct tracks and hit Play (see Fig. 2).Voilà! (You can download the results atwww.twittering.com/K6.)

LOCK, STOCK, AND BARREL

The Instruments list also provides access to the full range ofRoland's GS sound set. This GM specification extension containslots of great sounds, including nine drum kits, myriad synths,sound effects, and ethnic percussion, as well as some nicevariations on the more standard instruments. I chose the patch forthe MIDI bass line by double-clicking on the track and selectingSynth101 from the GS Piano and Chromatic Percussion category (seeFig. 3). You can also audition instrument sounds on thelittle piano keyboard at the bottom of the dialog box.

When you drop a sample onto a track, QuickTime PlayerPro automatically creates an Instrument definition consistingof preset envelope, LFO, and effects data. The default volumeenvelope has a fast decay, which is unfortunate if you just want tofire off your sample. The sound fades to silence quickly, so I hadto open the envelope editor by Option — double-clicking onthe track name.

That caused a new Edit button to appear on the New Instrumentfor Part dialog box, which provided access to a basic set ofInstrument parameters (see Fig. 4). I put the screen intoexpert mode by clicking on the lock icon; then I opened the Volumeparameters (see Fig. 5). There, I found the culprit. TheSustain Level was set to 50 percent, and the Sustain Time was setto 5,000 ms. This meant that the sound would fade to half volumewithin five seconds. Well, I didn't want my sound to fade, so I setthe Sustain Level to 100 percent and checked the Infinite Sustainbox. I also used the Overall Volume slider to adjust theinstruments' mix levels.

Expert mode offers a large number of parameters to play with,and many can radically alter an instrument's behavior and sound.For example, resetting the default Release Time to the maximumvalue effectively creates an instrument that plays the entiresample regardless of the triggering MIDI note's length. This isgood for percussion, but don't use it on a looped sample —the loop will never stop playing.

Other parameters create a wide range of effects. The Pitch andVolume LFO parameters can be set to add delayed vibrato to a fluteor tremolo to an electric piano. You can set stereo placementdefaults, play around with the filter effects, or transpose theinstrument to a different key. You can also modify the instruments'settings from the Roland GS bank to create wild new sounds.

TRY, TRY AGAIN

After I loaded my samples and set the envelopes, I simply playedthe movie to hear the MIDI file with the triggered samples. Ofcourse, I didn't get it right on the first try; some samples wereout of place, the bass wasn't loud enough, the hi-hat was too loud,and so forth. All I had to do was modify the MIDI data and reimportthe file.

QuickTime Player Pro's editing limitations, however,made this process tedious: I had to reload the samples and resetthe envelopes each time I imported a new version of the MIDI file.When I finally mixed everything correctly, I selected the Save AsSelf-Contained Movie command, which compacted the file to about 1.6MB. That was a bit larger than I hoped it would be, so Idownsampled the sound files to 8-bit, 22 kHz, which is theresolution the Roland sound set uses. After importing the audio andresetting the envelopes for the umpteenth time, I ended up with afile size of 770 KB for my three-minute song.

Whereas creating an entire song this way pushes the technology'slimits — and my patience — a simpler approach can alsoprove to be quite effective. Say you want to use your own kick andsnare sounds to enhance a mediocre-sounding GM rhythm track. Youcan easily break out the kick and snare trigger notes into separatetracks, import the MIDI and audio into QuickTime PlayerPro, and then save it as a QuickTime movie. Adding even acouple of sound effects, vocal riffs, or guitar lines in thatmanner can liven up an otherwise ordinary MIDI composition.

LOOKING AHEAD

Despite the difficulties, developing this kind of content forQTMA has advantages over other Internet audio systems. More than100 million machines have QuickTime, which means your music has ahuge potential audience. You can also use the system to createhigh-quality, low-bandwidth audio-enhanced MIDI scores forQuickTime videos, and the audio will lock to picture just like witha real movie.

On the other hand, QuickTime has its limitations. QTMA does notsupport compression when working with individual samples, eventhough QuickTime audio tracks can be compressed with the excellentQDesign algorithm. If I could have used compressed samples withinmy MIDI tracks, the three-minute song could have been reduced toabout 100 KB. There are other stumbling blocks as well. You can'tcreate multisampled instruments; you can't select custominstruments in your tracks using MIDI Program Change messages; andyou can't play your custom instruments from your sequencer (but youcan gain access to the internal Roland bank through the Open MusicSystem).

Fortunately, assistance is on the way. QuickTime 5 is already inbeta release (the final version may be available by the time youread this article), and it includes a major upgrade and overhaul ofthe MIDI architecture. DLS and SoundFont banks will be supported,which will make the entire content-development process easierbecause of the many tools available for creating audio in thoseformats. QuickTime is such a practical Internet streaming-mediatechnology that a widespread cross-platform installed base isguaranteed.

In any case, I will be watching my favorite softwaresynthesizer's evolution. The newest incarnation appears to beemerging as an audio engine for cell phones and other wirelessdevices. Who knows where it will pop up next?

Peter Drescher is a composer, piano player, and owner ofTwittering Machine, a project studio in San Francisco. He maintainshis Web site at www.twittering.com.

A ROSE BY ANY OTHER NAME

During the mid-1990s the SoundMusicSys technology that gave riseto Apple's QuickTime Music Architecture (QTMA) underwent a numberof incarnations. Steve Hales and Jim Nitchals started a new companycalled Igor's Software Laboratories to develop their technology forInternet applications. By that time, I was independent but stillproviding content for the system; I worked on sample libraries forBe and WebTV, which had already licensed versions of thesynthesizer for their operating systems.

That's when Thomas Dolby got involved. He had a company calledHeadspace and ideas about implementing interactive audio on theInternet. He discovered the Igor synth while writing music for theWebTV box; he liked it so much, he bought the company. Renamed theHeadspace Audio Engine, the system went through further changes andeventually emerged as Beatnik, the premiere interactive audioplug-in for “sonified” Web sites.

The fact that SoundMusicSys, QTMA, and Beatnik share a commonancestry explains why QTMA and Beatnik have so many similarfeatures and parameter options. I had a definite sense ofdéjà vu the first time I opened the Instruments editor inQuickTime Player Pro. The parameters had familiar labelsbecause they were based on the same set of data fields that Itweaked for years while designing instruments for Beatnik.