Learn the art of film scoring with QuickTime movies.
Most professional sequencers feature the ability to display digital video with synchronized sounds and music. This lets composers and sound designers take advantage of a wide range of familiar audio tools for projects that require scoring to picture. The video format used with these high-end sequencers is almost always QuickTime, because it is by far the most powerful, flexible, and ubiquitous engine for delivering multimedia content to desktops and the Internet.
According to Apple, QuickTime 4 has been downloaded more than 50 million times and is installed on more than 100 million machines. Because the software incorporates streaming and compression technologies, it is practically the de facto industry standard for digital movies on the Web.
Cross-platform portability is one of QuickTime's greatest strengths. It works on any PowerMac running Mac OS 7.5 or better, where it is implemented as a set of OS extensions. On PCs, the system consists of dynamically linked libraries and requires a Pentium processor, Windows 95 or later, a Sound Blaster - compatible sound card, and Microsoft's DirectDraw and DirectSound. You can control QuickTime with Java applets, making it an ideal platform for delivering audio and video to interactive Web sites.
So, if you aspire to be the next John Williams but don't have the backing of a Hollywood movie studio, you might try writing scores for QuickTime video clips first. you can use the same techniques as the big shots to produce a film score that is locked to picture yet is small enough to be posted on the Web. Aside from your desktop MIDI studio, all you need is QuickTime Pro to get started.
QuickTime is available as a free download from Apple's Web site (www.apple.com/quicktime). The latest version, 4.1.2, installs the required system extensions, QuickTime Player (see Fig. 2), and an encrypted version of QuickTime Pro. You must unlock the Pro software with a code purchased from Apple for $29.99, which enables the Save and Export features in the QuickTime Player application. You can then create a video clip, write a music score for it, and export a high-quality, low-bandwidth movie for Web distribution.
VIDEO CLIPPINGThe best way to begin is by developing your writing chops on short video clips, which you can acquire in various ways. If your computer system has an analog or digital video input, you can capture video segments from a VCR or DV camera. In addition, a variety of shareware utilities (called rippers) let you import video directly to your hard disk from an internal DVD player. Of course, a great many QuickTime movies are available to download from the Web - everything from dancing babies to hollywood movie trailers.
When my Berklee students express an interest in writing for picture, I always recommend that they first put together a 1- to 2-minute demo that shows off their best work. It's best to start with a short but strong portfolio piece, rather than a longer sequence that may end up seeming incomplete or weak in areas. After knocking your audience dead with a hard-hitting opener, you can present a more involved piece that is 4 to 5 minutes long.
Ideally, you should tell a story with your score, support the video images, or fill in parts of a narrative with background ambience. It is particularly exciting and challenging to score a clip that moves through several different moods or invokes a variety of psychological states. To produce this effect, you must vary tempo, dynamics, and tone colors to create music that dramatically underscores the action onscreen. The point is to demonstrate your ability to skillfully tailor engaging musical scores to a director's vision.
BANDWIDTH BLUESAfter you select a video clip to score, you should make a low-resolution reference version of it to use with your sequencer. Displaying full-screen, full-motion, full-color digital video places enormous demands on a computer's processing power, much more so than digital audio. Red Book audio (16-bit, 44.1 kHz, stereo) requires about 10 MB of storage for each minute of sound, whereas broadcast-quality video demands a whopping 36 MB for each second. That's an awful lot of data for the average desktop machine to move continuously from disk to video RAM.
You must cut the video bandwidth down to a reasonable rate (say, a few hundred kilobytes per second) if you want to use a video clip with your sequencer. This prevents excessive disk reads of enormous files, and it significantly reduces the load on your computer's CPU, which must simultaneously run MIDI, audio, and QuickTime. Moreover, it must process everything smoothly and in sync. Fortunately, the Pro version of QuickTime Player provides many features specifically designed to create high-quality, low-bandwidth versions of QuickTime movies for reference and for posting on the Web.
You can squeeze a movie down in many ways, though each degrades the quality to a certain extent. You can remove unused data, reduce the frame size, decrease the graphics bit depth, and apply various compression algorithms.
First, take a look at what you're starting with: open your video clip with QuickTime Player and choose Get Info from the movie menu. This opens a small window that presents a lot of detailed information about the movie and provides access to useful parameters. A pop-up menu in the upper left corner lists the tracks associated with the movie. In the upper right corner, another pop-up menu lets you view the parameters relevant to each type of track. The Movie option displays general performance data and copyright information. The Video Track option displays information about your clip's screen size, color depth, frame rate, duration, bandwidth, compression, and other important details. The Sound Track option displays several important parameters such as sample rate, resolution, duration, compression scheme, and volume settings (see Fig. 3).
PREPPING THE VIDEOTo reduce your video clip's size and bandwidth, you should first remove unwanted data. Because you'll be writing your own score, you don't need to keep the original soundtrack (if there is one). Go to the Edit menu and select Delete Tracks; if a Sound Track is present, delete it to create a "silent movie."
You can also edit out unwanted video, such as leading and trailing footage or glitched frames and scenes. Select a region of the video for editing by positioning the triangles underneath the progress bar in the QuickTime Player window (see Fig. 2). You can also hold the shift key and drag the cursor across the bar itself. The selected area becomes highlighted, so you can cut, copy, or paste the video to suit your needs.
When the clip is down to its bare essentials, select Save As from the File menu. You will be asked whether you want to Save Normally or Make Movie Self-Contained. The first option creates a "pointer" file that contains information about the movie and the edits you made, but lacks the actual video data. This allows for efficient file storage, because you could conceivably have multiple movie versions based on the same video images. For this situation, however, you want to apply the changes you made to the data (cutting out the unused parts), so choose the Self-Contained option. This creates a flat file that includes all the necessary data in one bundle. This format is cross-platform compatible and Windows friendly.
SHRINKING THE VIDEONow that your clip is in its final edited form, use the Export command in the File menu to produce a new low-bandwidth video clip for your sequencer. The Export dialog box provides access to a variety of output parameters that control QuickTime Player's extensive file conversion and compression features. For example, you can export a QuickTime movie as an AVI file for Windows or as a digital video stream, and you can export the soundtrack as an AIFF or WAV file at various sampling rates and resolutions. For this situation, select Movie to QuickTime Movie, because you want to end up with an MOV file.
Click on the Options button in the Export dialog box to open the Movie Settings dialog box (see Fig. 4). The Video and Sound settings displayed on the right do not necessarily represent the current movie format; they're simply suggested values for export. In the Video section, click on the Settings button to open the Video Compression dialog box, which lets you choose from a number of bit-squashing algorithms. Use Apple's standard Video codec for now, because it quickly encodes the data.
You can try the Sorenson Video compression scheme if you prefer; it's one of the most efficient in terms of file size and online transfer time. But be aware that it requires an enormous amount of computational power. The version that ships with QuickTime Player can take as long as 30 minutes to encode a single minute of 240 5 180, 30 fps video on a reasonably speedy machine. Sorenson sells a Pro version of its encoder, however, that is much faster and provides access to a number of additional processing parameters.
Regardless of which compression scheme you choose, set the Frames Per Second field to the original frame rate. (Refer to the Frame Rate from the Get Info dialog box for this number.) Leave the other parameters at their suggested values. Now go back to the Movie Settings dialog box and click on the Size button in the Video section to open the Export Size Settings panel. Set the frame size to 160 5 120 or 240 5 180, particularly if you intend to work at 30 fps, because this places a minimum burden on most machines. If you have a speedy CPU and a big monitor, you can try using the 320 5 240 resolution for better clarity, but remember that QuickTime will not solely compete for processor time.
In general, the object is to decrease your clip's bandwidth to a point where the QuickTime, MIDI, and audio drivers run smoothly and synch-ronously. When you've squeezed the clip as much as you can, click on Save to apply the changes. The final QuickTime movie should be appreciably smaller than the original, and the required bandwidth should be greatly reduced.
OPEN THE WINDOWAfter you import the video clip into your sequencer, create a MIDI tempo map. This lets you determine how many music measures are required for each section of the clip that needs to be scored. Although the names and locations within the programs may vary, the basic commands and their uses are similar for most sequencers, so I'll describe the process in general terms.
When you import the QuickTime movie to your sequencer, it typically appears in a dedicated "floating" window that is always visible - a useful feature if you only have a single monitor. The sequencer's transport controls are usually linked to the movie window so that the audio track and the video images are always locked in sync. The window size varies depending on what you focus on and how much onscreen real estate is available. It's a good idea to keep the movie in the upper right corner (see Fig. 5).
Next, you should display the SMPTE time in a separate window using big bold numbers. That makes visibility and editing easier. You might also need to set the frame rate of your sequencer to match that of the QuickTime movie. If your original clip was recorded at a low rate, such as 10, 12, or 15 frames per second, set the sequencer's rate to some multiple of that number, for example 24 or 30 fps. This does not affect the frame rate of your final exported movie because the setting is only used for recording audio and MIDI.
ON YOUR MARKThe next step is to mark the frames where you want the music to change or align to specific beats. For example, if you want the suspense music to start when the villain opens a door, mark the shot of the doorknob turning. If you want a character to tiptoe in time to the soundtrack (like Wile E. Coyote), mark the footsteps. The commands for doing that vary from sequencer to sequencer, but the principle remains essentially the same.
Set a marker at your video clip's first frame and name it something meaningful like "start." Then scroll through the movie and add markers for each of your selected cues. You should be able to edit the names and the SMPTE times of the markers graphically (or delete the markers altogether) on a timeline display or by typing items into a list. The sequencer automatically keeps track of the cue points so that they're numbered in the correct order. Once everything is set up properly, lock the markers in place so tempo changes in the MIDI score don't alter the SMPTE positions of the cues.
The marker list provides a convenient and efficient mechanism for positioning the movie and the music. Selecting adjacent markers can bracket your video clip's region between cues. Leave the marker list open, along with the video and SMPTE windows, because you will refer to them frequently when setting up the MIDI tempos in your score (see Fig. 1).
GET SETTempos are stored in a special track, as defined by the MIDI Specification. Using the Tempo track, you must next create a tempo-change MIDI event for each cue that you marked. To start, click on the first marker; this moves the song position pointer and sets the SMPTE numbers to the first cue in the audio, video, and MIDI windows.
Select your score's region between the current cue and the next one. Then tell the program how many bars and beats you want that region to contain (for example, 16 bars of 4/4 with the first beat of the 17th bar landing exactly on the next cue). Finally, have the sequencer calculate the tempo required to place 16 bars between the two marked SMPTE times.
Most sequencers have a variety of tools for creating tempo maps, letting you program quick changes, accelerandos, and ritards with frame-accurate timing. As long as the markers are locked to specific frames in the movie, you can change the number of beats between cues as needed to produce the desired musical effect. The computer recalculates the tempo settings automatically.
GO FOR ITNow comes the hard part: actually writing the music. The art and science of composing compelling film scores is outside this article's scope, but the important thing to consider is how the tempo map you've already created affects your themes and phrases. The point of the operation is to produce a musical flow from one cue to the next, whether smooth or abrupt, that supports and enhances the images' emotional character.
You should use the myriad sequencing tools at your disposal to record the MIDI and audio portions of your score. It doesn't really matter if you enter the data in step mode, cut and paste from previously produced sequences, or record live to a MIDI metronome, because the tempo map keeps the bars and beats in sync with your video clip's cues. What does matter is the final format you use to export the score from your sequencer.
If your score relies on samplers, outboard synthesizers, or live recordings of musicians and vocalists, you have to mix and master to a digital audio file format such as WAV, AIFF, or SDII. You can then import the file as a soundtrack into your QuickTime movie and compress it using a variety of algorithms. However, if you want to post your movie on the Internet, you might consider writing exclusively for General MIDI instrumentation. Not only will the resulting movie score be miniscule (offering quick and easy downloads from the Web), but you can take advantage of QuickTime's built-in MIDI synthesizer.
GETTING SOFTThe QuickTime Music Architecture is designed to play high-quality, low-bandwidth versions of General MIDI files by rendering sequences of notes into digital audio using a built-in sampled instrument library. Because the synthesizer is completely defined in software, the music will sound as intended on any Mac or PC. It is also frequently used by browsers to play MIDI files from the Web.
Apple licensed a 2 MB wavetable library from Roland for the QuickTime Musical Instrument set. Based on samples from the Roland Sound Canvas (the de facto standard for General MIDI sounds), the QuickTime library includes the regulation 128 instruments plus seven different drum kits and more than 100 additional patches from Roland's GS extension to the General MIDI Specification.
Polyphony is limited only by CPU power, and you can include multiple MIDI files in a single movie. QuickTime 4 supports stereo samples and even lets users reassign patch controllers, modify envelopes, and include custom samples for musical instruments and sound effects (see Fig. 6). Overall, it is a powerful, efficient, and flexible synthesizer that can provide composers with a fairly wide audio palette.
PUTTING IT TOGETHEROnce you create a killer musical score for your clip and you have the final MIDI or audio file mixed and mastered, combine the soundtrack and the QuickTime movie into a single file. Some sequencers insert the audio into the clip automatically, but QuickTime Player also allows you to add audio layers.
Open your audio file with QuickTime Player and choose Select All from the Edit menu. Copy the Sound Track, then open your video clip and position the diamond slider (which is on the progress bar) to the first frame. Hold the Option key down while you select the Edit menu and the usual Paste command is relabeled Add. This "merges" the soundtrack with the video, rather than "inserts" the audio over blank space. (You can also use the key combination Option+ Command+V.)
Next, select the Save As command and choose the Make Movie Self-Contained option. The file should now be the size of your video clip plus the audio file and might be quite large if you used a CD-quality, 16-bit, 44.1 kHz, stereo soundtrack. This isn't necessarily a big problem if you plan to distribute your clip on CD-ROM, but the final file should be as small as possible for low-bandwidth delivery on the Internet. Fortunately, QuickTime uses some amazing audio compression schemes to facilitate posting movies on the Web.
THE BIG SQUEEZEThe QDesign Music Codec 2 (QDMC2) is QuickTime's flagship audio compression algorithm and can provide astonishing compression ratios (as much as 50:1) while retaining a great deal of the original audio quality. The resulting movie's audio bandwidth can be small enough for streaming on the Internet (even at 28.8 Kbps modem speeds), and the sound quality is usually better than comparable MP3 files. QDMC2 is a perceptual codec that uses psychoacoustic principles to achieve dramatic results.
To apply the audio compression, choose Export from the File menu as you did when preparing the low-bandwidth video clip. Select Movie to QuickTime Movie as the Export type and click on Options. The dialog box should look familiar, but this time, click on the Settings button in the Sound section to open a menu of audio compression algorithms. Select the QDesign Music 2 compressor and set the other items in the box to match the bit-depth and sample-rate of your original audio file.
Saving the movie applies the compression to your soundtrack, which usually takes a while to process. Like Sorenson, QDesign sells a Pro version of the codec that gives you access to additional parameters, compresses twice as fast, and is optimized for performance on the new G4 machines. When your computer finishes number crunching, you'll have a high-quality, low-bandwidth version of your movie complete with original soundtrack that you can post on the Internet for distribution.
LOOKING AHEADApple plans to release OS X with a QuickTime update. The new version is likely to support 24-bit, 96 kHz, multichannel audio; a VST - like plug-in architecture; and latency of less than 10 ms for digital audio throughput. Import and export of Downloadable Samples may also be supported as a way of including custom instruments on the Macintosh and Windows platforms. And look for QuickTime to access the power and flexibility of the MPEG-4 file format.
In any case, QuickTime will continue to be a powerful, flexible, cross-platform technology for delivering multimedia content to users around the globe. Whether on CD, DVD, or the Internet, QuickTime movies always require good soundtracks, and directors are looking for musicians with the necessary technical savvy to produce engaging film scores in this brave new medium. So what are you waiting for? Put that QuickTime demo reel together today!