Picture Window

Scoring to picture the composition and placement of music to fit with video has moved beyond the worlds of film, TV, and industrials to the mainstream.
Publish date:
Social count:
Scoring to picture the composition and placement of music to fit with video has moved beyond the worlds of film, TV, and industrials to the mainstream.
Image placeholder title

Scoring to picture — the composition and placement of music to fit with video — has moved beyond the worlds of film, TV, and industrials to the mainstream. Thanks in large part to the stunning growth of YouTube and other Web-based video, the opportunities to put music to picture have never been greater.

Image placeholder title

Virtually every major digital audio sequencer offers video support of some kind. Import a video file and hit play in your audio app, and the video plays in sync with your sequence. But beyond that basic implementation, how does your application stack up as a scoring environment? To answer that question, we took a look at 11 of the most popular digital audio sequencers, covering both Windows and Mac. Though the recording, editing, and mixing capabilities that are their primary focus are all first rate, we were surprised at the differences in their abilities to work with video.

We'll start with an overview of the key video and scoring features that are common to most of the programs. Then we'll look at the highlights of each individual program.

We didn't cover multitrack audio editors because they don't use tempo as a timing reference, and they have little or no MIDI functionality. However, such programs are useful for postproduction mixing or for simply adding sound effects (see the sidebar “Mix to Pix”).

The programs covered in this story are Ableton Live 6, Apple GarageBand 3, Apple Logic Pro 7.2, Cakewalk Sonar 6 Producer Edition, Digidesign Pro Tools HD 7.3 and LE 7.3, Mackie Tracktion 3, MOTU Digital Performer 5.11, Sony Acid Pro 6, Steinberg Cubase 4, and Steinberg Nuendo 3. Many of these programs have lower-priced, reduced-feature versions. You may want to check manufacturer Web sites to be sure that those other versions have the features you want (see the sidebar “Manufacturer Contacts” online at www.emusician.com).

Start Me Up

If you sync up almost any piece of music against any video, some of the important moments of the video will likely be “hit” by important beats of the music, just by chance. But if you're doing a scoring job, chance won't cut it. You need to control which moments get those musical accents.

Taking into account the usable tempo range for the type of music you plan to write for a particular segment of the video (aka a cue), you typically want to come up with a tempo setting or tempo map that allows you to hit as many of the key points, within a frame or two, as possible with strong musical beats. Even if there aren't specific hit points that you need to accent, the tempo you choose should help the music “feel good” against the picture.

Besides tempo, another helpful variable when scoring is the start time of the music relative to the video. Changing how many “frames in” your music starts can impact whether a particular tempo will work. You typically finesse both the tempo and the start time to find the best combination. You can also experiment with the meter of the music, throwing in an occasional measure in a different meter (for instance, a 3/4 bar in a 4/4 composition) to change rhythmic emphasis.

Make Your Mark

When you're spotting your video (scanning through it and deciding what to emphasize), it's critical to be able to drop markers on the fly, and all of the programs covered here let you do that. Once you've got markers on the work space, the programs all make it easy (to varying degrees) to name and number them. With the exception of Live and GarageBand, they all let you lock those markers to absolute time values so that while you're experimenting with different tempos, preexisting markers stay at the same point relative to the picture.

Moving an audio clip to a specific SMPTE time is also a handy option. Sonar's Clip Properties menu, for example, lets you type in the exact location where you want your audio clip to start. Pro Tools' Spot mode makes it a snap to drop an audio clip at a specific SMPTE time. Logic Pro and Digital Performer have event lists that let you do the same thing. In Cubase and Nuendo, as you drag an event backward or forward in a track, its SMPTE position updates in real time, making placement easy.

Notable Additions

Some composers, especially those who work in the orchestral realm, prefer to work in standard notation when they write to picture. Several of the sequencers, including Logic Pro, Digital Performer, Cubase, and Nuendo, offer robust notation features. Sonar offers limited notation support.

Digital Performer (DP), Logic Pro, Cubase, and Sonar all show their markers in their notation windows. DP goes a step further with its QuickScribe Film Cues view, which shows your markers above the staves, with the beat they occur on clearly marked.

Snip Snip

When you're working on a scoring job, there are times when the ability to actually edit the video can be useful. Say you're given an entire reel from a project but have to write music only for a segment in the middle. If your sequencer lets you cut and paste the video, you can get rid of all but the portion you need to work on. Or, if your video has preroll before the actual material you're scoring, you can simplify your work flow by cutting it out so that the video starts right from frame 1 of the material you're scoring.

Cubase, Nuendo, Pro Tools, and Live offer basic cut, copy, and paste video editing. While these features are much more limited than what you'd get in a full-fledged video editor, they're helpful nonetheless.

Say Cheese

A video thumbnail track, which shows you the individual frames of a video file (depending on your zoom settings), is important for locating cue points. Having a dedicated, movable video window is also useful. All the programs except GarageBand (which has a stationary video window) let you drag your video window onto a second computer monitor to give yourself a separate display area.

The ability to preview your video on an external RGB monitor comes in handy too. Depending on the hardware you're using, this can enhance playback of the video, and it will also give you a more accurate image. In addition, it can help free up space for your sequencing windows on your primary monitor, though you can also accomplish that with a second video monitor. You'll want to output to an external device if, for instance, you are printing your final audio and video to a DV camcorder or other recording device.

Normally, your software passes the picture to the external monitor using a FireWire port, which means you'll need an external FireWire-to-RGB converter such as the Canopus TwinPact. All the programs except Tracktion, Live, and GarageBand can output video through FireWire.

In high-level TV and film work, surround sound is often called for. Most of the programs we looked at, including Acid, Pro Tools, Logic, Sonar, Digital Performer, Cubase, and Nuendo, are surround capable.

Ins and Outs

With all the video formats that exist, it's handy to have an audio app that handles different video-file formats. On the Mac, QuickTime (which uses a .mov or .qt file extension) is the most common type of digital video format. On the PC, QT, AVI, and Windows Media Video (WMV) are the most frequently found. But professionals at the high end and Web video users at the low end use numerous other formats. For example, many professionals require support for Avid OMF or Sony MXF formats (both of which are supported in Pro Tools HD), while work intended for the Web might require SWF (Flash), RMV (Real Media Video), MPG (MPEG-1 or MPEG-2), and other formats. (See the table “Video-Scoring Features Compared” for a list of formats supported by each program.)

Many of these programs can import QT and AVI, and some of the Windows programs support Windows Media Video. But beyond that, things vary dramatically. To handle some formats, you may need a dedicated video-editing program or format converter. RMV shows up only in Acid, which is also the only program to support Flash files.

Moreover, although all of these programs let you import various types of files, you can't assume that you can export a video (along with your newly recorded soundtrack), much less to a different format than you imported. For example, Tracktion and Live have no support for exporting video, and Sonar lets you import but not export MPG files. Convenient as it is to import QuickTime movies directly into your sequencer, remember that running a movie from your sequencer will add to the drain on your processor.

Fortunately, the ability to import a video file into your sequencer or audio application has obviated, for the most part, the sometimes dicey process of syncing to an external video deck. But if you do need external sync, all the programs but GarageBand sync to external timecode (with the appropriate peripheral hardware).

Click here for more of this story

One by One

Next we'll look at the individual programs (listed alphabetically by manufacturer) and give you a few ideas about each one's suitability for scoring. When reading our synopses of the various programs' scoring-related features, keep in mind that there is no right way to score to picture; it's only the results that count. Every composer has his or her own method for scoring and will use only the features in a sequencer that fit his or her work style. The sequencers that will appeal to the largest swath of audio-for-picture composers, therefore, are the ones that can provide a wide range of features that are both intuitive and efficient.

Ableton Live 6 (Mac/Win; $599 boxed, $499 download)

Live's video support is in its infancy, having just been added in version 6. According to Ableton, a lot more functionality will be added in subsequent releases. Currently, Live allows you to import several different movie formats and edit the imported video using cut, copy, and paste. But you can't export your finished product as a movie file. Import is easy, though: just drag the video file into an audio track in the Arrangement view. The QuickTime movie window that pops up is of the floating variety (that is, it always remains on top of the other windows), and its audio waveform display appears like a regular audio track (see Fig. 1).

Image placeholder title

FIG. 1: Importing a video into Live is as easy as dragging-and-dropping it into the Arrangement view. You can cut, copy, and paste the video from its audio track.

Setting the start point of your music is a breeze. Just select the video track and drag the Start Marker in the clip display. The movie will follow what you're doing, and you can stop at the point you want your sequence to start. If later you want to change the start point, no problem: just drag it again. Live's excellent manual even describes a method that allows the program to temporarily disable any video prior to the start point of your music.

The easy start-time manipulation, combined with Live's adroitness with tempo, makes it an excellent app for quickly finding workable tempos for your cues. A good way to do this is to drop in a drum loop of the style you're looking to compose in, set it to loop throughout the length of the video, and then start finessing the combination of tempo and start point until you find values that work well with the picture.

You can even experiment with using Live's Warp Markers on the video track (with that track set as the Master) to alter tempos within your sequence to hit particular events. You'll have to be careful that any changes you make are subtle, or they won't sound natural.

Live is missing some important scoring features. For example, it doesn't display timecode, only bars:beats:ticks. This complicates your ability to go to a particular frame location by scrolling to it with the counter, because except at ridiculously fast tempos, a single counter tick is larger than a frame. The work-around is to click on an empty track, as close on the timeline as you can to your event. The video will jump to the point in time corresponding to where you clicked. If you're zoomed in enough, you'll be able to move the transport to an exact frame location. If your video has SMPTE numbers burned in on it (aka window burn), it will be even easier to find specific frame locations.

You can mark spots with Live's Locators, which are similar to standard sequencer markers, but they can't be locked to absolute time. Also, the program doesn't let you change meter within a song.

With its unique work flow, Live is a good choice for less-demanding scoring situations such as band and Web-site videos. Although it doesn't have the breadth of features to be the primary sequencer on a complex film or TV scoring job, its creativity-inspiring features make it an excellent ancillary scratch pad for composing your cues.

Apple GarageBand 3 (Mac; $79 for iLife bundle, included on new Macs)

GarageBand (see Fig. 2) is by far the least expensive application that we looked at, and considering its price point and the fact that it's a consumer program, it has surprisingly robust video support.

You can import a range of video types into GarageBand, and you can even drag-and-drop the files into the timeline. The video's audio track is immediately split off into its own track, a video thumbnail track appears at the top of the main window, and a video window pops up in a window fixed on the right-hand side. The counter can read out in bars:beats:ticks or in absolute time, but not timecode.

Image placeholder title

FIG. 2: Although GarageBand''s scoring tools are limited compared with those of pro sequencers, it offers a number of export options, including letting you save your movie and soundtrack as an iPod-compatible video.

With GarageBand's well-implemented support of Apple Loops, it's easy to insert loops and start experimenting with tempo. You even get a modest collection of sound effects as part of the large Apple Loop collection that comes with the program. If you zoom in sufficiently, you can scroll to any frame location by dragging the Playhead.

GarageBand offers a number of different screen-size and compression options for exporting your movie/soundtrack combination: Full Quality, Video Podcast (iPod), Web Streaming, Web, and Email. The program is also integrated with the other iLife apps, so you can import movies directly from iMovie, and export to iDVD and iWeb. You can even add chapter markers for video Podcasts.

If you try to do anything more than simple scoring in GarageBand, you will run into its limitations. For one thing, you can use only one time signature and tempo setting per song. In addition, the markers can't be locked to absolute time, and they don't show bars:beats:ticks or timecode. Further, there's no way to offset music or video start times.

For adding music to a Web site or composing some incidental music for a basic video project, GarageBand may be all you need. But for professional scoring work, you'll want a more fully featured application.

Apple Logic Pro 7.2 (Mac; $999)

Logic Pro, Apple's flagship audio application, is stocked with useful features for video scoring. Movies can be opened in a floating or nonfloating window, and you have the option of opening a video thumbnail track as well.

Markers can be dropped in with a key command and viewed and edited in the Marker List. Within that window, you have the option to lock the markers and to toggle back and forth between their SMPTE and bars:beats:ticks positions by pressing the U key. You can view your markers against the timeline by opening up the Marker Track.

Image placeholder title

FIG. 3: You can adjust tempos to fit events in Logic Pro using its Scene Markers and its Beat Mapping Global Track (top).

Logic also makes the process of constructing a tempo map easy with its Scene Markers features. When used with Logic's Beat Mapping Global Track, you can graphically adjust the tempo to hit your marked locations (see Fig. 3).

Also handy is the Tempo List window, which provides fields for entering tempo and for offsetting the audio start against the movie. Once you've set locked markers that correspond to your important hit points, open both the Tempo List and the Marker List. You can then drag up and down to change values in the Tempo field of the former, watch the effect on your markers in the latter, and find the bpm setting that most closely hits your markers.

Logic Pro lets you easily program tempo and meter changes, and the Loop Browser window gives you access to the huge collection of tempo-flexible Apple Loops that are included with the program. You also get the same sound-effects collection that is in GarageBand. Logic's comprehensive group of soft-instruments and effects plug-ins gives you access to a wide range of sonic ingredients for your scores. Logic's EXS-24 sampler is extremely useful in scoring situations, particularly because it can convert Tascam GigaSampler libraries (a favorite of film composers) and stream their contents from disk.

When you're finished composing and mixing, the Export Audio To Movie command lets you combine whatever audio you recorded with the movie file in a new QuickTime movie file. New in Logic Pro 7.2, you can export a selected portion of the movie. That could be handy if you're working on a long-form project and are scoring small sections of it at a time.

With its scoring-friendly and comprehensive tool set, it's not surprising that Logic is the sequencer of choice for many professional film and TV composers.

Cakewalk Sonar 6 Producer Edition (Win; $619)

Sonar's handling of file import and export is a cut above the rest and will be especially handy if you need to prepare files for different purposes (see Fig. 4). After you've completed your score, select Export Video and you'll find WMV, Video for Windows, and QuickTime as the available formats. Once you pick a format, you have access to an extensive set of encoding options. For example, you can change the frame rate of a file, enable a setting to automatically optimize it for downloading, and change its frame size (all the standard NTSC and PAL sizes are supported). You can also add filters to alter your video's color, adjust its brightness and contour, and more. You'll find a number of options in the same window for modifying the audio, which is a convenient touch.

Image placeholder title

FIG. 4: Sonar 6''s import and export options for video are especially robust. You can edit many aspects of a video file, including its file format and frame rate.

Like most of the apps in this roundup, Sonar can display the individual frames of a video file and send those to an external DV device for viewing or recording. It will also show your video in a dedicated window and as single frames in its video track simultaneously. Right-clicking on the Video Preview window brings up a host of settings for customizing the video display. These include manually stretching the display to any arbitrary size, expanding it to full screen, setting the time-display format (minutes:beats:ticks, SMPTE, Frames, or None), and changing the background color. You can also adjust the resolution at which video will be displayed — a low resolution may be adequate for finding cue points and will save CPU cycles during playback. More important, you can change the time in your sequence at which the video will start to play and also set in and out points within the video file.

Sonar lets you drop markers on the fly and then lock them to specific SMPTE times. This is handy if, for instance, you need to hit a cue at a specific time in the video file and want to change the tempo of your sequence. It's also easy to import a video file without importing its preexisting audio track — just leave Import Audio Stream unchecked when you load the video. You can also have Sonar split an existing stereo audio track into separate mono files when importing.

Sonar 6's new AudioSnap feature can be used to easily align a beat or transient within a clip to hit a particular video cue. You can also use AudioSnap to fit the tempo of an audio track to match the length of a scene. In addition to just typing in a new duration for your clip, which is one of its less unique features, you can hold down the Ctrl key and drag the end of the audio clip to align with the end of the video, and Sonar will automatically change the speed of your audio without changing its pitch.

You can choose from several stretching algorithms to find the one that works best for your material (percussion, mono or polyphonic, and so on), and select either the real-time stretch or “offline” stretch function depending on the amount of stretching you're doing and the quality of output you're after.

With its robust surround features, extensive video-file support, and built-in V-Vocal VariPhrase and Perfect Space convolution reverb tools, Sonar is well equipped for a wide range of scoring tasks.

Digidesign Pro Tools HD 7.3, LE 7.3 (Mac/Win; price depends on hardware purchased)

Out of the box, the video functionality of Pro Tools HD and LE are quite different. HD offers a robust set of video- and scoring-to-picture-related features, whereas LE has very little. However, LE users can expand their capabilities significantly with the purchase of the DV Toolkit 2 software ($995), an add-on that gives them most of the same video support as is in HD. Unfortunately, DV Toolkit 2 is not compatible with Pro Tools M-Powered. The features described here apply to both Pro Tools HD and LE, except where noted.

Pro Tools imports a number of formats, including QuickTime, AVI, and MPEG. Your video opens in both a thumbnail track and a resizable video window. You get a set of basic video-editing tools, so you can cut, copy, and paste video sections right from the thumbnail track (see Fig. 5). There are two different ways to scrub your video with the Scrubber tool: scrub in the video track and the video will move with no audio sounding, and scrub in an audio track, in which case both video and audio will play.

Image placeholder title

FIG. 5: Pro Tools lets you display videos in both a QuickTime movie and a thumbnail track, and its Spot Dialog makes placing events easy.

The video track's audio is automatically displayed in its own audio track. Pro Tools' Playlist feature, which lets you have multiple nested versions (takes) of a track, is also supported for the video track. You can even have multiple video tracks, although only one can be active at a time. These would allow you to alternately view, say, two different edits of the same video with your music.

The program lets you export your finished video to a variety of QuickTime formats, supporting both PAL and NTSC, and gives you options for pull-up and pull-down (which compensate for speed changes when converting from film to video and video to film, respectively). You can also choose from a number of audio sampling rates. Pro Tools gives you flexibility with tempo, meter, and audio and video start times. The Identify Beat command lets you set the downbeat of bar 1 at any point on the timeline. The Current Time Code Position window lets you easily offset the timecode in the counter to match any burned-in timecode in your video.

One of Pro Tools' most useful features for video is Spot mode. It allows you to place events (both audio and MIDI) at precise SMPTE locations, making tasks like adding sound effects a breeze. Pro Tools also supports REX and Acid files by converting them into Region Groups, which then respond to tempo changes in the session.

Because Digidesign is owned by Avid, Pro Tools has been designed to interface with Avid products in several ways. For example, if you import an Avid video sequence into Pro Tools, you can see the Avid edits.

With a Pro Tools HD setup, you've got a top-notch industry-standard professional scoring environment. Pro Tools LE with DV Toolkit 2 offers almost as much functionality. However, with either of those options (especially HD), you will be spending more than you would with most of the other applications in this article.

Mackie Tracktion 3 (Mac/Win; Ultimate Bundle $319.99, Project Bundle $129.99)

At the time of this writing, Tracktion 3 was in the late stages of beta, and Mackie expected to release it well before this issue's press time. Our observations are based on a late beta version of the application.

Tracktion 3 has a number of video- and scoring-related features. It lets you import a QuickTime video file and open up a movie window. You can choose how big to make the window and whether or not the movie window will float. (Be aware that when you set it to float, it even floats over other programs you've opened above it.)

Image placeholder title

FIG. 6: The markers in Tracktion 3 can be displayed in a dedicated marker track (top).

Tracktion 3 lets you view its ruler and counter in either bars:beats:ticks, SMPTE, or elapsed time, but you can view only one format at a time. The program offers good flexibility with meter and tempo. It supports REX, Acid, and Apple Loop files, giving you the ability to import such files and then experiment with the tempo — always a plus when scoring. It's also worth noting that Tracktion 3 Ultimate Bundle comes with an entire DVD of looped content.

Tracktion's markers (limited to a total of nine) can be displayed in the timeline or in a separate marker track (see Fig. 6), and they can be locked to absolute time. There's no way to offset the beginning of bar 1 in relation to the video, but you can set an offset for the start of the video clip in either SMPTE or bars:beats:ticks, which gives you some flexibility.

The program doesn't let you split a movie's audio file onto a separate audio track (although you can mute the movie's audio), nor can it export your finished movie and audio into a new movie file.

In its current incarnation, Tracktion doesn't have a feature set to match its larger and more expensive competitors, but for those who only dabble in scoring, it offers a lower-cost alternative.

MOTU Digital Performer 5.11
(Mac; $795)

When it comes to the sheer number of audio-for-picture features, Digital Performer (DP) is the hands-down winner. It's got what you need for everything from adding music to a video destined for your Web site to scoring a major TV show or Hollywood movie.

Image placeholder title

FIG. 7: Digital Performer offers a large number of dedicated scoring-for-picture features, including the Find Tempo window (lower left), which calculates the optimal tempos (within a specified range) to hit the markers you''ve entered.

When you import a video file, a separate Movie window opens. The Movie window has a Mini-menu that offers options for FireWire video output, window size, copying the movie's audio to its own track, and more. You can also enable an option called Chase Graphical Edits, where the video transport jumps to whatever point you're editing in a graphical display. The Movie window has its own scrollbar at the bottom, from which you can control the transport, and it has handy arrow-shaped buttons that move the video forward or backward by one frame.

One limitation of DP's Movie window is that there's no option for it to float, so if you don't have a second monitor or at least a large screen, you may find yourself having to constantly recall it from under other editing windows. According to MOTU, a floating Movie window will be included in a future release.

In addition to the Movie window, you also get a thumbnail video track in the Sequence Editor. This track can be resized just like the MIDI and audio tracks, and you can even adjust the movie start time from right in the Sequence Editor.

Not only can DP's markers be locked to absolute time or remain in their relative song positions, but they're also visible in every edit window and extend vertically through each editing surface. This makes it really easy to drag events right up to them.

Between the aforementioned movie-start time, the very flexible tempo and meter control available in the Conductor Track, and DP's unique Song and Chunk structure — which allows you to have numerous sequences (each with its own SMPTE start time) nested in a single file — you get tons of flexibility for manipulating the critical variables in a scoring situation. For instance, each sequence (Chunk) can reference the same movie or its own separate movie, so users can organize multiple cues in a single DP project document.

DP has a unique feature called Find Tempo (see Fig. 7), which makes the process of figuring out a tempo that works with your cue a whole lot easier. After creating and locking your markers, you enter parameters such as the tempo range, how many frames early or late is acceptable for each hit (marker), and an acceptable range of offsets for the movie start time. The program then calculates a large number of possible tempos and shows you which ones come closest to hitting all your markers. The tempo list updates in real time as you make adjustments to the various parameters.

DP is the only application to generate its own visual film-scoring cues. You can set the program to generate — and output to FireWire or to the Movie window — streamers, punches, and flutters, which are particularly useful for scoring sessions involving live musicians (such visual cues are de rigueur in Hollywood orchestral scoring sessions). They can be helpful for cueing the solo home recordist as well. DP's programmable click can even be set up to output a visual click (using punches). All the visual cues can be included in the exported QuickTime movie. You can choose to export the entire movie or a selected section.

DP's unmatched quantity of music-for-picture features, coupled with its strong all-around feature set, makes for a very capable, professional, and self-contained scoring environment.

Sony Acid Pro 6 (Win; $399.96 boxed, $374.96 download)

Sony Acid (see Fig. 8) is best known for its loop-assembly and tempo- and pitch-shifting features, some of which make it particularly well suited for scoring to picture. It supports a single video track, which can contain only one video clip, but you can easily move the video clip to any start point in your project and set in and out points by dragging on the clip's ends. You can't, however, extend the clip past its original length by having it loop, as you can with audio and MIDI clips.

Acid supports a fair number of frame rates, including those used with 16 mm and 35 mm film, and though you can easily move audio clips to line up with points in your video, there's no way to automatically snap an audio clip to a SMPTE start time. You can preview your video on an external monitor and change several aspects of the dedicated video-preview window, such as whether it displays square pixels, used when displaying video on a computer monitor, or nonsquare pixels, as seen on a TV screen.

Image placeholder title

FIG. 8: Acid Pro 6''s Video Preview window can be moved anywhere on the screen. Its single video track will show the individual frames of a video file.

Acid's Time Markers are the key to its scoring capabilities. Unlike a standard position marker, a Time Marker stays locked to a certain SMPTE time even if the tempo of the music changes. Using the Adjust Tempo To Match Marker feature, you can force Acid to automatically adjust your soundtrack's tempo to ensure that key points in the music align with a scene in your video. For example, if you have an audio event in the middle of your third measure that must occur on frame 2,017, put your cursor at the start of the audio event and drop a Time Marker at frame 2,017, and Acid will change the tempo of your music so the event occurs at the correct frame.

Like some others in this roundup, Acid allows you to import a video file with or without an existing soundtrack, and then save the video with a new soundtrack. The included loop library will give you a lot of raw material to use in your scores. Sony continues to improve Acid's MIDI features, and with luck, you'll find all the MIDI tools you need.

Acid is missing many of the audio-editing features of its sibling Sound Forge, but either standalone or particularly in combination with that program, it is a very robust environment for scoring to picture.

Steinberg Cubase 4 (Mac/Win; $999.99) and
Steinberg Nuendo 3 (Mac/Win; $2,499)

Cubase's and Nuendo's video- and scoring-to-picture-related features are, for the most part, very similar. As a result, we'll cover both programs in a single section and point out where there are differences.

Both Cubase (see Fig. 9) and Nuendo give you a high-level, professional tool set for composing music to picture. Import a video to the Media Pool and drag it to a track, and a floating QuickTime window pops up, as does a video thumbnail track. For the latter, you can choose Snap Thumbnails, which ensures that the thumbnail track will be frame accurate in relation to the transport. You can also opt to show frame numbers (starting from 0).

Rudimentary video editing is also available, letting you cut and paste your video (or videos) at will. Both Cubase and Nuendo offer more than one way to scrub audio and video, including the Jog Wheel, a rotating circular control on the Transport Panel. Using the Jog Wheel, you can scrub or press buttons to move forward or backward one frame at a time.

Image placeholder title

FIG. 9: Both Cubase (shown here) and Nuendo offer a wide range of audio-for-video capabilities.

Markers in both programs can be viewed in either a separate marker window or the dedicated Marker track. In the Marker window, you can view your markers' positions in bars:beats:16th notes:ticks, timecode, elapsed time, samples, or a user-definable frame rate. In both programs, you can jump to a marker position by clicking next to it in the Marker window.

Cubase and Nuendo give you a multitude of ways to offset the start points of the video and audio. One method is through the Bar Offset feature in the Project Setup window, which lets you slide the entire starting point (of both audio and video) ahead by a specified number of measures. Set Timecode At Cursor makes it easy to synchronize your timecode start point with what's on your video's SMPTE burn window (if you have one). Both programs also give you total control of tempo and meter.

For putting together a tempo map, the Time Warp feature lets you graphically drag the timeline to match specific events, and when you do, the program changes tempo accordingly. It must be applied subtly if you want a natural-sounding track, but it can be very helpful for creating a tempo map that fits your hit points.

If you want to combine your newly recorded audio with the video track, the process is a bit more involved than in some other programs. First you have to mix your audio down. Then you choose the Replace Audio In Video File option. The program will prompt you to select the video file and then the audio file, at which point it will mix them together. The advantage is that your video stays in the same format it was imported in.

Although their video and scoring capabilities are mostly similar, the much-more-expensive Nuendo (which is intended more for postproduction environments) has a more robust feature set. For instance, Nuendo offers Insert Into Project At Timecode Position, a command that's handy for adding effects (in Cubase you can insert events at the cursor position, which is almost as useful). When you activate Nuendo's Edit mode, you can drag an event to any location in the timeline, and the video display will follow it in real time. It makes placing events at specific locations really easy. Nuendo also gives you pull-up and pull-down commands for working with projects originally in film, and it offers a lot more external hardware support options.

Overall, Cubase and Nuendo both offer a comprehensive tool set for professional video-and-film scoring work. Either program is a solid choice, but unless you're planning to do a lot of postproduction work, Cubase should give you all the functionality you need at less than half the price of Nuendo.

It's a Wrap

Deciding which sequencer to use for your audio-for-picture projects depends a lot on both the nature of the work and your budget. If you're doing only occasional Web-related video scoring, you may be able to get by with one of the less expensive programs. However, if you have the budget to buy one of the full-featured sequencers (Pro Tools, Logic Pro, Digital Performer, Cubase, Nuendo, or Sonar), you won't regret it. The control and flexibility that those programs give you is a necessity in the high-pressure world of professional scoring.

Senior Editor Mike Levine (www.mikelevine.com) has scored many national commercials, and has written music for CNN and the History Channel. Associate Editor Dennis Miller is a composer and animator. Check out his work atwww.dennismiller.neu.edu.



Though a digital audio sequencer is clearly the best tool for scoring to picture, dedicated multitrack audio editors can be very useful if you're using precomposed music (for example, from a production library). In fact, several such programs include helpful features for working with video.

Adobe Audition (Win; $349) has a terrific feature called Link Multitrack sessions that lets you bounce a multitrack audio mix down as a stereo file for export to Premiere or After Effects, which are both part of the Production Studio bundle that Audition comes in. (Audition can also be purchased as a standalone application.) If you then need to change anything in the mix, you can open the stereo file in Premiere or After Effects and it will split back out to a multitrack version in Audition. All the programs in the bundle can access the included Adobe Bridge, which is an excellent media-assets manager.

If you want to match the duration of an audio clip to a scene in your video, you can enable Audition's time-stretching option in the audio clip's properties, and then manually drag the clip's edge to the beginning or end of the scene. You can also set markers at the boundaries of the scene, then enable snapping to markers to make aligning the audio and video even easier.

Magix Samplitude 9 Professional (Win; $1,099) supports a large number of SMPTE time displays, including standard video rates (NTSC and PAL) as well as two common film rates: 40 frames per feet (used for 16 mm) and 16 frames per feet (used for 35 mm). It can import the most common video formats (AVI, MPEG, QT, and WMV) but can export video only in AVI format. Among its features is the ability to watch your video file update in real time as you change the start time of an audio clip. This lets you see exactly where in the video the clip will be placed. Samplitude 9 ships with Magix's Movie Editor, a very capable multitrack video editor.

Although Steinberg's WaveLab 6 (Win; $699.99) may not have the reputation of its sibling Nuendo when it comes to audio for picture, it's still quite useful. You can move video clips anywhere in the dedicated video track and change the clip's start and end points. You can also split a clip repeatedly and move the parts around at will. The program lets you choose from a number of timecode formats and will even display increments as small as hundredths of a frame.

WaveLab's Time Stretch To Cursor feature is especially handy. It lets you change the duration of an audio clip to match the length of a video clip. WaveLab can also export all markers in a project as a text list, which could be useful to take to a spotting session with your film's director.
Dennis Miller


David Lawrence is on a roll. This past year he composed the original score and cowrote one of the songs for the hit movie High School Musical (Disney, 2006), and scored the movie Cheetah Girls 2 (Disney, 2006) as well as CBS TV's hit series Jericho. As a composer in the Hollywood trenches, he experiences the pressures of the big-time scoring world daily.

Image placeholder title

David Lawrence

Lawrence, a MOTU Digital Performer user, appreciates that program's wealth of scoring- and video-related features. “It will act as a cruise control for any composer who likes everything mapped out,” he says. “All you have to do is start putting in the notes. Personally, I love to look at music much more organically than that; it is a question of subdividing tempos, and taking half measures and quarter measures, and making things sort of work because they're sort of organic that way.”

Depending on the situation, Lawrence will use various methods to figure out tempos. “Sometimes I'll look at a cue and I'll go, ‘Okay, I'm feeling 68 beats per minute. And then here I'm going to change it to 72, and over here I'm going to ritard it, and so forth.’ So you build your tempo track after you've played around with it for a while.”

Other times, he takes a different approach. “There are cues that you just can't map out. They just flow, they just are. And you have to just kind of free-flow it, look at it, respond to the film, and play.” He uses DP's Tap Tempo feature and establishes a tempo map to fit a rubato piano part he played to the picture. “And now you put your entire orchestra behind it, so anything that needs to be tweaked, or mildly quantized, or just edited or fixed or nudged is now on a perfectly working tempo map,” he says.

Lawrence gives a tip for tapping in a tempo, one that will work in DP or any sequencer with a similar feature: don't tap in quarter notes, but rather use a subdivided tempo, usually eighth or 16th notes. “You can actually follow your performance, because it's much easier to subdivide than it is to do quarter notes at slow tempos,” he says.

Because deadlines are always so tight for composers, Lawrence has learned to work really fast. “For better or for worse — and sometimes it's for worse — many producers and directors and companies think that you can get everything done now in an hour. And it's impossible, it's unfair, and it's cruel, but if you don't do it, somebody else will get that job,” he says. “The best advice I can give anybody is, as you get familiar with your sequencer, get away from the mouse and learn as many shortcut commands at the keyboard as you can. It really, truly saves hours in your day.”
Mike Levine


Chris Jordao is a staff composer at Big Foote Music and Sound (www.bigfoote.com), a busy music-production company based in New York City. Jordao, along with the other composers at Big Foote, uses Apple Logic Pro for his scoring work.

Image placeholder title

Chris Jordao

Jordao explains the work flow on a typical TV-commercial score. “The client sends us the picture, and it's not necessarily a QuickTime movie; sometimes it's a DV file,” he says. “There are multiple formats that they send us. We have a guy here that cuts the movie to 2 seconds before the first frame of the picture, which would be 00:59:58:00.”

Jordao then sets the Movie Start parameter at the bottom of Logic's movie window to 00:59:58:00, ensuring that the first frame of the picture will be at 01:00:00:00. “It's pretty standard here,” he says, “so we always know that our sessions are going to have that starting point.”

Once that's set up, Jordao can experiment with the tempo and SMPTE start time of the music to see how various combinations work with the picture. In Logic, both these variables are located in the Tempo List window. “Sometimes we want to have the movie start a little later, to make it hit the picture better to the music, so we play around with that,” he says.

According to Jordao, he'll often use Logic's markers as well, locked to absolute time. “Since we work in a 30-second format all the time, I'll put markers at 0 seconds and 30 seconds, so I have a clear view of how many bars I have for a track, and if it [the tempo] is going to fit my needs.”

He'll also frequently use the markers to outline the hit points. “That's an important thing too,” he explains. “Each composer here will work slightly different. Some will rely more on the markers, some less. I use the markers a fair amount.”

But Jordao doesn't get too wrapped up in all the sequencing features. “I try to keep Logic as just an intuitive program for me,” he says. “I probably only use about 50 percent of what it can do — I just want to have the music come through.”
Mike Levine

VIDEO-SCORING FEATURES COMPARED Acid Pro 6 Cubase 4/Nuendo 3 Digital Performer 5.11 GarageBand 3 Live 6 Logic Pro 7.2 Pro Tools HD/LE 7.3 Sonar 6 Producer Edition Tracktion 3

Cut/Copy/Paste Video X X X X Display SMPTE X X X X X X Export Movie and New Audio X X X X X X X Extract Audio from Video File X X X X X X X X File Formats Imported AVI, MOV, MPEG, SWF*, WMV AVI, DV, QT, WMV** DV, MPEG, QT AVI, DV, MPEG, QT AVI, DV, MPEG, QT AVI, DV, MPEG, QT AVI, MPEG, QT AVI, MOV, MPEG, WMV QT File Formats Exported AVI, MOV, MPEG, RMV, WMV AVI, DV, QT, WMV** QT QT QT QT AVI, MOV, WMV Markers: Lock to Absolute Time X X X X X X X Auto Move Audio to Video Frame Start X X X X X X X Notation Editing X X X X (limited) Offset Music Start X X X X X X X Offset Video Start X X X X X X X X Output Video to FireWire X X X X X X Offset SMPTE Display X X X X Rerender Video in New Format X X X Surround Capable X X X X X X Visual Cues X *SWF = Flash. **Windows versions only.