Imagine using a sample editor to change a vocal melody from major to minor or to accent the third beat of every measure using an amplitude boost. Sound tedious? Melodyne 1.0.1, from German newcomer Celemony Software, represents audio in a musical way and makes transformations such as those rather easy to do.
Celemony's Melodyne is a Mac program that takes a fresh and original approach to the sonic materials musicians work with. (A Windows version is in late beta as of this writing.) It offers many capabilities common in signal processing, sequencing, and transcription software, including melody detection and transcription, pitch shifting, formant correction, time compression and expansion, tempo adjustment, and audio-to-MIDI file conversion. Melodyne lets you work with those features in a musical context. It analyzes a monophonic audio signal and presents the results in terms of pitches and rhythms within an overall metric and tempo structure.
Melodyne's musical intelligence makes editing audio feel like editing music. Consider pitch correction, for example. With a conventional audio editor, you locate a sour vocal note visually or with a scrub tool; then, you select the samples, calculate the required pitch correction, apply a digital signal-processing pitch-shifting operation (with formant correction, preferably), and do your best to edit out or mask any resultant glitches. Even the best pitch-shifting algorithm might distort subtle musical characteristics of the altered event, such as the singer's vibrato or the pitch transitions to the surrounding notes.
With Melodyne you select a range of notes (rather than a range of samples) at a given measure and beat location; next, you drag it to the desired pitch level. The amount of shift could be a single cent, or it could be quantized in semitones or in terms of a scale and key, if desired. Formant correction is applied automatically. Because Melodyne knows about the internal characteristics of each note, you can tweak those characteristics, widening the singer's vibrato a little, for example.
I reviewed Celemony's Melodyne on a Macintosh G4/867 MHz with OS 9.2. The audio hardware was a Korg OASYS PCI card with version 2.0 software installed. I encountered no compatibility problems.
Setting up Celemony's Melodyne is simple — just run an installer program and enter a license key. Then, use the Preferences window to specify audio parameters. By default, Melodyne uses the Macintosh Sound Manager for audio playback and recording, but you can direct Melodyne to use an ASIO driver. I hooked up Melodyne to my OASYS hardware through ASIO and used it that way throughout the review period.
SEEK AND DEFINE
The heart of Melodyne is its melody-detection process, which operates on monophonic tracks. The program is not designed to analyze polyphonic music; its melody- and rhythm-detection algorithms require that each track contain a single-note melody or a monophonic percussion track. Tracks must be as clean and dry as possible. If notes within a track are not clearly articulated or are smeared together by reverb, Melodyne might detect spurious pitches or incorrect rhythms and durations.
Each track is associated with a sound file in AIFF, WAV, or Sound Designer II (SDII) format. You can record a file directly within Melodyne, or you can import one. If you import a stereo sound file, it will be handled as mono.
To edit a track, you must first let Melodyne analyze the signal and produce a melody-definition file. The melody definition is Melodyne's representation of the audio signal's pitch and rhythmic content. A separate melody-definition file is created for each analyzed sound file. Think of the melody definition as a template from which you can produce as many different versions of a track as you want. (Melodyne's authors make it clear that the initial melody definition might contain imperfections and will typically need some manual correction.)
Once you are satisfied with your melody definition, you can start editing the track. Before looking at Melodyne's editing capabilities, though, I'll investigate the melody-detection process.
To evaluate Melodyne's performance with reasonably clean input, I prepared a brief test example (concocted to challenge the software, not to please the ear) as a Mark of the Unicorn Digital Performer MIDI track. Fig. 1a shows the notation of the melody using Digital Performer's QuickScribe. I then recorded a note-perfect performance in the SDII format. The tune was rendered on a Kurzweil K2500 using a piano preset without reverb. Each note was sustained for 80 percent of its written duration to articulate each event clearly.
When the SDII file was imported into a Melodyne track, Melodyne performed an initial analysis of the sound file's temporal structure (but not of its pitches). Melodyne detected the correct tempo, time signature, and number of measures (see Fig. 1b). At that point, because no pitch analysis had been done, the audio signal was displayed in an envelope representation, much like that of an ordinary sample editor.
So far, so good — parsing a stream of audio samples and extracting rhythmic information is not a trivial software challenge. The next step was to invoke the Detect Melody command. When you initiate melody detection, you can optimize the process by giving Melodyne information about the expected pitch range and other properties of the source material. To see how well the program would do without any hints, I just used the defaults.
After that analysis, I brought up the Melody Definition window to view the analyzed melody (see Fig. 1c). That window displays the notated melody as well as an envelope display arranged on a pitch/time grid. As the window shows, Melodyne got all the pitches right — again, pretty impressive.
But Celemony's Melodyne didn't detect all of the rhythms. The repeated B naturals in measure 1 were not detected correctly, apparently because their articulation was too legato to suit Melodyne. To correct that, I selected the Note Separator tool from the toolbar at the top left of the window and clicked on the attack points of the two 16th-note B naturals. In Fig. 1d, the display is zoomed in to show the first measure as corrected.
In measure 3, however, I encountered a limitation: Melodyne doesn't detect or notate tuplets, so the triplets and quintuplets are not notated correctly. There is no way to fix this, but Celemony states that its programmers are working to overcome the restriction.
This may seem a somewhat mixed performance, particularly in terms of transcription and notation. But remember that Melodyne is primarily a tool for editing and correcting audio within a musical context. The first task of such a tool is to locate audio events in musical space and time. (You can't edit something you can't locate.) From that point of view, Melodyne did a remarkably good job.
Although Melodyne isn't primarily a notation tool, it does have potential in this area. I hope the authors will improve the notation features, at least with respect to handling tuplets. Correctly spaced, beamed, and tied notes would be nice, too.
Once you have a melody definition associated with a track, you can do detailed, nondestructive editing in the simple-to-use Editor window. First, you select a tool from the toolbar at the window's top left (see Fig. 2). Then, you select the notes you want to edit. The Editor provides some powerful selection features, such as selecting by pitch or by beat.
You can edit notes graphically with the mouse, type parameter values into edit fields, or use up and down arrows to change values incrementally. You can edit on the fly, hearing your parameter changes in real time during playback.
Rather oddly, Melodyne supports copy, cut, and paste edits, but not undo or redo. (According to the manufacturer, a multiple-level undo feature is included in the 1.1 version, which should be available soon.) However, a Reset button associated with each tool lets you undo the most recent change made by that tool.
When you select a tool, Melodyne changes the display so that you can manipulate the appropriate property. For example, when the Pitch Transition tool is selected, handles are superimposed on each event. By selecting and dragging a handle, you can vary the trajectory of pitch change between one note and the next. In Fig. 2, I have used that tool to produce an exaggerated upward pitch bend on the dotted quarter-note B in measure 2.
The Editor Window provides a powerful set of editing tools. The Pitch tools allow you to change the pitch of notes with a one-cent resolution, widen or flatten pitch fluctuations (such as vibrato) within notes, and adjust the pitch transitions between notes. Pitch-shifting automatically adjusts formants to preserve the timbral character of pitch-shifted notes.
The Formant tools let you shift the position (but not the bandwidth) of formants, which permits some interesting filtering effects. You can also adjust the formant transitions between notes.
The time-related tools allow you to reposition notes in time or change the duration of notes. When you perform either edit, the neighboring notes are also adjusted so that the metric structure of the melody is preserved. For example, suppose you have two half notes in a 4/4 measure. If you shorten the first note to a quarter note, the second note lengthens to a dotted half note. That is cool!
The Edit Time Handle tool is also time related. It lets you tweak the initial speed of a note, adjusting the attack time and the rate of internal pitch fluctuations.
The Amplitude tools allow you to increase or decrease the amplitude of notes or mute notes. You can also adjust the amplitude transitions between notes.
The note-separation tools let you divide a note into two parts, join two notes into a single event, or cut parts of the melody into free-standing segments.
The Arrangements window is equivalent to the project windows of conventional multitrack audio programs. The window displays one or more tracks, represented by track selectors with Solo, Mute, and Record buttons (left pane).
At the top of the Arrangements window's main scrolling pane is a timeline divided into measures and beats. Below the timeline are tracks displayed in an envelope format much like the Editor's. You can overlay the envelope displays with musical notation. Unpitched (or unanalyzed) tracks are displayed in percussion notation.
The Arrangements window also allows you to do a variety of edits on time regions within tracks, groups of tracks, or an entire arrangement. Edits include changing the tempo or time signature and shifting selected material around in time. One of the more interesting features of the Arrangements window is time adaptation of one track to another. To accompany my test melody with percussion, I imported a completely unrelated rhythm track into the arrangement. Then, using the Adapt Time command, I adjusted the tempo of the percussion track so that it synchronized perfectly to the melody (see Fig. 3).
The Temporary Play Offsets window lets you modify the pitch, tempo, or formants of the entire arrangement in real time. The changes are instantaneous and, for the most part, free of artifacts. If you like what you hear, you can apply those modifications to the arrangement permanently.
OTHER AUDIO FUNCTIONS
Melodyne isn't designed to compete with sophisticated multitracking software such as Pro Tools. Melodyne's Transport Bar and Mixer windows, however, do support basic recording, playback (normal, looped, fast-forward, and rewind) and mixing functions. The Mixer offers a conventional arrangement of controls such as input and output selectors, faders, pan pots, effects sends and returns, and level meters. The Mixer's rotary controls are sluggish and unresponsive. A crude reverb is the only effect available, which is unfortunate, as Melodyne does not support any effects plug-in formats. Mix automation is also not supported.
You can export all or part of an arrangement to an audio or a MIDI file and export audio as a stereo mix or as individual mono files per track. Likewise, a MIDI file can be written as a single multichannel file or as individual files for each channel. When exporting MIDI files, I noticed an apparent bug that caused Melodyne to omit the first note of one or more tracks, though all other notes translated correctly.
Professional users will probably wish Melodyne interfaced (other than exporting individual tracks) better with other audio tools. It would be nice, for example, if Melodyne could import and export tracks to and from other project formats, such as Pro Tools sessions or Open Media Format files.
Melodyne comes with a brief manual in German and English. The English documentation is well organized but poorly translated and, therefore, sometimes unclear. Celemony maintains an informative Web site where you can find additional information and a demo of the program.
Melodyne is a powerful and innovative (if pricey) tool for editing traditional music that can be represented in a more or less conventional pitch and rhythmic format. It won't appeal much to experimental composers, but Melodyne will probably find a niche in professional studios that need an efficient performance-correction tool. Although Melodyne is strong in its specialized domain, it is definitely not an all-purpose multitrack recording application. Most studios will use it as an adjunct to more mundane programs such as Pro Tools or Digital Performer.
Did I mention that Melodyne is fun to work with? Visit Celemony's Web site, download the Melodyne demo, and start having some fun with it yourself.
John Duesenberry's latest equipment acquisition is a Triumph Bonneville. It's very analog. Bikers and other readers are welcome to firstname.lastname@example.org.
Minimum System Requirements
G3/300; 128 MB RAM; OS 9.0.4
Celemony Software GmbH
Melodyne 1.0.1 (Mac)
FEATURES3.0EASE OF USE3.5AUDIO QUALITY4.0VALUE3.0RATING PRODUCTS FROM 1 TO 5
PROS: Unique musical approach to audio editing. Powerful set of tools for pitch, time, amplitude, formant, and other edits.
CONS: Transcription/notation does not handle tuplets, beams, or ties. No multiple undo/redo. Poorly translated English documentation.