FIG. 1: These plots of the strum (a) and fret (b) sensor signals over five repeat performances (represented by -different colors) indicate that the sensors can be used to play a song reliably while allowing some wiggle room in the precise placement of the arms.
Who among us hasn't played air guitar? Even accomplished guitarists have flailed their arms at some point in their lives, perhaps before they learned to play the real thing, striving to feel cool as they listened to Hendrix, Clapton, or Slash.
Now, thanks to research being conducted at Australia's Commonwealth Scientific and Industrial Research Organisation (www.csiro.au), air guitarists could soon do more than just feel cool — they could actually make music without strapping on an instrument. Dr. Richard Helmer, a research engineer at CSIRO, is developing what he calls a Wearable Instrument Shirt (WIS) with motion sensors embedded in the elbows of a long-sleeved T-shirt.
The textile-based sensors, which were designed at CSIRO to change their electrical resistance in response to being stretched or otherwise deformed, are connected to a small wireless unit with conductive threads sewn into the shirt. The unit monitors the sensors' resistance and converts that information into a digital signal that is transmitted wirelessly, via radio-frequency signals, to a receiver that is connected to a computer's USB port.
The computer runs custom software to analyze the data and reconstruct the elbow positions that generated them. This information is then used to trigger samples stored in the computer. The position of the left elbow selects the sample to play, while the right elbow activates the sound. The system can even distinguish between up and down strumming in the right elbow.
You might think that two bend sensors at the elbows offer little in the way of fine control, and you'd be right. But the software can compensate for that limitation by mapping the sensor data to larger song structures based on the idea that pop songs are usually constructed from relatively few sections and chords. For example, most pop songs consist of a verse, chorus, and bridge and four or five chords. These elements can be triggered in Simple mode (in which a few positions correspond to loop-based sections of a song) or in Fine mode (with more positions assigned to separate chords). In future incarnations, I can imagine smaller bend sensors in each finger of a pair of gloves that would allow individual notes to be played.
Helmer designed the system so that different arm positions can reliably trigger the associated song elements without requiring absolute precision. In other words, each gesture can fall within a range of positions and still trigger the same element. For instance, in one trial, he repeated a performance five times and plotted the sensor data (see Fig. 1), which clearly demonstrated that the gestures can be used to reliably play a song, even though they were in slightly different positions each time.
Guitar isn't the only instrument that can be played with the WIS; Helmer has also demonstrated a tambourine, drums, and more. Another possibility is sending MIDI messages in response to different gestures. In those cases, the left and right elbow gestures are interpreted differently by the software than they are for the guitar. In addition, the instrument and song definitions are independent, allowing them to be mixed in different combinations. You can even create new compositions using samples from an existing song.
One of the beauties of the WIS is that it can provide a satisfying musical experience for players and nonplayers alike. It takes virtually no musical training to play a song in the system's Simple mode, while trained musicians can take advantage of Fine mode to realize more-sophisticated musical structures. As a result, the WIS could open a whole new avenue of expression for electronic musicians as well as for their friends and families.