Mind Reader

Mobile music players such as the Apple iPod have changed the face of our industry for good or ill, depending on who you talk to. But one thing's for sure
Image placeholder title
Image placeholder title

FIG. 1: The team at UMBC uses the BodyMedia SenseWear armband to monitor the wearer''s physiological state. This information is used to select the appropriate music for the current situation.

Mobile music players such as the Apple iPod have changed the face of our industry for good or ill, depending on who you talk to. But one thing's for sure — people love to listen to these devices during almost every activity, from exercising to relaxing and everything in between.

For users with rudimentary computer skills, the process of acquiring, encoding, and selecting a seemingly endless stream of content can be daunting. To simplify some of those issues, a team at the University of Maryland, Baltimore County (www.umbc.edu), is working on an interesting project called XPod that automates some of the interaction between the player and its user.

A key component in the system is a physiological sensor called the SenseWear from BodyMedia (www.bodymedia.com; see Fig. 1). This armband is designed to monitor a variety of parameters, such as skin temperature, heart rate, and galvanic skin response. Previous experiments have demonstrated that such data can be used to determine emotional states (such as sadness, anger, surprise, fear, frustration, and amusement) with a high degree of accuracy — typically in the 70 to 90 percent range.

The SenseWear armband also has an accelerometer, which, as you might guess from the name, measures acceleration. That information is used to determine the user's level of activity: sitting, walking, running, and so on.

The XPod system “learns” the user's preferences, activities, and emotions and selects the most appropriate music to accompany any given situation. Using a client-server configuration, the server processes the incoming data from the SenseWear, combines that with the information entered by the user about what music they prefer under which conditions, and selects the tunes that would best fit the current situation.

The experimental setup includes a Windows laptop, which wirelessly receives the data from the SenseWear, executes all processing, stores all song data, and sends the selected songs to a PDA client that plays the music and provides the user interface. The laptop and PDA communicate via Wi-Fi, which also lets the user rate the music, skip to the next song, and otherwise control playback.

When the user presses Play, the server begins examining the data from the SenseWear. Using a series of algorithms, it determines the average and standard deviation of the incoming values and compares the results with predetermined ranges that correspond to active, passive, and resting states. Once the user's state is known, that information is passed to a neural-network engine, which compares the user's current state to states for which song preferences have been specified. Finally, it makes a musical selection and sends the data to the PDA.

Users can continually update the system by indicating their satisfaction with a given selection. That preference can be applied to the song being played as well as to the artist and genre as indicated by the song's metadata. The neural network learns the user's preferences by monitoring which songs are skipped under what conditions, eventually leading the system to skip songs it believes the user would skip anyway.

Initial experiments included monitoring several test subjects of different genders, ethnicities, and athletic abilities while the subjects were lying down, sitting at a desk, walking, and running. The values obtained during those trials allowed the UMBC team to develop algorithms that accurately determine any user's activity level.

This research is in its infancy, and the XPod currently selects music based on activity rather than emotion. Future versions could expand the selection criteria and become more sensitive to the user's state, automatically supplying just the right music for any situation. In addition, advances in miniaturization, such as those often featured in “Tech Page,” will eventually allow the server to be incorporated into the player. Soon, mobile music players could truly provide the soundtrack for our lives.