A new environment for evaluating speakers.
To many, the process of recording, storing, and reproducing sound seems entirely in the sphere of science. However, objective measurements tell only half the story. Sometimes audio equipment appears to perform exceptionally well after being measured for frequency response, dynamic range, noise levels, and so on, only to disappoint those who hear it. On the other hand, a device that measures poorly might get rave reviews from listeners.
The subjectivity of listener evaluations is a big problem. Variables such as listener experience and prejudice for or against a manufacturer play a significant role in the final analysis, not to mention room acoustics and associated equipment. Speaker evaluations are especially susceptible to those variables.
Among the companies working on the problem is Harman International, whose many subsidiaries include JBL and Infinity. To evaluate prototypes and finished products and compare them to the competition, Harman built a blind-testing room called the Multichannel Listening Lab (MLL).
In that room, several speakers are placed behind black acoustically transparent grille cloth so that listeners can't see them, and various audio clips are played through each speaker. A computer controls which speaker is used to play each clip, and a video display in the corner of the room identifies the speaker that's playing by an arbitrarily assigned letter. To prevent listeners from acclimating to a speaker's position, all speakers are securely mounted on pneumatic platforms, and the computer randomly shuffles their positions relative to each other in seconds (see Fig. 1).
The acoustic properties of the MLL are carefully designed. For one thing, the lab is markedly quiet, with a noise-criteria rating of NC5. (Recording studios are typically rated at NC15.) That rating system provides a subjectively derived scale for measuring the level of background noise, which has to be minimized in a critical-listening environment. In addition, when the room was finished, the reverb time was 0.25 second, which was deemed too dry, so drywall was added to increase the reverb time to 0.35 second, which is typical of a well-furnished living room.
Before listeners can evaluate speakers in the MLL, they must pass a hearing test and a series of computer-controlled training exercises designed to teach them how to be human spectrum analyzers. Those exercises (and the entire MLL facility and procedures) are based on research conducted by Dr. Floyd Toole, an acoustical expert and Harman's vice president of engineering. For example, his research indicates that listeners are much more demanding in terms of sound quality when the clips are played in mono. If the tests are performed in stereo, poor speakers are typically ranked much closer to good speakers, which makes it more difficult to distinguish quality levels.
During a recent visit with several other audio journalists, I participated in a sample test run in the MLL, which was conducted by Sean Olive, manager of subjective evaluation for Harman International. Three consumer speakers were mounted behind the grille cloth: a Boston Acoustics CR8, a B&W DM601, and a JBL S26. At the end of the testing, we learned that most of us had ranked the JBL S26 as the best speaker on most clips, which certainly pleased our hosts.
Toole's research indicates that trained listeners with unimpaired hearing tend to rank the quality of different speakers with surprising consistency in a blind test under controlled acoustic conditions. As a result, it is possible to turn subjective evaluations into objective results, though it's quite expensive to build a room like the MLL. Nevertheless, such facilities will become increasingly important as audio systems evolve from stereo to multichannel surround.