SOUND IDEAS
AUDYSSEY INNOVATIONS BLOG

Avatar 3D is showing to rave reviews in 2100 theaters, ESPN announced 85 sporting events broadcast in 3D starting with the FIFA World Cup in June, and all I heard about at CES was 3D televisions shipping this year.  It is a big deal.  I think… Or, it could be the next consumer electronics industry failed format.  There is no way to really know what 3D picture will become but there is huge momentum behind it and I think we are going to be hearing about it, watching it and talking about it for the next several years.

So, while we sit with our 3D glasses on being immersed in 3D video (and hopefully not getting headaches), I want to talk about 3D audio — because what’s the point of 3D video without an immersive audio experience?

Sound has an advantage over picture when it comes to rendering: the cues we receive from well designed and calibrated systems are closer to the real thing than what 3D displays can deliver.  Here’s what I mean by that:  I can stand behind a black acoustically transparent curtain next to a loudspeaker and with the right calibration make it so you can’t tell if it’s me or the speaker talking.  No display technology exists today that can fool you into thinking you are looking out of a real window.

One problem is display resolution and that is well on its way to being solved.  The other is depth perception.  No matter how good the 3D technology, the visual cues arriving at your eyes are still not the natural cues that we use in real world 3D viewing.  In the real world, our eyes focus at different distances through a process called accommodation.  But in 3D picture we always focus at the same distance: where the screen is.  It’s up to the 3D technology to make us believe that we are using accommodation, and that often leads to fatigue especially when we are asked to quickly focus on things popping out of the screen.

3D Audio is Not Possible from Two Speakers
Despite the claims, so-called 3D audio today suffers from a fundamental problem: the perceptual cues are virtual and demand that our hearing “accommodate” for them.  The math looks great on paper.  Start with two speakers that send signals to both ears.  Then pre-process the signal so that the crosstalk terms (left speaker to right ear and right speaker to left ear) are cancelled.  In theory, you then have turned the speakers into headphones that can talk to each ear individually and can reproduce virtual surround sound.  In theory…

In practice, the simplistic methods that abound in TV and PC sound systems don’t even come close to reproducing the correct cues.  Instead, they feed a little of the left channel signal to the right channel and manipulate its phase.  Then they do that to the right channel signal as well.  The results are phase artifacts that produce fatiguing sound.  What’s worse is that it changes every time you turn or move your head by as little as an inch.  The math clearly shows that  even in the ideal case, the right conditions can only be met in one infinitesimally small single point in the room.  So, forget about sharing the experience with anyone sitting next to you.

In order to do this correctly from two speakers or headphones you must have a very accurate model of how sound interacts with the listener’s outer ear and upper body.  This information is called a head-related transfer function (HRTF).  But, the HRTF characteristics are unique to each person so all kinds of artifacts start to show up when you listen through some “averaged” filter as all of these methods use today.  Two-speaker 3D audio systems today often try to disguise their inability to render sound correctly by artificially stretching the soundstage to the point where it no longer resembles the original.  The results are so fatiguing that most people turn off these effects and listen to what is natural from two speakers: stereo.

3D Surround Sound
True 3D audio must be capable of reproducing a seamless, enveloping surround sensation and also provide accurate localization of sound sources to match what is happening in the picture.  It must also be able to do this for everyone in the audience.

To do it right requires rendering acoustical and perceptual cues that human hearing relies on in the real world.  They must have the proper frequency response and timing to seamlessly combine with the direct sound from the front.  But more importantly, they have to come from the right directions found in research to influence our ability to perceive spatial sound.  True 3D sound requires more than two loudspeakers to get all this information to our ears and to also do it for our friends and family sitting next to us.

Surround sound today with 5.1 or more channels delivers a superior experience to 3D video in your home because it has evolved over the last 20 years.  And, now there are systems available from Audyssey and Dolby that go beyond 5.1 to add Wide and Height channels that properly enhance the perception of envelopment and immersion.

However, 3D surround sound isn’t fully evolved yet either.  With the right system and proper processing we can create a seamless, enveloping soundstage.  And we can accurately place individual sound sources on that stage.  But, how do we make a character whisper in our ear?  To do that we have to bring sound inside the bubble and that is still an unsolved challenge.  Headphones can do that quite well with the right recordings.  But, they can’t do the opposite: render sounds outside our head.  Solving this problem for loudspeaker-based surround systems will be the next leap in 3D sound.

A well-known home theater installer that I talked to at CES had this observation: “Until 3D video content gets out of the ‘in your face, popping out of the center of screen’ mode it will not be natural”. Avatar is a great success because it was created with the 3D theater experience in mind.  The test will come with follow-on content where 3D is utilized as a way to enhance the story or event instead of trying to dazzle us with effects. That will take some time.  For now, while the evolution of 3D video gains momentum, we can enjoy surround sound that is actually ahead of picture in realism.

Frank Filipetti is one of the most respected names in the music business. As a producer, mixer and engineer he has been at the forefront of digital music and production for decades – working with artists such as Elton John, KISS, Billy Joel and Carly Simon. He won a Grammy for Best Engineered Album for James Taylor’s landmark “Hourglass,” which also won that year’s Best Pop Album. Filipetti continues to work in music and film production and remains a major proponent of the surround sound experience for engineers and consumers alike.

PART I

How did you discover Audyssey?
Totally by accident. A couple of years ago I bought a Denon 5308 receiver, which included a consumer version of Audyssey MultEQ. I’d always been skeptical of automatic digital equalization so I held off on setting it up. But when I did try it I was amazed.

Did that prompt you to check out the Audyssey Sound Equalizer Pro?
Yes. I said to myself, ‘This is very interesting, I wonder if they have a version for professionals?’ So I called Audyssey and eventually got on the line with Chris who told me about the Sound Equalizer Pro. One thing led to another and he ended up coming up to New York and setting up the system for me.

Where did you install the Sound Equalizer Pro?
Chris and I set it up at my control room at Right Track Recording – a room I’ve had for 25 years; it’s been my main mixing and recording room for as far back as I can remember. I had always been happy and comfortable with the room, but it had a slight bump at 60 Hz, dipped at 90 Hz, and bumped again at 120 Hz. I had also been happy with the JBL 6300 monitors since those allow you to tune the low end. So, I had a room and a system that, except for a few small problems, were still excellent for my needs.

We set up the Sound Equalizer Pro and, lo and behold, as good as I thought my room sounded, this suddenly made everything sound so much tighter, cleaner and more focused.

How did the Equalizer improve the sound?
First I noticed the smoothness at the bottom end. Then I heard how spectacular the stereo imaging became. The phantom center was so much more precise. By equalizing both speakers the center imaging was spot on and so, too, the detail and stereo width: the speakers seemed wider and more focused at the same time. You could pinpoint things in the mix. It was quite a dramatic improvement in a room that was already professionally tuned and calibrated. Everyone noticed. Several days later I worked my first session using the Equalizer. I was recording David Sanborn and everyone, David included, was knocked out by the increased fidelity and clarity they heard.

Has the Equalizer changed the way you listen in the studio?
Yes. I typically don’t use big monitors in the studio. But I was so blown away by how good my small monitors sounded, I decided to go to the bigger ones, especially because there’s an option to use eight speakers in the setup. I was able to run both large and near field monitors in stereo through the Audyssey box at the same time. It was a room tuned for hip-hop clientele, and I didn’t like using the big speakers very much, but suddenly the system sounded musical and amazing even on those big monitors.

I setup a 5.1 system with the JBL monitors. The imaging in 5.1 was just frightening. Phantom images were incredibly precise because the speakers are calibrated to sound the same. The comb filtering you usually experience with multiple speakers was simply not an issue; every sound had a really stable place. Overall, I was just incredibly impressed.

What music did you use to test the system?
I test all my rooms and equipment with three tracks: First, the opening track on James Taylor’s “Hourglass” called “Line ‘Em Up.” It has an even frequency response, the bass is really tight and present, and I’m so familiar with James’ voice, I know when it’s right and when it isn’t. The track gives me a real perspective on how everything from sub to low mids to upper mids to air is doing. It’s my main reference track.

Once I’ve done my main tuning with that track, I go to a track off a Korn record I mixed in 2003 called “Another Brick in the Wall.” It’s very powerful with a tremendous amount of energy below 200 Hz.

Then I play the overture and intro to “Wicked.” It’s very orchestral and operatic. I generally listen to those three types of music to give me a broad perspective on how things sound: hi-fi, heavy metal and orchestral.

How did these tracks sound when the Equalizer was on?
Everything just sounded better. If something was punchy it got more punchy; powerful more powerful; high fidelity even higher fidelity.

For instance, listening to James with and without Audyssey was an ‘aha’ moment –an epiphany of sorts. In a room I was very used to and very happy with, I suddenly found an amazing, new degree of detail after only a fifteen-minute setup. It was frightening and exciting: I felt for the first time really comfortable thinking that maybe I could mix outside of the studio with a product like the Audyssey Sound Equalizer Pro.

Did you ever try that?
In fact two months later I was asked to mix an album at Martha’s Vineyard. Carly Simon wanted to mix her new album at her home because she didn’t want to travel to New York City in the summer. So I said, knowing that I had the Audyssey Equalizer with me, ‘I’ve never mixed outside of the studio before, but let’s see what happens.’

I cleared out her guest apartment, put out my system and, eureka, because of the Equalizer I could achieve the sound I needed without any additional floor or wall treatment. The sound was so good we were able to mix nearly the entire album at her home.

After my experience with Carly, I set up a system in my home with the Audyssey Equalizer. Now, I’m doing most of my mixing from home. I’ve recently mixed Billy Joel, Survivor, Dolly Parton and I’m working on a new project for Sony, which I’m completing here. It’s given me incredible freedom and people have been so happy with the results.

I knew things were right when my mastering engineers, Bob Ludwig and Ted Jensen, couldn’t tell the difference between the tracks I mixed at home from those mixed in the studio. That’s when I knew there was a new paradigm in place: we can now effectively operate from a home environment.

How have others colleagues reacted?
They’re a bit skeptical about products that seem to do equalization automatically. But I’ve become evangelical about Audyssey: I tell everybody in the business about this amazing box. The simple fact is it has literally changed the way I operate.

---

Tune in next month when Frank talks about the format wars, surround sound’s uncertain future, the democratization of the recording industry and the new music that moves him.

Do you have a subwoofer in your system? Great. Then your speakers are small. Before you get all upset, read on. This is one of those audio myths whose time has come to be busted. To understand why, we need to talk about Bass Management.

In the early days of home theater it was thought that in order to reproduce the full movie surround experience at home it was necessary to place 5 large loudspeakers in the room. The reason for the size was the woofers. To play at theatrical reference levels and reproduce the deepest bass available in the content requires each speaker to have 12” or larger woofers. Let’s just say that this theory didn’t get very far in the real world.

A better and more practical approach came after studying human perception. The mechanisms that we use to determine the direction of arrival of sound depend on the frequency. At high frequencies the wavelength of sound is small and so sound coming from the side is shadowed by our head. That creates a level difference between the sound reaching the ear closest to the source and the ear on the other side. Our brain analyzes these level differences and produces an estimate of where the sound is coming from. But at lower frequencies, the wavelength of sound gets longer and our head is not large enough to produce a level difference at the two ears. Instead, we analyze the difference in time of arrival of sound at the two ears. Sound arrives first at the closest ear and we use that to determine the direction. But even that ability fails us below about 80 Hz. The wavelengths get very large and it was found in listening tests that 80 Hz is the frequency below which most people can not localize the direction of sound.

Taking advantage of this apparent “deficiency” in our hearing was what made home theater practical for millions of homes. Five satellite speakers of reasonable size could now be used because they no longer required large woofers. A subwoofer (or two) can reproduce the lower octaves and it can be placed out of sight since its content is not directional.

But there is also a practical advantage: directing the bass to a dedicated subwoofer channel with its own amplifier greatly improves the headroom in the main channels. The idea behind this was proposed in a Society of Motion Picture Engineers (SMPTE) meeting in 1987. The participants could not agree on the minimum number of channels required for surround sound on film. Various numbers were being shouted out until a voice was heard from the back: “We need 5.1”. Everyone’s head turned around to look at Tom Holman. He proceeded to explain what he meant: Take the low frequency content from all 5 channels and redirect it away from the satellite speakers to the subwoofer. If we do the math, then the content below 80 Hz is 0.004 of the audible 20,000 Hz bandwidth. But 5.004 didn’t sound as catchy so Tom rounded up to 5.1. By the way, don’t make the amateur mistake of calling it 5 dot 1. It is a decimal: 5 point 1.

Fast forward to the early 90s when the first DSP powered home theater receivers started to appear. Along with progress came complexity. Some industry forces believed that Bass Management should be an option that could be turned on and off by the consumer. That’s not necessarily a bad idea, but to make an informed decision requires much more knowledge about the system than what was available to the typical consumer. So, the Large and Small rule of thumb was established. The idea was to look at the size of your speakers and decide whether their woofers were “large enough” to reproduce the lowest octaves at the required levels. It was a noble thought, but looking at it 15 years later I believe that it has led to nothing but massive confusion. The poor consumer was led to believe that Large is somehow a good thing and was then left wondering why there was nothing coming out of their subwoofer.

Redirecting the bass to the subwoofer relieves the receiver amplifiers from having to work on reproducing the low frequencies and this greatly improves the headroom. If you happen to be using Audyssey MultEQ for room correction, you will achieve much better low frequency performance because the MultEQ subwoofer filters have 8x higher resolution than the filters in the other channels.

Here is a better rule: All speakers are Small. In today’s complicated AVR lingo that just means: If you have a subwoofer you should always turn bass management on. Always. Even if your receiver clings to the past and automatically sets your speakers to Large.

Here is a trick question: “What if I correct the acoustical problems in my room, but I don’t like the resulting sound?” If you find yourself asking this question you have stumbled on the line between Reference and Preference.

Let’s look at what room correction aims to do. You start with a good (or not so good) set of speakers, you place them in a room and what you have is problems: acoustical problems. Sound from the speakers comes to you from many different directions. Some of it directly, but most of it after interacting with the floor, ceiling, walls, and furniture. Because each of these elements is at different distances from where you sit, the combined sounds arrive at slightly different times and what you hear is a form of distortion. Voices can sound unnatural, the low frequencies are muddy or boomy, and the high frequencies lack air and sparkle (yes, these are all technical terms).

A well-designed room correction system captures information throughout the listening area and analyzes it in the time domain. It then creates an equalization solution for each speaker and applies it so that the response matches a certain target sound. And here is where we first catch a glimpse of the Reference vs Preference line: What should this target sound be?

The answer lies earlier in the chain, where the content is made. The film industry adheres to a set of strict standards that are used in the creation of the content and in the reproduction of the content in movie theaters. These standards define the location, level, and frequency response (target sound) of the speakers in the audio system. They are in place so that content created in one location can translate perfectly to thousands of movie theaters. The same translation should apply when the content is played back in a home theater.

So, calibrating your home theater system “to reference” means that: (i) the levels of each speaker and subwoofer are matched to each other; (ii) the playback level of the system reaches a certain sound pressure level when the volume control on your AVR is set to “0”; (iii) the time delays for each speaker and subwoofer are adjusted so that sound from all of them arrives at the same time to the central point of the listening area; and (iv) the frequency response of each speaker and subwoofer is such that the perceived octave-to-octave balance is the same at home as it is in the dubbing stage or the movie theater.

Reference is a good thing because it gives us a way to reproduce the art the way it was made. But, you might say: “I like more (or less) bass.” OK, I’ve never heard anyone ask for “less bass,” but I suppose it’s possible. As soon as the word “like” is uttered we have crossed into the uncharted waters of the Preference abyss. It can be a fun place to be and there is nothing wrong with applying personal touches to your sound, especially if you are the one enjoying it. But know this: It’s not the job of a room correction system to determine your preference. That is entirely up to you. What room correction gives you when it delivers reference is a known baseline and the ability to apply preference consistently. Without it, boosting the bass for some content would not sound the way you want it for other content because you are not starting from a known condition. If you want to apply your preference, I have some words of advice: “Start with reference.”

My first “beyond 5.1″ experience was in January 2000 at the Alexis Park Hotel in Las Vegas. My colleague from the USC Immersive Audio Lab, Tomlinson Holman, was in the small room next door explaining why more channels are needed. This was followed by a demonstration of the world’s first 10.2 system to a few members of the press.

The world of sound had come a long way from the introduction of the first stereo LP in 1957. But in fact, with 10.2 we were moving back in time. It was in the late 30s that researchers at Bell Labs had looked into the question of how many loudspeakers are needed to reproduce a seamless, realistic sound scene. The answer at the time was discouraging: it turned out to be an infinite number–not exactly a high SAF (spouse approval factor) idea. Two channel stereo, however, was the only way to deliver sound to the consumer and it was limited by the LP groove that had two sides. Certainly better than mono, it didn’t come close to delivering an immersive experience.

The audio industry made great progress since 1957 and in the 1990’s we saw the introduction of 5.1 channel surround sound. Combined with the emergence of the DVD as the delivery medium, 5.1 surround became widely accepted in consumer homes because it created better sound immersion to match our growing video displays.

But since the 90s, the industry misdirected its focus to more esoteric endeavors: high sampling rates and higher bit depths for digital audio. While technically superior than the early digital formats found on CD in the early 80s, these developments did little to provide the consumer with a better experience. Human hearing limits for frequency range and dynamic range had already been met with 48 kHz sampling rates and 24 bit word lengths. Going to higher sampling rates simply filled up the discs with more data and required higher processing power to decode, but provided no audible improvement to the listener.

There have been attempts to push surround sound to more channels, but they were unfortunately not guided by proper acoustical, perceptual, or even logical approaches. 7.1 systems emerged by adding two more surrounds speakers in the location that makes the absolute smallest difference to the listener: directly behind them. Research in psychoacoustics tells us that human listeners are best at localization in the front hemisphere, cued by what we can see. Having more channels behind the listener than in front wastes resources and doesn’t go very far in providing a more immersive and enveloping soundstage.

To make an audible improvement in surround sound requires more Channels in the right locations. That’s what Tom was demonstrating at the Alexis Park during the 2000 CES. But not just any channels. They have to be placed in locations carefully crafted to produce the required cues that give us the impression of a wide, seamless, and enveloping soundstage.

Nine years later we are ready to go beyond 5.1 channels. Our company, Audyssey Labs just announced our latest technology, Audyssey DSX (Dynamic Surround Expansion). Drawing on our research in room acoustics and psychoacoustics we hope to extend an important dimension of sound…space.