3D is happening
Avatar 3D is showing to rave reviews in 2100 theaters, ESPN announced 85 sporting events broadcast in 3D starting with the FIFA World Cup in June, and all I heard about at CES was 3D televisions shipping this year. It is a big deal. I think… Or, it could be the next consumer electronics industry failed format. There is no way to really know what 3D picture will become but there is huge momentum behind it and I think we are going to be hearing about it, watching it and talking about it for the next several years.
So, while we sit with our 3D glasses on being immersed in 3D video (and hopefully not getting headaches), I want to talk about 3D audio — because what’s the point of 3D video without an immersive audio experience?
Sound has an advantage over picture when it comes to rendering: the cues we receive from well designed and calibrated systems are closer to the real thing than what 3D displays can deliver. Here’s what I mean by that: I can stand behind a black acoustically transparent curtain next to a loudspeaker and with the right calibration make it so you can’t tell if it’s me or the speaker talking. No display technology exists today that can fool you into thinking you are looking out of a real window.
One problem is display resolution and that is well on its way to being solved. The other is depth perception. No matter how good the 3D technology, the visual cues arriving at your eyes are still not the natural cues that we use in real world 3D viewing. In the real world, our eyes focus at different distances through a process called accommodation. But in 3D picture we always focus at the same distance: where the screen is. It’s up to the 3D technology to make us believe that we are using accommodation, and that often leads to fatigue especially when we are asked to quickly focus on things popping out of the screen.
3D Audio is Not Possible from Two Speakers
Despite the claims, so-called 3D audio today suffers from a fundamental problem: the perceptual cues are virtual and demand that our hearing “accommodate” for them. The math looks great on paper. Start with two speakers that send signals to both ears. Then pre-process the signal so that the crosstalk terms (left speaker to right ear and right speaker to left ear) are cancelled. In theory, you then have turned the speakers into headphones that can talk to each ear individually and can reproduce virtual surround sound. In theory…
In practice, the simplistic methods that abound in TV and PC sound systems don’t even come close to reproducing the correct cues. Instead, they feed a little of the left channel signal to the right channel and manipulate its phase. Then they do that to the right channel signal as well. The results are phase artifacts that produce fatiguing sound. What’s worse is that it changes every time you turn or move your head by as little as an inch. The math clearly shows that even in the ideal case, the right conditions can only be met in one infinitesimally small single point in the room. So, forget about sharing the experience with anyone sitting next to you.
In order to do this correctly from two speakers or headphones you must have a very accurate model of how sound interacts with the listener’s outer ear and upper body. This information is called a head-related transfer function (HRTF). But, the HRTF characteristics are unique to each person so all kinds of artifacts start to show up when you listen through some “averaged” filter as all of these methods use today. Two-speaker 3D audio systems today often try to disguise their inability to render sound correctly by artificially stretching the soundstage to the point where it no longer resembles the original. The results are so fatiguing that most people turn off these effects and listen to what is natural from two speakers: stereo.
3D Surround Sound
True 3D audio must be capable of reproducing a seamless, enveloping surround sensation and also provide accurate localization of sound sources to match what is happening in the picture. It must also be able to do this for everyone in the audience.
To do it right requires rendering acoustical and perceptual cues that human hearing relies on in the real world. They must have the proper frequency response and timing to seamlessly combine with the direct sound from the front. But more importantly, they have to come from the right directions found in research to influence our ability to perceive spatial sound. True 3D sound requires more than two loudspeakers to get all this information to our ears and to also do it for our friends and family sitting next to us.
Surround sound today with 5.1 or more channels delivers a superior experience to 3D video in your home because it has evolved over the last 20 years. And, now there are systems available from Audyssey and Dolby that go beyond 5.1 to add Wide and Height channels that properly enhance the perception of envelopment and immersion.
However, 3D surround sound isn’t fully evolved yet either. With the right system and proper processing we can create a seamless, enveloping soundstage. And we can accurately place individual sound sources on that stage. But, how do we make a character whisper in our ear? To do that we have to bring sound inside the bubble and that is still an unsolved challenge. Headphones can do that quite well with the right recordings. But, they can’t do the opposite: render sounds outside our head. Solving this problem for loudspeaker-based surround systems will be the next leap in 3D sound.
A well-known home theater installer that I talked to at CES had this observation: “Until 3D video content gets out of the ‘in your face, popping out of the center of screen’ mode it will not be natural”. Avatar is a great success because it was created with the 3D theater experience in mind. The test will come with follow-on content where 3D is utilized as a way to enhance the story or event instead of trying to dazzle us with effects. That will take some time. For now, while the evolution of 3D video gains momentum, we can enjoy surround sound that is actually ahead of picture in realism.
Tags: 3D
