Surround, Binaural, or Spatial Sound in Unity3D.

Over the last week I've done a lot of research into surround sound and how it works. Some things surprised me. Here's a summary of my findings.

Headphones are the best output device for creating surround sound. This is because the amount of sound reaching each ear can be precisely controlled, without also sending sound to the other ear (this happens with any speaker setup).

Our brains determine where sounds are coming from based on subtle changes created by the shape of our ears, head density (LOL) among other things. We can emulate this using 'head-related-transfer-functions'. Wikipedia has a good summary of this. If you have audio that has been processed using an array of HRTF filters, the best way to supply this sound to your ears is with plain old stereo headphones!

The bad news is that HRTF is very CPU intensive to compute. This is the reason we have dedicated surround sound hardware. Not all surround sound hardware implements HRTF, this headset from the same company simulates the effect, it is not true HRTF.

If it can be simulated using hardware, it can also be simulated using software, which is exactly what I've done. You can now provide virtual surround sound on any pair of stereo headphones, no extra hardware required. 

Unity currently simulates 3D sound by using volume to simulate distance, panning to simulate left and right, and the doppler effect to simulate relative velocity. However it does not provide any audio cues for front and back, or when sounds are occluded. The Spatial Audio package improves this by adding left/right phase adjustment, and frequency filtering to simulate sound from the rear.

The package also simulates sound damping as a result of being occluded. These subtle adjustments in sound give a significant improvement in spatial awareness to the player.

I've uploaded a web player demo so you can hear the difference for yourself.

Spatial Audio in Unity3D

I recently acquired an Oculus Rift.

It was easy to integrate into one of my existing projects, however during testing, I discovered Unity does not support 'surround sound' using stereo headphones.

After much research, trial and error I have found that getting 'true' (HRTF) surround sound really requires dedicated hardware. It's too expensive to run realtime on your CPU at the same time as your game.

I read that FMOD has a 'fake HRTF' implementation, however this is not exposed by Unity. Well, at least it can be faked right? After all, reality is not important, perception of reality is!

So I faked it. This is my SpatialAudioSource component.

www.differentmethods.com/packages/SpatialAudioSourceDemo.html

Just drag and drop the component onto any existing audio source. It also features audio occlusion. So if something gets between the Player and the AudioSource, the audio is altered accordingly.

When it's ready, I'll put it on the Asset Store.