Surround, Binaural, or Spatial Sound in Unity3D.

Over the last week I've done a lot of research into surround sound and how it works. Some things surprised me. Here's a summary of my findings.

Headphones are the best output device for creating surround sound. This is because the amount of sound reaching each ear can be precisely controlled, without also sending sound to the other ear (this happens with any speaker setup).

Our brains determine where sounds are coming from based on subtle changes created by the shape of our ears, head density (LOL) among other things. We can emulate this using 'head-related-transfer-functions'. Wikipedia has a good summary of this. If you have audio that has been processed using an array of HRTF filters, the best way to supply this sound to your ears is with plain old stereo headphones!

The bad news is that HRTF is very CPU intensive to compute. This is the reason we have dedicated surround sound hardware. Not all surround sound hardware implements HRTF, this headset from the same company simulates the effect, it is not true HRTF.

If it can be simulated using hardware, it can also be simulated using software, which is exactly what I've done. You can now provide virtual surround sound on any pair of stereo headphones, no extra hardware required. 

Unity currently simulates 3D sound by using volume to simulate distance, panning to simulate left and right, and the doppler effect to simulate relative velocity. However it does not provide any audio cues for front and back, or when sounds are occluded. The Spatial Audio package improves this by adding left/right phase adjustment, and frequency filtering to simulate sound from the rear.

The package also simulates sound damping as a result of being occluded. These subtle adjustments in sound give a significant improvement in spatial awareness to the player.

I've uploaded a web player demo so you can hear the difference for yourself.