The Great 3D Audio Content Overview: inspiring demo examples

3D audio sound has now arrived in many forms to the consumer sector. So the time has come to get overwhelmed! Therefore, with the help of the following overview, some light should be brought back into the darkness. Also, you can get some inspiration right away, because the application areas are absolutely creative, diverse and far from exhausted what is possible with 3D audio content.

It is not so easy to find a structure here, because there are many overlaps. In the end, I decided to differentiate according to “playback media”. Each medium is treated as a separate chapter, which in turn is divided into individual categories. After a short description regarding the used audio and playback formats, audio examples or further links follow. This way you can directly experience what is being talked about.

Radio Plays, Sound Experiences without visual reference

Audio content that is consumed without an additional visual stimulus can benefit immensely from 3D audio technologies. This includes e.g. radio plays, soundscapes, ambient recordings, ASMR, and of course music. The latter, however, will be considered separately as the implementation depends strongly on the intended effect of the 3D audio content.

But not only the immersion is enhanced, but there are also more narrative possibilities in terms of storytelling. Due to the spatial imaging possibilities of sound, 3D audio offers even more options to involve the listener in a captivating story or environment.

Linear 3D Sound Audio Games

These audio games can usually be enjoyed over headphones – so they can be listened to at home and on the road without any major hurdle. Therefore, binaural stereo is used here, which can either be recorded using binaural microphones or reproduced using appropriate software. Binaural audio recordings are intended for playback over headphones, as there must be no crosstalk between the speakers. This is the only way to accurately represent spatial impressions.

A famous piece that can already be called a “classic” is the Virtual Barber Shop. Here you can – surprise, surprise – be served in a virtual barbershop:

Non-linear 3D audio drama

The company Owlfield specializes in 3D audio games, for example. Here, non-linear 3D audio dramas are also offered. This means that the listener can influence the story by making decisions at the end of each chapter. Headphones should also be used for playback because of the binaural format for the full gaming experience.

As an example, there is a sample chapter of such an interactive 3D radio play on Youtube. However, the interaction aspect is omitted here. For the impatient: it really starts at 1:30min.

ASMR

Binaural microphones are also popular in the trendy ASMR (Autonomous Sensory Meridian Response) sector and used by so called ASMRtists. This is because the desired feeling of closeness and sensuality can be better conveyed. Although the three-dimensional aspect is not in the foreground here, the audio technology used is almost identical in the form of artificial heads. The only difference is how creatively the 3D microphones are used.

Ambiences

For relaxation, binaural ambient recordings are currently also very popular (marketed as 8D Audio in the example, more about this in the topic Music). So entertainment is not the focus here, rather background sprinkling or active stress treatment. Here, for example, is a rain recording with over 1.6 million clicks:

Sound installations

Sound installations are of course difficult to pigeonhole, but they should be briefly mentioned here anyway. In terms of 3D audio, these are usually circular or dome-shaped speaker arrangements that can create enveloping experiences. For technical implementation, channel-independent formats such as Ambisonics can be used. But spatial sound wave (wave field synthesis) is also a popular medium.

Since installations cannot be experienced here online, I refer to this interesting article about the Ambisonics installation “Kaleidoscope of Rooms”. Also worth mentioning here is Innsbruck’s Audioversum – a museum all about audio and walk-in sound installations full of 3D audio content.

Music

3D audio in music is, as can probably be guessed, a vast subject. Probably every 3D audio technology has been applied to music before. Therefore, it probably makes more sense to address the topic in terms of usage.

Music Streaming with 3D Audio

Meanwhile, some streaming services offer music mixed and mastered specifically in 3D audio, for more details click here. Currently, the audio formats used are AC-4 (Dolby Atmos Music) and MPEG-H (Sony 360 Reality Audio). The playback of these object-based formats can be done via speakers, as well as binauralized via headphones. Speaker playback is currently possible with soundbars like the Sennheiser Ambeo Soundbar or smart speakers like the Amazon Echo Studio to enjoy 3D audio content.

Those of you who have an Amazon Music HD account can search for the “Best of 3D Audio” playlist for corresponding audio samples or read the blog entry above to. Amazon Music HD, also known for its lossless audio quality, supports 3D audio content, offering a wide range of immersive listening options.

8D Audio

Not to be neglected is also the trend known as “8D Audio”. Although this is nothing special from a production point of view (you can find out why here ), it seems to have hit a certain nerve given the sometimes extremely high click numbers on YouTube.

Here is a typical example of “8D music”, where only the master file of the song is moved around the listener using a binaural panner:

Binaural Compositions

With 8D audio, usually a stereo file is simply moved binaurally around the listener. This may have a nice effect, but adds no value to the song compositionally. However, some artists use this technology to enrich their music. Music composed specifically in binaural format makes much better use of the available 3D sound field than 8D music. This is because different sounds can come from different directions here. The keyword is “envelopment”, so that the music is not localized in the head – as it usually is via headphones – but as if it were coming from outside (externalization). With this, you almost forget that you are wearing headphones at all. Here is a nice example from my dear colleagues Delta Soundworks.

Of course, 3D music can not only be produced via headphones. With Dolby Atmos, for example, and the aforementioned streaming providers, the audio objects provide the option of playing these works on several speakers. But this is reserved for people who have a hi-fi system with a multi-channel setup. However, smart speakers and soundbars will help to make 3D mixes more accessible in the living room in the future.

Stereo-Upmix

The stereo upmix should also be mentioned briefly here. This may sound niche at first, but soundbars and smart speakers in particular are increasingly making use of this technology. Here, they try to give ‘normal’ stereo content more spatiality through complex signal processing. The Sennheiser soundbar mentioned earlier as well as the Echo Studio smart speaker have this function, for example. I may reveal that my master’s student has devoted himself to this topic and has been able to make exciting discoveries.

Film, TV, Video

Audiovisual media are particularly difficult to structure. I have named this chapter now to mention what I consider to be the most important cornerstones. On a visual level, this chapter deals with linear 2D formats, because video games and VR will be covered separately.

Film: Cinema, Streaming

The medium of film is and always has been linked to the theme of immersion. It is therefore not surprising that 3D audio plays a major role here. Away from the cinema, good and enveloping sound is also in demand in the private home cinema area.
Most people will probably have heard of 5.1 surround sound before. Until today this playback system is used in cinemas as well as in the private sector. And rightly so, because surround sound undoubtedly offers good immersive listening experiences. What is missing, however, is directional information on the vertical plane, i.e. speakers on the ceiling. This is where newer audio formats like Dolby Atmos, Auro-3D, or DTS:X come into play, which include channel information for height speakers. Of course, movie streaming should also be mentioned here. Netflix offers surround sound and even Dolby Atmos in addition to stereo mixes for some movies with 3D audio content.

For technical implementation reasons, binaural examples are shown here. A scene from the American blockbuster Transformers catapults the listener into the middle of a shootout thanks to Dolby Atmos. Although the audio is binauralized, the scene should not necessarily be equated with an actual Dolby Atmos cinema experience.

But if you have a Netflix account and a soundbar or a multi-channel playback system, you can have a look yourself. Here is a list of movies that have been mixed in certain 3D audio formats. You can filter by medium (e.g. Netflix, Blue Ray Disc, etc.) and format (e.g. Dolby Atmos, DTS:X, etc.), among other things.

TV: Live, Broadcast, Shows

Of course, there are also movies on TV, so this subdivision on TV is intended for live broadcasts. There is also a lot going on here in terms of 3D audio technology. In addition to the aspects already mentioned in the film sector, interaction possibilities are becoming more and more of a thing in the broadcast sector.
Here, the Fraunhofer Institute is a pioneer with its MPEG-H format. In addition to 3D audio, the object-based format also offers interaction possibilities for the viewer. This makes it possible, for example, to turn up the volume of individual audio tracks, e.g. for speech intelligibility, or to switch them off. Theoretically, this format could even be used to pan individual tracks as audio objects in the room. As you can see and hear, there is no lack of technical possibilities for 3D audio content. The challenge at the moment is much more how to make this technology user-friendly and thus bring it to the market. But it also requires certain diplomatic or democratic rules, because, for example, does it make sense for viewers to be able to mute individual participants in political discussions? South Korea is a pioneer in this area, where the MPEG-H format is already on the air.

The following video gives a small insight into the possibilities of MPEG-H. It shows, for example, that perspective changes or switches from stereo to an enveloping sound are possible.

Learn more about next generation audio (NGA) right here.

Video: Online, Social Media

Online videos are not to be neglected either. There is also a considerable amount of content where 3D audio plays a role. On Youtube and Co. you can find some videos with 3D audio – usually binaural for headphones.

As an example one from my own fits quite well 😉 Sounds of Germany is a video series that presents some German cities using visual impressions and matching binaural sounds. In my Portfolio you can find more information about the project.

Another huge topic for social media are 360 Videos that we will cover in the section of virtual reality.

Gaming, Interactives

Similar to film, a symbiosis of image and sound is extremely important for video games. Here, too, the goal is to create the most immersive experience possible. Games even have the advantage of interaction. It is therefore only logical that 3D audio is becoming increasingly relevant here as well. In first-person shooters, for example, it is crucial to know where the opponents are. So spatial audio is used to localize them without having to see them as part of immersive sound experience. Many games are usually played with a headset, i.e. headphones plus a microphone instead on TV speakers. The topic of surround headsets is excluded at this point. Several products focus on the sound of computer games. Just this year, the colleagues from Dear Reality launched a plug-in with which you can integrate their Spatializer plug-in into the game engine Unity. But also most of the other popular engines, like Unreal Engine, offer possibilities to include spatial audio and as so-called middleware even more possibilities to make the sound more filigree from your game consoles.

One of the first-person shooter games that use binaural audio is Battlefield V. In the attached example, you experience a parachute jump before landing on the battlefield. Here, for example, the shots of the enemies can be located well, as mentioned.

Furthermore, I came across an interesting game called “Ear Hockey”, which was primarily developed for visually impaired or blind people. The game principle is similar to the classic game “Pong”, except that the ball can be located auditorily. The game is even available here for free download.

I also came across a list of games that use binaural audio. For those of you who want to research this further.

Also of interest is this article, which looks at the impact of immersive audio in games, using the game Hellblade as an example.

Virtual Reality

Who would have thought it, but 3D audio also plays a role in VR, as the regular reader here knows of course 😉 VR is currently probably the medium that can offer the highest degree of immersion. By using VR glasses, you can literally dive into other worlds that you can experience visually and auditorily from all directions.

VR experiences include VR games, 360° videos, and also social VR. The overlaps with the applications already listed are therefore pre-programmed. The devil is in the details, however, as 3D audio should be a basic requirement for each of these experiences, otherwise it can confuse the viewer rather than support the storytelling with immersive sound.

The playback medium of VR experiences are VR goggles, also called HMDs (head-mounted displays). These usually have headphones, which means spatial audio is decoded binaurally in real-time. Crucial to binaural audio playback in VR is head tracking. This means that head movements are recorded by the goggles so that both the image and the audio can be counter-moved.

From an audio production perspective, therefore, static, i.e. channel-dependent, formats cannot be used. Therefore, channel-independent formats such as ambisonics or object-based audio are used, which are only rendered binaurally during playback. In VR experiences, a basic distinction is made between 3 Degrees of Freedom (3DoF) and 6 Degrees of Freedom (6DoF). The latter means that you can move freely within the virtual environment. With 3DoF, you ‘only’ can look in all directions, which is not necessarily a disadvantage.

VR-Games

These video games can have either 3DoF or 6DoF. In any case, the audio reproduction is binauralized via headphones, as mentioned above.
One of the most famous video games made specifically for VR in recent years is Half Life: Alyx. Here is a short excerpt with binaural audio.

360°-Videos

These are videos that you can watch from all directions. So they have 3DoF, to use a VR term. 360° videos can now also be viewed without VR glasses. The web players of Facebook and Youtube support this format in image and sound (Ambisonics).
As an example, I again refer to my own project. Here you can watch the video of the #EUsavesLives 2 Kenya project.
For more good VR experiences I link to my blog post Best VR content with focus on VR sound.

Augmented Reality/Extended Reality

AR and XR mix experiences of real reality with virtual ones, so reality is digitally augmented. These extensions can be visual, auditory, or even tactile.

Soundwalks

In the 3D audio sector, GPS-supported soundwalks are a major topic in AR. Here, for example, the listener can perceive localizable sounds in public places that are not actually there; this can be useful for city tours, for example, to auditively recreate historical events. For such soundwalks it is important to know the position of the listener, therefore the smartphone usually serves as a tracker via GPS. The cell phone thus replaces the head tracker. Binaurally rendered audio content can then be heard via headphones.

With the free app “INSIDE MPHIL – St. Nikolai” you can listen to a virtual concert of the Munich Philharmonic Orchestra on a meadow behind the church in St. Nikolai. You can approach individual instruments and thus virtually walk through the playing orchestra. Here provides an overview of this.

Another soundwalk concept comes from Vienna. Here, a story is told from the past to the future around Vienna’s Heldenplatz. As a listener, you can follow the story through a walk and relive it in 3D audio. More information is available here.

Conclusion

I hope you can see from this overview how broadly 3D audio is now used. There is no guarantee of completeness, especially since new 3D audio content, new technologies, and new media are always emerging. Therefore this overview should show the current state of affairs and make curious about the further development of 3D audio. I am sure that there will be more to come! Been inspired? Feel free to let me know!