3D audio microphone: the future of sound recording

In a world increasingly characterized by 3D media, immersive audio technologies are playing an ever more important role. The 3D microphone in particular is at the heart of this evolution, enabling the capture of 3D audio with different channel configurations and spatial resolutions.

But what exactly is behind this technology and how will it shape the future of audio recording and hardware?

Relevance of 3D recording technology

3D audio recording technology has established itself as an essential tool in various fields such as film production, music, virtual reality (VR), and augmented reality (AR). Its precise recording of sound sources from different directions enables an immersive and spatial sound image.

Film and TV

In film production, 3D audio recording technology enables a realistic soundscape. By using ORTF 3D mics (Schoeps), for example, soccer broadcasts and other sport events can be recorded with precise localization of sounds and improved sound quality. This technology contributes to the creation of an authentic and dynamic soundscape.

It makes you feel like you’re sitting in the stadium, enveloped in sound with the right surround sound recordings. 5.1 surround has never quite caught on, but thanks to object-based formats such as Dolby Atmos or MPEG-H 3D Audio, 3D sound can be transmitted more easily to people at home.

Recording Music

3D audio technology is increasingly being used when recording music. By precisely placing instruments and voices in recording studios, sound engineers can create a spatial sound image that goes beyond conventional stereo recordings. In 360-degree productions in particular, the use of 3D audio enables an immersive listening experience.

There is no camera here to which the microphone has to be adjusted. This means that the possibilities for microphone placement in the acoustic space are virtually endless. In order to get an overview for music recording, I have discussed the various capsules for microphones and their arrangement here.

Take a look at Advanced Techniques in 360 Sound Recording.

Virtual reality and augmented reality

In the field of virtual reality and augmented reality, 3D audio recording technology plays a crucial role in creating immersive experiences. By precisely localizing sounds, VR and AR applications can create a realistic audiovisual environment. The use of 3D audio allows users to fully immerse themselves in virtual worlds and enjoy interactive experiences.

Virtual mics, like virtual mic one from Sennheiser, have established themselves here in particular. However, there are now more than enough other manufacturers such as Rode and Zoom. What is exciting about the Ambisonics microphone is the fact that the 3D space is recorded in the form of spatial axes. The signal is then converted from A to B format using software.

Depending on the use case

The relevance and use of 3D audio recording technology depends heavily on the respective use case. Depending on the requirements of the project and the available resources, the selection of the appropriate 3D audio recording method may vary.

The continuous development of this technology contributes to the fact that 3D audio is becoming increasingly important in various areas and is playing an ever greater role in audiovisual production.

Requirements for 3D audio in TV production

For the successful use of 3D audio recording technology in the TV environment, certain requirements are crucial in terms of mobility and recording quality, as well as the challenges of playback and the ability to compare different microphone configurations.

There are a number of accessories that need to be taken along for transportation, such as a windscreen for each of the capsules.

1. Mobility and quality: The mobility and quality of the recordings are decisive factors for the successful use of 3D audio recording technology in the TV environment. The focus here is particularly on mobile solutions that offer high audio quality combined with easy handling.
2. Comparison between ORTF 3D and Hamasaki Cube: In the context of the TV environment, different microphone configurations such as ORTF 3D and Hamasaki Cube are compared. ORTF 3D is characterized by its compactness and mobility, while the Saki Cube with its 20 microphones is less mobile in design. Another alternative could be Esma 3D.
3. Application contexts and problems with mobile solutions: Contexts of use such as stadium broadcasts benefit from mobile solutions such as ORTF 3D. Nevertheless, problems can arise in mobile use due to the size of ORTF 3D and the need for additional equipment such as tripods and recorders. 8-channel microphone signals are also not easy to distribute
4. Emphasis on post-production: Post-production plays a crucial role in solving problems that can occur with mobile recordings. Phase problems in particular must be compensated for in post-production to ensure optimum audio quality. This has to do with the spacing of the microphone capsules, which must be precisely maintained during assembly.
5. Challenges in the playback of multi-channel recordings: Challenges occur throughout the production pipeline when reproducing multi-channel recordings. This includes the loss of channels in mixing control and encoding, especially in stereo mixing regimes, as well as limitations in the transmission of uncompressed quality outside the cinema.
6. Realism in comparisons: When making comparisons between different microphone configurations, a high degree of realism is crucial. Comparisons under laboratory conditions are criticized because they do not take into account the differences and interference factors in the real production environment. In practice, you are often subject to the forces of nature and spontaneous rescheduling.

Taking these requirements and challenges into account is crucial for the effective use of 3D audio recording technology in the TV environment and contributes to improving the audio quality of TV broadcasts.

The acceptance and integration of 3D audio

While 3D TVs have lost popularity, there is a growing acceptance of 3D audio among consumers. This could be due to several factors.

Firstly, affordable soundbars and headphones with multi-channel audio have lowered the barriers to entry. As a result, they not only offer a wider range of uses, but also more interaction options, such as listening to podcasts or using voice control.

Another reason for the higher acceptance of 3D audio may be the availability of affordable content compared to 3D video. Platforms such as Netflix offer Dolby Atmos-enabled content, while the streaming infrastructure for 3D video is not yet widespread.

The integration of 3D audio into a device or platform is also increasing in number. Apple, for example, has integrated spatial audio into their headphones, which is being adopted by other manufacturers. However, despite this positive development, there are challenges in integrating 3D audio into different types of media such as music.

There is a need to promote the understanding and use of 3D audio through increased touch points.

Overall, there is hope that the long-term acceptance of 3D audio will remain in contrast to 3D video. This could be due to fewer barriers to entry and wider availability. The integration of 3D audio continues to evolve and could play a significant role in the future media landscape.

Technical aspects of 3D audio microphones

3D audio microphones, such as the Ambisonics microphone, are designed to capture sound in three dimensions – i.e. from all directions. 3D audio microphones use a variety of technologies and techniques to achieve this. These include binaural recording techniques, ambisonics and object-based sound recording.

By combining these technologies, 3D audio microphones can provide accurate and realistic sound reproduction that allows the listener to feel like they are in the middle of the action. An example of advanced surround microphone technology is the use of the Decca-Tree, which provides spatial and detailed sound pickup in 3D audio production.

The importance of a high-quality surround microphone and its accessories cannot be overemphasized when it comes to capturing high-quality 3D audio.

These microphones are crucial for capturing sound in a way that captures the depth, direction and ambience of the recorded space, making them ideal for video recording, HDTV production, film, and live performance recording and symphony recording.

But what good is all that technology, the price of which can go through the roof, if the audio signal is simply not ripping. With surround sound, there is simply no room for a recording made with a surround microphone because such microphone arrays are available.

You have to bear in mind that soccer broadcasts, for example, have completely different possibilities and requirements than tennis, for example. Sounds very similar at first, but the atmosphere that is captured here is dramaturgically very different.

Higher Order Ambisonics (HOA)

Higher Order Ambisonics is an advanced form of Ambisonics that incorporates higher orders of soundfields, leading to a more detailed spatial representation. While traditional Ambisonics typically goes up to third order, HOA systems can utilize higher orders, resulting in a more precise and realistic reproduction of sound events.

By employing HOA, engineers and artists can gain finer control over the spatial representation of sound, which is particularly advantageous in applications such as virtual reality, 360-degree video, and immersive audio.

Mic array

Microphone arrays can optimize the recording of signals to a single point, for example to an instrument or a speaker in the room. This is achieved by localizing the acoustic signal in the room.

As several microphones can be used, the arrangement of the signals can be determined as desired by software. In this case, we can speak of 3D miking.

There is usually no “right” or “wrong” when it comes to microphone placement. There are no limits to the design, but there is always an arrangement in technology that fits well for several use cases and prevails on the market.

Difference between 3D microphone and surround sound recording

The key difference between 3D audio and surround sound lies in the number of directions of the recording.

With surround sound, similar to stereo, we perceive sounds from the front, back, left and right. 3D audio goes one step further and integrates the recording of audio signals from speakers above and below. This makes the 3D soundscape even more realistic and immersive.

It reproduces the sound as we hear it every day, making the sound even more believable. There are various formats for Ambisonics, but usually channel-based formats such as 5.1.4 or similar are used. Here is an overview of the arrangement that could be used for virtual reality: https://www.vrtonung.de/virtual-reality-audio-formats/

Future of 3D audio recording technology

The future of 3D audio recording technology promises to be an exciting development that increasingly focuses on 360-degree productions and brings with it new challenges and potential.

• Changing focus: While the focus has previously been on multi-channel recordings, interest is now increasingly shifting towards 360-degree productions. This requires a reorientation of recording technology.
• Complexity in the mix: Multi-channel microphones inevitably capture all sounds in a scene, which makes mixing the audio track much more complex. Editing individual objects in mono, on the other hand, offers more flexibility and control over the atmosphere.
• Challenges of working with 3D audio: Editing 3D audio brings several challenges, including the complexity of editing and rotating recordings, as well as potential issues such as scene jumps that can affect the coherence of the sound picture. You suddenly have to think about things that were hardly ever an issue with stereo before.
• Dolby Atmos and Surround Atmo: Although Dolby Atmos is considered a promising technology, there is still skepticism about its effectiveness. Often the ORTF format is used for Surround Atmo to ensure more accurate and consistent audio reproduction. The 7.1.2 bed of Dolby Atmos is less suitable for rotation around the room axes.
• Potential of technologies such as Binaural Immersive Microphone Farming (Beam Forming): More precise directionality can be achieved through the use of microphone arrays. One example of this is a start-up from Israel that has developed a 128-microphone array to improve the recording of 3D audio. This allows software to further improve the surround microphone
• Future prospects: The increasing number of channels (e.g. currently usually eight) enables more precise control of sound sources and the suppression of background noise. However, current technologies only offer limited possibilities compared to future developments that promise even higher resolution and accuracy.

With a second-order Ambisonics microphone, for example, you would already need 9 channels.

Current and future areas of use

The use of 3D audio recording technology not only extends to current applications, but also opens up promising prospects for future areas.

One such area is virtual reality, which allows for dynamic adjustment of the sound environment to accommodate the user’s freedom of movement. In the VR environment, users can move and look around freely, which offers the potential to adapt the audio content based on these movements.

3D audio technology can therefore provide information about the user’s spatial environment. Spatial computing is more or less the same, but also describes a physical movement in space.

Compared to traditional media, VR offers a unique opportunity to demonstrate the added value and power of 3D audio. While in traditional media the audience is usually static, VR allows for active interaction that can significantly enhance spatial listening and immersion. However, these possibilities are often overlooked as the focus is often on traditional applications.

Another area that could benefit from 3D audio technology is the communications sector. The use of 3D audio can improve immersion in communication applications and create a more realistic sound environment.

Integration

The integration of degrees of freedom into the recording technology increases the relevance of multi-channel recordings. The more degrees of freedom there are, the more precise and immersive the sound can be reproduced, which in turn underlines the versatility and adaptability of 3D audio technology.

This includes utilizing spatial mic converters and rear microphones to capture audio from all directions, as well as employing virtual mic patterns for enhanced spatial resolution. Additionally, advancements in audio interfaces and USB microphones ensure seamless integration with various surround formats and computer systems.

Challenges and opportunities of 3D audio in different formats

Although 3D audio microphones offer many benefits, they also come with some challenges, including the need to convert recorded signals from A-format to B-format. There is also a need for some order to distinguish between recording 3D audio and isolated audio sources.

Capturing whole and isolated audio sources

A key aspect in new distribution formats such as virtual reality and augmented reality, for example, is the fine line between recording 3D audio and isolated audio sources. To overcome this challenge, it is recommended to record both 3D audio and isolated sources to ensure flexibility in post-production and processing.

This ensures that the speaker can effectively manage the various features of the audio signal.

Take a look at how you can master 360 Production Sound to explore more about recording techniques in immersive environments.

Despite the use of advanced tools, the difficulty lies in creating a credible sound image. In particular, adapting impulse responses for 3D audio in VR/AR is a challenge, as there is no smart solution to match isolated objects with the spatial sound.

Customizing

Further limitations arise when using VR binauralization techniques due to the artificial spatiality and lack of customization options. These techniques may not provide the desired authenticity and can detract from the immersive experience.

Another point is that the discussion about 3D microphones often concerns other aspects than the application in VR/AR. This can lead to misunderstandings and hinder the effective use of the technology.

It is therefore important to consider the challenges and requirements specific to VR/AR information and the respective formats and develop appropriate solutions. This includes optimizing multichannel setups in the studio and ensuring compatibility with various computer systems and outputs.

Conclusion

Considering the various factors and requirements when choosing a 3D microphone, it is clear that there is no perfect microphone. The decision for a specific microphone type depends on various factors, including the type of content, mobility requirements and available space.

5.1 sound remains an important aspect of 3D audio technology as it provides an immersive sound experience for viewers.

ORTF 3D could become the standard method for certain information such as broadcast. This is due to its good resolution, relative compactness, and features which make it practical to use.

However, it is important to emphasize that the choice of microphone type depends primarily on the specific content. Different microphone types may be suitable for different applications and the decision should therefore be made carefully to ensure the best possible recording quality.

Ultimately, selecting a 3D microphone can be both complicated and simple, depending on the specific requirements of the recording environment. Thorough consideration of the requirements and limitations is essential to achieve optimal recording quality and get the most out of 3D audio recording technology.