Advanced Techniques in 360 Sound Recording (Spatial Video, VR)

This article dives into the advanced techniques of 360 sound recording, drawing upon insights from the video titled “Audio Production for YouTube VR” by Tall Guy Films. The video, which can be found at https://www.youtube.com/watch?v=8o_v8FE7QHM, serves as a foundational resource for our exploration into the evolution of spatial audio design and the crafting of immersive soundscapes.

Below, we distill the key takeaways from the video, structured to provide a comprehensive understanding of the creative and technical nuances of recording spatial audio for virtual reality and 360-degree video projects.

The Evolution of Spatial Audio Design

In the realm of 360-degree videos, the immersive experience is paramount. The visual element surrounds the viewer, but it’s the spatial audio that truly enhances the realism, making the virtual environment convincingly real.

“When people watch 360 videos, you want people to watch around. This is easily the best use case for having 360 audio.”

This insight underlines the necessity of spatial audio in creating experiences that are not just seen but felt, transforming the viewing experience into something palpably immersive.

Crafting Immersive Soundscapes

Choosing the right tool for the narrative is crucial, especially when the story demands an immersive approach. the 360 degrees sound recording steps into the spotlight when we move beyond conventional video formats to explore narratives that require an all-encompassing perspective. Don’t use 360 degree technology for the sake of it. Try to find use-cases that won’t have been possible with standard stereo recording methods.

“If there is a format that works with just using a normal camera, then use a normal camera. But as soon as you have something with a point-of-use perspective, this is where it can get interesting.”

It’s not merely about the technical act of capturing sound but about embracing an opportunity to enhance the narrative depth. When the story unfolds in a space that invites the audience to explore and interact, the sound must follow suit, offering cues that guide and enrich the exploration.

It’s about capturing the essence of the moment and the environment, making the audience feel truly inserted into the narrative’s fabric, where the sound is as alive and interactive as the visual world it accompanies.

Microphone Technology and Spatial Recording

360 audio recording is an advanced stereo mic technique designed to capture sound from multiple angles in every direction around the microphone, creating a spherical audio landscape. Creating audio files that match the 360 spatial video with an immersive listening experience.

This method is essential for virtual reality (VR) and 360-degree video productions, where the goal is to immerse the listener in an environment that feels as real and multidimensional as the visual experience. By recording audio in this way, creators can ensure that viewers not only see the world in 360 degrees but also hear it in a similarly expansive manner, enhancing the overall immersive experience.

Can’t wait for the nerdy details of direct recording modes and different mic setups? Read my 360 microphone overview. Remember, there is no one stop, vr audio complete solution. Sometimes you need a dedicated ambisonic, computer or stereo mic too, sometimes a separate recorder, sometimes entirely different sounds.

Please don’t record binaural sound

In discussions about 360 audio, there’s often a mix-up between the terms “binaural audio” and “ambisonics audio,”. While both are related to spatial sound, they serve different purposes. Stereo or binaural audio, designed to mimic the way humans naturally hear sounds, offers a deeply immersive listening experience when using headphones.

However, it lacks the capability for head-tracking, making it less suitable for 360-degree videos where the listener’s orientation can change.

To bridge this gap, the quad-binaural microphone was developed, becoming an innovative addition to my arsenal. This specialized, stereo binaural mic combines the immersive qualities of binaural recording with the flexibility required for 360-degree content, allowing for head-tracking and thus preserving the listener’s sense of presence within a virtual space.

But this is where the Ambisonic Mic comes in. It generates an immersive sound field that can later be rendered to head movement or even proper speakers setups. Capturing VR audio required an ambisonic microphone and sound operators to use different recording modes with multichannel audio.

Ambisonics and Beyond: Choosing the Right Mic

In the quest to achieve the most authentic and enveloping soundscapes for VR, the selection of the microphone is crucial.

“Most of the time people would record 360 degrees of audio with an Ambisonics microphone, which is like the Sennheiser AMBEO,” I’ve observed.

Unlike traditional microphones, Ambisonic mics are specifically engineered to capture sound from all directions, making them indispensable for creating immersive audio environments. The Sennheiser AMBEO, for example, is renowned for its ability to record the depth, direction, and nuances of sound from two or three directions, offering listeners a truly three-dimensional auditory experience.

The use of such microphones is not merely a technical choice but a narrative one, enabling creators to construct soundscapes that are as rich and complex as the visual worlds they accompany.

Size matters

In the landscape of 360 audio recording, the size and mobility of recording equipment play a significant role in the production process. Microphones like the Sennheiser VR mic, while offering exceptional quality, often require additional cables and recorders. Despite three recording modes (upright, upside down and endfire) this can limit mobility and flexibility during on-location shoots.

On the other hand, devices like the Zoom H3-VR offer a more mobile solution since it’s a recorder with a built in ambisonic mic. However, this convenience sometimes comes at the cost of audio quality when compared to more elaborate setups. The Zoom H3-VR, while versatile and user-friendly, may not capture the same depth and nuance as larger, more complex microphone arrays. But can also be useful to stream events with an ios device with its onboard decoder.

Then there’s the ORTF 3D setup, which represents another extreme in the spectrum of 360 audio recording options. It offers standard stereo recording but provides a unique balance, offering a method to easily convert ambisonics audio, making it a compelling choice for those looking to capture detailed spatial soundscapes without compromising too much on mobility.

But I’ve only worked with such a surround sound microphone in extreme cases where the VR camera was equally huge.

Capturing Environmental Authenticity

To truly transport listeners to another place, capturing the authenticity of the environment’s sound is paramount. The ambient noise is what you get from the ambisonics microphone. But the raw ambisonics doesn’t get you far for mixing the pseudo binaural recording method with format ambisonic files later on. For the pseudo binaural recording method wav there is more you need:

“You need some additional isolated sound sources…this is why I have another sound bag…it’s basically also a multi-channel recorder with a pretty decent radio mic system,” I’ve explained.

This approach highlights the layered complexity of recording sound for VR. Beyond the use of Ambisonic microphones, incorporating isolated sound sources and employing a versatile multi-channel recorder are key to gathering a diverse array of sounds that enhance the virtual environment’s realism.

Whether it’s the rustling of leaves, the distant hum of a city, or the intimate timbre of a voice, each element contributes to the tapestry of the auditory experience. By meticulously capturing and integrating these sounds, creators can craft immersive soundscapes that not only complement the visual experience but also deepen the listener’s emotional and sensory engagement with the virtual world.

Overcoming Live Recording Challenges

Navigating the intricacies of audio live recording for 360-degree content presents unique challenges, particularly when it comes to microphone placement and managing environmental sounds. The goal is to capture pristine audio without compromising the visual integrity of the scene or the immersive experience.

Strategies for Invisible Mic Placement

In the world of 360-degree video production, the mantra “try to be as tiny as possible” is more than just advice; it’s a necessity. The last thing any of us wants is for our microphones to become unintended stars of the show, necessitating time-consuming and costly post-production work to digitally erase them from every scene.

This becomes especially pertinent in productions where the immersive experience is paramount, and any hint of the production process can shatter the illusion we’ve worked so hard to create. The challenge, then, is to strategically place our microphones in a way that they remain unseen yet effective, capturing every nuance of sound without drawing attention to themselves.

This often involves innovative solutions, from camouflaging mics within the set to utilizing miniature microphones that can be easily hidden from the camera’s all-seeing eye.

Adapting to Environmental Sounds and Noises

Capturing audio in live environments introduces a myriad of unpredictable elements, from the rustle of leaves in a gentle breeze to the sudden roar of an overhead plane.

“You need quite a lot of cable…and you also would want to preview what you are recording because if you just place the camera here…press record and run away you have no idea what’s happening.”

This insight underscores the importance of not just preparing for the expected but also adapting to the unexpected. The ability to remotely access transport controls becomes invaluable, allowing us to adjust settings on the fly and respond to the dynamic nature of live streaming environments.

It’s about striking a balance between the desire for audio fidelity and the practicalities of on-location recording, ensuring that our presence as creators is felt through the immersive quality of the sound, not seen in the final product.

Post-Production Virtual reality ambisonics audio

In the evolving landscape of virtual reality (VR) and 360-degree video production, the art and science of capturing and manipulating spatial audio. This exploration delves into two critical aspects of audio production for VR: the intricate process of mixing and editing for spatial realism, and the creative application of spatial audio in storytelling.

Mixing and Editing for Spatial Realism

In the realm of post-production, the emphasis on processing spatial audio to achieve hyper realistic spatial recording cannot be overstated. I’ve encountered scenarios where, due to mishandled recording sessions, the audio mix became an insurmountable challenge. Cross-talk from different radio mics created a cacophony that no amount of mixing could resolve.

“Try to think about sound as early as possible…try to go into a project as early as possible.”

It was a classic case of a sound engineer stepping into the complex realm of spatial audio without fully grasping the nuances, confidently asserting, “Sure, I’m your guy,” only to deliver results that fell short of the project’s needs. This experience serves as a cautionary tale: don’t be that person who underestimates the complexity of capturing VR audio.

The Creative Use of Spatial Audio in Storytelling

“Spatial audio can change the story or add value to the story,” highlighting the transformative power of immersive sound in narrative contexts. The role of the audio editor becomes crucial in utilizing spatial audio to its fullest potential, shaping how stories are told and experienced.

Through the creative use of ambisonic audio and binaural sound, storytellers can guide the audience’s attention, evoke emotions, and enhance the narrative immersion. Whether it’s through the subtle cues of ambient noise or the dynamic shifts in a sound field, spatial audio allows for a more nuanced and engaging storytelling technique.

Tools like Playback monitor etc.

This creative process is supported by tools like the Zoom Ambisonics Player software and other technologies that facilitate the easy conversion of the ambisonics files to audio, enabling audio professionals to experiment with different sounds and perspectives.

Ultimately, the goal is to create a complete solution for VR audio that offers listeners a one-stop immersive experience, free from the constraints of traditional audio production methods.

Incorporating these advanced techniques and creative strategies in audio production ensures that the final product—whether a video or audio editor, it’s mixed reality content, a live stream event, or a VR experience—delivers on the promise of immersive audio.

By addressing the challenges of recording, mixing, and editing spatial audio, creators can offer audiences a truly enveloping sound experience that goes beyond mere listening to become a key element of the narrative and immersive journey.

Picture lock is not picture lock

In the VR and 360-degree video production world, hitting that picture lock is crucial. It’s the moment I, as the sound engineer, get the final cut from the video team. That’s when the real fun begins: sound mixing. Something hard to put into a blog-post as a video free to download, but is part of coaching VRTonung learning.

But here’s a twist – sometimes, the video gets rotated post-picture lock to better fit the scene’s perspective. Sounds simple, right? Well, not so fast. This rotation, while great for visuals, can throw a wrench in the audio works. The good news? Rotating the Ambisonics b format files to match the video’s new orientation is easy.

This scenario highlights why clear communication between audio and video teams is non-negotiable. It also shows the need for sound engineers to deeply understand ambisonic audio files.

The Future of Interactive and Immersive Audio

As we dive into what’s next for VR and immersive audio, it’s clear we’re on the cusp of some game-changing innovations. The journey’s exciting but comes with its fair share of “uh-oh” moments, especially around getting sound folks in the loop early enough.

Ever been in those meetings where the sound guy gets the memo last?

“The problem is that most of the time sound guys are not involved in this question…it’s really hard to guess what will be the final result.”

Here’s the thing about 360 videos and VR – if you’re not pairing them with 3D video or audio editor already, you’re missing out. Remember what I like to say:

“in VR you can overlook an element but not overhear it”

It’s about making every head turn and every glance come alive with a sound that’s just as dynamic as the visual feast you’re serving up. With the tech getting smarter, we’re gearing up for a future where immersive surround sound doesn’t just complement the visuals; it completes them.