360 Music Video – Why the 3D audio potential was never reached?

Time and time again, attempts have been made in the past to revive the music video genre with 360 music video. Even if the initial excitement is no longer in our everyday minds, 360 music videos are anything but dead. It has just, unfortunately, rarely been used to its full potential. VR goggles and social media platforms, however, offer the possibility of all-around viewing, which can deliver real value both visually and auditorily.

And if that wasn’t enough, very few 360 music videos use spatial audio – a missed opportunity if you ask me. Of course, it’s easy to just leave the audio as stereo. But YouTube, Facebook, Oculus and others don’t support spatial sound in the form of Ambisonics and Dolby Atmos just for fun.

This article is therefore not so much about VR concerts; there is already a detailed article on this here. It is not so easy to draw a line here, however, because many music videos also feature a band or artist performing live. But we want to limit ourselves here to the VR360 category, i.e. linear videos, without getting lost in the discussions about the metaverse.

So: which music videos properly use the possibilities of 360° videos? What are some examples of videos that spatial audio would have really added value to? And what makes a good 360 music video, really? These are the questions I explore in this post.

How can a 360° music video be watched?

First, it’s important to differentiate how 360° videos are watched in the first place. Namely, whether they are watched in a browser with a mouse and keyboard, on a cell phone, or on VR goggles, there is a world of difference. It’s a bit like watching a movie with popcorn in the theater and then comparing how much fun it is on a smartphone on the train.

YouTubeVR provides a nice overview in four categories for this: VR.YouTube But for me, it’s only three, because VR goggles are very close to the experience with Cardboard. At least in the points that matter. So let’s start:

The full range with VR goggles

The user experience also plays a role, because with VR goggles, you turn your head physically. This also makes the rotation of dynamic head tracking of sound work better for our brains.

On VR goggles, you experience the full intended effect of the 360° video. Immersion is what we like to call the full dive into a digital reality. You can intuitively turn around and the image and audio move with you. On a stationary device, the situation is different – the movement must be actively initiated by the viewer. Or maybe the video is watched completely without movement.

For the sake of being complete, the inexpensive cardboards should be mentioned at this point. These are not exactly popular in the VR scene because a “breath-taking immersion” is often falsy promised. But the tracking of a smartphone that you put into a cardboard holder is simply not as good as that of an HMD (head-mounted displays). In addition, the field of view (FoV) is much smaller and the VR experience is limited by black borders on both sides. Certainly not as breah-taking as a VR headset.

Social media on the desktop with the mouse

However, when there is a computer monitor in front of us and we only adjust the image section with a mouse click, we do not move our heads. The sound does move and we can hear that the surround sound adapts depending on the viewing direction. However, the localization of audio sources is better with VR goggles as mentioned.

Gaze View on the smartphone

There is another intermediate step: You can also use your smartphone and utilize it as a “window into the VR world”. To do this, you hold the smartphone – a tablet is also possible – in front of your face and move your body around in a circle. That way, you can look around the scene and recognize the content on your display.

Due to its nature, this approach is most preferred in AR (augmented reality) applications. The reason is that with AR, you can see your space around the smartphone. On the smartphone, you continue to see this space through the built-in camera, only with additional AR objects. Thus, the environment is augmented.

With 360° music videos, you can already guess that it will be difficult to forget everything around your smartphone. You can also see that you’re still in your room and just staring at your smart device. That’s why the sound is very important here because it supports immersion. You can hear that you are in another world and looking at the display directly with your eyes supports this impression.

The agony of choice is actually none at all

The producer of such a video must take all these possibilities into account in the production – and thus also avoid focusing too much on the “back” parts of the picture. Exactly these can be drawn attention to with the sound. I always say:

“You can miss things in VR – but you can’t overhear them”.

Of course, this is just as true for 360° music videos. So it’s quite possible to use an additional sound design and combine it with your stereo music. Or you can remix your song in 3D audio to create an even more immersive VR experience.

But if you think you can choose one of the three platforms, you are wrong. Because only the consumer decides how he or she comes into contact with the 360° music video. As a producer, you can ask yourself whether you would like it to work particularly well for one medium. Or whether you want it to work equally well for all of them. With all the advantages and disadvantages that the technology brings with it.

What is the best 360° music video?

It is of course very difficult to award the imaginary Oscar for the best work here. Rather, the overview is meant to serve as inspiration as to what works already exist. What you like or don’t like about it is up to the reader to decide.

It is a fact that there are parameters that show how much work and thought has been put into a VR music video. The vast majority of productions, however, have to put up with my criticism that they simply tried it out to be innovative. There’s nothing wrong with that, I think it’s great when people experiment with new and immersive media.

But it’s usually not until the production that you realize that different laws apply to storytelling, production, and post-production – in short, to everything. The vast majority of directors are put off by this, and the first attempt was all that was left. It’s a shame because I speak from experience when I say that it is worth it to keep at it!

Gorillaz – Saturnz Barz (Spirit House) 360

Let’s start with a 360 video from Gorillaz. “Saturn Barz (Spirit House).” In this video, we see the four band members of the virtual band exploring an old house. The video does a lot of things right – when I watched the video. I could always follow the main action and it also made me want to turn around and explore the surroundings within the video.

However, the video is only available with static head-locked stereo, and that’s a real shame. Many scenes, especially those where the band members are talking or sound effects can be heard, would have benefited greatly from 3D audio!

360 music video with 3D audio

Now things get more exciting for audio. There are very few examples that have used 3D spatial sound. On the one hand, these are the so-called live performances. You see artists and also hear the sound from the corresponding direction, which makes total sense. On the other hand, there are abstract videos, where the sound was placed freely in the VR space. So now, at the latest, it’s worth unpacking the VR goggles or headphones!

EDEN – drugs (VR Experience)

Another great example of a successful 360 music video is the video “drugs” by Irish producer and songwriter EDEN. In it, dots, lines and shapes create an abstract world that is visually appealing on its own. I’m a big fan of audio or music visualizations where the image reacts to the audio waveform. Some of you may still know the iTunes Visualizer or Winamp.

The auditive part was produced in Dolby Atmos VR. This theoretically allows exporting, i.e. transcoding to Ambisonics. This would have made it available on YouTube in Ambisonics. But in my experience, something happens during the conversion…weird. As if the transients get broken. So here, the producers on YouTube preferred to go for the stereo.

In general, we can say that Dolby Atmos over headphones has some compromises, but it actually makes sense in VR. You have more of a feeling of being part of the scenery and don’t have this strong “in your head” localization. I have already written something about this topic on vrtonung.de – you can read it here: Click here.

In my opinion, the work here is untypical for Dolby Atmos. Dolby Atmos Music, for example, works heavily with the center channel. The format assumes that the listeners always look forward. This means that they are staring at a screen, just like in the cinema.

But in 360° videos, there is no “front” in the classical sense. Therefore, even here in the mix, the objects are placed more freely in the room or even encourage looking around. In fact, head tracking is less wanted with Dolby Atmos Music.

U2 – Song for someone 360 Version

And last but not least: There are also 360 music performance videos. A notable example is U2’s “Song for someone”. In the video, we first see only the guitarist and the image gradually opens up. Then cuts of scenes from all over the world are added – musicians performing the same song. Every now and then the image switches back to Bono and the band.

This video benefits greatly from spatialized sound. The precise localization allows the viewer to guess the position of the new instruments being added before you have turned your head in that direction and visually recognized the new musician. This 360 music video is relatively old for the VR age. However, it’s still outstanding and a true pioneering example.

Reeps One: Does Not Exist – VR Beatbox with 3D sound

I would say the following video speaks for itself. The picture editing, the locations and of course the sound are well done. Here, locations were also taken that are exciting for the sound. The 360 audio mix is a good mixture of sounds that were recorded in the studio. As well as sounds that sound as if they came from the location.

Conclusion

These videos and many more show that the potential of 360 music videos is far from exhausted. Especially due to the absence of 3D audio and thus fallback to stereo sound in videos, such as mehr erfahren there is still some room for expansion. This means that 360 music videos are a playing field that needs to be explored further. If you need help with this, you can simply contact me without obligation.