AES Southern German Section – The VR Audio Roundtable
For this month’s AES South German Section Research Colloquium, a roundtable of experts in VR Audio will be discussing the latest and greatest developments in the field, with full interaction with the audience!
Further details here: http://bit.ly/38vy7Kd
Speakers
- Przemysław Danowski (New media / audio / video / VR specialist. PhD candidate in spatial audio)
- Made Indrayana (CTO at Double Shot Audio)
- Ben Supper (Audio and acoustics consultant)
- Katja Rogers (Postdoctoral Researcher with the HCI Games Group, University of Waterloo)
- Martin Rieger (3D Audio Technology & Immersive Content Creator)
Roundup and Conclusion
Tactile manipulation of sound in VR and musical instruments
Przemysław Danowski said that VR brings new possibilities for the interaction between people, for example to make music together in a fully 3D environment with dedicated digital instruments. These are controlled by facial expressions and gestures. This approach reduces the separation between the musician and the audience, and music-making can be used as a social platform, as it was in the past. Not only musicians can create a VR world, an additional collaboration is possible where listeners can influence their musical experience.
More research is needed in this area to improve these algorithms in the future. Also, latency is still a big issue when wanting to stream instruments or vocals from the real world to VR. It was mentioned that it’s not just about connecting people around the world. It is also about the ways in which the physical world can be connected to the virtual world, how that connection is experienced, and how it can be improved.
Making 3d audio and VR more accessible
An important aspect that came up by Martin Rieger was how we as audio engineers can get spatial audio to the people. Even with VR headset becoming more prominent it can still be difficult to even share applications with spatialized audio. There is a lack of standards like MPEG-I and also a barrier for non-audio people to even be able to value our work. This is a topic that we need to approach as a community.
Low barrier to entry for VR Audio
Many VR audio applications, content, and learning materials are available to everyone, mostly for free. Made Indrayana points out there is already a large community of engineers and content creators engaged in this field of work. Institutes are working to standardize platforms to bring together the developments of contributors. There is widespread use of VR experiences that are not only utilizing sound to entertain. Where most people think of creative content in the music, film, and gaming industries.
Research is also being done to improve the experience of everyday things like virtual meetings. In addition, virtual prototyping and augmented troubleshooting are important for training. This is where humans deal with critical and important situations and must be as realistic as possible. Speaking of realism, the panel agrees that the term hasn’t really been defined.
Game audio interaction and research
It was pointed out by Katja Rogers that most first-time virtual reality viewers don’t even listen to the music of a game. It depends a lot on the application and the individual how much sound even matters. There is also a lot of research being done on the experience and impact of VR audio in different situations, rather than perfecting audio quality that is not even perceived by most listeners
importance of getting head-tracking correct
These facts led to an open discussion on how to design the interaction to create some kind of realism for the user as Ben Supper states. Questions were raised about whether some kind of realism is really the goal of virtual worlds. As we know from film sound, a larger-than-life soundtrack is desired. The next questions to be answered should be: what is the influence of audio in the virtual reality world, and how it can be designed to enhance the user experience.
We would like to thank all the experts and participants for taking part in the conversation. After an hour and a half, there was still a lot to discuss and there is so much happening in virtual worlds right now. We look forward to many great technologies in the future.
Below there is some additional information that was mentioned in the discussion.
Shared Sources and Links
Interactive project – Umwelten.
https://www.konzerthaus.de/de/umwelten
If I Had You – 3D Audio Demo.
https://www.youtube.com/watch?v=NgAUa1U-hF8
Coretet virtual reality instrument.
http://www.coretet.io/
Immersive Audio outside of the Box.
https://www.dbsoundscape.com/global/de/
Is there a difference between 3D audio and spatial audio in virtual reality?
https://www.vrtonung.de/en/360-heatmap-spatial-vs-static-sound/
Francis Rumsey – Spatial Audio-Reconstructing Reality or Creating Illusion?
https://youtu.be/y82nth2Pnwk
Hyunkook Lee – A Conceptual Model of Immersive Experience in Extended Reality.
https://psyarxiv.com/sefkh/