Join us for a deep-dive into the partnership of IIS and Friedrich-Alexander-Universität Erlangen-Nürnberg! The series starts now …
Part 1: MPEG-I and Jürgen Herre
Quite a lot has happened in since the International Audio Laboratories Erlangen were founded in 2010, just like one expects at an international research organization. All the more reason to give the “AudioLabs”, as they are usually called, a look-in and to introduce each of the research fields: At the joint institution of Fraunhofer IIS and Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) five professors, a staff of some 50 people, and international guests are researching a range of audio fields that are as complex as they are interesting. From Audio Signal Analysis all the way to Virtual Reality: The scientists provide important stimuli for the audio technologies of today and, even more importantly, for the ones to come (more).
In the first part of this series, we are visiting Professor Jürgen Herre who was one of the founding members of the AudioLabs just over a decade ago. After graduating in electrical engineering from FAU, he started his scientific career at Fraunhofer IIS in 1989. “I have developed algorithms for audio coding ever since, but I never thought about VR until 2017 – and most certainly not when I was a student,” he smiles. The graduated electrical engineer also has held the position of Chief Executive Scientist at the Audio and Media Technologies Department of Fraunhofer IIS for more than 20 years. This means that he has contributed to the evolution of the MPEG-4 and MPEG-7 Audio technologies as well as to mp3, mp3 Surround, MPEG Surround, Spatial Audio Object Coding, USAC, and MPEG-H Audio. He has been chair for audio coding since September 2010, and in recent years also of audio coding for Virtual and Augmented Reality at the AudioLabs, a responsibility he initially shared with Prof. Frank Wefers.
MPEG-I – Virtual Reality for the ears
Virtual Reality (VR) can transport people into a very realistic illusory world. In order to make that illusion seem entirely real, that is, to reach a so-called immersion, the audiovisual simulation has to be absolutely convincing. In future, the new standard for Virtual and Augmented Reality applications, MPEG-I, is meant to contribute to exactly that. To achieve this goal, Jürgen Herre’s team is working on the audio part of this field. Recently, they celebrated a great success: The technology they submitted to the MPEG Audio Standardization Committee was selected as the best system and won the Call for Proposals (CfP) in January 2022.
“We are working to make it possible to walk across the room during a virtual concert and to have a realistic audio experience at any position,” Herre explains. The currently available technologies such as MPEG-H Audio enable only three Degrees of Freedom (3DoF). This means that listeners can freely move their head while remaining at a fixed point in the room (this means they can turn it, move it forward, backwards, and sideways). The AudioLabs-team angled its research towards enabling 6DoF in the near future. This includes location and orientation data and, through this, makes possible the free movement through a room. For sound sources this means that, unlike before, each one has to be defined by three axis coordinates and three rotation angles. In addition to that, source extent and directivities are considered. MPEG-I – I for Immersive Media – serves as an overall system for audio and video reproduction and could not only be a revolution for virtual concerts but also for sports events and other virtual environments.
Moving through rooms of sound
The option for audiences to move freely through a virtual space requires the complex modelling of the environment. For a room’s reverb characteristics, an exact description of its geometry and surface structures is essential, because they reflect, scatter, and transmit sound. “Currently, we are working to find a way to integrate sound propagation in rooms and the famous Doppler-effect, that is, the sound from fast-moving sources, into smart phones in a cost-effective way,” explains Herre. “We are also researching the realistic reproduction of spatially extended sound sources such as beaches and the refraction of sounds with invisible sources, a phenomenon that is common in contorted rooms, for example in labyrinths.” To do this, the AudioLabs team researches how sound spreads around corners and obstacles and how this phenomenon can be reproduced in real time.
Header image: © Felix & Paul Studios