Fraunhofer’s Spatial Audio Microphone Processing SDK is designed to ease the high-quality capture of authentic spatial audio content on set. The automated processing algorithm is the first audio technology to be delivered under Fraunhofer’s upHear® brand of immersive audio innovations and Felix & Paul Studios is the first partner to incorporate and refine Fraunhofer’s algorithm into a virtual reality camera system.
“As the world’s only full-spectrum VR studio, we are dedicated to creating captivating, immersive experiences and sound plays a very important role in virtual reality storytelling,” said Sebastian Sylwan, Chief Technology Officer and Creative Partner at Felix & Paul Studios. “Fraunhofer is a world leader in digital audio technology and innovation and we are delighted to partner with them to incorporate their upHear Spatial Audio Microphone processing into our next generation camera system; the full integration with our cameras makes the capture of a faithful spatialized scene rendition an integral, no-overhead part of the filming process,” Sylwan continued.
“Our upHear Spatial Audio Microphone Processing solution will enable Felix & Paul Studios to enhance their immersive VR content with spatial audio in a simple, yet robust way,” said Jan Nordmann, Senior Director, New Media at Fraunhofer USA Digital Media Technologies. “We love to see our technological innovations in the hands of content creators and are very excited to work with the leading cinematic VR studio to enhance their sound capture and processing in the next generation of their camera systems.”
Fraunhofer’s Spatial Audio Microphone Processing was designed for professional and consumer 360-degree cameras and mobile devices, improving sound capture using the built-in microphones of the device. The algorithm automatically transforms captured sound signals in real time to any popular surround or immersive reproduction format, including First-Order Ambisonics (FOA) or Higher-Order-Ambisonics (HOA), immersive 3D audio with 7.1+4 height channels, or 5.1 surround for legacy systems, while preserving the authenticity of the original audio scene. Its flexible semantic signal analysis approach allows for adoption into devices with three or more microphones in various configurations, as well as more sophisticated configurations like in the case of Felix & Paul Studios’ cameras.
Jean-Pascal Beaudoin, Head of Audio at Felix & Paul Studios and Director of Sound at their subsidiary Headspace Studios further commented: “Fraunhofer’s technology allows us to faithfully and precisely capture scene-based audio that we can later combine with other sources for a fully immersive audio experience.”
Manufacturers, professional content creators, as well as consumers can benefit from the Fraunhofer Spatial Audio Microphone Processing technology:
- Manufacturers – the flexibility and customization of Fraunhofer’s spatial audio capturing algorithm to unique camera and mobile designs, as well as the possible integration of Fraunhofer upHear in devices, mobile applications or in post-production software can benefit various product categories.
- Content Creators – professional content creators will benefit from Fraunhofer’s audio quality standard through position-accurate 3D audio and a natural ambience that naturally carries the atmosphere of the captured situation. Furthermore, in post-production, sound designers can easily combine those audio scenes with additional sound elements like voice overs or dynamic objects.
- Consumers – integration of Fraunhofer’s spatial audio capturing technology in cameras and phones allows the ability to capture immersive sound to accommodate a video with the push of a button. This aligns with today’s consumer demand for products that are easy to use and provide high-quality recordings.
Header image © Fraunhofer IIS