Part IV: More Than TV – Diving into the Evolution of Broadcasting with Marcelo Moreno
Since the stone age, people gathered around the firepit to tell stories. Not too long ago, families were likely to meet in front of the TV during the evening primetime to watch their favorite show in the linear program. Today, the TV still is the centerpiece of many living rooms – even though the ways to consume content, to consume stories, have multiplied. The human desire for stories remained the same through the ages, the way we present and receive them has evolved. Being part of the progress of broadcasting services motivates Prof. Dr. Marcelo Moreno on the daily. That’s why he has been leading R&D and standardization groups in Brazil and abroad for many years now. Currently, he is visiting professor and researcher at the International Audio Laboratories Erlangen.
Moreno’s home country Brazil is about to have one of the most advanced broadcasting systems of the world. The so-called TV 3.0 project, which is led by the country’s Ministry of Communications, is in full swing. The aim is to provide viewers with a variety of new features, including personalized content, enhanced accessibility, an app-based TV experience, and immersive audio. Moreno contributes to the evolving new broadcasting system as a long-time member of the SBTVD Forum (Fórum Sistema Brasileiro de Televisão Digital). Last year, MPEG-H Audio was selected as the only mandatory audio codec for the future broadcasting system.
Changing (Daily) Lives through Application Coding
One major field of research for Moreno and his team at the AudioLabs is Application Coding. This innovative component of modern digital TV systems enables broadcasters, content producers, advertisers, and TV set manufacturers to deliver interactive multimedia applications to viewers. App-oriented television uses an approach people are familiar with through their smartphones and tablets. In the future, the viewers’ consumption journey may not start with tuning into one TV channel, but with selecting the desired content in an app. Apps can aggregate a content ecosystem and refer to apps from other broadcasters, providing viewers with personalized experiences. Speaking of personalization: TV 3.0 will make customized storytelling possible, which presents multimedia content users can interact with, even up to the point where they decide between different paths of a story to follow their favorite character’s point of view. Such interactive experiences must be enabled by multimedia applications provided by the content producer.
Not only personalization, but also accessibility is a big topic in the future of broadcasting. Audio description, language selection, and improved speech intelligibility contribute to that. For people with visual or hearing impairments, the selection of audio presets and objects may be controlled through voice commands or gestures. According to Moreno, the future of TV and its controls will be more diverse and accessible than it is today.
The Future of Broadcasting: Personalized, Immersive, Curated
The researcher expects truly immersive experiences to play a major role in tomorrow’s media consumption. For example, he anticipates what he calls “orchestrated pervasive storytelling” – a way to consume content that crosses many platforms and media – to emerge. The technique has already been put into practice in projects where a podcast was created as part of a fiction movie and expanded that story, or in a TV series that delivered background info through a book. Such cross-platform-consumption has the potential to be turned into an orchestrated, immersive experience delivering highly personalized stories. Multiple personal and public Internet of Things (IoT) devices may play a big role in this.
Stories could be personalized according to people’s daily routines and the events that shape their days could serve to unlock experiences that are part of a story they are currently participating in. In this thought experiment, several devices could be part of the story: at home, on public transport, and in the coffee shop. As soon as a “storytelling orchestrator” detects that the current situation of a given user comprises all circumstances required to unlock an experience, they will notify the user about the possibility to continue the story they started experiencing earlier. Friends may experience pervasive stories together and their location or proximity can even be used as part of the circumstances required to advance it. Audio and voice interaction would play important roles in this pervasive storytelling scenario played out in an open and connected world.
The Metaverse – a VR Parallel World Driven by Audio?
In research, there is no generally accepted definition of “the metaverse”. Some experts say it includes many types of technologies that complement real life with virtual services and content. This concept is more abstract and encompasses more than what is commonly understood as “the Metaverse“ constructed by tech giant Meta.
Audio consumption and voice interaction are easier to integrate in people’s daily routine than video content. That’s why scientists like Moreno focus on finding out how audio can sometimes be the main or only immersive medium in such a complex experience. He explores what kind of metaverse use cases can be established that could or even must have immersive audio as the protagonist since audio integrates seamlessly into people’s day-to-day life – at home, at work, driving their cars, walking the streets, and on public transport.
The endless possibilities and visions for future media consumption will surely keep Marcelo Moreno busy – and inspired.
Have you read part one, two, and three of our AudioLabs series yet?