Draft:Object-based audio

• Comment: See WP:Peacock. Shadow311 (talk) 19:40, 10 November 2024 (UTC)

---

Introduction

Object-based audio (sometimes called immersive sound or 3D audio) is an advanced surround sound multimedia technology. Its key concept is immersing the listener in the multimedia experience with surround sound enhanced with an overhead speakers and precise sound direction metadata. The speakers are assessed around the subject and their exact location is inputted into a mixing software. Then a 3D matrix is created, where each sound is encoded with metadata as fluidly moving objects, that are seamlessly continuous. This method enhances the listening experience with dynamic, immersive sound that surpasses traditional stereo, mono, and channel-based surround sound systems. Object-based audio has been in the consumer market since around 2012 when Dolby Atmos was introduced in Los Angeles for the first time with the movie Brave. Many competitors created their versions in the following years, such as DTS, MPEG-H or Auro-3D. Today, we can find object-based audio in cinemas, home entertainment devices, gaming consoles, headphones, mobile phones, home cinemas, music, virtual reality AR, podcasts, and e-books.

History

Object-based audio builds on top of popular surround sound technology. Even though the first immersive sound format was introduced in 2006 by Auro-3D, it added an overhead speaker but used channel-based approach. The first object-based audio technology for the public was introduced in 2012 with the movie Brave from Disney and Pixar. Other audio format manufacturers followed in the next years with their competing formats. MPEG-H from Fraunhofer was demonstrated for the first time in 2014 with the main focus on broadcasting, DTS launched its object-based audio format DTS:X in cinemas in 2015 with the movie Monk comes down a mountain. Auro-3D later presented its version AuroMax. The first consumer object-based audiovisual receiver was unveiled in 2015, created by Sony with enabled Dolby Atmos. Later on, in the same year, the first soundbar with Dolby Atmos was created by Yamaha. In 2017, MPEG-H format was adapted into ATSC 3.0television broadcasting in South Korea and in 2019 to existing HDTV services in Brazil. DTS furthermore expanded their line-up of products with DTS:X Pro in 2019

Technology

5.1 Surround sound - Channel-based audio

Object-based audio is a modern audio technology, offering a truly immersive experience. It adds a depth layer to a widely popular surround sound, that has certain limitations. A surround sound audio technique uses multiple speakers to enhance played digital multimedia content with improving spatial sound accuracy over regular mono or stereo playback. Traditional and the most common surround sound setup is called channel-based audio. This setup has a precise location for each speaker surrounding the listener. Channel-based audio (CBA) typically consists of 5 speakers with a subwoofer (5.1) or seven outputs with a subwoofer (7.1), where CBA 5.1. setup has a subwoofer, a center speaker, two rear and two back speakers. CBA 7.1. adds two speakers, one speaker on each side from the listener respectively.

However channel-based audio has its limitations. The main flaw is audio fixed to each channel or speaker. Even though the audio has been accurately mastered, the speakers might not be distributed around the room properly, which leads to inconsistency, and the listening experience will not be as intended by the creator.

Object-based audio creates a solution for this problem. It encodes the sound as dynamic objects with positional metadata containing a particular location and movement of the object. The renderer renders the audio content precisely, as the artist intended. Any object-based audio contains two types of audio content: Beds and Objects. Beds are channels from channel-based audio, where static audio comes from, typically dialogue, non-diegetic music, or ambiance. Objects use a Cartesian coordinate system (x,y,z) for placing audio tracks in the 3D matrix. Metadata then plays the desired dynamic audio relative to the listener and the speaker setup. They contain information about the audio file such as information about the file, location of the sound in the matrix, levels, panning, filters, fading, and much more. Object-based audio also enables interactive and personalised audio experience for each viewer in comfort of their home.

Use cases/applications

Cinema

Object-based audio is a technique in cinema sound design, enhancing the audio experience. Formats such as Dolby Atmos, MPEG-H or DTS:X use this technique to immerse the audience and to precisely reproduce the intended sound by the director and sound designer. Sounds are not limited to speakers like channel-based audio, they are joined together in a technique called Vector based amplitude panning. It positions virtual sources of sound into a 3D space, acting as an object. Then forms a triangle from the three closest speakers around the virtual object and adjusts the volume relative to its location in the 3D space. Object-based audio also uses Wave Field Synthesis and High Order Ambisonics (HOA). This enables the sound engineer to place the objects above, next, or underneath the viewers, creating an illusion of the exact location of each subject.

Television, broadcast

After the breakthrough of the object-based audio market in commercial cinemas, it shifted to consumer-oriented technology and content.  The leader in broadcasting-oriented object-based audio is the MPEG-H format invented by the Fraunhofer Institute. Although many consumers do not own the technology for reproducing this audio format yet, many audiovisual manufacturers include it in their hardware. MPEG-H allows the viewer to experience immersive and personalised broadcasting thanks to the ability to change volume presets, adjust dialogue volume, and more. Consumers can also access object-based audio in VOD applications, for example Netflix, Disney+, Max. Most of movies created after 2015 are mixed in object-based audio such as Dolby Atmos or DTS:X. Also, many older movies are remixed in newer formats, for example Jaws Remastered.

Music

Object-based audio is another tool for music producers to create unique sounding content. It adapts the concepts of object-based audio and applies them to music, enabling producers to immerse the listening audience. Many streaming music providers offer object-based audio music, for example Apple Music, Amazon Music, or Tidal. Music can be rendered from playback or live musical performance. Either listening to music on speakers, using technology like in cinema/home cinema, or with headphones. Headphones have two output sources, and have limitations. Therefore a new method to achieve the required 3D effect had to be developed. This method is called binaural audio, which achieves 3D sound effects by simulating spatial differences between two channels to mimic real-life sound directions.

Types

Dolby Atmos logo

Dolby Atmos

Dolby Atmos has been the market leader in immersive sound technologies since its unveiling in 2012. This technology is being used in around 6100 cinemas worldwide, with more than 6000 movies mixed in Dolby Atmos format. Dolby Atmos builds on Dolby Surround 7.1, expanding it to a 7.1.2 three-dimensional format. Upgrading from Dolby's AC-3 format to AC-4. Dolby Atmos supports up to 118 objects and 10 sound beds (channels) for the creative freedom of artists. It is available for theatres, home cinemas, soundbars, cars, and phone devices.

DTS:X logo

DTS:X

The main competitor of Dolby Atmos is DTS:X created by DTS in 2015. A key component of DTS:X is MDA, a free, open platform license for immersive audio creation. In comparison, Dolby Atmos had a headstart with disc media (DVD, Blu-Ray) and until 2023 DTS:X was not available on streaming services. Disney+ was the first to offer streaming content with DTS:X with the addition of IMAX Enhanced movies. The cinema version has a limitation of 64 uniquely rendered channels, and the home theatre version has a limit of 11.1 channels.

MPEG-H

Created by the Moving Pictures Expert Group, MPEG-H is a standard for 3D sound. Use cases are mainly in broadcasting, but also in streaming and VR/AR. It enables personalised audio: adjusting the volume for dialogues and switching between many audio channels with different languages.

Conclusion

Object-based audio is positioned to play a significant role in the future of immersive audio in home entertainment, cinema, gaming and music.