Immersive Audio Rollout Looms

Email_Content_MichaelGoldman

Hot Button Discussion

by Michael Goldman

The rise of immersive audio for cinema has entered an exciting new chapter of final testing and preparations for rolling out the format widely some five years after SMPTE’s TC-25CSS Cinema Sound Systems Working Group on Interoperability of Immersive Sound Systems in Digital Cinema first formed to standardize the delivery of immersive audio. Brian Vessa, Sony Pictures’ Executive Director of Audio Mastering, has been a big part of this evolution as the Founding Chair of the TC-25CSS committee and chair of the IMF Audio Essence Drafting Group in SMPTE’s TC-35PM Media Packaging and Interchange Technology Committee.

Vessa points out that the audio standards community has been “a bunch of busy beavers” in recent years, making sure that all the necessary standards got published, particularly the ST 2098 suite of immersive audio standards announced by SMPTE in the last year, and various related documents. These include ST 2098-1, which  Vessa calls the “parent document” for defining immersive audio metadata; 2098-2, which he calls the “main document”—the Immersive Audio Bitstream (IAB) specification; and 2098-5, which defines digital cinema immersive audio channels and soundfield groups—“basically, all the immersive audio channels and the current loudspeaker configurations for immersive audio,” he explains.

Along the way, he adds, SMPTE’s Digital Cinema technology committee, TC-21DC, worked in parallel with TC-25CSS to standardize the protocols necessary for making sure that immersive audio devices could interoperably communicate with Digital Cinema Packages (DCP’s).

“A number of associated documents were also recently created that specify how a IAB DCP that contains immersive audio should get made,” he says. “One document explains the MXF track file [ST 429-18] that IAB fits in; another the DCP operational constraints [ST 429-19]; the immersive audio KDM [Key Delivery Message, ST 430-1 addendum]; and the creation of a Digital Sync Signal and Aux Data Transfer Protocol [ST 430-14].”

Email_Content_MichaelGoldman

Hot Button Discussion

by Michael Goldman

The rise of immersive audio for cinema has entered an exciting new chapter of final testing and preparations for rolling out the format widely some five years after SMPTE’s TC-25CSS Cinema Sound Systems Working Group on Interoperability of Immersive Sound Systems in Digital Cinema first formed to standardize the delivery of immersive audio. Brian Vessa, Sony Pictures’ Executive Director of Audio Mastering, has been a big part of this evolution as the Founding Chair of the TC-25CSS committee and chair of the IMF Audio Essence Drafting Group in SMPTE’s TC-35PM Media Packaging and Interchange Technology Committee.

Vessa points out that the audio standards community has been “a bunch of busy beavers” in recent years, making sure that all the necessary standards got published, particularly the ST 2098 suite of immersive audio standards announced by SMPTE in the last year, and various related documents. These include ST 2098-1, which  Vessa calls the “parent document” for defining immersive audio metadata; 2098-2, which he calls the “main document”—the Immersive Audio Bitstream (IAB) specification; and 2098-5, which defines digital cinema immersive audio channels and soundfield groups—“basically, all the immersive audio channels and the current loudspeaker configurations for immersive audio,” he explains.

Along the way, he adds, SMPTE’s Digital Cinema technology committee, TC-21DC, worked in parallel with TC-25CSS to standardize the protocols necessary for making sure that immersive audio devices could interoperably communicate with Digital Cinema Packages (DCP’s).

“A number of associated documents were also recently created that specify how a IAB DCP that contains immersive audio should get made,” he says. “One document explains the MXF track file [ST 429-18] that IAB fits in; another the DCP operational constraints [ST 429-19]; the immersive audio KDM [Key Delivery Message, ST 430-1 addendum]; and the creation of a Digital Sync Signal and Aux Data Transfer Protocol [ST 430-14]. A document to standardize SMS (Screen Management System) to OMB (Outboard Media Block) communication protocol should publish soon [ST 430-17].”

Another document, called EG 2098-3, is still pending, he adds, “which gives guidelines for a cinema Immersive Audio Renderer, which will be very beneficial to manufacturers, but will not hold up the rollout.”

These combined developments are a game-changer for the audio industry because they now permit the standardized immersive audio bitstream to represent a single immersive audio mix and be distributed to theaters using a standardized IAB DCP. And that, in turn, means that any theater that has an immersive audio system in place that is compliant with SMPTE standards can play the DCP simply by receiving the immersive audio KDM from the content provider. In other words—immersive audio for cinema has finally reached the interoperability stage, meaning theaters outfitted with any of the current major immersive audio systems (Dolby ATMOS, Barco Auromax, and DTS:X) will be able to utilize the same DCP and play it into their own proprietary systems.

As far as the business of immersive theatrical audio is concerned, this reality means that manufacturers can enjoy a healthy competition for finding theatrical installations for their immersive audio systems, and more of them can join the fray. And, Vessa adds, these advances should also make life easier for studios and other content creators.

“This is a boon to theater owners, as they can now buy any SMPTE-compliant immersive audio system that fits their needs, and know they will be able to receive all available immersive audio content,” Vessa points out. “And on the content creation side—the mixing side—I won’t have to make a separate mix for each manufacturer’s system anymore. I’ll make one mix, and it will then be exported into the standardized IAB and placed in the IAB DCP for distribution to any immersive audio theater.”

Vessa elaborates that Digital Cinema Initiatives (DCI) has now officially accommodated immersive audio by updating its Digital Cinema System Specification (DCSS) to indicate how immersive audio—described as Object-Based Audio Essence (OBAE)—will be handled in digital cinema, and allowing for DCI’s Multiple Media Block (MMB) architecture to permit a second media block that can play OBAE.

All this means the wide rollout of immersive cinematic audio is about to hit, and it's looking promising, according to Vessa. The industry is pursuing the rollout initiative in partnership with the Inter-Society Digital Cinema Forum (ISDCF), which he describes as “a dedicated group of industry people from facilities, manufacturers, studios, exhibitors, and other walks of life within the digital cinema world that work together to solve real-world problems and help move technology forward.”

“Earlier this year, they conducted a hugely successful immersive audio plugfest,” he adds. “We tested all these standards that we have been talking about, got some equipment and test material together, and for the most part, the test material played on the various systems tested. There were a few small glitches, of course, here or there—there are always items we discover in these plugfests that are not implemented in the same manner between one piece of equipment and another. But it was especially encouraging to see that manufacturers had been paying attention to the standards while we were developing them, and then took the time to go to the bench and figure out how to make them work inside their various pieces of equipment.”

Vessa says another ISDCF plugfest is currently planned for January 2020. In terms of the rollout, he expects that if that event goes as planned, “then we are going to be at the point to begin the rollout of the IAB DCP early next year. And I expect [things will go as planned], because the manufacturers have the knowledge gained from the first plugfest, and have had a chance to update their software, plus more manufacturers will be bringing equipment this time.”

Vessa feels this paradigm shift could lead to the creation of more immersive audio content generally, and correspondingly, more places to view such content.

“As noted, having these delivery standards encourages healthy competition,” he says. “Because we now have a common bitstream, any manufacturer can get into the game and design around the standards, adding their own secret sauce to distinguish their system. For theater owners, until now, determining what system to buy was a crapshoot. Instead of buying a system that fits available content—all content fits my system. That’s a real blessing, because I think it will allow more people to come in and manufacture different theatrical systems at different price points and different sizes. Hopefully, at the end of the day, what this will mean is more immersive audio theaters will spring up.

“After all, right now, in the entire world, there are only around 5,000 or 6,000 immersive audio theaters and mixing stages. That sounds like a lot, but not when you consider there are more than 150,000 movie theaters and mixing stages in the world. So hopefully, these standards will encourage growth. The key task here is that the sound systems installed in theaters need to be up to the task of reproducing the soundtrack the way the creatives intended it, so we need to hold up that quality bar!”

Indeed, Vessa underscores that “pretty much every single blockbuster movie being mixed today is being mixed natively with immersive sound, and then we down-mix from that to 5.1 or 7.1 to play in those theaters as needed. We’ve been mixing in immersive sound for a number of years now, and it’s gotten to the point where it’s no more expensive to mix immersive than 5.1 or 7.1, so I expect that trend to grow.”

As excited as he is about the rollout process getting underway, Vessa expects it to be a measured one.

“I expect theaters will be gradual in upgrading their current immersive audio systems,” he says. “Manufacturers have already created the necessary firmware and software updates to upgrade existing immersive audio sound systems in theaters, but the actual upgrades will likely happen gradually, since theaters have the say as to when they elect to make those type of changes.

“I’m hoping it will be faster than the [slower rollout] of the SMPTE DCP to replace the INTEROP DCP we have had for years. We are just now rounding a corner to where we are getting the SMPTE DCP established in most theaters, but there are still some of them that have not upgraded their software and cannot play a SMPTE DCP, and for those, you still have to supply an INTEROP DCP.  But the big difference, of course, is we only have about 5,000 or 6,000 theaters to worry about at this moment—the ones that already offer immersive audio. At the end of the day, the studios will need to step up and push on exhibitors to upgrade their theaters if we want this to happen in a timely manner. The studios are eager to get to a single IAB DCP distribution model, rather than the multiple DCP’s they need to create now.”

Then there is the issue of upgrading theaters that currently do not feature immersive audio.

“It’s a bigger sound system—you have the physics of more equipment to deal with. I expect it will take a while for equipment prices and implementation costs to come down to where most theaters can afford an immersive sound system, but fortunately, costs have already come down from when they first came out, and that trend will continue. And, of course, new installations will come ready to play the common bitstream.

“But, sadly, most consumers don’t know what immersive audio is, and they probably aren’t deciding whether to go to a particular theater or not based on the ability to hear immersive sound. Surprisingly, it just hasn’t been promoted that much yet, even though it’s been around for quite a while. Hopefully, this will change as immersive audio becomes more available. Will consumers pay more for a movie ticket for it? Good question.”

There is, of course, another side to the immersive audio drama—systems designed for home entertainment. There, Vessa reminds, the Interoperable Master Format (IMF) plays a central role.  Vessa also points out that IMF was originally the brainchild of a dedicated group of studio engineers, including himself, who created the original DVP (Digital Video Package) specification and brought it to SMPTE for standardization. Eventually SMPTE’s TC-35PM technology committee was tasked with designing the IMF standards based on the DVP concept—these standards which were first published about six years ago. The IMF Audio Essence drafting group, which Vessa chairs, was responsible for the audio portions of the IMF standard.

When immersive audio for the home first arrived, IMF did not have standards in place to carry it, so the Audio Essence drafting group was tasked with creating them. A healthy synergy emerged with the work being done on the cinema side that allowed the standard to come together fairly seamlessly. The Immersive Audio Bitstream Level 0 Plug-in for IMF, known as ST 2067-201, emerged, specifying how IAB is wrapped into IMF. This allows IAB to be carried within IMF, rather than as a “sidecar,” as had previously been the case prior to the standard. Thus, ST 2098-2, the standard for the immersive audio bitstream in the cinema world, was woven into IMF.

“ST 2067-201 got published very recently,” Vessa explains. “We essentially used the work that digital cinema did to create ST 2098-2 and ST 429-18, but revising ST 2098-2 with the particulars that we needed to use it with IMF. This made a lot of sense and has precedence—from the beginning, we used as much as we could from the work that had already been done for digital cinema, and brought it to IMF. Remember that IMF was originally going to be called DVP as a nod to the Digital Cinema Package [DCP]. It was a conscious effort when we were working on IMF to utilize as many of the digital cinema standards as we could, and then change or expand where needed. This has already paid huge dividends, because manufacturers that had originally made equipment to create DCP’s found that it was not a big stretch for them to update their software to create IMFs, and that, in turn, gave us a good jump start on actually implementing IMF. And with the new ST 2067-201 standard being published, the manufacturers are already adding immersive audio capabilities to their IMF authoring equipment.

“Now, it’s to the point where immersive audio has taken a big step forward for home systems, and playback equipment is readily available. With IMF as the new mezzanine source, it is easy and straightforward to transcode immersive audio into home delivery bitstreams for streaming or physical disc.”

Vessa adds that an IMF plugfest is scheduled for early next year to test the addition of IAB to IMF, among other topics, and that he expects that rollout will likely get going early next year, as well.

This expected flurry of activity illustrates, in Vessa’s view, the process of standardizing immersive audio is heading in the right direction, “because it means the manufacturers are jumping right in, and showing they can implement it quickly.”

Where it all heads next, when and how the industry might evolve immersive audio for both cinema and home applications, is anyone’s guess. But Vessa emphasizes that the success of the effort to make the delivery of immersive audio a reality will wind up as the foundational building block of whatever does, in fact, come next.

“These standards are very forward-looking intentionally, in order to open the door for future development of new and improved immersive audio systems,” he says. “That was one of the reasons for the effort to standardize the delivery mechanisms, to make it clear that [more advanced immersive audio systems] are here to stay. That fact will eventually allow manufacturers to go in and start getting even more interesting audio systems together over time. I’ve always felt we were just scratching the surface of what was possible with immersive audio in terms of the systems we have now. But standards are needed for a vibrant delivery system to exist. Standards are what allow innovation to occur.”

For more information on these and other audio-related developments check out recent SMPTE webcasts, including “What is Immersive Audio and Why is it So Cool?”; “The Ins-and-Outs of ST 2098-2 Immersive Audio Bitstream”; and “ST 429-18 Immersive Cinema Track File and 429-19 Immersive Audio DCP Constraints.” 

News Briefs

4K Thursday Night Football

A recent report in TV Technology discusses a new plan by Fox Sports to immediately start producing its Thursday Night Football games in 4K with High Dynamic Range (HDR). The report explains that Fox committed to starting such broadcasts by the end of September onward, but adds that the technique used will be to capture games in 1080 HD HDR and employ a conversion process to create a 4K UHD HDR image for viewers with 4K displays, while down-converting to 720p for viewers watching on HD displays. The reason for that approach, according to the article, is that the network wanted to minimize resources such as server capacity that would be needed on those football broadcasts. “[Native 4k server capacity] would require four times the storage budget for four times the resolution,” the article relates. The article also states that a driving factor in the decision was to meet demand for 4K content from Fox Sports distributors. There is no timetable, however, as to when Fox will evolve to 4K UHD HDR production for all NFL games the network broadcasts.

New Sci-Tech Members

The Science and Technology Council of the Academy of Motion Picture Arts and Sciences recently announced the addition of seven new members, increasing the Council’s membership roster from 18 to 25. The new members are Bill Baggelaar, Senior VP of Production and Post Technologies for Sony Pictures Entertainment; Brooke Breton, a veteran film producer and visual effects producer; Buzz Hays, head of the Media and Entertainment team at Google Cloud Solutions; Arjun Ramamurthy, Senior VP of Technology at Twentieth Century Fox/Disney; Rachel Rose, an R&D Supervisor at ILM; Dave Schnuelle, VP of Technology at Dolby Laboratories; and award-winning cinematographer Mandy Walker. The council will have 16 members returning in addition to the seven new members, and its co-chairs for the 2019-2020 cycle are Visual Effects Branch governor Craig Barron and Member-at-Large Annie Chang.

Quantum Supremacy?

According to recent reports, quoting an original report in the Financial Times, Google researchers claim to have achieved what is known as “quantum supremacy,” which essentially means they have proven in an experiment the ability to make a quantum computer perform a task that traditional super-computers are unable to process. There is some confusion about the claim, however, because a white paper describing the details was posted, according to the MIT Technology Review’s coverage of the story, on a NASA Website but later taken down, and Google is not currently commenting on the particulars. Fortune Magazine’s story suggests the Google paper was published early on the NASA site, and thus taken down because it had not yet been fully vetted in a scientific peer review process. In any case, reports say the experiment in which quantum supremacy was achieved involved calculating the output of certain specialized circuits through a particular scenario that available supercomputers were unable to process. If the development holds up, it would be quite significant because of the possibilities of massive quantum processing power helping researchers to create new drugs, new materials, and to “turbocharge AI,” among other things, as the MIT Technology Review article put it.