Immersive Cinema Looms
Hot Button Discussion
by Michael Goldman
When discussing the current state of immersive media in theatrical environments, Steve LLamb, VP of Media Technology Standards at Deluxe, co-chair of the SMPTE 21-DC Digital Cinema Technical Committee, and chair of the 21DC Immersive Audio Drafting Group, emphasizes that providing strategic enhancements to theatrical viewing experiences may someday become crucial for content creators fighting it out in a multi-platform universe. After all, he says, “From the creative producer’s standpoint, if you are talking about immersive media, there is nothing more immersive than being able to take your content with you, wherever you go, whether at home or in your car or on a plane, on a mobile device.”
“That’s why many of the newer entertainment technology companies like Amazon and Netflix are pursuing that niche.”
Thus, LLamb suggests, immersive sound and picture in theaters may eventually become strategic tools in attracting consumers to the cinema, despite their growing list of alternative options. He adds that it is not a new concept, nor one that has been easy to implement up to now. Despite significant technical achievements in this area, he says the entertainment industry is still “in an embryonic/feeling-out period."
“The first roll-out was the 3D market, which pushed the adoption of digital cinema at one point,” he says. A few studios pushed 3D hard in the mid-2000s and thought they could sell more seats if they had a 3D version of their film. Then, 4K, higher frame rates [HFR] and other techniques were introduced, which have now been tried a few times on various movies. Now, techniques like HDR [high dynamic range] and EDR [enhanced dynamic range] are largely used, both in cinema and home theater viewing. That has probably had a lot more impact visually than either 3D or HFR. You can see just about any tentpole movie right now, for instance, in Dolby Vision HDR, a higher dynamic range format.
“And then, of course, somewhere in between all of those, the industry started pushing object-based audio, or immersive audio as we now call it.”
LLamb reminds that SMPTE published a suite of standards for immersive audio bitstreams in 2018 under the ST 2098 label, and now, within the next few months, the TC-25CSS Technology Committee, dedicated exclusively to Cinema Sound Systems, is slated to publish revisions to various cinema sound standards. 21DC, LLamb adds, is also slated within the next month or so to publish standards for the wrapping and packaging of Digital Cinema Packages (DCP’s) with ST 2098 immersive audio bitstreams. The immersive audio standards, of course, were built out of cooperative work in recent years with help from three companies in the immersive audio space—Dolby (ATMOS), DTS (DTS:X), and Barco (AURO). All of which means the audio side of the immersive media equation has advanced more seamlessly than the picture side.
“The original cinematic, immersive sound system was Barco’s AURO 11.1, which added height to the surround, so there were two layers of height,” he explains. “And then, Dolby came out with ATMOS, which was object-based audio, meaning you could not only add height, but you could do objects within a space—spatial audio. That was really cool and took on pretty fast, especially with sound mixers. And then, DTS:X came into that same area.”
ST 2098 was eventually built on the foundation of the ATMOS approach, so that “instead of being three proprietary versions, all three systems could play back a single standardized immersive audio bitstream,” he adds.
On the picture side, however, a standard approach remains a long ways off as it relates to immersive viewing formats, such as side screens with enhanced image capabilities, LLamb says.
“In 2015, Barco came out with Barco Escape, which was essentially a three-projection panoramic experience with two side screens, and incorporating AURO 11.1 for sound,” he says. “The format debuted in 2014 with selected scenes from the film The Maze Runner playing on three screens, featuring special content for the side screens. It was really cool, but it wasn’t so cool that side-screen content was only available for seven minutes. While that seven minutes was quite impressive from a visual perspective, the content was suddenly there and then suddenly not there from an audience perspective.”
Barco, however, closed down the Escape initiative last year. LLamb suggests one reason for that was the notion of creating, mastering, displaying, and distributing additional, unique content as a separate DCI package on top of the film itself was simply not practical for studios from a business point of view.
Korea's Screen X, which also offers expanded side screens for what the company calls “more creative possibilities,” is taking a different approach, but with a key difference, according to LLamb.
“In certain territories [particularly in Asia], Screen X is getting popular,” he says. “Instead of two digital cinema projectors on each side, which can be expensive, they use a series of smaller projectors on each side. But the big difference is the content. Rather than having the studio or content owner produce the content for those additional screens, Screen X creates the content themselves. Studios send their base DCP over to them, and they essentially fill in the gaps, so that you can have additional content for most, if not all of the movie.”
And, LLamb adds, there are other concepts out there for further “placing people inside a movie,” as he puts it.
“A Canadian company called D-BOX produces motion systems and motion-controlled seats for cinemas,” he says. “And there are other interesting things, such as systems produced by South Korea’s 4DX and MediaMation’s MX-4D in the U.S., which offer more of an amusement park type situation with seat motion, wind, flashing lights, and certain scents to augment the immersive feeling.”
However, although all these approaches are currently in play across the cinematic landscape, what will or won’t spark consumer interest in the long run to the point of making people want to spend more to go to the theater, filling more seats, is a harder question to answer. For one thing, LLamb suggests a big challenge for the industry is the fact that the cost and resources of producing high-value content for such presentations may not always be worth it in some instances.
This can be summed up by what LLamb calls the phenomenon of “version explosion,” which refers to the ongoing requirement placed on studios to continually master more and different versions of their movies to keep up. In a multi-platform world, this is already a challenge, even before factoring in the tantalizing possibilities of immersive versions.
“When ATMOS came along, it targeted creatives first, to get their appetites wet and they did some amazing things,” he says. “But that meant the studio had to create another version. That sounds simple enough, but if you tell a studio they need to create another version of a movie because there is a new sound mix, that means they need to spend money mixing it. They need to do the mastering and output, and then somebody needs to build another DCP for it. Now, instead of having a single traditional 5.1 or 7.1 surround-sound release, you have a 5.1 and a 7.1 and an ATMOS release. And by the way, each of these may or may not be close-captioned, or have audio descriptions for hearing impaired or visual descriptions for the visually impaired, which increases the release count exponentially.
“And so, this brings us into what everyone is calling version explosion. After ATMOS's re DTS came along. So then you had to do a fourth version. Maybe what was originally taking up 12 versions for a typical domestic release, now you have to do 14 versions, or 16 or 18, to get a single domestic release package together. Next, you add something like EDR or Dolby Vision, increasing the number of versions. At that point, you are also doing EDR 5.1, EDR 7.1, EDR ATMOS. You can go down the pipeline and ask yourself how many different versions? So now you might be going to 20 or even 30 versions.”
And then, he elaborates, “you have the flip side where you have to find a theatrical environment that can play the material. You obviously can’t send an immersive audio package to a non-immersive audio theater. You can’t play ATMOS DCP in a 5.1 auditorium or a Dolby Vision DCP in a normal, standard SD Xenon projector setup. That can’t happen any more than you can play 3D in a 2D auditorium.”
Therefore, to move onward to multiple screens creates even more time, money, and logistical issues to manufacture and distribute content. All of which is why “SMPTE came into play and, on request from the studios through DCI, did the immersive audio standard,” he explains. Now, LLamb predicts, the same situation will likely occur with HDR and EDR cinematic content until standards are achieved in that arena, as well.
“We have several companies working on different solutions for higher dynamic range,” he says. “Whatever that means—we haven’t yet fully defined the term, and that's part of the problem. So we may be in the same situation we were with immersive audio soon, and then the same thing with multiple screens. If two or three companies start having a lot of success with it, then obviously there will need to be one standardized format so that everybody can do it the same way.”
LLamb points out that, for now, the American Society of Cinematographers and the Academy of Motion Picture Arts and Sciences are among the entities taking the lead in exploring what HDR should or could be for cinematic presentations, and he expects in a year or two, some clarity will come to that quest.
The nature and difficulties of this process, however, raises the question of context—when is immersive content worth it, or not worth it, in terms of luring consumers to movie theaters?
“Do we need all this [immersive content]?” LLamb asks. “Well, we kind of did with 3D. Did 3D sell more seats? Yes, it did quite a lot, at least for a while. But would a 4K feature film sell more seats than a 2K version? Probably not. Would an HDR version sell more seats? Possibly, especially now that HDR is more in the consciousness of consumers in the home-entertainment space. Has immersive audio sold more seats? Probably, but now it is usually paired with HDR or EDR. So will more screens sell more seats? I don’t know; they might if the content is good enough. It’s simply far too early to answer these questions.
“It’s a Wild Wild West in that regard. And I think that’s a good thing because now that home entertainment is moving so fast, theatrical environments and content producers will need to be more aware. They are used to having that space entirely to themselves. If they want to grab people’s attention, especially people under 30, they will have to be more light and nimble.”
Immersive Sound for the Recording Industry
A recent Hollywood Reporter podcast interview with three-time Oscar-winning re-recording mixer Chris Jenkins examines, among other things, the issue of how immersive sound is impacting the recording industry. Host Carolyn Giardina spoke to Jenkins, also executive VP at the Universal Music Group, about his work mixing Ron Howard’s new documentary Pavarotti, which was mixed at London’s Abbey Road Studios in immersive Dolby ATMOS to bring life to reams of archival material of the legendary opera singer performing. During the interview, Jenkins points out that while immersive sound generally has typically been considered “a high-end format” for cinematic use, the recording industry, as he did with the Pavarotti project, is greatly interested in “democratizing it” for the recording industry. Jenkins says he expects new ATMOS-supported consumer sound technology to soon appear on the market, thus making it possible under certain circumstances for consumers to enjoy fully rich ATMOS mixes in their own homes.
A recent report in Wired suggests database provider MongoDB has made progress in tackling the issue of data breaches on behalf of large companies. The report says MongoDB is developing a new approach to this problem based on the notion of “radical simplicity” when it comes to encrypting databases, meaning the company is turning to the concept of Field Level Encryption to protect client data. The concept is not new—it means data travels encrypted end-to-end, becoming visible only to the sender and the recipient. But in the case of what the article calls “client-side encryption,” such databases would require a system login, specific key usage to process and decrypt specific pieces of data locally by the end user, and more, meaning even MongoDG and its employees and administrators could not access it. This approach does not address the issue of being able to access data using stolen end-user credentials. But it is designed, the article says, “to eliminate single points of failure,” meaning that people who steal administrative credentials or hack their way through software vulnerabilities to gain access still won’t be able to read encrypted data.