Hot Button Discussion
Approaching an Immersive Audio Standard
By Michael Goldman
While precise technical and timeline details about a single, internationally accepted standard for an interoperable, immersive audio format for digital cinema remain to be settled, one important aspect of this quest has crystallized in the past year or so, according to Peter Ludé, chair of the SMPTE Working Group on Immersive Sound (operating under the auspices of the SMPTE TC-25CSS Audio Technology Committee on Cinema Sound Systems). Ludé says the big breakthrough in the past year has not been technical in nature. Rather, it has been the fact that virtually everyone with a stake in the conversation has gotten publicly and enthusiastically on board with the industry's need for a single specification for the packaging, distribution, and theatrical playback of D-Cinema-based audio tracks that pushes past what was initially described in the original DCI (Digital Cinema Initiative) specification. Further, he suggests that key players across the industry now clearly understand that such a specification must build into its foundation the growing trend toward object-based audio playback methodologies, meaning audio tracks that include signals that carry their own metadata specifying a dynamic location for the sound to be played, along with other characteristics. This is in contrast to relying solely on the traditional, channel-based methodology of describing sound scenes using a set of audio signals that are intended for playback through specific loudspeaker configurations.
Ludé says that content creators in the form of Hollywood studios via the Digital Cinema Initiatives Consortium, and exhibitors, through their trade group, the National Association of Theater Owners (NATO), have now formally called for such a standard, in addition to the individual support the working group is getting from various manufacturers and different types of anecdotal data the industry has been producing on the benefits of immersive sound in the creative process.
"The manufacturers have long expressed strong willingness, interest, and enthusiasm about supporting a new standard," Ludé says. "They just needed a standard to support, and that has been our job. In the last year, what has been really pivotal is that the DCI--the six major studios--have made a public statement, that they strongly support an immersive sound format, and that they want it as quickly as possible. Having it, of course, will allow them to create single inventory DCP packages for their movies. In other words, the same version of the same movie can go out to everybody to enjoy the immersive sound, and that can only be done if we have a standard. They are already mixing the vast majority of their tentpole titles, if not all of them, in both the Dolby Atmos and Auro 3D formats, and they wouldn't be going to that trouble if they didn't see the benefit of immersive sound. Their statement made it very clear, a single standard is needed to simplify this need, and that was an important development."
"And then, in parallel with that, the National Association of Theater Owners issued their own public statement, in parternship with UNIC's Technology Committee, also saying we need a common international immersive format so that [theater owners] don't have to deal with so many different versions. Since different theaters have different systems with technology from different manufacturers, they want a solution that allows them to receive one copy of each movie and know they can play it in any of their auditoriums, because everything will be interoperable. So everyone has expressed strong interest in making this happen."
In fact, creative uses for immersive sound have greatly evolved in the past few years. Dolby has been working to analyze this trend ever since it launched the Atmos system into the marketplace. That company has been collecting statistics on what it calls "use patterns" for content mixed in the Atmos format, according to Charles Robinson, who heads up cinema sound technology research for Dolby Laboratories.
Robinson says the company has been soliciting input from the content creation community since it started developing the Atmos system to find out "what they most want and like about an immersive audio system." But Dolby has also been strategically analyzing finalized Atmos soundtracks to figure out when and how sound mixers have been using immersive capabilities, and when they have not been using them. A key point of this ongoing analysis, Robinson says, is to try and figure out when and under what conditions the addition of immersive capabilities to a feature film soundtrack can enhance its creative possibilities without bogging down the project's workflow in highly data-intensive environments.
Dolby has not formally published these findings thus far, he says, because the analysis is ongoing, but Robinson did tell Newswatch about some preliminary observations. Among them was the fact that sound mixing professionals are currently approaching immersive sound in a synergistic manner. They like and are actively using audio object capabilities, but are also continuing to think and work using a channel-based methodology for the time being.
"We are finding that mixers continue to specify much of the mix using audio channels," Robinson says. "In 12 out of 12 soundtracks that we analyzed, more sound energy was carried by channels than audio objects. That suggested to us that, for the foreseeable future, immersive audio should continue to support mixing to channels, and audio objects and channels must co-exist and behave in a consistent manner. In particular, the specification of object and channel location must share a common frame of reference."
A second use pattern, Robinson says, for the time being, is the notion of "virtual loudspeakers"--essentially, stationary audio objects always placed in so-called "preferred locations for static objects" such as, for example, along the left and right side walls, toward the front of a theater room.
These kinds of creative considerations point to an industry transitioning its approach and, therefore, a need for flexibility in any standard established. Beyond the creative need for flexibility, as Ludé notes, is the technical need for flexibility, or more accurately, interoperability. In fact, interoperability is at the center of the entire initiative, giving content creators the ability to insert into DCP packages audio data that can be played back in all different immersive systems and cinemas around the world.
The TC-25CSS group's previous study of all available requirements and technologies on immersive sound, led the group to focus most heavily on official proposals presented by Dolby and the MDA Cinema Proponents Group, a consortium that includes DTS, Doremi Laboratories, Ultra-Stereo Laboratories, QSC, Barco, and Auro Technologies. Dolby and MDA have both made proposals based on object-based systems, with Dolby's Atmos being the most prevalent such system currently installed in theaters around the world. The Auro 3D immersive system utilizes a height channel-based methodology, but Ludé suggests it has the capability to support object-based audio, as well. Both potentially contain elements useful for a standard in a world in which object-based mixing and channel-based mixing will likely co-exist for quite a while. Additionally, he points out that SMPTE follows "a due process standards' methodology," which means that any and all interested experts, groups, and manufacturers have a chance to weigh in. That process could take several more months after the working group draft is completed to take into consideration any other approaches or technologies before a standard can be finalized.
Still, while saying that that the development of an exact standard timeline can be "unpredictable," Ludé says so much progress has been made in recent months that he is hopeful a standard will be ready or close to ready for publishing before late 2015.
"We have been meeting frequently since the working group was formed last December," Ludé says. "On the other hand, this is a due process, so we want to make sure everyone's voice is heard. So far, no big concerns have come up, but it is a very complex process so, at some point, I expect some engineering debates will ensue. What we hope is that the working group will be able to submit a draft recommendation some time in early 2015, possibly even by the end of this year, and from there, it has to go through the process of collecting inputs and comments, to make sure all comments are thoroughly addressed. That can take a few months, but we hope to have the meat of the work done no later than early 2015, so we would expect that, by the end of 2015, or maybe even a bit earlier, to have the work done."
But even once a formal standard is achieved, there will be the matter of converting an entire industry over to it on a global scale on multiple fronts--the hardware manufacturer's side, the post-production side, and the exhibition side. For the post-production industry, Ludé emphasizes any hardware or software upgrades will, at the end of the day, be far less expensive than the current world "of having to mix things in several different formats, often starting from scratch. If they had their way, they want to create one mix, rather than having to go through all those different mixes."
"And for the cinema owners, for the most part, it will involve software upgrades to their existing infrastructures. We don't know what the actual, final standard will be yet, of course, because the work is still in progress. But the expectation is that it will be a relatively minor change from [the technology] that is in the field today. I think, for the most part, people are envisioning relatively minor software upgrades to get them from their current status to the new standard when the time comes. Everyone feels it is worth the time and expense to get to a single interoperable immersive format; that is the main reason this process is moving forward."
The other interesting issue is how the new standard will eventually impact the broadcast world and consumer technologies generally. The working group's efforts, of course, are focused strictly on cinema applications, but Ludé points out that many home entertainment technologies employ innovations that began life in the cinema. He suggests the concept of immersive sound will be no different, at least where home theater applications are concerned. Indeed, the concept is already in development, as a Dolby blog posting reported in Newswatch last month indicated. (Here is a followup FAQ on those plans from Dolby's director of sound research, Brett Crockett.)
"Obviously, theatrical releases have windows, and after those windows, they have home distribution, video on demand, all those other avenues," Ludé states. "Everybody understands that the less work one has to do to re-master from theatrical to those other formats, the better. And it is no secret that two of the key proponents in this initiative--Dolby and DTS--certainly have a presence in the consumer audio world, as well. I think there is some expectation that once a theatrical standard is put into place, there would be some relevance to be able to draw from that standard to support the home entertainment world. It would be something of a logical extension, and there are many examples of technologies that started out in cinema, ranging from sound itself, to color, widescreen, 3D, stereo sound; all those things were first developed for the cinema. This enhancement will likely be no different, but exactly how and when and by whom will be difficult to predict."
As the topic of the consequences of the arrival of flying unmanned vehicles--frequently being called "drones" these days--continue to be debated in military, business, government, and regulatory circles, their potential for moving image capture is becoming more exciting each day. Newswatch reported in a May 2014 news item, methodologies and guidelines for their use in broadcast journalism are becoming a major topic of conversation, and now, in a thoughtful column for the American Society of Cinematographers' (ASC) website, cinematographer John Bailey, ASC, offers an interesting perspective on how drones could be used to advance the art of cinematography from a creative point of view. Bailey discusses aerial shots from a camera drone for a film he recently worked on, and marvels at the expert use of a single HD camera on a drone in director Nabil Ayouch's Moroccan film, Horses of God. Bailey points out a great irony that the current debate over regulatory and safety constraints for drone technology could stunt the creative exploitation of this new tool by serious filmmakers, while much of the concern about them revolves around their potential for misuse by those who have no interest in filmmaking. Drones, therefore, are the first "promising technology adapted for filmmaking being put under such potentially devastating scrutiny" that Bailey can recall. Yet, he concedes, safety considerations are very real. Thus, he suggests "a real urgency" be put into the scrutiny process by those who see the technology "as a tool to expand cinematic style, and not just as a hip delivery system for Amazon."
SIGGRAPH Technical Papers
With the SIGGRAPH computer graphics show coming up 10-14 August in Vancouver, Canada, the annual previews of the SIGGRAPH 2014 Technical Papers program are under way, with some scholarly work that SIGGRAPH officials are confident will impact the world of computer graphics. In the press release announcing the 2014 Technical Papers, Adam Finkelstein, SIGGRAPH's 2014 Technical Papers Chair, said that, in addition to the traditional areas of modeling, rendering, animation, and imaging, some of the 127 technical papers accepted this year (out of 550 submitted) included work in "two emerging fields"--in the areas of fabrication-oriented design and learning. Links to abstracts of all the 2014 papers are available on the Web. Also check out the official SIGGRAPH 2014 Technical Papers preview trailer on YouTube.
HPA Hosts SMPTE 2014 Symposium
As one of the first manifestations of the recent initiative regarding plans to merge the Hollywood Post Alliance (HPA) and SMPTE by mid-2015, HPA will host a symposium leading into the SMPTE 2014 Annual Technical Conference and Exhibition in October. The daylong symposium, titled "Making Do with More," will take place at the historic El Capitan Theater in Hollywood on Monday, 20 October, and will feature sessions focusing on next-generation workflow concerns for the post-production community as the industry moves toward greater enhancements with picture and sound, ranging from higher resolution, contrast, and frame rates, to more sound objects, and much more. The event is one of the first joint efforts between SMPTE and HPA since their new partnership was announced. You can read more about it in the late July press release.