Hot Button Discussion
HFR's Next Steps
By Michael Goldman
Among the many digital cinema standardization processes currently in various stages of development, none is more dependent on input and interaction with the creative community than the issue of higher frame rates (HFR). That is especially true these days, as the industry's examination of how, where, and when HFR might, could, or should be woven into the digital cinema fabric on a formal basis is largely reliant on what it learns from ongoing experimental work by the creative community as part of a strategy to analyze HFR's technical impact on workflow and distribution, and its creative impact on viewers.
SMPTE Fellow John Hurst, current chair of SMPTE's Digital Cinema Technology Committee (TC-21DC), points out that SMPTE officially disbanded its Study Group on High Frame Rates for 3D and 2D D-Cinema Applications at the end of 2013. The reason was that the Study Group, co-chaired by cinematographers David Stump, ASC; Kommer Kleijn, SBC; and industry consultant Michael Karagosian, had completed the preliminary work it was asked to do. This included surveys of current industry practice in the HFR realm and the launch of a formal strategy for creation of test content that could be analyzed to answer many of the industry's questions about issues surrounding HFR imagery.
Actual supervision over creation and analysis of official test material has now been officially ported over to a new group under the auspices of the Motion Picture Academy's Technology and Science Council, a working group called the Next Generation Cinema Technology Working Group on Test Materials. In recent months, that group has planned a maximum of four specific HFR shoots, using different technologies in varying conditions so that it can then evaluate the resulting footage.
According to Stump, the first of those four shoots recently took place, with Stump serving as cinematographer during a production at the AMPAS Science and Technology Council headquarters at the historic Pickford Center in Hollywood.
"We used several different camera configurations in both 2K and 4K, and 'standard' and 'high' dynamic range modes," Stump relates. "We ran a dolly back and forth that looked out windows into pure daylight, and then dollied across an interior to the far end of a room, and back to the beginning again, making sure we had bright highlights outside the windows. We did it with a wide variety of frame rates, ranging from 24 frames/sec to 120 and a wide range of shutter angles, from 240 all the way to a 359° shutter."
That material is currently being processed at a variety of facilities around Hollywood. Stump says the working group will then scrutinize it over time, using what it learns from the first shoot to influence succeeding shoots, although no timeline for them has yet been developed. As Hurst explains, one of the factors they will be studying--a central concern in all this work--is the issue of compression in the distribution of HFR imagery.
"In particular, the [SMPTE] Study Group originally became concerned with the question of what an appropriate compression bit rate would be for high frame rate content," Hurst explains. "There is pretty much universal acceptance that the current DCI rate of 250 Mbits/sec is insufficient for HFR work. But the hardware sold today has limitations to it in this regard, so we are all interested in what bit rate is demanded by the entropy rate in the actual pictures. They determined there was no way to know that without intense analysis of test content. So that part of the discussion has moved to a new venue--the definition and production of the test content for higher frame rate imagery and also higher dynamic range imagery, since those two conversations can often get tangled up with each other. So that is under way, with the conversation having moved over to the Academy."
As that work is ongoing, Hurst emphasizes, there is currently no need for a timeline for results to come back to SMPTE for work to begin on particular HFR standards for 2D and 3D exhibition, nor should there be. After all, he points out, "there is still way too much experimentation that needs to go on for us to talk about standards right now. We would need to reduce the number of moving parts a bit before trying to write anything down."
By "moving parts," among other things, Hurst is referring to the fact that the Next Generation Working Group's current efforts are not the only work currently going on with HFR imagery. As was discussed in the September 2012 SMPTE Newswatch, filmmakers like Peter Jackson and James Cameron are already making major studio features this way and releasing them in specially configured theaters. Other filmmakers are using a wide range of technologies to experiment with the creative potential of HFR in numerous ways, long before the industry reaches any conclusions related to standards. Those projects will also have to be evaluated as part of the ongoing exploration of where HFR should properly head, in addition to the official work being done by SMPTE and the Academy.
Hurst emphasizes that any creative testing period of any length, by its very nature, means that those participating will come to the process "with a lot of different agendas. Some people who design compression algorithms will come concerned only with the complexity of the images, and how to approach that with their systems. Others will look at it entirely from an aesthetic point of view and how best to incorporate aspects of it into the storytelling process, among other things."
A host of related issues, of course, will come along with the process for certain segments of the industry. Post-production professionals, for example, will eventually need to know when there will be a way to achieve best practices in terms of mastering native HFR content for distribution to the reams of different platforms that now exist.
"For now, it is definitely a science project to master all this material for different platforms," Hurst says. "I assume, though, like all else, some conventional practices will emerge, and that is one of the agendas that people will bring to the content testing process. The question is not so much about, will a common practice be distilled eventually, but rather, how long will we be in chaos before they can figure it out? We might well be a decade or more into it before some rote practice comes out of it, when you think about how complex that situation is."
Then, there is the more general business issue of balancing the potential of the creative possibilities for HFR exhibition with the nuts-and-bolts costs of retrofitting cinemas to show such content, both during the current interim period where only a small group of filmmakers are forging ahead on major films, and again when standards and protocols begin to appear. As discussed in SMPTE Newswatch in September 2012, this is a dichotomous issue in the sense that two things are simultaneously true. First, technology's advance and the eventual requirements are not so onerous as to make this goal impossible or not worthwhile to achieve when the time comes, as compared to the industry's lengthy and difficult transition to digital cinema. But secondly, the business requirement to make the change to lure consumers, is not so pressing that the industry feels any particular rush to make a major switch on any specific timetable, at least for the time being, particularly when the jury is still out on how, exactly, the creative possibilities will enhance business opportunities to begin with.
Transitioning cinemas to showing HFR content, after all, will put new kinds of demands not only on cinema engineers and owners, but also on hardware manufacturers. Server and storage systems will have greater demands placed upon them, manufacturers will face greater requirements to upgrade software, test systems, and engineer solutions than ever before. Such systems will increasingly need to be interoperable to be valuable to their customers.
Those "moving parts" will take a long time to shake out, Hurst suggests. Still, none of this will be nearly as radical as the industry's last fundamental transition away from film exhibition and into the digital realm, obviously. But then again, Hurst points out, business and creative considerations will also have to transition, not just the technology.
"In the long run, nothing radical would need to change as cinemas transition, because the way that we make and distribute digital cinema packages is durable," Hurst explains. "The formulation would change a bit, but nothing like the changes we went through as we transitioned from film to digital. But I do think the business motivations for this are very interesting. Right now, only a small number of filmmakers are already looking at the increased reality of [HFR] as a filmmaking tool. A larger number of studio and exhibitor types are looking at it as a way to differentiate a new exhibition experience from other forms of entertainment, so there is a lot of gravitational pull to do this from the cinema side, almost more so than from the creative side. Therefore, as it starts to happen, a large number of filmmakers will need to go through a different experience as they learn a process for figuring out how to integrate higher frame rates into storytelling on a trial-and-error basis. And, to be sure, they won't all be gems. But those that do it right will produce something pretty special. Anytime that you introduce something new like this, you have to go through a long creative period where people have to first figure out what they can do or what they can get out of the process, and that just takes time."
Thus, Hurst adds, don't expect any rush to abandon 24 frames/sec, arbitrary though the creation of what eventually became, for over a century, the industry standard frame rate. Even more lifelike imagery can now be created by skilled filmmakers using higher frame rates. He anticipates that HFR is likely to remain a niche offering to consumers for quite some time.
"24 frames/sec is a stable baseline and will likely continue to be widely used for the bulk of the work that we see for a long time to come, even as we see new things coming along like HFR and other things, such as immersive sound and other advanced picture formats," he says. "I tend to think of these as special venue-type of distributions--not for the majority of sites. Sites that choose to equip themselves to exhibit these formats are doing so with a specific business goal in mind beyond simple cinema exhibition. They are trying to differentiate themselves from a competitor like another cinema, or a horizontal competitor in some other form of entertainment. The cost, because of the churn, will always be slightly higher than standard 24 frame/sec content which, because of its economy, will continue to rule for the foreseeable future."
IMAX Digital 4K 3D
There was a lot of buzz percolating around the digital cinema world early this summer when IMAX announced it had not only developed what it is claiming is the world's first fully integrated 4K (large format) 3D digital camera system, but that the system had already been in use recently by director Michael Bay's team to capture certain action sequences for his new release, Transformers: Age of Extinction. The IMAX announcement featured images and a short video feature showing the new IMAX 3D Digital Camera in action on the set of Bay's movie, and promoted the notion that the immersive IMAX large-screen experience that has become so famous through the company's historic 65mm film capture process could now be emulated with the new digital system capable of capture at full 4K resolution at an IMAX presentation aspect ratio of 1.9:1. A host of blogs followed the announcement, including some early analysis from Bryant Frazier at the Studio Daily site. Frazier reported that IMAX officials told him that among the system's engineering accomplishments was its design for placing the stereo lenses into the single housing, allowing for a smaller camera without a bulky rig, and making it a useful choice for the kind of action work for which Bay is known. Frazier also wonders whether true IMAX aficionados will be satisfied with seeing the format captured digitally rather than on 65mm film. But he points out that IMAX is claiming the company will continue to manufacture its 15-perf, 65mm film cameras as long as leading filmmakers are demanding that format from the company.
Atmos At Home
With the rapid evolution of ultrahigh-definition television, the corresponding need for an improved audio experience in the home that pushes past 5.1 surround sound to match the 4K viewing experience and emulate the immersive audio experiences now available in movie theaters has been the subject of much industry discussion. So-called 3D audio systems from manufacturers around the world were in various states of development as was discussed in the October 2013 SMPTE Newswatch, with multiple schemes proposed, ranging from 10.2 discreet channels to 22.2 multichannels, and more. But in June, Dolby, which has led the way in immersive cinema sound with its Atmos system, announced on the company's Lab Notes Blog, written by Brett Crockett, Dolby Director of Sound Research, that it is now attempting to bring the Atmos concept into home theaters. Crockett did not get technically specific in his post, but he suggested that the company is creating speaker module technology to complement standard home speaker systems that can essentially trick the ear and brain into hearing overhead sound even when your configuration does not permit overhead speakers. The approach, Crockett suggested, "comes down to understanding the physics of sound waves and understanding the way your brain interprets those sound waves." He added that Dolby is partnering with home theater receiver manufacturers and other hardware companies to develop technology to decode and deliver Atmos signals in home theater systems. No specifics on a timeline for the new product, beyond a promise that the company will be offering more details "over the next few months."
The post-production and technology communities are still analyzing the recent announcement that SMPTE and the Hollywood Post Alliance (HPA) have begun a process designed to eventually merge the two organizations in May of 2015. The announcement, made jointly on 19 June, stated that a partnership has been struck that will launch with SMPTE providing administrative support to HPA immediately and culminate with next year's merger. Officials from both organizations stated the move will eventually expand the influence of the engineering and technical communities by bringing them into closer collaborative contact with the creative community that relies so heavily on their work. Click here to read the SMPTE announcement, and click here for HPA President Leon Silverman's statement to the HPA membership.