Hot Button Discussion
The HDR Ecosystem
By Michael Goldman
While many image quality improvements are easily definable and explainable as the digital broadcasting world pushes into the ultra-high-definition (UHDTV) era, the question of high dynamic range (HDR) is less well-defined. At this year's NAB show, HDR was a hot topic certainly, and both professional and consumer display manufacturers are now vigorously promoting different ways in which they are improving the quality of their pixel offerings. But according to industry veteran Scott Daly, principal member of the technical staff at Dolby Laboratories, the concept of how best to create, store, transmit, and view content featuring higher dynamic range characteristics is evolving more into an industry challenge revolving around how to develop and maintain "a proper ecosystem" for HDR image viewing, rather than a question of what is the best display technology or the correct standard to maintain over the long haul.
The phrase "high dynamic range," after all, invokes a wide range of techniques in image creation, processing, and reproduction to improve luminosity generally. Various techniques, indeed, are required, to properly develop this HDR ecosystem from image capture or creation to display over time, Daly suggests.
"We developed several higher dynamic range displays over the years, as well as the related image processing, but we eventually realized that we needed a better signal for those displays [from broadcasters and studios]," Daly says. "We learned that if you make a higher dynamic range display and want to develop a market for it, the best approach is to work with studios to develop that better signal in a new format--signals that actually match the new high dynamic range displays. But then, one may ask, don't we need high dynamic range image capture for the content to begin with? Of course, a lot of companies are working on that--Sony and Arri, for example, and they have been steadily increasing dynamic range in their cameras in recent years. Their top-line cameras already feature higher dynamic range capture capabilities than most currently existing televisions are capable of showing, in fact. But for much content currently being produced, we don't have to wait for the cameras, because much of what is being produced involves computer graphics and image compositing. We can, of course, improve dynamic range in post using image compositing techniques by using layers that have just standard dynamic ranges, but also different exposures and rendered levels. Computer graphics is already in an unlimited dynamic range, depending on the system. A third way is to take imagery with standard range and color gamuts, and just push out larger luminance ranges in the color-grading step while using HDR reference monitors, as well as pushing colors outward to a wider gamut."
"Of course, when you expand the dynamic range of luminance, you also need to expand bit depth, as well. And that work is likewise going on to determine how to best allocate the bits. The technology is currently there to pretty much do the full pipeline. What is being hammered out right now are the standards and interfaces for people to be able to do this, but the basic ability to do the pipeline--the full ecosystem--already exists."
Some of Daly's own work has been involved with Dolby's development of the Dolby Vision suite of technologies the company has been showing in recent months, aimed at offering viewers "better pixels" in consumer displays with brighter highlights and darker shadows, among other improvements. That technology is now being incorporated into various new consumer display systems that tout improved HDR capabilities from manufacturers like Sharp, TCL, and Vizio, among others, some of which were demonstrated most recently at CES and NAB. Others in the industry are pursuing their own initiatives in this regard, including Philips, the BBC, and Technicolor, which proposed its own HDR encoding scheme earlier this year. Technicolor showed demos during the Consumer Electronics Show and is also planning to license such technology to display manufacturers in the near future.
Indeed, as The Hollywood Reporter recently reported, such initiatives are showing such promise across the industry that the International Telecommunication Union (ITU) is now in the process of examining how best to incorporate various HDR strategies for consumer displays into the wider plan for implementing the UHDTV initiative across the world in coming years.
Much of this HDR traction is happening based on continual technology improvements. But, Daly adds, such work also faces a harder-to-quantify hurdle as it moves forward: industry researchers must contemplate not only what they canoffer in terms of HDR improvements, but also what they should offer. In other words, what are the limits of human viewing capabilities, and more importantly, what are the preferences of different kinds of viewers--ranging from casual viewers to early technology adopters to expert viewers who take image quality very seriously, and so on? What kind of improvements is the human eye even capable of picking up and processing through the brain at the end of the day?
These are important questions, Daly suggests, because they are directly connected not only to what kind of technology and standards work the industry needs to pursue, but also to what kind of creative work artists could be doing with the resulting technology being made available to them on the larger landscape.
"On the display technology side, the technology is a moving target in that we constantly can improve things, but our ability to improve the experience can, at a certain point, bump into the limits of human perception where it might make no sense to go any further," Daly says. "And that can be a complicated subject. This is part of the debate over 4K, for instance. Some people say that with 4K, you can't see any difference, but with certain test targets and text, you clearly can see a difference. But then, when you get to 8K, with a 32-in. display, you might have to sit impractically close to see a difference. So that is an example of where display technology might outpace perception."
"However, at the same time, the reason for making a 4K or 8K panel is not just about increasing resolution. It is also about creating a wider field of view--a bigger display. Or, in the case of virtual reality, a smaller display. It is all based on certain assumptions of viewing in that sense. There are similar arguments about dynamic range and color gamut and bit depth. All of them, you can keep improving, but you might run into perceptual limits where it is hard to see any difference if you go too far in some situations."
That reality brings the conversation back to the ecosystem issue, Daly says. "With better displays, better signals, and new content creation approaches, such as new color grading procedures--imagery seen on those new displays can turn out to be clearly visibly improved and downright stunning. The whole thing has to be managed through a long and winding food chain, however, for the improvements to have significant meaning to the viewer."
If that is done properly--if the ecosystem is properly managed--Daly insists that industry research strongly suggests that higher dynamic range clearly does impact and improve the viewing experience and, in fact, has the potential to alter the creative experience altogether. He points to both demos that have been going on in recent months across the industry, conducted by various players like Dolby and Technicolor, as well as several studies that he says conclude viewers clearly prefer higher dynamic range when it is offered to them in the correct way.
"Dolby has offered some demos with content made by professional cinematographers showing higher dynamic range and wider gamut, shown on professional HDR displays that can go up to 4,000 nits," he says (read a recent article on some of those presentations). "What we think these show is that it is not just about images that are brighter. It is also about showing very low black levels. That is equally important. With bright images on one end and lower blacks on the other and a wider gamut, we can exceed the DCI P3 cinema gamut in terms of color volume. There are two important aspects of higher dynamic range: entropy and sensation. Entropy is about the detail that you can see, and sensation relates to the actual appearance of the luminances themselves. If you have sunlight reflecting off water and can have the luminances of both matching, it can potentially give the viewer the strong, visceral feeling of being in the real world, for instance."
Daly says that Dolby, the European Broadcasting Union (EBU), the École Polytechnique Fédérale de Lausanne (EPFL) University of Switzerland, and others have been conducting studies in recent years that he insists all point to similar conclusions--people watching material with professionally controlled and managed HDR imagery prefer them consistently. The 2014 NAB Show's Technology Summit on Cinema references the EBU and EPFL studies.
"Quite simply, this research suggests that consumers do see and appreciate the extra range that we can give them in terms of both luminance range and color gamut range," Daly declares.
If these conclusions are accurate and the ecosystem that can produce the kind of imagery that can lead to such conclusions can continue to grow and evolve, then "this opens up the palette for artists--the directors and cinematographers and others. Then, it offers them new opportunities for creativity," Daly suggests.
"If they don't know what they are doing, they could use it in a distracting manner, whereby it fights storytelling. But seasoned artists with good eyes will use it to support the story, or to take stories further than they were previously able to do."
Still, technology will march on and improve most viewing experiences to one degree or another, Daly points out. For consumer viewing, backlighting improvements alone, such as breakthroughs with things like Quantum Dot technology, discussed in the April 2014 SMPTE Newswatch will help make significant HDR strides, as will the arrival of laser projection technology in cinemas, discussed in the March 2014 Newswatch. But the perceptual debate will likewise continue to rage and make it difficult to standardize the HDR question any time soon, although issues related to perception of dynamic range imagery are among the topics currently being studied in various projects being run by SMPTE's Essence Technology Committee (TC-10E).
The question of how much to future-proof any standard alone is an ongoing debate, Daly says.
"If we really understand the [perceptual] range that makes sense for entertainment viewing, and we think the technology to achieve that may be within a 10 to 15-year time frame, then you could say we should standardize signals around that range and make everything future-proof to that," he states. "Such a standard would give headroom for the future. Then if the technology could actually exceed that in or beyond 15 years, we won't need it anyway since such a standard would be so close to the perceptual limits already. Another approach would be more short-term, to just say this is what displays can do today, so lets standardize around that. However, since we know visual system preferences are beyond what displays can do today, that would be short-sighted, since we know displays can and will get better, and we would likely have to change the standards again. I'm not sure where they will end up going with that, but it is a hot topic right now."
Open-Standard Cinema Surround Sound
As immersive sound systems for digital cinema continue to grow in importance, the industry has been hard at work pushing to develop an open-format, object-based playback standard for immersive sound in movie theaters. This process was recently explained in great detail in an article on the Editor's Guild website, penned by Mel Lambert. He discussed the cooperation between a coalition of studios and SMPTE to develop an open surround sound cinema format. The article explains that the work being done by SMPTE's special TC-25CSS working group is aiming to find an efficient method for delivering immersive audio to digital cinema playback systems in theaters, regardless of whose technology is being used in those theaters, and in what configuration. Lambert states that Dolby Laboratories has contributed one proposal built around its ATMOS immersive cinema audio system. MDA Cinema Proponents Group, which includes DTS, Doremi Laboratories, Ultra-Stereo Laboratories,QSC, Barco, and Auro Technologies, has contributed another. MDA is based on Multi-Dimensional Audio. The article discusses both proposals, but notes that the SMPTE working group is not looking to choose one technology over another. Rather, it is aiming to understand all approaches, and choose any or all elements it believes are necessary to properly build a single interoperable standard for immersive audio soundtrack delivery.
As certain exhibits at NAB showed this year, the use of flying unmanned vehicles to capture video imagery for use in broadcast journalism is a trend quickly gathering steam. With such new trends, however, new issues arise, and a recent article from TV Technology discusses one of them--the fact that there are not proper guidelines for regulating flying unmanned aerial vehicles (UAV's) being used by broadcasters. The article, written by Michael Grotticelli, points out that small, remotely controlled drones outfitted with miniature HD cameras are cheaper than news helicopters, and are capable of capturing extremely high-quality video imagery, including content that would be difficult or impossible to capture any other way, in some situations. He says that there is already a Professional Society of Drone Journalists (PSDJ) in existence, and the UAV technology already available--much of which was on display at NAB is quite impressive, including UAV's that operate on battery power and solar power, among other innovations. At the same time, Grotticelli points out, the Federal Aviation Administration (FAA) currently has no guidelines for operating broadcast drones in populated areas, but does have rules prohibiting using them for commercial use. There are also privacy concerns and the stigma of how similar technology is being used in military applications. Still, Facebook, Google, ABC, and many other entities are currently developing and experimenting with such technology, and how to make drones easier to use. The article suggests that, eventually, such technology will become a billion-dollar business with huge broadcasting implications.
HTML5 for Interactive Video
An interesting column on the Technorati website suggests that the flexibility and power of the HTML5 Web markup and content presentation language has great potential to be a boon for creating, manipulating, and using interactive video content for marketing and promoting products in a way that is fairly simple to create. The article states that a new company called Rapt Media has found ways to use HTML5 to allow companies to insert the kind of interactive and social media elements that the gaming industry has used in recent years in branding content, but in a fairly simple-to-build way largely using drag-and-drop methodologies. The article also notes that Rapt has focused on making the content playable and usable on small mobile devices, and it features a Q&A interview with Rapt's founder, Erika Trautman.