HDR’s Creative Impact

When she wrote a paper in 2019 for the SMPTE Motion Imaging Journal on the subject of how high dynamic range can impact the creative aspects of storytelling, Rory Gordon, senior colorist at Santa Monica-based post-production facility Arsenal FX Color, wanted to emphasize that “workflow has a strong impact on the creative experience,” as she puts it today. That remains true, Gordon insists, with the only thing changing since 2019 being the fact that “people are now more aware of HDR, they have a deeper understanding of what an HDR image can look like today.”

In particular, as a senior colorist who works primarily on television and streaming content of all genres, Gordon emphasizes that it is “really important” to comprehend the engineering process, and the technical side of the creative work that people like her do every day.

“The analogy I like to use is race car drivers,” she explains. “We, as colorists, are the drivers of the race car. Our job is to get it over the finish line. But if you are driving a car and spend some quality time with the pit crew or the mechanical engineers that built the engine, and you educate yourself about what things sound or look like when something is going wrong, you will have a better result. When you are driving along, and you have a deadline that is forcing you to blow through [the work] at top speed, you will be better qualified to see when you have a problem and what area it is coming from. You can then have a more educated conversation with the expert in that area and confirm what is going wrong. In my case, as a colorist, I can point you to the very specific direction where I think there is an imaging failure, and then [the engineers] can jump in. It’s much more collaborative that way.

“I think one of the greatest misunderstandings in our industry is that color scientists or engineers are not creative. Their work is a very creative discipline, especially in terms of the order of operations they use to solve problems. If you, as a creative person, can understand the way color scientists look at the big picture, the better off you will be. It might be a problem with out-of-gamut mapping, for example, that affects extreme edge points like highlights—ambulance lights in an image, for instance. You need to understand there were probably tradeoffs that the color scientists made in designing that color management system to preserve parts of the image that are generally more important, like skin tone.”

Gordon adds that it is important to understand that on the modern landscape where imagery is being viewed in a wide range of environments and on a seemingly endless wave of devices and monitors, those factors have the potential to change an image, or at least, how it emotionally impacts the viewer. So, the subtle aspects of color grading and technical tasks like managing metadata become more crucial than ever.

“The reason HDR is so impactful is not necessarily because of the imagery itself,” she says. “It’s because having brighter, more colorful pictures enable viewers to watch the images in a greater variety of surrounds, while still having a quality experience. There has been a lot of work done on how a surround will change an image. So, the idea that you can have greater contrast ratio is extremely impactful. It’s not necessarily that you need more contrast in order to have a pleasing image. It’s that the image has a greater chance of being perceived as contrasty in brighter surroundings.

“In terms of color gamut specifically, that can be a bit of a different ballgame because so few home displays today are fully covering Rec. 2020 by a substantial amount yet. But that being said, most of those displays are able to cover the P3 color range now, so that is similar to a theatrical experience. The idea of bringing in a more expansive image that feels theatrical because of the greater color gamut available [in a home viewing device] can make the content more special while watching it in your own home.”

Gordon refers to the issue of making sure that contrast or color relationships in content hold up on many different kinds of displays and in many different kinds of environments as the challenge of “relational color.” The rapid advancement toward greater HDR capabilities in consumer display technology, therefore, has created new challenges for both the engineering and creative communities to consider in designing and executing post-production workflows.

“People are working on these issues,” she points out. “You have ambient light calculations and implementations being put to use on displays. We now have Filmmaker Mode on LG OLED [monitors]. And that is the case with HDR, as well. Some of that is with [formats like] Dolby Vision, with the metadata doing a handshake with the display in switching over to the settings you need to have switched over. With Filmmaker Mode—that one just straight-up grays out some settings. The general idea is to make things easier for consumers who use these monitors. They may or may not like it, and that’s their choice, but it does make it easier for the intention that was set by filmmakers in a high-end color bay to be faithfully represented at home.”

Consequently, she suggests there is no way around the fact that, on this new landscape, both artists and consumers can benefit from learning about the huge range of home-viewing options.

“These displays can now go up to 1,000 nits and beyond and to P3 and beyond,” she says. “So, the piece that is really necessary for having viewers enjoy content on those displays is to have them understand what these different modes do. And the same thing for artists. Metadata is changing nowadays for different platforms and we have cool new tools that can make particular tweaks to make it easier to marry a standard def version with an HDR version. I personally think that if you can find ways to give people a little bit of new information from time to time, they can use it in a way that shows they are smarter than we give them credit for. To a degree, we have to be more generous with the benefit of the doubt in assuming viewers will be able to make good decisions with their home displays, and the same goes for artists and color scientists. After all, you don’t gain mass implementation of a new technology just by having the technology be good. You have to convince people that there is a way and a good reason to learn how to use it.”

For artists like Gordon, achieving the creative intention set by filmmakers so that it can be impactful on viewers no matter what viewing platform or format they are utilizing largely revolves around figuring out the best way to be “color target independent,” a goal that still remains, at least for now, somewhat elusive. By “color target independent,” Gordon means that the color workflow should ideally be able to utilize highly complex metadata in a seamless way that lets, for example, HDR mastered content be viewed in standard definition, or other formats, when necessary.

Indeed, Gordon emphasizes that HDR and SDR “are married now” through metadata and through the basic color workflow process.

“That’s the main thing that has changed in production,” she says. “We are now looking at how can we have a workflow that is color target independent? When we work on shows right now, we work in a log space, or in the ACES world, what we would call ‘scene referred space.’ The intention there is to be able to make creative decisions that are relevant and continue to be applicable regardless of whatever the color target is. That can be difficult when you consider that in the SDR world, if an image clips, it’s not a huge deal if you lose some detail because the dynamic range isn’t sufficient to make it distracting. But that same shot, in the HDR world, can be distracting.

“In the HDR version, when you get bright highlights that are 1,000 nits instead of 100 nits, you do see those details that are frequently distracting. Things like gels or filaments in a light or wrinkles in a cyclorama, for example. But, because we have the knowledge that those flaws will be visible at some point, we are typically able to finish even in SDR correctly so that they won’t be, because we can still recognize errors on our vectorscopes and waveform monitors. So, the idea is that in our workflow, we need to have a way to translate the relationships, like contrast ratio—we want to make sure our pipeline is portraying it in roughly equivalent relationship between SDR and HDR.”

Currently, there are three primary HDR flavors offered with modern televisions, and most HDR-capable content is mastered these days using one of them. Gordon says most of her television work is finished in Dolby Vision—a format widely supported by most high-end 4K and OLED televisions. Dolby Vision content travels with dynamic metadata to a user’s television, which lets the viewing device adjust color and peak brightness levels frame-by-frame. It is capable of producing up to 10,000 nits of peak cinema-style brightness, though few televisions are yet fully capable of taking full advantage of that feature.

HDR 10, the mandatory format for Blu-ray discs, sends static metadata into a video stream to calibrate the overall picture, but not on a frame-by-frame basis, and only goes up to 1,000 nits of brightness. More recently, however, HDR 10+ was introduced, and that standard also relies on dynamic metadata, and can go up to 4,000 nits of brightness. Both Dolby Vision and HDR 10+ support 12-bit color depth.

All three improve viewing brightness, shadows, and depth on modern televisions capable of receiving their input, and likely further HDR-related improvements for consumer viewing will come down the road. But Gordon points out that the industry is still far away from any type of official standard, and she doesn’t expect there to be one any time soon.

“Certainly, it would probably put less pressure and stress on productions and finishing houses if there was one single common HDR format, but I think this situation differs from the shift from SD to HD or from HD to 4K, because that is essentially one core item you are talking about—the image itself. With HDR, you have the image and the dynamic metadata that has to travel with it—they are both deliverables. So, for the time being, I see the industry being happy with different flavors as long as the end result is an image that, to the end viewer, essentially feels the same. There is no real pressure coming from the consumer to do it, because the viewer can get really great images right now whether it comes in the form of Dolby Vision, HDR 10, or HDR 10+. That’s why I think there is little motivation for people to standardize right now.”

At the end of the day, from the point of view of artists like herself, Gordon doesn’t see the fundamentals of her creative job radically changing as HDR proliferates and evolves. She thinks that camera tests with wardrobe, makeup, and production design are becoming more crucial than ever, and certainly there are “more gotchas” along the way—“more holes to fill, like times when you have to generate an SDR file first and times when you have to generate an HDR file first, which can impact your production schedule. And there are certain technical hurdles with certain types of images, like trimming lightning, for example. There, if you are working in Dolby Vision, it might require more attention, because with dynamic metadata, the system can take the peak brightness and shadow on average per shot, but if the scene is supposed to have a bunch of flashing lights, you have to find ways to instruct the software to allow certain frames to be brighter, darker, brighter, and so on. So, there is some juggling with more balls than in the past, but overall, it is still juggling. But as has always been the case, composition of the image is the most impactful thing on color. It still goes back to cinematography, production design, locations, wardrobe—things like that.”

As the industry trends further toward HDR and related developments, Gordon says she has self-educated herself on these trends, including consuming a series of books she highly recommends on this topic. Among then are “Color Appearance Models” by Mark D. Fairchild; “The Reproduction of Color” by R.W.G. Hunt; “The Art and Science of HDR Imaging” by John J. McCann and Alessandro Rizzi; and “Color Science: Concepts and Methods, Quantitative Data and Formulae,” by Günther Wyszecki and W.S. Stiles.