February 2017

SMPTE Newswatch Masthead

Hot Button Discussion

Creative Control as the Palette Expands
By Michael Goldman  

Over the years, Newswatch has examined the development, improvement, and standardization of modern image display enhancements by the technical community. The January 2017 Newswatch discussed how consumers perceive new image enhancements in typical viewing scenarios. That issue, however, connects to how content creators are using these enhancements. From that perspective, how has the perception of content creators been factored into the evolution of the so-called “expanded palette” provided by the technical community? By expanded palette, we mean, in particular, the three big-ticket visual improvements—high dynamic range (HDR), wide color gamut (WCG), and the use of higher capture frame rates (HFR).
Professor Harry Mathias has some provocative thoughts on this topic as his pedigree touches both the technical and creative realms. Mathias, an author, and professor of film at San Jose State University, worked for many years as a professional cinematographer and as an executive at various technology companies. He is also a SMPTE Life Fellow and was a founding member of SMPTE’s Digital Cinema Technology Committee (DC28) and Digital Cinema Standards Committee (21DC). His most recent book, The Death & Rebirth of Cinema: Mastering the Art of Cinematography in the Digital Cinema Age, addresses the struggle modern cinematographers face to develop methods to manage the new imaging technologies being hurled at them in order to keep control of the images they produce all the way to final exhibition or broadcast.

“Since the cinematographer’s work is judged by the ‘photographic’ images they produce and not the technology they employ, there is a constant struggle for the cinematographer to master and control the technology, without it overpowering the film’s visual impact,” Mathias explains.
In that spirit, he strongly suggests that improved dynamic range, color gamut, and faster frame rates, in any combination, though potentially startling from a technical point of view compared to what was possible even a few years ago, means little if doing so does not serve some specific creative function in the hands of filmmakers who know how to utilize them.

“Filmmaking is about telling stories visually,” Mathias says. “Tools that simply dazzle, but do not help tell the story are not useful in filmmaking and do not ‘improve’ cinema, per se. When they try to put dynamic range, color gamut, and higher frame rates all together, some people try to make the argument that three incremental improvements in visual quality combined somehow result in a seismic sea change in cinema technology. But I’m not sure that argument is valid. I think that’s why the three technologies often get lumped together, but I don’t believe they should be.
“A good example of a new tool [that fundamentally improved filmmaking] would be the Steadicam [which debuted in 1975],” he continues. “Nothing remotely like the Steadicam existed before [cinematographer] Garrett Brown invented it. It was a completely different imaging tool. He wasn’t taking a technology that existed, like a camera dolly, and improving it. He created a filmmaking tool that didn’t exist, and which changed filmmaking after it was invented. The main feature of the Steadicam is that it helps tell the story by making the camera more mobile. The same is not necessarily true of wider color gamut or higher frame rates or higher dynamic range, which in a sense are not technology breakthroughs in terms of storytelling and improving the quality of cinema.”

Thus, for the cinematographer, dynamic range, color gamut, and frame rates are no different than using light for image rendering—it’s all about how they are used to tell stories, and to use them, filmmakers must control them. To control them, they need assurance that the way they use these features creatively at the front end, on the set, is accurately maintained as the image travels downstream through post and eventually ends up in exhibition or broadcast. In this sense, strict standards are important to filmmakers, but the rigid application of visual trends resulting from new technological breakthroughs that require the standardization are not. “In other words, it’s not useful if you can’t control its image rendering,” Mathias emphasizes.

“That is the big issue regarding dynamic range from the cinematographer’s point of view, for example—not whether dynamic range is bad or good, but whether it is controllable and repeatable,” Mathias explains. “And whether it looks the same in every properly calibrated movie theater, or on every calibrated home television. Essentially, the enemy of that concept would be variable standards and different television manufacturers having different amounts of dynamic range rendering. If, when translating your work from one brand of TV screen to the other, or one movie theater to the other, it ends up looking different on different screens, that’s a very bad outcome. It is crucial that dynamic range is controllable and consistently reproducible. Some people might suggest that images with more contrast are better than images with less contrast, however, that is not always the case. In a creative situation where more contrast is needed, an image with more contrast is desirable. However, that image must be controllable in the same way that color reproduction has to be controllable, and so on.

“After all, a cinematographer spends his entire career learning how to control the image, to deal with the contrast and light levels that exist in reality, and to render the image as he intends. If the picture looks different on [different monitors], that is a disaster. Therefore, most cinematographers, if put in a situation where they are not controlling the rendering of the final image, would rather not have access to higher dynamic range than lose total control of that dynamic range. All the work they do on the set is pertinent to controlling the final rendering of the image that the audience will see in the end.”
Therefore, when examining new imaging trends individually, Mathias states “dynamic range is the most attractive to the cinematographer compared to the other developments where color gamut and frame rates are concerned, simply because removing the current limitations on contrast range and dynamic range of film-based cinema and early digital cinema imagers is obviously a good thing.”
However, Mathias says he believes cinematographers should be able to to use dynamic range creatively, and not think simply in terms of a wide dynamic range, but rather, more controllable dynamic range throughout the process, “for purposes of telling the story.”

Every picture or visual idea is not improved by adding more contrast, he emphasizes. “In some cases, you want low contrast. If you are shooting an overcast day, or a depressing interior or a dim living room—you don’t want high contrast. Contrast tells the visual story in the same way that graphic images tell the story. The important thing for the cinematographer is that they have a wider dynamic range available to them as an option, but can choose the dynamic range that is appropriate for telling the story. These creative decisions can be made only if the outcome of the image and the contrast of the image on the final screen can be predicted while shooting the image. If the end outcome is not predictable on the set, that is very disruptive to image-making creativity.”
Thus, Mathias urges cinematographers to learn how to control contrast creatively from a “whole system” point of view rather than “piecemeal.” By that, he means filmmakers need to ponder not only how they want to use contrast on set, but how their collaborators will be using technology to render that contrast later in post.
“As a cinematographer, if you can learn to control the system on set so that you know how it will render in the movie theater, then you have a predictable tool,” he says. “But if the system has nonlinearities along the way, resulting in unintended changes in the final picture, that is a problem. Many people hold a very troublesome concept that if we can just put more and more shocking extremes of contrast on the screen, we will get better images and entertain people more.

“In fact, there are several films in history that would not benefit from HDR. Would you want Casablanca to be a high dynamic range film? In films such as this, the correct rendering intent and the artistically successfully rendering is low dynamic range. Having said that, there are also situations in which high dynamic range would tell the story better. However, the idea that the film breakthrough of the future is going to be high dynamic range, and that audiences are going to line up to see high dynamic range, is a misguided concept.”

Moving to wider color gamut, Mathias argues that the industry also needs “to put a little reality” into the creative implications of the arrival of improvements such as the International Telecommunications Union’s UHD color specification, Rec. 2020, on the industry. He doesn’t quibble with the fact that this is “a breakthrough in color rendering,” but questions the creative implications from a storytelling point of view.
In that regard, Mathias suggests that since neither Rec. 2020 nor any future color standard will be likely to allow cinema or broadcast screens to truly display the entire range of colors the human eye can perceive in nature, we don’t miss those colors in terms of watching stories unfold on cinema or broadcast screens. Therefore, he ponders the importance of wider color gamut as a creative storytelling tool at the end of the day.
“Are there really colors that you cannot see on current movie screens that you miss? What would we gain by seeing colors that we can see with our eye, but that we can’t see in current movies? I have spent a lot of time critically looking at movies, and I certainly don’t miss any of the world’s colors. Nobody walks out of theaters saying, ‘boy I missed those visible colors that I couldn’t see in this film.’

“Again, I’m not arguing that wider color gamut isn’t a good thing, just as I’m not arguing increased dynamic range isn’t a good thing. I’m just trying to put it in perspective from a creative point of view. Once again, it is just a tool to be carefully used. While it is better to have an unlimited palette than a limited one, the key factor is that the mere act of expanding the range, whether it is color or dynamic range, does not necessarily do anything to make it a better movie or bring in a larger audience. That will depend entirely on the story and its visual representation.”
As far as higher frame rates, Mathias is less sure the expanded palette has much value. In fact, in this case specifically, he says, “I’m not entirely sure that this isn’t a potentially limiting (not freeing) use of imaging technology.”
He said he believes that a misinterpretation has evolved over the years regarding what some think are the deficiencies of the traditional 24 frames/sec rate of display, and therefore, the reasons for pursuing faster motion. That reason, of course, is the so-called pursuit of “realism” or what some people call “hyper realism” or greater clarity in the image, minus the appearance of motion blur in moving images. Motion blur represents what the human eye does see when objects move fast enough in the real world, Mathias insists, and therefore, eliminating it is not natural, rather than the other way around.

“Proponents of 48 frames and higher argue that motion blur is a bug, but motion blur is actually a feature because it makes motion look realistic,” Mathias states. “Wave your hand in front of your face. You see a blurred image—that’s how we see motion, that is how we know the hand is moving because it is blurred.”
By contrast, he continues, “if you shoot at 48 fps or higher, you get an exposure time of at least a hundredth of a second or higher as you go up. At that frame rate, you freeze motion blur, and you end up with a kind of a staccato of still images as an object moves. The argument is, instead of seeing motion as a blur, you see motion as a series of staccato still images, which is annoying because that is not the way our eyes see motion in nature. Look at what Disney did many years ago [in animated films]. They went from pencil sketches like Steamboat Willie to Fantasia. Walt Disney understood motion blur, and he painted it with an airbrush into each frame that contained motion. That was one of the visual breakthroughs introduced with that movie. Today, in modern animation, the motion blur rule of thumb is that if an object moves more than half its size between two frames, motion blur must be added by blurring the image either with a rendering algorithm [in computer animation] or if it is cel animation, with an airbrush.”

Mathias emphasizes that he is not arguing against the use of higher frame rates as a creative tool, just against what he views as a rising mythology that doing so somehow increases realism in moving images. In his opinion, “if the goal of movies is to entertain, to tell a story or to elicit emotions, then conflicting [the viewer] about motion rendering is not part of telling the story in and of itself. That doesn’t mean someone can’t use 120 frames to tell a story. I wouldn’t limit artists from any tools, but the fascination with the tool as the end result, with the technology as the goal—that’s what I’m worried about. The goal should still be storytelling, not just to show people an image with a technology they may never have seen before.”
Thus, Mathias cautions the technical community to think of enhancements and standards they create as creative tools for content creators, which need to enhance the ability to maintain creative control, rather than to limit that ability.

“[Some in the technical community] believe we just set engineering standards, and creative people can use them in any way, they choose” he says. “But if you break the workflow with a standard, if you take a situation where you can control lighting and exposure on set and predict the way it is going to look in a movie theater, and come up with a new automatic contrast transcoding standard for image contrast that breaks the creative workflow, you are not just changing that creative function, you are breaking it. The cinematographer must deliberately shoot low-contrast images to be assured they will stay low-contrast all the way through to the audience. We can’t get to a point where [HDR] becomes [so important] that studios decide they don’t want low-contrast movies or low-contrast sequences just because they have all this cool new contrast rendering technology available, thinking that’s what the audience ‘wants.’ In fact, the audience wants images that are visually appropriate to the story being told. The purpose of any changes or new [standards] should be to ensure that it is possible to make sure that the rendering on the final screen remains the director and the cinematographer’s intent, whatever that might be.”


News Briefs
HPA Retreat Chatter

Once again, the latest cinema trends were debated at the Hollywood Professional Association’s Tech Retreat in Palm Springs in late February. According to TV Technology’s coverage, topics discussed by various speakers included a resurgence in 3D thanks to the latest laser projection technology, 4K acquisition and delivery, the industry’s need to adopt more consistent color workflows along the lines of the Academy’s ACES initiative, live event streaming, and much more. But on the cinematic front, the article states that exhibition executives pointed out that it isn’t always technology upgrades that are luring moviegoers to cinemas these days—it’s upgrades, period. In particular, Neil Campbell of Landmark Cinemas of Canada, says exhibitors are now sinking major investment dollars into recliner seating, among other additions. Indeed, he claimed recliner seating is “the best thing that’s been invented in my career,” and that his company’s major investment in luxury seating in the last year happened to coincide with a two percent increase in domestic box office revenue. Conversely, in the Hollywood Reporter’s coverage of Campbell’s talk, he also stated that virtual reality experiments have not proven to be worthwhile investments for major exhibitors thus far.

Underwater VR
A recent article in the MIT Technology Review reports that two engineers are currently attempting to take virtual reality where no one has taken it before—underwater. According to the article, Stephen Greenwood, director of creative development at Discovery Digital Networks, and Allan Evans, co-founder of VR headset maker Avegant, have been working on a joint side project for several months to pair the concept of virtual reality with the concept of floating in an isolation tank. So far, they have developed prototype VR gear for floating underwater that the two men say could eventually be adapted for use in scuba-diving simulations or physical therapy, and other applications. The prototype consists of a waterproof Android smartphone attached to a 3D block of plastic and a snorkeling mask covered in black tape with biconvex lenses, like those used by Google Cardboard headsets. Users breathe with snorkels and take in an audio experience using an underwater MP3 player.  

Film Production Declined in 2016
Filmmaker Magazine recently published its third annual listing of all the U.S. theatrical releases shot on 35mm film stock in the past year, and in 2016, that number—27—dropped significantly from the previous two years. In 2014, 39 theatrical releases were shot on 35mm, and in 2015, that number rose to 64 films following the 2014 initiative by a coalition of high-profile filmmakers to keep Kodak in business. But the article suggests that, perhaps, the drop was a bit of an anomaly in the sense that many well-known moviemakers who prefer film—Steven Spielberg, Christopher Nolan, J.J. Abrams, and Quentin Tarantino among them—did not have movies released in 2016. Additionally, the roundup does not include 16mm productions, or television productions shot on film. However, the article also states that certain production centers for major studio projects in China, Australia, and South Korea, in particular—have continued to erode or eliminate their film production infrastructures. The article also points out that many filmmakers on tentpole movies in recent years have taken to incorporating various film formats into mixed media productions that include multiple digital and film platforms of various types. Among those in 2016 were Jason Bourne and Batman v. Superman: Dawn of Justice. Others trended to film, and sometimes, multiple film formats, strictly for creative reasons, such as Hidden Figures, a 16mm/35mm hybrid.