SMPTE Newswatch Masthead

Hot Button Discussion

New Tools, New Creative Possibilities
By Michael Goldman  
As 2015 arrives, it's appropriate to ponder the creative impact of some of the numerous technological advancements unleashed across the cinematic landscape that SMPTE Newswatch has been examining in recent months. By that, we mean, how have the technical capabilities provided by new and improved digital tools, workflows, and techniques impacted the creative community's ability in recent years to advance how they use image and sound attributes like 3D, immersive audio, higher frame rates, and higher dynamic range, among other things, to tell stories?
Experts suggest this is a tricky question, because virtually all these areas are in various stages of ongoing development. They are far from mature technologies with industry standard workflows attached, and are still in the experimentation process. Even their most profound creative potential needs to be balanced by filmmakers with everyday business practicalities and cold financial considerations regarding whether using such capabilities is even practical to begin with, among other factors. Therefore, while startling visuals and sound have evolved out of the creative laboratories of world famous filmmakers like James Cameron, Peter Jackson, George Lucas, David Fincher, and others in recent years, no final conclusions can be drawn on how such work will permanently impact the art of filmmaking or, for that matter, the larger filmmaking community. Nevertheless, the toolset available to the creative community continues to expand almost daily, and the experimentation with those tools goes on unabated, and will continue to do so for the foreseeable future.
At least, that's the perspective of many industry watchers, including longtime industry consultants H. Loren Nielsen, president and co-founder of Entertainment Technology Consultants (ETC) and a member of the Hollywood Post Alliance (HPA) Board of Directors; and Jerry Pierce, HPA's Vice President and a SMPTE Fellow.
"Consider high frame rate, and how that might impact a storyline," Pierce says. "One project [he heard about] under consideration was a film with two storylines-the real, or main, storyline and the hyper-real storyline, running simultaneously. For the hyper-real storyline, they considered using higher frame rate, sterile lighting, and a very clean, no grain look to represent reality as close as possible, making sure the movie has two different looks throughout. That's the kind of thing we expect to see more of from filmmakers. Together, filmmakers and technologists create these new technologies, and our creative partners test and explore how they can be used in storytelling. Some of these lead to fundamental changes in filmmaking-3D is a good example. It worked well for animation, but not for [live-action movies] until Avatar came out. That is when we knew we could tell really good stories in 3D, while also enhancing the viewing experience."
Both Pierce and Nielsen point out that the rise of digital cinema altered an equation that, creatively, had been unusually stable for more than a century in this regard--movies were made on film, shot and projected at 24 fps, using the same, basic workflow for the most part. Over time, filmmakers and audiences grew comfortable with an established cinematic language, Nielsen says. On the heels of the digital revolution, however, that cinematic language needs updating, she suggests.
"There is an artful use to all technology for the purpose of telling stories," Nielsen explains. "The language of film was developed over the last century and includes techniques like framing, camera movement, editorial, lighting, production design, color palettes, and sound design. Over the years, we have all learned to understand the language. We know that a fade from one scene to another refers to time passing, or that piecing cuts together gives us a story. Over the last few decades, as movies got more layered, and we succeeded in creating a language of visual effects, filmmakers became sophisticated in using those technologies to tell their stories. For example, David Fincher uses high-resolution cameras to allow reframing and stabilization of his images [most recently using the Red Dragon 6K camera system to shoot Gone Girl], and he cuts out performances from one take and pastes them into another. He might have a group of actors around a table in a scene from four different takes, all existing in the same frame.
"We are now entering a new world with higher frame rates, higher resolution, higher dynamic range, 3D, and even going further afield with virtual reality. So now, a new language has to be developed."
For instance, Nielsen points out that some filmmakers, since Avatar and Hugo, are evolving what she calls "more of a 3D language" for dramatic purposes and not just for the gimmicky thrills that the medium originally offered audiences. "Maybe they want to isolate characters and make them lonely, so they will position them behind the screen, rather than the front, establishing a sense of disengagement," she says. "That becomes a language element."
Both Nielsen and Pierce say that, similarly, the creative community is searching for ways to bring "meaning" to the use of high dynamic range technologies-something that "has more engagement" as Pierce suggests, or "speaks to us emotionally," Nielsen states.
"Maybe they will go to more dynamic range for a heightened, more excited response, and go to lower dynamic range to feel more intimate," Nielsen says. "At one time, we tended to use aspect ratio that way. Wide aspect ratio was used for openness and vistas and expanse, and smaller, tighter aspect ratios were used to depict intimacy. That has almost disappeared, partially because the 2.35 aspect ratio is rarely a bigger picture than the 1.85 aspect ratio [on today's fixed width screens]. We have lost some of 35mm film's acquired meaning, but have new opportunities to express these ideas with things like dynamic range."
Same thing with higher or variable frame rate filmmaking, Nielsen adds. The technology community can now offer a more practical implementation, and will hone that capability depending on what the creative community desires. But exactly how filmmakers will use HFR is far from understood. In fact, how to understand such things is part of the great unknown right now, she elaborates.
"We have to develop a new understanding of what things can mean, or if we still want to induce the same responses," Nielsen says. "The filmmaker might want to make us feel as if we're looking at a Rembrandt or Caravaggio painting, but if they are using higher frame rates, they may need to manipulate the image to make it darker, reduce saturation, or do something else differently than they would otherwise do today to evoke the same response."
In other words, storytelling mechanisms--the very language of cinema--are evolving and growing, and new ones are developing, as new technological capabilities are added to the toolkit. Certainly, the immersive entertainment concept--both subtly and not so subtly with video games and new virtual reality applications--will be part of this evolution. Additionally, the immersive concept can be applied cinematically to traditional film concepts, Nielsen says, with attempts to use 3D, immersive sound, and other techniques to make the viewer feel more a part of the environment they are watching. Nielsen suggests that Cameron succeeded in doing this in Avatar, a movie themed on environmental concerns, by using 3D to have delicate flowers, and other elements of the environment the characters were trying to protect, literally float around the audience.
"That's part of this new language--we feel we can surround ourselves with objects in 3D and object-based or immersive sound," she says. "This makes the audience more involved, creates a more intimate experience. But it is still very experimental. If filmmakers can use it to make us secure and safe, could they scare the audience, and would we like it if they did? Would audiences want to be inside a 3D fire, for instance? What if Backdraft were done that way? Would it be exciting or simply terrifying?"
On a more practical level, the technology community remains focused on delivering many of the positive attributes of film that were lost in the transition to digital, according to Nielsen. She says creative response has been strong as it relates to higher resolution cameras, deeper blacks in displays, and broader color gamuts, for instance. But a couple of practical issues of great concern to the creative and business sides of the content industry have not yet been adequately resolved by the digital revolution-long-term archival solutions and anything remotely resembling industry standard digital workflows.
"In fact, with digital cinema, there are also a whole new set of challenges," Pierce suggests. "As you use new capabilities, tools, and techniques, you have to then adapt the system to embrace that, and with digital cinema, it's an ecosystem that we have to [create and manage] today for this puzzle. Being able to continuously roll the camera is great, but now, you have ten times more footage to manage and edit. There is therefore a discipline that is potentially lost that used to exist with film, when they had to be more cautious about how much [film] was available in a 20-minute can. We have yet to see any workflow stabilize for all this--everytime you do a new project, you use a different workflow. If you tried a workflow successfully, would you use it again or try something else? And will all your assets, all your archival footage exist or get lost? We still keep our movies, even digital films, on film separation masters, because of the lack of a long-term digital archival solution. But we are losing materials used to make the movie in many cases. These are ongoing issues that impact both [business and creativity], that the technology community needs to address."
Pierce suggests that the nexus between creativity, technology, and business can't be separated, and many of the ongoing technological developments now offer, in some cases, stunning creative possibilities, which will only have relevance in the long-term if they can find a meaningful business model to make them useful. Therefore, he expects it is likely that many breakthroughs will end up serving primarily niche applications in the long-term, particularly on a modern landscape on which consumers have made it clear that having on-demand access to content on their personal devices at times and locations of their choosing are their priority. Thus, higher-quality content that will most directly benefit from immersive sound, deeper resolution, and all the rest will logically end up as what he calls "premium offerings" for those who want a break from their day-to-day content.
"Ultimately, this is all a business, so we always have to ask, what is practical," Pierce says. "These technologies have to pay for themselves in one way or another. So large, premium screens designed to offer a higher level of experience to cinemagoers is something the industry is trying to create. For that, they have to offer the 'full' experience, but it is not yet clear how much of that experience consumers would want or demand. It isn't just image and sound--they also want comfort and convenience, so bigger seats, assigned seating, beer sales, and things that have nothing to do with the image on the screen may be just as important to make those customers happy. The technology is there and we know what the filmmakers can do with it, but is there a market for it? That is a question we need to ask ourselves for each new development."
In any case, for all of these developments, building the aforementioned "ecosystem" on top of the foundation of the ongoing standards work being done by SMPTE and others across the globe rarely gets linked directly to the creative side of the industry. But, at the end of the day, the creation of entertainment content is a business, therefore such standards work and the ongoing construction of a digital ecosystem "is essential," Nielsen states.
"The effort to find, select, and define standards is essential, not only because it provides confidence for distributors that their product can be released to global networks repeatedly over time," Nielsen says. "The standards process also serves as a reality check. The compromise that is necessary in the development of standards is often helpful in creating a better, more robust solution. That goes hand-in-hand with the development of the technology and the creative use of the technology, and it assures that the business can operate reliably."

News Briefs

6K Alexa Bows
The large-format, 6K version of Arri's Alexa digital camera system made a splash in Hollywood in early December after initially being previewed this past fall at the Cinec camera show in Munich and then at the Cameraimage Festival in Poland. At its Hollywood debut, Arri officially launched the Alexa 65 and screened some of its test footage on a Barco 4K projector at the Linwood Dunn Theater on 10 December to an audience of cinematography veterans primarily. The Hollywood Reporter's coverage suggested that major industry names were duly impressed, including the ASC's Bill Taylor, David Stump, and several members of the AMPAS SciTech Council, as well as leading names from just about every major Hollywood post-production house, who praised footage shot by cinematographer Greig Fraser using a prototype version of the Alexa 65. In his coverage of the event, industry pundit Jon Fauer was also impressed with the footage, and said he viewed "two new Alexa 65 image formats that I had never seen before--1.78:1 and 1.5:1." Fauer suggested, "you could almost feel the winds of change" inside the theater because the presentation was able to impress an expert audience with "RAW uncompressed 6K ... compressed through a pipeline the diameter of a straw, thanks to antiquated DCI specifications." He also urged the industry to upgrade its DCI specs in response to ongoing developments like the Alexa 65.
Turn Smooth Motion Off?
David Heuring's blog on the American Society of Cinematographer's website recently ran a report about an effort spearheaded by cinematographer Reed Morano, ASC, to petition manufacturers of HD televisions to change default settings on new televisions away from the current motion interpolation, or "smooth motion" setting. Morano launched a petition to push this agenda on, and had received more than 9,000 signatures as of late November. The reason behind her crusade is that, in the opinion of many in the cinematography community, the motion interpolation setting on HDTVs, which was designed to reduce motion blur and is thus quite useful for watching live sports, for example, reduces the cinematic look of imagery shot at 24 fps, whether digitally or on film. This lessens the ability of viewers to see images as filmmakers originally intended. Morano told Heuring the petition is necessary to help "filmmakers protect our work." She suggested the industry could provide a "very simple button on the remote that toggles between 'cinema' and 'sports' settings," to allow the motion interpolation to be simply used by consumers where it is most effective, without interfering with artistic intent. Meanwhile, her petition offers this link to let owners of HDTVs know how to turn off motion interpolation on their televisions. 

Cardboard Rising
As mentioned in the October SMPTE Newswatch, Google's Cardboard Virtual Reality Kit has been promoting the concept of bringing affordable virtual reality (VR) capabilities to consumers via apps on their smartphones as an alternative to the high-end approaches being taken by companies such as Oculas VR, now a subsidiary of Facebook. This month, Google pushed the concept further, according to an article in TechNewsWorld, by announcing an updated app for what the article called the "minimalist Cardboard virtual reality viewer," new specs for Cardboard viewers, and plans to hire more people to work on the ongoing Cardboard project. According to the article, Google claimed that, as of December, more than half a million Cardboard viewers had been shipped, with apps readily available on Google Play. Analysts quoted in the article suggested that while the concept initially seemed so low-end as to be almost laughable when it debuted, it now appears Google is seriously building what one analyst calls "an emerging end-to-end VR ecosystem." The Cardboard concept basically provides users with a kit to build cardboard goggles with lenses that they can attach to their phones in order to transform them into VR viewers when running an increasingly growing number of apps.