<img height="1" width="1" src="https://www.facebook.com/tr?id=414634002484912&amp;ev=PageView%20&amp;noscript=1">
Donate
The 2024 Media Technology Summit
Donate

Virtual Production Initiative Gains Steam

December 27, 2021

According to Kari Grubin, a media and entertainment consultant who is leading project management for SMPTE’s Rapid Industry Solutions (RIS) initiative, and Erik Weaver, head of adaptive and virtual production at the Entertainment Technology Center (ETC) @ the University of Southern California, 2022 is likely to be “a really important year,” in Weaver’s words, for the proliferation of virtual production methodologies, tools, education, and projects.

“I expect it will be an especially important year for bringing [virtual production] down from the higher level where it is being used now for shows like The Mandalorian and The Book of Boba Fett, so that we can begin democratizing it for productions of all sizes,” Weaver suggests. “It’s already moved quite a bit beyond giant [virtual production volume] stages. You can do your own pitch-viz, previz, or whatever you need to do to produce a movie. Talented people can already create a version of their show Using Unreal Engine [the videogame rendering engine system now being widely incorporated Into many virtual production systems]. You can do edits, cuts, insert characters, do voiceovers—everything but actually film it. It’s really quite amazing what this technology is capable of, but it is much more than a so-called ‘production tool.’ It’s starting to change how we do things. Pre-production, production, and post are getting more bundled together using virtual production. About two-thirds of your shots that you would have had to clean up off a greenscreen, you can get done immediately on set. So, in that sense, it really is a brand-new discipline.”

Grubin adds that as on-set virtual production (OSVP) proliferates, it will open all sorts of doors to new options on how content can be built. “Many small productions or documentaries may not be able to travel across the globe,” she says. “Now, all of a sudden, they have an alternative they didn’t have before. You can sit in a room yet create a background. The commercial industry is doing this quite a bit. Things that were not accessible before are now suddenly accessible and affordable. There is expense, of course, but on the other hand, you can have Magic Hour all day long, rather than for 30 minutes on location. This has the potential to create a whole new way to tell stories.”

As they suggest, it’s a methodology that is already well underway for major, high-budget studio production work, with unique tools being woven together in a wide range of innovative ways to put people, props, sets, and other objects into environments where real-world cameras can’t go. However, the goal of democratizing it for wider industry use and more applications, by contrast, has a long way to go.

Achieving that goal, explains Grubin, makes up the RIS initiative’s first and current project, which launched midway through 2021. It now includes an Advisory Group that meets monthly, and which is planning major steps forward throughout 2022 and beyond.

“We have formed a partnership between SMPTE, the ETC, and other RIS Advisory Group members [including Epic Games, Unity, Arri, Canon, Sony, Dolby, the EBU, Samsung, RED, Planar, Movie Labs, and several others] to bring about solutions, unbiased collaboration, and a general democratization of the ecosystem,” she explains. “Over a year ago, SMPTE was looking to see what areas they could be of support and service to their members in, and so we did a survey along with some research on the role of on-set virtual production to understand the pain points. It was a very large survey across the industry, and we realized people were excited about this technology, but there were big gaps in being able to support it. So, the feedback overwhelmingly suggested there were two core areas that needed support. One was education, simply because there is a lack of trained and skilled people to be able to leverage game engines that also can speak content creation. We had a lot of filmmakers who didn’t understand how to use the tools, and a lot of smart technical people who understood the tools from the gaming and technology side, but who didn’t speak the same language as content creators.

“And then the other area was interoperability—making sure that equipment manufacturers would know how to plug into the virtual production environment, and to make sure graphic files and metadata can travel seamlessly through the pipeline. Initially, it was all being developed in siloed pockets by individual studios and industry organizations, and there were no industry-wide best practices or standards.”

Thus, to start, the initiative’s three primary workstreams that the Advisory Group is focusing on revolve around the issues of education, interoperability, and outreach. The first deliverable that rolled out from the initiative in 2021 was an On-Set Virtual Production Wall Chart
that is intended to eventually evolve and become interactive, to give people essentially a foundational visual map to follow while building a virtual production ecosystem. (You can find a short SMPTE video explaining the Wall Chart here.)

“We realized that virtual production can mean many things to many people,” Grubin says. “So, we decided we had to start somewhere to offer a base and a foundation to be able to grow people’s knowledge and help the entertainment industry navigate this process. We are looking at the designations within the Wall Chart to identify what companies fit into which part of the ecosystem, what educational assets tie into those areas, and we specifically designed the chart to be a simplified, high-level overview. But we plan on developing an interactive version to release in the next 12 to 18 months. The idea is to create interactive portals within the chart that people can take to learn about specific parts of this work, such as a section on virtual art direction (VAD), for example. You will be able to click into it and do a deep dive on that subject or find a person or company that can help you with that particular area of work.”

Meanwhile, the initiative is working hard to contextualize some of the stereotypes about virtual production—the key one being that it is prohibitively expensive for most players in the industry who are not major studios. Weaver points out that major productions, like the recent Star Wars-related streaming shows, have been spending huge amounts of dollars over the last couple years to use virtual production techniques extensively and those numbers will likely go up—money one might assume is simply part of larger visual effects budgets. But that’s not really the case when one digs deep into the numbers, he suggests.

“You might think they are simply stealing from the VFX world, but what is really happening is they are more stealing from travel budgets and housing budgets,” he explains. “So, while it is true that [virtual production] is more expensive than traditional visual effects techniques, in many cases that is being offset right now by wiping out all those costs associated with sending teams out across the world.”

He and Grubin add that the global pandemic has, in many cases, accelerated early adoption of virtual production methods across the industry. “In fact, we are at a unique crossroads where it has accelerated faster than some of the tools that support the connectivity, interoperability, and the educational portion,” Grubin says.

Meanwhile, the part of the RIS Initiative that revolves around outreach and luring participation and support from across the entertainment industry got off to a good start in late 2021 when SMPTE received an Epic MegaGrant from Epic Games in the amount of $150,000 to support RIS programs and projects.

“Epic reviewed the plan that we shared with advisory group members and provided those funds which the Advisory Group have designated for years two and three,” Grubin states. “The whole point is that RIS is a self-sufficient initiative, and all fundraising done with our advisory group sponsorship efforts goes to support what it takes to enable the initiative and the goals outlined.”

Along those lines, both Grubin and Weaver expect 2022 and 2023 to be a vibrant period for a flurry of initiatives designed to advance virtual on-set production across the industry. These will include a looming partnership between ETC and SMPTE as part of the initiative—details of which were not available at presstime—and much more.

“Anything we can push out quickly will happen early in the year, and then, we will focus on the work that takes more time,” Grubin explains. “That will be the timeframe for understanding the bigger technical issues and then finding and developing solutions for them in partnership with industry professional organizations, the guilds, the Advisory Group, and our many partners. We hope to have deliverables start coming out month-to-month, starting with a lot of educational pieces that we want to push to a broader audience.”

A significant part of those “bigger technical issues” will revolve around figuring out what areas of the larger virtual production paradigm do, or do not, need to be standardized. And if they do need to be standardized, to figure out how to move that process forward in a timely enough fashion that will make any potential standards meaningful in the long run as the technology and industry push forward at a rapid pace. Some specific pieces of the larger puzzle, for instance, are already standardized for the larger industry—things like the SMPTE ST 2110 standard for sending digital video over IP networks, and the SMPTE ST 2098 suite of standards for cinematic immersive audio, for example. Other pieces need more study and investigation to decide if they should be standardized, and still others have no practical need for standardization.

“The word ‘standards’ is very challenging for this area right now,” Weaver states. “When you look at a typical standards process, you are looking at something that could range anywhere from a year to three or even four years. But this field is moving at a lightning pace. So, we aren’t ready to lock down standards. We have to first define a set of understandings, best practices, which are important but are not standards, and then we will know what areas will be mature enough for formal standards. I’d say we probably have at least six to nine months to go before we can solidly lock down what we can march forward with as a natural standard.”

Grubin elaborates that “because of the complexity of what is required to create a true standard, the process can be inflexible and take too long to be relevant. The RIS initiative, which currently has over 45 members ranging from vendors to camera, hardware, software manufacturers and game engine companies, among others, will first want to figure out what the defining areas of interoperability need to be, and so on. Therefore, a lot of this work is about best practices, use cases, and providing a framework. The SMPTE standards community, after all, has various options for sharing important information beyond formal standards, such as its series of Registered Disclosure Documents (RDD), for example. Plus, a lot of entities have already done pieces of this work. Movie Labs, for example, has created a common visual language and ontologies for media creation that includes this kind of work. We are not looking to re-create that. Our job is to bring it into the wider community as a whole, so that it can be leveraged on a much wider scale. So, we will work with the ASC, Movie Labs, the ETC, Netflix, and many industry organizations to make sure [their initiatives] are brought in, and that we have common glossaries. And then, when there needs to be new standards work done, it will absolutely be done.”

Tag(s): Featured , News , Newswatch , RIS

Michael Goldman

Related Posts