Connections

While live-action film production during the pandemic was significantly helped by advances in virtual production, COVID-19 and the accompanying rise of virtual production has impacted animation also, according to experts who spoke Oct. 21 during the Virtual Production breakout panel session “Storytelling Reimagined: Animation” during the Introducing the MESAverse event.

Traditionally, artists spent tremendous amounts of time sketching and painting animated scenes frame by frame and iterations during each step of the process caused inefficiencies and increased costs for animation productions.

That started to change, however, with the rise of computer-generated imagery (CGI), real-time game engines and technologies including motion capture.

Moderator Sinan Al-Rubaye, chief experience officer at software development studio ICVR, posed a question to kick off the session: Why only recently did game engines become a thing for virtual production in film and TV?

“COVID-19 was a big game changer,” responded Brian Pohl, M&E technical program manager at Epic Games, whose Unreal Engine is one of the primary game engines that has been used for video games for several years and is now increasingly being used for virtual film and TV productions.

Virtual production provided a way to “address production in a way that would lower risk and profiles for bringing tons of people together,” Pohl noted.

“Somehow we had to leverage real-time technology, multi-user technology, being able to work virtually [and] the whole aspect of virtual production is catered towards, in many ways, addressing that concern,” he explained. So when COVID-19 hit… the studios were sent kind of scrambling” to figure out how to “meet these requirements, still stay productive, still produce content but without exposing our crews to risk.”

Like a Big Shining Star

And game engine technology was “kind of like a big shining star saying, ‘hey, look, you can use this to do collaborative projects, you can use it to potentially work together without exposing your crew to risk… It was in the right place at the right time,” Pohl pointed out.

Although “game engines have been used in the past” for film productions it was only for “previsualization-style services,” Pohl noted.

“COVID and virtual production has kind of pushed things forward to integrate live action and LED walls and all that kind of thing is coming to the forefront,” he added.

There has indeed been a “big explosion” in virtual productions of film and TV shows, “especially in the” visual effects (VFX) space, according to Adam Maier, producer at Reel FX Animation, who noted he started out as an animator working on films, TV shows and video games

But “for us actually it was more of a challenge,” Maier said, noting studios and financiers have had to be convinced to do something apart from the traditional workflow for this technology.

Real-time ray tracing was a major factor for his company, he said, noting it can save time and money with lighting at the beginning of the production process thanks to the new technology. Motion capture is a factor now also, he told viewers. All of those things “pushed us into this world” of virtual production, he said. “Being able to reuse these assets for games” has been another factor, he added.

As chief of product at ICVR, Chris Swiatek said he ends up “wearing a lot of hats” between serving as creative director, project manager, writer, educator and more.

“One of the biggest things that you guys touched on,” Swiatek told his fellow panelists, is “just being able to do more in one place – so being able to accomplish more of your steps within Unreal Engine, for example, and not having to pass off assets…. The biggest no-brainer of doing final rendering in Unreal Engine is that, as you said, you’re already doing your previz, you’re already doing your motion capture and starting to set up layouts within Unreal Engine.”

After all, Swiatek explained: “Having to take all of this stuff that you put together and then pass it off to another program for final rendering is an extremely kind of difficult and frustrating and, by extension, costly step. So I think it’s just a natural extension of the evolution of game engine rendering technology and hardware getting better that if you can afford to do the final rendering in the place where you’re already doing your previz and a lot of your production earlier on then that’s a big cost saver.”

The three experts went on to agree that it is now possible to produce a full feature film or episodic TV program in Unreal Engine.

“And I can say that because we’re doing it right now,” Maier disclosed. He conceded he “can’t say too much” about that plan, noting only that there are positives but also “a lot of challenges” for reasons that include the new pipeline involved and the training required.

Will producing a show fully rendered out of Unreal and using the engine for various steps be able to match the Pixar-level final renders coming out of a game engine?

“Probably not,” Maier admitted. “But we’re getting there and it’s moving very quickly in that direction, and we can meet and even exceed what we’re seeing on television right now and you are seeing people actually start to create feature films as well using game engine technology that honestly look good.”

Pohl agreed that it can be done now – and with smaller-sized teams. But it can be more costly than traditional methods, he said, adding: “New technology does not necessarily mean it’s going to come and be cheaper.” For one thing, new types of people will need to be hired, he noted.

“In general, I would say that it’s all dependent upon the vision of the director, what he’s trying to accomplish and the resources that are at his disposal or her disposal,” according to Pohl.

Don’t Assume

Don’t assume the film industry will go back to traditional work procedures after the pandemic ends, Pohl said. “A lot of people are starting to see the benefits” of working remotely or at least in a hybrid way with a global workforce, he pointed out. For instance, motion capture can be done by a team or person on one side of the planet, while another group can be in another location doing something else now, he said.

Pohl predicted that remote production will become a mainstream form of film and TV production.

Agreeing that it will become mainstream, Maier said it is “incredibly beneficial” to use a global workforce. Noting that a lot of people are making content now and are still learning how to use virtual production technologies, he said: “We’re not going to get it right 100 percent of the time” but predicted the industry will quickly learn “how to do it best.” Virtual production is not a fad, Maier added.

It is, however, always a concern that new tech may be used when it shouldn’t be used just because an organization has access to that tech, Swiatek said. This technology will become a larger part of more productions as more people learn how to use it and the tech keeps improving, he added.