The Challenge of Creating and Animating Giant Space Creatures
Legendary Pictures’ Pacific Rim: Uprising* is a sci-fi action thriller in which giant robots battle giant space creatures, called Kaiju, amidst a sea of destruction. As the much vaunted sequel to Pacific Rim, this action-packed film was released to wide admiration for its special effects. Creating animated characters like the Kaiju that look realistic is a challenging task, as any visual-effects (VFX) studio will tell you. And considering the sheer size of these creatures, DNEG—the VFX studio in charge of developing the animations for Pacific Rim: Uprising—had some real challenges. Creating any animated creature is a time-consuming process that requires massive amounts of work. Then, having them move in realistic ways presents a whole new set of challenges. Multiply that by the Kaiju characters’ sheer size and any discrepancies come front and center.
“DNEG constructed the Kaiju at a vast scale, and they required the greatest amount of anatomical detail we have ever been asked to produce,” says Theo Facey, Global Head of Creature at DNEG. “Bio-engineered by an alien race, the Kaiju’s anatomy is comprised of a variety of structures—bone, muscle, fascia, skin, sub-dermal plates, and bony spikes—all arranged to create a formidable array of organic weaponry.”
For DNEG, success came from taking advantage of cutting-edge artificial intelligence (AI) and machine learning (ML) software developed by Ziva Dynamics and powered by Intel® architecture. “In Pacific Rim: Uprising, we had a brief of a very specific number of characters, who were very demanding in terms of their quality and their feel. And very early on we identified it is a show where we wanted to push the boundaries of our production,” says Russell Bowen, CG supervisor at DNEG Vancouver.
Bowen also notes that creature work is some of the hardest in the industry—but that “software like Ziva Dynamics, supported by the power of Intel, can really remove some of the heavy lifting that is required, and allows the artists to really focus on the creative aspect of the work.”
According to Barnaby Legg, senior vice president of Creative Strategy at Legendary Entertainment, “innovation is our superpower” is an underlying theme from the film itself, and it also plays out in how Legendary Pictures and the Academy Award* winning DNEG VFX team approached bringing the Kaiju to life.
Using AI to Create Lifelike Computer-Generated Imagery (CGI) Simulations
Traditionally, animations are created by first building the characters and then animating them frame by frame. Any deformations and other dynamics are achieved by manually shot-sculpting their body shapes to achieve shot-specific desired results. Not only are these processes time-consuming and expensive, but there are so many variables in how characters move based on physics and size, their underlying anatomical structure, and more. As a result, if something doesn’t look quite right, it means that artists have to go back to the drawing board and determine which layer is off.
That’s where Ziva Dynamics comes in. Its flagship product, ZIVA VFX*, is an advanced simulation technology that mimics the physics of any material, which allows characters to move, flex, jiggle, and stretch organically.
“Our films are just going to get cooler and weirder and wilder while we have the AI software focusing on grounding it in the physical world.” — Barnaby Legg, senior vice president of Creative Strategy, Legendary Entertainment
ZIVA VFX works by creating physics-based computational models of the natural body features of real people and creatures. Then, the software solves those models with physics-simulation algorithms based on biomechanics and animated movements. For DNEG, this approach was the key to creating the most lifelike Kaiju possible—and in a way that was fast and efficient.
“The creative part of building something like a Kaiju … always sits with the artist,” explains Bowen. “But what’s going on behind the scenes, and the physical properties of how muscle moves, or skin, or even bones or exoskeletons, all of that can be simulated.”
Building and Animating the Characters
It all starts with creating virtual objects in the shapes of bones, muscles, fascia, fat, and skin; these are the building blocks of any anatomical character. With each sequential simulation of animation through the Ziva Dynamics* finite element method (FEM) solver, the physical properties of these materials automatically react and respond to the movement of the creature. This ability was critical to creating the characters in Pacific Rim: Uprising. For instance, with massive creatures like the Kaiju, the physics of hard plating next to supple skin could all be described holistically in Ziva Dynamics’ unified simulation framework, making it easier to achieve greater realism than with traditional methods.
Next, the material properties in ZIVA VFX specify the physics inherent to a simulated volume-object. Through a wide set of parameters and controls, users can describe how flexible, volume conserving, and dense an object is. Users also have the ability to change those physics when needed, which is particularly important in the case of the giant Kaiju, because when animating creatures that are so much larger than their surroundings, there is often a visual discrepancy between their movement and momentum to viewers. While scientifically accurate, it might not feel quite right when watching.
Because everything in the ZIVA VFX system is described geometrically and informed by anatomical knowledge and real-world data, artists can deconstruct and then rebuild a simulation rig programmatically. Ziva Dynamics provides a Python* module, called zBuilder*, that handles the complexity of loading, saving, mutating, and mirroring character-simulation setups. This meant that DNEG was able to generate simulations and dynamics for muscle and skin, which could more easily be transferred from one Kaiju asset to another, thus achieving physical consistency between the creatures and introducing efficiencies into the creature pipeline for each additional Kaiju.
As part of the development process, ZIVA VFX also allows for simulation results to feed into an ML process that can reduce the data model of an asset so that it can be fed into a real-time animation pipeline or game-engine implementation. This process was demonstrated in a collaboration between DNEG, Ziva Dynamics, and Intel during the Sundance Film Festival* earlier in 2018.
“All of our workstations and our render farm are powered by Intel® Xeon® processors. It’s because of that incredible hardware that films like this are possible. If you were to render the entire Pacific Rim: Uprising film on a standalone computer with 24 cores, it would have taken nearly 14 million hours of rendering, which works out to be about 1,600 years.” — Russell Bowen, CG supervisor, DNEG Vancouver
“Some of our shots had maybe 10 Kaiju in them. They had a huge number of characters, and each of those characters had upwards of 1.7 million polygons in some cases—there’s a real computational overhead in each of those characters,” says Adam Vanner, Creature Supervisor at DNEG. “Software like Ziva really helps us accelerate that process and allows us to produce characters in parallel and produce great-looking results with less resources than we had needed in the past.”
ZIVA VFX*, Empowered by Intel® Architecture
Creating a feature-length film with massive amounts of CGI requires equally massive amounts of processing power. To put it all into perspective, consider that the render model of the “Mega-Kaiju” monster was composed of 67 billion polygons. The DNEG team used 25 servers and 3.3 petabytes (PBs) of storage across London, Vancouver, and Mumbai, with more than 1 million daily submissions over the course of production. According to Bowen, “All of our workstations and our render farm are powered by Intel® Xeon® processors. It’s because of that incredible hardware that films like this are possible. If you were to render the entire Pacific Rim: Uprising film on a standalone computer with 24 cores, it would have taken nearly 14 million hours of rendering, which works out to be about 1,600 years.”
To work optimally, Ziva Dynamics software relies heavily on a number of Intel libraries for performance, and users can generate results by efficiently running a number of computationally expensive offline simulations that would otherwise be inefficient. “Intel’s CPU architecture lends itself extremely well to parallelized simulations that need a lot of compute power. Teams using our software running on Intel’s technology are able to decrease turnaround time on iterations of their work, run multiple simulations in parallel, and, as a result, deliver better results in a fraction of the time,” says Michael Smit, CCO at Ziva Dynamics.
The heart of the Ziva Dynamics solution relies on FEM physics simulation and ML algorithms to create and animate characters, which are run on Intel Xeon processors and Intel architecture–optimized frameworks.
Within ZIVA VFX, the Intel® Math Kernel Library (Intel® MKL) PARDISO* solver solves linear system equations in the FEM simulation software. This library features highly optimized, threaded, and vectorized math functions that can help improve performance. Without PARDISO, Ziva Dynamics estimates that solves would take twice the time they currently do.
Ziva Dynamics also uses Intel MKL Basic Linear Algebra Subprograms* (BLAS*) for matrix multiplication in Ziva Real-Time*, its real-time player, and asset-conversion process. With this product, users can take a virtual human “offline” asset and convert it into a character that runs in real-time environments. Additionally, simulation training designed to reduce the complexity of the data model and associated computation uses Intel MKL Linear Algebra Package* (LAPACK*).
Intel® Threading Building Blocks (Intel® TBB) is a widely used C++ library for shared-memory parallel programming and heterogeneous computing that provides a range of features for parallel programming. Intel TBB enables greater parallelization in both ZIVA VFX and Ziva Real-Time.
Finally, Intel® VTune™ Performance Analyzer assesses and fixes suboptimal computation speed, and Intel® Inspector assesses code correctness.
Extending the Capabilities and Reach of CGI
Ziva Dynamics solutions, underpinned with Intel® technologies, are changing the way many studios are approaching their CGI character building. As Facey says, “With Intel and Ziva at the core of our creature pipeline, DNEG is rapidly unlocking new production technology pathways to creating higher quality visuals.”
More Lifelike CGI Characters
With Ziva Dynamics software and Intel technologies, the aggregate results of the animations are based on physics—not on artistic interpretation. As a result, it’s simpler to achieve lifelike characters and physical characteristics when using computational models. Because the software is written against a number of Intel frameworks, users can generate results by running against a number of simulations, which have historically been extremely expensive, but can now be run across Intel server farms and workstations. Not only does this extend the possibilities for VFX studios, but it also enables anyone with a good computer (and access to a cloud render-farm account) to do higher-quality simulations for less.
“Legendary looked to DNEG to bring these characters to life with a level of realism that went far beyond anything we’ve done before. And in turn, DNEG looked to Ziva software to really empower that vision with the highest possible level of visual fidelity,” says Legg. “Tools like Ziva are smart enough to know how the physics of the real world work, and they can save a lot of time for our artists in terms of creating some of those animations on the fly. Our films are just going to get cooler and weirder and wilder while we have the AI software focusing on grounding it in the physical world.”
Higher Productivity
Underpinning the solution is the fact that, with Intel Xeon Scalable processor–based servers, multiple simulations can be run in parallel, which in turn enables incredible productivity for the studio. Not only does this get products to market faster, it can substantially reduce overall labor costs. In addition, the application of AI and ML to character creation frees artists from the painstaking shot-sculpting associated with hitting the right poses and jiggle in traditional rigging.
“Given the large scale in these Kaijus, who are 300-foot tall, iterating on how a muscle should move correctly with standard traditional software leaves you little time for the creative process,” says Bowen. “Ziva enabled us to get there faster. We were able to jump out of the design process sooner than we normally would and really start putting these creatures into production.”
In fact, according to Vanner, using Ziva Dynamics software enabled DNEG to reduce task times by nearly two-thirds. “For Pacific Rim: Uprising, we used Ziva to accelerate our production process. We were able to use Ziva in a way, which meant that we could take a task, which would maybe take two or three days, and produce a result in a single day. So we were able to get much higher-quality results much more quickly.”
This reduction in task time also translates to real benefits not just for artists, but for film companies. Legg adds, “Tools like Ziva are going to make production cheaper, which frees us up to divert those resources to bigger and better entertainment products for our industry. So that’s going to make a huge difference to the production pipeline, to the production schedule, and ultimately to the bottom line.”
Shortened Creative lterations
Adding AI and ML to character rendering means that many lower-level animation tasks are automated. As a result, skilled creatives can spend less time on repetitive 3D design tasks and more time focused on fine-tuning their art and achieving their creative visions. Using Ziva Dynamics software dramatically reduced the time between iterations. “Any way in which we can take something, which is a laborious, manual process and turn that into something, which is an automatic batch process means we can spend more time focusing on the creative aspects of the problem,” says Vanner.
Reusable Transmedia Assets
Assets created using Ziva Dynamics software are usable across multiple media, which makes porting intellectual property from film to film or on to video games faster and less expensive. In fact, the results of Ziva Dynamics simulations are great inputs to any kind of AI or ML process, which sets the stage for tremendous possibilities for content creators and developers across a multitude of industries that need to generate data representing the realistic movement and performance of creatures and humans.