VFX Production of Avengers: Endgame

Introduction

Hello!  My name is Dan DeLeeuw and I am the VFX Supervisor for Avengers: Endgame.  I have been interested in VFX from an early age starting with being a fan of Star Wars and Raiders of the Lost Ark.  Early on I would devour anything I could to help learn about the craft. The book "Techniques of Special of Effects Cinematography" was invaluable.  It is very outdated today but taught me a lot of the basics. I studied computer graphics at University and started with Dream Quest Images for my first job.

About a Shift From Practical To Digital Effects (1993-1995)

When I started at Dream Quest, we still had a model shop, machine shop, camera department, and a matte painting stage.  Miniatures photographed with motion control and composited optically were the norm. I started as the second employee in the digital department.  Our digital scanner was built on an optical printer bench and we filmed out on a solitaire. I’ve been in a position to see it all change. Digital composites and matte paintings were some of the first parts that took over from the practical techniques.  Eventually, digital models and creatures would take over as well. As digital grew more and more of the departments and floor space were taken over by computers. We definitely went through a learning period where we were trying to do more digitally. It was a great time when we could use the best tools from all the different disciplines.  By 2001, when Dream Quest was merged into the Secret Lab we had moved away from most of the practical solutions. Reign of Fire had the last models we ever shot.

Important Elements of VFX

Having a solid plan will always get you to the finish line.  As the shows have gotten bigger – some being larger than 2000 shots – we need a road map. Pre-Visualization is a key part of that plan. Pre-Vis informs the breakdown – a document that tells us how many shots we have and the type of work. The breakdown can be given to the other departments to know what we need. For instance, whether explosions are practical or which sets need blue screens.  Breakdowns will also be used for the bidding and used as a guideline for the visual effects houses to understand the work involved.

Working on Lighting of the Planets

The look of the planets started with our Art Department and Production Designer Charles Wood. They would create artwork that influenced the look of the different planets. During photography, a mood board containing images depicting the look of the planet was placed on set.  The Director of Photography would use these images as a guideline for lighting. VFX would work with the locations department and find a practical location we could use as a reference. For Vorimir, we used the dunes from Lençóis Maranhenses National Park in Brazil.  The plates were heavily processed to create an otherworldly feel. When combined with particle effects and atmospherics the planet came alive.

Body Morphing Techniques

When Scott shrinks, we swap out Paul’s body in the photography with a digital double.  The Digi-Double is created from a scan of Paul Rudd in costume. The model is textured and shaders are created to emulate the look of the practical costume.  When he shrinks, the Digi-Double is animated to make him appear that he is falling while simultaneously animating the model’s size. We add after images that we called the Disco Effect to emulate the multiple images that you see in the comics.

Working on the Final Battle Scene

The practical photography only contained approximately 17 heroes and 6 members of Thanos’ army in mocap suits.  It was important to give our actors appropriate eyelines so we dressed up 6 stunt performers to charge at them. The practical photography gets a laugh whenever anyone sees it.  We worked with Weta to create the thousands of characters that would exist in the final shot. Weta created behaviors for the armies that would be chosen by the software depending on circumstances in the battle.  In simple terms, the software would simulate a battle by pitting different opponents against each other and calling up the correct behavior to accomplish the attack. In our case, we wanted to emulate the long shots established in the other Avengers’ films.  The One’r in our case involved three practical elements – one for the armies charging at each other, one for Black Panther and his team, and one for Drax stabbing Cull Obsidian. The separate elements were “stitched” together used completely digital environments and characters rendered into cameras moves that would join everything together.  The size and technical complexity of these shots pushed Weta’s pipeline to its limits.

The Creation of Smart Hulk

To create Smart Hulk we started with the original digital model of the Hulk from Avengers.  We knew Smart Hulk would be a combination of Banner and Hulk and would be slightly smaller as a result.  We reproportioned the Hulk’s model into Smart Hulk to give us a feel for this smaller size. Once we decided on the proportions, ILM went in and re-sculpted him from the ground up.  Getting a great performance to read on the digital characters' faces required us to blend in just enough of Mark Ruffalo’s face to be believable. We used the Medusa system developed by Disney Research to capture incredibly detailed per frame geometry of Mark’s face. We could then use the tracking markers from Mark’s helmet-cam to solve for his facial movement and then map that movement onto the digital model.  Animators would then go back and correct the solves where necessary.  

Final Advice for Beginners

There are a lot of opportunities to get into the VFX business.  First off, you can start off at home. Many of the large software packages offer to learn versions that allow you to use the tools that we use. Maya for animation and rendering and Nuke for compositing have personal learning editions that can be downloaded. In terms of free software, you can use Blender and Da Vinci Resolve.  

It’s good to choose a focus for what you want to specialize in.  There are still generalists that can create shots on their own, but generally, artists will pick a particular discipline.  There are basically five main categories – modeling and rigging, animation, lighting, effects animation, and compositing.  Modeling involves the creation of digital models. If you can sculpt or are great at illustrating and character design, you might be interested in this area.  Rigging involves creating the skin and muscle systems and controls that the animators will use to animate with. If you’re interested in animation, try taking acting classes.  Lighting will involve recreating the lighting on set and making the digital models look great. This also involves texturing and creating material shaders to make the models look real.  Effects animators work with simulations to re-create fire, clouds, explosions, etc. Compositors are the ones that combine digital renders and live-action photography.  

Many art colleges offer courses for visual effects and animation specifically.  If you are interested in shaders, a programming background with computer graphics will be helpful.

Dan DeLeeuw, VFX supervisor

Interview conducted by Ellie Harisova