Working on Games and Films with Unreal, Houdini and Blender

Introduction

80.lv: Please, introduce yourself. Where did you study? What companies have you worked for? What projects have you contributed to?

Alvaro Garcia Martinez: My name is Alvaro García, and I'm a director at Mayhem Mirror Studios. I studied at CICE, Madrid, the School of Animation. My background actually comes from sports and martial arts. However, I changed the direction when I studied at CICE. For some time, I worked at Fusion Animation and later in VFX at MPC, DNEG, and ILM. The projects I worked on include movies like The Jungle Book, Pirates of the Caribbean, Transformers, Alien, Wonder Woman, Ready Player One, Life, Fast and Furious 9, The Matrix 4, Avengers, and Infinity War. That's more or less the entire professional background.

Simultaneously, I've been always working on my own projects. This includes Sumer in 2014 and The Seed of Juna in 2020 – both were pilots that currently developed to bigger projects like feature films. Last year, I turned to game design as well. This includes Crete, which is a biopunk prototype multiplayer and a story-driven game at the same time. I always move in-between departments and specializations.

Mayhem Mirror Studios

80.lv: How did the story of Mayhem Mirror Studios begin? What's the story behind the project?

Alvaro Garcia Martinez: It's a very interesting question. It started obviously with the pandemic – many companies started at that moment. I was working at DNEG with real-time solutions. I felt there were not enough projects, or speed, let's say, in the industry itself. As the animation business is huge there are giants in the industry, but the problem with the giants is that sometimes they move slower because such big companies have to approve many things to try R&D or real-time. So, I decided to quit DNEG as I thought that I could do the things that I had done in the companies completely on my own.

The nature of real-time is that as long as you have an idea, you can reproduce it quite fast. I decided to quit not only DNEG but the whole VFX circle – the classic DNEG/Framestore/MPC/ILM. So, I founded my own company to give it a try and build The Seed of Juna in real-time. I thought: "I've already spent 10 years working on the stories of others and now I want to create my own games and my own stories."

Main Strengths of an Artist

80.lv: What are you specialized in? What are your main strengths? What projects have you contributed to as a team?

Alvaro Garcia Martinez: My main strength is not to be a specialist in anything at all, and in fact, I don't have any specialist strength more than having a holistic approach to everything. This, sometimes, in VFX is not as simple as in other specializations. I basically work with most of the departments inside VFX, but the problem is I have always wanted to do more. Obviously, in companies, this is quite limited by the nature of the pipeline itself unless it's a very small show.

In those projects, at some point, they were surprised that, for example, when I was working on crowds, I was able to do cool presentations inside ILM because I was able to do good renderings. I was also able to do nice motion capture sessions and put things together and present almost a pilot, which wasn't expected because we were doing only crowds. That’s when I started to realize that I needed an environment where I could do most of these things. And real-time was perfect for me because, in only one shoot, I could do most of the things I always wanted to do.

If you are doing your projects, it is quite important to be completely agnostic to any department and just focus on what you want to do, what you want to tell, and what your aim is. Even if it's a very abstract answer, I think it's one of the most powerful things you can have as an artist. Don't be specialized at all, even though it's against most of the bias of anyone who wants to enter the industry. If you want to enter the industry, be specialized in something. If you want to survive as an artist, don't specialize in anything at all. That's how you put the balance there.

Unreal Engine

80.lv: Why did you choose Unreal as the main tool? What makes it a perfect match? What other tools do you use at your studio?

Alvaro Garcia Martinez: When I was working on Ready Player One, one of my friends, Ruben, knew that I was doing The Seed of Juna with offline renders like Arnold in Maya. At some point, he told me: "Hey, have you thought about the idea of doing The Seed of Juna in real-time?" At that moment, that was a crazy idea, but I have this kind of thing that when someone tells me something, even if I say no, it's bugging me.

I couldn't do the entire The Seed of Juna in real-time because we had already got 80% of the production. What we could do was to make a trailer with a specific Unreal Engine shot of The Seed of Juna. Recently, I came across the Facebook post I put two or three years ago. It was a picture of a blackboard that reminded me of the experience I got after making the trailer in real-time when I thought that if I had to start a new project, I’d have to use Unreal Engine as the core.

That was the very beginning, the early first draft on a blackboard of how the future was going to look like. What most of us would take for granted, at that moment, was me in a corner with a blackboard drawing Unreal Engine in the middle and then a lot of arrows pointing crazily to many other software programs. Again, there was no pipeline defined yet, there were no projects of real-time for feature animation. It was a very isolated feeling, but at the same time, pretty exciting.

That's why it's called workflow, it's not actually a line. It's a flow. For instance, I just had a small task in Avengers: Infinity War. We got two days to deliver 30 shots. The pipeline was slow. We needed to deliver things, release and approve them. In the end, we just sent links by email with Alembics. And we did that in just one evening.

Sometimes, the workflows or the pipelines are quite flexible. That's one of the things I learned from VFX in small teams. I brought that to my company thinking: "Yes, we got a little bit of a pipeline, but I'm not worried or scared to do things differently. Don't finish that model, give me a block out, I will find a way of replacing this." I had to iterate as fast as I could.

Mocap Workflow

80.lv: Your reel also shared a look at your work with mocap. What rigs are you using? 

Alvaro Garcia Martinez: In some videos, you can see optical solutions like Vicon. This was a collaboration. I was lucky enough to have the studio of ILM do the motion capture of The Seed of Juna. Ben Morris, the Creative Director of ILM, told me: "Hey, just stay one day and shoot”. And that was it. It was great. Obviously, I'm going to be always thankful for that because, to me, it was the first experience directing a mocap session in a bigger studio, not a living room.

Since then, I've worked on different projects in mocap at ILM like Ready Player One, etc. That's, again, the origin and the seed of my passion for directing motion capture. During the pandemic, we got an Epic MegaGrant in Mayhem Mirror Studios. That was a big reason for buying an Xsens solution, which is worth more or less the compromise of price and quality. This was more or less what we needed. The results were 90% optical operation adjustments.

My workflow basically is just preparing the takes I need for the next day, putting that in front of the screen, depending on what kind of takes those are. Sometimes I split them into parts, trying to always start with medium intensity, then high intensity, then low intensity.

Later, I do all the takes and that goes to Houdini. In Houdini, I do the retargeting with KineFX, which, in this case, we are using for a combat system. What we need is not just doing the retargeting, but also doing motion editing, acceleration, and breaking it into different parts if you are doing combos, so that if you press the button, the combo keeps going. That's why I create all the technical logic of the animation inside Houdini. That goes later to Unreal Engine 5, where we’ve got the characters waiting for the animations to do fighting, loops, or whatever action we need for the gameplay.

Besides, as we have a lot of cinematics both in The Seed of Juna and Crete, we have this specific workflow: we put the different takes together to create our cinematics. Again, we do it in Houdini, at least merging all the takes together to see the action and that goes to Unreal Engine. Simultaneously, we record the facial capture. Sometimes in a different take, sometimes at the same time. The facial capture goes with, in this case, Facegood.

Sometimes it goes with the same body take, sometimes it goes in another layer if it's, for example, a fight, and I just want to put expressions of anger and pain. I have a library of pre-recorded facial captures that I did with Facegood, and that goes directly to Unreal Engine.

MetaHuman

80.lv: What's your take on Epic's MetaHuman? Have you tried using this tech? What do you think about its possibilities?

Alvaro Garcia Martinez: Yes, I was lucky enough to participate in the closed beta, so I got a little bit of advantage in using the technology before the final release. One of the first things I noticed or one of the first things I realized the first day I used MetaHumans is that I needed to use it in my project, Crete. The problem was we had already got a stylized design. However, I was eager to use MetaHuman, so I thought this would be another challenge for me.

The first thing we tried was to not go completely stylized, so we could still use MetaHuman. At some point, I felt like we needed to push this even more. I reached out to Emilio Ferrari – a producer that I had always wanted to work together with. He posted a video on LinkedIn showing customizing MetaHuman rig, not just with the same topology, but with a completely different one.

You can put a cyclop with one eye, or you can put a dog with completely different topology and polycount and transfer all the blend shapes into the custom mesh. That was very impressive, so I reached out to him. We decided to work together. So, we took a ZBrush custom mesh with lower polycount from Ivan Lopez, the modeler of our main character Damien, and transferred all the architecture of MetaHuman into our head and face. When we managed to do this, we were absolutely over the moon. We were super impressed that this was working really well because we stylized the shapes in order to have harder edges. We got our own shade and there were different wrinkles, like in Spider Man: Into the Spider-Verse. I think you gain success when you use technology with a stronger direction, you have all the advantages, you have everything on your side.

MetaHuman itself is a really groundbreaking tool and technological achievement. It's a wonderful tool that, however, needs proper art direction in order not to feel like everything is a clone of itself. I can compare it to a space rocket: if I give some people a space rocket, not many of them are going to land it on Mars. It takes a lot of work if you want to have, let's say, polished results.

Main Tools

80.lv: What tools do you develop to speed up your projects? Have you tried experimenting with different procedural workflows? Also, what's your take on Houdini and open-source tools like Blender?

Alvaro Garcia Martinez: On the blackboard, I've mentioned before, I’ve already got two other different software programs apart from Unreal Engine. One of them was Blender and the other one was Houdini. It's exactly what I'm using right now. In the case of Houdini and proceduralism, it's quite its own definition. It really helps with the procedures that you don't want to do again and again. If you are in a small studio, and you need to create, for example, 100 plants, you just need to create one plant with specific rules and then you start iterating with different rules and states to have different variations.

That's where proceduralism is quite relevant for small companies – not just for simulation or using Houdini for effects – it's basically used for world-building. For instance, now, I'm working with the fibers, the set dressing in Houdini, and also, with retargeting. These are more or less the three areas I'm using Houdini for Crete.

I used Blender for keyframe animation, quick prototyping, and modeling. Our artists use different tools that I think are even more important than my expertise or my experience using Houdini or Unreal Engine. They use 2D art. This includes concept art, sketching, and storyboarding.

Without this, you're going to land into the Mr. Generic prototype. You need a lot of 2D art in order to have a beautiful project. This is something I want to highlight because it's so easy to open Unreal Engine and to drag and drop Quixel Megascans and have a shot in minutes, which I see as a little bit of missing art direction in most of the cases.

To me, one of the most powerful tools is a piece of paper and a pencil, digital or not. You need to be a 2D artist or you need to have one around you if you want to deliver something special and not another copy-paste. That's more or less my take on that. Again, these tools are really powerful and amazing, I also do programming in C++, I can do a lot of blueprints in Unreal Engine, but it doesn't matter if it's not self-contained into a powerful idea that appeals to the audience.

Alvaro Garcia Martinez, Director at Mayhem Mirror Studios

Interview conducted by Arti Sergeev

This interview was conducted during the December 2021 RealTime Conference