Uncharted 4: A Thief’s End is one of the most technically advanced games of this generation of game systems. To build
Uncharted 4: A Thief’s End is one of the most technically advanced games of this generation of game systems. To build this complex project Naughty Dog used a number of great tools to build this huge project, including Houdini. In a recent interview to SideFX two members of Naughty Dog’s team, dealing in VFX, Neilan Naicker and Raymond Popka talked about the way they used Houdini with Uncharted. It’s a great read especially if you’re into finding new ways of using Houdini. This tool is actually pretty affordable for indies now ($200 per year, which is crazy if you think about it).
u4_e3_1920x1080
Hello Neilan and Ray – first things first, a big congratulations on the release of Uncharted 4: A Thief’s End! It’s been quite the journey, no doubt. We are thrilled to have the opportunity to chat with you guys about some of the tremendous work that went on behind the game, so thank you for taking the time out to talk with us! Before we begin…Neilan, Ray – please tell us a bit about yourselves.
neilan_popka
Left to Right: Naughty Doggers Neilan Naicker, Dynamics Artist and Raymond Popka, VFX TD
NEILAN: I’ve been a Dynamics Artist at Naughty Dog for just over five years now, I moved here from South Africa. “Dynamics Artist” is a weirdly specific title, but I’m basically a Technical Artist who’s geared towards physics simulation. I create and script realtime physics objects that react to the character in the game, as well as pre-baked destruction and simulations, usually in Houdini. I also do tool programming for our 3D packages to support other artists.
uncharted-4_drake-scaffolding1
RAY: Hi, my name’s Ray and I started at Naughty Dog in the fall of 2014 as a contract FX Artist. It was my first Games job, coming from a pure post-production background.
How long have you been using Houdini?
uncharted-4_drake-looking1
NEILAN: I started learning Houdini 8 in around 2006 at my previous studio, BlackGinger in Cape Town, focused on film and commercial effects.
Back then, most of what I was working on was RBDs [Rigid Body Destruction], POPs [Particle Operators] and SOPs [Surface Operators], but I started picking up fluid and smoke simulation as they became available in later versions.
RAY: I’ve been using Houdini for about 7 years now; 5 years professionally and 2 years as a student.
When and how did you first come to discover Houdini in the world of 3D?
NEILAN: Weirdly enough, aside from very briefly for university projects, Houdini was my first real exposure to working in 3D at all. I came at the industry from a programming angle, which is maybe why Houdini seemed like such a natural fit. My previous studio hired me to be a tools programmer, but they had a small but growing department of Houdini enthusiasts. Houdini seemed exciting to me, so I did everything I could to learn it when I could.
2016-06-16 09_26_29
RAY: I found out about Houdini from my school, Savannah College of Art and Design, where they taught two courses in Houdini. I spent a winter break learning the basics before starting the courses, which were taught by the prestigious Ken Huff. Previously I had been a general CG artist, using 3D software like 3ds Max as far back as middle school, but Houdini was a completely different way of thinking. Once I went Houdini, I never looked back.
uncharted-4_drake-truck-leap1
What are some past projects you’ve done prior to joining the prestigious ranks of Naughty Dog?
NEILAN: When I left in 2010, BlackGinger had done a lot of high profile commercials and three international films. Most of them featured Houdini effects heavily, lots of RBD, particle and smoke simulation work.
RAY: I previously worked at Rhythm & Hues, The Mill, Psyop, Method Studios, and a few others. I did a mix of commercial and box-office work including Life of Pi, Percy Jackson: Sea of Monsters, Hercules, and the iPhone 5C reveal.
Can you tell us an overview of the pipeline at Naughty Dog?
RAY: Back in Naughty Dog FX land, we have a ‘do whatever it takes’ approach to making art for our FX. Within our artist team we use pretty much every 3D package and plugin for FX to make art; Houdini, FumeFX, Krakatoa, RealFlow, NUKE, 3ds Max, thinkingParticles… everything you can think of. We render out static images or image sequences and turn them into textures and flipbook images.
Then we have proprietary tools to script our particle behaviors and custom shader writing packages for the complex materials our FX use in-game.
NEILAN: Our pipeline is largely Maya based. Generally, whatever we work in has to be converted to a Maya scene to get it into the game. When I first joined, I was one of maybe three people at most who used Houdini at all, so I wrote a lot of scripts for getting our game data between Maya and Houdini. That’s about the extent of my workflow, as I’m usually working in one or the other, but the studio at large uses a lot of ZBrush and Substance.
uncharted-4_drake-sully-stairs1
How many Houdniks are at Naughty Dog, if we can ask?
NEILAN: Not as many as I’d like! Raymond’s probably the most active Houdini user who’s joined the studio, but right now there are maybe four or five people that use it with varying degrees of frequency.
RAY: Houdini is pretty scarce here; before I came, Neilan and my lead Eben were using a license of Houdini 10 for various purposes. I convinced them to get us a master license of the latest and greatest, which I found crucial in my R&D adventures.
uncharted-4_drake-sully-farm1
All of your trailers and videos are captured in-game, so the cinematics are all taken real-time from your game engine, which kind of removes any post effects that Houdini would normally be used for. So…what was Houdini used for in Uncharted 4?
NEILAN: On my end, I’m mostly using Houdini to create pre-baked animations, either through DOPs [Dynamic Operators] or SOPs [Surface Operators], which then get transferred to our engine via joint animation in Maya. On a game like Uncharted 4, that’s mostly rigid body fracturing and destruction. Then there were a few little SOPs networks for things like cloth deflations and moving liquids.
Having worked on the in-game dynamic rivers and deforming water, can you share with us the process it took to generate the necessary flow maps for them?
figure-1a
Background geometry (Top Image) and sculpted base water height (Bottom Image)
RAY: The dynamic water system in Uncharted 4: A Thief’s End is very complex and required a large amount of setup work to get working.
From the Houdini standpoint, it was primarily used for texture generation and determining parameters for the generation of the meshes in-game. I would start off with the background geometry and start with a rough water guide from design, as water levels needed to be highly art-directed. I would then start to sculpt the base water height as I saw fit.
figure-1b
figure-2
Since we could only use 8-bit textures we had to render the 16-bit height encoded in a 2-channel 8-bit map. To generate the flow maps, I used a mix of simulation and hand drawn curves to fill the needs of the art direction. For the large level chunks I would run a simulation to get a nice reference of the flow intensities and directions, then hand draw them with modifications as I saw fit.
Initially I tried to just use the raw flow from the simulations but there were always issues that I would have to paint over or redo. Drawing flow curves is very meditative :-). I would then paint in the flow intensity and wave height to suit the environment and gameplay. The end result was a texture with flow direction and intensity in RG channels and wave height in the B channel.
figure-3a
Simulation for flow map generation
figure-3b
Hand drawn flow curves
figure-5
Bounding boxes for generating in-game meshes
I would also export a normal map and alpha map to improve the look of the river. Once the textures were ready I would need a bounding box set to generate the in-game meshes, which was easy to deduce and export within Houdini ROPS [Render Operators] (right image).
figure-4
Texture with flow direction & intensity in RG channels, wave height in B channel
All that, coupled with a water uber-shader, written partially by me, gives us in-game rivers and oceans (see images below). In the end it was a relatively cumbersome pipeline that I plan to renovate and streamline for the next Naughty Dog title, both art side and pipeline side. It is worth mentioning though that once you had a set of textures in the game for a river you could paint in Houdini and have it update in-game relatively quickly, making art iterations pretty smooth. We don’t have a game ‘editor’ of any sort so this was the next best thing.
Overall I would say Houdini gave me a very quick turnaround for developing all the components and data needed for our rivers, there were at least 15 large river chunks that had to be hand painted. So having everything in one place as opposed to an array of python tools in Maya was nice, as well as being able to easily modify the setup whenever I felt it necessary. The power of Houdini workflow…
figure-6a
In-game river
figure-6b
In-game oceans
What else did you do with that power?
RAY: Texture based vertex animation is something that Unreal had been flaunting in their dev blog so I looked into creating a similar system for background animals. Foreground animals are actors with skinned meshes which we can only handle so many in game. Vertex animation allows us to make hundreds of animation geometries in-game with little graphical / computational overhead. I R&D’d a network to export alembic animation almost entirely in COPs [Compositing Operators], and only required a second UV set on the geometry to animate correctly; COPs are super underrated. I used our particle system to spawn thousands of running chickens.
figure-7
Something we could never do with the traditional skinned actor pipeline. In production we made most birds, and some ground animals with vertex animation, we even made a background crowd for the city tower background.
figure-8
Eventually, we developed a python toolset out of Houdini so other artists could use the system, as our geometry files are based on Maya scene files and more difficult to integrate with Houdini than something like Unreal.
Are there other prototyped systems you developed during the production of Uncharted 4?
The impact-particle emitter system was a novel R&D effort that was used in a handful of animations for our E3 demo. I had a system that would analyze pre-simmed destruction animation from foreground for contacts and output data for spawning effects in-game during animation. (EFF was the name of our scripts that specify what particles to spawn and when, during animations in-game). It was a nice tech demo which I attempted to port to Maya via Engine but found pipeline-breaking issues with how Maya handles animation/caches and Houdini, like low level stuff that probably won’t be fixable, sadly. I might revisit it in the future, if the project calls for it.
Neilan – you were in charge of the destruction side of things… and certainly there was quite a bit of it throughout the game. Were there any tricky ones that we can keep an eye out for?
NEILAN: There are a few wall breaks and collapses in the Madagascar market fight, as the truck’s crashing around, those had to be done on a strict time schedule for E3, so that was tricky. There’s this wooden rope bridge that breaks in the island chapter (“At Sea”), which took a while. The animation for the character running and jumping had already been done, so it was a lot of very specifically timed glue and constraint breaks of planks and ropes. To make things worse, it transitions to a real-time physics setup halfway through the collapse, so it had to hit very specific poses and velocities to keep the transition as smooth as possible. That took a long time to get right.
Please tell us about the workflow of generating the destruction in Houdini and then integrating that data into the rest of the pipeline.
office_houdini
NEILAN: When it comes to fracturing, I use two different approaches. Either I’m cutting very precise shapes to match a texture (or because the sequence happens close to camera), which means I’m cutting by hand, usually in Maya. For larger jobs (or where the texture detail is some distance from camera) I rely on Houdini for fracturing. I made an HDA that encapsulates a lot of my routine workflows for clustered fractures, and adds edge detail as efficiently as possible.
What is this fast fake fluids trick that you did, for simulating animated drinks?
NEILAN: There are two different techniques I came up with, one’s a realtime method for the crowds in the auction that the player can bump his way through, but we also needed some sloshing and pouring liquids for a few closer shots of whisky and wine glasses in cutscenes.
glass_network
I decided we’d do them as prebaked animations, so for a few, I got a little bit crazy and did some DOP fluid simulations, but there was no way I was going to have enough time to do that for all of them. I hacked something together that worked out pretty well, it’s just a constrained Bullet rigid body on a pendulum which drives a water surface, with SOP noise for sloshing based on the velocity of the pendulum.
It let me get away with a lot of keyframing, like easily raising/lowering liquid levels or motion at specific timings, without having to deal with all the eccentricities of fluid simulation. So that made it extremely fast to turn around in Houdini, then every “simulation” is shrinkwrapped with a fixed point count surface so we can get it into Maya as a joint animation.
How do you go about troubleshooting things in Houdini?
NEILAN: Forums, mostly. The SideFX forums and OD|FORCE are great resources!
Which version of Houdini are you using? …and how many versions of Houdini have been gone through since the start of Uncharted?
NEILAN: When I started at ND in 2011 (mid Uncharted 3), we’d been using Houdini 10 – I think that’s the earliest version that had been used at the studio – and we used Houdini 15 to finish out Uncharted 4.
uncharted-4_truck-street1
You guys are probably kicking back and maybe didn’t have time to check it out, but if you did – what are your thoughts on the recent release of Houdini 15.5?
NEILAN: I wish I’d had more time to take a proper look, but I love the look of the new modelling tools. I’d also love to play around with the crowds stuff – that’s a personal interest of mine, and I’d love to find a work related reason to dive in too.
RAY: I haven’t had a chance to play with Houdini 15.5 yet, and the new features don’t really pertain to my work, but I’m sure 16 will have some cool new features I could use.
If you could be granted two things to change or be created brand new in Houdini, what would they be?
NEILAN: I wish I felt more inclined to do more cloth work in Houdini. Truth be told, it might just be that I haven’t really gotten to grips with the FEM solver, but I find that I’m more likely to stick to Maya for cloth, as it makes our workflow easier. I also think there’s a lot I don’t like about rigid body solves right now. It’s hard to beat Bullet for speed, but I find it can be too volatile to get stable, good looking results close to camera. It’s hard to go back to RBD given the speed difference.
RAY: I think it would be nice to have some kind of super optimized version of the smoke and flip solvers which reduce the flexibility and customization for making fast, simple simulations. A lot of work I see my coworkers make in FumeFX are relatively simple setups but are in general easier to populate and quicker to sim in that package. I know the shelf tools have been getting better and we do have partial GPU simulation, but they should keep pushing that area for more streamlined setups.
What and/or who inspires you in the work you do today?
NEILAN: Every now and then on the forums I catch a glimpse of some of the systems people are creating great work with, like crowds, or sand/fluids/pyro and I super wish I had the time to play around with those solvers! Otherwise, playing other AAA games, and seeing what other VFX studios are doing with technical effects is really inspiring.
RAY: I personally find inspiration in a lot of the SIGGRAPH white papers and dev talks with other game studios. They’re a bounty of bleeding edge technologies that may not necessarily be ready for everyday use, but are definitely something to aim for in terms of future innovations.
uncharted-4_drake-close-up
Any words of advice for people just starting out in Houdini?
NEILAN: Stick with it! I don’t think it’s unfair to say there’s a relatively long learning curve with Houdini compared to other packages, and that can be discouraging, but if you put in the time to understand its systems thoroughly, it absolutely pays out. Also, learn Python/VEX!
RAY: I would tell people learning Houdini to try to do as much as possible without simulations. Learning to do things with just SOPs will make you a way more in-depth user than someone who just runs simulations all day.
Aaand lastly.. what games are you playing these days? PC or Console(s)?
RAY: Game wise, I’m a fan of adventure, strategy, roleplaying games. My favorite recent games have been Overwatch, Stellaris, Fallout 4, The Witcher 3, Metal Gear Solid: Phantom Pain, oh and Uncharted: A Thief’s End
NEILAN: Yes! PC and PS4 mostly. I just finished a really long XCOM 2 campaign, and I’ve been catching up on some indies I missed. Right now, I’m looking forward to playing a lot of Overwatch!