LogInFreek Hoekstra talked about his passion for Houdini and mindblowing experience of developing and promoting procedural methods for next-gen game development.
Technical artist Freek Hoekstra talked about his passion for Houdini and mindblowing experience of promoting new procedural methods for next-gen game development that allow large game studios to optimize the workflow immensely.
Introduction
80.lv: Could you tell us a little bit about yourself? You’ve started your work a couple of years ago working on a lot of EA games, plus you’ve been an intern at SideFX. A lot of work was going on there. Could you tell a bit about your career?
Yeah, so I’ve worked at EA and SideFX as you said, working on titles like FIFA, UFC, Need for Speed and more and if you have seen my work before it’s probably the one below. If you haven’t seen it, it’s probably also worth a quick look as it describes what I do far better than words can.
Everything started when I was about 8-9 years old, and my friend had a game Command & Conquer: Red Alert. We were obviously young and crap at the game, but tried to figure it out (while learning English, too) and finally found out that there was a level editor. We started coming up with ingenious ways of torturing the AI players and giving ourselves more resources as we saw the AI’s armies relentlessly explode into our ever increasingly ridiculous defenses. Shortly after my school notepad was filled with level designs and ingenious master plans befitting an evil genius. But it was a good start, and by making the game unbalanced in all kinds of ways I learned a lot about game balance in general.
When I was about 11-12 I got a copy of Unreal, and it had a level editor, too. It was way more complex and had all kinds of bugs with the BSP editor, but there was a Dutch magazine, Computer Idee, that had a 3-page introductory tutorial, so I went to the library downloaded the editor on 3 separate floppy disks and got to play around. My computer was awful so when Unreal Tournament 2003 came out I couldn’t hope to run it, so I dabbled a bit with Half-Life and whatnot. When I was like 15 years old and finally started working, I got my hands on a faster computer and a cheap copy of UT 2003. Soon UT 2004 followed and it came with a version of Maya PLE (by the way, Autodesk, you should bring PLE back).
Maya PLE was a free personal learning edition and UT2004 came with a whole raft of Unreal tutorials made by 3Dbuzz. This was a treasure trove of 3D modeling, UVing, basic animation custom model importing, basic scripting, even setting up a basic vehicle. I hit the game development student jackpot, and my grades soon suffered accordingly as a result (i.e. greatly).
I then managed to form a small mod team and started working on a mod for Genome, which was basically an Unreal 2k4 full conversion that had insect-like monsters with no ranged attacks, but they were plentiful and would try to overwhelm the player in close combat. We were a small team but we learned a lot.
This work was ultimately abandoned, but it proved good enough to get into the NHTV, IGAD program (now University of Breda).
University
I visited a few programs, but IGAD stood out. The supervisor Andrew Paquette showed me some students’works done after only a few months of instruction, those were photorealistic vehicle models in NURBS. That convinced me that the school wasn’t wishy-washy and I had to go there. So I did. I had to choose art or programming, and as a level designer who had been mostly ignoring high school, I figured Art would be best. Also, I was bad at math, so it was obviously art.
For roughly 2.5 years we were taught modeling, rigging, texturing, animation, scripting, and engine basics. We would also be paired with programmers and stuck together 1 day a week for the game lab to make a game in 14 weeks (so in fact, 14 days). After 14 weeks we made this:
It might not look that good, but for a team of 5 people working with almost no tools or experience making this in 14 days was pretty awesome. I made the characters, their rigs, and animation, plus some more modeling and level design work. Overall it was a great success.
The following years I did pretty well, specializing more in game design and level design as well as the traditional art courses.
I then met Kim Goossens. Kim is a special guy, let’s say that. He taught me very high-end rigging (in which I wasn’t that good) and then introduced me to scripting in Houdini. I was terrified, I knew I had dyscalculia and, my god, code? But Kim relentlessly let me suffer and am I glad he did. I made a procedural bridge, fairly simple but detailed. It took me weeks of headaches to get into it but I got it, and that was neat.
I then had my internship at Playlogic Games working on Fairytale Fights. It was an interesting internship, but then disaster struck. I hit my head, REALLY hard. This is actually the first time I write publically about it, and it’s still pretty difficult.
I got tunnel vision, lost the ability to play guitar, and after a few weeks, I ran into a friend, who ripped my phone out of my hands after I pulled up a picture of her on my phone because I failed to recognize her. I hit rock bottom.
I finished my internship, but it didn’t go well. As I had to specialize, I came back to level design, but even that felt wrong. Nothing I made looked like I wanted it to, and I had to take a break. I broke down at school and cried for like an hour. Thankfully my peers were great and consoled me, which made me feel loved. My parents had had enough, so they said, screw this, you need a break, you’re coming with us to New Zealand. We went there and for 2 months I had no computer, no internet, no phone, no nothing, just me, my parents and the world. And it was exactly what I needed.
But I forgot so many things… how was I ever going to be good at this stuff again.
When I came back, I found out that I had 2 days to decide what to do again or wait another 6 months. So I thought let’s do level design again. But then I asked my 2 most trusted professors, Andrew and Kim, for a piece of advice, and Kim said he had a few nice tutorials on CMIVFX on road creation and suggested trying it.
I didn’t just want to do a tutorial, so I decided to make an automated railway system using some of Kim’s techniques and also inventing some new stuff. Everything went well and I got 9/10 mark and was pleased that I had proven I could do good work again.
*this image was created by intersecting 2 lines representing the railroads, everything else is completely procedural: the nuts, the bolts, the electric lines, the intersection generation, everything! Find the full write-up of the project here (PDF).
I was ready to graduate, but I was still squeamish about proceeding in proceduralism so I chose to revert to my specialty (Level Design and Detailing). To it Kim did something rather unexpected: he said NO. I was furious. Kim had no legal authority to decide that for me, but he said that I needed to do procedural level design, as I am good with Houdini and should use it. Incensed, I wrote him an angry email mentioning that in Houdini one cannot make stuff that can be used in an engine like Unreal as it was a closed source. Kim calmly wrote back: never get emotional in an argument, give me 2 days.
He worked with Jan Pijpers, and together they figured a way out. Two days later there was a prototype of a simple asset being placed in Houdini and getting used in Unreal. It was very rudimentary, but it proved it could be done. This was the moment I realized I was wrong, and I decided to trust my professors one last time. It has been the best decision of my life.
I then went on and made a tool to create a level inspired by games like Uncharted in under an hour. The idea was simple: just specify the big strokes to have control and let the computer do the tedious work of placing all the plants, trees and blades of grass, UVing of the terrain, optimization, etc. Find more information here (PDF). It’s another full writeup so with this you should be able to recreate what I did!
I graduated with a 10/10 mark and got job offers from all over the globe which left me very unsure about what to do. Not sure what to do, I decided to say no to all of them and have an internship at SideFX software instead. I figured it would be a 6-month test to see if I could live abroad or I’d be homesick.
Eventually, it turned out to be way more than that, as I got a chance to make a City Generation demo and present it at SIGGRAPH, the worlds foremost 3D graphics convention. The talk I presented there can be seen here:
I was also able to show it to some game developers who ended up customizing the tech for their games. Naturally, it drew even more attention to me.
After this demo, John Courte, Ken Xu, Judith Crow and I sat down and talked about the drawbacks of the workflow. One of the main issues was going back and forth, between DCC and the engine, or say, Houdini and Maya. Ken made a bold suggestion to embed Houdini into these engines. Another intern (I believe, McKendree Schilthuis) had made a crazy prototype before and it inspired him. It was a crazy idea that ultimately became the Houdini Engine and made Houdini available in Maya, Unreal, Unity or any other platform you wanted.
After my internship at SideFX, I went to EA, hoping to work on SSX with Caleb Howard who had given a wonderful talk at SIGGRAPH about volumetric modeling. Find the video here.
SSX, however, didn’t get a sequel, but I ended up working on FIFA, NHL, Madden, NBA and more,
doing procedural helmets for characters, characters scanning, crowd systems, photogrammetry for characters, and, finally, modeling tools for Need for Speed Payback. I had a great time there and my great colleagues Caleb Howard, Ivan DeWolf, and Bryn Wyka and many more, they all taught me so much.
I could keep talking for days, but this is probably already way too long, so let’s get to the rest of your questions!
Houdini Advantages
80.lv: Could you talk about the way the procedural nature of Houdini proves to be so interesting for the game developers? What are the elements and tools, which you think show the biggest interest for the users?
One of the biggest things that make Houdini interesting for the game developers is the ability to solve problems of scale and deal with things that are simply too great in number to do by hand or that require a lot of precision, especially in combination with tedious repetitive work which will bore people and make them make mistakes. A good example of this was the helmet fitting I mentioned above. When you are a company like EA and you have thousands of character over multiple franchises that wear different types of helmets it can save a LOT of time to do that automatically. Suppose there are 2000 heads and 8 different helmets with 3 different sizes each, that’s 16000 helmets to place… now imagine also having to fit the strap and heads being improved all the time, meaning they need to be re-fit continuously. That’s a job that costs a lot of time artists could have spent on actually making the game look better, so our system did it in an hour or so whenever required.
Complex Level Generation
80.lv: Let’s talk about the generation of the levels with Houdini. This package provides some amazing landscape generation tech, and we’re interested in the way you suggest using these tools for the generation of the game worlds of a more complex nature. How does that actually work and what way can a person with little Houdini knowledge approach this task?
When it comes to the complex level that includes anything from mountains and dungeons to corridors and interiors – keep in mind that if you can think of it, Houdini can make it. Houdini basically just wants you to store all the steps and then give it the input to work on. Let take a road: the simplest input might be a curve, and then we tell Houdini the steps to go from that curve to a road. And that is basically a lofting operation. We probably want to do some smarter things like automatically UVing that road or have some controls over something like the number of potholes on them but the principle is the same. Let’s take another example of a building: if you make the basic shape, we can divide that shape into tiles of wall segments, and place those instead of the simple box you gave as an input. Now if you were to make a dungeon, you would start with a simple layout polygon of the floors, allowing Houdini to add the walls and detailing with maybe tiles for hard walls and procedural rock generation where it’s more organic.
It’s really not that hard once you learn how to break a problem down into its most basic steps.
Always think of it this way: what is the simplest basic representation of an object, its simplest form and how would I add all the detail to reach the highest detail? How can I make rules to go from A to B?
I usually start to show Houdini with these simple setups: grid – paint – node – scatter (set to use color for density) – copy_to_points. This setup places objects (that you input into the copy) onto the points that get scattered where you paint a brighter color. This simplified option can be easily extended. Instead of painting the color you can calculate, say, how much light an area receives. and grow plants based on that amount. It’s all just numbers. It doesn’t matter whether I or some other process does it. What matters is that the right value is placed in the right spot.
I decided to make a simple scene to explain how this works.
It’s simple but here you can see how it takes a shape and makes a fence out of it plus places animals inside:
.hipnc file (fence asset)
Filling the World with Organic Elements
80.lv: How do you fill the world with all the content? With the procedural approach, you can potentially influence all the little elements like tree branches, the size of the rocks, etc. You can even generate the way the tree would grow within a certain natural environment! That’s crazy. Could you talk a little bit more about the way it works?
We can grow a tree based on how much light it receives, doing a simulation of where branches catch that light, even doing self-shadowing. Typically though we don’t go that far as games like instances and repeated objects. But it can definitely be done just like Anastacia Opara showed (her talk is amazing). Start at 8:07 or watch the entire talk because it is worth your time.
It’s more common to have a set of different assets that grow in different areas and at different ages like pines, oak trees etc. They have their own preferences: pine trees are happier in the cold and need less water, and we can take that into account when we generate their “likelihood values” to get different masks for different types of trees. Then we can add things like different age, scaling them up and down a bit, randomize rotation and the human eye is easily fooled.
We can do some more real-time trickery as well such as coloring the leaves based on the placement or adding a real-time vertex shader to bend our trees a little differently.
When we want to replicate something organic it’s all about looking at the patterns. Need rock distribution? Let’s make a water flow for it. Plant distribution? Look at what the plant needs: temperature, water, sunlight etc. Every plant has its own needs, so figure them out and simulate to get natural results.
The final trick is noise. Noise is basically just adding extra “randomness” to a system, for example, if we have a rule that says, plant X can only grow within 10m of a river, that’s going to make a rough line. We add noise, make a gradient based on the distance of the river and subtract a value from the noise to hide that edge and make our simulation look more clever then it really is.
80.lv: How do you manage to fill the whole scene so organically? (We believe this approach is similar to the things done with the Horizon Zero Dawn and Far Cry 5).
Yeah, it’s exactly what I was saying before. What these teams have done incredibly well was looking at the rules and following them: why do plants grow this way? where do they grow, why? How do the mountains form (strata rock generation)? Ghost Recon Wildlands did a great job with this too, even procedurally generating small towns and squares.
Etienne Carrier gave a wonderful talk at GDC about the work on Far Cry 5 and the procedural vegetation systems. I highly recommend checking it out if you want to know more about organic object placement.
Multi-level Spaces Production
80.lv: What do you think are the ways you can explore some of the potential of Houdini with level production? Is it not just about the creation of open spaces, but also generating dungeons, complex multi-level spaces? What is a good way to jump into it and figure out how to work with that?
That’s absolutely true!
I decided that it would be easier to show than to tell what Houdini is capable of, so I just made a quick small demo file to show some examples. It goes from the input at the top to the one at the bottom completely automatically:
Here is an explanation of how it works:
The example has some polygonal shape as an input but you could also paint on a plane, remove the white section and then generate walls on the edges, and there you go. You just need some input. You could even generate some random curves and make it completely procedural. I typically prefer a bit more control but for a completely procedural dungeon crawler that may be the best option.
Now closed spaces aren’t necessarily harder, just different. Basically, you just invert this system, invert the walls, and hey presto. The example has some modeled shapes as an input but you could also paint on a plane, remove the white section and then generate walls on the edges, and there we go – an enclosed space.
If that’s a cave, we will make the walls look like the rock. In this case, we can either place pre-scanned rocks or generate procedural rocks as I have done here. If it’s more like a castle dungeon, then we could use wall tiles to automatically be placed along the walls similar to the balustrades or the fence asset. Like this:
.hipnc file (manmade dungeon)
It goes from the left input to the final result on the right, creating stairs, balustrades, adding details to walls and some basic details on the floors (note, that this could also be a building generator if turned inside out).
After that, we want to fill the space with no matter what – desks, torture devices or just organic things. in the example to keep it simple I just placed things more or less randomly, but things may be placed based on the amount of sunlight or around campfires etc. We can model a distribution of items as well, whatever rules you can come up with go crazy.
In the Dungeon example, you could imagine me placing torches and enemies, etc. based on all kinds of rules, such as the distance between torches to get to a nice amount of illumination, in-game difficulty settings, etc.
Ironically, one of the hardest things to decorate is an office, because it has such rigid rules about where people typically place things and it combines big open spaces with things that both align to walls and are in open space.
Ironically, an office is one of the hardest things to decorate because it has such rigid rules about where people typically place things. Nature is more forgiving. If we place a desk at an angle in the middle of an office room it will look odd, so we need to figure out how people place things, which is actually often harder then nature rules.
Custom Details Production
80.lv: How do you think developers can achieve the necessary level of detail with the procedural generation? In your earlier demos with the buildings we saw a basic level of authenticity, but we guess right now you can actually create some very good detailed procedural architecture and customize all the small details like color, chimneys, etc. What are the ways you can enrich the level of detail and make the pieces of the environment look as if they were done by an artist and not a machine?
The buildings demo I did was made in a few weeks (for a game studio doing an open world game we were demoing for) and it could be vastly improved. But it does its job and proves that instead of an artist placing each tile, a computer can do it. The main issue here isn’t even the tool but the fact that I had to make my own window tiles during those weeks, so they were bad-looking assets. If they were modeled properly, the tool would make totally photorealistic buildings.
The main idea is that we want to take the tedious job of the artist placing all the lamp posts next to the road or all the windows in a building, blades of grass etc. But adding the things that make that building unique is what the artists excel at, so I tend not to leave it for them. I take the non-glamorous boring repetitive wo
However, as I said above it’s totally doable, to create it procedurally and would in certain cases be useful, for example, when making a dungeon crawler with random levels with mood and atmosphere. But note that it means a lot of extra work and often I found that when it’s not a requirement to deliver the final product totally procedurally, my time is better spent on removing only repetitive, slow cumbersome tasks from the artists and letting them work on the unique parts. The more time I save them the more they can focus on adding that atmosphere. Sometimes they need to remove or change everything I add and it also takes time.
But to give a more recent better-looking example, here is a building generator by Anastasia Opara:
Adjusting Elements
80.lv: Could you talk a bit about the way you’ve been working on the importing of such content in a game engine like Unity or UE4? How do you make sure everything fits together and trees, for example, don’t hang in the air above the terrain? How do you optimize it?
Importing content into engines typically goes in either of two ways: exporting some geometry and importing it (for unique meshes) or exporting some custom file like XML that tells the engine where to place instances of meshes the engine already has imported (like, say, trees or something).
For everything instanced it’s the second approach, the first is for the things that are unique.
For fitting together, the procedural approach can work well in 2 ways: either with neatly fitting elements or organic. If it’s neatly fitting tiles, that’s easy – they just fit. We only make sure the scales are consistent and everything should just line up. When we place objects it can be a little trickier, for example, we typically scatter points onto the surface (or project them down onto the terrain) to avoid floating trees. The place where it can get hard is when you have to place an object like a table on a non-flat surface. If the ground is not flat, one or two contact points (legs in this case) will be floating. To avoid it we can flatten the area, adjust the area under the legs or try and pick a decent average that doesn’t look too bad from any angle.
However, we often place either rocks or trees and sticking them to the ground is easy. It can get tricky though when there is a combination of both, or when they have to blend from one to the other.
Finally, when it comes to the in-game performance, procedural is great because we can achieve high-level controls and based on some heuristics adjust. For example, if we have a bad framerate in a particular section of the forests, we can just lower the number of objects there and as the tools improve turn the quality up and down whenever we need during the production process. Thus we can always run as optimal as possible.
Another trick is combining several meshes together into clusters based on the terrain, for example. Let’s take a field you need to place the grass on: most of it is flat but there is a section with a hill. In this case, we can either just place small patches everywhere and have hundreds of thousands of these patches or we can spawn a big mesh there, say, 10×10 meters of grass and fill the hill section with smaller patches to avoid floating grass blades. The real trick is just measuring where the expensive version is needed based on the curvature of the terrain.
Recommendations for Houdini Learners
80.lv: For a content creator, how would you recommend starting on game asset and level generation with Houdini? What would be a good starting point: a tutorial, some presentations?
The best part is that Houdini has a free apprentice edition that lets you do anything from the full version: fluid effects pyro destruction, modeling, compositing and so on. It’s awesome, so go and download it here.
And while it’s downloading, watch the video below video which is just a simple introduction, but it covers a lot: UI, getting around, making a simple asset, making controls, and how to make that tool reusable. Sorry, the audio is a little crappy.
Houdini works with Nodes, therefore it’s a bit different from Maya or 3ds Max. I know it’s a little weird but hang in there. If you liked the project, continue with part 2:
More in-depth:
I have made a few files for you that cover some of the questions I answered today.
I recommend starting with the fence file, it covers a lot of basics and does so in a simple and easy-to-play format. See how you can modify the asset, make the fence taller, add additional bigger poles or a gate, check how the fence reacts and how you can extend the system.
Then look at the dungeon file and see if you can figure out how it works. Here’s the explanation just in case:
These assets are simple intentionally, but you can probably see that they can be powerful and easily extendable to create more complex and impressive things.
After that take a peek at the manmade dungeon file and the video below. It has more complex logic about placing objects along the walls and making more specific structures.
Finally, I also made a simple forest distribution file. Try to figure out how it works without help.
Use the paint node to paint “desirable” locations, (one could calculate them too, of course! Especially if he/she watched Etienne’s video) and see what happens. Why does the same value result in trees going in one place, and bushes and grass somewhere else? I hope you can answer that question! You can also think of a way to add foliage around the path, draw a curve and remove trees near it. You are totally free here. Experiment and have fun!
For further for tutorials, I recommend:
- Entagma for more abstract effects.
- Kim’s tutorials: they go pretty deep but cover every section quite well. I started with them initially.
- And if have free money, there are great ones over on CMIVFX.
Good luck out there, and if you have questions feel free to shoot me an email (you can easily find the address on my website).
Freek Hoekstra, Procedural Artist
Interview conducted by Kirill Tokarev