Procedural Optimization With Houdini: Case Study

The second part of our talk with Adolfo Reveron explains how you can build environments procedurally with Houdini and UE4.

Adolfo Reveron discussed creating environments procedurally with Houdini and Unreal Engine. The developer broke down one case in detail and shared tips on how to approach the VEX language, set up different elements of the environment, and more. Make sure to read the first introductory part to learn more about the importance of Houdini and how it can be used for different development tasks.

Introduction

Why create a modular level procedurally inside UE4?

Producing environments is a highly consuming task. It involves creating a meaningful level design layout and proper art dressing on top of it.

This is addressed in a linear workflow traditionally (first, do the design, then art) but this is a stiff and outdated way to work as in practice these stages overlap each other and any layout change leads to destroying and redoing existing artwork.

Producing a game or a movie is an alive entity and needs to evolve as it progresses. That's why the Houdini engine comes in handy: it can run any tool (aka HDA or Houdini digital asset) inside other platforms like UE4, Unity, Maya, Max, and Cinema4D, manipulated and evaluated in real-time. So, what about designing an HDA to produce those modular levels inside Unreal Engine on the fly?

Designing an HDA for modular level building

You can find some projects about creating a modular environment with Houdini and Unreal Engine 4, like this excellent tutorial by Simon Verstraete

After some research, I realized that procedural artists in charge of creating most of the tools, already know the measurements of the wall modules and/or stick to round values (1m, 1.5m, 2m, and so on), which eases things a lot because you can carve those distances in Houdini and rely on them to build your system. But wait, that means you would need to tweak the tool depending on your inputs! What if you add a new wall module 1.75m width?!

Once I spotted that, I decided to aim for a more powerful approach: design the tool so that adapts to different module widths, which would boost the HDA’s flexibility. In addition, I also included the ability to create fully dressed rooms, not only corridors/open areas.

With this approach, I can change those measurements in different projects very easily, so that the tool will pick the modules randomly, align, and finally scale them to assemble the walls perfectly.

Houdini supports three coding languages: Hscript, Python, and last but not least: VEX. This is pretty similar to C++ and I embraced it since the first day learning Houdini. Despite being challenging (and painful) at first, I assumed it was key for bending Houdini to my will, so I just accepted it as my weird little-talkative buddy, standing by me on a daily basis, who I had to end up loving. The more I knew about it, the more I felt it provided true control. Nowadays I use it as much as I can, in combination with Python, and can’t imagine addressing anything without them, no matter the complexity of the project. If you are in the early steps with it, believe me: the effort will trade sooner or later, just hang on and keep pushing.

Regarding the Modular Environment Project, VEX calculations, despite not being too complex, became a bit challenging until I found a good approach for every little problem within the project. Most times it is not about knowing all secret functions and expressions but about stating what you need to do step by step as if describing the process/algorithm.

Breakdown

Goal

To put it simply, the goal is designing a Houdini tool (HDA) that will allow the user to create a finished game-ready environment with minimum effort and maximum flexibility. I found a good practice to state in a single line what the tool is going to do. Sometimes when developing a tool, you lose focus very easily as you progress, because every little aspect of a project involves problem-solving and making design decisions. A few little deviations from the original purpose of the setup stacked together might lead to undesired results. I ended up asking myself many times: 'ey, ey, hold on a second, stop. What was the purpose of the tool? I wanted to create a tool that makes dogs, and I am trying to output horses now. I need to go back to the track'.

To help me keep focused, I like writing down basic design guidelines, as if coming from a client’s brief :

Rely on in-engine curve editor for creating layout.
The ability to create elaborate silhouettes
The assets will be fed dynamically from the editor’s content browser. That means the tool must be unaware of the modules it is gonna be receiving as input.
Modular (for this demo, I used Megascan assets)

Overrides

The idea is to be able to allow the user to create rooms from simple shapes, which will be processed and connected to the floor layout (explained in the "Floor layout" section).

To do so, I firstly processed every shape coming from the editor as “override input”, so that I could generate a grid-based floor.

The first thing I do is creating a box for every input, which is easily done with a "for each connected piece" loop containing a bound node. This will ensure the creation of a simple box, no matter the shape used by the user to feed the tool with (you can drag any existing asset as an input, even a T-Rex, which might lead to instability of the tool).

Then removing all faces except one, dividing it using the “bricker” option, according to the selected size. Finally, using the “snap” node-set to the grid.

Again, a good practice I got used to some time ago is trying to solve as much as I can through VEX, like removing the overlapping faces in this example. I find amazing the way this simple use of the ‘intersect’ function saves so much time and prevents topology issues.

It is extremely useful to get in touch with as many VEX functions as you can, so from time to time, I scroll down through the vex reference site, look for a function I never heard of, and try it. At first, it might look like you are wasting your time, but I found this to pay off in time, always.

However, the more experienced you are, the better you balance your coding efforts. Sometimes it is overkill because you might spend a full evening solving something to later find a node doing exactly that in a more efficient way, which might seem I lost my time.

But in general, the more I do it, the more I think it was a good decision because this forced me to be more and more proficient with this scripting language.

Floor layout

I start copying a box with dimensions promoted to a parameter (this will allow tweaking the width of the corridors later). The curve is resampled to points spaced 1m. However, this doesn't ensure the points will be correctly aligned to the grid. (A point can be sitting in position P=(1.25, 2.01, 35.6), no matter its distance to its neighbor point).

To fix this, I preferred to rely on this rounding trick, which snaps the points based on the "grid" variable. It is a cheap and pretty powerful way for moving the points around, you get unexpected point placements when playing with the threshold, by the way.

//Ground curve
@P.y=0; 
//
float grid=chf("Floor_grid_unit_size");
@P.x=rint(@P.x*grid)/grid;
@P.z=rint(@P.z*grid)/grid;

Then I remove overlapping, assign basic normals, and, finally, create a point in the middle of each primitive with this code. This way I found out that despite not existing such a “@P” attribute for primitives, it actually works as a primitive centroid when fed into a VEX function (there are many different ways to work when snapping stuff to the grid, and I found this approach works nicely). I use these points to copy a grid later. I finally merged this grid with the previous one created within the "overrides" section.

Walls

This is the heart of the tool. Below is a visual overview of the process.

I start by isolating the "in-between" points from the corner points with the group node and some auxiliary primitive, like a platonic shape. I tend to use less geo if possible and set this shape to be a pyramid because it is the simplest closed shape, but I am not sure whether this is the most efficient way of grouping points with second-input geo - like a sphere set to primitive.

Once I resampled the line, I tell each point: "You will be assigned to instance a module of X-meter width". With some VEX, I report the system the possibilities (check red arrow) and assign this "width_pick" randomly. PS: I forgot to promote this to a parameter so that this array of suitable widths is assembled dynamically based on user input.

//Goal: oofset points within a prim according to possible offsets
//(aka modules_widths) 
//1. Find first and last point of prim
//2. Move all prim points to first point position
//3. Find the prim axis
//4. For each point, offset it randomly according to
//the axis+suitable offsets
//5. Do this for all points
//6. Measure both original and current prim (offsetted)
//7. Remove points with measure higher than original prim
 
float wall_widths[]={1.5,2,4};
float seed,pick;
seed=chf("Seed");
 
//input possible offsets
int pts[]=primpoints(0,@primnum);
int primpts=len(pts)-1;
int pt0=vertexpoint(0,primvertex(0,@primnum,0));
int pt1=vertexpoint(0,primvertex(0,@primnum,primpts-1));
vector prim_axis=normalize(point(0,"P",pt1)-point(0,"P",pt0));
 float perimeter=primintrinsic(0,"measuredperimeter",@primnum);
 vector tempZeroPos,newPos;
float tempWidth;
 foreach(int pt;pts)
{
tempZeroPos=point(0,"P",pt0);
setpointattrib(0,"P",pt,tempZeroPos,"set");
pick=int(rint(fit01(rand(seed+pt+@primnum),0,len(wall_widths)-1)));

tempWidth=wall_widths[pick];
 
setpointattrib(0,"width_pick",pt,tempWidth,"set");
setpointattrib(0,"prim_axis",pt,prim_axis,"set");
setpointattrib(0,"perimeter",pt,perimeter,"set");
}
 //f@pick_width=wall_widths[pick]
//v@prim_axis=prim_axis;
 i@pt0=pt0;
i@pt1=pt1;
i@primpts=primpts;
f[]@wall_widths=wall_widths;

Doing the offset comes next, and it involves grabbing previous points' random widths and accumulate those. As a result, the points are offset.

Comparing resulting, expanded prim and original unexpanded y can tell which points are beyond original bounds. In this case, I tried using the xyzdist() VEX function. It is super-handy and one of my favorite functions as it provides you with a ton of information: the distance from a point to closest geo, the prim number, and its parametric UVs.

int pts=len(primpoints(0,@primnum))-1;
vector cen=v@P;
 
float widths[]=f[]@widths;
float tempWidth;
float offset=0;
int pt_counter=0;
int startPt=i@pt0; //first point of each prim
 vector startPos=point(0,"P",i@pt0); ////Grab position of each first point
vector prevPos=startPos;
vector newPos={0};
 
foreach(float w;widths)
{
offset=widths[pt_counter-1];
newPos=prevPos+v@prim_axis*(offset);
setpointattrib(0,"P",startPt+pt_counter,newPos,"set");
prevPos=newPos;
pt_counter++;
}
 
f@new_perimeter=offset;
i@pts=pts;
f[]@widths=widths;
f@new_perimeter=offset;

After removing passing points, I also needed to adjust the final point, which required more VEX scripting than expected. I compared the original primitive with its “crippled” version (removing passing points) which threw a gap. Comparing the gap measurement with the available module widths resulted in four different possibilities that needed different actions:


//possibilities:
//1. Gap inferior to min width
//2. Gap equals min width
//3. Gap major to min width
//4. Gap inferior than last pt widht_pick
 
int pts[]=primpoints(0,@primnum);
int last_pt=vertexpoint(0,primvertex(0,@primnum,len(pts)-1));
 
float gap=f@gap;
float min_width=min(f[]@wall_widths);
float widths[]=f[]@wall_widths;
 
//1. Gap minor to min width
if(gap&lt;min_width) { removepoint(0,last_pt); 
float width_pick=point(0,"width_pick",last_pt-1); 
float k=(gap+width_pick)/width_pick; vector scale=set(k,1,1); 
setpointattrib(0,"scale",last_pt-1,scale,"set"); } 

//2. Gap equals min width
 if(gap==min_width) 
{ 
setpointattrib(0,"width_pick",last_pt,min_width,"set"); 
} 

//3. Gap major to min width 


//4. Gap inferior than widht_pick float width_pick=point(0,"width_pick",last_pt); if(width_pick&gt;gap)
{
//Check if existing matching w value for that gap first
int solved=0;
foreach(float w;widths)
{
if(w==gap)
{
setpointattrib(0,"width_pick",last_pt,w,"set");
solved=1;
break;
}
else
{
}
}
 
if(solved==0)
{
int index=0;
float closest_width;
float min_difference=9999999;
float difference;
foreach(float w;widths)
{
difference=abs(w-gap);
if(difference&lt;min_difference)
{
min_difference=difference;
closest_width=widths[index];
}
index++;
}
 
setpointattrib(0,"width_pick",last_pt,closest_width,"set");
float k=gap/closest_width;
vector scale=set(k,1,1);
setpointattrib(0,"scale",last_pt,scale,"set");
 
}
 
}

The only thing remaining was to split the points according to the random width assignment and pair that with the proper unreal_instance attribute.

Dressing

Instancing the props was the easiest part. I used the grid layout (and its perimeter) resulting from combining the corridors and rooms as a starting point to place the elements (pillars, tables, chairs, benches, decorative props, etc) in order of importance. For instance: pillars were rated higher, so wherever there is not a pillar, a table could be placed. Wherever there is no pillar nor table, a chair, and so on. In other words, I kept removing available areas as I kept spawning assets, like in the following example.

Removing the first and last point of each prim (tips):

int pts=len(primpoints(0,@primnum));
int last_pt=vertexpoint(0,primvertex(0,@primnum,pts-1));
int first_pt=vertexpoint(0,primvertex(0,@primnum,0));
 
setpointgroup(0,"last_pt",last_pt,1,"set");
setpointgroup(0,"first_pt",first_pt,1,"set");

Removing points too close to the corridor:

int neirs[]=nearpoints(1,v@P,chf("Search_radius"));
if(len(neirs)&gt;1) removepoint(0,@ptnum);
 
//I consider a gap between pts of at least 1m

Whenever facing tasks I've met before I try to vary my approach as much as I can to force myself to use different VEX functions and approaches. For either removing or grouping points based on the distance to another geometry, I usually use xyzdist() and nearpoints().

Conclusions, questions, and limitations

Q: Does it work in other engines like Unity?

Houdini Engine plugin nicely connects with Unreal and Unity (but I read somewhere that SideFX also provides the Houdini API to build your own custom plugin). I developed and tested the tool within Unreal already, so I'd focus my answer on Unity: Yes, it might work, but I would need to care about two subjects: interface-related and marshaling data-related.

For example, there are special attributes used to communicate between Unreal and Houdini, like "unreal_instance". In the case of Unity, it is called "unity_instance".

Regarding the interface, you need to know the way other platforms display the data, which will affect the way you tailor the tool. For instance: 3dsMax is not able to make a graphic representation of a ramp, whilst Maya does not support to type-in values lower than 0.001.

From my experience, it is a bad practice to aim for a tool to work in all platforms supported by the Houdini Engine plugin, which leads to dysfunctionality.

Q: What would I change if creating the tool from scratch?

The more HDAs I create, the more I realize that authoring a tool is like climbing a mountain: only when you reach the top and realize other suitable paths and dangers avoided, you get the big picture. Therefore, chances are I would solve things differently. Many times I feel tempted to redo some parts or even restart from scratch, but time is limited so I query myself: does the tool meet design requirements and behavior is reasonably good? If the answer is yes, the job is done.

I hope you liked the breakdown, visit Reveron3D for more stuff like this.

Adolfo Reveron, a Houdini Procedural Artist

That wraps up our talk with Adolfo on procedural content creation. You can find the first introductory part here in case you missed it. Did you like the overview and the case study? Share your thoughts in the comments below.