Creating Moonlit Clouds Simulation in Houdini

Introduction

Thanks so much to 80 Level for your interest in me, Patrick Schwalbe, and my project Dancing Moonlight. I am a 3D technical artist and my specialties are animation, rigging, and special effects. My favorite thing to do is to make literally anything in Houdini; it has become an obsession. I just love the procedural workflow so much. 

I have been a 3D artist since 2000 and have been focusing on the technical side of 3D that whole time. I love problem-solving and learning new things. That includes learning new software and plugins for those programs. I have known the whole time I've been doing 3D that I will have to keep learning new things every day, and that's the part I love the most. 

I've been doing 3D animation and technical direction my whole adult life, which is a crazy thing to wrap my brain around! I've developed these skills while working on medical animation, advertising, and 3D product development for retail. 

Here are the projects I've worked on:

Harmony :: I made everything :: personal project

Xray Body in Motion - Yoga :: shading, rigging, look-dev :: made while employee of Hybrid Medical Animation

Starry Night Embroidery :: everything :: made while employee at Target

Liquid Gold :: everything :: personal project

Patrick Schwalbe Commercial Reel :: credits in video :: made while employee of Gasket

Houdini

I learned about Houdini through my professional work using Autodesk's Maya. I was introduced to a plugin for Maya called Soup that I LOVED. It was a plugin that allowed Maya to be more like Houdini. That is how I discovered that I liked working in a very procedural way. It took many years of waiting before I could get my hands on my own Houdini license, but Soup was an amazing toy to play with in the meantime! 

I love creating new things all the time. I come up with very lofty ideas and I work patiently to learn all that I need to know before finalising any of my projects. I love seeing that smooth motion and those complex details in the final results. Some people create things and then never look at them again. I make art that I want to look at, so I refine and refine until I am very happy with how it turns out. I don't like to finish things halfway so I HAVE to be patient and it's worth the wait, for me. After I'm done with a personal project, I will keep going back, again and again, to learn from the result or just to simply enjoy the show. 

I have to give credit to many tutorial creators out there. Entagma, Johnny Farmfield, Junichiro Horikawa, and VFX Grace are some of the people I lean on all the time. The Houdini community is also very generous with handing out free .hip files. They've been a very helpful group of people! Houdini Blueprints is a great example of this.

The Dancing Moonlight Project

I learned a LOT about realistic cloud formations and movements from this tutorial course from VFX Grace. I have had a mild obsession with clouds for the past few years. I have a long-term goal of making a CG short film that is all made up of clouds. I am in the learning phase of that project right now. Dancing Moonlight is a practice project for that larger project coming... someday. It will be made using some combination of Redshift, Renderman, and KarmaXPU clouds.

Actually, this cloud project (coming someday) is the first of a 3-part series I have planned. The first video that will come out will be called "Nimbus", then "Wanderer", then finally I'll release "Majestic". They all gradually increase in complexity, so I've ordered them accordingly. This means I will need to learn how to create clouds and have lots of control over them; that part is done! Then, I need to learn how to efficiently render them... That's a work in progress right now. I only have 1 computer, so it's been slow. Next, I need to learn how to generate fantasy landscapes and how to do character rigging in Houdini. I already know how to do these in Maya but I also want to learn this in Houdini.

For the final project, Majestic, I will need to combine what I've learned in the first 2 videos in order to complete it. Like I said, I'm patient and I have ambitious goals!

References

I stare at the Minnesota sky anytime I am outside or riding in the car. I constantly bother my wife by saying, "Oooh, look at those clouds!" I take lots of photos and I save any beautiful image I come across online and I take MANY, MANY, notes. I will save a note with a reference image and describe which shot will be driven by its look/feel. I describe the action that will take place and I will place it in sequence. I have the whole project storyboarded this way in great detail already. I just need to do the work.

Now that KarmaXPU clouds were so successful and Renderman 25 just came out with huge increases in speed, and Redshift's volumes now have the multi-scattering capability, I am on really good footing to proceed.  I plan on using some combination of all 3 renderers to make "Nimbus" an eventual reality. I also took a lot of inspiration from this very stylized image on Instagram.

Workflow

I used SideFX Houdini, Karma XPU (renderer), Topaz Video, Adobe After Effects, and Adobe Media Encoder for this project.

I had the idea of using a photo-stacking effect to composite together lots of light angles to create the final look/feel. This turned out to result in animated light. So, I also made a short video. I used Topaz Labs to interpolate the frames to make the video play slower/longer. I also used it to upres it a tiny bit. I didn't plan on making a video of Dancing Moonlight... it just worked out that way!  

Houdini's Pyro

I wrote a Python script that creates a base cloud setup. It creates the whole suite of nodes needed to go from simulation to cached output VDB sequence. I created 10 different simulation sequences. For each iteration, I tweaked different settings to make versions with less wind, less density, and more buoyancy to push the clouds to be taller, adding more detail to get the "cauliflower" look, etc.

My process, at its most basic, is: watch tutorials, iterate, and iterate some more, get to a point where you're happy and hit render. This is my process because I don't work on Houdini effects with a team of people and I can't ask anyone around me important questions. There is a lot of trial and error. There is a lot of note-taking so I don't have to learn the same concepts over and over again. Inside Houdini, the basic workflow is to simulate your pyro cloud, bake it to cache sequence, bring that sequence into a separate render file, assign shaders/lighting, and render out .exr image sequences. I also followed a lot of the advice on Sergen Eren's page.

Rendering

I started from scratch loading each cache file into the OBJ context. Each cloud got its own Geometry Node with its individual volume VOPs. That way I could position and handle their densities, individually. In the OUT context, I created a Karma Driver node which automatically does a lot of the USD setup for you! That was really convenient. 

Simply using Karma XPU as the renderer and using the XPU Pyro Preview as the shader on the clouds made all the difference in optimization. The speed increase and the beautiful results speak for themselves. 

Karma is new and exciting to use. It is very fast but you need to learn a whole new way of doing things, especially if you are working in both the OBJ context and the LOPs context. It is my understanding that you cannot use Principled Shader VOPs in KarmaXPU. There are many other differences. I use MaterialX shaders so I can use the CPU and the GPU simultaneously. I like to do as much scene building as possible inside the SOPCreate nodes to keep as much as possible in the Solaris/LOPs context. That way I don't run into workflows where I'm doing everything twice, once in OBJ and again in LOPs. But for Dancing Moonlight, I broke this rule of thumb because I found out I could just use the Karma render node in the OUT context, saving a lot of setup time. It worked out okay without any redundant work, whew!

Tips for Beginners

Start out small. I sometimes need to remind myself that I can go faster if I keep the tests small. Here is an image of an early render test. I didn't try to do render tests with the whole scene at once. I hid all the other clouds and just focused on these two until I was relatively happy with the results. 

Simulating and rendering clouds (photo-realistically) will be slow and will take up a lot of space on your computer. You need a lot of voxels to get this amount of detail. I've made simulation cache sequences that are hundreds of MBs per frame. Sometimes I go crazy and they're over a gig per frame. The resolution you need to render out at needs to be high in order to get the little wispy details of the clouds. You can't render small and expect that detail. If anything, I've been rendering high and reducing the resolution afterwards because of how anti-aliasing works. 

Denoising is key. You should have some noise in your clouds. Some noise is good but too much will look incorrect. The actual video will always have some amount of noise (especially in the shadows) so don't try to get rid of all of it. Allowing some small amount of noise in the final renders will also reduce render times. Because I will be primarily doing animations, I needed a denoiser that considers the sequence of images when it denoises. That's where Topaz Labs' tools came in. Some denoisers only consider a single frame at a time. This can result in denoised looks that "flicker" over time in the final video. 

Here are some links for useful tools:

• Soup, a plugin for Maya to make it more like Houdini
• Entagma, Houdini tutorials and .hips
• Johnny Farmfield, Houdini tutorial and .hips
• Junichiro Horikawa, Houdini tutorials
• Realistic Dynamic Clouds, Houdini tutorial by VFX Grace
• Houdini Blueprints, Houdini .hip files
• Topaz Labs, denoising/uprezing tool
• Cloud Modeling Series

Thanks so much, 80 Level, for your interest in me and my project Dancing Moonlight.

Patrick Schwalbe, Technical Artist

Interview conducted by Theodore McKenzie