Paul H. Paulino showed a cool technique to create a cavity map in Mudbox and Zbrush.
Paul H. Paulino shared a cool technique to create a cavity map in Mudbox and Zbrush that can be super helpful for your organic texturing/shading process. It is also considered an Auxiliary Map along with the Ambient Occlusion map and Curvature Mask.
WHAT IS AND WHY DO I NEED A CAVITY MAP?
Cavity maps are essentially a black and white mask which will give you access to crevices and high-frequency details on your model. As you will notice when we are baking the map in Mudbox, the cavity map is an Ambient Occlusion operation, but with a “sharper” and darker look into holes and recesses.
The Cavity Map can be handy if you have an organic sculpt and you want to mask pores or other similar details. You can also blend it on top of your diffuse to add more complexity to your textures.
Keep in mind that these techniques I’m going to showcase are focused on VFX assets which have UV’s. So before you start make sure to check if your asset has proper UV’s otherwise you might have problems.
USING MUDBOX
Pros
– Clean map
– Not a lot of options (which will also be a con)
– Simple to use
Cons
– Naming Convention
– Not a lot of options
1 – IMPORT YOUR MESH
Open Mudbox and import your HIGH RES geometry.
2 – REBUILD YOUR MESH SUBDIVISIONS
Click on “Rebuild subdivision levels” like the image below so you can have multiple subdivision levels in your mesh.
If this process works, you will be able to see all the subdivision levels on your mesh.
Common errors: Sometimes this option doesn’t work properly because Mudbox is sensitive to mesh errors. But don’t worry, if that’s your case, simply bring the same mesh with a lower resolution and you’ll be okay.
3 – EXTRACTING
Go to UV’s & Maps > Extract Texture Maps > New Operation.
Choose Ambient Occlusion Map. Here you can also name your extraction operations if you end up having more than one.
Okay, so this part might get a little bit confusing but don’t worry, I will try to get your head around this part without getting too technical. After a ton of experiments, these are the setting that I’ve been for almost everything.
On the screenshot below I highlighted the areas that you should pay more attention while adjusting your settings.
1 – Here you can add your LOW res mesh or the lowest level of your HIGH res. (If you rebuilt the subdivision level on step 01 it will give you a few options)
2 – Add your HIGH res mesh.
3 – Change your quality to Best and your method to Subdivision.
4 – On Image Properties, just make choose to select the resolution you desire. In this case I choose 4K just for the sake of this tutorial, but it is up to you.
5 – Here’s where the magic happens. The only option that I modify is the filter. The default option is 0.01, but to force the AO map get into the crevices, we are going to lower the filter to something around 0.000001. You can play with this value and see if you can get different outcomes.
6 – The final step is to choose your bit depth and your output folder. I usually keep it simple and choose 16-bit RGBA TIFF maps as my output. You can also keep the “Preview as diffuse” option checked so you can see how the map looks like in Mudbox.
If your mesh doesn’t have any problems, you should get a window like the ones below.
And if you have selected the “Preview as diffuse” option, you will see a preview of your map like this one:
Now you are ready to go! If you want to use this map in MARI, you will have to rename it manually to UDIMs; or you can use this awesome renaming method created by my friend Jens Kafitz.
USING ZBRUSH – METHOD 01 – MULTI MAP EXPORTER
Pros
– Fast
– If you’re gonna be painting in MARI, it gives an exporting to change the extension to UDIM.
– Simple
Cons
– Hard to predict the output
1 – IMPORT YOUR MESH AND ADD SUBDIVISIONS
Import your mesh into Zbrush. Make sure it has enough subdivisions (4 million polys or more)
2 – MULTI MAP SETTINGS
Go to Zplugin > Multi-Map Exporter.
On the screenshot below I highlighted the areas that you should pay more attention while adjusting your settings.
1 – Check the Cavity Option
2 – Select the desired resolution
3 – Here you can find the main settings for your Ambient Occlusion. The settings above are the ones I usually use, but it takes a lot of trial and error. Below you can see the information from the Zbrush Documentation website
- Cavity Blur: Higher Cavity Blur values will soften the cavity map effect.
- Cavity Int: Higher Cavity Intensity values give a harder edge and greater contrast. Negative values will give an inverted map.
- Use Curve All SubTools: The Cavity Curve for the current subtool will be used on all visible subtools. Only applies If the ‘SubTools’ option is on.
- 16Bit: When on a 16 bit single channel file will be exported; when off the file will be 8 bit with 3 channels.
- Colors: When on, the Cavity map will be generated using the main and secondary colors in the Color palette(only available as an 8 bit file). Turn off for black and white.
4 – Click on File Names and choose your UV tile ID format
5 – When you’re done, click on create all maps to extract your map.
USING ZBRUSH – METHOD 02 – TEXTURE FROM POLYPAINT
Pros
– Fast
– If you’re going to be painting in MARI, it gives an exporting to change the extension to UDIM.
– A lot of control over your output
– You can see your map before exporting it
– More options than the other two methods
Cons
– More options than the other two methods
1 – IMPORT YOUR MESH AND ADD SUBDIVISIONS
The first step to this tutorial is to import your mesh into Zbrush and add a few levels of detail. The amount of subdivisions will vary from asset to asset and you’ll know if it’s enough when you see that your mask/model isn’t faceted.
2 – CHANGE YOUR RENDER MODE TO FLAT
This step will allow you to preview your mask accurately without any light/shadow information.
3 – MASKING
Go to Tool> Masking > Masking by Cavity. Here’s where the magic happens.
As you can see, one of the greatest things about this method is the fact that you have a lot of control over your map. It might be confusing for some people, so if you want understand what these settings do, here’s a quick explanation from Zbrush Docs:
- Blur – The Blur slider will apply a blur to the cavity mask. By setting the slider to 100 the cavity masking will be softened. To best view cavity masking, set Preferences>Edit>Masked Object Dimming to 5 and change the material to Flat Color.
- Intensity – The Intensity slider creates a harder edge around the cavities. Negative values will invert the Cavity Mask.
- Cavity Profile – The Cavity Profile curve controls how the cavities on the mesh are masked. The left side of the curve is the base of the cavities, the right side is the surface. The dark area of the curve is the masked area and the light area is unmasked.
With that in mind adjust the values accordingly and press Mask by Cavity.
If you followed the steps correctly, you should see something similar to the image above. The mask preview is surprisingly fast and accurate. Now we can start adjusting our mask.
4 – EDITING YOUR MASK
By default, the mask by smoothness already give us a great map, but luckily we can go even further and get a better result. On top of the masking tab, you can see a variety of options to adjust your mask. You can use blur, grow, sharpen and shrink to get a result that suits your needs. Go ahead and test each one.
Another great thing about this method is the fact we can invert the mask before exporting it.
5 – FILLING YOUR MASK WITH COLORS
When you’re finally happy with your mask, it is time to bake colors onto the mesh so we can export it later. With your mask selected, invert it so the white area can receive the color and go to the Color menu and change your color to Black and press Fill Object.
Now clear your mask and you’ll see that your mask is now a texture baked into your geometry, ready for export.
6 – EXPORTING THE CAVITY MAP
If your mask result looks similar to mine, you’re ready to export your map. Go to Zplugin > Multi-Map Exporter and select Texture From Polypaint. This step is very similar to the second extracting method we tried before.
Press OK and click on Create All Maps. Select your output directory, and you’re ready to go, and don’t forget to change your extension name to UDIM’s.
BEING CREATIVE WITH YOUR MASK
While baking your map, keep in mind that you don’t have to use just one option. When I’m working on my textures, I usually like to export two or three different masks.Since the baking process doesn’t take too long, I benefit from that to create multiple versions which can be very helpful later on.
HOW TO USE A CAVITY MASK?
Like I mentioned above, the cavity map can be especially useful when you are working with organic models. If you have a good sculpt, you’ll be able to have access to pores and crevices without effort.
Having the mask can help you improve your dirt mask, like the one shown below:
Or you can overlay it on top of your diffuse channel, giving a more interesting look and enhancing your colors with details from your sculpt.
Another great use of a cavity map is to apply it to your rough/gloss maps. It can give you a nice highlight breakup in the recessed areas.
That being said, it doesn’t mean you can’t use it for hard-surface objects. If you have a piece which has a sculpt pass on it and you want to get into cuts and scratches, the cavity can come in handy there as well.