Generating PBR Materials From Photogrammetry Scans Using ArtEngine

Federico Capriuoli discusses a project where he extracted several PBR materials from a single scan of a medieval church using Unity ArtEngine

In case you missed it

You may find these articles interesting

Introduction

My name is Federico Capriuoli. Based in Italy and a civil engineer by training, I am passionate about scanning. I work for Acas3D, an organization focused on surveying historical sites and structures in 3D using laser scanning, photogrammetry, and drones for the purpose of preservation. I have a particular focus on capturing underground sites, which can prove challenging, given space and lighting constraints in these conditions!

Though my work is a niche within architecture, I leverage very similar workflows to those used in gaming, and thus my toolset looks quite similar. Indeed, I rely heavily on tools such as Lightroom, Reality Capture, Meshlab, ArtEngine, and Marmoset Toolbag.

Recently, I began using ArtEngine, a material authoring tool that leverages an example-based workflow and AI to drastically speed up the material creation process. I’d previously used the software to clean up scans for past preservation efforts, but I thought with this project, it’d be interesting to see if I could extract high-quality PBR materials from some of my old photogrammetry scans using ArtEngine. 

For the subject matter, I chose a scan I did of a small medieval church in Pisa. The church, called the Cappella di Sant'Agata, was about to be restored, and so my task at the time was to create a high-resolution 3D replica to be used as a reference to create technical drawings.

Here were the end results of that scanning project:

My Scanning Workflow at a Glance

In terms of my workflow for this initiative, I leveraged laser scanning and terrestrial photogrammetry methods. I shot on a cloudy day to maximize diffuse light and minimize shadows and used a color checker to calibrate colors.

To generate the laser scans, I used a FARO® Focus Laser Scanner. The scans needed to be done at a very high resolution (3mm \ 10m). Each scan produced a point cloud of about 90 to 95 million points, which meant that the 19 scans totaled almost 2 billion points!

For the detailed photogrammetric survey, I used my Nikon D850 and 35mm and 50mm lenses. I shot in RAW, color corrected the photos in Lightroom, then brought them into Reality Capture for processing. I created a high poly model (145 million polygons) and five UV maps at 16K. I then extracted plans, sections, and orthophotos from the model to generate the technical drawings. Finally, to optimize for online viewing, I reduced the model to 100,000 polygons and five 4K textures.

Material Generation with ArtEngine

As mentioned, I recently started using ArtEngine to do surface scanning work. Indeed, another one of my projects involves mapping all the medieval walls of my city, Pisa, and creating a library of seamless, 8K textures—one material for each wall. 

For this project, I wanted to illustrate something of the opposite: Is it possible to extract several materials from a single scan? Spoiler alert: with ArtEngine, yes!

First, I generated several baking planes (Blender for baking plane creation, xNormal for texture reprojection) from areas of the scan from which I wanted PBR materials. The planes varied in dimensions.

For the floor materials, I imported only the normal and diffuse maps into ArtEngine.

Here is where the ArtEngine magic begins. For each material, I began by importing only the Diffuse and Normal Maps into ArtEngine. From the Normal Map, I generated the height and ambient occlusion (AO) maps. From the Height and Diffuse Maps, I created the Albedo Map. Between these steps, I employed a fair amount of adjustment nodes, such as Gradient Removal and Normal Intensity, to fine-tune the results.

After creating all the maps, I used ArtEngine’s Seam Removal node to remove seams. I then created a second version of this material using the Mutation and Mask Paint nodes to generate a larger material that tiled at scale without any visible artifacts.

Here are my end results for the brick and wooden door materials:

And there you have it: several full PBR materials from a single scan!

I hope you learned something new about scanning workflows and feel inspired to create and give ArtEngine a try. There’s a free 30-day trial as well as a GDC promotion to get the tool for $19/mo for the year (vs. the regular price of $95/mo) if you purchase before May 17.

If you’d like to learn more about my work, I invite you to visit my website or Sketchfab portfolio, connect with me on LinkedIn, or shoot me an email at f.capriuoli@gmail.com.

Federico Capriuoli, 3D Scanning Expert

Join discussion

Comments 0

    You might also like

    We need your consent

    We use cookies on this website to make your browsing experience better. By using the site you agree to our use of cookies.Learn more