Guilherme Rambelli has shared a breakdown of photogrammetry process behind his latest personal project.
When I go out to scan assets to assemble an environment I usually look for a few things in particular, to ensure what I’m capturing will solve (Photogrammetry 3D result) with quality enough to be used.
Weather – I keep in mind that the flatter the light is when you capture the subject, the better my asset will look after I treat it; The better it will blend and behave in any light condition and scenes. The reason is that it won’t or will have VERY little Highlight/Shadows baked into the Base Color map, which my Albedo Map will derive from.
I try to shoot the images in Raw 32 Bit so there is a little room for error, and the exposures can be equalized later in post.
Shoot with the fastest shutter speed possible, keeping the Camera ISO as low as possible to avoid Noise in the image set. Shoot with 30-50% overlap between images to Guarantee that every image shot will collaborate in my 3D asset solve. F-stop may vary according with the lighting, a good range is between 9-13 but is possible that the capture location doesn’t have enough light to support so I just keep in mind those values but don’t compromise the image quality.
* Sharp and Crisp images, F-stop 11 is ideal, ISO the lower the better.
Asset reusability – I always try to capture a limited amount of asset, that would have the most reusability possible, meaning: interesting enough, but not too iconic so people wouldn’t be able to see it repeating throughout the scene.Usually in natural landscapes, the terrain features and “props” tend to repeat themselves quite often, and share common features due to the wearing and weather condition of the location.Sometimes a few good quality captures can take you farther than multiple half good assets.
There are a few things you can do with your image set before feeding the photogrammetry software with them.There are a couple different softwares out there that can help you deal with extensive amounts of data in a more automated way. My last few year in my adventure to improve my process using photogrammetry I ended up using Lightroom, and here are a the main tweaks I do in every single image set captured and briefly its “whys”.
Lightroom Right Tab, Under “DEVELOP” 1- Calibrate White balance, 2 – Highlight slider set to -100, 3 – Shadows slider set to +100, 4 – Enable Profile Correction (select Lens Mode.) 5 – Under the menu *Photo>Develop settings>Match Total Exposures.
Those tweaks will flatten your image as much as it can with all the 32 bits of color info from the RAW image file. Step 5 will help you fine tuning your image exposure big time, and it will save you some time during capture, adjusting your exposure before every picture.
Once all of that is done, take a look in your image set and see if the color range feels somewhat close to the albedo values you would expect from the material properties from the assets you captured.
Make sure to remove any image your believe is not sharp enough, so it doesn’t influence your photogrammetry solve, and don’t decrease the sharpness of your texture in the next step.
There are a several photogrammetry softwares out there, each with its own advantages. The 2 softwares that I had the best results were Agisoft Photoscan, and Reality Capture. The highlights in each software for me were:
The steps to generate the photogrammetry 3D asset is quite simple and can be summarized in: Camera Alignment, Rough Cloud, defining solve volume, Dense Cloud, Texturing, Happiness! haha
There are multiple ways to deal with all the data acquired from solving a photogrammetry asset. In this project I used the current state of my personal Workflow, and I hope people can take advantage of it, and can improve it so Photogrammetry can be something more and more common for real time use, and a process more friendly and approachable for anyone.
From Reality Capture, I generate and export the following assets:
*Under export options, keep assets into Chunks. so 60 Dif files are generated
Now baking ->
To bake all the maps Necessary to have full control of the asset and its surface features, two sets of baked maps were necessary during production of an 3D Asset using Photogrammetry.
First maps to bake – xNormal
Once you load all your High Poly chunks, select all the High Poly pieces at once and right click>Base texture to Bake, and select the 64K Map exported from Capturing reality. Once the setup is done, you can bake the following maps.
Second set of maps to bake Using 6M Mid Poly asset rather than 85M polys due to performance limitation in Substance Painter
Once all the maps are done, set up the asset backed maps in its slots so the smart masks and filters can work properly in substance.
* Normal Map, Base Texture and Cavity Maps are baked in Xnormal due to allow the load of all the 60 Chunks from the High Poly asset.
Set up Fill Layer as following Base Color – Base color map from Xnormal
Metallic – 0 Value (vary from asset)
Height – Height Map from Xnormal
Setup Input maps baked in Xnormal + Maps baked in Substance Painter.
To start to create the Roughness valuer another Fill Layer was created to contribute with the Roughness values, using the Base Color map from Xnormal, once again, and a Level filter to tweak values and generate a first pass Roughness map.
The first step for creating the first step to delight the base color texture, was creating a folder with it’s mask filled with the cavity map
The next step was to create a Level to invert the Cavity map values and tweak it so it would mask strictly the areas more affected by shadows/ Occlusion
Now that anything in the folder will affect an defined area; a filled layer with a color from choice, picked from the original Base Color map + reference of albedo standard color chart is introduced. Masked by with Smart masks and generators, it adds color variation and helps equalize the color values of the areas once defined as more shadow and occlusion affected
As a second pass to Flatten the Albedo, I create a folder masked by a Dirt generator to define the areas that needs to be affected by the second delight pass
The save process is continued with the Fill Layer that lives inside the folder just created. A Filled with a neutral color value is masked by a Dirt Generator and in addition to a Level to introduce a second pass of color values in the areas most affected with shadows and occlusion, flattening the Base Color Map so the asset can better behave in more varied light setups and scenes.
To ensure Texture density in case some areas of the scan has a few blurred pixels and not sharp details.
I used a Filled Layer with a Concrete Substance material, and set the blending mode to Overlay, so an extra pass of was introduced and would guarantee that no texture detail discrepancy would happen if the asset is placed in any scene right next to a procedurally generated texture or so.
To help integrating the assets with each other, I added a soft pass of Dust so a common weathering feature was distributed throughout all the assets scanned.
Adding passes of weathering on top of the scanned asset can benefit the asset, helping it feels like it belong to the same world where all the non scanned assets lived, introducing a soft procedural look to the asset.
Keep in mind that the scanned asset already contains a lot of weathering features scanned along with the asset; so is quite easy to over detail the asset, and losing the real world feel that a photogrammetry asset have.