Red Dead Redemption Fan Art: Photogrammetry Workflow

Introduction and Career

Hi. My name is Sergey Rudavin, I am a photographer from Ukraine, who now gets more and more immersed in the world of 3D graphics. I am extremely interested in transferring the real world to the virtual one, at the same time maintaining maximum photorealism and details.

I studied photogrammetry on my own, collecting information on the Internet and finding articles or people who could help me. For instance, Dmytro Dokunov and Vlad Kuzmin helped me. Thank you, guys.

I started learning in 2014 after the release of The Vanishing of Ethan Carter. In it the guys used photogrammetry. The world changed for me. I started scanning everything, but a lot of it did not work out well. Now It's easier, I try to immediately understand how the Reality Capture program will create a 3D model, even before I take the camera out of the bag. Of course, sometimes there are misses. After each shooting, I run home to find out whether the model works or not. Probably photographers, who have shot on film, run home to develop it and find out whether it worked or not, experience such a feeling. 

After I started to get the gist of photogrammetry, I realized that it was not enough and I needed to know how to process the model, clean the texture, prepare it. 

So, I went to 3D graphics courses in my city of Kharkov – 3dmaya.

Here I discovered ZBrush and Marmoset Toolbag, all thanks to Anna Beganskaya. She was my teacher. I did my homework straight on my 3D scans. Thus, immediately understanding what I needed, for what I needed it, and how it all would be better with it.

There are already two games in which you can find my scans. I am incredibly pleased, and it really motivates me to keep doing what I am doing. The games are Chernobylite and Half-life Alyx. And there are two other serious gaming projects for which I am freelancing. In general, I like that I can cooperate with different companies, create various projects. And the most important – I can meet many talented 3D artists and think about joint projects.

About the Model

To scan something inside the game, it turned out to be quite interesting and spontaneous. Most importantly, I definitely did not expect such a number of views, comments, and that now I would be writing this article at all.

To begin with, I no longer look at the world through the eyes of an ordinary person. I look at everything with the thought of how to scan it and where to use it.

Red Dead Redemption 2 is my favorite game today. And, of course, when I play it, I still think that there are objects that can be scanned, then I stop myself and say, ‘Wait, this is already a game, why do you need this?’ 

And when I decided to print a souvenir from the game on a 3D printer, I realized that first, it needed to be scanned. So, one free evening I sat down to make screenshots inside the game.

It was difficult to choose what to scan in this game! I would send it all to print, this game is a masterpiece. The house that I chose stood separately, nothing prevented it from being scanned freely.

I realized that there could be two problems. 

The first problem was to make a free camera inside the game. I managed to do it.

The second was for Reality Capture to recognize the screenshots as photos. This happened automatically and this was good because Nvidia Ansel registered the focal distance in the exif of the file. But I also had plan B. You can choose the focal distance yourself and register it inside Reality Capture. 

Working in NVIDIA Ansel

As a gamer photographer, I paid attention to the Nvidia Ansel program after the release. I did the first tests a little later inside the Metro Exodus game. I created several 360 panoramas and added them to my feed on Facebook. I like how Facebook displays them considering the gyroscope. Photogrammetry did not work out for me in this game. Now I understand that this was because I could not set up a free camera inside the game.

It is not difficult to enter the program. The game itself reminds us to press Alt+F2. After that, with simple settings, you can set up the camera and start collecting material.

I took 1167 screenshots. I have done everything exactly as I do in real life on my DJI Mavic 2 pro.

Reality Capture Workflow

The upload of material into Reality Capture is standard.

Vlad wrote an article about this. I did everything as described in it, only instead of photos, I uploaded screenshots from the game. 

Tweaking the Model

To lighten the mesh and prepare it for printing, I used two tools in ZBrush -- Decimation Master and 3D Print Hub. For my tasks, it is enough.

The only thing is that such detailed models should be printed on SlA 3D printers. FDM will not provide high quality. 

Texturing

I did not go deep into creating UV texture mapping, since I only needed mesh for printing.

The models that I create together with Igor Pogrebelniy for our store undergo serious processing. But I am sure that this one article will not be enough to describe and show the whole process of it.

Other Data Sources for Photogrammetry

Photogrammetry can be also created from photos and videos. I tried to create it from a video taken for the Rag'n'Bone Man - Human clip.

There is a moment when the camera is spinning around the guy. But then I could not get a 3D model of good quality, as I did not have enough frames. 

On the whole, if the quality of the texture is not important, you can try to create something from a 4K video from drones, for example. But assuming you have some good fly-bys around the object.

I love it when the texture is high-quality and in focus. Therefore, I create photogrammetry only from photographs.

But when I only need mesh, then there are tools thanks to which I can create a 3D model. A screenshot is no exception.

Sergey Rudavin, Photographer 

Interview conducted by Arti Sergeev