LogInRebellion's Ludovico Antonicelli has shared exclusive insights about 3D scanning and game asset production.
Introduction
80.lv: Could you tell us a bit about the way you've got into Rebellion? We've talked a long time ago on 80.lv and we'd love to hear more about your journey?
Ludovico Antonicelli, Senior Photogrammetry Artist at Rebellion: Hello! It’s been a while right? Yes, we talked back in 2017 when I was just out of university. There is a good chance that the first 80 Level interview helped me get my first gig so massive thanks. I worked on Forza Horizon 4 as an Environment Artist and in 2018 I was contacted by Rebellion, as it was starting to build a brand new photogrammetry team to support the development of Sniper Elite 5. Although at the time I had already been into scanning, these past 4 years have been a deep dive into perfecting our pipeline and creating an extensive asset library.
The Team
80.lv: Tell us a bit about your team, what do you guys do, how many people work there and what kind of tasks you are tackling today?
Ludovico Antonicelli: The photogrammetry project started with just two members, myself and Kristóf Rosu. There are many more environments, characters, and material artists who were trained and occasionally took part in our trips, especially in the early days. As Sniper Elite 5 (SE5) production rumped up, we started hiring a dedicated team, which is now known as Rebellion Digital Asset and includes 4 additional Photogrammetry Artists, 3 Outsourcing Artists, and a Vehicle Artist, a Location Scout, and our Head of Outsourcing.
We take requests from the project's Lead Artist and Art Directors and take care of the full prop production pipeline from location scouting to final art implementation into Asura, our proprietary engine. We also maintain a face-scanning rig, but the shooting and final art production are completely handled by the character team as that’s a big task on its own.
As there are always projects in the works at Rebellion, we are constantly being asked for new assets to keep us busy. We, as most senior members of the team regularly undertake R&D projects, such as improving the color pipeline and maintaining our equipment. We need to stay on top of technology so we constantly look for pipeline and software improvements. Lately, we have been looking into improving our asset management as maintaining the library can be quite tedious work, but also very important.
The Evolution of Photogrammetry
80.lv: The world of photogrammetry sure has changed a lot. What were the main changes in the process that influenced your workflow? Maybe some particular tools?
Ludovico Antonicelli: Photogrammetry is a relatively established technology but it has exploded in the last few years, especially in game development. This is due to better cameras and computer processing power that can handle more photos and bigger meshes and textures. Games are also always pushing for better graphics and newer consoles can also handle larger polygon count which goes together with producing assets starting from a high-res scan. Software-wise in the last few years, RealityCapture has become more reliable and its API allowed for extensive automation, which is crucial in big productions.
Against popular opinion, we don’t make extensive use of Houdini for further automation. While the software is great, we found it is not appropriate as each prop is different and if we want it to look at its best, we cannot rely on the sole node pipeline for anything other than secondary rough organic props. Anything remotely glossy or hard-surface needs special attention to make sure there are no geometry smoothing artifacts that can easily pop up when using decimated topology and automatic UV and baking.
Tools and Equipment
80.lv: What kind of equipment do you use for your scanning needs? Could you talk about cameras, and other gear you're using, such as lighting tools or lenses?
Ludovico Antonicelli: We use Sony’s Alpha 7R IV cameras which have a 61-megapixel sensor. This gives us a level of detail that enables us to ensure our raw scans are futureproofed. This means that if in the future we want to rework a low poly asset we can reproject bigger textures starting from the raw reconstructions.
We use a cross-polarized flash which gives us a nice flat color, removing the need for any delighting. Having controlled lighting means that we can shoot at any time of the day on interior and exterior and we can set up our camera exposure settings to filter more photons which results in ideal photos: low noise, long depth of field, and appropriate exposure time to reduce motion blur. This also gives us a great geometry reconstruction and reduced clean-up times in ZBrush, making the whole workflow less expensive.
Of course, we use Macbeth chars to balance exposure and temperature, which is important to have a consistent library and save time when color correcting the photos through Adobe Camera Raw.
The last piece of equipment worth mentioning is the Scale Bar which allows us to do ground control and scale the asset to real-world size. This is very convenient as scale sometimes can be subject to interpretation and since the game is made by many artists it is easy to mess this up.
80.lv: There's a perception that you can successfully scan with a phone currently? Is that true?
Ludovico Antonicelli: No...
Just kidding, of course, you can. It is a good way of practicing to obtain full coverage of your subject. Often beginners can get overwhelmed by all the technicalities and it can be hard to find your way with all the information on the internet: reflex or mirrorless? zoom or prime? JPEG or RAW? How much RAM? Etc…
The secret to obtaining a good scan starts with good camera positions. Get out there and try to align all the photos, it doesn't matter which software you use. I would prefer to hire an artist using a phone who isn’t scared to squat and get dirty compared to someone that successfully aligned 3 photos with its new expensive camera.
It goes without saying that for sharper results you need adequate equipment. Once you get some good scans you might want to consider renting a professional camera and learn about exposure. After many tree stumps and shoes, you will want to scan something bigger which requires more photos and therefore a better camera.
The Advantages of Photogrammetry
80.lv: What makes photogrammetry so useful? What are the biggest production advantages of this particular workflow?
Ludovico Antonicelli: If accurately planned, photogrammetry will make realistic asset production fast and more consistent. I will not say that it will make it easier (or harder) as it will depend on the studio resources and artists' knowledge. Your production will need to be as streamlined as it can be to make the costs lower than traditional production. Taking photos and processing them is the least time-consuming bit and it can be a great way to give art directors an idea of the look of the game early on. You will need a handful of skilled artists to complete the job though, after having a reconstructed model there is still a ton of work ahead to finish the asset.
You can’t talk of advantages without considering the disadvantages. There could be hidden caveats to slow down your productions. Scanning anything historical or modern enough may need to be double-checked to avoid copyright infringement. If you are scanning in a different country you also need to check the local rules around public photography and filming. You want to avoid the “guerrilla photogrammetry” approach and double-check all the legal stuff before you get to the location.
Game Development
80.lv: Applicable to gamedev in particular, how do you work with assets? Optimizations, modularity, some tweaks, and changes? In what way do you adapt all the photogrammetry material to gamedev requirements?
Ludovico Antonicelli: A robust pipeline and documentation help the team to be on the same page. Every asset has its own set of challenges so your workflow should not be too restrictive regarding techniques. After establishing some common rules, we give artists the freedom to tackle each asset freely. This also opens up the possibility for workflows the lead didn’t think about. It pushes the team for constant improvements.
We taught our artist to be meticulous in making appropriate low polys and UVs as our games are scalable for most consoles so we need hand-crafted topology and consistent texel density that looks great on all kinds of hardware. Most of the assets' lowest LODs are automatically decimated starting from the one above as, at this point, the balance between fidelity and cost tends to make us want to save budget instead. This is especially true if you have a very convenient decimation tool hooked directly into the game editor which is very capable of maintaining the silhouette even at very low polycount.
We have quite a few Substance 3D Painter filters that help to speed up the texturing phase and make the look more consistent between different assets. One of the most time-consuming tasks during this phase is making sure the PBR metallics have the appropriate PBR value.
There are special types of assets that need additional care like anything that involves gameplay metrics: chairs, doors, windows. Also, these are usually part of a building kit as you want them to be used modularly. The most challenging assets are certainly vehicles as they require extensive use of overlapping and mirrored UVs to reduce texture count.
Conclusion
80.lv: What do you think are the new things that are going to change the game dev process in the same way as photogrammetry? What's next? How will photogrammetry change?
Ludovico Antonicelli: I am torn about how to answer this question. Gamedev involves so many different branches of ever-evolving technology that we could talk about this all day. As an artist, I will certainly keep an eye on any tools that will speed up asset creation. I don’t think we are too far from getting rid of manual UVs and LOD creation for good.
Regarding the “What’s next?” part of the question, I expect more advanced software that will streamline the process even further. Currently, file handling is a time-consuming part of the pipeline when reviewing work. Uploading and downloading big mesh and textures can take a lot of time and in big productions those minutes of waiting add-up. I am keeping an eye on Nira which allows you to view highly dense meshes and big textures through a web viewer that streams the rendering from their server to your screen. In a nutshell, it’s as if Sketchfab and Stadia had a baby together, but much more than that.
Below is a 100 million triangle and 32k texture raw scans rendering real-time through Nira. You can also open this on your phone.
And finally, we are recruiting! We are looking for a senior artist to join our scanning team. If photogrammetry is not your thing, we also have many other roles open across all disciplines. Head over to careers.rebellion.com to check them out.
Thanks for the opportunity of having a chat with 80 Level again. Happy scanning!