LogInJama Jurabaev and Oleg Zherebin from BigMediumSmall shared their character creation pipeline using a mix of photogrammetry and FaceBuilder and discussed how the clothes were made for The Wild West pack.
In KeenTools we develop plug-ins for Foundry's Nuke, Blender, and After Effects, among which is FaceBuilder, which lets you reconstruct faces in 3D from photos. This add-on was meant to be primarily a VFX tool. It created a buzz among Blender artists and eventually found its niche in gamedev as it greatly simplifies face sculpting in the first stages of creating a character.
The example we’d like to share with you today is exactly that kind of story. The BigMediumSmall team create highly detailed characters, weapons, and environment assets. When trapped by the first wave of COVID-19 in London with no access to photogrammetry equipment, they started using FaceBuilder actively in their work.
We’ve talked to Jama Jurabaev and Oleg Zherebin, the studio founders, about the work they were involved in at that time, The Wild West asset collection, how they created character faces from scratch, and the tricks they had to do due to the total lockdown. Here’s what they said.
About BigMediumSmall
Jama Jurabaev: Oleg and I have been friends for many years. We both live in London and help each other with various errands. Those used to be things like getting a visa, now it’s mostly work issues. For example, the real headache for a concept artist is working with assets. There are lots of pretty good architectural resources on the internet, but searching for characters is tough. You have to go through scattered bits in different setsб and the quality varies a lot. The good works cost a fortune, while the bad ones aren’t of much use. So we decided to make something of our own and set up the BigMediumSmall studio where we started creating highly detailed characters and environments. At first, we targeted the concept art market but then expanded to game development and filmmaking.
From the very beginning, we decided that we would model only what we are passionate about. We were both big fans of knights and medieval armor, so we devoted our first characters to those – we traveled around the UK, shot castles, searched for armor kits, and ate fish and chips for three months in a row. The result of all that was our debut collection Medieval featuring preset poses and animated clothes. All our low-poly characters were made from high-poly concepts. In terms of usage, we generally strived for simplification, so they were fully rigged and had precise polycount and all necessary texture maps for easy export.
In our second collection named Post Apocalypse, we dwelled on the corresponding topic and made military-style assets with a slight touch of sci-fi. There were soldiers, medics, and even flamethrower operators among the characters. As they say about the artists living in the former USSR countries, they don’t even need to search for references in that case. They may simply look out of the window.
Now then, two collections later, we were going as planned when we realized that if we wanted to achieve a higher output, we needed to find a quicker approach to faces. The thing is, all characters we had made before had masks on their faces – the knights wore helmets, the soldiers were dressed in chemical suits. Of course, we prepared faces for them. While full faces were not necessary for the knights as their helmets covered all of their heads except for a thin eye hole, the soldiers required more effort. And we did a good job, but all the steps from sculpting to texturing and so on had to be done manually. In fact, in the early versions of the pack, one face was simply cloned to all characters.
Aiming for 10-15 characters in our next collection, using that manual approach, we would have to spend 8 months solely on their faces. That was way too long for our pricing. We needed a solution.
Lockdown Messed Up Plans
Jama Jurabaev: After a while, we got down to The Wild West pack, with cowboys, horses, and a small Western town. In the process we had many challenges, especially when working on the environment. Yet the hardest one was bare faces.
As we were making highly detailed assets, the pipeline we developed for this collection included a lot of photogrammetry. We had advanced equipment at hand and we knew how to do it right. So we had successfully scanned two characters just before COVID-19 hit the UK and the country was on lockdown. We were stuck at the point when everything was prepared for scanning but we were not able to invite anybody to our office due to the strict quarantine restrictions. At some point, we almost gave up and were ready to turn the characters into outlaws by hiding their faces under rags.
Anyways, we searched for a different method and eventually came across FaceBuilder. By that time we already knew about KeenTools from one of the CG events, so the decision of implementing their add-on into our pipeline came without much hesitation. Inviting people to the studio wasn’t possible, but with that add-on in the library, all we needed was to get photos from the actors. Fast forward a few months, and we ended up with a combination of FaceBuilder results and photogrammetry.
FaceBuilder Pipeline
Oleg: As a test, we used FaceBuilder to create one of the main characters in that pack, named Sheriff Bert.
We had already tried FaceBuilder when it was in beta, so there appeared no difficulties in the workflow. We prepared a set of 8-10 photos taken from different angles and got started.
We created a 3D head dummy, combined all the photos, and got the following model.
In order to improve the quality and make it look more realistic, we did some additional sculpting, fixed the textures on the head and behind the ears, and replaced the default eyes. Sure, you can’t transfer all the details from photos to 3D that easily, especially if that’s your main character, meaning we were to polish it anyway. Also, we had to add a neck and a torso. The thing is, while the current version of FaceBuilder comes with a neck, the early version didn’t. And many people of the Wild West often walked around with an open neck.
As for the mesh, we used the one generated by FaceBuilder, except for the topology – it was replaced with our own template we always use for scanning. We used ZBrush for geometry transfer, where it can be smoothly done with the help of ZWrap. Morphing one mesh into another takes only a few control points on the character's eyes, nose, and ears. I’d like to note that the current version of FaceBuilder lets you use the generated mesh without additional topology polishing since they’ve added three different topology options: Low, Medium, and High Poly.
After all the polishing and adding more hair and clothes, we got a sheriff with a very distinctive look.
However, our ride wasn’t over yet. Apart from the main characters, we needed a crowd. We enjoyed making our characters look so detailed, but it took too much time and we still had to populate the town with a lot more inhabitants under a very tight schedule. That’s when FaceBuilder helped us out once again. We had a set of pictures of people taken from the front so we quickly made a bunch of good-looking heads.
The workflow remained as before: we replaced the topology, added necks, and sculpted the necessary details. We did far less polishing here compared to the main characters – the only thing they had to do is to look good from a distance. If you make heads quickly, that results in greater diversity, which adds more realism to the crowd, versus making one head and then simply copying it many times.
We took the built-in FaceBuilder UV and drew only some textures like the torso for example. Of course, it has minor drawbacks, like eye distortion, but you can’t avoid it. Generally, the UV workflow is pretty comfortable, you can easily fix the texture in Photoshop and add eyebrows or nose without sending it to Substance 3D Painter or other texturing software.
We used Maxface UV, one of the FaceBuilder built-in maps. It’s very handy because the whole head gets unwrapped as a single texture. We could add such details as eyebrows to the original front-view photos in Photoshop. But that only works for single pictures. It was easier to transfer many details to texture as we had the full set of photos from all sides.
Jama: We are very happy with the result. It is the best outcome we could get in the current COVID reality using only photos. In many ways, it is a great compromise between time and quality. And yes, it’s the quickest way to turn my Tajik nephew into an American cowboy!
Creating Clothes, Facial Animation, and Game Engines Compatibility
Jama: When releasing The Wild West, we didn’t adjust the characters for blendshape animations, we just rigged them. However, as we plan to grow from concept art to production-ready models, we are going to introduce preset body animations in our future collections. And, of course, we are thinking about facial blendshapes, if only we had enough time for that!
Talking about clothes, we make them in Marvelous Designer. Our workflow is, in a word, hybrid – some of the things we scan, others we do manually. For example, a hat is a static object, so it’s easier to scan it than model by hand. That’s why we shot the actor with the hat on his head. But our segment is pretty specific – we don’t make assets that you could put right into the movie, although the characters in this set are pretty well-made. They got meshes, clean topology, all well-optimized, so they could actually be used just like that.
Oleg: To make pieces of clothes fit with each other, we retopologized the whole geometry. The meshes were animated with waypoints, all folds were prepared for mesh deformations, exporting, and further development. We do some of the parts manually through Marvelous simulations, no news here, it’s a standard workflow for clothes in the industry. For instance, the trousers on one of the characters were made with photogrammetry, that’s why they look so realistic.
As for the faces, we combine automation and the manual approach here too. The ratio depends on the desired quality and the amount of time we have. Creating clothes is a topic that deserves a separate article.
Jama: As for game engines, the previous collections were not ready to be used out of the box. In fact, they were sets of PBR materials displayed as separate textures. When uploading models to Unreal Engine, you needed to readjust the nodes. But starting from The Wild West, we added import setups for Unity and Unreal. You only click on the character and get it in Unreal with all materials already applied.
Honestly, we envy modern concept artists who have assets like these. Their absence in our time was a real pain in the neck. And again, they’re very popular within indie and small post-production studios. Sure, we have not reached the desired pace yet, but we are going to release a few collections at a time by the end of the year, all pretty cool and unique. We will continue using 3D scans in our future projects. Manual work is great but it’s so time-consuming! As for faces, we will opt for the pipeline we implemented in The Wild West – a mix of FaceBuilder and photogrammetry.
Conclusion
Jama and Oleg have kept their promise to develop new assets. The interview took place in 2021, and soon after BigMediumSmall released three new collections made in collaboration with acclaimed concept artists.
The first one, The Lost City, a collab with Jakub Kozłowski, is devoted to a study of an abandoned Aztec city in the middle of South America’s jungles. Among new features are a fully rigged main hero and his mule with 7 preset poses, as well as Unreal Engine 5 compatibility.
The second collection is Mech Squad, full of highly detailed mechas and their pilots, looking like they have just stepped off a top sci-fi movie screen. This is no surprise since this bundle was made together with Min Guen, Sr. Concept Artist at Coalition (Gears 5 video game), former concept artist at FuseFX (Snowpiercer TV series), who took part in many other sci-fi projects.
The third pack – Industrial Zone – was designed with the help of Michael Yoshimura. This time, the team focused on handcrafting the vast industrial environment with highly detailed assets such as pipes, steel, and concrete structures, scaffolds, girder structures, warehouses, and set dressing elements. As for characters, BigMediumSmall created a set of photorealistic, fully rigged models of plant workers with an insane level of attention to the smallest detail! All assets are set up and ready to use in Unreal Engine.
As we were preparing this story, news arrived about the new work of Jama and Oleg’s team – the Grand Bazar collection – dedicated to Middle Eastern culture. The bundle includes high-quality Middle Eastern assets, a variety of handcrafted props like dishes, trees, and fruits, 24 authentic photorealistic characters, and even a fully rigged camel!
There is more to the story. Remember the Post Apocalypse collection Jama and Oleg mentioned in the beginning? Those characters and vehicles became the core assets in the short film Irradiation by filmmaker Sava Zivkovic made in Unreal Engine.
By the time Irradiation was released, the BigMediumSmall team had enhanced the collection, optimized it for Unreal Engine, and updated the characters' faces. As they mentioned above, they used to use only one face copied to all other characters. Now, under respirator masks, each of them has a unique appearance. The workflow they used here was a proven one – just like with The Wild West, the base models were first created in FaceBuilder for Blender and then polished by manual sculpting.
Go to BigMediumSmall’s website to check them out!
FaceBuilder is evolving too. Now you can use built-in FACS blendshapes to create draft facial animation. We’ve also added Epic Games’ LiveLinkFace app support to transfer pre-recorded facial animation onto FaceBuilder models.
In the latest 2022.2 release, we’ve introduced the new ‘MH’ texture map which is compatible with the MetaHuman texture layout. So now you can not only use the mesh to MetaHuman plugin for Unreal Engine to create your metahuman characters based on the 3D models exported from FaceBuilder but also refine the facial texture transferring skin features from photos in a more convenient way, making it perhaps the most efficient way to transfer a real person to a game engine with a high level of likeness!
All of that is available on KeenTools official website.