Breakdown: Sculpting and Animating a Detailed Evil Jester Character
Introduction
My name is Sia, and I've been learning 3D character art for 3 years. I've always been that kid who's always drawing, but it was never consistent for me — I'd get bored and quit more times than I could count. Now, looking back, I think I always knew that I wanted to work on games: growing up, my favorite game was The Sims 2, where I'd always spend hours creating characters. At some point, I even discovered The Sims 2 Body Shop and started redrawing textures in Paint to create custom clothing pieces.
My family was against me pursuing a career in the arts, so I got a government-funded Bachelor's degree in linguistics instead. I had tried to dive into 3D several times, but my ancient laptop couldn't run the software, so it never went far. In late 2022, I got laid off from my tech recruiter job, and I decided that I was done doing a job that I hated. In January 2023, I started learning Blender for ArchViz, but by summer I got bored with it and decided to focus on characters. It was also around the time Baldur's Gate 3 was finally released, which was a huge inspiration and a role model.
In early 2024, I joined a game jam, where in a team of four we had to deliver a game MVP in three months. I was a solo 3D Artist on this project, and this experience eventually helped me land an internship at a mobile game company shortly after. It was a very interesting experience, since the game we worked on was pushing the limits of mobile gaming — a third-person RPG with card-based combat, a character customization system, and pretty advanced graphics.
But my heart has always lain in AAA/console games, so I joined Think Tank Training Centre while still working full-time to develop a solid portfolio that can land me my next opportunity. Just before my last term, the company I worked for ran out of funding, and they had to close the project, so mentorship was the only term where I got to be a full-time student.
Evil Jester
Evil Jester (or just Jester) was meant to be the crown jewel of my time at Think Tank. I had very high hopes for the project: I wanted to develop complex cloth elements, include some armor, and have a visually interesting character with a lot of accessories. I didn't settle on the concept art until the very last minute, since nothing ticked all of my boxes.
I do have a strict rule when it comes to choosing concept art — I have to love it. I came across Jester by Dariya R. on Pinterest, and it turned out to be an actual artwork, not AI slop — and it was love at first sight. I didn't even run it by my mentor first; I was so certain that the concept was "it". I just started the blockout that same day, so he couldn't say no to me.
Dariya R.
I really came to love the MetaHuman pipeline for my characters. It really brings everything together and gives me more freedom in terms of presentation, plus it is a real game engine, so it feels like having an actual game-ready asset. I started by making a MetaHuman character inside of UE5.7 — I didn't really focus much on what it's going to look like; I just need it to vaguely represent what I'm going for and give me an overall idea.
I'm using a MetaHuman to Maya plugin to get access to the basemesh that I'm going to use for sculpting. I usually end up merging body and head at this stage, since keeping them separate causes more issues inside of ZBrush. At this stage, I was also actively looking up references for as many elements as I could; it's always helpful to have a real-life counterpart to rely on, in case a certain element doesn't make sense. I was also lucky enough to hear back from the Concept Artist herself, so she was able to give me some insight for a few character elements I was struggling with.
PureRef
I then made the blockout in ZBrush. At this stage, I really like to use a mix of ZModeler for modeling, DynaMesh and ZRemesher for sculpting — I really like to establish good topology as early as possible, especially for hard surface elements. My other go-to is GN ZBrush/Maya, since I often end up improving my ZRemesher topology, which can also save time later in retopology and ensures good subdivision for high-poly.
All cloth elements were then recreated in Marvelous Designer. I usually try to find real-life patterns when creating outfits, but I mostly end up modifying them quite a bit. This project was no exception, and it took me a full week to figure out my patterns. The trickiest parts were the jester sleeves, sash, and vest skirt, since I really wanted to try and capture folds from the original concept art. I ended up simulating gloves and boots too, since I wanted to add an extra layer of realism to them.
I couldn't figure out how to achieve the concept art look for the sleeves, since the fabric strips would just keep clashing; they wouldn't hold their shape and would just fall apart. So I decided to cheat a little and insert transparent patches of fabric in between strips to keep them together. There was a lot of fabric manipulation and simulation tweaking, and I ended up improving the shape further inside of ZBrush, but I was quite happy with the look I achieved in Marvelous Designer alone.
For the sash and shawl I was convinced about going for lace from the very beginning, but I wasn't sure if I needed to include it in the high-poly stage. I ended up finding fantastic vintage lace and shawl references online and using them to create alphas in Photoshop. For the shawl, I deformed my UVs to match the alpha and then finalized the ornamented edge for both the shawl and sash inside Substance 3D Painter, using custom alphas as brushes.
I then exported my alpha texture and used it in ZBrush. Dagger and neck brace patterns utilized a similar pipeline — I created a bunch of small alphas in Photoshop based on what I could grasp from the concept and then retopologized, UV unwrapped and pushed my assets to Substance 3D Painter where I created displacement maps for ZBrush.
High-poly took 10 weeks to wrap up, and this time I decided to push the details such as wear and tear, stitches, etc. It was easily the most artistic part of this stage. I ended up creating custom alphas using Photoshop and Substance 3D Painter, fabric folds are a mixture of alphas and hand sculpting; leather was fully hand sculpted.
Face
I decided from the start that Jester's face is not going to be the focal point of this project, but I still tried to keep it interesting. I've seen people refer to Jester both as "he" and "she", and it's very satisfying to me, since I intentionally went for a more androgynous look. I chose Bill Skarsgård as my main inspiration for the facial structure, mostly because of his Pennywise performance in the "IT" movie. I was quite liberal with sculpting, since facial likeness wasn't a priority.
Jester's hairstyle was an interesting element to develop. You can't really make out what the hair should look like from the concept art, so I decided to go for the wet, slicked-back look. I wanted each strand to look like it's got a ton of styling wax in it, and I mostly relied on my imagination, since I couldn't get a hold of good references. I went for a strands-based groom, since it just works best for a MetaHuman setup, and I have figured out the best pipeline for it on my previous project.
There's a huge bald spot on the head since it's only meant to be viewed from under the hat, but it has meme potential! For the Unreal Engine setup, I heavily relied on documentation and Andrew Giovannini's tutorials, which proved to be incredibly helpful — he provides altered scripts for Maya that do all the prep work for the final Alembic export.
Retopology
I always try to simplify my retopology process where I can. As I've mentioned before, I really like to establish good topology early on, which saves me a lot of time. As far as good topology goes — there is a small shortcut that I like to take, which also ensures that my meshes are animation-ready. For anything that is skintight even in the slightest, I use body topology for best skinning results.
I always keep optimization in mind and try my best to make use of every triangle. I would delete any hidden faces, collapse edges to close off meshes, mirror assets where I can get away with it and not lose on quality, etc. My character has a lot of elements, so I had to rationally use my polycount budget. I ended up under 150k.
UV-unwrapping is luckily never a big issue for me. I genuinely just get to relax and enjoy the process. My usual approach is trying to separate elements by materials and keeping certain parts separate for convenience (I just ask myself "Can this piece be hidden?" or "Would it make sense to have a separate texture in case I want to optimize/swap/etc").
There were a few complications I ran into while creating the low-poly, though. I created a visual breakdown for the most challenging parts of this outfit in case my pipeline and approach can help someone.
Texturing
Jester's face is based on a TexturingXYZ scan (also pretty sure that it was a female scan), which I then refined in Substance 3D Painter. I didn't really intend on rendering Jester without a hat or a mask, so I wasn't overly concerned with makeup. I was going for a distressed look to match the character aesthetic. My main inspiration for the look was Bill Skarsgård’s "Crow" makeup, which happened by accident, since I didn't start scoping makeup inspo until the texturing stage.
Makeup texture was probably the easiest part of the project, since I just got to paint it the way I wanted. I tried my best to make the look believable, which was a combination of paint layers, curvature, cavity and levels.
Outfit texturing always ends up being a trial-and-error run where I just try various materials, develop Color Variation, Roughness, Specular, etc. The most important part of texturing is having enough references and just building layers of grunge until it looks believable. I chose velvet as the main red fabric, silk for white fabric, two types of metal for variety, and multiple leather options to make it look believable.
Animation
As mentioned before, Jester is MetaHuman-based, so that saves a lot of time in terms of body rigging — you just get it for free. I still needed to rig the outfit, but truth be told, the latest versions of Unreal Engine offer a wide variety of skinning tools that are pretty good and in many cases manual skinning (for example in Maya) isn't necessary.
The best pipeline that I found for my character was to export a "Combined Skeletal Mesh" of the body from the MetaHuman Creator window, import all of the assets that need to be rigged as Static Meshes and convert them to Skeletal meshes, attaching them to the body Skeletal Mesh. Then I usually transfer weights and manually go in with the manual tool to fix any skinning issues.
I didn't really need to skin any simulated components. Vest, sash, shawl, and cape were made to be Chaos Cloth assets, while chains, ropes, and belts weren't skinned at all, as I've decided to use Rigid Body Physics. I'm not a developer, nor am I a Technical Artist, but I had a vision, and I needed to find a way to make my simulated garments look good. I had various complications, as my meshes weren't merged and if simulated they would just fall apart (vest had laces, cape had trims and fur), so I decided to use proxy meshes.
The new DataFlow system inside of UE5.7 wasn't explicitly set up for this, and I needed to understand how to do it. The best I could find was Unreal Engine documentation mentioning using LODs as proxies. That's what I ran with. I had to rework my topology to function well in simulation. That meant combining all of my meshes into one continuous mesh, rearranging topology so it could deform well. After some testing, I ended up with a quite different simulation topology for the cape and vest.
Another problem was making sure my Physics Asset (PA) was letting me have a multi-layered garment setup without clipping. For that, I created multiple PAs — one to compensate for Jester sleeves and leg armor to allow the vest, sash, and shawl to run smoothly, and another PA to compensate for the vest so the cape doesn't clip through, but also doesn't bulge and stick out.
Rigid Body Physics is not a new concept and has been around for ages. As usual, I did my research and tried to find applicable content, but most tutorials were covering older versions of the engine, while others didn't cover the MetaHuman Blueprint setup, which was key for me. Everything was done inside of the Unreal Engine. I ended up with three separate assets — hat accessories, belt system with bag and dagger, ropes from the boots.
I then created Skeletal Meshes with new skeletons attached to each asset and manually went in to assemble chains of bones for every physics body that I intended to have. I then skinned my assets accordingly and created PAs for each of them. The step that took the longest was actually making PA work. I created physics bodies for every bone and logically assigned some of them as Kinematic and some as Simulated. The belt system was the hardest to make, since it had to collide with the outfit, as well as collide with itself (bag and dagger).
It was working pretty well past the initial setup with correct margins and tolerances, but I just kept getting terrible clipping and jittering issues. It took me hours of tweaking to get it to the place where I felt good about it. Once I was satisfied with PA, it was time to test it in Jester's Blueprint — the way I intended for all of this to work is attaching every asset onto a character socket (hat accessories to the head socket, etc), and then enabling collision and assigning it to be a PhysicsActor. But what I didn't know is that this approach ONLY works with the MetaHuman's head. My belt system and ropes remained motionless.
I was very frustrated, since no one had brought up a way to fix this issue, so I had to make it work somehow. The fix came to be purely by accident — instead of attaching the simulated skeletal mesh to the socket, I had to use a "scene" component attached to that same socket, to which I then attached my asset. Somehow that extra padding fixed the issue, and I was beyond happy with myself.
To demonstrate the complex setups I've invested my time into, I knew I needed to do an animation, and not just some default movement. The idea for reverence came naturally to me, as it only made sense for the character, but I couldn't find anything close to what I had in mind. I don't own any mocap equipment, and the only markerless option was the Rococo AI solution, which I tried previously and wasn't happy with my results, so I didn't push it. I've tried Cascadeur one time in the past, and I liked the overall concept, though I was concerned about retargeting. But once I started looking into the software, I noticed the new plugin for Unreal Engine that allows you to iterate your animation with real-time feedback inside of the Sequencer, which was ideal for me.
I was worried about the quality of my animation, since I have no animation background besides my personal projects where I often pose my characters and do simple idles, but that's it. Cascadeur has an AI interpolation option, but for this specific animation it proved to be more problematic than any other type, giving me a lot of jittering and "snapping" into poses, so I stuck with standard interpolation types. I had a video of myself doing a reverence, so that was my main inspo besides imagination. The animation looked very jumpy at first, but I eventually made it work for the character. Because of the nature of my character, Jester can get away with quite a bit of "unnaturalness" to the movement. It made the character look more mysterious.
The final cinematic for Jester was created a day before I posted the project on ArtStation. The cinematic actually has two separate Sequences. One of them is solely based on Mixamo animations that I retargeted to my character a while back. I wanted to showcase the cape and fur cards that I've just added at the time, so I just sat on that video unsure what to do with it.
Then, a week before publishing, I found out about the new Markerless Mocap plugin in UE 5.8 and decided to give it a go. I was so impressed with the tool that it was a must for me to try it out. So I went ahead and recorded a video of myself doing reverence and idle, processed it through the plugin, and created a Sequence. I got very lucky with the environment too — I had in my possession the "Opera" environment by RagTarock, which proved to be a perfect match to my Jester.
Lighting
For rendering purposes, I've had three separate studio-like setups with varying lighting. I didn't go for anything crazy, since my main goal was to present my character, so it had to be well lit.
I've figured out that spotlights just work best for me; obviously, there's a lot of nuance in making materials look good in any given lighting. I always start with a key light to give me ideas for the rest of my setup. The second light is usually intended for the face to make it pop more, and another fill light to eliminate dark spots. A well-placed rim light is also a very strong addition to the light rig. While working on Jester, I found out about False Color — a very useful tool that's been used in professional cinematography forever.
It shows over- and under-exposed areas and everything in between, which helps to save hours of look development and eliminates guesswork. For my static images, I always use EXR format that then gets pushed to Photoshop. I use the Camera Raw Filter to color correct and tweak my renders, which has proved to be the best approach for me. For videos, I use Apple ProRes format plus Premiere Pro for editing.
Conclusion
Evil Jester took four months to complete, plus an extra two weeks to finalize all of the breakdowns and cinematics. I have this problem where I can miss something or it just totally flies over my head in the rendering stage, so I usually end up redoing my renders multiple times, which takes a lot of time — and I honestly haven't found a way to battle this. For example, at some point I hid the hat accessories and totally forgot that they existed until I noticed it in my "Evil Jester Demo — Final" video and had to rerender the whole thing.
I knew this character was going to be a challenge, hence my picking this concept to begin with. What I really like about the concept is that I was able to break it down into blocks — I had a clear plan in case I ran out of time. My priority was to make a baseline character — face, hair, hat, outfit, dagger, and bag. The next block was the cape without fur, then the fur itself. The only element that I decided to abandon was the hobbyhorse staff. I was very confident in picking it up later, but eventually decided against it, since the character was already rendered and developed. Adding that extra weapon meant redoing a big chunk of work for something that arguably won't add much of an impact.
If I had to point out the biggest challenge of this project — I'd say Unreal Engine integration for sure. I chose to do all of that with not much time on my hands, no real tech support, and just my intuition to rely on. I also didn't really HAVE to do any of this — it wasn't a part of my assignment. I chose to do so because I am very committed to my characters, and if I don't make them "work" — I feel like I didn't do a good enough job. Just rendering a static pose is not cutting it for me.
I was very lucky to be mentored by arguably one of the best 3D Character Artists out there — Wayne Chu from Larian Studios. He pushed me to give it my best, hyped me up when I brought up crazy ideas and weird pipelines, supported me on every stage, and made sure not to let me get too comfortable, meanwhile not trying to do my work for me and letting me breathe, think, and push through. I also had a lot of support from my Think Tank peers and friends who enjoyed seeing me work on this character and were there for me.
When I was only starting my path, I was so inspired by various 3D Artists — all I wanted was to get better at this craft and hopefully get to their level someday. And here I am 3 years later with multiple decent projects under my belt and ready to conquer the world. My biggest goal is to be a part of something that I will be proud of, something that will inspire others. I really wish that every artist in the industry never loses the spark and remembers why they started doing this in the first place.
