Nightmare Reaper: How an Indie Studio Created a Retro-Inspired FPS Game

Nightmare Reaper uses a distinctive "2.5D" visual style with sprite-based enemies inside fully 3D environments. How did you approach recreating that classic DOOM-era look while still taking advantage of modern techniques?

I aimed to recreate the flat sprite look of old shooters for various reasons, but mostly because I love how it feels and think they have many gameplay advantages. I was very careful about implementing them in a way that would not sacrifice the feeling of sprites, but also use modern techniques to improve them where I thought it would fit.

As with everything in this project, with very limited resources and time, the final implementation wasn't perfect, but it was hitting most/all of the important notes in order to produce satisfying gameplay and visuals for the target audience. It's also important to note that this project started on a completely custom engine and that I ported it to Unreal Engine in order to have more graphical effects at my disposal.

So, many decisions might not fit in other projects. I also had zero experience with Unreal Engine, so many things aren't optimal, and I would certainly do many things differently nowadays. It's more important to ship a good game than a game that is perfectly made, though, especially if your resources are very limited.

Many enemies appear as billboarded sprites that always face the camera. How did you handle lighting, shading, and perspective so these sprites feel naturally integrated into the 3D world rather than appearing flat?

There are many ways of doing sprites, but only one that I consider "right". The sprites in Nightmare Reaper (and every classic sprite-based game) don't face the camera, but rather the camera's orientation. Many modern "boomer shooters" made to look like old-school games often make the mistake of having the sprites face the player, or in other terms, the camera.

While it can look good in some situations, it really breaks in others, and when making a game aimed at an older audience, I feel like this is an aspect that you have to get right. But then again, many games have been successful while doing sprites in a way I don't enjoy. Sorry for the tangent, but I think it's important to note that when you're making a game inspired by classics, it can get incredibly difficult to get it right because of the passion of the fans.

I think that sprites never felt flat. When you're playing a game like the original Doom, you quickly forget that the enemies are sprites and they just feel like 3D enemies that occupy the space organically. I didn't try to change any of that in Nightmare Reaper. I enjoy that these pixelated enemies fit the space. I actually feel that low-poly models are slightly more jarring and less immersive.

2D pixelated graphics have many advantages over 3D, and that's why there's always going to be games with that art style. But that's another topic. What I actually did change in Nightmare Reaper is adding dynamic lighting, shadows, bump mapping, and fancy effects like reflections and material rendering. I always wanted the game to have shadows to improve atmosphere and immersion.

I didn't do any special shadowing implementation for the sprites. Shadows cast from flat sprites appear on surfaces and look good. If you really pay attention to them, when a sprite is showing its edge to a light source, the shadow will become paper-thin, which does look weird, but absolutely no player notices that. It just looks good all the time, and it's easy to put it out of your mind.

Bump mapping is another thing that just seamlessly worked without any extra work. Every sprite has a "bump map" that will make the engine light it as if it had depth, and this just further improves the 3D look of the sprites. Sometimes spectacularly so. These effects and most others simply work great with sprites, and all I had to do was experiment, and I immediately noticed how well it works. It was really surprising.

Retro shooters traditionally relied on pre-rendered sprite frames for animation. Did you follow a similar approach for Nightmare Reaper, or did you experiment with hybrid techniques such as 3D-to-sprite rendering or procedural animation?

I don't like sprites that look pre-rendered. In classics, even if the sprites were actually pre-rendered (or made with physical models), every frame was combed through by hand to make sure that the pixels themselves looked good. If this isn't done, it just looks worse to me, as a hardcore fan of these classic games.

I think 3D-to-sprite is the absolute worst way to do it. It always ends up looking like a pixelated mess. I think it could be done in a way that looks acceptable, depending on the filter and art style, but I'll still vastly prefer hand-made. Which is what we did with Nightmare Reaper. Absolutely every frame of animation is hand-drawn. Every pixel was placed there on purpose.

It really makes the art pop out despite the low resolution, and the artist's style really comes through. This is a giant amount of work, though, but we're not afraid of that at Blazing Bit Games. As for procedural animation, I did experiment with it, but it looked really awkward and artificial. You still had to spend a lot of time modifying them by hand, so why not just make it all by hand and have the advantages of that?

What tools and pipeline did you use to create and animate the sprite-based enemies and weapons, and how did you ensure they feel cohesive with the fully 3D environments?

Nightmare Reaper is built in Unreal Engine, so sprites are just materials applied to flat billboards (custom ones because Unreal Engine billboards have limitations). There's a base material that offers sprite rendering, and every sprite uses it, or a version of it, to draw properly. The materials use different map inputs, such as the Texture Map for colors, the Bump Map for bump mapping, the Emissive Map for emissive elements like glowing eyes, etc.

As for tools, I'm personally using Photoshop 6.0 from the year two thousand. I use a free online tool to generate Bump Maps from Height Maps, and that's about it, really. As I said, we do things by hand. We definitely have the opinion that it's often just better to grind at it than set up a complex pipeline. The other artist on the project, Brouzouf, uses Corel Painter, Asesprite, and a more modern version of Photoshop for file conversions.

Many retro-inspired shooters struggle with readability when there are dozens of enemies and projectiles on screen. How did you approach visual clarity and effects design in Nightmare Reaper's chaotic combat scenarios?

I would say that Nightmare Reaper is definitely guilty of readability issues. This is mainly due to how low the art resolution is. The smallest pixel is actually not that small. Since I'm making a game I love, this was never a huge issue to me because I have an easy time parsing the visual clutter. I did try to improve it in many ways.

Visual clarity is most important when combat starts, so when that happens, the game ever so slightly desaturates while enemies keep their brighter colours. Enemies also have a certain degree of visibility through all the particles. A red outline can appear when they're behind an explosion, for example.

These effects are very rarely noticed by players, but they make the game feel ever so slightly better. Another trick is our excellent audio design. A tapestry of sounds will alert you about enemies and dangers at any point in time, even if they're not in your field of view.

Nightmare Reaper features an enormous variety of weapons with randomized stats and enchantments. From a technical standpoint, how did you design systems that could support such a large number of combinations without breaking balance or performance?

Like almost every system in Nightmare Reaper, this is all managed through very fast C++ custom code. The weapon system itself is very lightweight and doesn’t really affect performance in meaningful ways. What affects performance are the projectiles and their physics. This doesn't seem like the focus of the question, but essentially, pooling projectiles is very important, and thankfully, Unreal Engine can support a large number of physics objects with low performance impacts.  

The weapon system itself is a completely generic weapon implementation. By setting a list of variables to specific values, you can completely change how the weapon behaves. That means that if it wasn't for art and sounds, making a new balanced weapon would be(and has been) done in about an hour. This means that the weapon code is a bit complex, but we're not afraid of that either. Another layer of complexity and performance concerns comes with weapons in multiplayer, but that's mostly another subject.

The levels blend handcrafted design with procedural elements. How do those systems work together behind the scenes to keep runs feeling unpredictable while still maintaining a strong level of flow?

Don't tell anyone, but the levels aren't randomly generated at runtime. Most random level generation in games is done with rooms that fit a grid shape, but I've never enjoyed that, and I wanted truly organic levels with interesting shapes. Generating levels like that will cause them to coil over themselves and overlap.

The perfect solution was to pre-generate tons of potential levels and pick one at random at runtime. The elements of these pre-generated levels are then also randomized to a degree. This makes it so levels load lightning fast, and you never get the same one. Unless you die a lot on the same level.

I've decided to use handmade rooms that interlock instead of making it fully random because that was the fastest approach to make great-looking levels and keep things looking random. Making an algorithm that makes truly memorable rooms with their own identities is possible, but really hard.

By making them by hand, each room can have a theme and a point of interest to help players remember where to go next if they have to backtrack. There's still randomness to these handmade rooms, of course. Every level has its own identity, too. When generating the random level, there are variables you can set to have a certain level setup and challenges.

So even if the rooms are different and don't connect in the same way, you still have to solve the same "puzzle" to get through this specific level. For example, a level may require a red key, a switch puzzle, and be made of a specific number of rooms of specific types.

As for how this was all made, custom C++ again. It was a ton of work to make an algorithm that would produce levels randomly like that, but again, by writing a short script, you can generate a custom level in an hour.

With the new co-op mode allowing up to four players through the entire campaign, what were the biggest technical challenges in adapting a primarily single-player experience to multiplayer?

Nightmare Reaper was never designed for multiplayer, so this was a massive undertaking. I already had a lot of experience making multiplayer games. I worked at Ubisoft as a network programmer for almost 6 years. One big hurdle was discovering and working around Unreal Engine's limitations and issues around multiplayer systems.

Completely changing the state machine/order of events to game states was a real challenge, especially since I started this task with zero experience with Unreal Engine's multiplayer. The biggest challenge, though, was weapon replication. I had to change the system to synchronize weapons around inputs instead of actual weapon functionality because that would be too taxing on the bandwidth.

I also had to implement a hit validation system so that the server would be the authority on what happens in the game and could relay that to everyone.  That means predicting where clients were at the time they fired a shot and re-adjusting things that happened on clients to reflect what actually happened on the server authority.  

This was especially difficult because Nightmare Reaper uses real physical gravity-affected projectiles for every bullet. Unreal Engine doesn't synchronize physics, and they can behave extremely differently because of lag.

Nightmare Reaper's combat is extremely fast and effects-heavy. What kinds of optimization techniques did you rely on to maintain smooth performance while so much is happening on screen?

Heavy use of custom C++. This is much faster than using Unreal Engine blueprints. Another key aspect was the pooling of projectiles and also explosions, blood, gibs, and anything that might result in a future action. Depending on quality settings, we also changed a lot of systems.  

We could reduce rendering and processing distance of objects or lights, reduce the quality of particle effects like rain, remove water physics, use simple custom C++ physics for some objects, and limit the maximum number of objects of certain types at once. Unreal Engine/modern engines definitely come with a certain amount of bloat that is difficult/impossible to overcome, and I'm not fully satisfied with the results.

The game's retro aesthetic extends to its UI, minigames, and progression systems. How did you approach designing these elements so they feel nostalgic but still intuitive for modern players?

I honestly don't think UIs for shooters have changed that much since the days of "boomer shooters". A key aspect was keeping the chunky pixel art look everywhere. I tried to limit the amount of inputs as much as possible and use default controls that were the most familiar. I also made sure from day 1 that the game would work "well" with a gamepad, so that forces you to simplify and streamline a lot of things.

I also spent a great deal of effort to reduce the amount of text in the game. An image is much clearer and faster to understand, and I tried to design things that are as obvious as possible. I also don't want to hold the player's hand too much, and this was really important for me, so every part of the game is trying to show you that you have to pay attention and figure things out by yourself.

Looking back at Nightmare Reaper's development, were there any surprising technical discoveries or creative breakthroughs that significantly shaped the final look and feel of the game?

The game has evolved dramatically since the first concept, so there have been a lot of such events. The game was originally going to have flat level design, like Wolfenstein 3D, but I added some verticality to test, and I couldn't go back after that.

I think that using physics objects for bullets was another big one that I wasn't sure was going to work, but it feels really good to shoot because of that, and the physics handle it well. It's hard to say what else is a highlight because the whole development of the game was a lot of experimentation and transformation through play. That's why the game is fun, because it was shaped by me playing the game and guiding it towards what was fun for me.

Blazing Bit Games

Interview conducted by David Jagneaux