3d artist Josh Van Zuylen discussed the creation of CyberRunner – a phenomenal environment work, done in UE4.
3d artist Josh Van Zuylen discussed the creation of CyberRunner – a phenomenal environment work, done in UE4. This is an incredible level of craftsmanship, worthy of the the ‘Cyberpunk 2077‘.
Introduction
Hey, I’m Josh Van Zuylen and I’m from Melbourne, Australia. I’m the Lead Environment Artist at Zero Latency where we make free roam, virtual reality on a warehouse scale. I have worked on a number of projects across a lot of platforms including, VR, PC, Console and Mobile. These games range from sports, Rugby Challenge 3, on current gen console all the way down to the puzzle adventure games, PuzzleWiz on mobile. Most recently I have worked on Zero Latency’s world premier launch content, Zombie Survival, Engineerium and Singularity, while developing ground breaking techniques and approaches for large scale virtual reality. Primarily self-taught, I was formally educated at the Academy Of Interactive Entertainment, Melbourne Campus and Graduated with awards in 2013.
I have been creating objects from Lego, drawing environments and playing video games since I was a child. When I was 15 years old, I was looking up one of my favorite developer’s website “Bungie”, to look at the halo news. With curiosity, I checked out the careers page and what qualifications were required for the experienced artist role. I downloaded a copy of 3ds Max and once I opened it for the first time I immediately looked at the UI and said “Bugger that!” Curiosity again got the better of me though and few days later I opened it again and made a few cubes. Twelve hours later I had a house and was inspired, a career as an Environment Artist was inevitable.
My inspiration for CyberRunner started when I purchased the Blu-ray edition of Blade Runner. I had seen it years before as a kid, but never realized how much the style resonated with me until then. I also have a soft spot for almost anything cyberpunk, as I can picture myself living in those universes in my head. I went on to look extensively at the behind the scene footage and concepts from Blade Runner, delving deeper and deeper into the designs.
In this project, the style and image I wanted to create was of a brutal cold world, especially at street level where the downtrodden struggle to live. It’s a linear world where, quite literally, the higher you are the better off you are. The streets are filled with violence and shady activities and the people’s voices are being suppressed. The world I created attempts to reflect this.
Start
One day I found myself sketching out streets in 3D and playing with the style, it wasn’t even intentional that I make an environment, but I was having too much fun to stop.
Because I had been sketching out the environment in 3D I had a fairly good idea of how it was all going to fit together, I built proxy meshes for almost all of the props that appear and for some that were cut, this gave me a good idea of how many assets I would need and how long it would take. The environment spans about 4 city blocks, excluding the background dressing.
One of the biggest things about the universes I was trying to capture in this project is the art deco, dystopian future vibe. I usually play a lot with design and styling while in the proxy stage as it gives me a good idea of how something could look. This helped heaps when trying to make assets align with the style; you can see how some assets evolved over time, from the picture above.
Mesh Production
When I started this project I had aimed to really level up my skills, so I decided no matter how small the asset, everything would get proper treatment and be fully fleshed out. Because of this I quickly realized I was not going to be able to finish this the way I usually did things.
My pipeline though it changed a lot, consisted of any one of these programs at various times:
- Maya – Block out
- Maya – Proxy model
- Maya – High poly sub-div model
- Maya/Zbrush – Sculpting if needed
- Maya – Low poly and UV mapping
- Knald/xNormal/Painter – Baking High to low
- Photoshop/Quixel/Painter – Texturing
- Photoshop – Emissive technical set up
- UE4 – Asset set up and rendering
I revamped my Maya pipeline and started looking into custom tools, hotkeys and my own kit bash sets to make the menial tasks that required little thought, go faster. I also switched to using a pen and tablet as my main input device and made heavy use of Maya’s gestures. This improved my productivity by a fair amount, freeing me to focus on things that matter more. I started researching industrial design as well as manufacturing processes to lift my hard surface to the next level. I kept in mind a lot of things, from proper functionality and structurally sound constructed assets, right down to how the metal would be pressed and folded in the die. This level of detail enabled me to create objects that are assembled plausibly as a physical asset in 3D. In most instances I would say I did not go far enough in this regard and I am continuing my studies on this into the future.
I went through several iterations of my texturing pipeline as this is what was taking the longest to begin with. Normally the asset’s baked maps would come into Photoshop and I would texture everything by hand. I ended up creating an insane list of tools in Photoshop that let me export into Marmoset ToolBag and use mask based texturing workflows, very similar to DDO, in Photoshop. I ended up being able to paint in scratches and paint chips based on a complex masking system that sampled the 3D object’s maps data, in order to represent areas of wear and weathering accurately.
I made about half the assets this way in the end and unfortunately it was still too slow. The tools only let me edit one map type at a time i.e. diffuse. I switched over fulltime to DDO as we were using it at work at the time. This halved the amount I time I spent texturing, which was awesome and enabled me to spend more time on the details and design. Later in production I started using the Substance tools more and ended up texturing a few assets like the Police Spinner in it.
Materials
While working on the forms and shapes I wanted to play with, I was also looking at colours and how I was going to light the scene. Using the right colours for the project, was an interesting one for me, I had always assumed that the scene would have more fluorescent or saturated colours, on a large amount of the metallic and painted surfaces. In actual fact I found that the world presented in a more serious undertone, when the surface colours were quite dull, beige for instance. Colour is a big deal for me, so I had to add it in a way that made sense for the world, thankfully this was a no brainer as the Blade Runner movie and Cyber Punk world are littered with neon and emissive surfaces. These emissive surfaces ended up being the main source for all of the ambient lighting in the scene.
When starting the development on the shaders, I really wanted to push what most people out there are doing and take it to a whole new level. Focusing on bringing every surface to life in one way or another, I did this using a variety of techniques. I looked into subtle features mostly, including focusing on details a lot of people usually miss, such as the subsurface glow of a glassy emissive surface and the bounce lighting it produces. I didn’t use layered materials much in the environment as it made it easier to do more with the asset’s textures, if I went down a traditional texturing/shading pipeline.
One of the cool things I built was the wetness feature that I created for all of the assets in the scene. It gave me great control over surface wetness, water drops, flowing water, gathered and pooled water and general small water drops on the assets. This gave me control over every aspect as well as being able to mask it via vertex colour if it’s only partially wet.
The holograms were another pretty crazy shader. These had a ridiculous amount of features, segmenting the image into RGB pixels when close enough, glitching when an object is interfering with the projection pattern, scanlines, 60hz flicker, small image warps, large image warps and random signal loss.
The force field works in a similar way, but uses displacement based glitches that happen randomly across the whole surface, scrolling patterns every which way and flashing lights and text.
Unreal’s material editor is awesome and feature rich; it makes everything super easy and thanks to the profiling tools I was able to make the really expensive shaders run a little faster, without sacrificing quality.
Action
One of the things I had set out to do with this project was to make people “want” to watch the video. There are many projects out there that have a 30-60 second flythrough or pan over the environment and focus on what are typically quiet parts. This makes sense as an Environment Artist, you want to show off how you have fingerprint oil on the hand rail of your polished oak staircase railing, but I did not want to do it this way. Instead I tried to use action and the active world in order to get people to look at these details. A great example is the Police Spinner flying in after the logo, it flies past one of the buildings and lights up facade, pipes and structural reinforcing.
I’m no expert particle artist by any means, but looking at a lot of references, made me realise it was a massive part of the atmosphere of Blade Runner and CyberPunk themes. I dived into the depths of UE4’s particle editor, as well as Maya’s fluid simulations, in order to achieve all the smoke, steam, rain and sparks you see throughout the world. Parts of the world are always under constant repair, thus steam pipes are leaking, water mains dripping, electrical equipment sparking, etc.
I love making things feel alive even if it’s something like concrete or a light. The scene had a multitude of rain effects that ranged from normal rain and spray, to water moving down a surface, gathering and then falling off at the surface’s lowest point. The wetness of the scene was a huge part of the vibe, I invested a lot of time into creating many different particle effects and shader features, to achieve this look.
The animated water drops were pretty cool to make, they are a media texture that plays a video of normal mapped water drops. I created this effect by taking a spray bottle into my back yard and spraying a black piece of cardboard with milk, so it contrasted for the post work. I took the recording into Photoshop where I made the image a black and white mask, then turned all of the white areas into a normal, using layer styles. Once that was done, I exported the video as a normal map media texture.
I used the same alpha I created for the water drops earlier, applied a lens blur filter and exported that texture into UE4, then plugged it into the camera’s post processing for lens dirt. This gives a great sense of water dripping down the lens as it rains.
I spent a fair amount of time studying the properties of each type of particle. Looking up heaps of videos of the real deal on the internet was my main source; I ended up rebuilding particles a few times because I would always find better references. Sometimes I would just look out the window and watch the rain.
Lighting
Lighting was a massive area of improvement for me over the course of the project and I started out with fairly bland lighting. I received a lot of help from my colleges in this area and probably rebuilt the entire scene’s lighting 3 times in order to get the right look. I knew I wanted a lot of reflections across the wet surfaces and for assets to glimmer and shine, with all of the world’s neon lights. I tried to put as many emissive surfaces, as would physically make sense, into the world in order to make it come alive.
I built a wetness shader to help me achieve a more realistic shading scenario, this helped shade and highlight objects, when presented with a strong emissive.
As I had a lot of lit particle effects, lighting these properly was a fair challenge. Constantly positioning lights and effects so they light each other nicely while helping the scene, was an interesting proposition, as well as the performance overhead that came with so many real time lights. In the end I settled on a mixed solution of baked area lights and real time point and spotlights, the latter of which usually has particles associated with it.
Extensive use of the post processing inside of Unreal Engine also helped make the lighting pop. Here is a breakdown on each of the render passes including, unlit, AO, pre-post and post.
About mid 2016 after I had taken a few weeks off work, I was working away bouncing back and forth between the scene and the Police Spinner, when I realized the scene composition sucked. Making a very brave decision, I ripped the entire environment in half and flipped large parts and built two entire new alleyways. This enabled me to shift the focal point from the end of the main alleyway to a building at the newly built cross-roads.
Doing this gave a massive boost to viewer retention and added a much larger punch effect once the music kicked in. I didn’t have any single asset focal point on the building facade; rather I wanted people to take in the forms and verticality at the intersection. I’m not employing anything terribly fancy to do this, rather just lines leading to the building corner as well as the lighting being intense and vibrant.
Constantly adjusting the scene lighting, set dressing and composition was a day to day affair, I didn’t hold on to anything too long even if I thought it was right. This is a methodology I employed across the entire project in order to achieve the desired look.
Because I was constantly adjusting everything, building new assets and knew roughly how I was going to present it once it was done, I spent a lot of time set dressing and building composition for the fly though video. The video was created over the course of the entire project, getting new angles, cutting shots and iterating on ones already done. Ultimately the video was shot entirely in engine, in one master sequence with no alterations in post.
The iterative process is super present throughout the whole project; I started building the world as soon as I had my first asset in engine, playing with lighting and shaders too. The world grew around me, as it was more realized; I began to think less of what the world is supposed to look like and more about how the people lived. This process created a world infinitely more detailed.
Sometimes I would walk around the environment in VR and just meditate on the vibe to find direction when I was having a problem. The most problematic areas are where streets end, trying to figure out how to terminate a road is an awkward thing, and you have to think about how and why it is being blocked off.
Verticality
Because the world of CyberRunner exists between Blade Runner and Cyber Punk, where you have mega cities and mega corps, the new world is built upon the old in order for there to be enough room. This means you have old buildings that are the foundations for mega buildings where the wealthy live, these stretch well into the sky and above the layer of silt and pollution. This was not a particular main focus of my environment but it was an important aspect. I wanted to portray the height and depth of the city through the use camera angles and lack of lighting. Using long zoomed shots created a lot of dark areas which contrasted with the lit areas. The street lights don’t reach the upper limits of the environment so I made use of nice soft lighting to give an ambience, as well as feature spotlights to light up the façade of the buildings, and complement the rest of the shots.
Using Toolbag
I have quite a large amount of assets that I built for the project, many of which will never see the light of day, but I knew I wanted to give them their time to shine without making a video that went for 10 minutes. The way most of my assets are constructed made it easy to bring them into Marmoset ToolBag to render them. This was fun in itself, to see how I could make even some of the more boring assets pop. It’s also good to show how everything looks without the UE4 shaders on it.
Police Spinner
The police car, or as it’s referred to in Blade Runner “Police Spinner”, was a project in itself. Spanning half the project’s life it was slowly built and iterated on in-between the rest of the environment.
Since it is one of the most iconic pieces of Blade Runner, it started with a huge amount of research. Carefully studying the Blu-ray release of the film for shapes, details and the way it moved, a fair amount of reference came from the behind the scene shots and scale models. In classic movie fashion all of the cars full size and scale models where different, this meant a lot of picking a choosing what details to lose and what to keep.
In the end I went with aspects from every design and ended up with probably a model truer to the movie version then the movie versions. Not wanting to give up a chance to put a little bit of my own flair on to this beauty of a design, I changed several aspects to bring it away from the movie version. I ended up with something that has been built upon, something that’s a few generations later than the spinner we see in the original film.
I added details such as, levitation pads instead of wheels, LEDs instead of incandescent and fluorescent lights and gave these elements some movement to make them look a little newer. As the propulsion system has been completely rebuilt, you can expect a different rear underside. The rear body has a slight modification to allow more ventilation, with a light weight design for the new engines.
The car was fully modeled in Maya Sub-D, and mostly built by eye until I found blueprints for it, thankfully I was no more than 2cm out at any point, so it was a quick fix as it was in sub-d. Slowly pushing up the resolution and keeping the industrial design methodology in mind, I was able to complete a full high poly Police Spinner totalling 12 million triangles.
All of the texturing and baking was done in Substance Painter; I made use of Substance designer for making all of the materials and then made smart materials inside of Painter. It was a very interesting challenge as it was the most complex asset I had textured inside of Substance Painter. Substance Painter is super powerful and enabled me to texture the car, while on the train to work, by lowering the working resolution on my lower spec tablet and then bumping it all the way up to 8k for the Marmoset ToolBag Renders.
Once all this was done, I constructed the spinner Unreal Engine and created a blueprint to make it wobble and move about realistically as if it was hovering. I also had control over thrusters and lights and replaced the proxy model I had been animating at the same time. All of the actual locomotion in the trailer you will see is actually animated in Unreal Engines Sequencer along with the camera shots and triggered events.
This enabled me to iterate on camera angles, weird looking animations and add more animations faster than previously could. Once I was done I could drag it along my master sequencer timeline like it was nothing and re-order every event that happened in the flythrough.
I started this project back in September 2015 and finished in January 2017. As I work full time at Zero Latency, I worked on the project after work and on weekends. I estimate that amount of time I spent working on the project equated to approximately 6 months of full time work.
