The founder of Labrodex Studios Jim Ivon talked about the way his company is creating futuristic VR environments for the upcoming UE4 game.
Approaching modern VR
Before getting started, I wanted to say thank you very much for the opportunity to speak about Labrodex and our current project. My name is Jim Ivon, and I am the owner of Labrodex Studios located on Long Island. I founded the company in early 2016 with a focus on VR / AR entertainment titles. When mapping out the company, there were several challenges we needed to address. The first was finding top talent on the east coast. We began aggressively recruiting for industry professionals and currently have a fantastic mix of AAA talent, industry veterans and recent college graduates.
We have a few projects in the works, with the primary one being a VR sci-fi shooter with RPG and exploration elements. We are extremely excited about this project and how it’s coming along. We haven’t released substantial information to the public yet, but details are coming soon. We’re going to share a few screenshots with 80.lv today to give you a sneak peek at the game. Our goal with internal projects at Labrodex is not just to create great games, but to create new gaming universes centered around deep and original IPs.
We opened our doors in November 2016 and by December 2016 we had 5 people on staff. We’ve been growing quickly since then and currently have over 20 people on the Labrodex team.
We are striving to combine the best elements of a AAA firm while maintaining the creativity, flexibility and speed of a boutique Indy studio. For example, as part of planning our first project, I wanted to include several AAA components. These included hiring a well-known, major award-winning composer to develop a unique score for the title. In addition, we hired a NY Times bestselling sci-fi author to write the prequel novel to the game. The book ends where the game begins and should be out later this fall. This will give a fantastic story foundation for the game.
Without giving too much away, the city itself is a key part of the project. We hired one of the world’s leading architectural concept artists to design the major buildings and city elements. We’ve taken those designs and recreated them in VR. The final look is fantastic. This individual has had dozens of his futuristic buildings, complexes and entertainment facilities built all over the planet.
The main challenge has been finding the balance between an authentic environment while maintaining optimal performance. From the outset, the team wanted to create a realistic world that you could interact with and be a part of in VR. This makes it incredibly challenging since you need a certain quality level of assets, textures, materials and lighting to achieve a real-world look. Combine that with the mandatory performance specs needed for VR and you create a formidable development environment.
There have been some incredible VR experiences to date, and we are hoping to build on those experiences. Our goal is to take VR another step forward by offering a compelling and rich story that brings the player into the game. One thing I can share is that you will be able to travel around large levels without teleporting or point and click movement. We’ve built what we feel is a fun and natural way to travel.
VR was a perfect fit for the title since scale is a major factor in the story. VR gives that incredible feeling of realism when looking at tall structures, machines or even a boss or two!
Initially our writing team had to catch up to the design team. We were doing everything simultaneously, and although this was extremely fun and an exciting way to build, it certainly helps to have the story ironed out for the level designers. Each level was sketched out and planned according to the missions and look of that level.
One issue we found was that VR accentuates the flaws in design. Issues that are not visible on a monitor become painfully obvious in VR. It took us a while to get comfortable with the level design direction overall. We built numerous test levels to “try out” certain looks, heights, scale and to test the believability factor. There were many times where scenes looked great on screen, but looked very different in VR.
We then adopted the approach of putting key assets into VR as soon as the base model was built. Because of the quality limitations in VR, this meant rethinking how items were textured, how they stood out at a distance and what they looked like up close. For example, humanoid characters take on a slightly different look in VR versus on screen in ZBrush. By avoiding substantial efforts to retopo, UV and texture an asset until it was viewed in VR, we’ve managed to speed of the development pipeline. Nothing is worse than realizing you need to go back to the beginning because the look of the model in VR is vastly different than what it was on screen.
We’re still evolving and learning as we go. Every day presents a new challenge that keeps us on our toes and humble. We wanted to keep that “wow” factor and needed to rethink what was important and what was mandatory. Even minor changes, like adding a simple stationary light can change performance in VR.
We started working heavily on the first demo in December 2016 and by January 2017 we had a very rough proof of concept in the works. In early January, we hired our Executive Producer who has over 20 years of AAA experience. After he saw what we had, we talked about an aggressive plan to make GDC with a vertical slice of the game. The staff was on board and this was our first major challenge as a company. Could we make a basic prototype that would get the feeling of the game across in only 2 ½ months of development time?
The demo for GDC was done in Unity and we set an aggressive schedule to get it completed on time. The demo included an outside area, along with one level of the game to demonstrate travel, immersion and the unique elements of the game. The GDC demo had voice overs, complex animations, effects, missions and some pretty cool AI mixed in. The reaction was very positive from the companies we met with, which included Microsoft, Sony, Valve and Twitch, to name a few. That said, I was saying a little prayer to myself each time someone put the headset on in hopes they would have a great experience.
We of course had a tremendous amount of work to be done after GDC. We accomplished what we set out to do, which was to show we could put together a vertical slice quickly that captured the essence of the game.
In order to make GDC, we needed the internal team to focus on the core assets that defined the game, while finding alternative ways to create the secondary assets. One of the major stumbling blocks was the need for a fully modeled cityscape that surrounded the rooftop your character starts on. At this point, we had about 10 people involved in the project. Instead of pulling those resources to create the city, I decided to check out TurboSquid to see if we could find some ready built assets that fit into the game.
Because this was VR, we needed to make sure the assets were already optimized, met certain texture and material requirements and looked the part once you put the headset on. After searching and locating a few asset packs of sci-fi themed buildings, I had our level designer look at the details and verify they would work. What’s great about TurboSquid is the amount of detail you get when looking at their models. You can clearly see if they are game ready, what the poly/tri count is, have they been textured, what maps come with them, have they been unwrapped, animated and what file types are included. We made the purchase and decided to try the building pack out in game. Within a few hours we had a skyline surrounding the rooftop and it saved us a couple of weeks of development time so we could stay on schedule and make GDC.
Once we returned from GDC, we switched to Unreal and have been recreating all of the assets. We want every model in the game to be unique, original and created by our art team. This includes the buildings and items we used in the original demo.
We’ve tested many ideas out in VR to get our level building in line with the vision of the game. For example, we wanted to test out a camera moving along a cityscape while the player is attached to the camera. Instead of needing to create a new city, we were able to reuse the existing TurboSquid buildings to quickly assemble a city skyline. This was a major time and cost saver for this small experiment.
This has been the biggest learning curve to date. We could do several articles on the trials and tribulations of optimizing content for VR. One of the lessons we learned quickly was to profile the project constantly. We’ve had dozens of instances where a model, particle effect or animation looks fantastic onscreen, runs extremely well in the editor, then when you bring it in VR it kills performance.
Now, when we make any major changes or additions to the game, we profile the project immediately. We check everything against the base scores we had previously. FPS, latency, draw calls, lightmap density, shader complexity, triangles on screen and everything else. We record these results and refer to them when running a new set of profiling tests. With numerous people contributing to a project, we found it very helpful to constantly profile levels. Something as simple as adjusting a particle effect can wreak havoc with performance in VR.
There is also the extra challenge of getting things to run at a constant 90fps. We try to consider everything when building a level in VR. We’ve had intense discussions with the level designers about building specific geometry to maximize resources, reduce calls and limit drops in framerate. We’ve spent a good amount of time and effort to put real thought in not only how levels and models can “look cool”, but how can we design them with VR in mind. For example, we are making an effort to design intentional, story driven occlusion culling based geometry as part of our strategy.
Getting scale correct was something we decided to do right from the start. Unlike a first-person shooter, in VR you get the entire surround feeling. This makes it incredibly important to get scale correct. We found a key way to do this is to scale things according to real-world sizes. This meant actually measuring items in engine.
For Unreal, you can measure items by hitting the orthographic view icon, select the appropriate view, zoom in or out on your object, then hold the middle mouse button down and pull. You’ll see the measurement in Unreal units. We did this with all character models to make sure they appeared as they would in the real world. This simple step made designing the rest of the environment much simpler. Measure your characters first to make sure they match real world scale, then you can use them to assess the rest of the items in your level by placing them around the world. This works especially well for determining the realism of objects at a distance.
When designing in VR, we are trying to take everything into consideration. Are we using LODs on all effects? Do we have aggressive LODs setup on all of our models? Have we setup custom collision boxes to avoid unwanted performance hits? Are we designing in a modular way, but not too modular? Are we merging specific actors based on visibility? Instancing meshes where we can? Adjusting light map density? Avoiding movable or stationary lights unless absolutely necessary? And of course, profile, profile, profile.
The biggest tip we have that might help VR developers is as soon as your framerate drops, stop what you are doing and find the culprit. Once reprojection kicks in, you will drop to 45fps from 90fps. You might have only lost a few fps, but to avoid volatile framerates, the engine drops you to a new max number that you can maintain without spikes. As soon as framerate drops, stop and figure out what happened. It will most likely be the last thing added.
Originally we waited and said “let’s optimize everything all at once!”. This was a mistake for us, since there are hundreds of things affecting performance. Eliminating one at a time proved too daunting. We ended up rebuilding the level from scratch, and as we added items back in, profiled the level and eventually found the problem areas. Once you get to that magic 90fps, do everything you can to stay there. Your team will love you for it later on.