Resident Evil 7: The Use of Photogrammetry for VR

IGN Japan did a wonderful report from the CEDEC 2016, although the translation comes from the all powerful NeoGAF and user Dusk Golem (thank you, man!). The report mostly talks about the things, which are relevant for the newest Resident Evil, but there are a couple of technical details, which might be interesting for our readers. Mostly because the developers talked a lot about the production of VR games for PSVR and the use of photogrammetry in modern gamedev. Here are the most important bits of information from the presentation.

• Resident Evil 7 started conceptual development in 2013, and started actual production in January 2014. PSVR was decided for the project in November 2014, and they used The Kitchen Demo as a testing the water build to both tease the basic concept of RE7 and how it worked with VR.

• Capcom talks about their experience with VR to other companies and gives tips for those who want to develop VR games for PSVR. They mention if making a longer game, like they are with RE7, it is important to give the user options due to VR tiredness, some have no problem while others get sickness, etc. So for RE7 they’ve been testing and making changes to it all throughout development, and recently implemented VR options for speed, acceleration, and more. But also they let users go in and out of VR mode at any time, in the middle of gameplay you can pause and switch on or off to VR mode on PS4 if the gamer wishes to, and they can switch back at anytime to the other. They believe giving options is important, and the content of the game is the same whether you play in VR or not, which they think is important to make users not locked out of content just because they’re not playing in VR, as some can’t handle or afford that and VR should be focused on user experience. They also give a lot of technical information of how they handle depth, lighting, etc., for VR, and elements that developers may not initially think about which effects users in VR.

• Depth of field is important, and though they have a VR and non-VR at any time mode and the content is changed, there are slight variations made to the game in both modes to best support both modes. They mention some gameplay elements are slightly altered to make best use of most modes of play, but much of the change is mostly technical. An example they give is that their item inventory in RE7 looked like it was a 2D image in front of everything else in VR, which made it look unnatural, so they made alterations when in VR mode game to make it fit in the world much better and have more 3D depth when in VR. Melee weapons, like the axe, were also close to the player vision and looked weird to everything else and lacked depth, so they made some visual changes to make it look a lot better in VR as well.
• The method of how they do it in RE7 is through photogrammetry, They’ve made an advanced method of photogrammetry for games in RE7’s RE Engine. It is 3D scan technology which allows the computer to generate a model and textures by rapidly taking pictures of a person/object from all angles in rapid succession and generating the model and textures from those photographs, which are then touched up by 3D modelist in the follow-up. They have a few different methods to generate the models from the data for different type of things, but the end result is the same.
• For RE7, the reason for introducing this technology is because it got to a point where the technology they developed could create high-quality, equal to that of human hands effort, at a fraction of the development time, which allows them to make a lot of high-quality assets much more rapidly and make much more detailed environments and models, meaning they can make low-cost high-quality model data at a much faster rate than could be done previously. They mentioned the method could work with any type of photograph and they tested all sorts of different things for what caused the best results. Even taking shots at different angles with an iphone camera could work, but the quality would be a bit lower. They found a few best methods for RE7, but mention they found some camera techniques that had creepy results which they have put into practice for some of the elements of RE7, it creates a very uncanny effect.
• Capcom created a full-fledged 3D scan studio for RE7 they plan to use with future projects, as well as the RE Engine itself. They bought over 100 different types of cameras. 3D modelist still on board for the project, since there are certain aspects the system has difficulty with, like hair, and having fantastical elements are also important, so there will be need still of 3D modelist in the near future. This isn’t a method to get rid of them. They tested this technology a bit in Umbrella Corps, but it had limited use in the Unity Engine, which is why they worked to make it completely compatible with the RE Engine they were working on. They say a little over half of RE7’s assets were created using this method.
• They talk about things that would be harder to make in RE7 with this system, such as creatures who are not humanoid or zombies. They tease that RE7 indeed does have some of those type of monsters certainly, some of it was done by hand with 3D modelist if it was too impossible to recreate in reality, while others were done by tricky practical effects and make-up artist. They say the atmosphere of RE7 is really important, and some of the techniques they used made the development at times feel more like they were working on a horror movie than a horror game, as they’d have people dressed on set and hiring actors and using techniques to make such creatures in person.

• A word to people who try to follow what they do; the quality of the model will come to to the quality of the actor, make-up, etc. The system helps animate as well, so having actors who can act is actually really important.
• For development of the environments, they did a lot of location shooting. They went to various creepy areas in Japan, including Nara in Japan, and took a lot of different angled shots of things for their systems. They made sure to capture a lot of specific things, like a deer head, lanterns, gardens, They used equipment to combine several cameras together to take a shot at once from different angles simultaneously and automatically adjust brightness, color, focus, etc. They also worked with a lot of locations and set-up ‘miniature studios’ in various locations to take a lot of shots away from their actual studio.

• Now, there was some difficulty. The setting of RE7 is Louisiana in America. While it’s not uncommon to go overseas for scouting with projects, it was completely impractical to take all of this equipment over there to go scouting. They did do some outsourcing, but a lot of it was on their own shoulders. They scouted Southern USA and decided which things to take shots for and model from hand and which things to try and use their system for. Also in the United States there’s a bigger culture for filming. they worked with some people in the Hollywood scene to use southern houses and other locations for use in their game. Copyright of things they were taking photos of was one other extensive area they had to research and work with Hollywood studios to help achieve.
• They did a lot of unusual photography as well. They mention they did work for example in a meat processing plant. Some things they even used Claymation and other effects to achieve.

• To create some things involved Frankenstein monster-ing varying things to create a whole. He teases as an example one monster in the game was constructed of a lot of different pieces of meat they got from the meat processing plant.
• The director says while the system certainly had its difficulties, it was a lot of fun to do and they hope some of the unorthodox ways they made RE7’s world helps to make a high-quality, detailed, and a video game world unlike any other that gamers have experienced. The method even with its difficulties did actually save them a lot of time and money, cutting down the time it would of taken otherwise to make so many assets and of such quality by 40%. He did point out the system would definitely not work for every game, but it can be helpful to certain type of games and he believes has a place in horror games going forward. He says there’s good chemistry and bad chemistry for such a system, while it can save in costs and development time, it has its own challenges, and can fit some projects like a glove and others it wouldn’t fit at all.

Here’s a bunch of small photos from Famitsu, which were taken during the presentation.