A New Approach to Capturing Relightable Volumetric Videos

Google has developed The Relightables, a new system that features a spherical cage holding 330 programmable LED lights and approximately 100 cameras to capture volumetric video.

The team states that the new project is all about innovations. First, the system's cameras act as depth sensors, capturing 12.4MP depth maps of its subject. It presents a geometric and machine learning reconstruction pipeline to synthesize video from the captured data.

The project captures reflectance maps, time-synced to the depth maps, which manipulate the lighting of the subject in an AR or VR scene instead of using the original lighting.

"While significant progress has been made on volumetric capture systems, focusing on 3D geometric reconstruction with high-resolution textures, much less work has been done to recover photometric properties needed for relighting," states a paper from the team. "Our method combines the ability to realistically relight humans for arbitrary environments, with the benefits of free-viewpoint volumetric capture and new levels of geometric accuracy for dynamic performances."

The system could be used for augmented reality content, immersive storytelling, game production, and more. The team also noted that the project can change the way people communicate.

You can find the full paper here.

Built for Creators. Read by the Best

Partner with 80 Level

Comments

0

Leave Comment

Built for Creators. Read by the Best

Partner with 80 Level

Comments

0

We need your consent