Capture Photorealistic 3D Scenes On Your Phone With Gaussian Splatting

Scaniverse is an app developed by Niantic that empowers users to create 3D models with stunning levels of detail with photogrammetry directly on their phones. The app's reconstruction algorithms support all modern iOS devices and don't require any extra steps or previous experience, you just have to point and scan, as simple as that. Scans could be easily edited just like photos by cropping, rotating, tweaking exposure, or contrast, as well as exported in popular formats such as OBJ and FBX for compatibility with 3D modeling software and game engines.

Now, Scaniverse supports Gaussian Splatting as well, allowing for the creation of incredibly realistic scenes of any size with an unprecedented level of detail directly on-device. For those unaware, Gaussian Splatting is a relatively new real-time 3D rendering method for images taken from multiple viewpoints.

It starts by reconstructing a point cloud from a set of input images. These points are converted to gaussians, each represented by its orientation, scale, opacity, and color, and then optimized to fit the input images, resulting in an extremely representative model. In addition, splatting reconstructs geometry far out into the distance, allowing your scans to become truly immersive experiences.

Unlike meshes, splats are powerful enough to represent transparency, reflections, and view-dependent lighting, making them a perfect choice for shiny objects, reflections, and subsurface scattering. Furthermore, since splat parameters are learned, they can adapt to any type of scene with the final result so realistic that you might mistake renderings for a video.

To get started with Gaussian Splatting in Scaniverse, simply open the app, press New Scan, and choose Splat. After the scan is complete, click Process for Scaniverse to start training your splat, which takes around a minute.

If you're not satisfied with the result, you can press the Enhance button for as many iterations as it takes to look good. There's also the option to create a short video of the final splat, which you can either save on your device or export in PLY format.

The developers also prepared a step-by-step tutorial you can check out below:

If you're not sure whether you need to create a mesh or a splat, take a look at Scaniverse's short guide below:

Create a splat if:

• You want a photorealistic result with high-fidelity lighting and reflections
• You want to capture beyond the 5-meter range of the LiDAR scanner or our photogrammetry mode
• You want a result that models the background and sky

Create a mesh if:

• You want to use the result with other 3D software or game engines
• You want a standalone 3D model without a background
• You need precise measurements
• You want to model a scene that doesn't have a well-defined subject (for example, a scan containing all rooms in a building)

Your older scans can also be converted to splats if you have the data saved. Simply open a scan, select Reprocess Scan, and then Splat. While Gaussian splatting is a pretty new technology, there are some plug-ins that can be used with game engines such as Unity and Unreal Engine, as well as 3D modeling software like Blender.

The developers highlight that the entire process of splat capture and training happens locally on a user's device and the data never gets sent to the cloud unless specifically chosen to do so by sharing a splat.

Learn more about Scaniverse's Gaussian Splatting here and join our 80 Level Talent platform and our Telegram channel, follow us on Instagram, Twitter, and LinkedIn, where we share breakdowns, the latest news, awesome artworks, and more.