LogIn
The algorithm can recreate both the object's geometry and material.
Researchers Delio Vicini, Sébastien Speierer, and Wenzel Jakob demonstrated a new AI-powered algorithm that is capable of recreating real-world objects and turning them into digital copies of themselves.
According to the demonstration paper released by the team, the algorithm uses the physically-based differentiable rendering technique and signed distance functions to represent shapes. By analyzing reference images and known environment illumination, the AI can learn about an object's geometry and material and produce a thorough digital copy of the object.
"Physically-based differentiable rendering has recently emerged as an attractive new technique for solving inverse problems that recover complete 3D scene representations from images. The inversion of shape parameters is of particular interest but also poses severe challenges: shapes are intertwined with visibility, whose discontinuous nature introduces severe bias in computed derivatives unless costly precautions are taken," comments the team. "Shape representations like triangle meshes suffer from additional difficulties since the continuous optimization of mesh parameters cannot introduce topological changes. One common solution to these difficulties entails representing shapes using signed distance functions (SDFs) and gradually adapting their zero level set during optimization."
We also recommend checking out Two Minute Papers' recent video that describes this technique in more depth:
You can read the entire paper on differentiable SDF rendering here.