NVIDIA's RANA Allows Animating & Relighting Avatars

NVIDIA has presented RANA, a relightable neural avatar that requires only a short video clip to be animated. Simply put, it takes a video of a person in any pose and then animates it while adjusting the lighting based on the given environment.

The company presented a "novel framework to model humans while disentangling their geometry, texture, and also lighting environment from monocular RGB videos." It estimates the coarse geometry and texture of the person and then learns an articulated neural representation for photorealistic image generation. RANA generates the normal and albedo maps and then uses spherical harmonics lighting to generate the shaded image in the target lighting environment.

The tool is pre-trained using synthetic images, and NVIDIA believes that it "leads to better disentanglement between geometry and texture while also improving robustness to novel body poses." It also presented a photorealistic synthetic dataset, Relighting Humans, to evaluate the performance of the approach.

Learn more about the research here and don't forget to join our Reddit page and our Telegram channel, follow us on Instagram and Twitter, where we share breakdowns, the latest news, awesome artworks, and more.