LogInThe approach reconstructs realistic strand-based geometry of human hair from video or images.
A team of researchers from the Samsung AI Center proposed Neural Haircut, a novel approach that enables precise reconstruction of hair geometry at the individual strand level from either a monocular video or multi-view images, even in challenging lighting conditions.
Their method comprises two main stages. In the first stage, they achieve joint reconstruction of coarse hair and bust shapes, along with hair orientation, utilizing implicit volumetric representations. Moving on to the second stage, they estimate strand-level hair reconstruction by effectively integrating the coarse volumetric constraints with learned hair strand and hairstyle priors from synthetic data through a single optimization process.
To enhance the accuracy of the reconstruction, the researchers introduced image-based losses during the fitting process, leveraging a new differentiable renderer. This combined system achieves impressive realism and personalization in the resulting reconstructed hairstyles.
"We reconstruct hair strands using geometry texture," reads the paper. "At each iteration, we sample a set of random embeddings from the texture and obtain corresponding hair strands using a pre-trained on synthetic data strand parametric model. These strands are supervised using geometric and rendering-based constraints.
The geometric loss (Lgeom) makes sure that hair strands do not deviate from the volumetric reconstruction and have proper orientations.
The silhouette-based and neural rendering losses (Lrender) utilize our proposed soft hair rasterization technique based on hair quads with learnable strand-based appearance texture to predict the silhouette and the RGB image.
Lastly, (Lprior) acts as a regularization penalty that improves the physical plausibility of obtained strands using a pre-trained synthetic hairstyles diffusion model."
You can learn more here and access the code over here. Also, don't forget to join our 80 Level Talent platform and our Telegram channel, follow us on Threads, Instagram, Twitter, and LinkedIn, where we share breakdowns, the latest news, awesome artworks, and more.