Using AI for Real-Time 3D Face Reconstruction

Take a look at a work that proposes using a neural network to deal with facial reconstruction.

What is your take on facial alignment and real-time 3D face reconstruction? There are so many possibilities here, but are such things even possible today? Take a look at a work that proposes using a convolutional neural network to deal with the tasks. The method is said to be capable of generating high-quality outputs, creating each image in less than 10 milliseconds. What’s the magic here?

First, let’s check out a quick introduction from Two Minute Papers:

Here is an abstract:

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

We propose a straightforward method that simultaneously reconstructs the 3D facial structure and provides dense alignment. To achieve this, we design a 2D representation called UV position map which records the 3D shape of a complete face in UV space, then train a simple Convolutional Neural Network to regress it from a single 2D image. We also integrate a weight mask into the loss function during training to improve the performance of the network. Our method does not rely on any prior face model and can reconstruct full facial geometry along with semantic meaning. Meanwhile, our network is very light-weighted and spends only 9.8ms to process an image, which is extremely faster than previous works. Experiments on multiple challenging datasets show that our method surpasses other state-of-the-art methods on both reconstruction and alignment tasks by a large margin.

Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, Xi Zhou

You can find the full paper here.

Links:

Paper
GitHub

What are your thoughts on the proposed method? What do you think about its limitations?

Special brush for halloween, perfect for detailing your ghosts, devils and various killing machines!

You might also like

1
2
3
4
5

A Week After "Basically Announcing" Minecraft 2, Notch Basically Cancels It

Instead, he and his team will focus on the previously-announced retro-style roguelike.

+2

Rumor: Possible Release Date for Grand Theft Auto 6 Revealed

A video game store from Uruguay appears to have disclosed the launch date for the gaming industry's most anticipated title.

+2

Breaking: Unity Suddenly Lays Off Numerous Developers With a 5 AM Email

Apparently, the entire Unity Behavior team was cut, alongside many other employees.

+4

65 Brushes
by Mels Mneyan
These 65 brushes is everything you need to draw materials, textures and other things.
Cartoon Water Shader
by Adam Homoki
Cartoon Water Shader is an easy to use highly stylized water material, and river tool with many features. Demo scenes also included!