The Neural System That Creates Avatars From a Single Portrait

Check out MegaPortraits – a new system that is capable of creating high-resolution neural avatars from a single portrait.

Take a look at MegaPortraits: One-shot Megapixel Neural Head Avatars – a new system capable of creating megapixel avatars from a single portrait image created by Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Aleksei Ivakhnenko, Viktor Lempitsky, and Egor Zakharov.

To get one-shot megapixel neural head avatars, the creators of the project advanced the neural head avatar technology to the megapixel resolution with a focus on creating realistic avatars, especially when the appearance of the animated source image is significantly different from the resulted animated image.

MegaPortraits creates a high-resolution avatar in two stages. First, it captures the source image and reenacts it using the driving motion using the base model. Then, during the second stage, the system performs the enhancement of the output image frame by frame using a high-resolution model. The two models are trained using a large-scale dataset of videos and a medium-scale dataset of high-resolution images.

At each step, the system samples two random frames – the source frame and the driver frame. The model imposes the motion of the driving frame (like facial expression or the head pose) onto the appearance of the source frame and produces the final image.

According to MegaPortraits creators, the system can provide a much better cross-reenactment quality compared to similar systems so the output image can deliver a better reproduction of the source image including the appearance and facial expressions. However, the authors admit that the method underperforms in the region of shoulders and clothing and state that they will address this problem in the future.

You can learn more about MegaPortraits here.

The Neural System That Creates Megapixel Avatars From a Single Portrait

Join discussion

Comments 0

You might also like

We need your consent