Smart Relighting Through a Webcam

Soumyadip Sengupta, Brian Curless, Ira Kemelmacher-Shlizerman, and Steve Seitz published their article named A Light Stage on Every Desk where they describe the mechanism behind their relighting tool. 

Normally, a light stage is a whole spherical light cage that includes a moveable digital camera, a light source, two polarizers in front of the light, and the camera and a computer. It is used to capture and relight subjects with the help of multiple light sources and cameras. The tool developed by UW's team makes it possible to use any monitor as an at-home light stage. 

The algorithm that turns your average monitor+webcam setup into a miniature light stage uses time-varying illumination that appears on your monitor to relight your face as the picture in front of you changes (like when you watch a vibrant, colorful video on YouTube and your face reflects those splashes of color). Inspired by the original concept of a light stage developed by Debevec et al., the team decided to make a relighting tool by training a deep network on images plus monitor patterns of a user so that it can predict images of that user under any target illumination. 

This is a big step forward since traditional light stages are expensive and require a lot of space and pricey equipment. Webcam-based light stages can be used, for instance, to improve lighting for video calls and conferences, live streams, etc. With the help of the newly developed algorithm, the video can be relit with a ring-light pattern resulting in a temporally consistent well-lit video.

The image below shows how the 'mini light stage' changes the lighting and color reflections on the face.

You can read more about the relighting algorithm in the original article. Also, don't forget to join our new Telegram channel, our Discord, follow us on Instagram and Twitter, where we are sharing breakdowns, the latest news, awesome artworks, and more.