Smart Relighting Through a Webcam

Researchers from the University of Washington, Seattle developed an algorithm that allows changing lighting scenarios during webcam calls without additional physical lights.

Soumyadip Sengupta, Brian Curless, Ira Kemelmacher-Shlizerman, and Steve Seitz published their article named A Light Stage on Every Desk where they describe the mechanism behind their relighting tool. 

Normally, a light stage is a whole spherical light cage that includes a moveable digital camera, a light source, two polarizers in front of the light, and the camera and a computer. It is used to capture and relight subjects with the help of multiple light sources and cameras. The tool developed by UW's team makes it possible to use any monitor as an at-home light stage. 

The algorithm that turns your average monitor+webcam setup into a miniature light stage uses time-varying illumination that appears on your monitor to relight your face as the picture in front of you changes (like when you watch a vibrant, colorful video on YouTube and your face reflects those splashes of color). Inspired by the original concept of a light stage developed by Debevec et al., the team decided to make a relighting tool by training a deep network on images plus monitor patterns of a user so that it can predict images of that user under any target illumination. 

This is a big step forward since traditional light stages are expensive and require a lot of space and pricey equipment. Webcam-based light stages can be used, for instance, to improve lighting for video calls and conferences, live streams, etc. With the help of the newly developed algorithm, the video can be relit with a ring-light pattern resulting in a temporally consistent well-lit video.

The image below shows how the 'mini light stage' changes the lighting and color reflections on the face.

You can read more about the relighting algorithm in the original article. Also, don't forget to join our new Telegram channel, our Discord, follow us on Instagram and Twitter, where we are sharing breakdowns, the latest news, awesome artworks, and more.

Join discussion

Comments 1

  • C Mono

    I recommend clicking through to the original article. You wouldn't believe it looking at the unfortunately badly chosen image for the article, but this actually looks very impressive when you see it in motion.


    C Mono

    ·4 months ago·

You might also like

We need your consent

We use cookies on this website to make your browsing experience better. By using the site you agree to our use of cookies.Learn more