LogIn
Researchers together with NVIDIA presented a GAN-based approach that introduces disentangling time-lapse sequences in a way that allows separate, after-the-fact control of overall effects, like weather, cyclic effects, or any other that can destroy the fluidity of a time-lapse.
The model enables re-rendering the sequences in ways that would not be possible with the input images alone. It can stabilize a long sequence to focus on an object under selectable, consistent weather.
The GANs (Generative Adversarial Networks) are conditioned by the time coordinate of the time-lapse sequence.
"Our architecture and training procedure are designed so that the networks learn to model random variations, such as weather, using the GAN's latent space, and to disentangle overall trends and cyclic variations by feeding the conditioning time label to the model using Fourier features with specific frequencies."
The approach can amend some of the practical difficulties in capturing long time-lapse sequences, such as temporary occlusions, uneven frame spacing, and missing frames.
Simply put, it allows creating time-lapses in particular conditions without flicker due to random effects, such as weather, as well as cyclic effects, such as the day-night cycle.
You can check out the research and the code on GitHub. Also, don't forget to join our new Reddit page, our new Telegram channel, follow us on Instagram and Twitter, where we are sharing breakdowns, the latest news, awesome artworks, and more.
100% procedural material made in Substance Designer.