LogInThe new GauGAN system uses generative adversarial networks to create highly realistic scenes.
The new GauGAN system uses generative adversarial networks to create highly realistic scenes.
NVIDIA has recently revealed a very cool new tech, which uses powerful AI tech to turn primitive doodles. The company says that ‘the tool leverages generative adversarial networks, or GANs, to convert segmentation maps into lifelike images’. Here’s a short video, which shows how it works.
NVIDIA believes that GauGAN could become a powerful tool for creating virtual worlds to architects, urban planners, landscape designers and game developers. This instrument provides a quick way to prototype ideas and make rapid changes to a scene. It should be perfect to brainstorm designs with simple sketches.
Bryan Catanzaro, vice president of applied deep learning research at NVIDIA, believes GauGAN is a “smart paintbrush”. It can quickly fill in the details inside rough segmentation maps. The software users can ‘draw their own segmentation maps and manipulate the scene, labeling each segment with labels like sand, sky, sea or snow’. The system, which has already seen a million images, will fill the space automatically.
GauGAN can produce convincing results because of their structure as a cooperating pair of networks: a generator and a discriminator. The generator creates images that it presents to the discriminator. Trained on real images, the discriminator coaches the generator with pixel-by-pixel feedback on how to improve the realism of its synthetic images.
After training on real images, the discriminator knows that real ponds and lakes contain reflections — so the generator learns to create a convincing imitation.
The tool also allows users to add a style filter, changing a generated image to adapt the style of a particular painter, or change a daytime scene to sunset.
The research paper behind GauGAN will be presented at CVPR conference in June. If you’re at the GPU Technology Conference, don’t forget to try out GauGAN at NVIDIA booth.