NVIDIA AI Turns Sketches into Photorealistic Images

NVIDIA AI Turns Sketches into Photorealistic Images

The new GauGAN system uses generative adversarial networks to create highly realistic scenes.

The new GauGAN system uses generative adversarial networks to create highly realistic scenes.

NVIDIA has recently revealed a very cool new tech, which uses powerful AI tech to turn primitive doodles. The company says that ‘the tool leverages generative adversarial networks, or GANs, to convert segmentation maps into lifelike images’. Here’s a short video, which shows how it works.

NVIDIA believes that GauGAN could become a powerful tool for creating virtual worlds to architects, urban planners, landscape designers and game developers. This instrument provides a quick way to prototype ideas and make rapid changes to a scene. It should be perfect to brainstorm designs with simple sketches.

Bryan Catanzaro, vice president of applied deep learning research at NVIDIA, believes GauGAN is a “smart paintbrush”. It can quickly fill in the details inside rough segmentation maps. The software users can ‘draw their own segmentation maps and manipulate the scene, labeling each segment with labels like sand, sky, sea or snow’. The system, which has already seen a million images, will fill the space automatically.

GauGAN can produce convincing results because of their structure as a cooperating pair of networks: a generator and a discriminator. The generator creates images that it presents to the discriminator. Trained on real images, the discriminator coaches the generator with pixel-by-pixel feedback on how to improve the realism of its synthetic images.

After training on real images, the discriminator knows that real ponds and lakes contain reflections — so the generator learns to create a convincing imitation.

The tool also allows users to add a style filter, changing a generated image to adapt the style of a particular painter, or change a daytime scene to sunset.

The research paper behind GauGAN will be presented at CVPR conference in June. If you’re at the GPU Technology Conference, don’t forget to try out GauGAN at NVIDIA booth.

Join discussion

Comments 9

  • DigitalGod

    Even Top Notch Artists will be replaced by AI. You have no idea what you are talking about. If you do, only very superficial. At the end you are only an employee. You dont have any contact or experience to the High End Echelons we worked on. In 20 years, 40% of workforce working today will be out of jobs. First we will get worldwide financial crash, then AI takes over.

    Admin will remember my words in not distance future.

    0

    DigitalGod

    ·7 months ago·
  • sanek94cool

    Besides, if you'd be involved in project budgeting you would be aware that the costs are growing and using cheap alternatives is inevitable. This is the business. first of all.

    0

    sanek94cool

    ·7 months ago·
  • sanek94cool

    If you hate people that can make your life easier and see the threat in everything related to AI then you can hardly call yourself an artist. Rather than a kid who likes to be in a comfort zone.

    0

    sanek94cool

    ·7 months ago·
  • sanek94cool

    This is sad only for cheap projects and artists having no desire to grow.
    This technology in particular will make life easier for those who often use photostock services.

    0

    sanek94cool

    ·7 months ago·
  • crowbringer

    Cheap art will be just that. The true value lies in commitment and hard work. This is just a tool and without a creative mind operating it the results will hold no magic. Did photography kill painting? I think both are doing and developing rather well.

    0

    crowbringer

    ·7 months ago·
  • Matt

    Doesn't they say the same thing about photography when it was emerging? ;)

    0

    Matt

    ·7 months ago·
  • Drew

    Agreed. This is just depressing and is a detriment to society. If this keeps advancing at its current rate, good art will be so trivial to generate that it won't be special anymore. Art will slowly morph into a banal distraction, with creating an original piece being as easy as applying an Instagram filter. The role of the human artist will change from a craftsperson to someone who picks a bunch of parameters, gives it to the AI, and chooses the best output.

    This type of technology is a threat to the very existence of art as a craft, will completely devalue artwork, and will make the journey of training to become an artist obsolete. I hate these researchers for what they're doing to a field that I love.

    0

    Drew

    ·7 months ago·
  • Jason

    I disagree. There will always be demand for real artists. Like any other digital software, this is just a tool with the possibility to help artists create compelling worlds faster and add realism that would otherwise have taken days to make using other methods. As a 3D character artist, I would love to use this to create quick backdrops to place my characters in to enhance final renders.

    0

    Jason

    ·7 months ago·
  • Alan

    This is sad : /
    Ai advancements eventually will kill art.

    0

    Alan

    ·7 months ago·

You might also like

We need your consent

We use cookies on this website to make your browsing experience better. By using the site you agree to our use of cookies.Learn more