New AI Imaging Technique from NVIDIA
Subscribe:  iCal  |  Google Calendar
Milwaukee US   17, Jun — 22, Jun
New York US   17, Jun — 20, Jun
St. Petersburg RU   19, Jun — 21, Jun
TBA BR   22, Jun — 24, Jun
Amsterdam NL   25, Jun — 28, Jun
Latest comments
by Yes
8 hours ago

Gabe is amazing and im happy to have him basicaly be the face of gaming. If you have something against him reply to this, i would love to discus

by Kai Marshall
24 hours ago

Awesome! So proud of you!!! <3

Positive site, where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work. rv altitude

New AI Imaging Technique from NVIDIA
24 April, 2018

A team of researchers at NVIDIA has presented a new state-of-the-art deep learning method that is capable of editing images and reconstructing a corrupted image with holes or is missing pixels. The team also states that it can be used to edit images by removing content and filling in the resulting holes. The method revolves around a process called “image inpainting” which could be implemented into photo editing tools to remove unwanted content by adding a realistic computer-generated alternative.

Existing deep learning based image inpainting methods use a standard convolutional network over the corrupted image, using convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes (typically the mean value). This often leads to artifacts such as color discrepancy and blurriness. Post-processing is usually used to reduce such artifacts, but are expensive and may fail. We propose the use of partial convolutions, where the convolution is masked and renormalized to be conditioned on only valid pixels. We further include a mechanism to automatically generate an updated mask for the next layer as part of the forward pass. Our model outperforms other methods for irregular masks. We show qualitative and quantitative comparisons with other methods to validate our approach.

Image Inpainting for Irregular Holes Using Partial Convolutions 

“Our model can robustly handle holes of any shape, size location, or distance from the image borders. Previous deep learning approaches have focused on rectangular regions located around the center of the image, and often rely on expensive post-processing,” the NVIDIA researchers said in their research paper. “Further, our model gracefully handles holes of increasing size.”

The researchers started by generating 55,116 masks of random streaks and holes of arbitrary shapes and sizes for training. 25,000 more have been generated for testing.

An example of the masks generated for training.

During the training phase, holes or missing parts are said to be introduced into complete training images from the above datasets in order to enable the network to learn to reconstruct the missing pixels. Then, at the testing phase, different holes or missing parts, not applied during training, are introduced into the test images in the dataset by the team, to perform unbiased validation of reconstruction accuracy.

The outputs for missing pixels used to depend on the value of the input that was supplied to the neural network for those missing pixels, which led to artifacts (color discrepancy and blurriness, for example). The NVIDIA team developed a method which uses a “partial convolution” layer to solve the problem.

Are you excited? You can learn more about the news here


Leave a Reply

Related articles