logo80lv
Articlesclick_arrow
Research
Talentsclick_arrow
Events
Workshops
Aboutclick_arrow
profile_loginLogIn

New Generation Method Turns 2D Images into 3D Models

DreamCraft3D produces high-fidelity and coherent 3D objects.

Researchers have presented a new hierarchical 3D content generation method: DreamCraft3D. It takes a 2D picture created by a text-to-image model and turns it into a high-fidelity 3D object.

To create consistent geometries, the researchers use score distillation sampling – a way to generate samples from a diffusion model by optimizing a loss function – via a view-dependent diffusion model, but this might compromise the texture quality. To improve the texture, they propose bootstrapped score distillation.

They then train a personalized diffusion model, Dreambooth, using augmented renderings of the scene to give it a 3D understanding of the scene. "The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene."

"Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting."

With this method, the authors try to solve the issue of inconsistency of generated 3D objects, which other models suffer from, and the results look pretty nice, at least on paper. Text-to-image diffusion models still have issues with finer details, like text, so you won't find any in the presentation. However, the 3D models made by DreamCraft3D could still streamline asset production and make artists' lives easier. 

Read more about DreamCraft3D here and join our 80 Level Talent platform and our Telegram channel, follow us on InstagramTwitter, and LinkedIn, where we share breakdowns, the latest news, awesome artworks, and more.

A procedural Substance material that will help you set up beautiful eyes in no time.

Error
(2000-0001)

Join discussion

Comments 0

    You might also like

    A Week After "Basically Announcing" Minecraft 2, Notch Basically Cancels It

    Instead, he and his team will focus on the previously-announced retro-style roguelike.

    Rumor: Possible Release Date for Grand Theft Auto 6 Revealed

    A video game store from Uruguay appears to have disclosed the launch date for the gaming industry's most anticipated title.

    Breaking: Unity Suddenly Lays Off Numerous Developers With a 5 AM Email

    Apparently, the entire Unity Behavior team was cut, alongside many other employees.
    • Zombie VDM Brush
      by Nicolas Swijngedau

      40 hand sculpted zombie detail brushes all contained within one Multi-Alpha brush.

      Error
      (2000-0001)
    • Hospital Props
      by Dekogon

      This project includes assets, maps, materials, Blueprints, and effects created in the Unreal Engine. Each asset was created for realistic AAA quality visuals, style, and budget.

      Error
      (2000-0001)

    We need your consent

    We use cookies on this website to make your browsing experience better. By using the site you agree to our use of cookies.Learn more

    ×