NVIDIA Updates Omniverse with Generative AI and OpenUSD

At SIGGRAPH 2023, NVIDIA talked about a range of fantastic AI features it had been working on. We have covered them in this article, but one of them deserves to be discussed in detail.

The company announced a major release of Omniverse, its platform allowing users to create OpenUSD-based 3D workflows and applications. This new version is offering applications and services for developers to optimize and enhance their 3D pipelines with the OpenUSD framework and generative AI. It "accelerates the creation of virtual worlds and advanced workflows for industrial digitalization."

One significant update is that Cesium, Convai, Move AI, SideFX Houdini, and Wonder Dynamics are now connected to Omniverse via OpenUSD.

NVIDIA also introduced advancements to Omniverse Kit, the engine for developing OpenUSD apps:

• New modular app building — A new Omniverse Kit Extension Registry, a central repository for accessing, sharing, and managing Omniverse extensions, lets developers easily turn functionality on and off in their application, making it easier than ever to build custom apps from over 600 core Omniverse extensions provided by NVIDIA.
• New developer templates and resources — New application and experience templates provide developers getting started with OpenUSD and Omniverse with a major headstart with minimal coding.
• Boosted efficiency and user experience — New rendering optimizations take full advantage of the NVIDIA Ada Lovelace architecture enhancements in NVIDIA RTX™ GPUs with DLSS 3 technology fully integrated into the Omniverse RTX Renderer, and a new AI denoiser enables real-time 4K path tracing of massive industrial scenes.
• Native RTX-powered spatial integration — New extended-reality (XR) developer tools let users build spatial-computing options natively into their Omniverse-based applications, giving users the flexibility to experience their 3D projects and virtual worlds however they like.

It also upgraded its USD Composer, a customizable RTX-accelerated app for OpenUSD scene assembly and composition, and Audio2Face, an AI-powered tool for creating facial animation from an audio source.

Omniverse USD Composer now lets 3D users assemble large-scale scenes, while Audio2Face includes multilingual support and a new female base model.

What's more, the company announced a variety of frameworks, resources, and services to accelerate the adoption of OpenUSD. There is a new Omniverse Cloud APIs helping implement and deploy OpenUSD pipelines and applications and new Omniverse connections.

NVIDIA is also planning to deepen its connections with Adobe Substance 3D by making Adobe's Firefly generative AI models available as APIs in Omniverse.

Wonder Dynamics is connected to Omniverse with OpenUSD export support through its AI platform Wonder Studio, which automatically animates, lights, and composes computer-generated characters into live-action scenes.

You can also import Luma AI’s reality-capture models in USDZ format and build content compatible with spatial computing platforms such as ARKit and RealityKit – useful for developers working on apps for Apple's Vision Pro headset. Khronos Group’s OpenXR open standard is now supported as well.

Good news for Houdini users: it is now possible to load Houdini Digital Assets directly into the Omniverse viewport. Moreover, Cesium for Omniverse enables 3D Tiles, and the Cadence DataCenter Design Software is available via Omniverse, helping users see computational fluid dynamics simulations.

NVIDIA has been working hard on its products it seems. Find the full list of changes here and join our 80 Level Talent platform and our Telegram channel, follow us on Threads, Instagram, Twitter, and LinkedIn, where we share breakdowns, the latest news, awesome artworks, and more.