Nvidia Corp. today announced a major release to Omniverse, its hyper-realistic real-time 3D graphics collaboration and simulation platform, that will allow enterprises to build better 3D models and scenes using OpenUSD and artificial intelligence.

Nvidia Omniverse acts as a platform that provides artists, engineers, creators and others a suite of “metaverse” tools to collaborate remotely on virtual worlds at large scale to recreate and simulate anything they can imagine. With this powerful tool, they can simulate virtual worlds that include buildings, cars, airplanes and factories for the purposes of video games or industrial solutions including “digital twins” that can react with actual physics just like the real world.

Updates to Nvidia Omniverse announced today during SIGGRAPH 2023 included enhancements to its use of Universal Scene Description, also known as OpenUSD, an open framework supported by Nvidia that allows artists and engineers to collaborate on large-scale 3D projects. These included a new portfolio of technologies that connect cloud and generative AI application programming interfaces.

Using generative AI and OpenUSD will permit developers to build whole new applications and 3D environments in completely new ways. Using the power of AI, they can simply type what they want and have USD specifications created or renders generated, saving time and energy.

With ChatUSD, developers get access to a generative AI large language model that can generate Python-USD code scripts from text and answer questions about USD. It was developed using Nvidia’s own NeMo artificial intelligence framework, which allows anyone to become an OpenUSD expert by bringing them up to speed and allowing them to write code quickly. It operates similarly to how OpenAI’s ChatGPT works to generate computer code, but it’s trained on OpenUSD Python scripts and can work with what the artist or developer already has on hand.

DeepSearch is a large language model agent that can do fast semantic searches through 3D models and untagged assets based on text prompts. Many enterprise developers and artists have vast stores of 3D models at their fingertips and search them constantly, but they’re not always in well-categorized or searchable stores. Having a search agent to which they can just pose questions is intended to make short work of finding exactly what they want.

“Just as HTML ignited a major computing revolution of the 2D internet, OpenUSD will spark the era of collaborative 3D and industrial digitalization,” said Nvidia co-founder and Chief Executive Jensen Huang.

With access to the new Omniverse USD Composer, developers, artists and engineers will be able to build advanced scene compositions quickly at large scale at the same time and easily share them between colleagues and different applications, Nvidia says.

Nvidia recently joined with Apple Inc., Adobe Inc., Autodesk Inc. and Walt Disney Co.’s Pixar Animation Studios to form the Alliance for OpenUSD, an industry body that will build and develop OpenUSD, earlier this month. Under the auspices of the alliance, the group will work toward the adoption of OpenUSD and provide a written specification of the standard while furthering its development. Nvidia has also provided a roadmap of features for the framework that will make it more useful for autonomous industrial robots and industrial AI.

Adobe and Nvidia also announced further collaboration for Omniverse with plans to bring Adobe Firefly, Adobe’s family of image generating foundational AI models into the platform. Adobe introduced Firefly as a suite of image generating AI tools that operate similar to the popular image generating models such as Stable Diffusion, Midjourney and OpenAI LP’s DALL-E 2. The company added it to Photoshop in May, allowing users to use “generative fill” to modify images using text prompts.

“The latest Omniverse update lets developers tap generative AI through OpenUSD to enhance their tools, and it allows enterprises to build larger, more complex world-scale simulations as digital testing grounds for their industrial applications,” said Rev Lebaredian, vice president of Omniverse and simulation technology at Nvidia.

Other major improvements to Omniverse include a new Kit Extension Registry that will provide developers a central repository for accessing and sharing Omniverse extensions, which are plugins within the application. This will give them an easy resource for discovering what they’re using and how it is affecting the application and let them quickly turn on and off functions for building custom apps from more than 600 different core Omniverse extensions provided by Nvidia.

Nvidia also boosted efficiency for Omniverse with new rendering optimizations that take advantage of new Nvidia RTX graphical processing units with DLSS 3.0 technology found in Ada Lovelace architecture. The platform also has a native RTX-powered extended-reality developer tool suite for “spatial computing” experiences for developing virtual and augmented reality applications.

Image: Nvidia