As supply chains become more complex, the demand for intelligent and autonomous robots has only grown and the tools needed to address this need require artificial intelligence and advanced simulation.

Nvidia Corp. today at CES 2023 announced major updates to Isaac Sim, its robotics simulation tool that allows designers and engineers to build and test virtual robots in realistic environments, such as warehouses, factory floors and city streets, in order to train their AI and design them better.

Using this simulation technology, companies can tap into Nivida Omniverse, a hyper-realistic collaboration platform capable of creating and operating 3D worlds and metaverse applications in the cloud. This allows teams that are split up around the world to easily sit down and collaborate with very little down time.

“Developing robots is a multidisciplinary endeavor,” said Deepu Talla, the vice president of embedded and edge computing at Nvidia. “Mechanical engineers, electrical engineers, computer scientists, and AI engineers come together to build a robot. With Isaac Sim in the cloud, these teams can be located around the globe and still collaborate efficiently.”

A new capability added to Isaac Sim allows human characters to be simulated in warehouses or manufacturing. There they can be tasked with familiar activities, such as stacking boxes or pushing carts. The robots can then go about their usual actions and robot programmers can watch them go through their commands. At the same time, the robot AI can learn to avoid the people and their potentially erratic movements.

As a result, if any hazardous conditions would arise with people, they could be headed off before putting robots and humans together. This means that engineers can reprogram the robots to better deal with the presence of humans, redesign the factory or warehouse floor to accommodate people and robots, and adjust safety features to eliminate hazards before they happen.

Isaac Sim takes advantage of Nvidia RTX technology, an advanced platform for ray tracing and AI for designing large-scale models, to allow designers to build and render physically accurate data of simulated environments. This includes RTX-simulated LiDAR, a type of laser emission sensor that robots use to guide themselves, so that robots in the virtual world behave exactly as they would in a real environment, including under various lighting conditions or in response to reflective materials on the floor.

Robotics researchers will also have access to a broad library of new simulation-ready 3D models that they can simply drop in for rapidly building physically accurate simulated environments. This includes everything from prebuilt large-scale warehouses, warehouse parts and popular robots. With these libraries, researchers can be ready-to-go to test robots in prototype environments and quickly start building.

Nvidia has made advances in a number of different simulation tools for Isaac including in Isaac Gym, which provides AI reinforcement learning, and Isaac Cortex, a collaborative robot programming tool. Additionally, the company is introducing Isaac ORBIT, which provides simulation for operating environments for manipulator robots to learn and plan motions for picking up, putting down and moving objects.

Talla said that arguably Nvidia is well on its way to “having a thousand to a million times more virtual robots for every physical robot deployed” speaking to the growth of the platform. It is already in use by over a million developers and over a thousand companies that rely on many different parts of it for their operations.

Telexistence Inc., a Japanese remote-controlled and AI robotics development company, has deployed over 300 beverage restocking robots across 300 convenience stores in Japan. Aiming to improve safety, the German national railway company Deutsche Bahn AG trains robotic AI models using Isaac Sim to deal with edge cases such as luggage falling off racks or tumbling out of carriages.

Robotics researchers and developers can download the new and improved Isaac Sim platform today and learn about platform features and capabilities.

Image: Nvidia