Google LLC’s artificial intelligence research unit DeepMind today unveiled a trio of new advances that it says will help robots make better, faster and safer decisions in the wild.

The advances, which include new training systems for artificial intelligence systems designed to power autonomous robots, help improve the way robots collect data and boost their generalization capabilities, among other things.

DeepMind said the goal of its research announced today is to create robots that can better understand and perform more complicated tasks, without being trained from scratch for each one.

AutoRT

The first advance is a new AI training system called AutoRT that combines large language models and visual language models with specialized algorithms for robots. The goal is to scale robotic learning and teach them how to perform useful real-world applications.

According to DeepMind, AutoRT is a technique that can help to teach multiple robots to perform different tasks in a range of environments simultaneously. The robot uses a VLM to understand the environment and the objects it’s seeing, while an LLM is deployed to suggests and select appropriate tasks for the robot to carry out.

The AutoRT system was tested extensively over seven months and safely trained 20 robots simultaneously. All told, DeepMind generated a rich dataset of 77,000 robot trials that involved them performing 6,650 individual tasks.

DeepMind explained that AutoRT incorporates a “robot constitution” that applies safety rules to the underlying LLM-based decision maker. The rules are based on Isaac Asimov’s Three Laws of Robotics, which stipulate that human safety always comes first, and that the robot should avoid any task involving people, animals, electrical devices or sharp objects.

Additionally, it relies on established safety rules that are applied to traditional robotics. As an example, a robot will automatically stop performing a task if it is stressed beyond a certain limit, such as if it tries to pick up an object that’s too heavy.

SARA-RT

A second system, Self-Adaptive Robust Attention for Robotics Transformers, is designed to help robot Transformer models learn to perform new tasks in the most efficient way possible.

DeepMind explained it uses a technique known as “up-training” to fine-tune robotic Transformer models. It has the effect of converting “quadratic complexity to mere linear complexity,” reducing the computational effort of the original model while increasing its speed.

DeepMind’s researchers said SARA-RT is the “first scalable attention mechanism to provide computational improvements with no quality loss.”

.
They explained that SARA-RT can be applied to numerous Transformer models, including those that process spatial data from robot’s depth cameras, expanding their application in the robotics industry.

RT-Trajectory

Finally, DeepMind outlined a new kind of AI model called RT-Trajectory that introduces visual contours to robot motion descriptions in training videos. By doing this, it can help robots to generalize more effectively and improve their understanding of how specific tasks should be performed.

The new technique works by overlaying a 2D trajectory sketch of the robot arm in training videos, providing the model with a more convenient, low-level visual cue that aids the learning of robot control strategies.

The researchers carried out a test involving 41 unknown tasks on a robotic arm powered by RT-Trajectory, and found that it more than doubled the performance of existing models. Overall, it achieved a 63% task success rate, versus the 29% success rate for an older technique known as RT-2.

DeepMind said it hopes these new systems will be adopted by the robotics industry and help it to build safer, more efficient and useful robots that can perform a wider range of tasks.

Images: DeepMind