Startup Covariant today detailed RFM-1, a large language model that allows industrial robots to ask for instructions on how to perform a task when they can’t complete it on their own. 

The artificial intelligence model also lends itself to other tasks. According to Covariant, RFM-1 can generate short videos that depict industrial robots performing tasks such as moving merchandise between boxes. Those videos, the startup claims, can be used to equip industrial robots with the ability to plan their actions.

Covariant, officially embodied intelligence Inc., is backed by more than $100 million from Index Ventures and other investors. It provides software for powering warehouse automation systems such as robotic arms. The company’s software reduces the amount of time it takes to program robots for merchandise processing tasks, a chore that historically required custom code and a significant amount of time. 

The newly debuted RFM-1 language model features 8 billion parameters. It was trained on images and videos captured by warehouse robots that run the company’s software. Covariant also used data generated by those robots’ built-in pressure sensors and other components, as well as information from the public web.

There are multiple research datasets geared toward training robotics-optimized artificial intelligence models. However, Covariant says those datasets mainly contain footage of warehouse automation systems deployed in labs. The footage the company used to train RFM-1 was captured by robots installed in production environments, which makes it a more valuable source of knowledge for the AI model.

RFM-1 is capable of processing natural language instructions. As a result, warehouse robots powered by the AI model can be taught new tasks using plain English prompts. A worker could, for example, instruct a robot to pick up items from a pallet and place them on a nearby conveyor belt.

Programming warehouse automation systems to perform new tasks historically required writing custom code, which can take a significant amount of time and effort. According to Covariant, enabling workers to perform the task with natural language prompt could ease robot fleet management.

The company says that a robot powered by RFM-1 can not only understand instructions but also ask for them. “If a robot is having trouble picking a particular item, it can communicate that to the robot operator or engineer,” Covariant researchers detailed in a blog post. “Furthermore, it can suggest why it has trouble picking the item. The operator can then provide new motion strategies to the robot, such as perturbing the object by moving it or knocking it down, to find better grasp points.”

RFM-1 is not limited to language processing tasks. According to Covariant, it doubles as a video generator.

RFM-1 can generate short clips depicting a robot performing tasks such as moving merchandise. The company says those clips enable automation systems to perform “online decision-making through planning.” When a robot is asked to perform a new task, it could use RFM-1 to generate a video that shows how the task should be performed and then study the video to identify the best course of action.

Covariant will start rolling out RFM-1 to customers’ warehouse robots in the coming months. In the longer term, the company plans to create more advanced AI models that can automate a broader range of robot configuration tasks. Covariant will increase the amount of training data it collects to support the development effort.

“We expect to scale up our data collection speed by at least a factor of 10 through the robots coming into production soon,” the company’s researchers detailed.

Photo: Covariant