Physical Intelligence, a startup researching practical artificial intelligence models to create “brains” for robots, announced today it has raised $400 million in new funding.

The New York Times broke the news that the round was led by Jeff Bezos, founder and executive chairman of Amazon.com Inc., Thrive Capital and Lux Capital. Other investors participating in the round included leading AI firm OpenAI, Redpoint Ventures and Bond. The funding round valued the company at about $2.4 billion post-money. It follows a $70 million seed round in March this year led by Thrive Capital.

Co-founder and Chief executive Karol Hausman previously worked as a scientist on robotics at Google LLC and is joined by a team of researchers formerly from the University of California at Berkeley and Stanford University. The company seeks to build a universal AI model for robots that can understand the physical world so they can complete complex multipart tasks.

“What we’re doing is not just a brain for any particular robot,” Hausman told the New York Times. “It’s a single generalist brain that can control any robot.”

According to the company, today’s robots are specialists, most industrial robots exist to do a single task or a series of simplified motions. These robots can work around small changes in their environments but can’t adapt easily to extremely messy or complex spaces such as homes or other real-world places.

“AI could change that, allowing robots to learn and follow user instructions, so that programming a new behavior is as simple as telling the robot what you want done, and the robot can itself figure out how to adapt its behavior to its environment,” Physical Intelligence said in a blog post last week.

The company’s answer is an AI model named π0, or pi-zero, a general-purpose robot foundation model that provides a way for users to simply ask a robot to perform a task just like they can talk to a large language model for a chatbot assistant. Unlike an LLM, it needs to span a larger variety of data from text, images, video and “physical intelligence,” which is the embodied experience of moving limbs, grasping objects, manipulating them and taking other actions.

Using pi-zero, Physical Intelligence has demonstrated fine-tuning the AI model to get robots to fold laundry, make coffee, bus tables and assemble boxes. In the case of bussing a table, the robot had to identify the difference between trash and dishes. It would have to throw away trash into a bin while putting dishes into a bussing tray. However, it needed to get trash off plates, which it could learn to shake off before placing them in the tray.

The company said the biggest challenge in creating a generalist model is the lack of large-scale multitask and multirobot data right now. As that data set grows, it will help provide a stepping stone for larger frontier generalist models for more capable dexterous robot brains.

“We believe this is only a small early step toward developing truly general-purpose robot models,” the company said. In the same way that LLMs are a foundation model for language, generalist robot models provide foundation AI for physical intelligence.

There also exist similar robot control foundation models such as OpenVLA, a 7 billion-parameter open-source model, used commonly by academic researchers for experiments, and Octo, a 93 billion-parameter model. Parameters refer to the number of internal variables the model uses to make decisions and predictions. The company said its pi-zero outperformed OpenVLA and Octo on most complex tasks.

Bringing “brains” to robots is becoming a long-term trend for the tech industry. Last year researchers from Google unveiled a robot using PaLM-E, a 562 million-parameter model, that could understand basic single voice commands such as picking up and delivering objects. Nvidia Corp. also announced Project GR00T, a general-purpose foundation model for bipedal humanoid robots earlier this year.

“Our experiments so far show that such models can control a variety of robots and perform tasks that no prior robot learning system has done successfully, such as folding laundry from a hamper or assembling a cardboard box,” the company said. “But generalist robot policies are still in their infancy, and we have a long way to go.”

To get there, Physical Intelligence said, it will not just take a lot more data, but also a collective effort from the entire robotics community. The company said that it has several collaborations with companies and robotics labs that will help refine hardware design and the use of data from partners for pretrained models to work toward that vision.

Image: Physical Intelligence