Palo Alto-based Rhoda AI, a developer of foundational artificial intelligence robotics models, said today it has raised $450 million in eary-stage funding to train intelligent robots using publicly available internet videos.

Premji Invest led the Series A round, bringing the valuation of the startup to $1.7 billion.

The internet has been a prolific ground for providing examples of human activity. It’s a perfect place to get candid presentations of almost every task, from picking up items of every kind to packing boxes, sorting objects and completing various manual jobs.

Currently, most robots are trained through a process called teleoperation. This requires specialized remote-control equipment, including gloves and external sensors that collect positional data that can be translated into movements. Some systems can also work from numerous videos of actors taking the same action over and over.

This limits the total amount of training data that can be input into robotics models. To handle this, most robotics companies extrapolate from these limited data sets by generating synthetic datasets.

However, with the vast amount of data already available on the internet of hundreds of thousands to millions of humans already doing everyday jobs in natural environments, researchers have ready-made datasets.

Rhoda’s team said they believe that online videos will bridge the gap between limited data for orienting tasks by supplementing the small amount of robot telemetry with a broader array of natural human movements.

“In the case of teleoperation, if the phone orientation changes, that might be enough to cause the model to fail,” Chief Executive Jagdeep Singh told Bloomberg. “Whereas in our case, the model it’s seen so many other examples of objects that are at different orientations, it’s able to generalize.”

Teleoperation can give a robot a framework to understand the basics of a task, but it cannot give a robot the wherewithal to adapt. That’s where online videos come in to provide the context, generalization, different orientations, failure states, varied approaches and edge cases. Something that’s difficult to reproduce even with the “fuzzy logic” of synthetic generation.

The company calls the approach Direct Video Action and said it allows setups such as robot arms to deal with edge conditions better than current AI training models. The company added that it has run successful tests using off-the-shelf parts with an automotive firm and has plans to license its AI model to customers.

Singh said that Rhoda plans to build humanoid-style robots and has plans to make its own hardware to ensure quality for real-world work.

Other backers joining the round included Khosla Ventures, Singapore’s state-owned investor Temasek Holdings Pte and venture capitalist John Doerr.

Rhoda joins a number of companies drawing investor attention for a growing advancement called physical AI, where AI bridges the gap between the physical and digital worlds, allowing machines to perceive, reason and interact with 3D environments in real-time. Although reasoning and perceiving robots first come to mind, physical AI includes self-driving cars, adaptive assembly lines, robotic surgery and smart buildings. Where traditional AI “thinks,” physical AI is capable of “thinking, then acting.”

Image: SiliconANGLE/Microsoft Designer