Stockholm-based startup Rerun.io AB announced today it has raised $17 million in seed funding to build out a multimodal data stack for what it calls “physical AI.”

Today’s round was led by Point Nine and saw participation from Sunflower Capital and existing investors Costanoa Ventures and Seedcamp. Angel investors such as Vercel Inc. Chief Executive Guillermo Rauch and Y Combinator General Partner Nicolas Dessaigne also participated in the round, which brings Rerun’s total funding to date to $20.2 million.

The startup is building a data infrastructure platform that’s becoming increasingly sought after as generative artificial intelligence capabilities shift into the real world, via robots, drones and autonomous vehicles. Its database and visualization tools are built from the ground up to support the various types of data that’s streamed from these “physical AI” sources, including video streams, audio, sensor readings, tensors and 3D visualizations.

Its multimodal data stack supports important features such as visual debugging, which can help developers to piece together a robot or drone’s prior movements to understand where things might be going wrong. It enables companies to import the data from these intelligent machines rapidly, and then use it to create spatial visualizations and conduct detailed analytics.

Rerun believes there’s going to be big demand for this kind of platform because physical AI is rapidly becoming a thing. Indeed, it was a major focus at Nvidia Corp.’s annual GTC conference this week in San Jose. As more and more autonomous vehicles hit the road, as humanoid robots begin working alongside humans in factories and warehouses, and as spatial computing moves into the mainstream, a bottleneck is building up in the supporting data infrastructure.

Rerun co-founder and CEO Nikolaus West (pictured, center, alongside his co-founders Emil Ernerfeldt and Moritz Schiebold) points out that most of the world’s gross domestic product arises from the physical world, so that means there’s a massive opportunity for physical AI to transform the global economy. “But the current data stack wasn’t built for this reality,” he added.

Advances in robotics, drones, spatial computing and so on mean that AI systems are interacting with the real world more than ever before. Such systems rely on an assortment of cameras, lidar and sensors to perceive the world around them and guide their actions, but the data they produce doesn’t play nicely with existing database tools. As a result, it becomes difficult to understand and analyze this information properly, and that’s slowing down progress.

“Say a robot drives into a wall. Why did it do that?” West asked. “You need tools that allow you to see the world through the eyes of the robot to know why it acted the way it did. And that’s what we’ve built.”

The startup’s database is designed to support complex multimodal data that’s created by 3D scenes, video streams and tensors, and it’s combined with an open-source visualization toolkit that supports use cases such as simple observation and experimentation and training. Both tools can be accessed for free, but the company is building a commercial-grade version of its database that will become generally available later this year.

Analyst Holger Mueller of Constellation Research Inc. told SiliconANGLE there are many different data types around these days, and all of them are best handled in a database that’s specialized towards each format. But Rerun changes that with its ability to deal with multimodal data forms in a single platform, he said.

“It’s great to see this kind of innovation that attempts to tackle a real problem,” Mueller enthused. “It’ll be interesting to see what use cases the company tries to fix first, and how the entrenched competition will react to this kind of alternative. Just when we thought databases were getting boring again, Rerun spins that idea on its head, and there are exciting times ahead.”

The open-source versions have already gained significant traction, with a vibrant Discord community and users including Meta Platforms Inc., Google LLC, Hugging Face Inc. and more.

According to the startup, most robotics and drone developers have had to build their own data infrastructure and visualization systems, but doing so takes a long time and the results are often subpar.

Rerun Chief Operating Officer Schiebold said this has led to a bottleneck that affects the speed of product development.

“There is a data flywheel in physical AI, and the smarter your system is, the more units you can deploy, which allows you to collect more data to train on, making your system even smarter,” he explained. “Rerun helps teams run more experiments faster, which is the biggest lever on development speed.”

Point Nine Partner Ricardo Sequerra Amram said he was most excited about the strength of Rerun’s open-source products. “Its work on open source has allowed it to gain the trust of some of the most ambitious companies in the world of physical AI,” he pointed out.

Looking forward, Rerun wants to use the money to expand, and it plans to double the size of its team by the end of the year, while doubling down on its plans to launch the commercial version of its multimodal database. At the same time, it will continue to advance its open-source products to integrate them more deeply with robotics and other physical AI devices.

Photo and images: Rerun