Baseten Labs Inc., a startup making it easier for developers to run artificial intelligence models in production, today announced that it has closed a $40 million funding round.

IVP and Spark Capital led the investment with participation from several existing backers. According to Forbes, the Series B raise values Baseten at more than $200 million.

Performing inference, the task of running AI models in production, can require a significant amount of time and effort. Developers have to prepare the infrastructure on which the model will run for sudden traffic spikes. They must also ensure that the AI replies to user prompts in a timely manner, avoid cloud cost overruns and complete dozens of other technical tasks.

San Francisco-based Baseten is working to ease the process. It offers a platform that promises to automate many of the tasks involved in running production AI environments, starting with the initial model deployment phase.

After developers train a new neural network, they have to package it into a format that can run on their organization’s cloud infrastructure. Baseten has developed an open-source tool called Truss that promises to speed up the process. The startup claims that AI models can be installed on its platform with a few lines of code using the tool.

Once a neural network is running in production, Baseten uses an autoscaling engine to ensure user requests are processed quickly. A sudden surge in requests can overwhelm an AI model and slow down response times. When the autoscaling engine detects a usage spike, it creates replicas of the AI model and distributes the extra traffic among them.

It also automates the task of deleting replicas once user activity returns to normal levels. The company says its platform enables developers to implement a scale to zero approach, or an arrangement in which an AI workload shuts down completely when it’s not actively used. That helps avoid unnecessary cloud costs.

Baseten provides a dashboard that enables developers to track an AI model’s infrastructure usage, as well as related metrics such as prompt processing times. A complementary observability tool eases the task of troubleshooting technical issues.

“Our native workflows serve large models in production so users don’t need to think about version management, roll-out, and observability,” Chief Executive Officer Tuhin Srivastava (pictured, center left, with the company’s other co-founders), wrote in a blog post. “In 2023, we scaled inference loads hundreds of times over without a minute of downtime.”

Companies can access Baseten’s platform as a managed cloud service or deploy it in their own Amazon Web Services and Google Cloud environments. According to Forbes, the software maker’s platform is used by about 20 large organizations and tens of thousands of developers. Its annual revenue is reportedly in the “mid single-digit millions.”

To further expand its customer base, the company plans to nearly double its 25-person sales and marketing team by year’s end. Baseten’s newly closed funding round will also finance product development initiatives. The company plans to roll out support for more cloud platforms, as well as tools that will help optimize the performance of customers’ AI models and make them easier to train.

Photo: Baseten