OctoML Inc., a startup founded by University of Washington computer scientists to make artificial intelligence models more efficient, today disclosed that it has closed a $85 million funding round. 

Lead investor Tiger Global Management was joined in the round by returning backers Addition, Madrona Venture Group and Amplify Partners.

OctoML launched in 2019 to commercialize an open-source framework called Apache TVM that was developed by its founders at the University of Washington. Apache TVM and the commercial platform that OctoML has developed based on the framework enable developers to boost the performance of their AI models. According to the startup, it’s helping some enterprise customers make their machine learning workloads up to ten times faster than before.

Optimizing an AI model’s speed not only allows calculations to be completed faster but also reduces infrastructure costs. If a neural network’s performance is doubled, it can perform the same number of calculations as before using half as many processors. Increasing AI performance allows companies to make do with less hardware and thereby reduce their infrastructure expenses.

The challenge is that optimizing an AI to achieve maximum performance has historically been highly complicated. OctoML is working to ease the task. 

One factor that makes AI optimization complicated is that there are numerous different approaches to increasing a neural network’s performance. It can be difficult to determine which specific set of optimization methods would be most suitable for a given project. 

The process is made even more difficult by the fact that a company’s choice of hardware also factors into the speed of the neural network. There are many different AI chips to on the market, each with different specifications. Because of the technical differences between chips, neural networks often have to be optimized separately for each processor, which increases the amount of work involved in the task even further.

“The ever-growing ecosystem of ML hardware backends and diverse models are generating an insurmountable amount of manual work to optimize and fine tune models before deployment,” said OctoML Chief Executive Officer Luis Ceze. “This is resulting in skyrocketing costs, significant delays in time to production and impeding new use cases in resource-constrained edge devices. These challenges are threatening the progress of AI by limiting its accessibility; it’s also putting ML on a path that is not environmentally sustainable.”

OctoML’s answer to the challenge is to automate the task with machine learning. The startup’s platform analyzes a neural network and automatically finds the set of optimization techniques that can provide the biggest performance boost. OctoML also helps companies determine which AI chip they should use with their neural networks for the best results.

One of the methods that the startup uses to boost AI performance is known as quantization. The method works by compressing data before it’s ingested by a neural network to reduce the total number of bits that have to be processed. Fewer bits to process means computations can be carried out faster. Quantization also increases performance in a second way: Because the data processed by the AI has been compressed, it can travel faster between the memory and compute modules of the underlying chip.

A technique called operator fusion is also among the methods that OctoML uses to help enterprises run AI software more efficiently. With operator fusion, some of the calculations that a neural network uses to process data are compressed into a single, streamlined calculation.

OctoML’s announcement of its funding round today follows a number of key partnerships. Over the last few weeks, the startup detailed that it’s been working with Arm Ltd., Advanced Micro Devices Inc. and Qualcomm Inc. to provide improved support for their chips. That OctoML’s AI optimization technology caught the attention of three of the biggest names in the semiconductor industry may have been one of the factors that led investors to provide it with another $85 million.

The startup said that it will use the capital to boost its partner ecosystem and expand product development efforts. OctoML has raised $132 million from investors to date. 

Photo: OctoML