CentML Inc., a startup working to improve the performance of artificial intelligence models, today announced that it has closed a $27 million seed round.

The investment was led by Alphabet Inc.’s Gradient Ventures startup fund. Nvidia Corp., Deloitte Ventures, Thomson Reuters Ventures and Radical Ventures contributed as well.

CentML was founded in 2022 by AI researchers from the University of Toronto. The company has built an open-source tool, Hidet, that can increase AI models’ inference performance and lower the cost of running them. CentML also maintains a second open-source project that focuses on speeding up AI training.

Hidet is a compiler, or a tool that allows developers to turn raw source code into a functioning program. It’s specifically optimized to process neural networks’ source code. While assembling an AI model, Hidet makes optimizations that help increase the model’s inference performance.

One way the tool speeds up neural networks is by optimizing their code with a technique called operator fusion. Operators are components of an AI model responsible for, among other tasks, deciding which of the model’s artificial neurons should participate in a given computing task. Hidet can “fuse” multiple operators into a single code component, which improves performance by reducing the need to move data to and from memory.

The tool packages the AI it compiles into so-called CUDA kernels. Those are pieces of code optimized to run on Nvidia Corp.’s graphics processing units. The key feature of a CUDA kernel is that it can be distributed across a large number of GPU cores, which improves performance. 

CentML says its software speeds up AI models without decreasing their accuracy or requiring developers to manually change code. In one internal project, the company made Meta Platform Inc.’s open-source Llama 2 model run three times faster. CentML claims some users have achieved even bigger performance improvements.

Besides accelerating processing times, speeding up an AI model also reduces infrastructure costs. If a neural network’s performance is doubled, it can crunch data at the same speed as before using half as many GPUs. This means companies have to buy fewer chips for their machine learning projects.

Alongside Hidet, CentML offers another open-source tool called DeepView. It allows developers to monitor how an AI model is trained and find ways of speeding up the process. DeepView is designed to optimize neural networks built with PyTorch, a popular AI development framework.

“The proliferation of generative AI is creating a new base of developers, researchers, and scientists seeking to use accelerated computing for a host of capabilities,” said Vinod Grover, director of CUDA and compiler software at Nvidia. “CentML’s work to optimize AI and ML models on GPUs in the most efficient way possible is helping to create a faster, easier experience for these individuals.”

CentML reportedly plans to invest the proceeds from its seed round in product development and research initiatives. The company, which currently has a 30-person workforce, will also hire additional employees to support its growth efforts. 

Image: Unsplash