Luminal AI Inc. has outlined a plan to tear down Nvidia Corp.’s formidable software moat in the artificial intelligence chip industry after raising $23 million in early-stage funding.

Today’s Series A round was led by Standard Capital and saw participation from existing investors such as Y Combinator, SagaVC, Liquid 2 and Felicis, bringing Luminal’s total amount raised to date to more than $28 million. It comes about a year after the startup raised $5.3 million in seed funding.

Everybody knows about Nvidia these days. The chipmaker has ridden the generative AI wave to a staggering $5 trillion market valuation, surpassing rivals such as Apple Inc., Alphabet Inc. and Amazon.com Inc. to become the most valuable publicly traded company in the world. But anyone who assumes that it’s solely Nvidia’s graphics processing units that set it apart would be mistaken, for the real reason for its dominance of the AI computing industry is not just silicon, but software.

For almost two decades, Nvidia’s proprietary Compute Unified Device Architecture or CUDA software framework has been the real differentiator, serving as a virtually impenetrable barrier that prevents rivals such as Advanced Micro Devices Inc., Google LLC and Amazon Web Services Inc. from stripping away its market lead. Though those rivals have developed some very powerful GPUs and custom application-specific integrated circuits for running AI workloads at lower costs than Nvidia’s hardware, they haven’t been able to match the scale of Nvidia’s software ecosystem.

Virtually every major AI framework, including TensorFlow and PyTorch, has been built natively on Nvidia’s CUDA software, enabling developers to run their AI models on Nvidia’s GPUs without any modifications. But to run their models on alternative chips, developers need to rewrite much of the underlying code, which is challenging and time-consuming work. That’s why the vast majority of AI workloads still run on Nvidia’s chips.

Many developers don’t want to be paying the exorbitant fees needed to rent Nvidia’s GPUs or purchase them outright, but they also don’t want to spend months at a time fine-tuning their codebases to get their AI models to run optimally on alternative hardware, and it’s this challenge that Luminal intends to solve for them.

The startup’s solution is a two-pronged architecture platform that consists of its proprietary search compiler and a fully managed inference cloud that combine to eliminate the hassles of bringing AI workloads onto different chips.

Co-founder and Chief Executive Joe Fioti told SiliconANGLE that the compiler works by systematically exploring different ways of running neural networks on the target chip in order to come up with the most optimal path. It benchmarks each variation until it finds the best possible solution, and then it automatically generates the code developers need.

Once the code is ready, it caches the implementation so it can be reused across recurring inference workloads. In this way, it allows developers to run their AI applications across heterogeneous chip architectures with minimal fuss, Fioti said.

“We started out building in the open on GitHub as AI adoption took off, and quickly saw how much demand there was for a more efficient way to run AI models across different hardware,” Fioti said of the company’s origins.

Luminal’s compiler and inference cloud are designed primarily for running high-throughput, continuous-uptime AI applications that demand massively scalable inference. Think applications such as AI coding tools, automated image generation engines, AI video generators and real-time voice agents.

“The cost of running AI shouldn’t limit who gets to build it,” Fioti said. “We’re building the infrastructure to make high-performance inference more accessible, giving developers more choice beyond Nvidia’s ecosystem without forcing them to compromise on speed or cost.”

Luminal launched its compiler and public cloud-based inference infrastructure just four months ago and adoption is growing rapidly. It’s already processing more than 45 trillion tokens per month – a number that’s roughly equivalent to 10% of the total tokens consumed so far by Together AI Inc., an $8 billion-valued infrastructure unicorn that has been serving the market for more than four years already. The startup has established a strong partnership with AWS, and runs a significant amount of its workloads on that company’s Tranium processors.

Going forward, Luminal’s plans include expanding its engineering team, scaling its go-to-market operations and optimizing its compiler and inference infrastructure to support more chip hardware and AI models.

Standard Capital’s Dalton Caldwell said he’s betting on Luminal because AI inference is the fastest-growing market in the world. “We backed it because of its deep technical expertise and its demonstrated proof that it can squeeze fantastic performance out of heterogeneous hardware,” he said. “Luminal’s ability to get the most out of GPUs that have already been purchased … is incredibly high leverage.”

Though Luminal has had a promising start, it’s certainly not the only company trying to break Nvidia’s CUDA stranglehold on the AI industry. Both Google and AWS have their own things going on, notably launching alternative AI frameworks to PyTorch that make it easier for developers to create AI models and applications that can run on their native processors.

Luminal also faces competition from startups such as Spectral Compute Ltd., which offers a programming toolkit and compiler that enables CUDA-based code to run natively on any AI-accelerated chip. There are also open-source efforts such as the UXL Foundation’s OneAPI, which converts CUDA code into software that can easily run on any silicon.

Image: SiliconANGLE/Dreamina