The GPU cloud market has so far been shaped by its largest buyers, who can commit to multiyear contracts for scarce accelerators. As artificial intelligence workloads tilt from training toward inference, some neocloud providers are betting the next surge in demand will come from a far broader customer base.

Demand for neocloud capacity is shifting as AI startups prioritize burst capacity, or extra computing power they can tap for short spikes in demand. For Comet Cloud Inc., the opportunity lies with small and midsize businesses, which hyperscalers have largely pushed to the back of the line for GPU allocation, according to Thomas Kato (pictured), founder and chief executive officer of Comet Cloud.

Hyperscalers once rented GPU capacity by the hour, day or week. “You go to [Amazon Web Services] now; they want a multiyear contract from you. They’re picking who they’re taking,” Kato said. “To the small and medium businesses, they’re not even coming up on that scope at all to be able to have that AI. Really that’s our focus.”

Kato spoke with theCUBE Research’s Dave Vellante at the Dell Technologies New York Symposium, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed dedicated AI infrastructure, the shift to inference and the cost of financing GPU clusters. (* Disclosure below.)

Small businesses emerge as the next GPU cloud customer

Comet Cloud’s path to AI infrastructure began in payments. Kato’s credit card processing business led him into Bitcoin mining, which he came to see as an arbitrage of electricity, a cost consideration that also plays a major role in AI computing. The company now offers dedicated clusters rather than shared capacity, a model aimed at customers such as medical offices and law firms that need their data isolated from other customers.

“What we do is we put together clusters and we’re giving clusters to a whole company. All their information, all their security, everything is within that cluster. There’s no one else tied or connected to it,” he said.

For now, Comet Cloud supplies bare metal infrastructure to partners such as Andromeda, a compute marketplace that recently standardized storage across more than 50 GPU providers using Weka NeuralMesh. Comet Cloud plans to move further up the software stack and turn its sites into token factories, according to Kato. Price is a big part of the pitch, particularly for Nvidia Corp.’s B300 systems.

“[Hyperscalers are] going to probably charge you $15, $20 an hour, maybe more. You’re going to come to someone like us and it’s down to $4 an hour. The difference is it’s astronomical,” he said.

Lower prices matter more as inference overtakes training, since Kato expects small businesses to need tokens for everyday tasks such as marketing and customer service. Through its payments merchants and affiliates, including one that works with 50,000 oncology offices, Comet Cloud can deliver third-party AI software to small businesses along with the compute to run it, he noted. Most of those businesses do not yet use AI beyond consumer chatbots, he said.

“As everyone’s developing these [AI tools], it’s going to make them more efficient, more profitable … When that’s developed and we can offer to them, that’s where it’s really going to change everything,” Kato added.

Building that capacity takes power, GPU allocation and capital. Comet Cloud has 300 acres under contract on the West Coast and is beginning a relationship with Dell Technologies Inc., whose AI Factory business has reached an inflection point. Financing presents another major challenge: A single 48-rack cluster of Nvidia’s next-generation Vera Rubin systems can require $1 billion to $1.2 billion in contracted revenue to secure financing, according to Kato. Those economics will decide how far a GPU cloud aimed at small businesses can scale.

“We need the price of tokens to go so low that everybody could afford to use it for their businesses and use those tokens,” he said. “That’s where we should end up.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Dell Technologies New York Symposium:

(* Disclosure: TheCUBE is a paid media partner for the Dell Technologies New York Symposium. Neither Dell, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE