Enterprise artificial intelligence is entering a cost-conscious phase, and hybrid infrastructure is where that reckoning plays out. As inference spending climbs, business leaders are weighing where models run, which models they use and how much of their data they are willing to hand to someone else.

Those cost considerations are reshaping buying decisions across industries, including among financial institutions trying to turn siloed data into real-time action. More than 6,500 enterprises have adopted the Dell AI Factory with Nvidia updates, according to Dell. A sharp shift in token economics is changing what customers ask for, according to Kyle Dufresne (pictured), senior vice president of sales at Dell.

“It’s incredible to see just over the last three months where we saw 80% of the tokens being generated from frontier models. Three months later, 80% of the tokens in the world are being generated by open models,” Dufresne said. “Why? Because of cost. And everybody talks about tokenomics.”

Dufresne spoke with theCUBE Research’s Dave Vellante at the Dell Technologies New York Symposium, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed tokenomics, hybrid infrastructure and how Dell is helping customers plan AI investments. (* Disclosure below.)

Inference costs push hybrid infrastructure toward the edge

Inference is now growing faster than training, though both are expanding quickly, according to Dufresne. That shift is changing where AI work happens, with Dell seeing inference run wherever data is created — in the cloud, in the data center and out to PCs.

“We’re seeing the business owners saying, ‘Oh my gosh, these costs are getting really high for inferencing. How do we reduce those costs without curbing innovation?'” Dufresne said. “We’re finding it’s happening across that continuum but more and more right out at the far edge.”

The economics also raise a strategic question about control. Companies have to decide whether they want to outsource their data and differentiation, and they are now pricing AI based on where workloads run, which models they use and where their data sits, Dufresne explained. Dell has framed its latest AI Factory with Nvidia updates around helping customers get results on infrastructure they control.

“We’re seeing every single [chief financial officer] in the world that has implemented these structures saying, ‘Gosh, my token costs are exploding,'” he said. “‘How do we drive costs out?'”

Supply is the other side of the ledger. Memory prices are pushing technology costs up rather than down, and about 20% more memory will reach the market next year against roughly 200% more demand, Dufresne estimated. Dell is responding with collaborative planning, steering customers toward more readily available components and sharing advance visibility into upcoming cost increases. Many customers find their cost of capital is lower than component inflation, so they are buying earlier.

“It’s not only just being able to provide the technology,” Dufresne said. “But also to be able to share with customers what’s going to happen in the future so they can make better decisions today on the use of their capital.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Dell Technologies New York Symposium:

(* Disclosure: TheCUBE is a paid media partner for the Dell Technologies New York Symposium. Neither Dell, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE