As enterprises push into next-generation computing, the old rules of cloud no longer apply. In their place comes what has been dubbed the neocloud, a specialized layer of infrastructure built for scale and security, engineered to handle workloads well beyond yesterday’s limits.

Cirrascale CEO and CTO Dave Driggers discusses neocloud economics with theCUBE at theCUBE + NYSE Wired: AI Factories – Data Centers of the Future.

At the core of an ever-demanding infrastructure market is a divide. Training massive artificial intelligence models requires cutting-edge hardware, while inference demands something different: cost efficiency, low latency and the ability to scale resources closer to where people live and work. That divide is what makes neocloud infrastructure appealing for balancing the needs of very different models, according to Dave Driggers (pictured), chief executive officer and chief technology officer of Cirrascale Cloud Services.

“All models are not just created equal,” he said. “You’ve got small models, medium-sized models, large models and gigantic models … The bigger the model, the bigger the hardware that needs to be utilized. The smaller the model, the only way to drive the cost and the performance and scale is through smaller hardware.”

Driggers spoke with theCUBE’s Dave Vellante at theCUBE + NYSE Wired: AI Factories – Data Centers of the Future event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how repurposing hardware from training to inference underpins neocloud economics.

Neocloud strategies for AI infrastructure

Neocloud — specialized, vertically integrated compute and networking built for AI workloads — is carving out a niche by optimizing the lifecycle of graphics processing units. That means rethinking how systems are built for high-intensity training — which are often refreshed every 12 to 18 months as Nvidia Corp. accelerates its AI GPU release schedule — so they can later find a second life powering inference. Even if the GPUs aren’t the most efficient for smaller models, once depreciated they deliver cost advantages that make reuse practical, according to Driggers.

“Our goal and our requirement is we have to build to repurpose that equipment into inferencing,” he said. “It’s the second life; the long tail for that equipment. It may not be ideal for inferencing when it first launches, but once it gets depreciated and in its second life, our main thing we deal with it is repurpose it toward inferencing.”

That model of repurposing doesn’t just serve providers: It defines the limits of what enterprises can realistically take on themselves. Building large-scale, real-time systems internally is costly and inefficient, leaving companies reliant on neocloud providers that can balance the very different demands of training and high-stakes inference, Driggers explained.

“With training, I may have one person running 1,000 GPUs by themselves,” he said. “[If the] network gets cut, nobody even notices it. When it’s inferencing, that network is mission critical; totally different animal.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of theCUBE + NYSE Wired: AI Factories – Data Centers of the Future event:

Photo: SiliconANGLE