The rise of artificial intelligence has taken the world by storm, with new developments like ChatGPT being the talk of the town. A major stumbling block arises, however, because AI is incredibly computational intensive.

Anyscale Inc. is tackling this challenge using an open-source framework called Ray, which speeds up the experimentation cycle and enhances developer productivity, according to Robert Nishihara (pictured), co-founder and chief executive officer of Anyscale.

“To actually succeed with AI, you’re typically not just running it on your laptop; you’re often running it and scaling it across thousands of machines,” he said. “One of the goals for Anyscale is really to make that easy … we’re building Ray, which is an open-source project. Companies like OpenAI use Ray to train their models, like ChatGPT. Companies like Uber run all their deep learning, companies like Shopify, Spotify, Netflix, Cruise, Lyft, Instacart, you know, ByteDance; a lot of these companies are investing heavily in Ray for their machine learning infrastructure.”

Nishihara spoke with theCUBE industry analyst John Furrier for a CUBE Conversation ahead of the AWS Startup Showcase: “AI/ML: Top Startups Building Foundation Model Infrastructure” event. They discussed how Anyscale and Ray are simplifying AI workloads. (* Disclosure below.)

How Ray aims to make developers’ lives easier

Since one of the main value propositions of Ray is to move fast, it enhances developer productivity, according to Nishihara, simplifying the process by taking the infrastructure work off a developer’s critical path.

“I think there are really two main things when people choose to go with Ray and Anyscale,” Nishihara stated. “One reason is about moving faster; it’s about developer productivity. I think probably the phrase I hear the most is companies saying that they don’t want their machine learning people to have to spend all their time configuring infrastructure. All this is about productivity.”

By offering developers the infrastructure needed to create applications, Anyscale enhances the success rate in AI, according to Nishihara, who said that this benefits many industries because AI is transformative.

“I think there’s the potential here to really solve an important problem, to get to the point where developers don’t need to think about infrastructure, don’t need to think about distributed systems,” he stated. “All they think about is their application logic and what they want their application to do.”

A stepping stone toward foundational models

As the foundational model trend continues to gain steam in generative AI, Ray is a perfect fit because it renders a common infrastructure when it comes to training, fine-tuning, serving, data ingestion, pre-processing, and hyper parameter tuning, according to Nishihara.

“Well, foundational models are hugely important for the industry broadly,” he pointed out. “Now, Ray fits in and Anyscale fits in a number of places. First of all, they’re useful for creating these foundation models. Companies like OpenAI use Ray for this purpose. Companies like Cohere use Ray for these purposes. IBM … there’s of course also open-source versions like GPT-J created using Ray.”

Given that companies are looking for managed infrastructure, Anyscale supports different workloads, according to Nishihara, who said that they include batch inference-related workloads and those requiring natural language processing and computer vision.

“They’re three main workloads that companies run on Anyscale, run with Ray,” he noted. “It’s training-related workloads, serving and deployment-related workloads, and batch processing. And it’s across many different industries. We work with companies doing drug discovery, doing gaming or e-commerce, doing robotics or agriculture.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s pre-event coverage of the AWS Startup Showcase: “AI/ML: Top Startups Building Foundation Model Infrastructure” event:

(* Disclosure: Anyscale Inc. sponsored this segment of theCUBE. Neither Anyscale nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE