Artificial intelligence solutions company Groq Inc. — not to be confused with X’s new chatbot, Grok — is the creator of the first Language Process Unit, the company says.

With its LPU Inference Engine, Groq can deliver the world’s fastest generative AI, and the fastest way to accurately run a large language model, according to the company’s CEO, Jonathan Ross (pictured). It believes in an AI economy powered by human agency and envisions a world where AI is accessible to all.

“The best way to think about [LPU] is all of these other architectures you’re hearing about are really good at parallel compute,” Ross said. “Some of them are really good at sequential compute. The thing is, you can’t produce the hundredth word until you’ve produced the 99th. Very much like a game of go or chess, language is the same thing. It’s just a larger space. So, we are very good at sequential problems.”

Ross spoke with theCUBE industry analysts John Furrier and Savannah Peterson at SC23, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed LPU and what it means for gen AI, what makes Groq different from other AI chip vendors and how LPUs will increase energy efficiency in AI usage. (* Disclosure below.)

Preparing for an automated future

Rather than building a chip, Groq started off by developing software to build a system. With this system, rather than having only a few chips working, several chips are running, with each chip doing its own small part and then handing off the task to the next chip.

“We’re actually 10 times faster, 10 times cheaper, 10 times lower power,” Ross said. “Because we start with a compiler, we can often get software working 20 times faster. If I hadn’t just shown you how fast it was, you wouldn’t believe me. But that’s real, and we’re ramping up our production.”

The conversation continued with the group discussing what the vision of the future was regarding what kind of applications would be developed, with democratization assuming enablement across developers. While we have the hammer, we haven’t yet invented the nail, according to Ross.

“There are going to be a bunch of things that are possible that we haven’t envisioned yet, because we haven’t imagined what you could do if you had a large language model available 24/7, not just limited to speaking to one person or small group at a time, but could speak with a large number,” he said. “The applications haven’t been envisioned yet. They’re not even in science fiction yet.”

The group ended the discussion with some brainstorming about how companies, developers and even entrepreneurs of the creative class should configure their data, especially as their current configuration will eventually be old and antiquated compared to emerging architecture. It’s not big data but big compute that makes AI such a behemoth in tech, Ross explained.

“When training Llama 2 70b, the number of GPUs that you’d need for that would be about $3 to $5 million worth. But if I was to transfer all the data it was trained on, on my terrible cellular plan, that’d be about $40,000,” Ross said. “We’re now starting to talk about and working with people to build compute centers rather than data centers. The difference is you don’t need a bunch of them all around; you can have a couple of them.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of SC23:

(* Disclosure: TheCUBE is a paid media partner for SC23. Neither Dell Technologies Inc., the main sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE