Groq Inc. on Monday announced that it has secured a $1.5 billion commitment from Saudi Arabia to expand its delivery of artificial intelligence chips to the country.

The deal comes about six months after the company raised $640 million in funding from Samsung Electronics Co. Ltd., Cisco Investments and other backers. The deal valued Groq at $2.8 billion.

Last year, Groq announced an initiative to build an AI data center in Dammam, Saudi Arabia. The facility is optimized for inference, or the task of running neural networks in production after they’re trained. According to Reuters, the $1.5 billion commitment announced this week will go toward expanding the data center.

The facility is powered by Groq’s flagship LPU, or Language Processing Unit, chips. The company says its processors are 10 times more energy-efficient than graphics processing units. Moreover, Groq claims that the LPU is easier to program, which means deploying AI workloads on the chip takes requires less time and custom code.

Nvidia Corp.’s graphics cards can run not only large language models but also a range of other workloads. Groq’s LPU, in contrast, is optimized specifically for LLMs, which is one of the factors behind its efficiency. When engineers design a chip focused on a narrow use case, they can remove some of the components that ship with more general-purpose processors such as GPUs, which lowers electricity usage. 

Graphics cards break down AI processing tasks into simpler steps. When a chip completes a step, the hardware resources that were used to complete the calculation can be immediately reassigned to the next computation. In practice, however, the process of reassigning hardware resources to workloads is often slowed down by technical hiccups. 

Groq says its LPU streamlines the process. The chip has a mechanism that automatically decides what piece of data a given set of circuits should process, how and where the output should be sent. Groq says that this arrangement enables AI workloads to better utilize its LPUs’ on-chip compute resources. 

Another way the company promises to boost efficiency is by improving the way the chips in an AI cluster exchange data. 

LLMs typically run on not one processor but several. To coordinate their work, those processors regularly exchange data, which is carried out with the help of specialized networking chips. Groq claims its LPU’s design reduces the need for external networking components, which cuts costs and makes AI clusters powered by the chip easier to program. 

The company ships its LPUs with an internally-developed compiler. The compiler turns customers’ AI models into a format that the chips can more easily process. Along the way, it optimizes those models to make better use of the underlying hardware, a task that developers usually have to perform manually. 

Groq sells its chips as part of an appliance called the GroqRack. The system includes eight servers, which in turn each feature eight LPUs. The processors are linked together by an internally developed interconnect dubbed RealScale that promises to remove the need for external switches.

One GroqRack can provide 12 petaflops of performance when processing FP16 data points, which are commonly used by AI models to hold information. One petaflop equals a million billion computing operations per second.

Groq also makes its chips available on a managed basis via a cloud platform called GroqCloud. The company this week updated the platform to let customers to run workloads in its new Dammam data center.

Groq’s Video Feature on theCUBE

Here is an exclusive interview with Groq’s Mark Heaps, vice president of brand and chief technology evangelist.  He addressed the news on camera.

Image: Groq