Google Cloud today announced the imminent launch of its most powerful and energy-efficient tensor processing unit to date, the Trillium TPU.

Google’s TPUs are similar to Nvidia Corp.’s graphics processing units, which have emerged as the most popular hardware for running generative artificial intelligence workloads. With Trillium, Google is hoping to provide companies with an alternative to Nvidia’s GPUs.

Unveiled today at Google I/O 2024, Trillium is the company’s 6th generation TPU. It’s designed to power the most demanding generative AI models and workloads, providing significantly greater compute, memory and networking capabilities than its existing TPUs.

Amin Vadhat, vice president and general manager of ML, Systems & Cloud AI at Google Cloud, said the Trillium TPUs deliver a 4.7-times increase in peak compute performance compared to the TPU v5e, which is its current most powerful chip, announced at Google Cloud Next last August. To achieve this higher performance, Google put in a lot of effort to expand the size of the matrix multiply units or MXUs that perform computations, and also increased its overall clock speed.

In addition, the Trillium GPUs feature twice as much High Bandwidth Memory capacity and bandwidth, while the Interchip Interconnect bandwidth has also been doubled. According to Amin, these enhancements will deliver significant improvements in training time and serving latency for large language models.

Another new innovation is third-generation SparseCore, which is a specialist accelerator for processing the ultra-large embeddings found in advanced, AI-based ranking and recommendation systems. With this, the Trillium TPUs will be able to train the next generation of foundational AI models and serve them with much lower latencies and costs. At the same time, they deliver energy-efficiency gains of up to 67%, meaning lower running costs and a smaller carbon footprint than the TPU v5e, Amin said.

Google said Trillium will launch later this year with the ability to scale to more than 256 TPUs in a single high-bandwidth, low-latency pod. And by leveraging Google’s advances in pod-level scalability, multislice technology and Titanium Intelligence Processing Units, users will be able to link hundreds of individual pods of Trillium TPUs to build multi-petabyte scale supercomputers and data center networks.

Powering the next generation of AI

Some of Google Cloud’s customers have already made plans to put the Trillium TPUs to use. For instance, Google DeepMind will use the new chips to train and serve future generations of Google’s Gemini models, and do it more quickly, efficiently and with lower latency than before. Moreover, a company called Deep Genomics Inc. is planning to use the Trillium TPUs to advance drug discovery with AI.

“Our mission is to create and apply advanced AI models to push the boundaries of what’s possible in the field of RNA therapeutics,” said Deep Genomics founder and Chief Information Officer Brendan Frey, who expects the Trillium TPUs to improve training and running of its AI foundation model 100-fold. “Advanced AI models to push the boundaries of what’s possible in the field of RNA therapeutics.”

The Trillium TPUs will also be integrated with Google Cloud’s AI Hypercomputer project, which is a supercomputer platform that’s designed to run cutting-edge AI workloads. The AI Hypercomputer integrates the most advanced TPUs and GPUs with open-source software frameworks and a flexible consumption model to support the development of extremely powerful new AI workloads.

When Trillium launches, it will be added to AI Hypercomputer as another hardware option alongside Google’s TPU v5p and Nvidia’s H100 GPUs. The company is also planning to integrate Nvidia’s new Blackwell GPUs with the AI Hypercomputer, but hasn’t said when this will happen, so it’s not clear if the Trillium TPUs will be made available first.

Image: SiliconANGLE/Microsoft Designer