Google LLC today shared early details about a new iteration of its TPU chip for running artificial intelligence workloads that more than doubles the performance of the previous generation.

Google’s TPU, or Tensor Processor Unit, is an application-specific integrated circuit designed with AI in mind. The new TPU model that the search giant detailed today is the fourth iteration of the chip. The third and second iterations, which have also demonstrated impressive performance, are available to enterprises via the Google Cloud Platform.

Google says the fourth-generation TPU has demonstrated 2.7 times higher average performance than its third-generation predecessor. The search giant compared the chips by measuring how fast they trained five popular AI models during the recent MLPerf industry competition. The fourth-generation TPU produced the strongest results when running the Mask R-CNN model, an image segmentation AI for use cases such as autonomous driving, which it trained 3.7 times faster than Google’s earlier chip.

The performance jump is the fruit of major under-the-hood changes. Google engineer Naveen Kumar detailed in a blog post that the company has significantly increased the TPU’s memory bandwidth, or the speed at which the chip fetches data from memory to process it, and improved its ability to carry out specialized calculcations. 

“Google’s fourth-generation TPU ASIC offers more than double the matrix multiplication TFLOPs of TPU v3,” Kumar detailed in the post. Matrix multiplication is a type of mathematical operation AI models use to process data, while a TFLOP is a trillion floating point operations per second. For perspective, the third-generation TPU v3 against which the new chip was compared can manage 420 trillion operations per second.

Kumar wrote that customers can expect more information on the new TPU “soon.” It’s quite possible the chip will eventually land in Google Cloud given that the two previous-generation TPUs are already available for rent on the platform. However, customers may have to wait a while: ExtremeTech reported that the fourth-generation TPU was placed in the research category during the MLPerf contest, which means it won’t be commercially available for at least six months.

The new chip’s speed enhancements are particularly notable given that the third-generation version it outperforms helped break multiple records during the same contest. Google used 4,090 third-generation TPUs to build what it calls the world’s fastest AI training supercomputer. The system set new records for six out of eight MLPerf benchmarks and trained four of the tested models in under 30 seconds. 

Image: Google