UPDATED 12:36 EDT / FEBRUARY 12 2018

CLOUD

Google’s TPU chips for AI are now available on its public cloud

Google LLC today bolstered its public cloud platform with the addition of Tensor Processing Units, an internally designed chip series specifically built to power artificial intelligence workloads.

A single TPU (pictured) consists of four application-specific integrated circuits paired with 64 gigabytes of “ultrahigh-bandwidth” memory. The combined unit can provide up to 180 teraflops, or a trillion floating-point operations per second, a standard unit of computer performance. Later this year, Google plans to add a clustering option that will let cloud customers combine multiple TPUs into a “pod” with speed in the petaflop range, or 1,000 times as fast as a teraflop.

The company didn’t share any more performance details in today’s announcement. However, a blog post penned by two of Google’s top engineers last year revealed that the pods used internally at the time included 64 TPUs with a combined output of 11.5 petaflops. For comparison, the world’s most powerful supercomputer comes in at 93 petaflops, although it’s worth noting the search giant most likely didn’t use the same benchmarking methods to measure the TPUs’ speed.

Either way, the chips represent a major addition to the Google Cloud Platform. When Google first gave the world a glimpse of the TPU’s specifications last April, it divulged that the chip can run at least certain machine learning workloads 15 to 30 times faster than off-the-shelf silicon. This includes graphics processing units, which lend themselves particularly well to machine learning models. Made mainly by Nvidia Corp. as well as Advanced Micro Devices Inc., they remain the go-to choice for most projects today.

As a result, Google’s cloud customers should gain the ability to train and run their AI software considerably faster. Jeff Dean, leader of the Google Brain team behind machine learning research and development at Google, said on a Twitter thread that a single cloud TPU can be used to bring an implementation of the popular ResNet-50 image classification model up to 75 percent accuracy in 24 hours at a total cost of less than $200.

Google has created several pre-optimized neural network packages that customers may run on TPUs. The lineup includes a version of ResNet-50, as well models for machine translation, language modelling and identifying objects within images. Companies can also create their own AI workloads using Google’s open-source TensorFlow machine learning engine.

Customers that prefer using conventional graphic cards for their AI projects also received a new feature today. Google has added GPU support to its Kubernetes Engine service to allow for machine learning models to be packaged into software containers. The latter technology provides a lightweight abstraction layer that enables developers to roll out updates and move an application across environments more easily.

The new TPUs are available from $6.50 per unit per hour, while GPUs rented through Kubernetes Engine will be billed at Google’s existing rates for each supported chip model.

How ready the chips are for a wide variety of AI tasks, especially for serious computer workloads, remains uncertain. “TPUs are a good place to kick the tires but not necessarily a place to run production workloads,” said Patrick Moorhead, president and principal analyst at Moor Insights & Strategy. “GPUs are the best way to do training. Getting locked into TPUs means getting locked into GCP and TensorFlow.”

What’s more, Moorhead’s colleague Karl Freund wrote in Forbes, Google’s cost and speed claims don’t really hold up. “One does have to marvel a bit at Google’s marketing audacity, considering the plain facts: Google’s 4-chip beast, replete with 64 GB of expensive High Bandwidth Memory, is roughly 33 percent more expensive per unit of performance than Nvidia’s single chip, 1-year old Tesla V100 GPU accelerator,” which he added is up to three times faster.

And Google isn’t alone in pursuing its own AI chips, either. Chip giant Intel Corp. has been touting its latest central processing units for AI workloads, as well as custom chips known as field-programmable gate arrays. And today, The Information reported that Amazon.com Inc. is developing its own AI chip that would help its Echo smart speakers and other hardware that uses its Alexa digital assistant do more processing on the device so it can respond more quickly than it could calling out to the cloud.

With reporting from Robert Hof

Image: Google

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU