Spotify to abandon own hosting infrastructure and move to Google Cloud


In a coup for Google, Inc., music streaming firm Spotify AB has announced it’s moving its service to the Google Cloud.

Founded in 2006, Spotify has until now hosted its service by itself in data centers in its native Stockholm, as well as the United Kingdom and the United States, but has decided that the cloud has matured enough to be able to provide the same service at a lower cost.

Operating our own data-centers may be a pain, but the core cloud services were [previously] not at a level of quality, performance and cost that would make cloud a significantly better option for Spotify in the long run.” Spotify Vice President of Engineering and Infrastructure Nicholas Harteau explained in a blog post.

“Recently that balance has shifted. The storage, compute and network services available from cloud providers are as high quality, high performance and low cost as what the traditional approach provides.”

Data stack advantage

Why Google and not a competing service such as Amazon Web Services or Microsoft’s Azure was explained as coming down to the tools Google provides, or its data stack advantage if you like.

“What really tipped the scales towards Google for us, however, has been our experience with Google’s data platform and tools,” Harteau said. “Good infrastructure isn’t just about keeping things up and running, it’s about making all of our teams more efficient and more effective, and Google’s data stack does that for us in spades.”

According to a post on the matter, from the Google Cloud Platform Blog, Spotify is now implementing Google Cloud Datastore and Google Cloud Bigtable, as well as also deploying Google’s Cloud Networking services, such as Direct Peering, Cloud VPN, and Cloud Router, to transfer petabytes of data.

On the data side, Spotify is abandoning Hadoop, MapReduce, Hive and a series of home-grown dashboarding tools, in favor of Google Cloud Pub/Sub, Google Cloud Dataflow, Google BigQuery, and Google Cloud Dataproc.

From traditional batch processing with Dataproc, to rock-solid event delivery with Pub/Sub to the nearly magical abilities of BigQuery, building on Google’s data infrastructure provides us with a significant advantage where it matters the most,” Harteau added.

The move itself will take some time, with Spotify hosting over 1 billion playlists from 75 million users, along with a music catalog of over 30 million songs.

Image credit: mager/Flickr/CC by 2.0