Microsoft Windows Azure, Ubuntu and the Big Data Connection

Much has been made already of the new services and features that are coming to the Microsoft Windows Azure cloud. But while everybody (myself included) talks about the new infrastructure-as-a-service and Azure Web Sites components of the enhanced Azure, there’s something interesting going on with Microsoft, Canonical, and the rising market for big data applications in the cloud.

In a blog post, Gianugo Rabellino, Microsoft’s chief open source evangelist, writes about some of the partnerships that were only covered in passing during yesterday’s Meet Windows Azure keynote.

These partnerships, including CleardbCloudant and Lucid Imagination, bring MySQL, CouchDB and Apache Solr services, respectively, to the Azure platform, enabling large-scale database services for cloud applications “without the worry of provisioning and maintaining their databases.” CouchDB and Solr will also be available for customers to run themselves with Azure-optimized installation packages. And finally, 10gen is finally bringing the long-promised improved MongoDB installation experience to Azure

And, of course, Azure already has the just-renamed Windows Azure SQL Database, as well as Apache Hadoop support.

But the big data connection goes deeper than just these partnerships. For a look at what could be the real future of big data on Windows Azure, step back and take a look at the new IaaS layer. Now, SUSE, CentOS and Ubuntu machine images are already available for Windows Azure, but only the last of those – and its primary developer, Canonical – are giving lip service to the big data and cloud future.

In his own blog entry, Canonical CEO Mark Shuttleworth notes that the company has started to use Ubuntu’s Juju service deployment/management tool to spin up 2,000-node Hadoop clusters in Amazon EC2. With Azure, Shuttleworth says that Canonical finds that it’s “very valuable to have a new enterprise-oriented cloud with which to evaluate performance, latency, reliability, scalability and many other key metrics for production deployment scenarios.”

Shuttleworth goes on to explain his view that Microsoft (wisely) wants Azure to be more than just a Windows IT environment extension, and that Ubuntu on Windows Azure is a growth opportunity for both platforms.

From where I’m standing, it seems like those managed application services, plus Azure’s rapidly-maturing Hadoop support, plus the ability to deploy Hadoop as needed on Linux VMs, plus all those relational and non-relational database partners seem to make Azure the Swiss Army Knife of big data cloud application development.

I’m not certain there’s another public cloud service provider who has all these pieces in play at this point – but as the big data market explodes, I certainly expect that to change sooner rather than later.