theCUBE Live At EMC World 2014

Enterprise storage juggernaut EMC Corp. is rolling out support for Cloudera Inc.’s market-leading Hadoop distribution to its Isilon line of scale-out arrays in a bid to capture a bigger slice of the unstructured data pie, which is expected to account for 79.2 percent of all capacity shipped by 2017. The move is equal part necessary and ambitious.

It’s necessary because Hadoop is fast becoming CIOs’ platform of choice for digesting their vast troves of uninstructed information, with complementary open-source projects such as the recently graduated Apache Storm now extending the framework to other fast-rising use cases like stream processing. It’s ambitious because EMC is gambling control over a vital part of its ecosystem for a chance at achieving solid footing in what is, at the end of the day, still an emerging market.

Up until now, the Isilon series has only been compatible with Pivotal HD, a homegrown Hadoop distro that it spun off into Pivotal Software Inc. with the creation of the subsidiary last year in addition to a number of other non-core assets. Support for the array family is one of the main advantages that the distro offered over more popular alternatives such as Cloudera’s CDH, an edge that has all but been blunted with today’s move. But the gambit holds the potential to more than make up for that setback.

The direct compatibility between Isilon and HDFS – the Hadoop file system – enables users to make data stored on their appliances available for analysis in the batch processing platform without a manual migration, which consumes times and resources better spent elsewhere. That’s an undeniably positive change from the customer point of view, but for EMC, it introduces the prospect of organizations moving workloads off Isilon and onto competing offerings from the likes of NetApp Inc. and the soon-to-be-split Hewlett-Packard Co., which both already support Cloudera’s distro.

However, that risk is easily offset by the benefit of enabling customers to make use of existing hardware investments in their Hadoop implementations. This not only allows CIOs to save money on infrastructure but also take advantage of the advanced data services available in Isilon, which Hadoop doesn’t come near to matching at this point in time. That functionality makes a compelling point for CIOs to scale their analytics clusters on top of the array family, which – taking into account that the amount of uninstructed data is set to increase by a factor of ten through 2020 – opens up a massive growth opportunity for EMC.

That Isilon can help make Hadoop more viable for mission-critical workloads is an equally big plus for Cloudera, with the integration putting it in a position to target more use cases. Plus, the company can now boast of being the only distributor to provide support for the array family, although that will probably change now that EMC is trying to expand its presence in the Hadoop ecosystem. There’s a also a good chance that the storage giant will extend its commitment to the framework beyond Isilon to other parts of its portfolio, potentially even matching NetApp with a dedicated converged infrastructure offering for the platform. Either way, it’s clear that EMC’s Hadoop journey is only just beginning.