Stalwart enterprise data warehouse vendor Teradata is partnering with Hadoop start-up Hortonworks to improve integration between the two technologies and, as Teradata puts it, to reduce the “confusion” created by the nascent but growing ecosystem of Big Data vendors.

The partnership will include joint technology development and services offerings to help customers create highly optimized and integrated Teradata-Hadoop deployments, understand when to use Hadoop versus when to use Teradata, and how to identify areas where the two technologies should be used in tandem.

Hadoop or a Data Warehouse? Why not both?

By “confusion,” I believe Teradata is referring to those vendors and market watchers that believe Hadoop will eventually supplant traditional data warehousing technology all together, becoming a single platform for processing, storing and analyzing all manner of data, structured and unstructured, in batch and in real-time. Start-ups like Hadapt and Platfora espouse this view.

The more popular view, and the one being taken here by Teradata and Hortonworks, is that SQL-centric data warehousing and MapReduce-style approaches like Hadoop are complimentary. Specifically, this view holds that Hadoop is an ideal place, due to its inexpensive but powerful processing and storage capabilities, to apply structure to large volumes of unstructured data which can then be fed into the data warehouses and other upstream systems, integrated with structured corporate data, then mined and analyzed for new insights.

The move is an interesting one from Teradata’s perspective considering its acquisition of Aster Data last year. Aster Data applies a unique blend of MapReduce in a SQL wrapper to support the development of large-scale parallel applications. At Hadoop World, Teradata Aster Co-President Tasso Argyros positioned Aster as a less expensive, easier-to-deploy-and-manage alternative to Hadoop, though he acknowledged the Aster database doesn’t scale to Hadoop levels. (See video below).

It appears Teradata has come to the conclusion that Hadoop does indeed have a role to play in Big Data environments, along with its market leading enterprise data warehouse and Aster’s high-speed, scalable analytic database. Other analytic database vendors, including HP Vertica and EMC Greenplum, have come to the same conclusion and developed Hadoop connectors (and in EMC’s case a full-fledged Greenplum-Hadoop appliance) to move data back-and-forth between systems.

The partnership makes obvious sense for Hortonworks, which will benefit from Teradata’s marketing operation to get itself in front many more potential customers (not unlike Cloudera’s partnership with Oracle on this front.) As my colleague Klint Finley reported, Hortonworks underwent a quiet management shuffle last week but, according to founder and now CTO, the company is still committed to keeping its Hadoop distribution 100% open source and making money by providing paid-for training and technical support.

ServicesANGLE

Whether Hadoop, and Hive in particular, eventually matures to the point that it becomes practically capable of real-time querying and other functions currently the domain of data warehousing is an open question. But for the foreseeable future, Wikibon believes that Hadoop and data warehousing (both “traditional” data warehousing and Next Generation Data Warehousing – see here for the distinction) are indeed complimentary Big Data approaches and that enterprises should engage service providers and the vendor community to identify beneficial co-deployment use cases.