Industry consortia have a mixed track record in computer industry history. At their best, they create standards the market can rally around, thereby simplifying user decision making (think W3C). At their worst, they foster bickering and divisiveness that shoves innovation into a corner.

So we approached the announcement of the Open Data Platform (ODP) two weeks ago with some skepticism. ODP is a consortium of vendors that advocates for a standard version of the core Hadoop stack upon which all vendors can agree. The group is led by Hortonworks, Inc. and Pivotal, Inc., and includes an impressive roster of partners that include Hewlett-Packard Co., IBM, EMC, SAS Institute, Inc. and General Electric Co.

However, ODP notably doesn’t include Cloudera, Inc., the seven-year-old Hadoop pure-play that is unquestionably the current leader in the market. It also lacks support from MapR Technologies, Inc. and several of the large cloud service providers, including Amazon Web Services.

Cloudera’s absence is the most conspicuous, though, and that’s not surprising, since ODP is clearly a ploy by its competitors to shake it from its number-one ranking. But does that make the consortium a bad thing? Not necessarily.

Open source divide

Let’s look at some of the factors underlying the creation of ODP. The market for software platforms has essentially shaken down to two strategies, both based on the open-source model. One is to build proprietary extensions on an open-source foundation (the Cloudera model) and the other is to distribute pure open source platforms and make money on support, training and other ancillary activities (the Hortonworks and Red Hat, Inc. model).

Hortonworks is the lead dog in ODP. It makes no money from selling software, so its success is based upon amassing a large customer base and broad support from other industry players. Its core Hadoop stack is the same one being promoted by ODP, so the support of a broad range of heavyweight software companies and enterprise users is an important endorsement. Hortonworks stands to be the preferred Hadoop supplier for customers doing business with those companies.

Other companies have a more defensive interest in supporting ODP, most notably Pivotal. Although Pivotal executive Leo Spiegel told theCUBE (see video below), “We’ve been all in on open source for a long time,” the reality is that the company has struggled with its open-source database strategy. Pivotal laid off many of the employees working on its open source database offerings last fall. Lacking a strong Hadoop position, it has nothing to lose by adopting a pure open source approach.

Pivotal has had one big open-source success: Cloud Foundry, the open platform as a service framework that has gained broad support and is the foundation for IBM’s Bluemix cloud development platform. Pivotal’s Hadoop strategy is to try to re-create Cloud Foundry’s success with technologies like Greenplum, GemFire and HAWQ, which it has submitted for consideration as core Apache projects. Allying with Hortonworks improves its chances of success.

Many of Hortonworks’ partners in ODP have more defensive motivations. IBM, for example, has nothing to lose by joining the group. Its own Hadoop implementation was not gaining any traction, and supporting ODP is consistent with its longtime commitment to open source, a strategy it used successfully in the 1990s to slow Microsoft’s momentum. HP also has nothing to lose since it has committed to supporting the Hortonworks stack and even invested $50 million in the company last summer. EMC spun off its Hadoop strategy with Pivotal.

Cloudera challenge

In the opposing corner is Cloudera, a company that has no interest in making the Hadoop stack a commodity. Although Cloudera is an active member of the Apache Software Foundation (ASF), it makes money by selling proprietary extensions to Hadoop, such as the Cloudera Manager management suite. Cloudera’s challenge is to stay ahead of the market by providing superior functionality to the open-source equivalents of products like Cloudera Manager. That’s why the OPD’s inclusion of the Ambari open source management framework in its core spec was a deal-killer. Adopting Ambari would essentially be capitulating to Hortonworks.

Cloudera needs to protect its leadership position. It has raised nearly $1 billion in financing and claims to have logged $100 million in revenue in 2014, with the majority coming from software licenses. That number is impossible to verify for a privately held company, but if true it would make Cloudera’s business roughly twice the size of Hortonworks’.

Cloudera argues that the ODP is effectively a competitive governing body to ASF and that there is no need to standardize the mature Hadoop and MapReduce code bases. “The file system has been stable for years. There are no meaningful differences that are causing customers problems,” said Cloudera co-founder Mike Olson in an interview with SiliconANGLE last week. “MapReduce is well-understood and not changing. There is no problem that OPD solves”

Not surprisingly, Hortonworks CEO Rob Bearden disagreed. ASF is a development organization while ODP is principally focused on packaging, he said in an interview with SiliconANGLE last week. Hortonworks is “ten thousand percent” behind ASF as the governing body for Hadoop development. “We fully support and embrace all of the governance and stipulations of the ASF for how things are built in the upstream,” he said. “What ODP does relates to how that technology is productized, packaged and released. That’s not part of the ASF charter.”

The proliferation of Hadoop stacks has been an impediment to the market’s growth, Bearden said, and that creates unnecessary complexity for customers. Agreeing on a common stack “brings consistency, reliability and predictability to how customers make decisions,” Bearden. “Our model is to make the market function, not fracture it.”

Standoff

Cloudera’s and Hortonworks’ business models are at opposite ends of the spectrum. “Pure play open source service businesses are hard to scale,” Olson told us. “Everything about our platform is open source, but we reserve the right to build tooling unique to us, not to lock our customers in but to lock competitors out. I don’t want to drive the market to $0 and let someone else monetize the business.”

Bearden’s rejoinder was pointed: “The fact that Mike Olson can’t run a 100% open source business doesn’t mean we can’t,” he said. Noting that companies like Red Hat have prospered with pure-play business models, he said the market presents plenty of opportunities to find revenue streams without selling software licenses. Asked why some customers appear to be happy to write checks to Cloudera, he commented wryly, “Part of the world still drives Pontiacs.”

Wikibon believes that the ODP will sharply redefine the competitive choices in the Hadoop market. Customers now have a clear choice between a small number of distributions, and the competitive field has boiled down to Hortonworks vs. Cloudera. MapR will do fine with its niche strategy, but it is not calling the shots. Although Hortonworks’ Bearden asserted in our interview that open source is a superior platform for innovation, the fact is that Cloudera’s customers pay it for functionality they can’t find elsewhere. The company’s success is based upon continuing to stay ahead of its open-source competition.

For customers, the market is boiling down to two clear choices: Cloudera and MapR for best-of-breed tools or an ODP-compliant distribution for maximum choice and compatibility. If ODP can keep its members in line with the mission and avoid forking the core Hadoop platform, it will be a powerful force to contend with. Hortonworks and its partners must stick to the organization’s charter, put competitive bickering behind them and focus on the needs of the customer. At the same time, the pressure will be on Cloudera to stay far enough ahead of the open source curve to justify its premium pricing. An industry schism isn’t bad if it forces every to hew more closely to the needs of the customer.

Wikibon’s Jeff Kelly (@jeffreyfkelly) and Stu Miniman (@Stu) discuss the future of open source, particularly in the world of big data, and the impact of the formation of the Open Data Platform in a CUBE Conversation (19:11)