UPDATED 13:10 EDT / FEBRUARY 24 2015

Despite hiccups, Big Data has a big future says Wikibon’s Kelly

Big_Data_SVBig Data enables an inflection point in business, and it presents a $1 trillion-plus opportunity for vendors and practitioners, says Wikibon Principal Research Contributor Jeff Kelly.  In his keynote to theCUBE’s BigDataSV evening program last week in parallel with HadoopWorld, Kelly addressed six questions that are often raised by the Wikibon community when discussing Big Data and also presented an overall optimistic picture of the future.

Where does the Big Data market stand and what are enterprises adopting this technology doing with it?

Kelly said adoption is still immature, partly because practitioners have to put together many piece parts of unfamiliar technologies to create a working system. Even large enterprises often do not have the requisite skill sets. Furthermore, they do not understand what to do with the technology to maximize return. The near-universal initial application in Fortune 1000 enterprises is to archive large amounts of data from more expensive platforms such as Oracle production databases. The result, he said, is that a Wikibon survey of practitioners two years ago found that the average return on investment was 55 cents on the dollar. Today, little has changed, and the ROI, if anything, has shrunk.

Big Data has a much larger potential for value creation by helping companies to understand their customers better, identify business opportunities, and create efficiencies in core business activities where a savings of two to five percent can total millions of dollars. But what enterprises need is a unified Big Data technology platform that is enterprise-ready and that supports business applications.

What can we learn from previous data-oriented markets like business intelligence (BI) and data warehousing (DW)?

First, Kelly said, the history of these markets shows us that consolidation happens. In BI, for example, Business Objects was bought by SAP SE and Cognos by IBM. That consolidation has already started in the Big Data market, with Teradata Corp. buying Revalytics, Hadapt and others.

However, this may be a good sign for the technology, Kelly said. Big vendors like Teradata, IBM, and Hewlett-Packard Co. know how to harden technologies to meet enterprise needs and build software platforms from the piece-parts. They also have the marketing and sales contacts with enterprise C-suites to sell those platforms, which are assets startups usually lack.

What about open source generally, and the Open Data Platform announcement in particular?

Hortonworks to go public in early 2015, predicts analyst | #BigDataNYC

Jeffery Kelly – Principal Research Analyst at Wikibon

One major concern Wikibon members have raised is that once big vendors enter a new market innovation tends to slow down. Kelly (left) thinks the Big Data market will avoid this problem in part because it is driven by the open source community, and in part because the big born-in-the-cloud practitioners who created the Big Data technologies, such as Yahoo!, Inc., Netflix, Inc., Google, Amazon, Facebook and Twitter, continue to drive innovation. They are constantly developing Big Data tools that they donate to the open source community, where those tools become the basis for new startups. Some of those tools will eventually be absorbed into the Big Data platforms the major vendors are beginning to build, continuing to advance the capabilities they provide to their users.

Kelly sees the Open Data Platform as an initial attempt by the vendors, in this case including Pivotal, IBM, Hortonworks, Inc., SAS Institute, Inc. and practitioners including General Electric Corp. and Verizon Communications, Inc., as a first attempt at developing a Big Data foundation for enterprises. In general, Kelly said, the early expectations that this technology would disrupt the big traditional vendors was mistaken. “The big vendors will make a lot of money on Big Data, but that is good,” he said.

Where’s all that startup money going?

“Crazy amounts of money have been raised by the Big Data startups,” Kelly said. The three core Hadoop vendors – MapR Technologies, Inc., Hortonworks and Cloudera, Inc. – alone have raised more than $1.6 billion in venture capital. Adding in the NoSQL vendors raises that above $2 billion. Add all the other startups at HadoopWorld and the total probably exceeds $3 billion.

The money is needed in part because building the market takes time and work. This is separate from technology innovation. Building an enterprise-grade platform, for example, requires establishing standards and developing an understanding of enterprise needs, “which may not be the sweet spot for startups.” Building sales and distribution channels to reach those customers also takes investment. The startups have to compete with major vendors like IBM, EMC and Hewlett-Packard, who already have those channels and are being aggressive about using them. Startups have challenges gaining mindshare against that kind of competition.

What does the spate of consolidation mean?

Startups have a challenge building long-term sustainable businesses when the big vendors are playing such a large role so early in the market. The question is whether any independents will be left in five or 10 years. Kelly believes one or two can survive, with Cloudera and Hortonworks best positioned. They work with Hadoop itself, which is the foundation of the market, and are positioned to build the kind of platform that most companies need. The startups that focus on a particular tool or other segment of the stack will probably not survive as independent companies. Those that aren’t acquired will disappear. But new startups will appear to develop new tools often based on new contributions to the open source community from the big cloud practitioners.

Is this the beginning of the end or the bubble, or is there something next that matters?

Kelly believes this is only the beginning. Even as the technologies mature, companies face their own maturation process dealing with non-technology challenges such as compliance, ethics, internal political issue and developing a more sophisticated understanding of how to get the most from their Big Data investment. While the initial use cases have focused on saving money inside IT, the real value will be in identifying new business opportunities and savings in major business processes and expenses such as fuel costs for airlines and large freight companies. Internet-of-Things (IoT) data is an exciting area with major possibilities, for instance.

Vendors will do well in the Big Data market over the next decade, Kelly predicts, but the real winners will be the companies that harness the technology creatively. He estimates that practitioners will create $1.2 trillion in new value from Big Data over the coming decade.


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU