Gartner’s recent estimate that Big Data is a potential $200 billion business opportunity misses the mark by orders of magnitude said Tresata Founder and CEO Abhi Mehta
in the Cube at the Strata + Hadoop World 2012 Conference in late October. But, he says, the opportunity is primarily for businesses that can use Big Data analysis to revolutionize their organizations and operations.

“We’ve both been saying this for two years now. And then Gartner comes out finally after two years and says well it’s a $200 Billion opportunity. Well, it’s bigger than that.

“We were talking to the head of credit cards at one of the biggest global banks. He looks at our solution for underwriting, and he goes, ‘If we do this well, the payoff is not in the millions. It’s not in the billions. It’s in the trillions. Because we can redo the banking industry off of it.’” No one, he says, is writing that story.

Unlike many past IT-driven opportunities, however, the major value is not in IT investments but rather in the ways in which businesses can leverage Big Data analysis to revolutionize their business and operational models. “Big data is not a technology opportunity,” he said to Wikibon Co-founder and Chief Analyst David Vellante and SiliconAngle Founder and CEO John Furrier. “We’ve been saying that together for two years now.” In fact, he predicts that the bottom will drop out of the base hardware and operating system-level markets.

In part this is because much of the basic technology, such as Hadoop, MapReduce, and HTFS [High Throughput FIle System], is Open Source. While companies will look to companies like HortonWorks for expert support to ensure that the Big Data systems on which they will increasingly depend run smoothly and they may well use Big-Data-as-a-Service companies to do specific kinds of analysis, the basic software licenses are free. The IT market will move up the stack to the services and applications level.

And in part this is because the software stack needed to support Big Data business analysis will be much simpler than the traditional RDBMS software stack, leaving fewer components to sell. Much of that traditional stack was designed to get valid results given the limitations of the technologies and data sets available. Big Data, for instance, eliminates sampling entirely. Instead of creating a statistically valid sample set from a large populations — say the entire consumer population of the United States — Hadoop and Map Reduce allow researchers to work with the entire population. Instead of relying on data from surveys, which are by their nature limited in part by the subsets of any population willing to take the survey in the first place, they can look at data on the online actions and social media postings and connections of virtually the entire population.

In traditional sampling-based research, statistical modeling plays a central role. In Big Data research it may not have any role whatsoever. “If I’m to assign intent [that is what consumers, for instance, intend to do in the future] that model has not been written,” Mehta says. “I think the fundamental principle that bottom-up analysis always wins always holds true…. The big question we ask ourselves is this: If I were to build the next generation data analytics company, what am I building it on? If it’s HTFS, which I think is the right answer….then do you need layers on top of HTFS to make a solution like mine more useful? The answer, Dave, is, I don’t. What this means for the traditional software suppliers I don’t know.

“The big realization companies are having is software is free. Should you pay for HDFS? No, you shouldn’t. It’s the iOS of data. It should be free.” Companies will buy applications to run over that base software, although as in the smartphone market some of those will also be free. “Enterprise IT will go the way of iOS and the iPhone. You’re going to buy incredibly cheap hardware for almost nothing, like you buy an iPhone And every two years you get an upgrade for almost no cost. I call it the iPhone-ization of IT. ”