In a report now available, IDC sees the big data and services market growing to $16.9 billion by 2o15. That’s up from $3.2 billion in 2010, a 39% compound annual growth rate (CAGR).

IDC does not provide its reports to journalists for review so what it actually covers is fairly sketchy.

IDC did post its methodology for the study, which it outlines as follows:

• Deployments where the data collected is over 100 terabytes (TB). IDC is using data collected, not stored, to account for the use of in-memory technology where data may not be stored on a disk.
• Deployments of ultra-high-speed messaging technology for real-time, streaming data capture and monitoring. This scenario represents Big Data in motion as opposed to Big Data at rest.
• Deployments where the data sets may not be very large today, but are growing very rapidly at a rate of 60% or more annually.

IDC Analyst Ben Woo said in an interview today that the market is still in its early stages. Most projects are minimal in scope but there is real proof that adoption is growing. Initial revenues are small but that’s more a factor of most projects being open source.

In a press announcement,  Woo said infrastructure investments are growing fastest at a 44% CAGR. Storage shows the strongest growth opportunity, growing at 61.4% CAGR through 2015.

Woo said about 250 vendors play in the big data market. It is early to determine  market share. It’s too fractured at this point.

I asked Woo about Hadoop and its role in the big data landscape.  He was not dismissive but clear in making his point that Hadoop is just one approach.

He said Hadoop has garnered a lot of attention due to its origins and original adoption in the Web community.

Woo would not comment about how the IDC report compares to Wikibon’s market estimate for big data. Wikibon Analyst Jeff Kelly wrote in his report that the market is $5 billion and by 2017 will be $53.4 billion, representing a 58% CAGR.

Kelly also broke down the vendor space:

Of the current market, big data pure-play vendors account for $310 million in revenue. Despite their relatively small percentage of current overall revenue (approximately 5%), these vendors – such as Vertica, Splunk and Cloudera — are responsible for the vast majority of new innovations and modern approaches to data management and analytics that have emerged over the last several years and made big data the hottest sector in IT.

It’s clear that IDC sees the market as a nascent one. But that is hardly surprising. But a young, fast growing market also means an exciting one for all the players. Testament to that is Woo’s view that there should be considerable mergers and acquisition activity in the next year. This is not necessarily because technology vendors have gaps in their offerings. It’s due to the wide spectrum of opportunities that affect us all in all aspects of business and in our own personal lives.

Services Angle

Like the Wikibon report, IDC puts an emphasis on services. At Strata last week, we heard repeatedly about the need for services to help companies become better scientists. Wikibon’s report puts CSC and Accenture in the to 10 among vendors with big data revenues. IBM is number one on the list. Clearly the market is in its early stages when services providers have such a large role in its definition.