The Big Data market reached $18.6 billion in 2013, a 58 percent growth rate over 2012, writes Wikibon Principal Research Contributor and Big Data Analyst in his just released “Big Data Vendor Revenue and Market Forecast 2013-2017”. Big Data-related services revenue makes up 40% of the total, with hardware at 38% and software at 22%.

To learn more about Big Data tools, vendors, and services, watch theCUBE’s coverage of its own event, BigDataSV 2014, February 11-13 on SiliconAngle.tv.

The major drivers of market growth through the year, he writes, include:

• Growing confidence in Big Data products and services among enterprise buyers as a result of improved marketing from the vendors,
• Growing maturity of those Big Data products and services, including the release of YARN,
• The addition of better privacy, security and governance capabilities in those products as well as improved backup and recovery and high-availability for Hadoop specifically, and,
• The growing number of partnerships, particularly reseller agreements and technical partnerships between Big Data and non-Big Data vendors.

However, he writes, barriers to adoption still exist in the market. These include:

• A lack of best practices for integrating Big Data analytics into existing business processes,
• Concerns over security and data privacy,
• Continued “Big Data washing” by legacy IT vendors,
• A volatile and fast developing market, and,
• A lack of polished Big Data applications for solving specific business problems.

The introduction of YARN (Yet Another Resource Negotiator) was an important step in technological development. Introduced in 2013 after three years of development, it provides the structural foundation for Big Data analytics to move beyond MapReduce-style batch processing. YARN, he writes, is critical to the future of Hadoop, ensuring that it will grow beyond backroom data science projects to take a prominent role in modern data architecture. YARN is complimented by several moves by vendors to better integrate the open source Big Data framework with existing data management infrastructure and legacy databases.

Market Forecast

Wikibon forecasts that Big Data market growth will slow slightly in 2014 to 53 percent, reaching $28.5 billion for the year. It is currently on pace to reach $50 billion in 2017, a 38 percent compound annual growth rate over the six year period from 2001.

As the market matures, he writes, Big Data applications and cloud-based services to gain an increasingly important role, with mainstream users and late-adopters adopting polished applications and services from cloud providers to sit on top of their Big Data infrastructure.

Kelly warns that the public’s ire at the NSA fed by the continuing revelations in the news is likely to shift focus to large commercial companies as the public develops a better understanding of how social networks, retailers, banks and other businesses use Big Data. Industry must address these concerns sooner rather than later. This makes good business sense and is the responsible thing to do.

As with all Wikibon research, this report is available without charge on the Wikibon Web site. IT professionals are encouraged to register for membership in the Wikibon community, which allows them to connect with peers, participate in Wikibon research, comment on published research, and publish their own questions, tips, Professional Alerts and longer research.