The global big data market will grow from $18.3 billion in 2014 to a whopping $92.2 billion by 2026, representing a strong compound annual growth rate of 14.4 percent, according to Wikibon’s newly published 2016 – 2026 Worldwide big data Market Forecast.

The research firm called 2015 “a breakthrough year for big data”, with the market growing by 23.5 percent, led mainly by Hadoop platform revenues. That was evident in the explosive growth of Hortonworks Inc. and other Hadoop vendors, as well as the rapid adoption of Apache Spark and other streaming technologies, Wikibon says. This growth in big data is being fueled by a desire among larger enterprises to become more data-driven, as well as the emergence of new, Web-based, cloud-native startups like AirBnB Inc., Netflix Inc. and Uber Technologies Inc., which were conceived with big data at their core.

Software market to surge

Wikibon breaks down global big data market revenues into three segments: professional services (40 percent of all revenues in 2015), hardware (31 percent) and software (29 percent).

However, Wikibon’s projection for 2026 shows a markedly different split, with rapid growth in big data-related software set to ensure that that segment overtakes the other two to account for 46 percent of all big data spending in the next ten years, with professional services at 29 percent and hardware at 25 percent. This shift will occur due to the development of better quality software that reduces the need for big data-related services, Wikibon said.

The big data software segment is expected to grow at a CAGR of 20 percent between 2014 and 2026, which is significantly faster than the hardware and professional services segments. Indeed, Wikibon said it expects to see substantial growth in all four sub-segments of big data software through 2026, let by data management (14 percent CAGR), core technologies such as Hadoop, Spark and streaming analytics (24 percent CAGR), databases (18 percent CAGR) and big data applications, analytics and tools (23 percent CAGR).

Wikibon is also predicting strong growth in the big data hardware segment, with revenues set to grow at a 12 percent CAGR through 2026.

This growth will come at the expense of professional services, however, which Wikibon says will decline as the ecosystem gains more experience with big data tools, methods, and conventions. Over time, this experience will translate into “new and higher quality applications”, thereby reducing the need for professional services. Nonetheless, it’ll take some time before this happens; Wikibon predicts that professional services will remain the largest market segment until 2022, growing at an 11 percent CAGR. As we enter the next decade however, Wikibon is confident that professional services will be subsumed by new cloud options that become increasingly viable for a larger portion of the big data stack, thus slowing the segment’s growth to just a three percent CAGR between 2022 and 2026.

Fueling big data’s growth

Wikibon cites a number of factors that will drive this rapid rate of big data adoption, including “demand-side” growth drivers such as maturing data lakes, evolving intelligent systems of engagement and emerging intelligent self-tuning systems.

The analyst firm also highlights important supply-side growth drivers, such as the public cloud and increasing administrator and developer productivity as well as adjacent technology growth drivers such as the Internet of things (IoT) and rich media, both of which represent a significant opportunity in the healthcare, entertainment and surveillance industries.

Obstacles

One obstacle to big data’s growth is the relative immaturity of technologies. Wikibon says that even today’s most popular big data technologies, such as Hadoop and data lakes, still need to undergo a massive transformation if they’re to expand their presence beyond early adopters. However, Wikibon expects such systems to become more simplified in the coming years, with newer technologies like Apache Spark being an important factor.

Another major obstacle is the much-discussed “skills gap,” which Wikibon says remains the number one barrier to big data adoption in many enterprises. Another barrier is the problem of big data governance, or rules for use. The first problem can be alleviated in the coming years due to simple supply-and-demand dynamics as well as the accelerated delivery of standards and conventions for big data programming models, administration processes and insight-delivery methods. The problem of data governance will be trickier to solve, however, because it transcends technology. Wikibon doesn’t expect best enterprise practices for data governance to emerge until 2020 at the earliest.

Recommendations

Wikibon urges enterprises, users and vendors alike to embrace the potential of big data analytics. Enterprises need to begin exploring how big data can be applied to their business processes at the earliest opportunity, Wikibon says, starting out with small proofs of concept in order to develop skills and gain experience. For users, Wikibon recommends they educate themselves about the potential of big data and how it can help them both today and tomorrow. For vendors, the main objective should be to rapidly develop a big data software ecosystem in order to help customers realize the potential of big data.

*Disclosure: Wikibon is owned by the same parent company as SiliconANGLE*