Big Data is officially “a thing” with senior executives at Fortune 1000 firms indicating they’ll increase spending on collecting and analyzing data sharply over the next few years. In addition, the majority of enterprises recognize the importance of having someone to lead their Big Data efforts, and are increasingly hiring Chief Data Officers to do so.

These are the main takeaways from NewVantage Partners LLC’s latest Big Data Executive Survey 2016, which shows that Fortune 1000 companies have moved beyond the ‘hype’ surrounding Big Data projects and are now throwing resources at them with abandon.

The new normal

According to NewVantage, which quizzed 44 executives at Fortune 1000 firms for its survey, Big Data has reached the point where it’s safe to say it’s hit “mainstream adoption”. Some 62.5 percent of companies claim to have at least one instance of Big Data in production, compared with just 31.4 percent in 2013, while in another yardstick, 26.8 percent of companies reported they plan to invest greater than $50 million in Big Data projects by 2017, up from just 5.4 percent of companies that said the same in 2014.

NewVantage notes that 69.6 percent of companies state Big Data is “very important” or “critical” to their business success as further evidence of its mainstream adoption.

“Big Data has reached a point of mainstream adoption within Fortune 1000 firms,” said Randy Bean, CEO and managing partner of NewVantage Partners. “This survey has over the past 4 years reflected the evolution of executive perspectives as firms have come to terms with the opportunity represented by Big Data.”

With Big Data now being seen as essential for enterprises, it’s hardly a surprise to learn that the role of Chief Data Officer (CDO) has now become an established one among larger firms. The survey found that 54 percent of Fortune 1000 firms now employ a CDO, up from 12 percent in 2012. In addition, 20 percent of companies said the CDO is the executive who takes primary responsibility for their company’s Big Data initiatives, while 14.3 percent say the CDO is also the main sponsor of their Big Data projects.

Besides looking at Big Data’s rate of penetration in the enterprise, NewVantage’s survey also looked at the main factors driving that adoption. It found that those factors are numerous, but by far and away the biggest one was partnerships between business and technology organizations and leadership. Indeed, 33.9 percent of firms cited business and technology cooperation as the most critical factor in business adoption of Big Data, way ahead of any other factor.

The second most important factor was deemed to be strong business sponsorship, cited by 23.2 percent of firms, but surprisingly, forces like technology leadership and technology selection had minimal impact on companies’ adoption of Big Data. This shows us that partnerships and cooperation with business leaders are the keys to launching successful Big Data projects.

As for Big Data investments, some 37 percent of firms said that being able to garner greater insights into their businesses and their customers was the single biggest driver of investment into Big Data. This was closely followed by the advantages gained from speed that Big Data offers (faster time-to-answer, faster time-to-decision, and faster speed-to-market), cited by 29.7 percent of all firms. Clearly, most companies therefore see Big Data as an opportunity to seize market advantage by gaining and acting on insights faster than their rivals.

However, that wasn’t the only reason cited for investing in such projects, with other drivers including “greater analytics capabilities” (9.3 percent) and the “opportunity to create a data-driven culture”(9.3 percent) being cited.

Photo Credit: Global Water Partnership – a water secure world via Compfight cc