In this era of data-driven business, way too few companies are walking the walk.

That’s the gist of the 2020 Big Data and AI Executive Survey by NewVantage Partners LLC to be released Monday. In their introduction to the report, authors Tom Davenport and Randy Bean mince few words about their disappointment with the progress responding organizations are making toward becoming more data-centric.

“Continuing in the current vein or giving up on the objective of data-driven organizations and cultures isn’t really a viable option,” they write. “That would mean the eventual demise of legacy organizations in favor of digital-native firms.”

The results of this year’s survey of executives at more than 70 large enterprises, which will be released on Monday, documents the reason for their concerns. Less than 40% of respondents said their organizations compete on analytics or manage data as a business asset, a number that has changed little from previous surveys. Only 31% call their organizations data-driven and just 28% report success in building a culture around strategic data use. More than 90% cited people and process challenges as the biggest impediments to those objectives.

The survey, which has been conducted annually since 2012, primarily covers financial services firms, although healthcare companies represented one-quarter of respondents this year. The small sample size is offset by the large number of blue-chip enterprises that respond and the consistency of results over the years.

Enterprises are investing more than ever in data technologies. Nearly 65% of firms reported that they spent more than $50 million on data and AI in 2019, up from 40% the year before. More than 90% are investing in artificial intelligence, although fewer than 15% have any AI projects in widespread production.

Progress slows

But there are also signs that enterprises are moving less deliberately than in the past toward data-centric thinking. While 52% of firms said they’re accelerating their investments in big data and AI, that’s way down from nearly 92% last year. The 38% of organizations that call themselves data-driven has changed little over the past four surveys.

Becoming data-centric “is counterintuitive for most legacy companies,” NewVantage Chief Executive Bean said in an email interview with SiliconANGLE. “It requires a new orientation and way of thinking. Some will make it; a good number will probably not.”

Those who don’t “will be increasingly vulnerable to disruption by digital-native companies and competitors who have made the transition,” Bean said. And, he added, in a reference to Ernest Hemingway’s 1926 novel “The Sun Also Rises,” this kind of failure tends to happen “gradually, then suddenly.”

The CDO rises

The survey also documented some important shifts in attitudes toward the role of the chief data officer, a position that barely existed a decade ago but will be established in 90% of global organizations this year, according to Gartner Inc.

While initially adopted primarily by regulated firms to meet reporting requirements, the CDO is increasingly taking on a strategic role, according to the survey. Almost 55% of respondents said their CDO is primarily focused on revenue generation, up from 44% in 2017. A minority 45% said the job is more compliance-oriented.

However, the number of companies that reported having a CDO position declined slightly from previous surveys, a finding that Bean said reflects “some quite open disillusionment” with the role.

“I think it is more a case of getting it right than abandoning it,” he said. “The intent is noble, but the execution needs to get better.”

There has also been a change in perception of what background makes for a good CDO: Nearly 49% of respondents said the best candidates would come from outside the organization, up from 34% in 2018.

At the same time, the percentage of executives who favor insider candidates dropped by more than half to under 16%. “There is a view that an outside perspective is needed if you are going to seriously change the way you operate today,” Bean said.

Judging by the number of organizations struggling to be more data-driven, that may be the only option.

Image: Gerd Altmann/Pixabay