Over the last few years, the emergence of the Chief Data Officer (CDO) has become more clear. Initially, this new C-level role was more prevalent in healthcare, finance, government organizations. More recently, it has been bleeding into more commercial companies. In a live segment introducing Day Two of the MIT CDOIQ conference, theCUBE hosts Dave Vellante, Jeff Kelly, and Paul Gillin discussed the latest developments around the CDO evolution.

One idea noted by Vellante is that the “CIO role actually morphing into the COO and CTO, with the CDO emerging and reporting to the COO.”

Gillin explained this is a normal occurrence. “What is information but data? We have a CIO and CDO, data equals information, why do we need two people to handle that role?” With the CIO being more of an infrastructure role, that is how the position should naturally evolve.

Where data governance fits in

Vellante stated that when most companies implement Big Data projects, there’s a lack of oversight in how they relate to governance. Kelly confirmed that “data and analytics products have often been siloed within different departments,” and because of this lack of oversight at corporate level, “tying them to corporate governance is difficult.”

As employees have increasing access to free social services, they need to be aware of the implications of using them, and the data going out. The question remains “how do you use all that information ethically, legally? I don’t think there’s a lot of oversight right now. Most organizations treat them as siloed projects and they don’t see it as a wider view in governance,” Kelly added.

Gillin said one of the biggest problems was “we tend to think of data as being structured.” In reality, 80-90 percent of data is not structured. The big governance challenge will be to find a way to make sense of that unstructured information.

Mentioning his recent Big Data practitioner survey, Kelly, an analyst with Wikibon, spoke of relevant findings pertaining to the CDO conversation. Kelly said one of the questions was regarding the identification of the biggest barriers associated with moving Big Data analytics projects into production. “From a nontechnical perspective, it’s confusion around privacy and compliance issues,” he recalled. “People don’t understand what their responsibilities are and they don’t want to get into legal or ethical issues.”

Vellante stressed that such findings “speak of the lack of some kind of data czar.” Gillin explained that “the CDO job fundamentally is not about owning data, but about owning governance, how the data is handled, communicating that to everyone who is involved.”

The current state of data warehousing

Discussing the current state of data warehousing, Vellante asked if, in its current form, the technology was a dinosaur. “The approach taken for 10-20 years has failed to live up to its expectations, I think we can all agree on that,” said Kelly, stating that data warehousing proposes a “rather inflexible data model.”

There is a “difference between understanding the content and the context,” Kelly went on. “Everyone can understand the content, what it actually means in the context, that’s the real challenge.”

Vellante emphasized that a large number of Kelly’s practitioner survey respondents said they shifted resources from data warehousing to Hadoop, and more will do so by the end of the year. Kelly said there was some nuance in that finding, as a little over 60 percent have shifted one workload or another from traditional mainframe or data warehousing to Hadoop. Most commonly this refers to workloads pertaining to transformations, business intelligence, or reporting, he explained.

Gillin added that data warehousing has had its biggest impact in fields such as marketing and customer relations, or retail where there has been significant payoffs. “In banking, healthcare, manufacturing, I have not seen examples of data warehousing paying off,” he said.

“Hadoop is sustainable as long as you as an organization continue to innovate,” Kelly stated, when new assets were being developed and added. “Hadoop and Big Data are just a platform and an enabler. If you continue to innovate, and look to exploit new values inherent in your data, then there can be sustainable growth, if you can build that culture of always exploiting your data to its fullest.”