As artificial intelligence accelerates across industries, unstructured data has emerged as both a critical asset and a growing challenge. Its value hinges entirely on how well it’s managed, governed and protected.

With unstructured data continuing to proliferate, how can companies adeptly approach long-term data management, security and governance?

“This concept of unstructured data management is becoming part of the operational cadence within many Fortune 1000 firms,” said Mark Ward (pictured, right), chief operating officer of Congruity360. “The reason is that it’s the number one cost center for storage growth because of the storage cost growth. You can imagine that the compute equivalent is growing as well. When you add AI into the equation and the GPUs associated with it, the costs continue to go up.”

Ward and Mike Krafton (left), global information governance strategist at Marsh & McLennan Companies Inc., spoke with analyst Christophe Bertrand during an exclusive conversation on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the growing complexity and opportunities of data management in the AI era, touching on the chaos of unstructured data and the importance of classification and governance in secure AI adoption. (* Disclosure below.)

Unstructured data, AI readiness and lifecycle management pose mounting challenges

The challenge of managing, utilizing and securing unstructured data streams is growing. Covering everything from emails and documents to files in SaaS platforms, such as Dropbox, unstructured data now represents the biggest cost center for storage and the largest attack surface for cyber threats, according to Ward.

“This concept of cost combined with the security threats associated with it, and eventually the desire to turn this cost center into a revenue or competitive advantage center with AI, has put a lot of very important minds in the industry together to figure out how to do it,” he said. “We’re fortunate to be in a position where we’re learning from many end users as to how they want to adopt it.”

The next data challenge is lifecycle management. Beyond its technical demands, lifecycle management has evolved into a structural and cultural challenge for companies, according to Krafton.

“You’ve taken a small, controllable set of data, and now you’ve spread it amongst the environment,” he said. “We need a representative unstructured data composition analysis. We need to know what exists where. The idea that we’re going to be able to simply point these incredible AI tools at a body of data and magic happens isn’t possible. It needs a human in the loop. We need to bring some logic and some sense to merge these two mindsets together and to start to facilitate the true value of unstructured data and its impact on AI.”

Historically, data classification has focused on structured data — databases, ERP systems and CRM tools — because those environments are more manageable in size and scope. But the future lies in unstructured data. Tools that can analyze metadata and attributes across petabytes of files, in both legacy and cloud-based systems, are rapidly becoming essential, according to Ward.

“Having the ability to go out and prosecute these data sources in a heterogeneous fashion with one single reporting dashboard is manna from heaven for many of the customers that we’re dealing with,” he said. “It’s giving them the insights that they need to take that next action. That action could be anything from remediation by deletion for data that is aged out.”

With rising cloud costs and increasing regulatory scrutiny, many organizations are reevaluating their cloud strategies. Smart migrations require visibility and planning, not just bulk transfers of data. A single platform that manages heterogeneous environments — on-prem, cloud, SaaS — is vital to enable seamless, policy-driven movement of data, Ward added.

Here’s theCUBE’s complete interview with Mark Ward and Mike Krafton:

(* Disclosure: Congruity360 sponsored this segment of theCUBE. Neither Congruity360 nor other sponsors have editorial control over content on theCUBE or SiliconANGLE) 

Photo: SiliconANGLE