Driven by breakthroughs in artificial intelligence, cloud infrastructure and advanced data platforms, the landscape of AI data governance is rapidly evolving.

This technological shift is not only reshaping how businesses manage and leverage data, but also heralding a new era of intelligent applications and enhanced operational efficiency. With industry leaders, such as Snowflake Inc. and Databricks Inc., pioneering these advancements, the focus is on democratizing data access, bolstering security and governance and integrating AI to anticipate and meet future demands.

TheCUBE’s Savannah Peterson and Dave Vellante discuss AI data governance.

“That’s where AI could come in,” said John Furrier (pictured, left), executive analyst at theCUBE Research. “That’s why the cloud is so critical to this discussion, because cloud and the data platforms are going to be integrated and the AI apps running on them is what’s going to be the big focus of this conversation today.”

Furrier spoke with theCUBE Research’s Savannah Peterson (middle) and Dave Vellante (right) at the Supercloud 7: Get Ready for the Next Data Platform event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the transformative impact of AI, data governance challenges and the evolution of data platforms.

AI data governance driving transformational changes in data infrastructure

The latest research and surveys suggest that data is rapidly transforming, with significant changes anticipated in the data layer due to advancements in infrastructure and AI. This rapid evolution is driven by improvements in infrastructure, with new hardware and formats emerging, leading to a stronger data ecosystem. The integration of AI into data platforms is expected to change the way businesses operate, enabling real-time analysis and decision-making capabilities.

“There are hundreds of billions of dollars of waste spent on supply chain and the way that the industry handles it is they build up inventory and that’s just a waste,” Vellante said. “If you ever get frustrated because you’re supposed to get an order show up and it shows up in pieces, that problem is going to go away because people will be able to anticipate problems.”

The integration of data and DevOps, coupled with AI, is creating a new paradigm in data management and utilization. The confluence of data and cloud technologies is essential for the development of intelligent applications, according to Vellante.

“The picture that I’m depicting here is that the data management needs a set of kind of standards in the middle to allow … a very diverse application of the data at the top and very diverse kind of storage and computer at the bottom,” he said.

The shift toward open data formats and governance is seen as fundamental to the future of data platforms, enabling more seamless and efficient data management. The survey conducted by theCUBE Research and ETR revealed that while only 15% of respondents currently use open data formats, a significant number are planning to adopt them, indicating a strong trend toward more open and accessible data environments, according to Furrier.

“As infrastructure closes the loop and gets stronger, the data layer’s going to change very rapidly,” he said. “The data’s clear that this is evidently happening right now in the market.”

Addressing the challenges of data governance and security

With the increasing importance of data as a proprietary asset, ensuring its security and proper governance is important. There is a necessity for strong governance frameworks to manage the complexities of modern data platforms, according to Furrier and Vellante.

“Same problem that we had with back in the old data warehouse days. That whole data pipeline is going to get disrupted to John’s point with open data formats,” Vellante said. “Democratizing that data, putting data in AI in the hands of business users, that can actually create not only data products, but new applications on top of that with low-code and no-code. It’s just that we’re entering a new world.”

AI data governance, in the form of the integration of AI with data platforms is expected to drive the development of new applications that can predict and preempt future needs, enhancing everything from supply chain management to consumer experiences, according to Vellante.

“We’re entering a new era and it’s really … what the next data platform is all about,” he said. “Whereas the current modern data platform was about separating compute from storage and doing everything in the cloud, the next data platform is all about allowing any computing engine … it’s about opening up the data elements, making them coherent and then building new intelligent applications on top of that.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the Supercloud 7: Get Ready for the Next Data Platform event:

Photo: SiliconANGLE