The trend has been clear for some time: Data is changing, the script is flipping and the intelligent data platform is rising. Generative artificial intelligence is set to change the data layer completely.

The ongoing platform shift was a central focus of the Supercloud 7: Get Ready for the Next Data Platform event, which ran July 30-31. According to John Furrier (pictured), executive analyst for theCUBE Research, the platform shift absolutely targets what will happen next as infrastructure improves.

Rob Strechay, John Furrier, Savannah Peterson and Dave Vellante of theCUBE talk about the intelligent data platform during Supercloud 7.

“As infrastructure closes the loop and gets stronger, the data layer is going to change very rapidly,” Furrier said during Supercloud 7. “The data’s clear that this is evidently happening right now in the market.”

The opportunities for innovation and many other subjects were explored by Furrier, Dave Vellante, chief analyst at theCUBE Research, and a full team of analysts at theCUBE Research. They discussed the future of AI infrastructure and the rise of the intelligent data platform.

Here are three key insights you may have missed from Supercloud 7:

1. Enterprise customers are conflicted on data governance.

Supercloud 7 included a discussion of Enterprise Technology Research survey results recently published by SiliconANGLE involving more than 100 joint Databricks Inc. and Snowflake Inc. accounts. The results suggested that enterprise customers are conflicted about the best route possible when it comes to balancing data trust with a desire to move fast and innovate amid the rise of the intelligent data platform.

When asked about their preferences for open table formats, most ranked security and governance and lock-in avoidance first and second. But there’s a “swing vote” at play here, according to Vellante.

George Gilbert of theCUBE Research talks about ETR survey results during Supercloud 7.

“There’s 14% that said data security is less important than creating a stack that allows for rapid innovation. Why is that the swing vote?” he asked. “It’s because ETR correlated that and found it. Those were the ones that were most likely to [leave] Snowflake and move to Databricks as an example, because they’re saying ‘damn the torpedoes’ in governance.”

When one starts talking about open table formats, they must essentially start abstracting governance from the purview of any one vendor, according to theCUBE Research’s George Gilbert. That means thinking about governing all of the data in one’s data and application estate.

“Snowflake’s appeal was, ‘Hey, we’ll take care of governance. You bring whatever tool you want, go through our engine, and it’s all seamless.’ And then they were like, ‘We’ll extend that to all Iceberg tables, and we’ll synchronize governance to the Polaris catalog,’” Gilbert said. “You have governance across your Snowflake data and your open Iceberg data.”

Meanwhile, Rohan Kumar, corporate vice president of Azure Data at Microsoft Corp., has suggested that its Microsoft Purview Data Governance solution would see users set policy and tag all data — not only across Azure data, but across clouds, on-prem, and edge. Microsoft would author the policies, and users would set the tags.

“No one governance rules them all is really the message there,” Vellante said.

As database vendors navigate the shifting landscape of compute engines for AI, theCUBE analysts see changes ahead. Table formats have changed the game, according to Sanjeev Mohan, principal at SanjMo and analyst for theCUBE Collective.

“If I can build reports and dashboards very easily in Snowflake with an amazing user experience, great. But if I want to do AI and machine learning, maybe I should use Databricks,” he said. “If I want to do my data cleansing and data prep, I should use Apache Spark. It’s not making copies of the data and moving it around, but it’s changing the compute engine.”

2. Companies are seeking to respond to the intelligent data platform.

Amid the rise of the intelligent data platform, companies are making a concerted effort to enhance scalability, flexibility and security. That plays out in several ways, including for the multinational retail corporation Walmart Inc.

The retail giant has sought to utilize large language models for online and physical commerce. The company’s machine learning platform, Element, has changed the game for its cloud operations, according to Anil Madan, senior vice president of cloud and data platforms at Walmart Global Tech.

Microsoft’s Dipti Borkar talks about the future of data platforms.

“We manage from a data life cycle, thousands and thousands of data sources. The data lifecycle is data moving in more in a raw format, coming into a central lake. It goes through certain transformations and quality to create a data catalog,” Madan said. “That data catalog helps very much power every aspect of our analytics and machine learning.”

Meanwhile, Databricks has also sought to advance the evolution of data platforms by emphasizing the use of open formats and AI integration, with an eye on driving efficiency in data management. When it comes to future-proofing data platforms, open formats and interoperability between different data engines are crucial, according to Ali Ghodsi, co-founder and chief executive officer of Databricks.

“[Customers are] super excited to see interoperability between these formats,” he said. “They don’t want to see incompatibility between Delta Lake and Iceberg. The fact we’re unifying them in project UniForm, and now you can get both of those and you get full compatibility of your data, they loved it.”

Microsoft also has its eyes on advancing the future of data platforms as it shifts to pure open formats, with Microsoft Fabric in particular. It represented a dramatic change for the company, according to Dipti Borkar, vice president and general manager at Microsoft.

“[We moved] all our engines to reengineer these computes, to now read these native formats,” Borkar said. “We support Delta Lake, and Iceberg is landing very soon. The reason that these are important, again, customers get a choice.”

3. There’s an effort to enhance data platforms with open standards and interoperability.

Among many key insights at Supercloud 7, it quickly became clear that the evolving landscape around the intelligent data platform includes a focus on the adoption of open standards and seamless interoperability. Snowflake, for instance, has sought to balance solutions integration with the adoption of open standards.

Snowflake, of course, has made Polaris open source. The company’s goal is to commit to open-source principles and enhance its products through collaboration, according to Benoit Dageville, co-founder and president of product at Snowflake.

Snowflake’s Benoit Dageville talks open-source principles during Supercloud 7.

“Our customers, or a class of our customers, is asking us for interoperability between tools,” Dageville said. “Many tools are accessing the same data, and they want Snowflake to be one of these tools or one of the systems accessing this open data. They ask us to come to their data versus their data coming to us, and they want to avoid vendor lock-in.”

Vast Data Inc., meanwhile, has aimed to create a unified data pipeline that integrates multiple clouds and data centers. That allows companies the ability to federate multiple clouds with the Vast DataSpace, according to Jeff Denworth, co-founder of Vast Data.

“Essentially, it allows you to kind of flow all your files and all of your records that sit in tables across a series of independent cloud platforms so that you can have one unified pipeline,” Denworth said.

For some companies, there’s been a shift toward platform-centric models. TransUnion LLC has sought to embrace that new shift given its tens of thousands of daily data sets and more than 50 billion data points, according to Venkat Achanta, chief technology, data and analytics officer at TransUnion.

“We curate that data, the data quality, cataloging of the data, more importantly connecting that data we call identity resolution, trying to figure out where this food this data belongs to and then serving up that data into intelligent predictive applications and cognitive applications for banks and the consumers to consume,” Achanta said.

Stay tuned for the complete video playlist, part of SiliconANGLE’s and theCUBE Research’s coverage of the Supercloud 7: Get Ready for the Next Data Platform event.

Photo: SiliconANGLE