As organizations embrace cloud optimization strategies, they’re moving beyond traditional analytics toward advanced data infrastructures.

This shift is undertaken to power operations, inform business decisions and deliver differentiated products. As generative artificial intelligence and large language models influence how software is developed and deployed, data is no longer just an asset to analyze; it’s becoming the foundation of innovation and competitive advantage, according to Pete DeJoy (pictured), co-founder and senior vice president of product at Astronomer Inc.

Astronomer’s Pete DeJoy talks to theCUBE about cloud optimization strategies.

“We’ve actually seen the secular trend flip over the last really three or four years where companies have moved from using data to analyze their business to using data to operate their business,” DeJoy said. “And AI has obviously accelerated that tenfold, but now not only they’re operating their business with data, they’re differentiating their business with data. And as a result, there’s been this huge focus on data infrastructure and data foundation.”

DeJoy spoke with theCUBE Research’s John Furrier for theCUBE’s “Cloud AWS re:Invent Coverage,” during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed rethinking data’s role, steering away from mere analysis toward becoming a vital operational force, utilizing open-source tools, data orchestration and generative AI.

How cloud optimization strategies redefine data orchestration and innovation

Organizations are realizing that while analytics once helped them understand where they’ve been, operational data-driven workflows tell them where they’re going. This evolution calls for a stronger, more flexible infrastructure capable of supporting huge volumes of data from multiple domains, DeJoy argued.

“We built Apache Airflow…we’ve built a great partnership with AWS and other public clouds and making sure that Airflow is the de facto standard for data orchestration,” he said. “On the commercial side, we’re mostly working with high-scale, sophisticated data engineering and data platform teams that are building shared services for Airflow internally. One of our big kind of shared customers with AWS is Marriott.”

As generative AI redefines the relationship between developers, data and infrastructure, it’s also pushing organizations to rethink workflows. Many assume that generative AI is merely a shortcut to productivity, a convenient tool for code generation, DeJoy explained.

“You can think about high context developer experience, wherein your IDE assistant knows what databases you have access to, what schemas you have access to, where you should be reading from, where you should be writing to, what the API schema that you’re trying to pull from looks like,” he said. “All of that is deeply embedded into the core developer experience. On the other side of the house, we’re very interested in how we can use agents to go do auto-remediation at the data infrastructure level and help people save money.”

By aligning strategies with cloud computing trends, enterprises can fully leverage emerging frameworks and orchestrations for greater impact. It’s about “build, run and observe” cycles where developers quickly write and test code, operationalize their workflows and continuously monitor outcomes, DeJoy concluded.

“On the build side, we have a bunch of great developer experience abstractions that help people move more productively,” DeJoy said. “On the run side, we’ve built a bunch of IP into the way we manage and run Airflow for customers…And on the observed side, we have a full suite of data observability tooling that gives you data lineage across your whole data platform and proactive alerting for potential bottlenecks in your supply chain of your critical data products.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s “Cloud AWS re:Invent Coverage”:

Photo: SiliconANGLE