As artificial intelligence and high-performance computing workloads evolve, hybrid cloud data performance is becoming essential for enabling scalability and efficiency in modern infrastructures.

This integration is driving seamless data movement, expanded capabilities and improved performance. Key developments include integrating accelerated hardware, such as graphics processing units, with highly performant data pipelines that handle massive datasets, according to Phil Curran (pictured), cloud strategist at WekaIO Inc.

WekaIO’s Phil Curran talks with theCUBE about hybrid cloud data performance.

“The infrastructure is fundamentally different to support these types of workloads,” he said. “They have varied data performance characteristics. They’re usually in some kind of a data pipeline. They require handling of millions of tiny files with sub-millisecond latency, in other words, incredibly low latency, while at the same time delivering massive [input and output] and high bandwidth. And they’re usually massive data sets.”

Curran spoke with theCUBE Research’s Rob Strechay at Microsoft Ignite 2024, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how WekaIO’s high-performance data platform integrates with Microsoft Azure to optimize AI and HPC workloads through seamless data portability, scalability and performance enhancements.

How hybrid cloud data performance optimizes AI and HPC workloads

As a software-based solution, WekaIO is designed to seamlessly manage the complex data performance needs of modern workloads, providing scalability and speed. The company’s ability to deliver consistent performance across hybrid environments, whether on-premises or in Azure, has positioned it as a great partner for enterprises transitioning to cloud-based systems, Curran explained.

“What we’re doing with Microsoft, and in particular within the Azure ecosystem, is providing that massively performant and highly scalable data solution that can really power these AI and these HPC applications so the customers can do more in Azure with those workloads,” Curran said. “We’re helping customers often bridge from an on-premises HPC or an on-prem AI environment and into Azure. Customers are finding … that we’re actually able to bridge from on-prem into Azure with the customer.”

Meanwhile, AI’s potential to transform industries such as life sciences is becoming a reality, thanks to technologies that support massive data and performance requirements. With hybrid cloud data performance solutions such as WekaIO and Azure, organizations can seamlessly manage and scale AI workloads across environments.

“What they’re doing is capturing medical imaging data in their labs and then bursting that into the cloud, into Azure, to run their analytics workloads,” Curran said. “The way that they’re leveraging Weka is [to] capture the data on-prem, feed that into the Azure analytics environment, run that high-performance data processing in Azure and then capture the results and then archive it all down … Because we leverage things like autoscaling, customers can actually scale up an entire Azure environment to run that environment, and that’s exactly what this customer’s doing.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of Microsoft Ignite 2024:

Photo: SiliconANGLE