The rapid advancement of AI hardware, particularly in high-performance computing, is revolutionizing the enterprise computing infrastructure, with new data- and AI-centric solutions emerging quickly. Recently, a plethora of industry leaders showcased high-performance computing solutions in this bourgeoning market at theCUBE and NYSE’s AI Infrastructure Silicon Valley – Executive Series 2024 event.

TheCUBE’s John Furrier and Dave Vellante are on set in Palo Alto, CA, discussing high-performance computing.

“AI is transforming the entire tech stack, from the edge to the core, pushing the boundaries of what’s possible in computing and data management,” said John Furrier, theCUBE Research’s executive analyst. “The innovations in AI hardware are not just about speed — they’re about unlocking new levels of efficiency and scalability across the entire infrastructure.”

During the event, held at theCUBE’s livestreaming studio in Palo Alto, California, Furrier and Dave Vellante, theCUBE Research’s chief analyst, spoke with thought leaders who are trailblazing AI hardware solutions and, as a result, revolutionizing enterprise computing.

Here are three key insights you may have missed from theCUBE’s AI Infrastructure Silicon Valley – Executive Series 2024 event coverage:

1. AI infrastructure’s transformation, driven by advancements in Wi-Fi and networking innovations, is reshaping high-performance computing networks.

The evolution of networking and Wi-Fi technologies is accelerating AI infrastructure transformation, ensuring quicker, more effective data processing for AI, augmented reality and virtual reality applications. In addition, the shift toward on-premises infrastructure and intelligent network configurations drives high-performance computing, positioning modern networks as the backbone of future innovation.

Wi-Fi 7, designed with a focus on reducing latency, is critical in supporting the high-performance computing requirements of emerging technologies, such as AI, AR, VR and HPC, according to Vijay Nagarajan, vice president of wireless connectivity at Broadcom Inc. This new focus on latency is a windfall for companies using AI at the edge, where quick, seamless data processing is a requirement.

A new feature called Multi-Link Operation “makes sure that you have multiple lanes available across [the] frequency band,” Nagarajan explained. “Think of it like a carrier aggregation for cellular, being able to aggregate traffic across different bands to make sure that you get very reliable, very low-latency traffic.”

Parallel to these advancements in wireless connectivity, AI infrastructure transformation drives innovations in high-performance computing networks. One stand-out development is the shift toward on-prem infrastructure, according to Raj Yavatkar, chief technology officer of Juniper Networks Inc. This trend is driven by the need to address cost and data privacy concerns as enterprises increasingly build large GPU clusters for AI applications, which require high-performance, low-latency networks to manage data transfer efficiently.

TheCUBE Research’s John Furrier talks with Broadcom’s Vijay Nagarajan about Broadcom’s approach to Wi-Fi standards and wireless connectivity.

“Everybody is building huge GPU clusters,” Yavatkar said.

Networking is essential because the data must be transferred between GPUs after these clusters are fed data from storage networks, he added: “[The] more exciting part is that a lot of the infrastructure is not just being built in hyperscalers; it’s also being built by enterprises on-prem, which is a big shift in the market.”

The evolution of networking technology itself is another pivotal aspect of this transformation, according to Nagarajan. As AI integrates more deeply into networking, the need for more intelligent, automated network configurations grows. Navigating a complex regulatory environment is also critical, and Broadcom collaborates with global regulators to secure new spectrum.

“In 2020 April was when the FCC opened up about 1.2 gigahertz of spectrum for Wi-Fi,” Nagarajan explained. “Since then, about 70% of the world’s gross domestic product has opened up the six-giga span for new Wi-Fi.”

This additional spectrum offers more bandwidth, enabling enterprises to deploy wider channels and support more devices without compromising performance.

Advancements such as the Ultra Ethernet Consortium’s efforts to evolve Ethernet standards help ensure networks can keep pace with the demands of modern AI applications. This includes integrating intelligence-based configurations that automate network setups and manage the rigorous demands of AI workloads.

“What’s happening with GPU-based clusters, these things are coming together,” Yavatkar said. “The standard Ethernet … performs as well or better than InfiniBand-based networks. That is being recognized by [the] industry now by creating this Ultra Ethernet Consortium … to add some new capabilities to match the training workloads and inferencing workloads.”

As both wireless and wired networks continue to evolve, the collaboration between these technologies will be the foundation of high-performance computing environments that drive AI innovation.

Here’s the complete video interview with Vijay Nagarajan:

2. Integrating advanced AI architectures and full-stack solutions increases AI workload efficiency, paving the way for high-performance AI applications to be scaled across industries.

Advancements in AI computing are innovating data centers and democratizing access to powerful AI technologies, enabling efficient, scalable integration across industries.

Arm Holdings PLC’s vision of ubiquitous computing is reflected in its focus on democratizing AI by enabling AI workloads to run more efficiently on CPUs. The company’s approach highlights its role in advancing AI infrastructure’s efficiency and performance and ensuring AI hardware technologies are accessible and scalable to support widespread, cross-sector adoption.

“We have a number of capabilities … as part of our technology that makes our CPUs a lot more performant to run AI workloads,” said Ami Badani (pictured), chief marketing officer at Arm. “We call these matrix extensions and, essentially, you’re able to do vector math and those sorts of things on the CPU that you were never able to do before at almost the same token rate as you can do on other accelerators.”

Similarly, SambaNova Systems Inc. is redefining AI performance with its innovative Reconfigurable Dataflow Architecture, according to Rodrigo Liang, co-founder and chief executive officer at SambaNova. This architecture dynamically maximizes computing resources, enabling the company’s AI chips to excel in many applications by simplifying large language models’ integration into operations.

TheCUBE Research’s John Furrier and Dave Vellante talk with Arm’s Ami Badani about the trends for next-gen semiconductors.

“People want to go to production,” Liang said. “We’re able to do that [at] 115 tokens per second, which is 10 times faster than anybody else out there, and we run it at less than 19 kilowatt[s].”

As AI becomes more central to many industries, Arm is innovating its data center design, replacing the traditional model of rack-and-stack servers with supercomputer clusters handling the heavy demands of AI workloads. The company’s strategy focuses on optimizing the entire tech stack, from firmware and drivers to the application layer, ensuring the power and efficiency of its devices.

“It’s a new way to think about your data center,” Badani said. “From the chip to the systems to the interconnect and the networking, all of that stuff is the way you think about your data. We’ve fundamentally changed the way that most people think about their data centers from chips then to systems now.”

In parallel, SambaNova’s advanced chip design and the company’s full-stack approach simplify the deployment of complex AI applications within enterprise environments, according to Liang. Eliminating the need for low-level programming and manual configurations allows organizations to integrate AI technologies, such as Llama 400B and GPT-4-like models, into their infrastructure more efficiently. This approach enhances AI performance and reduces technical barriers to adoption, making advanced AI more broadly accessible across industries.

“We build our own substrates that run the largest models and the most complex models, and we run them faster than anybody else,” Liang said. “On the other bookend, we build a full stack that allows you to actually deploy these models without having to learn CUDA [and] without having to do all the low-level manual work. You can just download these models and then run them really fast.”

Here’s the complete video interview with Ami Badani:

3. As AI infrastructure evolves, new technologies allow enterprises to scale more efficiently by optimizing resource usage, enhancing performance and reducing costs — unlocking new opportunities for productivity and innovation across sectors.

The need for integrated, high-performance systems has never been more critical as AI advances, leading to cutting-edge technologies that supercharge AI infrastructure, such as Celestial AI’s Photonic Fabric. Meanwhile, Pure Storage Inc.’s innovative approach to storage is designed to ensure that storage solutions remain up-to-date and efficient without the need for costly replacements.

“Hardware and systems are cool again,” said Charles Giancarlo, chief executive officer of Pure Storage. “A lot of this is being driven by the excitement around AI but also the recognition that what’s going on in data centers, what’s going on in the world of compute, is accelerating at this point and creating a lot of opportunity for productivity for businesses and for people alike. What we’re seeing in the AI space is that it’s opening up multiple layers of opportunity.”

Similarly, Photonic Fabric, an optical interconnect technology platform, connects AI accelerators using photonics, offering high bandwidth, low latency and low power interconnectivity, according to Dave Lazovsky, co-founder and CEO at Celestial AI. These features are essential for advancing AI performance to the next level.

TheCUBE Research’s John Furrier and Dave Vellante talk with Pure Storage’s Charles Giancarlo about the evolution of storage hardware in hybrid environments.

“What the Photonic Fabric enables is not just an increase in the energy efficiency in the performance associated with compute-to-compute fabric interconnectivity, but, because we have a technology platform that enables bandwidth that is equivalent or superior to HBM3 or HBM4, we have the ability to disaggregate memory,” Lazovsky said. “Photonic Fabric enables it with optically interconnected HBM.”

As enterprises seek to optimize their AI infrastructure, they are increasingly focusing on high-performance computing solutions that are sustainable, scalable and capable of adapting to changing needs. The evolution of storage toward more flexible solutions, such as Pure Storage’s virtualized systems, aligns with the broader industry trend of reducing redundancy and improving data accessibility for AI models.

“We’re creating an environment where on our arrays — all of that data — appears as a pool of data,” Giancarlo explained. “It’s accessible by AI — what AI wants is access to data. Now you have to have the proper security mechanisms in there, role-based access controls, but then you want to allow it to have access to the data where it sits instead of having to be able to replicate it.”

In parallel, Photonic Fabric addresses the growing demand for efficient memory scaling, providing a pathway to manage AI computing workloads more effectively without relying heavily on traditional GPU-based memory controllers. This approach enhances performance and reduces costs associated with AI infrastructure by optimizing resource usage.

“There’s two drivers for the package getting bigger,” Lazovsky said. “It’s an increase in the compute horsepower per package … so there’s a need for more and more high bandwidth memory across AI acceleration systems. By eliminating the use of GPU as the world’s most expensive memory controller, if you don’t need the flops and you’re just buying GPUs for memory capacity, there’s a more efficient way to do it.”

Here’s the complete video interview with Charles Giancarlo:

To watch more of theCUBE’s coverage of this event, here’s our complete event video playlist:

Photo: SiliconANGLE