Faster chips alone no longer define the AI buildout. Hundreds of thousands of processors now work together as one machine, so AI infrastructure hardware is designed and judged as a complete system. That system runs from the optical links between racks to the memory, storage and power that keep GPUs busy.

The next phase will be defined by the bottlenecks around the processor. Copper is giving way to light, and the memory wall is moving rather than falling. Power systems built on cycles of up to a decade must keep pace with chips that turn over every year. Demand is not in doubt: Memory and optical laser capacity are booked out for years.

“A lot of the focus in AI has been very much on compute and on memory, but our view is actually AI is a data system,” said Irving Tan (pictured), chief executive officer of Western Digital Corp. “Ultimately, you need the entire ecosystem to work together and to be able to move in lockstep.”

Tan and other industry experts spoke with theCUBE Research’s John Furrier during The CEO Summit in Santa Clara, California. The inaugural summit marked the 35th anniversary of the Chinese American Semiconductor Professional Association. The interviews covered optical interconnects, inference economics, memory pooling, maskless lithography, physics-based computing, power resilience and AI-driven chip design.

Here are 11 standout insights from the event:

1. Hard drives gain throughput as storage becomes the AI data layer.

Data creates intelligence and storage makes it persist, so data must be split into hot, warm and cold tiers across the cloud, the edge and devices, Tan explained. Citing IDC, he noted that the amount of data generated is projected to triple to more than 700 zettabytes over three years. Western Digital expects demand for data storage to grow at a compound annual rate of at least 25% over five years.

Explore theCUBE’s in-depth discussion.

2. Connectivity becomes the choke point for AI data centers.

Semtech Corp. supplies the chips inside the optical transceivers and copper cables that link thousands of AI processors into a single computer. Every link flap can force a job to restart, so connectivity now governs GPU uptime and, with it, revenue, according to Hong Hou, president and CEO of Semtech. Hyperscalers want more bandwidth at lower power, which leaves more of a data center’s fixed power budget for compute.

Check out theCUBE’s complete interview.

3. Inference economics push chipmakers toward workload-aware racks.

Agents now consume tokens around the clock, making performance per dollar per watt the critical metric of the inference era rather than raw teraflops, explained Sunghyun Park, co-founder and CEO of Rebellions Inc. The step beyond today’s rack-scale systems is racks tailored to whether a workload is latency-critical, throughput-hungry or energy-constrained. Models change every few months, so Rebellions designs flexible hardware and sees the network as the next bottleneck as context windows grow.

Watch the full interview from theCUBE.

4. Maskless e-beam lithography targets the shift from Moore’s Law to advanced packaging.

Moore’s Law is reaching its limit, and advanced packaging that combines logic, memory and input/output chips is taking over as the industry’s driving force, noted David Lam, founder of Lam Research Corp. and CEO of Multibeam Corp. Multibeam shrank an electron-beam column from 80 centimeters tall to the size of a marking pen, so arrays of columns can write a wafer in parallel. The resulting maskless lithography avoids the reticle limit that forces large quantum and AI chips to be stitched together.

See theCUBE’s full coverage.

5. CXL memory pooling gains new life as the memory wall persists.

The memory wall may not be solved for five years, emphasized Greg Ernst, chief business officer of ScaleFlux Inc., who recently joined from Intel Corp. Even an 8 billion-parameter model serving many users needs far more memory than high-bandwidth memory can hold. Compute Express Link, or CXL, lets multiple servers share a pool of memory. Hyperscalers that reuse older DDR4 modules in those pools are reviving interest in the technology.

Don’t miss the full interview on theCUBE.

6. In-memory computing takes aim at the memory supply crunch.

New memory fabs take three to four years to come online, leaving most memory from top suppliers sold out through 2028, according to Faris Sbahi, co-founder and CEO of Normal Computing Corp. The startup is moving away from high-bandwidth memory toward commodity LPDDR5. It also performs some inference calculations directly in memory to cut cost and energy use.

Hear the full story on theCUBE.

7. Physics-based computing aims for human-level intelligence in a single rack.

Unconventional AI Inc. is building a fully integrated machine that uses the time-varying physics of the real world to compute, much as a rolling ball inherently knows its next position, shared Naveen Rao, co-founder and CEO of Unconventional AI. The company’s first system targets human-level intelligence at about 10,000 watts in a single rack, compared with the brain’s 20 watts. Rao aims to keep driving that figure down until it beats biology, cutting the cost of intelligence enough to open up new form factors.

Catch the complete conversation on theCUBE.

8. Optics replaces copper as AI clusters outgrow electrical links.

Power limits are pushing operators to link multiple smaller data centers with fiber, and converting electrical connections inside the data center to optical ones is the next growth driver, highlighted Michael Hurlston, president and CEO of Lumentum Holdings Inc. The company has grown fourfold in a little more than a year, yet it runs about 70% below demand and is essentially booked through 2029. Indium phosphide laser chips are a key constraint, so Lumentum is investing in fabs and packaging as if the buildout will last another four or five years.

Watch theCUBE’s full sit-down.

9. Power systems race to match AI’s one-year hardware cycle.

Power infrastructure has historically moved on cycles of six to 10 years while AI hardware turns over every year or so. Melton Chang, president of the Power Systems Division at Schneider Electric SE, described that mismatch as the industry’s core challenge. AI loads can swing from 30% to 150% in milliseconds. Those swings threaten voltage and frequency stability and cause resonance that shortens the life of on-site generators. Schneider uses digital twins, simulation and real-time data to anticipate those swings.

Don’t miss the full segment on theCUBE.

10. Self-verifying AI agents speed chip design without sacrificing trust.

Custom silicon can no longer take 18 to 24 months when models change every three to six months, yet designers still need deterministic, trusted results from electronic design automation tools, according to Ankur Gupta, head of EDA IC and systems at Siemens Digital Industries Software. Siemens runs its design engines on Nvidia Corp. GPUs for 10- to 12-fold speedups. It also uses AI agents to automate the setup and debug phases of design.

Check out theCUBE’s complete interview.

11. d-Matrix scales memory-centric XPUs from chip to rack for inference.

d-Matrix Corp. has moved rapidly from accelerator cards to full rack-scale systems built around its Corsair XPU, according to Sid Sheth, founder and CEO of d-Matrix. By stacking memory directly on top of the processor — rather than placing it beside compute on an interposer — the architecture opens up both capacity and bandwidth, two of the core bottlenecks in AI inference. The company accelerated that shift by integrating into the Nvidia AI factory ecosystem to avoid rebuilding power, cooling, networking and software from scratch.

See the full interview on theCUBE.

Photo: SiliconANGLE