As Nvidia Corp.’s GPU Technology Conference kicked off Monday, one of the more interesting product announcements was the introduction of data processing units, also known as DPUs. 

Nvidia is widely known as the market leader in graphics processing units and recently announced plans to acquire Arm Ltd. to move into central processing units or CPUs. Now, it’s pitching something called DOCA or data-center-on-a-chip, an architecture that improves the performance of data center networking, security and storage.

A simple way of thinking about a DPU is that it’s a network interface card or NIC that has been beefed up to perform many of the tasks that the CPU on the server did — a SmartNIC, as VMware Inc. called it in an announcement with Nvidia last week. By offloading these functions, the server can handle more bare metal, virtual or containerized workloads. With traditional architectures, the NIC handled basic networking functions while the CPU on the server performed tasks such as infrastructure management and software-defined storage, security and networking.

With a DPU, Nvidia has taken a Mellanox NIC and loaded it up with silicon and software to boost server performance. At GTC, Nvidia announced two flavors of DPUs:

• BlueField-2 DPU is a Mellanox ConnectX-6 DX SmartNIC with ARM cores. The “card” (it’s actually a server on a board), is fully programmable and has data transfer rates of 200 gigabytes per second and accelerates network, security and storage functions such as segmentation, root trust, key management, RDMA/RoCE, block storage and compression.
• BlueField-2X is the Bluefield-2 but with Nvidia’s Ampere GPUs on it for artificial intelligence purposes.

During a pre-event call with analysts, Manuvir Das, Nvidia’s head of enterprise computing, made a bold prediction that “every server shipped into a data center” will have a DPU in it. Although I think that statement is a bit aggressive, I do think DPUs will gain significant traction.

The base BlueField is ideally suited for servers with high-density virtual machines or containers. The BlueField-2X is the more interesting product because it can give servers that perform AI tasks a significant boost. One of the low-hanging fruit use cases is real-time security analytics where AI is used to identify anomalous traffic that could indicate a breach. With AI and DPUs, that task can be done at line rate, limiting the impact of a security incident.

The concept of a NIC with extra processing capabilities isn’t really new, but Nvidia’s timing is good. For those who have been around the industry for a while might remember the 3Com NIC with an embedded firewall or the Chelsio and Intel TOE (TCP offload engine) that had improved network processing on the card. Those were early versions of the DPU, and they failed miserably. What’s different this time?

Back in the day, I used to think the TOEs and other products like it were a good idea but were somewhat of a solution looking for a problem. A decade ago, the average server utilization was under 40%, so there wasn’t much reason to offload processing from a server when the server wasn’t even close to being maxed out. 

Today, that’s not the case as virtualization and containers have pushed servers to the limit. In fact, many businesses I’ve talked to are pushing server utilization into the 90% range before upgrading. Replacing a server can be highly disruptive and can cause unnecessary downtime so companies push these out as long as they can. Dropping one of these DPUs in can extend the life of the server for months or even years.

The other factor driving the need for faster computing, driven by AI. During his GTC keynote, Nvidia Chief Executive Jensen Huang noted that AI is “vastly more parallel, and thousands to millions of times more compute-intensive. The method of developing the software is different. The computing infrastructure is different. The tools are different. The software runs differently, and even the method of deployment is different. AI requires a whole reinvention of computing stack.” The DPU enables the compute stack to change by shifting resources out of the server, letting the server do more things it was designed to do.

The factor that makes me believe DPUs are here to stay is ecosystem support. Last week at VMWorld, VMware announced it would use the DPUs in its Cloud Foundation. In addition to VMware, Red Hat, Canonical and Check Point Software have announced support for the DPUs. Also, a number of server manufacturers, including Dell, Asus, Lenovo and Supermicro will be integrating the Nvidia DPU into their systems. 

Any technology that goes into a data center must have a strong ecosystem as it needs to work with so many other components. The original wave of similar cards had virtually no third-party support, but the Nvidia BlueField seems to have plenty.

Information technology pros in companies large and small are all concerned about squeezing more out of the dollar already spent. Servers today are having to perform workloads that are far more processing-intensive than just a few years ago. DPUs can give those same servers a badly needed kick in performance, enabling businesses to move into the AI era.

Zeus Kerravala is a principal analyst at ZK Research, a division of Kerravala Consulting. He wrote this article for SiliconANGLE.

Image: Nvidia