Last year, the discourse about high-performance computing trends focused on the increasing importance of software, as opposed to hardware, in steering innovation there.

Now, a new factor is at play. Since its recent explosion, artificial intelligence has begun to cut across several HPC operations within the enterprise.

“I think there is an intersection between HPC and AI,and it boils down to the fact that they’re using more and more intensive hardware nowadays,” said Peter Nguyen (pictured, right), senior product manager at Dell Technologies Inc. “AI used to be a lot of research; it would be one or two nodes, one or two GPUs and a lot of testing. But now, with all the things going on with OpenAI, and with those things being near completion, you can start throwing some horsepower at it.”

Nguyen and John Lockman (left), distinguished engineer, AI and high-performance computing at Dell Technologies, spoke with theCUBE industry analyst Dave Vellante at the ISC High Performance event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the AI/HPC confluence, with particular respect to the work being done with the Omnia project. (* Disclosure below.)

The Omnia project in detail

By enabling capabilities such as high-performance data analytics, HPC has changed the way organizations work, particularly on data-intensive use cases, such as meteorology, life sciences and engineering. But a problem has since cropped up: Infrastructure cluster sprawl has overcome data centers, consuming disproportionate amounts of critical IT resources. To combat these inordinacies, Dell Technologies conceived the open-source Omnia project.

“What’s amazing is we’ve completely open-sourced the tooling,” Lockman explained. “A set of Ansible playbooks and a group of developers here at Dell are continuing to develop new features for it. The heart of the project is to enable everybody to build a simple Slurm-based HPC simulation and modeling cluster or get started in the Kubernetes space if you’re doing AI.”

Omnia’s chief purpose is to simplify the deployment of consolidated infrastructure through open-source software, servicing domains such as HPC, AI and data analytics. The software stack is designed to handle a spectrum of processes all the way from firmware updates to operating system provisioning, Lockman added.

“The Omnia project will handle your firmware updates, BIOS configurations and RAID configurations,” he said. “But it will also provision operating systems to your choosing and set up a very large variety of Dell and Nvidia switches, including InfiniBand and high-speed Ethernet. It then takes that entire pile and turns it into a logical cluster.”

While Kubernetes is an open-source technology, a lot of it is packaged into fast-changing microservices. Thus, Omnia provides a prescription of the best tools enterprises can harness to fit a particular use case, according to Nguyen.

“Dell has taken kind of a different approach where we’ve created the open-source stack and given you the ability change out bits and pieces here and there, but it’s still open source,” he said. “Where Omnia comes in is that it’s this big wrapper that goes around everything.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the ISC High Performance event:

(* Disclosure: Dell Technologies Inc. sponsored this segment of theCUBE. Neither Dell Technologies nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE