Just as oil propelled Standard Oil Co. Inc. to a position of dominant industrial power in the late 1800s, data is doing the same for a number of technology firms today. Half of consumer online spending in the U.S. is controlled by Amazon, a company that relies extensively on mining data so it knows what you want before your buy it.  Nearly all revenue for digital advertising in America flows to two companies: Facebook and Google.

The list goes on, but one constant is clear. Just as oil spawned the growth of many industries, data is reshaping the technology stack. From robotics (think autonomous cars) to the entire field of data science, a new era of innovation is underway. This is forcing any enterprise that handles data to innovate as well, and there may be no better example of this transformation than what is taking place in the storage layer, where the application of machine learning and artificial intelligence is refining data management in new ways.

“Pick your domain; there’s going to be innovation across a lot of fronts, across a lot of traditional vertical industries that is all going to be about data and driven by data. And there has to be an infrastructure layer that actually enables it,” said Mike Cordano (pictured), president and chief operating officer at Western Digital Corp.

What that infrastructure will look like over the next two to three years was the focus of discussion earlier this month, as Western Digital unveiled new storage technology that is designed to handle the explosion of data across the enterprise stack. Cordano stopped by theCUBE, SiliconANGLE’s mobile livestreaming studio, and spoke with host Jeff Frick (@JeffFrick) at the Western Digital Presents: Innovating to Fuel the Next Decade of Big Data event in San Jose, California.

Frick also separately interviewed Janet George, fellow and chief data officer at Western Digital, and Brendan Collins, vice president of product marketing devices, at Western Digital. They discussed key elements of the company’s new storage technology, enterprise trends that are shaping data management tools, the important role of AI and machine learning in bringing compute to data and potential innovations in the future. (* Disclosure below.)

Watch the complete video interview with Mike Cordano below:

New approach for data explosion

The challenge facing Western Digital and other companies in the storage space is how to innovate scaled growth while allowing multiple software applications to access data and transform it into something useful. Storage industry veterans could see that perpendicular magnetic recording technology, or PMR, introduced a decade or so ago, was beginning to run out of steam in the face of exponential increases in data demand.

So Western Digital embarked on a journey to develop new storage technology that would be faster, dramatically expandable and cost-efficient. Investments in two different technologies — heat-assisted magnetic recording, or HAMR, and microwave-assisted magnetic recording, or MAMR — yielded a verdict: MAMR.

Unveiled by Western Digital at its headquarters event this month, MAMR uses an oscillator-generated microwave field for greater reliability and more capacity in handling data storage needs. Hard drive capacity can be increased to at least 40 terabytes, with 2.5 million hours of reliability.

“When we go to MAMR, everything is seamless, everything is transparent, and it’s great,” said Collins, who cited an “order of magnitude difference” in the heat level between HAMR and MAMR.

Watch the complete video interview with Brendan Collins below:

Western Digital’s announcement is also significant because it heralds the evolving shift from bringing data to compute, to moving compute to data, a change made necessary because datasets are rapidly becoming too large to transfer in any substantive way.

“We want to drive this architecture that brings compute to the storage and enables some things that just can’t be done today,” Cordano said.

Applying AI and machine learning

The data science team at Western Digital sees the development of MAMR, with its speed and capacity, as an opportunity to apply artificial intelligence and machine learning tools within the constructs of the data itself. This is becoming increasingly important because many businesses are moving toward an “internet of things”-based model where data is captured and analytics are applied in real-time rather than stored indefinitely for later use. Retail is a prime use case for this, as merchants yearn for real-time data to reduce abandoned transactions.

Real-time data streaming architectures, such as Lambda or Kappa, are the kinds of tools receiving interest by the data scientists at Western Digital, according to George, as the company develops new storage technology. MAMR is seen by Western Digital as instrumental in data science and management.

“It’s a breakthrough announcement both in the field of machine learning and AI,” George explained. “We can capture data at the raw form, and we can do the learning at raw form without human interference.”

Rise of genetic algorithms

Learning from raw data opens a world of possibilities for the enterprise to refine models in such a way that can make decisions rapidly at the edge. The research team at Western Digital is looking toward the future where genetic algorithms can be employed in AI models, quickly discarding useless solutions and acting swiftly on the one that works.

“Right now, I’m thinking about genetic machine learning algorithms,” George said. “With genetic algorithms, we can quickly assign a cost function, and we can kill all that don’t fit, arriving to the fastest, newest technology node.”

This vision of machine learning and AI applied in the context of the storage world highlights where the read/write industry is headed. Moving beyond devices, storage is transitioning to a place where it can influence technology across the enterprise.

“We’re going to continue to innovate above [devices] into platforms and systems,” Cordano said. “There’s a lot of technology innovation that we can do which enhances the offering that we can bring to market when we control the entire technology stack.”

Watch the complete video interview with Janet George below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of the Western Digital Presents: Innovating to Fuel the Next Decade of Big Data event. (* Disclosure: TheCUBE is a paid media partner for the Western Digital Presents: Innovating to Fuel the Next Decade of Big Data event. Neither Western Digital Corp., the event sponsor, nor other sponsors have editorial influence on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE