It’s been a year of hype and innovation, especially when it comes to artificial intelligence. In what has represented the biggest story of the year, it’s now difficult to imagine a world without AI being infused into everything.

But with the advent of AI, so too have companies been forced to grapple with how they must handle the future of data. Exponential growth has led to a huge need for enterprise data storage, with next-generation storage solutions projected to be valued at over $150 billion by 2032. Meanwhile, given the fact that data platforms today see a massive total addressable market estimated in the tens of billions of dollars, every vendor with a data-related offering is vying for a piece of the market.

“A data platform is a comprehensive solution that not only stores data, but also offers application programing interfaces for data ingestion, methods for data processing addresses data governance — that’s a really important one now — and ensures scalability and performance because you want to use it at cloud scale no matter where you are,” according to theCUBE industry analyst Rob Strechay. “It can be delivered as a build-your-own platform or as a service.”

Still, there are various solutions on the market today when it comes to data management, and not every approach is comparable. In this deep dive, theCUBE steps back to analyze one company’s offerings — that of Vast Data Inc.

This feature is part of SiliconANGLE Media’s ongoing series with theCUBE exploring the latest developments in the data storage and AI market.

Data storage transforms

As the world of data storage transforms and AI is factored in, organizations have no choice but to respond. For Vast, that has involved an evolution beyond being just a storage company.

“Our observation is that people are now starting to architect strategies to collect tons more data because, for the first time ever, they can actually go and process it,” said Jeff Denworth, co-founder of Vast Data, in June, on the company’s quest to redefine data infrastructure. “Big-time investors in the field and successful companies are realizing the value of data hoarding. This shift from real-time data to historical data is a trend that is not hard to achieve anymore, and democratizing access to this valuable resource is the true game-changer.”

Since that interview in June, more has been revealed about how Vast Data is looking to change the game by revolutionizing data management amid the AI era. Here’s what the future looks like as Vast unveils its vision and continues its quest to evolve from its beginnings as a storage company.

Long before the current AI boom, Vast was working on AI — the company launched its data platform in 2019 but began working on a storage system designed to support the AI of the future in 2016. Even before the company started building the storage system, the inspiration was always AI, according to Renen Hallak, founder and chief executive officer of Vast.

“It was all about how do we give fast access to a lot of information to enable these new algorithms and these new applications, and this is one more step that we make in that direction,” Hallak said.

Moving forward, Vast sees itself partnering with and providing underlying infrastructure for various leaders, such as Nvidia Corp., and becoming an enabler for clouds. In addition, because the platform analyzes huge amounts of data in real time, organizations can make data-driven decisions previously thought unthinkable, according to Hallak.

“As we progress, what we see more and more is clouds — and not just the top three clouds that everybody knows and talks about — we see more and more specialized AI clouds, and we see the big clouds spilling over sometimes into these more regional clouds in order to provide services to their customers faster,” he said. “We are enabling those clouds.”

The company’s ambitious plans to power a new generation of AI models — those that don’t just parrot simple responses — came into view in August when it announced its new global data infrastructure for AI called the Vast Data Platform. The platform aims to unify data storage, database and virtualized compute engine services in a scalable system.

The Vast Data Platform comprises four key components. It begins with Vast DataStore, a scalable storage architecture that eliminates the need for tiering. The second innovation is Vast DataBase, which combines the capabilities of a traditional database, data warehouse and data lake into one platform.

There’s also Vast DataEngine, which the company says serves as a global function execution engine that consolidates geographically distributed on-premises data centers and cloud regions into a single computational framework. Finally, there’s Vast DataSpace, a global namespace that permits any location to store, retrieve and access information from any other location.

The potential of data platforms

What Vast ’s data platform does is go far beyond conventional storage protocols and enables a new way of thinking about data, according to theCUBE industry analysts. It’s a reimagined data platform, allowing developers to utilize data wherever it is located, which is crucial in the era of AI.

“You have to rethink the platform,” said analyst George Gilbert. “If data becomes the platform, then you want to be able to program or compute on it or learn on it independently of where it is … you have to rethink all that so that you can bring together files, objects, tables, unstructured and structured data, streaming data coming in transactionally, turning into analytic data.”

In announcing its Vast Data Platform, the company also announced the integration of its new offering with Nvidia’s DGX AI supercomputing platform. It emerged out of the desires of companies, according to Tony Paikeday, senior director of product marketing, AI systems at Nvidia.

“The work that Vast does mirrors our own design philosophy, and how we look at the full AI developer stack and what’s required to give these teams productive work capacity that can scale linear with the size of system that’s being used,” he said. “Our customers have asked us time and again to simplify this, give us better performance with scale, better utilization, better productive work capacity for data scientists and developers. That’s what this coming together is all about.”

Vast DataSpace takes on significance here, given the fact that everyone wants the flexibility of a hybrid model of being able to compute where they choose, according to John Mao, vice president of global business development at Vast. DataSpace allows the company to stretch its namespace in a hybrid manner to wherever the customer chooses.

In practice, what we’ve been talking about here comes into play in the world of complex, fast-moving global financial markets, where data is crucial. Aquatic Capital Management (Aquatic Group LLC) manages systematic investment strategies and combines a development platform, along with sophisticated research and disciplined risk management. The company turned to Vast given its capabilities.

“As Aquatic’s businesses evolved, as they’re building these prediction systems, the more different multi-variant data sources that they can assemble, the more accurate that their models can become,” Denworth said.

2023 has seen a rapid pace when it comes to advances in AI, and the months ahead promise to be no different. Companies will need to respond swiftly when it comes to their responses to data management.

For Vast, the evolution beyond being just a storage company promises the reimagination of data platforms, en route to a future where data becomes the API to the real world. It’s the culmination of a long journey, one that theCUBE industry analyst Dave Vellante has been watching closely.

“Years ago, when I saw Vast launch, I was like, OK, hey, another storage company. That’s kind of cool,” he said. “These guys are bringing new thinking to the market in a way that [is] starting to tweak our idea of what that future data platform is going to look like.”

Image: Created with the assistance of DALL·E