Data is at the core of today’s digital revolution, but its refinement processes have become the real points of differentiation for enterprises.

As companies convert raw data into actionable insights, artificial intelligence signals a burgeoning ecosystem of management models. It’s a startup haven of sorts, with cloud-native newcomers attributing success to the scalability of modern computing.

The current wave of AI startup investment has received $16.5 billion in capital over the three months of the fourth quarter 2020, according to financial analysis from PitchBook Data Inc. The boom is continuing, with the latest figures from International Data Corp. showing a 16.4% year over year growth for the overall AI and machine learning market in 2021, with the AI software platforms market coming in strongest at a predicted compound annual growth rate of 32.7% between 2020 and 2024.

AI has already become an essential part of many enterprise technology solutions. The McKinsey & Company Inc. 2020 “The State of AI in 2020″ survey found that 50% of companies had incorporated AI into at least one process, with marketing and sales the most common functions.

O’Reilly’s “AI Adoption in the Enterprise 2021” report shows this adoption very evenly spread between sectors, with highest adoption in the technology, financial, healthcare and life sciences sectors.

Looking into the reasons behind the funding of AI startups and how cloud-scale computing supports new models with industry-wide implications, theCUBE, SiliconANGLE Media’s video studio, reviewed recent exclusives with AI startups and reached out to industry experts for their opinions.

Before we dive in, it’s important to clarify the term AI. While many interchange the terms AI, machine learning and deep learning, AI encompasses both the fields of machine learning and deep learning. It’s this more inclusive term we reference in this article. Wikibon Inc.’s chief analyst, David Vellante, gives a comprehensive explanation of the differences in his Breaking Analysis: Moore’s Law Is Not Dead & AI Is Ready to Explode.

Incubating the big data egg

No matter how human workers may welcome or fear its dominance in the workplace, AI is more than a futuristic way to reduce repetitive work in favor of more creative tasks. Where the technology truly becomes mission-critical for enterprises is its ability to automate aspects of data management — at cloud scale.

“AI-enabled automation is the next wave for data management tools,” said John Santaferraro, research director at data management research and industry analysis firm Enterprise Management Associates Inc. He sees the use of AI and ML in business intelligence becoming a differentiator, potentially creating barriers to entry that will change “the face of how information is accessed, delivered and acted upon.”

In this regard, it’s a bit of a chicken-and-egg scenario: The data boom needs intelligent solutions, while such solutions require data to build upon.

“Companies realize that just having ML algorithms is not enough,” Ajay Khanna, chief marketing officer for external data platform provider Explorium Ltd. told theCUBE’s Natalie Erlich in a recent interview. “That is not a competitive advantage — everybody has the same algorithms.”

Explorium is among the startups benefitting from AI’s funding frenzy. On May 17 the company closed $75 million in additional investments, doubling its previous two rounds for a total of $127 million. The trend is a sign of the growing appetite for external data and adoption of AI and machine learning, according to Khanna, and raises new questions of access for the burgeoning industry.

The wider the variety of data a company has access to, the more advantage it has, Khanna told theCUBE. “What we are seeing is an increase in trust and getting access to these external data sources as a competitive advantage, and then having that access easily and being able to easily use that external data into your analytics, into your ML models,” he said.

AI takes self-improvement lessons

The British comedy “Fawlty Towers” has a long-running gag on the miscommunications between Spanish waiter Manuel and hotel owner Basil Fawlty. It’s a good illustration of the problems that occur when AI attempts to gain understanding from poor or biased data.

AI that trains itself on poor data provides results, but they are based on inaccurate or insufficient information. So the results, as with Basil and Manuel, are chaotic and confused. Unfortunately, the analogy ends there. Although the implications of misunderstanding in comedy lead only to laughter, inaccurate judgments made by AI algorithms can have long-term and serious implications for people discriminated against on the basis of outdated or inaccurate data sets.

Addressing this means making AI more self-aware so that it can identify potentially biased statistics and ensure it is trained on accurate and well-curated data. One company working on this is Fiddler Labs Inc., whose co-founder and Chief Executive Krishna Gade, recently spoke with theCUBE host John Walls.

“You need to understand the AI system, you need a way to probe it, to interrogate it, to understand how the system is making predictions, how is it being influenced by various inputs you’re supplying to the system,” Gade said, explaining how Fiddler’s algorithms work to provide “explainable AI” by examining and exposing bias before the AI is entrusted with real-world applications. “When [AI] is being used for criminal justice scenarios or when it’s being used for clinical diagnosis scenarios, being able to ensure that the system is making unbiased decisions is very, very important.”

Fiddler and Explorium, along with fellow AI startups ChaosSearch Inc., Coralogix Ltd. and Ahana Cloud Inc. are taking part in the upcoming AWS Startup Showcase: The Next Big Thing in AI, Security & Life Sciences. AI is one of three tracks at the event, which aims to provide a forum for next-generation solutions in cloud-scale data that have built out solutions on AWS infrastructure. (* Disclosure: TheCUBE is a paid media partner for the event. Neither Amazon Web Services Inc., the sponsor for theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

“A big bet” is how ChaosSearch described the decision to base its software-as-a-service data lake platform on AWS S3 storage. It was 2015, and the wager’s premise was that “customers would want to analyze such large data sets that the only place to process queries on such a vast amount of data would be on cloud object storage,” Thomas Hazel, founder and chief science and technology officer for ChaosSearch, told theCUBE.

That such a decision could be a risk seems almost laughable in the light of today’s reliance on cloud-scale data. In less than two decades, data has gone from a connectivity byproduct to mission-critical.

But the increase in data alone could not have resulted in the data-driven society we see today. It is advances in AI that animate data deposits and connect the dots, revealing the wisdom hidden within the mess of miscellany.

Image: Shutterstock