Scaling artificial intelligence initiatives successfully involves developing appropriate data strategies, choosing the right tools and putting together an effective team. A KPMG report highlights that 65% of CEOs in the U.S. anticipate a significant impact of AI on their organizations in the next three to five years.

Explainability and security are crucial for using AI in regulated industries, as large language models are susceptible to prompt attacks and need to provide explanations for their decisions, according to Amr Awadallah (pictured, left), founder and chief executive officer of Vectara Inc. Developers struggle to achieve high accuracy in AI applications, but Vectera claims to solve this problem and offers high-quality results, although many are skeptical due to the abundance of tools for building prototypes.

“The problem is, many of them don’t believe us when we say that because there’s so many tools out there that make it simple to build a prototype. They think because they can build a prototype that they can make this work in production,” Awadallah said. “We warn them, we actually tell them, many of our customers. The way it works with them is we have the first meeting and they will say, we can do this ourselves. Go ahead try to do this yourselves. You’re going to come back and complain about many things.”

Awadallah and Yangqing Jia (right), founder and chief executive officer of Lepton AI Inc., spoke with theCUBE guest host Howie Xu, at the “Supercloud 6: AI Innovators” event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the challenge in AI applications to ensure accuracy and reliability while also finding cost-effective ways to run AI models at scale in regulated industries.

Balancing AI accuracy and security in regulation

Building an efficient retrieval augmented generation pipeline is difficult due to the need to fine-tune multiple models and ensure proper feedback and back propagation, according to Awadallah. There is an option of leveraging existing generative models and open-source resources for building a hallucination evaluation model that performs factual consistency checks.

Customers have the ability to balance and control the complexity of writing and deploying gen AI applications depending on their use case, Awadallah added.

“You have a trade off now. If you increase your bar on the accuracy of the result, then there might be less questions you’re able to answer. If you say I want 100% accuracy, then there will be now out of 100 questions you could answer maybe you’ll only answer 60%,” Awadallah said. “It depends on the use case. The good thing is we give our customers the ability to balance these things so they can control them depending on the use case they’re after.”

AI applications may not generate direct revenue, so it’s important to run AI models at scale economically and efficiently, allowing people to choose the best balance and launch models seamlessly on a cloud native platform, Jia explained.

“We built this cloud data platform allowing people to actually naturally evaluate and find the best balance. Of course. we have the fastest run time,” Jia said. “But people can actually choose among a spectrum to find out the best way to launch models. And then when they decide how to do so, we have the cloud-native platform to allow people to seamlessly scale things.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the “Supercloud 6: AI Innovators” event:

Photo: SiliconANGLE