There’s an unprecedented rate of attempted integration of generative artificial intelligence into enterprise operations, and plenty of gaps left to fill.

As organizations strive to streamline processes, elevate customer experiences and discover new horizons of innovation, the adoption of generative AI has opened up significant market opportunities. Amid this early excitement, however, challenges loom large on the path to realizing AI’s full potential in enterprise applications.

“The [AI] capability that we got the most excited about was the ability to follow instructions,” said Arjun Prakash (pictured, right), co-founder and chief executive officer of Distyl AI Inc. “When InstructGPT came out … there was this aha moment where we realized that this wasn’t just something that you could use to write letters or edit emails … it’s also something you can use to give instructions and actually carry out tasks that could have meaningful operational impact at enterprises.”

Prakash was joined by Jerry Liu (second from right), co-founder and CEO of LlamaIndex Inc., as they spoke with Howie Xu (left), theCUBE panel host, AI and data executive, at the “Supercloud 5: The Battle for AI Supremacy” event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed some of the gaps in gen AI and the state of fine-tuning in large language models.

Unlocking the potential of knowledge automation

In the rapidly evolving landscape, companies are increasingly looking to harness the power of AI to unlock new opportunities. However, there are major headwinds to overcome regarding generative AI’s enterprise deployment, according to Liu.

“A lot of people are trying to build LM applications these days, mostly to build prototypes, and they’re finding it hard to productionize,” he said. “There’s a few core issues. One is hallucination … it might not actually understand some of the outputs. The other piece is that a lot of people are building software systems around outlines, and they’re still figuring out the best practices for doing so.”

One of the key takeaways from this panel was the concept of retrieval augmented generation, or RAG, which involves combining a knowledge base with a language model, enabling more efficient and accurate information retrieval. RAG is an area where significant progress is being made, with growing enterprise adoption. However, it is not without its challenges, primarily because of the need to carefully handle parameters and data at various stages of the process, according to Liu.

“This is exactly where the point about adding more parameters to the system comes in, because the moment you build retrieval, in addition to the language model, you have to think about how does your retrieval system work,” he said. “How do you load in data … then how do you figure out how to retrieve it? A lot of failure points aren’t just due to the outline. It’s due to the selection of parameters at the earlier stages of the process.”

Navigating choices in generative AI

A key debate in the ever-evolving landscape of generative AI centers around the concept of fine-tuning, which is the process of modifying pre-trained language models for specific tasks or domains. Today it has gained plenty of attention and discourse within the AI community.

“What we have found is that the information loss from fine-tuning is larger than the accuracy gains from treating it as an information retrieval problem outside of the large language model itself,” Prakash said. “We have really good techniques of doing high reliability and predictable information retrieval. It’s going to be a work in progress to get fine-tuning to the point where you can trust it for information as well.”

However, organizations are often caught in a perpetual cycle of trying to match the capabilities of the next AI model iteration. This raises questions about the long-term viability of fine-tuning as AI models continue to advance rapidly.

Fine tuning may be a temporary solution as AI models become more powerful and costs decrease, according to Liu. The trajectory of AI capabilities suggests that the need for fine-tuning may diminish over time, aligning with an exponential growth curve in model capabilities.

“A lot of people are doing fine-tuning,” he said. “The reason is that, with a current set of models, it allows you to squeeze out better performance for less cost. So, there’s certain types of tasks that are very specialized and you can definitely fine-tune something much smaller and much cheaper versus just you using like GPT-4 and GPT-3.5.”

Watch the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the “Supercloud 5: The Battle for AI Supremacy” event:

Photo: SiliconANGLE