Aquant Inc., the provider of an artificial intelligence platform for service professionals, today introduced “retrieval-augmented conversation,” a new way for large language models to retrieve and present information to users that allows them natively to act as a guided domain expert instead of receiving and presenting knowledge as a single all-in-one answer.

AI large language models need external knowledge to update their general training when people pose them questions, this is often done with retrieval-augmented generation, or RAG. This is a type of workflow that retrieves relevant information from external sources, improves accuracy and reduces hallucinations.

These workflows can search hundreds of thousands of documents and retrieve the most context-relevant knowledge to present it to the LLM before it provides an answer all at once. But what if there’s a better way to get the information to the AI and the user?

RAC can be thought of as an expert technician that is aware of its capacity and capabilities, Indresh Satyanarayana (pictured), vice president of product technology and labs, and the father of retrieval-augmented conversation, told SiliconANGLE in an interview. It helps the AI look at a user’s question and ask follow-up questions to fill knowledge gaps and generate tailored solutions.

“[RAG] is agnostic to domain, indifferent to the user’s real goal, and blind to whether the answer is actually useful in context,” he said. “RAC replaces this ask-answer paradigm with a problem-resolution mindset. It engages in a dialogue.”

According to Satyanarayana, unlike RAG, RAC introduces dynamic turn-taking, much more like a human conversation with an expert in the field in question. It’s designed to provide “bite-sized actions,” which he says avoids cognitive overload for the user.

For example, when used to assist a technician performing a repair on an engine in a vehicle, RAC would go step-by-step through the engine type, parts and symptoms. In contrast, a RAG system would produce a complete document for an AI application, which would either display all at once or to produce a conversational interface a developer would have to parse.

Not only that, RAC can incorporate even more data points into its conversational context depending on the persona developers want to build into their AI app.

“It retrieves not only manuals but transactional data, job history, parts catalogs, internet of things readings, and key performance indicator targets, then reasons over that richer context to recommend the action that best balances cost, risk and time,” said Satyanarayana.

Satyanarayana stated that retrieval-augmented conversation does not fundamentally replace RAG; it will still perform the retrieval-augmented portion. Documents still need to be searched and retrieved, and this aspect will guide the conversation for the user.

On the other end, developers will have a chance to decide how “chatty” their app acts. It can do one-to-one questions solving one ambiguity at a time and then provide a final answer after they have all been resolved.

Alternatively, they could develop an app that can resolve multiple questions at once, the way some people can hold multiple threads of conversation at the same time — like many open tabs in Chrome while researching — before resolving the problem.

“There is a place where some customers will still choose the single document RAG response,” Satyanarayana said. “But largely our goal is to redo the conversation layer, move away from generation and really focus on conversation.”

Satyanarayana said he expects that over the next two years, RAC could become a primary mechanism for guided AI retrieval and guidance. He likened it to coaching, because it treats outcomes as goals and solutions instead of direct generations, collaborating with the user instead of dumping an answer on them.

Photo: Aquant