Building artificial intelligence agents that can interact with each other reliably across services presents technical challenges that have never been tackled before. According to one AI researcher, that will limit their use to a narrow set of business processes for the next few years.

Niloufar Salehi (pictured below), an assistant professor at the University of California at Berkeley, said traditional data processing systems are built to deliver predictable results. But nothing is predictable about machine learning. The same algorithm may produce entirely different results depending on context.

Building trustworthy agentic AI systems involves solving problems that have never before existed. “The way databases work is that you have a data schema and you know what sort of things are there, like accounts and deals,” she said. “What agents are doing is much more unstructured and adapts over time. It’s a completely different way of thinking about data structure and schemas, and we don’t have the right way to build that yet.”

Salefi: Building shared memory will be a huge step toward making AI agents more reliable. Photo: Niloufar Salehi

Random paths

The mathematical term is stochastic, a reference to systems or processes that are inherently random or involve uncertainty. One well-known example of a stochastic algorithm is the Facebook newsfeed, which varies based on the preferences of the person reading it.

“The moment systems become stochastic, you no longer open your Salesforce application and see the same thing you saw yesterday,” she said. “That means reliability is a huge issue.”

It’s also an extremely difficult problem because many variables are involved. In addition to the billions of parameters used to train machine learning models, agentic AI involves interactions among multiple agents, each operating on stochastic algorithms.

Solving the problem requires creating persistent, shared memory that agents can tap to learn from past actions, Salehi said. Current AI models approach each situation as if it’s entirely new, even if the same scenario has occurred repeatedly in the past.

“If an agent has a problem, it tries multiple ways to solve it,” she said. “If it comes across the same problem tomorrow, it goes through that process all over again. Building out the shared memory that lets these agents coordinate is an extremely difficult feat of engineering.”

Researchers are working to extend the context length or maximum amount of short-term memory a model can employ to make actions more predictable and repeatable. Salehi pointed to MemGPT, a research project attempting to build a memory manager for large language models, as a potential solution. “Giving agents a longer-term memory they can share will be a big unlock,” she said.

Language barrier

Salehi became interested in human-computer interaction shortly after receiving her Ph.D. in computer science from Stanford University in 2018. Working with a team of physicians using machine translation in emergency rooms to communicate with non-English-speaking patients, she was surprised to discover that error rates were alarmingly high.

“Chinese translation had a 20% error rate for commonly used sentences, such as emergency room discharge instructions,” she said. “It was 8% for errors that caused potentially significant harm, such as telling someone to continue kidney medication they were supposed to stop. That’s what got me thinking about how to make agentic systems more reliable.”

She is optimistic that agents that can coordinate across services and companies will be practical within the next five years, but they probably won’t be used in the ways people expect. The reasons have little to do with technology.

For example, training agents to schedule meetings is difficult because people’s time is so personal and many of us keep mental notes about managing our schedules. “It’s hard to automate when human factors aren’t known or not written anywhere,” she said.

Better candidates for automation are well-known processes that could benefit from better coordination. “I think we’re pretty close to making those systems happen,” she said. “The next five to 10 years will be all about solving extremely difficult engineering problems to get agents to work together and reliably.”

Image: SiliconANGLE/Microsoft Designer