Artificial intelligence performance startup ArthurAI Inc., said today it’s adding a powerful new tool to its suite of AI monitoring services.

The new Recommender System Support tool is designed to help companies ensure their AI systems are safely deployed and managed well, in order to improve the accuracy of AI-powered recommendation engines.

The startup explains that AI recommendation systems are some of the most widely used AI tools today. They’re deployed by companies such as Netflix Inc. to recommend what users should watch next, as well as Spotify Inc., which uses AI to suggest songs people might like, and Amazon.com Inc., which does the same for product recommendations.

The use of recommender systems extends to other areas such as social media and email marketing. For instance, the posts that appear in someone’s Facebook feed are influenced by a recommender system, while the marketing emails people receive are often guided by AI. Such systems work by analyzing customer data to predict what people are interested in and generate tailored recommendations. When used correctly, these AI systems can improve customer satisfaction and increase revenue growth and engagement.

However, using AI recommender systems is not an easy task, as these models are often prone to performance issues that result from a phenomenon called “data drift.” This refers to the gradual change of the underlying dataset over time. Such changes can include the user’s behavior, content formats and population demographics. As AI recommendation systems suffer from data drift, their recommendations become less accurate and less relevant.

Arthur, which is focused on monitoring AI models and improving their performance, said it’s introducing the Recommender System Support tool in Arthur Scope, which is a service designed to detect and react to data drift in such systems. With its availability, Arthur says, it can ensure the continued relevance, accuracy and effectiveness of AI-powered recommendations. In other words, it’s a proactive monitoring tool that helps to maintain the integrity and performance of AI recommenders.

Arthur listed a number of interesting capabilities within its new tool, including a comprehensive dashboard that provides an overview of the health of each model, with metrics such as Precision@k, Recall@k, MAP@k, nDCG@k, MRR and Ranked List AUC. It can also generate advanced queries, filters and data visualizations to help users better understand those metrics, while systematically measuring a system’s performance against real-world data, in order to gauge the extent of data drift.

There’s a configurable alert system to notify engineers and developers when a model’s data drift metrics deviate from a predefined threshold. that enables swift repairs to be initiated and segmentation tools to analyze the model’s performance for different user segments.

The new capability is the latest in a number of recent additions to Arthur’s platform. In August, it announced an open-source tool called Arthur Bench to help companies choose the correct generative AI model based on their data and proposed workloads. Then in December it followed up with the launch of Arthur Chat, a retrieval-augmented generation platform that enables existing chatbots such as ChatGPT to be enhanced with a company’s own datasets to build more accurate and specialized AI models.

Arthur co-founder and Chief Executive Adam Wenchel said running an AI recommender system is similar in some ways to driving a car. The problem, he said, is that many companies do so without a temperature gauge or check engine light, which makes it difficult to maintain top performance.

“With Arthur’s new Recommender System Support, enterprises can remain confident that their recommender systems are constantly in check and will consistently deliver high-quality, personalized user experiences, ultimately protecting revenue streams and customer trust,” Wenchel said.

Image: Arthur