Application monitoring startup Groundcover Ltd. today announced the launch of a new observability tool that offers code-free, real-time visibility into artificial intelligence applications powered by large-language models.

The new tool is customized for large language model observability and covers applications such as AI agents, which go beyond general chatbots by automating tasks on behalf of humans, as well as retrieval-augmented generation pipelines that expand the capabilities of LLMs with custom data, and “tool-augmented workflows.” As an added benefit, it ensures all of the data generated by these apps remains safely ensconced inside the customer’s computing environment, enhancing security.

Groundcover, which bagged $35 million in funding in April, is the creator of the industry’s first fully-managed “bring your own cloud” observability platform. It has the advantage of letting customers store their observability data in their preferred location – either on-premises or in the cloud of their choice.

Like other observability providers, Groundcover is focused on monitoring applications and infrastructure performance. It does this by collecting data on error messages, performance metrics and user activity and analyzing it in real time to help companies spot unusual behavior that might indicate a problem. When it finds something, it attempts to discover what’s causing that behavior, so it can be fixed – either automatically or by humans if necessary.

Groundcover says its observability platform is differentiated by its flexibility, enabling companies to store their data in a location of their choosing. That allows them to meet whatever security and privacy requirements they’re obliged to adhere to. It offers unlimited data capacity and simple, predictable pricing, employing extended Berkeley Packet Filter or eBPF technology to gather information such as logs, traces and metrics directly from the Linux kernel.

EBPF is another differentiator. It’s an emerging observability protocol that stands out from legacy approaches thanks to its ability to execute programs that can exfiltrate monitoring data within the Linux kernel without altering its source code in any way. It helps to ensure observability operations remain isolated and inobtrusive, while simplifying the export of data to third-party platforms.

With its new LLM Observability offering, Groundcover says it can capture every interaction with LLM providers such as OpenAI and Anthropic PBC, including metrics such as prompts, completions, latency, token usage, errors and reasoning paths. That makes it easier for teams to track the performance of their apps and debug any problems and also helps with cost optimization, the company said.

For instance, it can help teams to identify why some outputs fail, where context shifts across turns, why “hallucinations” occur, why latency is increasing, and how AI agents come to make their decisions. The data can be stored wherever the customer prefers.

Groundcover said specialized observability is necessary for LLMs because they’re built and operate in very different ways from traditional applications. Almost 70% of companies now use LLM-powered applications, but lack the proper tools to track and monitor their performance. Moreover, LLM applications are rapidly increasing in complexity, evolving from simple, prompt-based chatbots to multistep agents that are far harder to keep pace with and understand.

Groundcover Vice President of Product Orr Benjamin said LLM-based applications simply don’t fit with traditional observability models.

“By using eBPF, we deliver complete insight into AI pipelines with zero instrumentation and zero data egress,” he explained. “Teams can understand exactly how their AI apps behave in production without changing their code or exposing sensitive information.”

Image: SiliconANGLE/Microsoft Designer