Responding to growing demands for improved governance in enterprise artificial intelligence, application programming interface specialist Kong Inc. has unveiled an enhanced version of its Kong AI Gateway.

AI Gateway functions as a retrieval engine for large language models and provides the infrastructure for managing the entire AI governance lifecycle. The 3.10 version takes the technology a step further by fully automating retrieval-augmented generation, known as RAG, a method for improving AI results with contextual data.

“We don’t ask the developers to go and build those RAG pipelines anymore,” said Marco Palladino (pictured), co-founder and chief technology officer of Kong. “The Gateway will query the vector database, add relevant data and then send it to the model. In other words, RAG is enabled by default, and that matters because the hallucinations are perhaps the biggest drawback of using AI. We want to reduce hallucinations as much as we can to improve the response quality of our applications.”

Palladino spoke with theCUBE’s John Furrier about today’s product launch, discussing why AI Gateway could become the world’s most comprehensive platform for managing AI and APIs and the importance of AI governance. (* Disclosure below.)

How AI Gateway patches up LLMs’ security issues

Kong recently raised $175 million in funding for its suite of API services. The company has brought its experience in APIs to AI, automating RAG while also providing users with greater security for their vector databases. The new version of AI Gateway sanitizes Personally Identifiable Information (PII) data from the model, ensuring regulatory compliance.

“AI cannot keep a secret,” Palladino said. “This new version … allows us to apply a blanket policy to remove all PII data that’s being sent to a model across every team and every application, every agent that’s using AI. And at the same time, we will reinsert that PII information on the way back so that the end user experience is not being compromised.”

Vector databases have been critical to the new stage of AI applications, as well as agentic AI, which has become the hot new thing in enterprise this year. Agents essentially function as virtual employees, according to Palladino, and are the new frontier for automating business processes.

“We’re entering a new era of AI, the era of custom agents,” Palladino explained. “One of our customers, for example, is a top Fortune 500 pharmaceutical organization. They’re running our AI Gateway in production. They were able to reduce by 50% the time it takes to document a new drug when it enters the market to document the clinical trials when a new drug is preparing to enter the market.”

In addition to improving model accuracy with automated RAG and providing greater security across the board, Kong is bringing its API technology to the table. Approximately 80% of the world’s internet traffic is APIs, and that percentage is only going to grow, according to Palladino. The proliferation of APIs has caused issues for organizations that have not been able to centralize their application access.

“The ability of an organization to move fast, innovate fast and build new applications, it’s [predicated] on the ability to know what APIs they have,” he said. “Without having a central repository for APIs … managing all of these technologies, the organization is operating in a siloed way. So, the customers that are partnering with us are the ones that understand the strategic importance of owning your connectivity layer because it is going to allow us to execute faster.”

Here’s theCUBE’s complete video interview with Kong’s Marco Palladino:

(* Disclosure: Kong sponsored this segment of theCUBE. Neither Kong nor other sponsors have editorial control over content on theCUBE or SiliconANGLE) 

Photo: SiliconANGLE