Vectara Inc., a startup that helps enterprises implement retrieval-augmented generation in their applications, has closed a $25 million early-stage funding round to support its growth efforts.

The company announced the investment today. It said that FPV Ventures and Race Capital jointly led the round, a Series A raise, with participation from a half-dozen other backers including Samsung Next. In conjunction with the funding announcement, Vectara debuted an internally developed large language optimized for retrieval-augmented generation use cases.

AI models can usually only answer questions using information from the dataset on which they were trained. Retrieval-augmented generation, or RAG, allows LLMs to draw on information from outside their training datasets. Increasing the amount of knowledge available to a language model enables it to generate higher-quality output.

Implementing RAG features in an application historically involved a significant amount of manual coding. Palo Alto, California-based Vectara offers a software platform that promises to ease the workflow. It provides many of the code components necessary to build RAG applications in a prepackaged format.

Before a RAG-powered AI model can start using the data in a document to generate answers, the data has to be turned into so-called embeddings. Those are mathematical structures that neural networks use to represent the information they process. Vectara’s platform provides features that make it easier to generate embeddings for RAG projects.

Another task that the software promises to simplify is the process of implementing a reranker. When a RAG-powered AI model draws on third-party data sources to generate an answer, some of the records at its disposal may be more relevant than others. A reranker is an algorithm that ensures AI models incorporate the most relevant information into the answers they generate.

Vectara also promises to ease several additional aspects of the RAG development workflow. The platform can, among other things, speed up the interface design work involved in building AI chatbots. Vectara says its software also lends itself to powering other services such as website search bars and employee-facing document analysis tools.

The latest addition to the platform’s feature set is an LLM called Mockingbird that debuted in conjuction with today’s funding announcement. According to Vectara, the model is specifically optimized for RAG use cases. The company says that Mockingbird significantly outperforms GPT-4 and Google LLC’s Gemini-1.5-Pro on Bert-F1, a benchmark used to evaluate how well RAG models turn data from external sources into prompt responses.

Vectara has equipped Mockingbird with a so-called structured output feature. The capability allows the LLM to package its prompt answers into JSON, a data format commonly used to move information between systems. The format will make it easier for developers to make output from Mockingbird available in their applications.

There are several open-source frameworks that can turn text generated by LLMs into JSON files. According to Vectara, it opted to integrate JSON support directly into Mockingbird instead of relying on an external framework because the former approach improves output quality. 

“We found during our evaluations that training an LLM to produce structured output and then constraining output produces much better results than using an LLM that has not specifically been trained to produce structured output,” Vectara engineers Suleman Kazi, Vivek Sourabh, Rogger Luo and Abhilasha Lodha explained in a blog post today. “In order to train Mockingbird to produce structured output, we created a dataset of thousands of challenging real-world examples of JSON outputs, matched to their schemas and descriptions of what the LLM should produce and used it as training data for the structured output task.”

Vectara will use its funding round to advance its product development efforts. The company also plans to invest a portion of the capital in international go-to-market initiatives. 

Image: Unsplash