Graphon Inc., a startup with technology that makes artificial intelligence models better at processing large datasets, launched today with $8.3 million in funding.

Novera Ventures led the seed round. It was joined by more than a half dozen others including the venture capital arms of Perplexity AI Inc., Samsung Electronics Co and Hitachi Ltd.

Today’s most advanced large language models have a context window of 1 million tokens. That means a single prompt can only contain up to 1 million tokens worth of data, which corresponds to a few thousand pages of text. LLMs’ context window limits their ability to process larger datasets.

AI developers get around the context window limit using RAG, or retrieval-augmented generation, tools. Those are software modules that can analyze a dataset with more than 1 million tokens, extract key records and make them available to an LLM. 

RAG tools can also prioritize the records they extract based on their relevance to a given prompt. However, they struggle to identify connections between records, which limits their usefulness. For example, a RAG system that extracts malware signals from a large cybersecurity dataset may not be capable of determining whether those signals describe different cyberattacks or a single hacking campaign.

Graphon has developed a software platform that addresses the challenge. It can analyze a dataset with more than 1 million tokens, identify key patterns and save them to a so-called persistent relational memory. LLMs can then extract the patterns from the persistent memory without hitting their context limits.

Graphon’s platform reportedly identifies patterns in datasets using small AI models with about 200 million parameters. Those models carry out processing with the help of graphs. A graph is a data structure that contains information about relationships between objects. Such data structures lend themselves well to, among other tasks, representing useful patterns in business datasets processed by LLMs.

Graphon’s platform also makes use of mathematical objects called graphon functions. They can be used to scan a business dataset stored as a graph for records that are connected to one another. Christian Borgs, a computer scientist who helped invent graphons, is a technical advisor to Graphon.

“AI has spent the last decade learning to mimic language,” said Graphon founder and Chief Executive Officer Arbaaz Khan. “But the world isn’t made of tokens, it’s made of relationships. By preserving that structure, we make foundation models more accurate and more useful at enterprise scale.”

Graphon is one of several venture-backed startups working to increase the amount of data that LLMs can ingest.

Subquadratic Inc., which raised $29 million last week, has developed a version of the transformer architecture that can process up to 14 million tokens per prompt. The company claims that its LLM design is also significantly more hardware-efficient than its predecessors. Another recently funded startup, Standard Intelligence Inc., is using a different technology called a masked compression objective. It increases LLMs’ effective processing capacity by removing unimportant data from prompts.

Image: Unsplash