Inflection AI Inc. has switched its Pi chatbot to a new large language model, Inflection-2.5, that it says can nearly match the capabilities of GPT-4 across several key areas.

The company detailed the upgrade today. It also shared new information about Pi’s recent user base growth, as well as its plans to make money from that user base.

Inflection AI is led by Chief Executive Officer Mustafa Suleyman, who previously co-founded Google LLC’s DeepMind artificial intelligence research group. The company is backed by more than $1.5 billion in funding from Microsoft Corp., Nvidia Corp. and other investors. Inflection launched its flagship product, a chatbot called Pi, last May.

The initial version of Pi shipped with a large language known as called Inflection-1. The company disclosed today that it trained the LLM with 4% of the processing power OpenAI used to build GPT-4. Inflection-2.5, the new LLM that debuted today, was trained using ten times more processing power than Inflection AI’s first-generation model.

The extra infrastructure resources helped the company unlock a significant increase in response quality. According to Inflection AI, Inflection-2.5 more than doubled the score of its predecessor in an evaluation that involved a series of math questions. It demonstrated an even more significant improvement in a test that comprised coding tasks.

Inflection AI also evaluated its latest model across several other benchmarks, most notably a subset of the BIG-Bench dataset. That’s a collection of Google-developed questions specifically designed to be difficult for LLMs to solve. Across all the benchmarks Inflection AI used, Inflection-2.5 achieved average performance that trailed GPT-4 by less than 6%. 

Together with the benchmark results, the company today released new data about Pi’s market traction. Suleyman told Axios that the chatbot’s installed base includes one million daily active users and has been growing about 10% per week for the past two months. The average Pi chat session lasts 33 minutes, with Inflection AI claiming “higher monthly stickiness than leading competitors.”

Inflection AI reportedly plans to monetize the chatbot by offering a paid subscription-based version. According to Axios, the company may eventually introduce another pricing model that will charge customers based on productivity milestones, such as when “someone reaches a particular goal they had set out inside of Pi.”

The company faces competition from not only ChatGPT but also Google’s Gemini chatbot and a range of heavily-funded startup rivals. One of those rivals, Anthropic LLC, released a new version of its flagship LLM earlier this week. Claude 3 can process prompts with up to 200,000 tokens and promises to outperform GPT-4 across several types of tasks. 

Image: Inflection AI