Together AI Inc., the operator of a cloud platform optimized to run open-source artificial intelligence models, has raised $800 million from investors.

The startup stated in its funding announcement today that the Series C deal was led by Aramco Ventures. Nvidia Corp., Vista Equity Partners, General Catalyst and several other institutional backers contributed as well. Together AI is now worth $8.3 billion.

Together AI’s platform includes a serverless inference service that developers can use to run open-source AI models, which removes the need to configure graphics cards and network equipment. It claims its serverless environments provide about twice the performance of the fastest alternative.

The company also sells three other inference services. Two use dedicated infrastructure that offers more reliability guarantees and customization options than its serverless offering. The third service, Batch Inference, prioritizes cost-efficiency over speed. It provides an up to 50% price reduction for models that don’t require the ability to answer user prompts immediately.

Under the hood, Together AI’s platform is powered by Nvidia chips and a custom software engine called ATLAS. It uses a machine learning technique called speculative decoding to speed up customer workloads.

Speculative decoding enables engineers to integrate their AI model with a second, lighter neural network. When a user enters a prompt, the lighter algorithm quickly generates a draft response. The main model then checks it for errors, makes any changes that may be necessary and delivers the prompt response to the user. That process is considerably faster than having the main model generate the output on its own.

Usually, the lightweight algorithm that creates draft responses has a fixed configuration. Models with a fixed configuration often become less accurate over time. According to Together AI, its ATLAS technology addresses the issue by automatically adapting the lightweight model to changes in user requirements. The company claims its software can speed up some inference workloads by 400%.

Customers can also use Together AI’s platform to fine-tune open-source models. It provides access to training clusters with up to thousands of graphics cards. Developers can manage the clusters using Kubernetes, which is relatively easy to use, or a tool called Slurm that offers more customization options.

One of the main challenges involved in AI training projects is that graphics cards sometimes experience technical issues. In some cases, chip failures can introduce errors into the training workflow. Together AI’s training clusters include software that automatically detects and remediates technical issues.

The company disclosed today that its annual bookings topped $1.15 billion in the second quarter. Its platform is used by several thousand organizations including LG Inc.’s AI research lab, Cohere Inc. and the Mozilla Foundation. 

Together AI will use its newly raised capital to buy more infrastructure. It hopes to grow its public cloud’s capacity by a factor of 50 over the  next five years. In addition, it plans to enhance its training and inference features.

Photo of Together AI’s founders: Together AI