OpenAI today debuted a new video generation model that it says can render clips too complicated for earlier algorithms.

The model, Sora 2, will be available in ChatGPT and via an application programming interface. It also powers a new consumer-powered social app called Sora.

Sora 2 is the successor to an eponymous video generator that OpenAI launched last year. According to the artificial intelligence provider, its new algorithm is significantly better at modelling physical phenomena such as buoyancy. As a result, it can generate complex clips such as videos that depict Olympic gymnastics routines and objects floating on water. Earlier models struggle at such tasks. 

According to OpenAI, Sora 2’s higher output quality partly stems from the way it manages the errors that sometimes emerge during the video generation process. In many cases, the model can mitigate those errors without substantially decreasing video quality.

“Prior video models are overoptimistic — they will morph objects and deform reality to successfully execute upon a text prompt,” members of the Sora development team wrote in a blog post today. “For example, if a basketball player misses a shot, the ball may spontaneously teleport to the hoop. In Sora 2, if a basketball player misses a shot, it will rebound off the backboard.”

Sora 2 improves upon its predecessor in other ways as well. Notably, it can generate not only video but also audio. The model is capable of generating speech in multiple languages and sound effects for the clips it creates.

According to OpenAI, Sora 2 supports more detailed prompts than its predecessor. Users can enter multi-sentence descriptions of the clips they wish to generate. It’s possible to customize details such as the camera equipment that Sora 2 should simulate, the texture of the objects depicted in a clip and the way light is reflected off those objects.

Prompts can include not only text but also a brief video. Sora 2 is capable of inserting the likeness of the person or object the uploaded video depicts into the clips it generates.

The model’s ability to take video as input is the basis of a new social app that OpenAI debuted today. The app, which is available for iOS on launch, allows users to generate clips, insert their likeness into those clips and remix content created by other users. Videos generated with Sora 2 are displayed in a centralized feed.

“We are giving users the tools and optionality to be in control of what they see on the feed,” the Sora developers detailed. “Using OpenAI’s existing large language models, we have developed a new class of recommender algorithms that can be instructed through natural language.”

The Sora app is initially available in the U.S. and Canada on an invite-only basis. OpenAI plans to “quickly” launch it in additional markets. Down the line, the AI provider will bring Sora 2 to ChatGPT and its developer API. Users of the paid ChatGPT Pro plan will have access to an enhanced version of the model called Sora Pro 2.

Image: OpenAI