Luma AI Inc., a developer of image and video generation models, today announced that it has closed a $900 million funding round.

Saudi artificial intelligence company HUMAIN led the Series C investment. AMD Ventures, Andreessen Horowitz, Amplify Partners and Matrix Partners participated as well. CNBC reported that Luma is now valued at more than $4 billion.

Luma’s flagship algorithm is a video generator called Ray3. The company describes it as the world’s first video reasoning model. Luma says that Ray3 can generate clips with realistic motion effects and 16-bit HDR, a technology that helps boost rendering quality.

Users can create a clip by uploading an image and typing in instructions on how it should be animated. Ray3 also makes it possible to provide so-called visual annotations. A designer could, for example, draw a line on a photo of a car to show the direction in which the vehicle should move once it’s animated.

Luma offers Ray3 alongside a series of image generation models called Photon. According to the company, the algorithm family offers better output quality than several popular alternatives for one tenth the cost. The most hardware-efficient Photon model can generate an image with a resolution of 1920 pixels by 1080 pixels for under half a cent.

The company provides its models through an application programming interface. It says that its software has been adopted by multiple film studios, advertising agencies and in-house marketing teams. In September, Adobe Inc. integrated Ray3 into its Firefly suite of AI design features.

Luma developed Ray3 and Photon as part of a research effort focused on inventing “useful general intelligence.” It plans to achieve that goal by training its algorithms on only text, the approach taken by some language model developers, but also multimodal data such as images and videos.

Luma’s team includes the inventors of several foundational multimodal AI technologies. One of those technologies is NeRF, a technique that allows a model to turn photos of an object into a three-dimensional representation. That representation makes it easier to generate additional three-dimensional content such as videos. Members of the Luma team also helped invent DDIM, a method of speeding up image generation models. 

The new funding will be used to finance its research efforts. According to the company, its engineers are developing technology that will make it possible to train AI models on petabytes of data. AI training datasets are usually significantly smaller.

“To create AI that can help humanity in the physical world and expand our understanding of the universe, we need to build systems that can learn from a quadrillion tokens of information — roughly the collective digital memory of humanity — contained in video, image, audio and language,” said Luma co-founder and Chief Executive Officer Amit Jain.

Luma also plans to release AI software for robots. During development, a robot’s neural networks are trained in a simulation that mimics the environment they will be expected to navigate. Video generation models can help improve the quality of such simulations.

Image: Luma