Open generative intelligence startup Stability AI Ltd. launched a new AI model called Stable Video 3D late Monday that can transform still images into 3D objects and videos.

The new model is called Stable Video 3D is built on the foundation of Stability AI’s Stable Video Diffusion, a image-to-video model that can take a still image and producing a short photorealistic video clip with motion based on the company’s image model.

“By adapting our Stable Video Diffusion image-to-video diffusion model with the addition of camera path conditioning, Stable Video 3D is able to generate multi-view videos of an object,” the company said in the announcement. “Additionally, we propose improved 3D optimization leveraging this powerful capability of Stable Video 3D to generate arbitrary orbits around an object.”

The new model enhances the company’s already existing research into producing 3D objects and videos since the release of Stable Zero123 last December. At the time, the company said that Zero123 could “generate novel views of an object” and “demonstrate an understanding of an object’s appearance from various angles.” The new models greatly improve on this model with better view consistency, greater illumination and superior reliability for 3D meshes, the research team said. 

According to Stable Diffusion, SV3D also outperforms alternative 3D AI models such as Zero123-XL.

With a better understanding of illumination around the object, SV3D objects also appear to have better lighting all around the 3D rendered objects as well as more consistent shading when rotating as well in comparison to 3D reconstructions.

The model comes in two variants. SV3D_u generates videos where the camera orbits the 3D image that uses still images as input with the viewpoint remaining unchanged. SV3D_p will allow users to build videos with camera motion around the 3D object in any path that they want. The latter model is made for spinning 3D motion and it can also accommodate creating static images from any angle.

The new model was trained on a very large set of objects and items, making it perfect for e-commerce, retail and gaming use cases. However, the resulting 3D meshes may not always be an exact match to the original object because the AI model isn’t entirely aware of the entire object — for example hidden surfaces. But it will always attempt to make the best guess as to what the entire object is going to be.

The company said that SV3D is already available for commercial purposes with a Stability AI Membership, which starts at $20 a month. Developers can get access to the open-source model weights on Hugging Face for noncommercial use and read the research paper.

Image: Stable Diffusion