Noted artificial intelligence personalized chatbot service Character.AI Inc. unveiled a new video generation model named AvatarFX on Tuesday that allows users to take images and turn them into lifelike AI chatbots.

Available to a limited number of users in closed beta mode, the model can take images and animate them so that they can speak, sing and emote. It can maintain strong consistency with photorealistic faces, along with audio, animated limb, hand and body movements, while supporting videos with multiple speakers.

“Our model performs well across a wide range of styles and scenarios, from realistic humans to mythical creatures, and even inanimate objects with faces,” Character.AI said in the announcement.

To build the model that generates the video, the company said, its multimodal team designed a specialized parameter-efficient training pipeline that allows a diffusion AI model to produce realistic lip, head and body movements based on audio sequences. Diffusion is an AI image generation method of gradually adding noise to training data and then reversing the process to produce a new image.

On the audio side, the company uses its own proprietary text-to-speech voice models.

To keep costs low, Character.AI explained that the training used state-of-the-art distillation techniques to reduce the number of steps for diffusion during runtime. That greatly reduces the computation needed at deployment without degrading quality.

Character.AI said AvatarFX differs from other avatar-generating models in that it can generate top-quality video of 2D animated characters, 3D cartoon characters and nonhuman faces with ease. It can maintain consistency even in long-form videos and doesn’t need text-to-image generation.

The company stressed that even during this test phase, the company is focusing on safety for user-uploaded content to prevent misuse and deepfakes. Dialogue written by users will be passed through content safety filters and photos of minors, high-profile politicians or other notable figures will be blocked. A watermark will also be applied to videos to make it clearer that it is AI-generated content.

Character.AI has recently faced multiple lawsuits alleging that its personalized chatbot AI characters behaved in a manner that promoted self-harm and violence with children. In one case, a mother in Florida claimed her 14-year-old son’s suicide was a consequence of his addition to one of the company’s hyperrealistic chatbots.

In response, the company rolled out a new Parental Insights feature in March to enhance safety for teens. The feature includes a dedicated model for teen users, enhancements to detection and intervention systems for human behavior, and improved AI model responses, and it offers parents and guardians insights into minors’ activities on the platform.

Image: Character.AI