ByteDance Launches AI Video Diffusion Model Loopy for Realistic Audio-Driven Portrait Avatars

Sep 5, 2024, 04:44 AM

ByteDance has launched a new AI model named Loopy, which creates realistic portrait avatars based on audio input. This innovative video diffusion model uses long-term motion dependency to generate lifelike videos without the need for spatial templates. Loopy can make avatars move in sync with sound, showing emotions through eyebrow, eye, and head movements. It works with diverse visual and audio styles, generating natural motion from audio, including subtle expressions. The model is applicable to both human and non-human images and outperforms existing methods in naturalness and quality across various scenarios.