Kandinsky Lab releases open 29B model that generates video with synced audio
Kandinsky 6.0 Video makes five-second clips with 44 kHz lip-synced audio from text or an image, in 3B and 29B sizes. Weights, code and Diffusers support are public under the MIT license.
Kandinsky Lab has published Kandinsky 6.0 Video, a family of diffusion models that generate video and audio together, with weights on Hugging Face and an accompanying paper that was among the day's most-upvoted on the site, at 23 votes. The family has two sizes: Lite at 3 billion parameters and Pro at 29 billion. Nobody outside the lab has published tests of them yet.
Both models make five-second clips at 24 frames per second with synchronized 44 kHz audio, including lip-sync, from a text prompt or a starting image, according to the model card. Full-HD output at 1920 by 1080 comes from a separate super-resolution stage. The Hugging Face page lists six checkpoints, including distilled versions and a 1-billion-parameter upscaler.
The paper says a dual-stream CrossDiT architecture connects a pretrained video stream to a newly trained audio stream through bidirectional cross-attention. The lab says it trained with supervised fine-tuning, reinforcement learning and distillation, and that the reinforcement step cut Pro's speech word error rate by 47 percent. Those are the authors' figures, not independent results.
The lab says Pro beats its predecessor, Kandinsky 5.0 Video Pro, and is competitive with leading audio-video generators, especially on speech quality. The lab has released code and checkpoints, plus a Diffusers integration, under the MIT license. The Hugging Face pages showed only a few hundred downloads each within about an hour of upload, so uptake is unmeasured.