joint-generation

Temporal Context Routing improves script-aligned timing in joint audio-video generation by mapping script timing onto shared video-audio temporal axes, reducing errors in shot boundaries and dialogue.

VA-Judger is the first reward model for joint video-audio generation that uses human preference feedback to evaluate holistic quality, including a dataset and benchmark, and demonstrates significant improvements in human preference rates when applied to models like LTX-2.

OmniVAE is a jointly trained audio-video VAE that uses segment-level contrastive learning and feature distillation to align latent spaces, improving joint generation quality and synchronization in text-to-audio-video generation.