Paper page - UniAVC: Unified Audio-Video Creation via Progressive Training and Asymmetric Scheduling

Source: https://huggingface.co/papers/2610.24791

Abstract

UniAVC trains separate audio and video branches on abundant unimodal data before aligning them with scarce paired data, using independent noise schedules and asymmetric supervision to enable joint synthesis, cross-modal generation, and instruction-guided editing.

Unified audio-video creation requires a single model capable of text-conditioned joint synthesis, bidirectionalcross-modal generation, andinstruction-guided editingacross both modalities. Existing methods rely heavily on paired corpora, which are scarce for joint generation and even rarer for editing. Consequently, they are spent learning single-branch generative and editing capabilities that abundant unimodal data could readily teach, rather than on the cross-modal coupling. Furthermore, prevailing systems apply a single sharednoise scheduleto both modalities, neglecting two critical limitations. First, due to the significant discrepancy in information density, video requires prolonged high noise to organize spatial and motion structure, whereas the sparser audio signal can settle earlier during the synthesis process, the dynamics a uniform schedule might compromise. Second, lockstepdenoisingleavesasymmetric workflows, such as conditioning on one modality to synthesize or edit another, untrained. We present UniAVC, which addresses both obstacles. It first trains each branch onunimodal datasetto full generative and editing competence, and subsequently dedicatespaired datato exploit cross-modal correspondence.Modality-decoupled schedulersmeanwhile warp thenoise scheduleindependently from a shared timestep, while subtasks with one clean modality directly supervise asymmetric configurations. These settings respectively enable intent-aware editing, which preserves one modality intact while enlisting it to constrain the other, and post-inferenceself-refinement, which regenerates only an unsatisfactory branch against an accepted one. UniAVC thus spans multiple creation tasks from a single set of weights without task-specific adapters.

View arXiv pageView PDFProject pageGitHub21Add to collection

Get this paper in your agent:

hf papers read 2610\.24791

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### TK3105/UniAVC Text-to-Video• 23B• Updatedabout 23 hours ago • 1 • 8

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.24791 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.24791 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.