AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation

Hugging Face Daily Papers Papers
audio-video-generation reinforcement-learning diffusion-models multimodal-learning ai-research

Summary

The paper proposes AV-GRPO, a modality-anchored reinforcement learning framework for joint audio-video generation that improves generation quality, semantic alignment, and cross-modal synchronization over existing methods.

Recent years have witnessed major progress in joint audio-video generation. Existing models still suffer from limited per-modality fidelity, insufficient text-modality alignment and weak cross-modal synchronization. While reinforcement-learning post-training offers a promising remedy, directly adapting it to joint audio-video generation is challenging. Heterogeneous multimodal rewards entangle learning signals and complicate credit assignment. Joint optimization of two modality towers is computationally expensive given their divergent dynamics. Moreover, synchronization evaluation difficulty depends on paired samples, preventing fair reward comparisons. We propose AV-GRPO, a modality-anchored online diffusion RL framework, and 5DAV, a decoupled, difficulty-controllable training dataset. AV-GRPO includes three key modules: (1) modality-anchored rollouts to disentangle learning signals and stabilize difficulty; (2) trajectory-locked frozen-tower optimization to reduce cost and reassign credit; (3) adaptive objectives and perturbation strengths tailored to modality-specific dynamics. This converts coupled multimodal preference learning into unimodal subproblems for precise reward attribution and better synchronization. Our 5DAV dataset decouples samples across five dimensions for systematic training. Experiments on JavisBench and VABench demonstrate AV-GRPO outperforms LTX-2.3 in generation quality, semantic alignment and cross-modal synchronization under LoRA and full fine-tuning. Ablations confirm our designs. Code and data: https://github.com/zhiyuxu03/AV-GRPO
Original Article
View Cached Full Text

Cached at: 09/25/26, 03:46 PM

Paper page - AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation

Source: https://huggingface.co/papers/2609.29816

Abstract

Recentyearshavewitnessedmajorprogressinjointaudio-videogeneration.Existingmodelsstillsufferfromlimitedper-modalityfidelity,insufficienttext-modalityalignmentandweakcross-modalsynchronization.Whilereinforcement-learningpost-trainingoffersapromisingremedy,directlyadaptingittojointaudio-videogenerationischallenging.Heterogeneousmultimodalrewardsentanglelearningsignalsandcomplicatecreditassignment.Jointoptimizationoftwomodalitytowersiscomputationallyexpensivegiventheirdivergentdynamics.Moreover,synchronizationevaluationdifficultydependsonpairedsamples,preventingfairrewardcomparisons.WeproposeAV-GRPO,amodality-anchoredonlinediffusionRLframework,and5DAV,adecoupled,difficulty-controllabletrainingdataset.AV-GRPOincludesthreekeymodules:(1)modality-anchoredrolloutstodisentanglelearningsignalsandstabilizedifficulty;(2)trajectory-lockedfrozen-toweroptimizationtoreducecostandreassigncredit;(3)adaptiveobjectivesandperturbationstrengthstailoredtomodality-specificdynamics.Thisconvertscoupledmultimodalpreferencelearningintounimodalsubproblemsforpreciserewardattributionandbettersynchronization.Our5DAVdatasetdecouplessamplesacrossfivedimensionsforsystematictraining.ExperimentsonJavisBenchandVABenchdemonstrateAV-GRPOoutperformsLTX-2.3ingenerationquality,semanticalignmentandcross-modalsynchronizationunderLoRAandfullfine-tuning.Ablationsconfirmourdesigns.Codeanddata:https://github.com/zhiyuxu03/AV-GRPO

View arXiv pageView PDFGitHub12Add to collection

Get this paper in your agent:

hf papers read 2609\.29816

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### Dr-Loser/AV-GRPO Text-to-Video• Updatedabout 2 hours ago

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.29816 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.29816 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

VideoGen-Agent: Reinforcing Video Generation Agents

Hugging Face Daily Papers

The paper presents VideoGen-Agent, a reinforcement learning-based multimodal agent that coordinates tools for video generation, significantly improving performance on the new VABench benchmark.

DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models

arXiv cs.LG

This paper identifies weaknesses in existing reinforcement learning methods for diffusion language models—lack of temporal credit assignment and biased likelihood estimates—and proposes DACA-GRPO, a plug-and-play enhancement that introduces denoising progress scores and stratified masking likelihood, achieving consistent improvements across reasoning, code generation, and constrained generation benchmarks.