UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models

Hugging Face Daily Papers Papers
diffusion-models reinforcement-learning text-to-image optimization udm-grpo

Summary

UDM-GRPO introduces a stable RL training framework for uniform discrete diffusion models, boosting GenEval accuracy from 69% to 96% and OCR benchmark accuracy from 8% to 57%.

Uniform Discrete Diffusion Model (UDM) has recently emerged as a promising paradigm for discrete generative modeling; however, its integration with reinforcement learning remains largely unexplored. We observe that naively applying GRPO to UDM leads to training instability and marginal performance gains. To address this, we propose \Ours, the first framework to integrate UDM with RL. Our method is guided by two key insights: (i) treating the final clean sample as the action provides more accurate and stable optimization signals; and (ii) reconstructing trajectories via the diffusion forward process better aligns probability paths with the pretraining distribution. Additionally, we introduce two strategies, Reduced-Step and CFG-Free, to further improve training efficiency. \Ours significantly improves base model performance across multiple T2I tasks. Notably, GenEval accuracy improves from 69% to 96% and PickScore increases from 20.46 to 23.81, achieving state-of-the-art performance in both continuous and discrete settings. On the OCR benchmark, accuracy rises from 8% to 57%, further validating the generalization ability of our method. Code is available at https://github.com/Yovecent/UDM-GRPO{https://github.com/Yovecent/UDM-GRPO}.
Original Article
View Cached Full Text

Cached at: 04/22/26, 10:35 AM

Paper page - UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models

Source: https://huggingface.co/papers/2604.18518

Abstract

Uniform Discrete Diffusion Model integrated with reinforcement learning through novel optimization strategies achieves state-of-the-art performance in text-to-image tasks and OCR benchmarks.

Uniform Discrete Diffusion Model(UDM) has recently emerged as a promising paradigm for discrete generative modeling; however, its integration withreinforcement learningremains largely unexplored. We observe that naively applyingGRPOto UDM leads to training instability and marginal performance gains. To address this, we propose \Ours, the first framework to integrate UDM with RL. Our method is guided by two key insights: (i) treating the final clean sample as the action provides more accurate and stable optimization signals; and (ii) reconstructing trajectories via thediffusion forward processbetter aligns probability paths with the pretraining distribution. Additionally, we introduce two strategies,Reduced-StepandCFG-Free, to further improve training efficiency. \Ours significantly improves base model performance across multiple T2I tasks. Notably,GenEvalaccuracy improves from 69% to 96% andPickScoreincreases from 20.46 to 23.81, achieving state-of-the-art performance in both continuous and discrete settings. On theOCR benchmark, accuracy rises from 8% to 57%, further validating the generalization ability of our method. Code is available at https://github.com/Yovecent/UDM-GRPO{https://github.com/Yovecent/UDM-GRPO}.

View arXiv pageView PDFProject pageGitHub10Add to collection

Get this paper in your agent:

hf papers read 2604\.18518

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper2

#### Yovecents/URSA-1.7B-IBQ512-UDMGRPO-GenEval Text-to-Image• Updated1 day ago • 23 • 1 #### Yovecents/URSA-1.7B-IBQ512-UDMGRPO-PickScore Text-to-Image• Updated1 day ago • 54 • 1

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2604.18518 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2604.18518 in a Space README.md to link it from this page.

Collections including this paper2

Similar Articles