RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework
Summary
RAD-2 presents a unified generator-discriminator framework for autonomous driving that combines diffusion-based trajectory generation with RL-optimized reranking, achieving 56% collision rate reduction compared to diffusion-based planners. The approach introduces techniques like Temporally Consistent Group Relative Policy Optimization and BEV-Warp simulation environment for efficient large-scale training.
View Cached Full Text
Cached at: 04/20/26, 08:28 AM
Paper page - RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework
Source: https://huggingface.co/papers/2604.15308
Abstract
A unified generator-discriminator framework for autonomous driving motion planning that improves stability and performance through diffusion-based trajectory generation and reinforcement learning optimization.
High-level autonomous driving requires motion planners capable of modeling multimodal future uncertainties while remaining robust in closed-loop interactions. Although diffusion-based planners (https://huggingface.co/papers?q=diffusion-based%20planners) are effective at modeling complex trajectory distributions, they often suffer from stochastic instabilities and the lack of corrective negative feedback when trained purely with imitation learning (https://huggingface.co/papers?q=imitation%20learning). To address these issues, we propose RAD-2, a unified generator-discriminator framework (https://huggingface.co/papers?q=generator-discriminator%20framework) for closed-loop planning (https://huggingface.co/papers?q=closed-loop%20planning). Specifically, a diffusion-based generator is used to produce diverse trajectory candidates (https://huggingface.co/papers?q=trajectory%20candidates), while an RL-optimized discriminator reranks these candidates according to their long-term driving quality. This decoupled design avoids directly applying sparse scalar rewards to the full high-dimensional trajectory space, thereby improving optimization stability. To further enhance reinforcement learning (https://huggingface.co/papers?q=reinforcement%20learning), we introduce Temporally Consistent Group Relative Policy Optimization (https://huggingface.co/papers?q=Policy%20Optimization), which exploits temporal coherence to alleviate the credit assignment problem. In addition, we propose On-policy Generator Optimization, which converts closed-loop feedback into structured longitudinal optimization signals and progressively shifts the generator toward high-reward trajectory manifolds. To support efficient large-scale training, we introduce BEV-Warp, a high-throughput simulation environment that performs closed-loop evaluation directly in Bird’s-Eye View (https://huggingface.co/papers?q=Bird%27s-Eye%20View) feature space via spatial warping (https://huggingface.co/papers?q=spatial%20warping). RAD-2 reduces the collision rate by 56% compared with strong diffusion-based planners (https://huggingface.co/papers?q=diffusion-based%20planners). Real-world deployment further demonstrates improved perceived safety and driving smoothness in complex urban traffic.
View arXiv page (https://arxiv.org/abs/2604.15308)View PDF (https://arxiv.org/pdf/2604.15308)Project page (https://hgao-cv.github.io/RAD-2/)GitHub207 (https://github.com/hustvl/RAD)Add to collection (https://huggingface.co/login?next=%2Fpapers%2F2604.15308)
Get this paper in your agent:
hf papers read 2604.15308
Don’t have the latest CLI?curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2604.15308 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2604.15308 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2604.15308 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
ReflectDrive-2: Reinforcement-Learning-Aligned Self-Editing for Discrete Diffusion Driving
ReflectDrive-2 is a new discrete diffusion planner for autonomous driving that uses reinforcement learning to enable self-editing of trajectory tokens, achieving high performance and low latency on the NAVSIM benchmark.
Accelerating Disaggregated RL for Visual Generative LLMs with Diffusion-Based Parallelism and Trainer-Assisted Generation
This paper introduces DigenRL, a disaggregated RL framework for diffusion-based generative LLMs that uses generation-axis pipeline parallelism and trainer-assisted generation to improve throughput by 1.56-2.10x over existing systems.
FlowR2A: Learning Reward-to-Action Distribution for Multimodal Driving Planning
FlowR2A proposes a novel method that combines dense reward supervision with dynamic proposal generation using a flow-matching decoder for multimodal driving planning, achieving state-of-the-art results on the NAVSIM benchmarks.
@dair_ai: Highly-recommended read from MIT on the part of RL with verifiable rewards that everyone keeps hitting. RLVR only optim…
This paper from MIT proposes an adversarial generator-discriminator framework that combines verifiable rewards with a learned signal from human demonstrations to address issues like diversity collapse, unnatural responses, and reward hacking in RLVR training of language models.
Sample-efficient Transfer Reinforcement Learning via Adaptive Reward Shaping and Policy-Ratio Reweighting Strategy
This paper proposes a safe transfer reinforcement learning framework for autonomous highway lane-changing, using adaptive teacher intervention and reward shaping to improve sample efficiency and safety. Experiments show over 52% improvement in safety and 5% improvement in efficiency over baselines.