Tag
Introduces Staleness-Adaptive Trust Regions (SAT) to stabilize asynchronous reinforcement learning by adaptively controlling update intervals based on staleness. Evaluated on a decoupled asynchronous RL setup using Qwen3-30B-A3B-Base, achieving improved results.
This paper presents AsyncOPD, a fully asynchronous on-policy distillation pipeline for LLMs, systematically studying the effects of stale-policy data and proposing estimator designs that improve training throughput by 1.6-3.8x while maintaining comparable accuracy.