Tag
A paper examining a single-rollout asynchronous RL recipe combining the Muon optimizer, token-averaged loss, truncated importance sampling, and auxiliary penalties for training long-horizon repository-level issue resolution agents, showing improved training success over baselines across two model families.
This paper introduces Retroactive Advantage Correction (RAC), a closed-form bias correction method for delay-aware RLHF that handles asynchronous reward signals by queuing and reinjecting delayed rewards with a V-trace-style clipped residual update.
This paper presents AsyncOPD, a fully asynchronous on-policy distillation pipeline for LLMs, systematically studying the effects of stale-policy data and proposing estimator designs that improve training throughput by 1.6-3.8x while maintaining comparable accuracy.
This paper addresses the missing old logits problem in asynchronous reinforcement learning for LLMs, proposing exact and approximate correction methods to improve training stability and performance.