Tag
论文研究了LLM在可验证奖励强化学习（RLVR）下的优化几何特性，发现梯度能量会逐渐集中到不断收缩的有效子空间中，进而提出自适应秩优化器Ares，根据梯度谱能量分布动态调整更新秩，在多个模型规模和RLVR任务上取得优于主流优化器的表现。
This paper systematically analyzes how reinforcement learning post-training reshapes flow-based VLA policies (such as π0.5 and GR00T N1.5/N1.6), finding that RL induces low-rank parameter updates concentrated in the action expert's Timestep Modules. The authors show shift-vector update directions strongly predict task success and use them to improve RL-trained policies without additional training.