Tag
该论文提出，在KL正则化控制下，策略的log odds隐含其harness（提示、工具、控制逻辑）的偏好，部署性能等于harness潜力上限减去策略失配；实验在MiniWoB++和Terminal-Bench Lite上证明，仅用固定模型比较harness无法预测训练收益，而事先测得的ceiling与mismatch能准确预测。
这篇论文研究了RLVR中的off-policy数据问题，证明了在覆盖性与局部表达性条件下，完整未来KL信用可使精确近端优化器成为更新的唯一平稳策略，并提出ProximalKL方法，在Qwen3-1.7B和Qwen3-4B上相比GRPO在数学基准上最高提升2.58%的微平均准确率。
A research paper exploring KL-regularized policy optimization, likely relevant to RLHF and reinforcement learning from human feedback, posted on arXiv.
This paper studies KL-regularized contextual bandits and shows that greedy sampling can achieve logarithmic regret without explicit eluder-dimension dependence for both reward and preference feedback.
This paper introduces a game-theoretic approach to fine-tuning language models that optimizes the trade-off between reward and deviating from a reference policy, providing a principled method for setting the KL regularization coefficient.
Proposes LARA, a framework for safe inference-time alignment that uses Lagrangian dualization to derive an augmented reward from separate reward and cost models, improving the helpfulness-harmlessness tradeoff without retraining.
This paper proposes SDPG, a self-distilled policy-gradient framework that combines on-policy self-distillation with verifier advantages and KL regularization to improve reinforcement learning stability and performance.