Tag
本文提出 Multiscale Policy Optimization（MPO），通过在轮次尺度上对齐信用分配与策略更新（结合多尺度优势估计 MAE 和轮次级重要性采样）来改进长时程 LLM 智能体的训练，在搜索、工具使用和自动化任务上对 4B 和 30B-A3B 模型均取得一致增益。
The paper proposes Doob Policy Optimization (DoobPO), a likelihood-free RL fine-tuning framework for diffusion and flow-based generative policies that uses Doob's h-transform to learn a ratio model and distill the tilted target policy via conditional drift matching, achieving strong results on MuJoCo continuous-control benchmarks while supporting PPO and GRPO-style updates.
论文提出 Gibbs-Gap Policy Optimization（GGPO），一种针对可验证奖励的离线/异步 RL 方法，通过最小化 KL 调整奖励的 log-mean-exp 对比来避免高方差重要性权重，并在数学推理基准上优于 GRPO 和 AGRO。
LM2LM trains LLMs with reinforcement learning to propose effective language-model improvements, using a frozen research world model as a scalable reward signal with periodic real executions for reliability, enabling Qwen3.8-27B to outperform GPT-6-sol at 33% lower data-generation cost.
VASPO 将自蒸馏教师与 PPO 中的 critic 结合，通过 token 级优势与有界概率比率改进推理训练，在六个数学推理基准上相比 PPO 显著提升 Qwen3-4B 与 OLMo3-7B 的准确率。
提出了Cross-Behavior Policy Optimization (CBPO)，通过共享的固定参考回归目标，将离线示范数据与在线/离线策略rollout统一到单一损失函数中进行LLM推理后训练，在数学推理基准上显著优于SFT→DAPO两阶段管线。
提出了RSIPO框架，通过技能与策略的协同演化实现智能体强化学习的递归自我优化，在ALFWorld上以Qwen2.5-7B达到98.6%的成功率，显著超越GRPO等基线。
该论文提出 Adaptive Reference Guidance (ARG)，通过分析前缀续写的 KL 散度来平衡参考解引导与自由生成，用于 GRPO 中全失败组的轨迹构建，在 Qwen3-4B/8B 上取得最高的聚合 pass@12。
该论文提出Convex-Concave RL (CCRL)，在对数密度比坐标下将策略迭代的精确目标建模为DC约束的DC程序，从而超越凸代理近似，恢复CPI、NPG、TRPO、AWR为特例，并在诊断性MDP和医疗模拟领域相较调优的PPO取得更优表现。
LASER提出了一种基于潜在空间伴随匹配（latent space adjoint matching）的离线强化学习算法，通过熵正则化避免潜在空间策略坍缩和利用critic伪影，在40个OGBench任务上以固定超参数取得SOTA性能。
A research paper exploring KL-regularized policy optimization, likely relevant to RLHF and reinforcement learning from human feedback, posted on arXiv.
An analysis shows that BPO and Score Centering in RL post-training converge to the same full-vocabulary token gradient under ideal conditions, with Score Centering acting as bias correction against off-policy drift rather than variance reduction.
提出 KL 正则化策略优化（KLPO）框架，通过在采样器处锚定 KL 正则项实现无 critic、无重要性权重的异步 RL 更新，仅需每 prompt 一次 rollout，并证明 SPPO、GPO、REBEL 和 BPO 均为其特例。
The paper derives Bellman-certified sensitivity envelopes for rounding continuous row-mixture policies into sparse binary policies in finite MDPs, proving unavoidable linear dimension dependence while showing candidate-specific bounds and local curvature integration substantially tighten pre-deployment return guarantees.
iADD proposes an incremental Feynman-Kac training method for diffusion policy optimization that provides better alignment-diversity tradeoffs, and theoretically shows that only late-timestep updates in reverse diffusion may harm diversity, contrary to prior work.
The paper introduces Semifactual Credit-Augmented Policy Optimization (SCAPO), a GRPO variant that uses semifactual stability scores for finer token-level credit assignment in reinforcement learning with verifiable rewards, improving AIME accuracy on Qwen3 models by up to 5.63 percentage points.
The paper proposes DEEPO, a dual-stage reinforcement learning optimization method to reduce hallucination in multimodal large language models by addressing weaknesses in the correction chain from reward to parameter update.
This paper introduces Reinforcement Learning with Decomposed Subtasks (RLDS), a method that decomposes trajectory reward into per-subtask advantages to improve credit assignment in reinforcement learning for language model agents, showing significant gains on high-heterogeneity agentic benchmarks.
This paper introduces SCOPE, a joint training method for computer-use agents that improves both task completion and safety, using a synthesized dataset and achieving strong performance on benchmarks OSWorld and OS-BLIND.
This paper introduces GVPO++, a novel post-training method for large language models that integrates KL-constrained reward maximization, enhancing training stability and enabling on-policy distillation.