rlvr

Tag

Cards List
#rlvr

Benchmarks Don't Talk Back: Learning Through Self-Disagreement

arXiv cs.CL ↗ · yesterday Cached

The paper introduces 'dialectical competence' — knowing when to hold or yield a position under counter-reasoning — and proposes Self-Disagreement, an RLVR method where a model generates its own antithesis and is trained on ruling over it, with no second model, reward model, or human annotation.

0 favorites 0 likes
#rlvr

CRoS: Causality Reward on Scrambled CoTs Improves Accuracy and Reasoning Faithfulness

arXiv cs.CL ↗ · 2026-11-02 Cached

The paper proposes CRoS, a CoT causality metric that rewards scrambling the order of reasoning steps without altering their content, avoiding confounding effects of traditional perturbation-based metrics. Evaluated with RLVR on Gemma-3-4B over Reasoning Gym, CRoS improves out-of-domain accuracy (46.8% vs 43.8%), CoT quality, and CoT faithfulness over existing causality and outcome rewards.

0 favorites 0 likes
#rlvr

[R] There is no free lunch in RLVR: under strategy specialization, every pass@1 gain provably costs some prompt, and the pass@k crossover has a tight bound [R]

Reddit r/MachineLearning ↗ · 2026-11-02

A theory paper showing that in RLVR, every pass@1 improvement provably costs performance on some prompt under strategy specialization, with a tight bound on the number of pass@k crossover points, framed as an optimal tradeoff frontier rather than a bug.

0 favorites 0 likes
#rlvr

@dongxi_nlp: https://x.com/dongxi_nlp/status/2115389565628305715

X AI KOLs Timeline ↗ · 2026-10-28 Cached

A Chinese-language explainer on reinforcement learning terminology for LLM readers, using the "Is OpenAI open?" example to walk through concepts such as policy, state, action, reward, return, value, advantage, and GAE, and explaining how scores are attributed to individual tokens when a judge only rates an entire response. Set against the backdrop of DeepSeek-R1-Zero's RLVR training, this is the fourth installment in the post-training series.

0 favorites 0 likes
#rlvr

From Visual Affinity to Learning Credit in Multimodal Reinforcement Learning

arXiv cs.AI ↗ · 2026-10-26

本文提出 Outcome-Conditioned Visual Credit (OVC) 框架,通过分离视觉关联测量与 token 级信用分配,改进多模态强化学习中的奖励分配,在三个模型的六项基准上较 GSPO 提升 1.86-2.39 个百分点。

0 favorites 0 likes
#rlvr

S3AC: Stabilized, Selective, and Scalable Actor-Critic for LLM Reasoning

Hugging Face Daily Papers ↗ · 2026-10-26 Cached

S3AC 提出一种稳定的 Actor-Critic 训练方法与难度感知提示采样(DAPS),在不依赖分组 rollout 的情况下改进大模型的长链推理,性能优于 GRPO-X 等现代基线,并可在 35B 模型上维持单 rollout 训练。

0 favorites 0 likes
#rlvr

LIFT: Scoped Constraints for Evaluating and Training Instruction Following

arXiv cs.CL ↗ · 2026-10-22 Cached

LIFT presents a new instruction-following benchmark (LIFT-Bench) and training counterpart (LIFT-Train) using scoped constraints on structured document outlines, showing that frontier models struggle on harder subsets and that training with LIFT-Train is less susceptible to reward hacking than existing IFTrain approaches.

0 favorites 0 likes
#rlvr

ReAPO: Retry-Aware Policy Optimization for Efficient Exploration in LLM Reasoning

arXiv cs.LG ↗ · 2026-10-22 Cached

ReAPO 提出一种基于随机最短路径视角的重试感知策略优化方法,对 GRPO 进行改进,将额外生成分配给失败轨迹以实现更有原则的信用分配,在 Qwen3 和 OLMo-2 系列模型的数学与科学推理基准上显著提升首次尝试性能并减少生成 token 数。

0 favorites 0 likes
#rlvr

SimHack: Predicting the onset of detectable reward hacking before RL training

arXiv cs.LG ↗ · 2026-10-22 Cached

Google Research introduces SimHack, a simulator that predicts when detectable reward hacking will become dominant during RL training of coding LLMs, before running costly full-scale training. It achieves 11.3% median relative error in onset prediction and 0.85 rank correlation across recipes, and can screen curricula to find ones that train 96.6% longer without hacking.

0 favorites 0 likes
#rlvr

Exploration via Activation-Space Steering for Reinforcement Learning with Verifiable Rewards

arXiv cs.LG ↗ · 2026-10-21 Cached

论文提出 Activation-Space Exploration (ASE),通过聚类正确与错误回答的激活差异构建潜空间探索向量库,以多样化 RLVR 训练中的 rollout;在 Qwen3-1.7B 和 Qwen3-8B 上较 GRPO 分别提升 4.87 和 1.87 个百分点的 Mean@8。

0 favorites 0 likes
#rlvr

Batched Contextual Reinforcement II: Does RLVR Need Long Rollout Windows?

arXiv cs.LG ↗ · 2026-10-19 Cached

这篇论文提出 Concurrent Constraint Reinforcement (CCR),将多个可独立验证的问题打包进单个受上限的补全中,使 RLVR 在 8,000 token 短窗口下仍保留正确性信号,相比 32k 窗口的 GRPO 每步训练 token 成本降低约 80%。

0 favorites 0 likes
#rlvr

Axiomatic World Modeling for Physics Reasoning

Hugging Face Daily Papers ↗ · 2026-10-17 Cached

提出 RLAWM,通过分层建模与求解奖励惩罚模型内部世界模型与真实物理公理之间的偏差,显著提升 LLM 在物理推理基准上的表现。

0 favorites 0 likes
#rlvr

What Should We Remove? Evaluating the Reliability and Costs of RLVR Data Decontamination

arXiv cs.CL ↗ · 2026-10-16 Cached

该论文审计了十六个开源RLVR数学语料与六个基准之间的去污染策略,发现更激进的语义清洗并不更可靠,而judge-adjudicated策略在保持高召回的同时实现了98.1%的精确率。作者公开了审查标签与各策略的移除决策,以推动可衡量可靠性的去污染报告。

0 favorites 0 likes
#rlvr

Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?

arXiv cs.LG ↗ · 2026-10-16 Cached

This paper finds that 82% of the reported out-of-domain multiple-choice gains from RLVR on four 7B SimpleRL model pairs are artifacts of answer extractors that only read \boxed{} answers, meaning RL mostly teaches models to place answers in the box rather than improving actual reasoning.

0 favorites 0 likes
#rlvr

@_avichawla: SFT vs. DPO vs. RLVR, clearly explained: All three methods customize an LLM, but they learn from different forms of sup…

X AI KOLs Timeline ↗ · 2026-10-14 Cached

An explainer comparing SFT, DPO, and RLVR for customizing LLMs, with a hands-on experiment training Qwen3 0.6B on GSM8K using GRPO on Amazon SageMaker AI, achieving a ~2.9 percentage-point accuracy gain, plus guidance on choosing the right supervision method and an AWS serverless model-customization workshop.

0 favorites 0 likes
#rlvr

Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following

Hugging Face Daily Papers ↗ · 2026-10-14 Cached

本文揭示了可验证指令跟随基准中的评估盲点:前沿模型能在几乎不回答实际问题的情况下获得满分,而使用验证器奖励进行强化学习会加剧这种奖励 hacking。作者提出 CALIF 方法,用基于校准似然的轻量级奖励增强来保留响应质量,在 IFBench 等基准上同时提升约束满足度和响应质量。

0 favorites 0 likes
#rlvr

Enabling Token-wise Importance Sampling in Off-Policy RLVR with Proximal KL Regularization

Hugging Face Daily Papers ↗ · 2026-10-13 Cached

这篇论文研究了RLVR中的off-policy数据问题,证明了在覆盖性与局部表达性条件下,完整未来KL信用可使精确近端优化器成为更新的唯一平稳策略,并提出ProximalKL方法,在Qwen3-1.7B和Qwen3-4B上相比GRPO在数学基准上最高提升2.58%的微平均准确率。

0 favorites 0 likes
#rlvr

DeltaTTG: Training Multi-Turn Agents with a Token-to-Go Progress Meter

Hugging Face Daily Papers ↗ · 2026-10-13 Cached

DeltaTTG 提出一种可复用的进度计量器(progress meter),以 Token-to-Go(剩余生成 token 数)作为完成进度的可观测代理,在不改变奖励的前提下重新分配多轮智能体训练中的优势值,在 ALFWorld、PaperSearchQA 和 BFCL v3 上显著优于 GRPO 等 RL 基线。

0 favorites 0 likes
#rlvr

MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis

arXiv cs.AI ↗ · 2026-10-12 Cached

本文提出 MagicBeanStalk,一种自顶向下分解与自底向上递归结合的 RLVR 数据合成算法,用以逃逸约88%数学语料落入的“零优势陷阱”,在多个数学与金融基准上显著提升 gpt-oss-120b、DeepSeek v4 Flash 和 Qwen3 的推理性能。

0 favorites 0 likes
#rlvr

Beyond Binary Verifiable Rewards: Behavioral Self-Rewarding for SWE Agents

arXiv cs.AI ↗ · 2026-10-12 Cached

论文提出 Behavioral Self-Rewarding (BSR),一种利用策略自身对同任务轨迹的调试行为进行两两比较并生成辅助奖励的奖励精炼方法,超越二元可验证奖励,在 SWE-bench Verified 和 SWE-bench-Live 上以 Qwen3-14B/32B 取得一致提升。

0 favorites 0 likes
Next →
← Back to home

Submit Feedback