Tag
The paper shows that PRM-Pruned Fragment Grafting (PPFG), a cost-minimal inference-time cross-trajectory transfer technique, is statistically indistinguishable from independent parallel CoT across three reasoning LMs and six benchmarks, with the inertness traced to poor injection targeting and selection-dominated pruning dynamics.
The paper introduces TIPS (Thinking-Induced Process Supervision), an outcome-only reinforcement learning framework that trains generative process reward models using only outcome-level rewards, achieving 85.2 F1 on ProcessBench with just 3.2K outcome-labeled trajectories and outperforming existing trained PRMs and prompt-only judges like GPT-5.4-Instruct.
This paper formulates PRM stress testing as a quality-diversity search problem using MAP-Elites, characterizing what archive coverage can and cannot certify. Experiments on Qwen2.5-Math-PRM-7B reveal aggregation-dependent vulnerabilities, and a LoRA repair protocol reduces exploit rates.
This paper introduces Prefix Utility Model (PUM), which evaluates LLM reasoning prefixes based on their utility (improvement in solve rate) rather than local correctness. PUM shows strong performance in mathematical reasoning tasks across selection, search, and reinforcement learning.
This paper introduces Distributional Process Reward Models, using conditional optimal transport to calibrate PRMs for more accurate success probability estimates in inference-time scaling. It demonstrates improved calibration and downstream performance on mathematical reasoning benchmarks like MATH-500 and AIME.
This paper proposes unsupervised Process Reward Models (uPRM) that eliminate the need for human annotations by using LLM next-token probabilities to identify erroneous reasoning steps, achieving up to 15% accuracy improvements over LLM-as-a-Judge and performing comparably to supervised PRMs as verifiers and reward signals.