step-level-feedback

The paper introduces TIPS (Thinking-Induced Process Supervision), an outcome-only reinforcement learning framework that trains generative process reward models using only outcome-level rewards, achieving 85.2 F1 on ProcessBench with just 3.2K outcome-labeled trajectories and outperforming existing trained PRMs and prompt-only judges like GPT-5.4-Instruct.

BetaPRM is a process reward model that predicts both a step-level success probability and the reliability of that prediction using a Beta belief from Monte Carlo continuations, enabling adaptive computation allocation that reduces token usage by up to 33.57% while improving accuracy.