LLM-as-a-Verifier: A General-Purpose Verification Framework
Summary
LLM-as-a-Verifier introduces a probabilistic verification framework that computes continuous scores from LLM logits, scaling across granularity, repeated evaluation, and criteria decomposition. It achieves state-of-the-art results on multiple agentic benchmarks and provides dense feedback for RL.
View Cached Full Text
Cached at: 07/07/26, 06:42 AM
Paper page - LLM-as-a-Verifier: A General-Purpose Verification Framework
Source: https://huggingface.co/papers/2607.05391
Abstract
LLM-as-a-Verifier introduces a probabilistic verification framework that scales across multiple dimensions to improve solution correctness assessment and agent performance across various benchmarks.
Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identifyverification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduceLLM-as-a-Verifier, a general-purposeverificationframework that provides fine-grained feedback foragentic taskswithout requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions,LLM-as-a-Verifiercomputes the expectation over the distribution ofscoring token logitsto generatecontinuous scores. This probabilistic formulation enablesverificationto scale along multiple dimensions: (1)score granularity, (2)repeated evaluation, and (3)criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scalingrepeated evaluationandcriteria decompositionconsistently lead to additional gains inverificationaccuracy through variance and complexity reduction. We further introduce acost-efficient ranking algorithmfor selecting the best solution among candidates using the verifier’scontinuous scores.LLM-as-a-Verifierachieves state-of-the-art performance onTerminal-Bench V2(86.5%),SWE-Bench Verified(78.2%),RoboRewardBench(87.4%), andMedAgentBench(73.3%). Beyondverification, the fine-grained signals fromLLM-as-a-Verifiercan also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show thatLLM-as-a-Verifiercan provide dense feedback for RL, improving the sample efficiency ofSACandGRPOon robotics andmathematical reasoning benchmarks.
View arXiv pageView PDFProject pageGitHub409Add to collection
Get this paper in your agent:
hf papers read 2607\.05391
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.05391 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.05391 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.05391 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
@Azaliamirh: Check out LLM-as-a-Verifier: a simple, cheap, & general-purpose self-improvement technique that boosts performance on "…
LLM-as-a-Verifier is a simple, cheap, general-purpose self-improvement technique for agentic tasks, using fine-grained scoring and logprob-based ranking to achieve SOTA on multiple benchmarks like SWE-Bench Verified and Terminal-Bench V2.
@gurtej__gill_: This new paper from Stanford, Berkeley and NVIDIA feels like a massive piece of the puzzle when it comes to test time c…
This paper from Stanford, Berkeley, and NVIDIA introduces LLM-as-a-Verifier, a general-purpose verification framework that uses token logits for continuous scoring. It achieves SOTA on multiple benchmarks including Terminal-Bench V2 (86.5%) and SWE-Bench Verified (78.2%), and provides fine-grained signals that can accelerate RL training.
@omarsar0: NEW AI paper worth bookmarking. This is something I called early, and this paper confirms it: verification has emerged …
This paper from Stanford, NVIDIA, and UC Berkeley introduces LLM-as-a-Verifier, a training-free verification framework that uses continuous scoring from LLM logits to improve accuracy across coding, robotics, and medical domains, achieving state-of-the-art results on multiple benchmarks.
Logic-Regularized Verifier Elicits Reasoning from LLMs
Introduces LoVer, an unsupervised verifier that uses logical rules (negation consistency, intra-group and inter-group consistency) to improve LLM reasoning without labeled data, achieving performance close to supervised verifiers on reasoning benchmarks.
LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL
LLM-as-a-Tutor introduces a framework that extends LLM's role from judge to tutor by dynamically adjusting prompt difficulty through pairwise comparison and constraint addition, improving instruction-following performance in reinforcement learning.