Tag
LLM-as-a-Verifier is a simple, cheap, general-purpose self-improvement technique for agentic tasks, using fine-grained scoring and logprob-based ranking to achieve SOTA on multiple benchmarks like SWE-Bench Verified and Terminal-Bench V2.
This paper from Stanford, Berkeley, and NVIDIA introduces LLM-as-a-Verifier, a general-purpose verification framework that uses token logits for continuous scoring. It achieves SOTA on multiple benchmarks including Terminal-Bench V2 (86.5%) and SWE-Bench Verified (78.2%), and provides fine-grained signals that can accelerate RL training.