Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation

# Inducing Reward-Free Judging Rubrics that Reduce Over-Crediting in Agent Evaluation
Source: [https://arxiv.org/html/2608.13564](https://arxiv.org/html/2608.13564)
Darragh Quinn, David Dylan, Roisin Healy, Fionn Carroll, Maeve Donnelly, and Cormac SheehanD\. Dylan, D\. Quinn and M\. Donnelly are with Trinity College Dublin; R\. Healy and C\. Sheehan are with University College Dublin; F\. Carroll is with Dublin City University, Dublin, Ireland\.

###### Abstract

Evaluating language\-model agents at scale increasingly relies on a second language model as an automatic judge, because the gold signal, an executable environment reward, is expensive, slow, or unavailable at deployment time\. Such a judge is a reward\-free proxy whose value depends on whether it can be trusted, yet existing judges either hand\-write the scoring rubric, as inG\-Eval, or fine\-tune the judge’s weights, and both tend to credit fluent but unsuccessful trajectories as successes\. We instead induce the text of an agent\-judging rubric from a small set of ground\-truth\-labeled trajectories, grounding it in true outcomes\. We presentRubricForge, which evolves a judge rubric by reflective evolution against labeled trajectories to maximize agreement with the environment reward, freezes it, and applies it to held\-out trajectories in one model call with no environment access\. The optimized artifact is human\-readable text, so every verdict is attributable to named criteria\. Using one frozen 7B model as both agent and judge, onτ\\tau\-bench \(173 labeled trajectories drawn from 220 rollouts\) andWebShop\(160\), the principal gain is faithfulness rather than raw agreement\. The edge over a genericG\-Evaljudge is not statistically significant \(McNemarp=0\.248p=0\.248\), and absolute\-score calibration marginally favors the generic judge \(\|err\|\|\{\\rm err\}\|difference−0\.048\-0\.048,p=2×10−4p=2\\\!\\times\\\!10^\{\-4\}\)\. YetRubricForgeover\-credits failed trajectories roughly half as often \(0\.1150\.115vs\.0\.1730\.173false\-pass rate onτ\\tau\-bench, with three over\-credit catches and zero reversals\) and ranks gradedWebShopoutcomes more faithfully \(Spearman0\.4100\.410vs\.0\.3700\.370\)\. For a reward\-free evaluator the false\-pass rate, not aggregate agreement, is the deployment\-relevant quantity, since a false pass ships a broken agent whereas a false fail merely costs a retry\.

###### Index Terms:

Automatic evaluation, evaluation metrics, intelligent agents, interpretability, large language models, LLM\-as\-a\-judge, prompt optimization, reward modeling, trustworthy machine learning\.

![Refer to caption](https://arxiv.org/html/2608.13564v1/x1.png)Figure 1:RubricForgeat a glance\. An evaluation rubric is*induced*from ground\-truth\-labeled agent trajectories by reflective evolution, then frozen and applied to held\-out trajectories with one model call and no environment access\. Left: against a genericG\-Evaljudge, the binary\-agreement edge is directional and*not*statistically significant \(McNemarp=0\.248p=0\.248\)\. Right: where it matters, the induced rubric over\-credits fluent\-but\-failed trajectories about half as often \(0\.1150\.115vs\.0\.1730\.173false\-pass rate onτ\\tau\-bench\), the deployment\-relevant error for a reward\-free evaluator\.RubricForgeis thus*harder to fool than to agree*\.## IIntroduction

Language\-model agents—systems that interleave reasoning with tool calls to act in an external environment—are now evaluated, debugged, and ranked at a scale that no longer admits human grading of every trajectory\[[65](https://arxiv.org/html/2608.13564#bib.bib25),[43](https://arxiv.org/html/2608.13564#bib.bib16),[52](https://arxiv.org/html/2608.13564#bib.bib27),[47](https://arxiv.org/html/2608.13564#bib.bib4)\]\. The cleanest evaluation signal is an*environment reward*: a programmatic oracle that inspects the world state after an episode and returns success or a graded score—a database hash in a customer\-service simulator, an attribute match in a shopping environment, a passing test suite for a code agent\[[63](https://arxiv.org/html/2608.13564#bib.bib30),[62](https://arxiv.org/html/2608.13564#bib.bib123),[14](https://arxiv.org/html/2608.13564#bib.bib127)\]\. Such oracles are the ground truth this paper is built on\. But they are also the very thing one rarely has when it is most needed\. During development one wants to score thousands of candidate rollouts cheaply; in deployment the environment is the production system itself, where a destructive “ground\-truth” probe \(issuing the refund, placing the order, mutating the record\) is exactly what must*not*be run merely to grade the agent\. The field has therefore turned to the*LLM\-as\-a\-judge*: a second language model that reads a trajectory and predicts whether the agent succeeded\[[71](https://arxiv.org/html/2608.13564#bib.bib111),[28](https://arxiv.org/html/2608.13564#bib.bib110),[10](https://arxiv.org/html/2608.13564#bib.bib120)\]\. The judge is a*reward\-free proxy*—a learned stand\-in for an oracle it never queries\.

A proxy is only as useful as it is trustworthy, and judges are known to fail in characteristic ways: they prefer verbose, fluent, confident answers\[[53](https://arxiv.org/html/2608.13564#bib.bib116)\], they favor their own generations\[[37](https://arxiv.org/html/2608.13564#bib.bib117)\], and they reward surface form over substance\. For an*agent*judge these biases concentrate into one particularly costly mistake\. A trajectory can read beautifully—the agent authenticates the user, narrates a plan, and signs off with a polished confirmation—while the underlying action failed: the booking errored out, the record was never committed, the purchased item does not match the request\. A judge that rewards the narration*over\-credits*the failure\. We argue that this single error type, not aggregate agreement, is what determines whether a reward\-free evaluator is safe to deploy, because the two ways a binary judge can be wrong are not symmetric\. A*false pass*—scoring a truly\-failed trajectory as success—silently certifies and ships a broken agent: it inflates reported success rates, masks regressions, and selects bad policies during optimization\. A*false fail*—scoring a truly\-successful trajectory as failure—merely costs a retry or a discarded sample\. The risk is asymmetric, so the metric should be too\.

This paper studies whether the*text*of an agent\-judging rubric should be*written*or*induced*\. The dominant practice writes the rubric by hand—G\-Evalprompts a strong model with human\-authored criteria\[[28](https://arxiv.org/html/2608.13564#bib.bib110)\]—or fine\-tunes the judge’s*weights*\[[19](https://arxiv.org/html/2608.13564#bib.bib112),[20](https://arxiv.org/html/2608.13564#bib.bib113),[74](https://arxiv.org/html/2608.13564#bib.bib114)\]\. We take a third path\. We keep the judge model*frozen*and instead optimize the rubric*string*by reflective evolution against a small set of ground\-truth\-labeled trajectories, using each trajectory’s true environment outcome as the supervision signal\. The induced rubric is then frozen and applied to a held\-out, by\-task\-disjoint split with a single model call and no environment access\. We call thisRubricForge\. Because the artifact under optimization is human\-readable text, the resulting metric is interpretable by construction: every verdict is attributable to named criteria the practitioner can inspect, edit, and audit\. And because induction grounds the rubric in true outcomes rather than in the judge’s untethered intuitions, we hypothesize that it is more resistant to over\-crediting fluent failures\.

Our central result separates two questions that the literature usually conflates: whether the judge agrees with the oracle in aggregate, and whether it fails in the dangerous direction\. On a hard binary success label the frozen 7B judge is near its agreement ceiling, andRubricForgedoes not significantly out\-agree a genericG\-Evaljudge; on gradedWebShopthe generic judge is in fact marginally better calibrated in absolute score\. We report both outcomes without qualification\. On the false\-pass rate, however,RubricForgeover\-credits fluent failures roughly half as often, and the entire margin between the two judges onτ\\tau\-benchreduces to three trajectories, all of which are over\-credit catches inRubricForge’s favor with zero reversals\. Aggregate agreement is therefore the wrong headline for a reward\-free evaluator: induction leaves it essentially unchanged while sharply reducing the error that determines whether the proxy is safe to deploy\.

This result speaks to a broader question about optimizing in the space of prompts\. When the objective being optimized is decoupled from real outcomes, an optimized prompt can drift or collapse; grounding the same optimization in checkable signal is what keeps it useful\.RubricForgegrounds rubric induction in labeled trajectory outcomes, which is precisely what lets an induced proxy resist drifting away from the true reward, and we develop this connection to Goodhart’s law in[Section˜II](https://arxiv.org/html/2608.13564#S2)and[Section˜VIII](https://arxiv.org/html/2608.13564#S8)\.

Contributions\.The main contributions of this paper are summarized as follows\.

- •RubricForge: automatic metric induction for agents\.We introduce a method that evolves a judge rubric against ground\-truth\-labeled trajectories with a*frozen*backbone, then freezes the rubric and scores held\-out trajectories with one model call and no environment access \([Section˜IV](https://arxiv.org/html/2608.13564#S4)\)\. Induction reliably improves validation agreement \(0\.654→0\.7690\.654\\\!\\to\\\!0\.769onτ\\tau\-bench,0\.667→0\.7500\.667\\\!\\to\\\!0\.750onWebShop\) and produces concrete, human\-readable, trajectory\-grounded criteria\.
- •Faithfulness, not agreement, is where induction pays off\.We showRubricForgeover\-credits failed trajectories about half as often as a generic judge \(0\.173→0\.1150\.173\\\!\\to\\\!0\.115false\-pass onτ\\tau\-bench; three fluent failures caught, zero reversed\), never scoring worse thanG\-Evalon any item—while stating explicitly that the binary\-agreement margin is directional and*not*significant \(McNemarp=0\.248p=0\.248\)\.
- •Better ranking of graded outcomes\.RubricForgeranks gradedWebShopoutcomes more faithfully \(Spearman0\.4100\.410vs\.0\.3700\.370\), even though absolute calibration marginally favors the generic judge \(\|err\|\|\{\\rm err\}\|difference−0\.048\-0\.048,p=2×10−4p=2\\\!\\times\\\!10^\{\-4\}\)\. We separate ranking faithfulness from absolute calibration and report both\.
- •A reward\-free evaluation protocol and an over\-crediting probe\.We package an oracle\-reward ceiling, a false\-pass rate, a ranking\-versus\-calibration split, per\-criterion leave\-one\-out attribution, and difficulty/fluency/length stratification into a reusable protocol \([Section˜VII](https://arxiv.org/html/2608.13564#S7)\), and connect the false\-pass rate to proxy\-objective misalignment and Goodhart’s law\.

Roadmap\.[Section˜II](https://arxiv.org/html/2608.13564#S2)surveys LLM\-as\-a\-judge, reward modeling, agent benchmarks, prompt optimization, and proxy misalignment\.[Section˜III](https://arxiv.org/html/2608.13564#S3)formalizes reward\-free trajectory evaluation\.[Section˜IV](https://arxiv.org/html/2608.13564#S4)presentsRubricForgewith an algorithm and an architecture diagram\.[Section˜V](https://arxiv.org/html/2608.13564#S5)details datasets, baselines, and protocol;[Section˜VI](https://arxiv.org/html/2608.13564#S6)reports the main table;[Section˜VII](https://arxiv.org/html/2608.13564#S7)is the faithfulness diagnosis\.[Section˜VIII](https://arxiv.org/html/2608.13564#S8)discusses implications and limitations, and[Section˜IX](https://arxiv.org/html/2608.13564#S9)concludes\.

## IIRelated Work

We organize the literature into five themes: \(a\) LLM\-as\-a\-judge and automatic evaluation, where the false\-pass failure our probe measures is named; \(b\) reward modeling, RLHF/RLAIF, and learned evaluators; \(c\) agent benchmarks and evaluation; \(d\) prompt optimization and evolution; and \(e\) proxy/metric misalignment and Goodhart’s law\. Throughout, we mark the differentiator ofRubricForge: prior judges either*hand\-write*the rubric or*fine\-tune*the judge’s weights, whereasRubricForge*induces*the rubric*text*by evolution while keeping the judge model frozen\.

### II\-ALLM\-as\-a\-Judge and Automatic Evaluation

Using a strong language model to grade the outputs of another has become the default scalable alternative to human evaluation\.G\-Evalprompts a capable model with a hand\-written rubric and a chain\-of\-thought form\-filling protocol to score natural\-language generation\[[28](https://arxiv.org/html/2608.13564#bib.bib110)\]; MT\-Bench and Chatbot Arena established LLM and human pairwise judging as a benchmark substrate for chat models\[[71](https://arxiv.org/html/2608.13564#bib.bib111)\]\. A complementary line trains dedicated*open*evaluators: Prometheus and Prometheus 2 fine\-tune models to apply fine\-grained, user\-supplied score rubrics and to approximate GPT\-4\-level judgments\[[19](https://arxiv.org/html/2608.13564#bib.bib112),[20](https://arxiv.org/html/2608.13564#bib.bib113)\]; JudgeLM, PandaLM, and Auto\-J fine\-tune scalable judges for instruction\-following and pairwise comparison\[[74](https://arxiv.org/html/2608.13564#bib.bib114),[55](https://arxiv.org/html/2608.13564#bib.bib115),[26](https://arxiv.org/html/2608.13564#bib.bib118)\]; FLASK decomposes evaluation into skill sets\[[66](https://arxiv.org/html/2608.13564#bib.bib138)\]; and aggregation schemes such as branch\-solve\-merge and panels of diverse juror models reduce single\-judge variance\[[40](https://arxiv.org/html/2608.13564#bib.bib137),[50](https://arxiv.org/html/2608.13564#bib.bib139)\]\. The same machinery is increasingly turned on*agents*rather than single responses—PersonaGym scores persona\-conditioned agents and CharacterEval grades multi\-turn role\-play behavior\[[41](https://arxiv.org/html/2608.13564#bib.bib66),[49](https://arxiv.org/html/2608.13564#bib.bib65)\]— which is the regimeRubricForgetargets\. Surveys catalog the rapidly growing space\[[10](https://arxiv.org/html/2608.13564#bib.bib120),[24](https://arxiv.org/html/2608.13564#bib.bib121),[25](https://arxiv.org/html/2608.13564#bib.bib122)\]\.

Crucially for us, the same line documents that judges are*biased estimators*that fail in a specific, dangerous direction\. LLM evaluators are not fair: they are swayed by answer position, verbosity, and superficial fluency\[[53](https://arxiv.org/html/2608.13564#bib.bib116)\]; they recognize and favor their own generations\[[37](https://arxiv.org/html/2608.13564#bib.bib117)\]\. These biases are exactly what produce a*false pass*—scoring a fluent\-but\-failed trajectory as a success—which is the error our over\-crediting probe isolates and measures\. Viewed through this lens, an induced agent rubric is a*learned proxy*for the true environment reward, and the false\-pass rate is the operational measurement of the gap between that proxy and the reward\[[60](https://arxiv.org/html/2608.13564#bib.bib3)\]\.RubricForgediffers from all of the above in*what*is optimized\.G\-Evalfixes a human\-written rubric; Prometheus and JudgeLM move the judge’s weights\.RubricForgeleaves the judge model frozen and optimizes the rubric*string*against ground\-truth labels—cheaper than fine\-tuning, more faithful than hand\-writing, and interpretable by construction because the artifact is text\.

### II\-BReward Modeling, RLHF, and Learned Evaluators

Learned evaluators have a long lineage in preference\-based alignment\. Reward models trained on human comparisons drive RLHF for summarization and instruction following\[[46](https://arxiv.org/html/2608.13564#bib.bib129),[34](https://arxiv.org/html/2608.13564#bib.bib128)\]; RLAIF replaces or augments the human signal with AI feedback and constitutional self\-critique\[[22](https://arxiv.org/html/2608.13564#bib.bib119),[4](https://arxiv.org/html/2608.13564#bib.bib75)\]; and direct preference optimization recasts the language model itself as an implicit reward model\[[39](https://arxiv.org/html/2608.13564#bib.bib130)\]\. “Self\-rewarding” and “meta\-rewarding” schemes let a model generate its own training rewards or judge its own judgments\[[67](https://arxiv.org/html/2608.13564#bib.bib81),[57](https://arxiv.org/html/2608.13564#bib.bib82)\], and self\-feedback loops such as Self\-Refine and tool\-interactive critique iteratively improve an output against the model’s own criticism\[[29](https://arxiv.org/html/2608.13564#bib.bib15),[9](https://arxiv.org/html/2608.13564#bib.bib74)\]\. The reliability of these learned rewards is itself now benchmarked, e\.g\. RewardBench measures how well reward models track held\-out preferences\[[21](https://arxiv.org/html/2608.13564#bib.bib131)\]\.RubricForgeshares the goal of a learned evaluator but targets a different object and supervision\. Reward models score*responses*from*scalar preference*data and are typically consumed by an optimizer;RubricForgescores entire*agent trajectories*against*programmatic outcome*labels and is consumed as a frozen, reward\-free*measurement*instrument whose verdicts are human\-auditable rather than a black\-box scalar\.

### II\-CAgent Benchmarks and Evaluation

The agents we judge come from interactive benchmarks with executable rewards\.τ\\tau\-bench scores tool\-agent\-user dialogues in retail and airline customer\-service domains by hashing the final database state against a gold state, yielding a strict binary reward\[[63](https://arxiv.org/html/2608.13564#bib.bib30)\]\.WebShopprovides a graded reward measuring how well a purchased product matches a natural \-language instruction’s attributes and price\[[62](https://arxiv.org/html/2608.13564#bib.bib123)\]\. Broader suites—AgentBench across eight environments\[[27](https://arxiv.org/html/2608.13564#bib.bib124)\], WebArena for realistic web tasks\[[72](https://arxiv.org/html/2608.13564#bib.bib126)\], GAIA for general assistants\[[30](https://arxiv.org/html/2608.13564#bib.bib125)\], and SWE\-bench for repository\-level coding\[[14](https://arxiv.org/html/2608.13564#bib.bib127)\]—all rely on programmatic success checks\. The trajectories themselves are produced by agent policies such as ReAct\[[65](https://arxiv.org/html/2608.13564#bib.bib25)\], reflective and tree\-structured variants\[[43](https://arxiv.org/html/2608.13564#bib.bib16),[64](https://arxiv.org/html/2608.13564#bib.bib80)\], experiential and memory\-augmented agents\[[69](https://arxiv.org/html/2608.13564#bib.bib20),[56](https://arxiv.org/html/2608.13564#bib.bib21),[35](https://arxiv.org/html/2608.13564#bib.bib24)\], generative agents\[[38](https://arxiv.org/html/2608.13564#bib.bib26)\], reasoning bootstrappers\[[68](https://arxiv.org/html/2608.13564#bib.bib28)\], and self\-evolving or recursively\-composed agents\[[52](https://arxiv.org/html/2608.13564#bib.bib27),[47](https://arxiv.org/html/2608.13564#bib.bib4),[61](https://arxiv.org/html/2608.13564#bib.bib5)\]\. Many of these agents are moreover*retrieval\-augmented*, grounding their actions in fetched evidence rather than parametric memory alone\[[23](https://arxiv.org/html/2608.13564#bib.bib38),[16](https://arxiv.org/html/2608.13564#bib.bib39),[3](https://arxiv.org/html/2608.13564#bib.bib49),[42](https://arxiv.org/html/2608.13564#bib.bib47)\], which only widens the space of fluent\-but\-unfaithful traces a reward\-free judge must screen\. These benchmarks supply the ground\-truth labelsRubricForgeinduces against;RubricForgeis orthogonal to them, learning a reward\-free proxy that predicts their oracle from the trajectory alone, so it can grade rollouts when re\-running the oracle is too costly or unsafe\.

### II\-DPrompt Optimization and Evolution

RubricForgeinduces its rubric with reflective prompt evolution rather than gradient descent\. Automatic prompt search spans Monte\-Carlo instruction generation \(APE\)\[[73](https://arxiv.org/html/2608.13564#bib.bib76)\], optimization\-by\-prompting where an LLM proposes improved prompts from a trajectory of past attempts \(OPRO\)\[[58](https://arxiv.org/html/2608.13564#bib.bib19)\], evolutionary prompt search \(PromptBreeder\)\[[5](https://arxiv.org/html/2608.13564#bib.bib77)\], and the DSPy line that compiles and optimizes language\-model pipelines\[[17](https://arxiv.org/html/2608.13564#bib.bib48),[18](https://arxiv.org/html/2608.13564#bib.bib18)\]\. We specifically build on GEPA, which uses natural\-language*reflection*over execution traces to mutate prompts and has been shown to rival reinforcement learning at far lower sample cost\[[1](https://arxiv.org/html/2608.13564#bib.bib17)\]\.RubricForgeapplies this machinery to a target it was not previously used for: the single mutated component is the*judge’s own rubric*, the “rollout” is a*judging pass*rather than an environment rollout, and the fitness is*agreement with the ground\-truth label*\. The optimizer is reused unchanged; the contribution is the reward\-free induction*objective*, not a new search algorithm\.

Reflective prompt evolution sits within a broader family of*self\-improvement without weight updates*\. Models can bootstrap from their own high\-confidence generations\[[12](https://arxiv.org/html/2608.13564#bib.bib78)\], marginalize over sampled reasoning paths\[[54](https://arxiv.org/html/2608.13564#bib.bib79)\], and revise prior attempts—though the limits of unaided self\-correction are now well documented: it helps with external feedback but can fail or even hurt when the model grades itself\[[13](https://arxiv.org/html/2608.13564#bib.bib83),[15](https://arxiv.org/html/2608.13564#bib.bib84)\], which is precisely whyRubricForgegrounds its reflection in*external*ground\-truth labels rather than the model’s own confidence\. The induction stage is also a form of*test\-time*adaptation: it spends extra compute to fit an instrument before deployment, echoing test\-time scaling\[[45](https://arxiv.org/html/2608.13564#bib.bib85),[32](https://arxiv.org/html/2608.13564#bib.bib86)\]and the classical meta\-learning and test\-time\-adaptation program\[[6](https://arxiv.org/html/2608.13564#bib.bib87),[33](https://arxiv.org/html/2608.13564#bib.bib88),[11](https://arxiv.org/html/2608.13564#bib.bib99),[31](https://arxiv.org/html/2608.13564#bib.bib104),[48](https://arxiv.org/html/2608.13564#bib.bib107),[51](https://arxiv.org/html/2608.13564#bib.bib108)\]of learning\-to\-adapt from limited supervision—here adapting a*rubric*, not weights\.

### II\-EProxy/Metric Misalignment and Goodhart’s Law

That an optimized proxy drifts from the true objective is Goodhart’s law: once a measure becomes a target it ceases to be a good measure\[[8](https://arxiv.org/html/2608.13564#bib.bib132)\]\. In machine learning this manifests as reward over\-optimization and reward hacking—policies that exploit an imperfect reward to score highly while violating the intent\[[2](https://arxiv.org/html/2608.13564#bib.bib135),[44](https://arxiv.org/html/2608.13564#bib.bib134),[36](https://arxiv.org/html/2608.13564#bib.bib136)\], with characterized scaling laws for how proxy reward and true reward diverge under optimization pressure\[[7](https://arxiv.org/html/2608.13564#bib.bib133)\]\. A reward\-free judge is itself a proxy at risk of this drift: an ungrounded judge that learns to reward fluency*is*a Goodharted metric\. Our over\-crediting probe is a direct, operational measurement of this drift for evaluation metrics—the false\-pass rate is how far the proxy has slipped toward rewarding form over outcome\. The enabling contrast is grounding\. ToolTree grounds an agent’s planning in per\-instance, programmatically checkable feedback\[[59](https://arxiv.org/html/2608.13564#bib.bib2)\]; analogously,RubricForgegrounds rubric*induction*in per\-trajectory ground\-truth labels, which is exactly why the induced metric resists drifting toward fluency where an ungrounded generic judge silently does\. Grounding a small\-sample objective in real outcomes rather than an internal preference is the same move that few\-shot preference alignment makes\[[70](https://arxiv.org/html/2608.13564#bib.bib71)\]\. This grounded stance contrasts with ungrounded prompt\-space optimization, in which an aggregate objective decoupled from real outcomes can drift or collapse; here a grounded objective instead confers faithfulness\.

## IIIProblem Formulation

We formalize reward\-free trajectory evaluation and state the induction objective\.[Table˜I](https://arxiv.org/html/2608.13564#S3.T1)summarizes the notation\.

TABLE I:Notation used throughout the paper\.Setup\.An agent acting in an environment produces a trajectoryτ\\tau—a sequence of actions \(tool calls and user\-facing messages\) interleaved with observations \(tool results and user replies\), terminating in a final committed answer\. After the episode an oracle returns the environment rewardr⋆​\(τ\)r^\{\\star\}\(\\tau\), either binary \(e\.g\. a database\-state hash inτ\\tau\-bench\[[63](https://arxiv.org/html/2608.13564#bib.bib30)\]\) or graded in\[0,1\]\[0,1\]\(e\.g\. the attribute \-match score inWebShop\[[62](https://arxiv.org/html/2608.13564#bib.bib123)\]\)\. We define the binary success labely​\(τ\)=𝟏​\[r⋆​\(τ\)\>0\]y\(\\tau\)=\\mathbf\{1\}\[r^\{\\star\}\(\\tau\)\>0\]\. The oracle is the ground truth; the defining constraint of*reward\-free*evaluation is that at scoring time the evaluator may*not*query it, because doing so is expensive, slow, or, in a live system, destructive\.

The induced metric\.A reward\-free evaluator is a function that maps the trajectory*text*x​\(τ\)x\(\\tau\)to a prediction without environment access\. We realize it as a frozen judge backbonegθg\_\{\\theta\}conditioned on a rubric stringρ\\rho:

Mρ​\(τ\)=gθ​\(ρ,x​\(τ\)\)=\(p^,s^,z^\),M\_\{\\rho\}\(\\tau\)\\;=\\;g\_\{\\theta\}\\big\(\\rho,\\,x\(\\tau\)\\big\)\\;=\\;\(\\hat\{p\},\\,\\hat\{s\},\\,\\hat\{z\}\),\(1\)a single deterministic call \(temperature0\) returning a pass verdictp^∈\{0,1\}\\hat\{p\}\\in\\\{0,1\\\}, a graded scores^∈\[0,1\]\\hat\{s\}\\in\[0,1\], and a one\-sentence rationalez^\\hat\{z\}\. The text renderingx​\(τ\)x\(\\tau\)exposes only what an observer of the interaction would see—actions, tool results, user replies, counters, and the final answer—and never the oracle reward or gold world\-state, so no label leaks into the judge’s input\.

Objective\.Let agreement on a labeled set be

A​\(ρ;𝒟\)=1\|𝒟\|​∑τ∈𝒟𝟏​\[p^ρ​\(τ\)=y​\(τ\)\]\.A\(\\rho;\\mathcal\{D\}\)\\;=\\;\\frac\{1\}\{\|\\mathcal\{D\}\|\}\\sum\_\{\\tau\\in\\mathcal\{D\}\}\\mathbf\{1\}\\\!\\big\[\\hat\{p\}\_\{\\rho\}\(\\tau\)=y\(\\tau\)\\big\]\.\(2\)Rubric induction seeks the rubric maximizing agreement on the labeled training pool, with selection on a held\-out validation pool to avoid overfitting the exact training trajectories:

ρ⋆=arg⁡maxρ∈ℛ⁡A​\(ρ;𝒟tr\),selected by​A​\(ρ;𝒟va\),\\rho^\{\\star\}\\;=\\;\\arg\\max\_\{\\rho\\,\\in\\,\\mathcal\{R\}\}\\;A\\big\(\\rho;\\,\\mathcal\{D\}\_\{\\rm tr\}\\big\),\\quad\\text\{selected by \}A\\big\(\\rho;\\,\\mathcal\{D\}\_\{\\rm va\}\\big\),\(3\)over the spaceℛ\\mathcal\{R\}of candidate rubric strings reachable by the evolution operator\. The judge backboneθ\\thetais*frozen*: optimization moves only the textρ\\rho, not the weights\. The frozenρ⋆\\rho^\{\\star\}is then applied once per held\-out trajectory in𝒟te\\mathcal\{D\}\_\{\\rm te\}\.

What we actually optimize for\.Agreement \([Equation˜2](https://arxiv.org/html/2608.13564#S3.E2)\) is the*training*signal, but it is not the deployment\-relevant quantity\. We separate two error directions\. On a truly\-failed trajectory \(y=0y=0\), a*false pass*\(p^=1\\hat\{p\}=1\) silently certifies a broken agent; on a truly\-successful trajectory \(y=1y=1\), a*false fail*merely costs a retry\. We therefore single out the false\-pass rate,

FP​\(ρ\)=\|\{τ:y​\(τ\)=0∧p^ρ​\(τ\)=1\}\|\|\{τ:y​\(τ\)=0\}\|,\\mathrm\{FP\}\(\\rho\)\\;=\\;\\frac\{\\big\|\\\{\\tau:\\,y\(\\tau\)=0\\ \\wedge\\ \\hat\{p\}\_\{\\rho\}\(\\tau\)=1\\\}\\big\|\}\{\\big\|\\\{\\tau:\\,y\(\\tau\)=0\\\}\\big\|\},\(4\)as the primary*faithfulness*criterion, and treat raw agreement,κ\\kappa, AUC, ranking \(Spearman\), and absolute calibration as complementary diagnostics\. The central empirical question is then sharp: does grounding the rubric in labels \([Equation˜3](https://arxiv.org/html/2608.13564#S3.E3)\) reduceFP\\mathrm\{FP\}\([Equation˜4](https://arxiv.org/html/2608.13564#S3.E4)\)—make the proxy harder to fool—even where it does not move raw agreement?

## IVMethod: RubricForge

RubricForgehas three stages:*induce*a rubric by reflective evolution against labeled trajectories,*freeze*it, and*apply*it to held\-out trajectories as a one\-call reward\-free metric\.[Figure˜2](https://arxiv.org/html/2608.13564#S4.F2)diagrams the pipeline and[Algorithm˜1](https://arxiv.org/html/2608.13564#alg1)states it\.

INDUCE \(reflective evolution vs\. ground truth\)APPLY \(reward\-free, one call, no env\)Labeled trajectories\{\(τi,ri⋆\)\}∈𝒟tr\\\{\(\\tau\_\{i\},r^\{\\star\}\_\{i\}\)\\\}\\in\\mathcal\{D\}\_\{\\rm tr\}Validation pool𝒟va\\mathcal\{D\}\_\{\\rm va\}Frozen judgegθg\_\{\\theta\}score with rubricρt\\rho\_\{t\}Reflect onmis\-scored cases⇒\\Rightarrowproposeρt\+1\\rho\_\{t\+1\}Rubric pool /select byA​\(⋅;𝒟va\)A\(\\cdot;\\mathcal\{D\}\_\{\\rm va\}\)Freezeρ⋆\\rho^\{\\star\}Held\-outτ\\tau∈𝒟te\\in\\mathcal\{D\}\_\{\\rm te\}\(no env access\)One call:Mρ⋆​\(τ\)M\_\{\\rho^\{\\star\}\}\(\\tau\)Verdict\(p^,s^,z^\)\(\\hat\{p\},\\hat\{s\},\\hat\{z\}\)\+ criteria𝒞\\mathcal\{C\}Faithfulness /over\-credit probeFP\\mathrm\{FP\}, ranking, attributionp^\\hat\{p\}vsyynext roundbestFigure 2:RubricForgearchitecture\.Induce\(top, blue\): the frozen judgegθg\_\{\\theta\}scores training trajectories with the current rubricρt\\rho\_\{t\}; a reflection step reads the cases whose verdict disagreed with the ground\-truth label and proposes an improved rubricρt\+1\\rho\_\{t\+1\}; candidates are pooled and selected by validation agreement \([Equation˜3](https://arxiv.org/html/2608.13564#S3.E3)\)\.Freeze: the best rubricρ⋆\\rho^\{\\star\}is frozen\.Apply\(bottom, green\): for each held\-out trajectoryMρ⋆M\_\{\\rho^\{\\star\}\}emits a verdict\(p^,s^,z^\)\(\\hat\{p\},\\hat\{s\},\\hat\{z\}\)in a single call with no environment access, feeding the faithfulness / over\-crediting probe\. The judge backbone is never updated; only the rubric text is optimized\.### IV\-AStage 1: Rubric Induction

The optimizer is GEPA’s reflective prompt evolution\[[1](https://arxiv.org/html/2608.13564#bib.bib17)\], reused unchanged;RubricForge’s novelty is in the component, the rollout, and the fitness it is pointed at\. The single evolved component is the rubric stringρ\\rho\(the judge’s system prompt\)\. The “rollout” for a candidate rubric is not an environment rollout but a*judging pass*: the frozen backbone scores each training trajectory withρ\\rho, and the per\-trajectory fitness is binary agreement with the gold label,

s​\(τ;ρ\)=1​\[p^ρ​\(τ\)=y​\(τ\)\],s\(\\tau;\\rho\)\\;=\\;\\mathbf\{1\}\\\!\\big\[\\hat\{p\}\_\{\\rho\}\(\\tau\)=y\(\\tau\)\\big\],\(5\)so the optimizer’s objective is exactly agreement \([Equation˜2](https://arxiv.org/html/2608.13564#S3.E2)\)\.

The reflection step is where grounding enters\. After a minibatch is scored, the mis\-scored trajectories—those where the judge’s verdict disagreed with the true label—are collected, and the reflection meta\-prompt presents the current rubric together with these failures and their true labels, asking for a revised rubric that would label them correctly and generalize\. Two design choices keep the induction honest and leakage\-free\. First, the rubric author \(the reflecting model\) is told the true label*only for the training trajectories it is revising*, and is explicitly instructed to describe*observable trajectory evidence*\(which signals indicate the user’s needs were met versus the agent guessing\) rather than to mention the reward—so the evolved rubric encodes*symptoms of success*, not a memorized answer key, and the deployed judge never sees a label\. Second, the seed of the search is not an empty or strawman rubric but the same competent generic G\-Eval\-style rubric used by ourG\-Evalbaseline; any rubric the search accepts must therefore beat a real judge, not a degenerate one, so reported gains are not an artifact of a weak starting point\. Candidates are pooled and the rubric maximizing validation agreement \([Equation˜3](https://arxiv.org/html/2608.13564#S3.E3)\) is retained\.

### IV\-BStage 2: Freeze

The selected rubricρ⋆\\rho^\{\\star\}is frozen verbatim\. If the search never beats its seed,RubricForgeexplicitly records that the effective rubric is the generic default; in our runs the search did improve over the seed on both benchmarks \([Section˜VI](https://arxiv.org/html/2608.13564#S6)\), accepting evolved rubrics into pools of size66\(τ\\tau\-bench\) and44\(WebShop\)\. Freezing makes the metric a fixed, reproducible instrument: the same trajectory always receives the same verdict\.

### IV\-CStage 3: Reward\-Free Application

At test time the frozenρ⋆\\rho^\{\\star\}is the judge’s system prompt and the rendered held\-out trajectoryx​\(τ\)x\(\\tau\)is the user message; a single deterministic call returns the verdict \([Equation˜1](https://arxiv.org/html/2608.13564#S3.E1)\)\. A fixed output contract, kept separate from the evolved rubric so the optimizer shapes criteria rather than formatting, requests a strict JSON object\{pass,score,reason\}\\\{\\text\{pass\},\\text\{score\},\\text\{reason\}\\\}\. Parsing is robust—the first brace\-delimited object is extracted and decoded—and*conservative*: any network or parse failure falls back top^=0\\hat\{p\}=0, so a malformed judge call never spuriously*passes*a trajectory \(it can only abstain toward failure, the safe direction for a reward\-free evaluator\)\. Because the rubric is text, its named criteria𝒞\\mathcal\{C\}can be applied individually to attribute each verdict to the criterion responsible, which we exploit in[Section˜VII](https://arxiv.org/html/2608.13564#S7)\.

Algorithm 1RubricForge: reward\-free metric induction0:labeled pools

𝒟tr,𝒟va\\mathcal\{D\}\_\{\\rm tr\},\\mathcal\{D\}\_\{\\rm va\}; frozen judge

gθg\_\{\\theta\}; seed rubric

ρ0\\rho\_\{0\}; budget

TT; minibatch

bb
1:// Stage 1: induce

2:pool

←\{ρ0\}\\leftarrow\\\{\\rho\_\{0\}\\\}; evaluate

A​\(ρ0;𝒟va\)A\(\\rho\_\{0\};\\mathcal\{D\}\_\{\\rm va\}\)
3:for

t=1t=1to

TTdo

4:sample parent

ρ\\rhofrom pool; draw minibatch

B⊂𝒟trB\\subset\\mathcal\{D\}\_\{\\rm tr\},

\|B\|=b\|B\|=b
5:for each

τ∈B\\tau\\in B:

\(p^,s^,z^\)←gθ​\(ρ,x​\(τ\)\)\(\\hat\{p\},\\hat\{s\},\\hat\{z\}\)\\leftarrow g\_\{\\theta\}\(\\rho,x\(\\tau\)\);

s​\(τ\)←𝟏​\[p^=y​\(τ\)\]s\(\\tau\)\\leftarrow\\mathbf\{1\}\[\\hat\{p\}=y\(\\tau\)\]
6:

F←\{τ∈B:p^≠y​\(τ\)\}F\\leftarrow\\\{\\tau\\in B:\\hat\{p\}\\neq y\(\\tau\)\\\}\{mis\-scored cases\}

7:

ρ′←Reflect​\(ρ,F,true labels of​F\)\\rho^\{\\prime\}\\leftarrow\\textsc\{Reflect\}\(\\rho,\\,F,\\,\\text\{true labels of \}F\)\{describe symptoms, not the label\}

8:if

A​\(ρ′;𝒟va\)≥maxρ∈pool⁡A​\(ρ;𝒟va\)A\(\\rho^\{\\prime\};\\mathcal\{D\}\_\{\\rm va\}\)\\geq\\max\_\{\\rho\\in\\text\{pool\}\}A\(\\rho;\\mathcal\{D\}\_\{\\rm va\}\)then

9:pool

←\\leftarrowpool

∪\{ρ′\}\\cup\\,\\\{\\rho^\{\\prime\}\\\}
10:endif

11:endfor

12:// Stage 2: freeze

13:

ρ⋆←arg⁡maxρ∈pool⁡A​\(ρ;𝒟va\)\\rho^\{\\star\}\\leftarrow\\arg\\max\_\{\\rho\\in\\text\{pool\}\}A\(\\rho;\\mathcal\{D\}\_\{\\rm va\}\)
14:// Stage 3: apply \(reward\-free, no env access\)

15:return

Mρ⋆:τ↦gθ​\(ρ⋆,x​\(τ\)\)M\_\{\\rho^\{\\star\}\}:\\tau\\mapsto g\_\{\\theta\}\(\\rho^\{\\star\},x\(\\tau\)\)

## VExperimental Setup

Benchmarks and trajectories\.We evaluate on two agent benchmarks with executable rewards\.τ\\tau\-bench\[[63](https://arxiv.org/html/2608.13564#bib.bib30)\]covers retail and airline customer service with a strict*binary*database\-state\-hash reward; we dump220220labeled rollouts and admit them*whole tasks at a time*\(all of a task’s cells stay in one split, so no task leaks across the train/val/test boundary\) into a class\-stratified85/26/6285/26/62split, which fills its budget with the173173cells reported below; the remaining4747surplus cells are held out\. The173173enter the experiment at a∼\\sim15%15\\%positive rate\.WebShop\[[62](https://arxiv.org/html/2608.13564#bib.bib123)\]provides a*graded*attribute\-match reward; we collect160160trajectories, of which116116carry graded scores\. For each benchmark we dump trajectories from two agent policies—the standard ReAct loop\[[65](https://arxiv.org/html/2608.13564#bib.bib25)\]and a self\-evolving RSEA policy—so the judge sees both terse and elaborated traces\. Each trajectory is rendered to judge\-readable text exposing only observable signals \(actions, tool results and errors, user replies, step/error counters, and the final committed answer\); the oracle reward and gold world\-state are withheld from the renderer\.

Frozen backbone\.A single frozen Qwen2\.5\-7B model serves as*both*the agent backbone and the judge, at temperature0for deterministic, reproducible verdicts\. Using one frozen 7B model for everything is a deliberate stress test: it removes any capability gap between agent and judge and isolates the effect of the*rubric*rather than a stronger grader\.

Splits\.Trajectories are partitioned*by task*into train/validation/test at roughly50/15/3550/15/35, so no test task appears in induction\. Onτ\\tau\-benchthe173173admitted cells yield8585train \(1212positive\),2626validation \(44positive\), and6262test trajectories \(1010positive,∼16%\\sim 16\\%\), summing to85\+26\+62=17385\\\!\+\\\!26\\\!\+\\\!62=173at an overall∼15%\\sim 15\\%positive rate;WebShopuses8080train and2424validation, with5656graded test trajectories for the calibration analysis\.[Table˜II](https://arxiv.org/html/2608.13564#S5.T2)consolidates these per\-split trajectory counts, positive rates, reward types, and induction budgets in one place; note that the test positive rate \(∼\\sim16%16\\%\) is held close to the pool rate \(∼\\sim15%15\\%\) by the by\-task class\-stratified split, so the agreement numbers below are not an artifact of a skewed test label\.

TABLE II:Dataset and protocol statistics\. Trajectories are split*by task*\(no task crosses a split boundary\); positive rate isPr⁡\[r⋆\>0\]\\Pr\[r^\{\\star\}\>0\]\. “Graded” counts the subset carrying a continuousr⋆∈\[0,1\]r^\{\\star\}\\in\[0,1\]used for the calibration analysis\. Budget is GEPA judging calls during induction\.Baselines\.We compareRubricForgeagainst \(i\)*Oracle*, the environment reward itself \(a ceiling, agreement1\.01\.0by construction\); \(ii\)G\-Eval\[[28](https://arxiv.org/html/2608.13564#bib.bib110)\], the generic hand\-written judge—the same rubric that seeds induction, so the comparison isolates the value of grounding; \(iii\)*FewShot*, the generic rubric augmented withk=4k\\\!=\\\!4in\-context labeled trajectory examples \(no rubric evolution\); \(iv\)*Heuristic*, a non\-LLM surface\-feature rule; and \(v\)*Majority*, the train\-majority class predicted for every item \(degenerate but a standard sanity floor on an imbalanced label\)\.

Metrics\.For binary agreement we report Accuracy with a bootstrap95%95\\%confidence interval,F1F\_\{1\}on the positive class, Cohen’sκ\\kappa\(which, unlike accuracy, penalizes the degenerate all\-negative predictor\), and ROC\-AUC\. For graded outcomes we report Spearman’sρ\\rhoand Kendall’sτ\\tau\(ranking faithfulness\) and mean absolute error\|s^−r⋆\|\|\\hat\{s\}\-r^\{\\star\}\|\(absolute calibration\)\. The faithfulness primary metric is the false\-pass rateFP\\mathrm\{FP\}\([Equation˜4](https://arxiv.org/html/2608.13564#S3.E4)\)\.

Significance\.Every judge\-vs\-judge comparison carries a paired test: McNemar’s exact test on per\-item agreement for binary verdicts, and a paired bootstrap on absolute error for graded scores\. We mark significance as\*\*\*p<0\.001p\\\!<\\\!0\.001,\*\*p<0\.01p\\\!<\\\!0\.01,\*p<0\.05p\\\!<\\\!0\.05, andnsotherwise, and we label directional\-but\-not\-significant gaps*ns*explicitly rather than implying they are wins\. Confidence intervals are10410^\{4\}\-resample nonparametric bootstrap\.

## VIResults

[Table˜III](https://arxiv.org/html/2608.13564#S6.T3)reports the mainτ\\tau\-benchtest results\. We read it through finding\-titled observations rather than a single aggregate verdict, because the honest story is split across metrics\.

TABLE III:Main results onτ\\tau\-benchtest \(n=62n=62,∼\\sim16%16\\%positive\)\. Judge↔\\leftrightarrowground\-truth agreement\. Best non\-degenerate value per column inbold\. Majority \(all\-fail\) and Heuristic \(pass\-almost\-everything, acc\.0\.1610\.161\) are degenerate predictors; their inflated metrics are bracketed and excluded from the per\-column best\. McNemar column tests each judge againstRubricForgeon per\-item agreement \(pp, significance\)\.↑\\uparrow: higher is better\.Finding 1: Induction works, and the rubric is concrete\.Reflective evolution improves validation agreement from the generic seed in both domains:0\.654→0\.7690\.654\\\!\\to\\\!0\.769onτ\\tau\-bench\(a pool of66rubrics,208208judging calls\) and0\.667→0\.7500\.667\\\!\\to\\\!0\.750onWebShop\(a pool of44,156156calls\)\. The inducedτ\\tau\-benchrubric is not an opaque tuned string but five named, trajectory\-grounded criteria—successful authentication \(no repeated failed attempts\), order verification \(no “order not found”\), request fulfillment with explicit user confirmation, error\-free tool usage, and clear user communication—each phrased in terms of observable trajectory evidence \([Appendix˜A](https://arxiv.org/html/2608.13564#A1)\)\.

Finding 2: The binary\-agreement edge is real in direction but not significant\.RubricForgeattains the highest accuracy among informative judges \(0\.7740\.774vs\.0\.7260\.726forG\-Eval\) and the highest informative Cohenκ\\kappa\(\+0\.092\+0\.092vs\.\+0\.026\+0\.026\);κ\\kappais the honest headline here because it discounts the trivial benefit of predicting the majority class on a16%16\\%\-positive label\. But the McNemar test againstG\-Evalis*not*significant \(p=0\.248p=0\.248\): on a hard binary label, a frozen 7B judge is near its agreement ceiling, and we do not claim a significant binary win\. We say so in the abstract and here\.

Finding 3: Majority is degenerate;κ\\kappaexposes it\.The Majority baseline posts the highest*accuracy*in the table \(0\.8390\.839\) purely by predicting “fail” for every trajectory on an imbalanced label—yet itsκ\\kappaandF1F\_\{1\}are both0, and McNemar versusRubricForgeis not significant \(p=0\.289p=0\.289\)\. This is the canonical reason accuracy alone is the wrong headline metric for reward\-free evaluation: a metric that never identifies a success is useless for ranking agents, whichκ\\kappacorrectly reflects and accuracy hides\.

Finding 4: FewShot and Heuristic are clearly worse\.Adding four in\-context examples without rubric evolution*hurts*\(0\.6130\.613accuracy,κ=−0\.019\\kappa=\-0\.019; McNemarp=0\.009p=0\.009,\*\*\): the demonstrations bias the judge toward over\-passing fluent traces \(its predicted\-positive rate jumps to0\.320\.32\)\. The non\-LLM Heuristic collapses to near\-zero accuracy \(p<0\.001p<0\.001,\*\*\*\)\. Neither is a competitive reward\-free evaluator\. These two significant gaps confirm the test has power; the non\-significantG\-Evalgap is therefore a genuine near\-tie on binary agreement, not low power—which is exactly why the next section moves the analysis to where induction does separate\.

Finding 5: The error decomposition shows the tie is one\-sided\.[Table˜IV](https://arxiv.org/html/2608.13564#S6.T4)unpacks each informative judge’s6262test verdicts into the full confusion count \(TP/FP/FN/TN\) together with its predicted\-positive ratep^\+\\hat\{p\}\_\{\+\}, false\-pass count, and false\-fail count\. The decomposition makes the “near\-tie” precise and reveals it is*asymmetric*\.RubricForgeandG\-Evalagree on their true\-positive count \(22\) and—critically—on their false\-fail count \(88each\): induction does*not*cost a single extra retry\. The whole difference is in false passes, which fall from99\(G\-Eval\) to66\(RubricForge\), dragging the predicted\-positive rate from0\.1770\.177toward the gold∼\\sim0\.160\.16\(0\.1290\.129forRubricForge\)\. FewShot moves the opposite way—its demonstrations pushp^\+\\hat\{p\}\_\{\+\}up to0\.3230\.323and its false passes to1717, the mechanism behind its significant accuracy loss\. Reading the table by*error direction*rather than by aggregate accuracy is what exposes that theG\-Evaltie is entirely a false\-pass tie thatRubricForgewins;[Figure˜4](https://arxiv.org/html/2608.13564#S7.F4)visualizes the same decomposition\.

TABLE IV:Error\-direction decomposition onτ\\tau\-benchtest \(n=62n=62;1010pass,5252fail\)\. Each row splits the judge’s verdicts into true/false positives/negatives, its predicted\-positive ratep^\+\\hat\{p\}\_\{\+\}, the deployment\-relevant false\-pass count/rate \([Equation˜4](https://arxiv.org/html/2608.13564#S3.E4)\), and the merely\-costly false\-fail count\.RubricForgeandG\-Evalshare an identical false\-fail count \(88\); the entire margin is the33fewer false passes, soRubricForgeis*never worse on any item*\. Degenerate predictors \(Heuristic, Majority\) are bracketed\. Best informative value per column inbold; forp^\+\\hat\{p\}\_\{\+\}“best” is closest to the gold positive rate0\.1610\.161\.
## VIIAnalysis: Faithfulness and Over\-Crediting

If binary agreement does not separateRubricForgefrom a generic judge, what does? This section answers with the faithfulness probe: the over\-crediting \(false\-pass\) rate and its case study, per\-criterion attribution, the ranking\-versus\-calibration split, and stratification by difficulty, fluency, and length\.

### VII\-AThe Over\-Crediting Probe

[Figure˜3](https://arxiv.org/html/2608.13564#S7.F3)reports the false\-pass rate \([Equation˜4](https://arxiv.org/html/2608.13564#S3.E4)\): how often each judge scores a*truly\-failed*trajectory as a success\. Onτ\\tau\-bench, of5252ground\-truth failuresRubricForgeover\-credits66\(FP=0\.115\\mathrm\{FP\}=0\.115\) versusG\-Eval’s99\(FP=0\.173\\mathrm\{FP\}=0\.173\)—about half as often\. Decisively, the two judges disagree on*exactly three*τ\\tau\-benchitems, and all three areG\-Evalfalse\-passes thatRubricForgecorrectly fails, with*zero*reversals \(no item whereRubricForgeover\-credits andG\-Evaldoes not\)\. The entire margin between the two judges in[Table˜III](https://arxiv.org/html/2608.13564#S6.T3)*is*these three over\-credit catches:RubricForgeis never worse thanG\-Evalon any individual item\. On gradedWebShopthe gap is narrower but in the same direction: of4747failuresRubricForgeover\-credits2222\(0\.4680\.468\) versusG\-Eval’s2323\(0\.4890\.489\), with22caught and11reversed\. The takeaway is the paper’s thesis in one number: induction’s benefit is concentrated in the dangerous error direction even where it is invisible in aggregate agreement\.

![Refer to caption](https://arxiv.org/html/2608.13564v1/x2.png)Figure 3:Over\-crediting \(false\-pass\) rate on truly\-failed trajectories\.RubricForgeover\-credits fluent\-but\-failed trajectories about half as often asG\-Evalonτ\\tau\-bench\(0\.1150\.115vs\.0\.1730\.173\) and slightly less often onWebShop\(0\.4680\.468vs\.0\.4890\.489\)\. Onτ\\tau\-benchthe only three judge disagreements are threeG\-Evalfalse\-passes thatRubricForgecatches, with zero reversals\.[Figure˜4](https://arxiv.org/html/2608.13564#S7.F4)recasts the same result as a two\-way error decomposition\. Stacking each judge’s6262test verdicts by error*direction*\(left\) shows the false\-fail bars are identical height \(88each\) while the false\-pass bar shrinks from99to66; theRubricForgeconfusion grid \(right\) makes explicit that the33caught cases all move out of the over\-credit \(FP\) cell and into the true\-fail \(TN\) cell, with the false\-fail \(FN\) cell untouched\. This is the visual statement of “harder to fool than to agree”: the only cell that changes is the dangerous one, and it changes in the safe direction\.

![Refer to caption](https://arxiv.org/html/2608.13564v1/x3.png)Figure 4:Error\-direction decomposition onτ\\tau\-benchtest \(n=62n=62\)\. Left: verdicts stacked by error type—both judges make the same88false fails \(amber\), so the only difference is the dangerous false\-pass bar \(red\), whichRubricForgecuts9→69\\\!\\to\\\!6\. Right:RubricForge’s confusion grid; the33caught cases leave the over\-credit \(FP\) cell for the true\-fail \(TN\) cell while the false\-fail \(FN\) cell is unchanged\.RubricForgeis thus never worse thanG\-Evalon any item\.
### VII\-BCase Study: The Three Caught Trajectories

Because theτ\\tau\-benchmargin reduces to three trajectories, we can inspect them exhaustively \([Table˜V](https://arxiv.org/html/2608.13564#S7.T5)\)\. Each is a fluent airline trajectory thatG\-Evalpasses with high confidence \(s^∈\[0\.85,0\.90\]\\hat\{s\}\\in\[0\.85,0\.90\]\) andRubricForgefails—and in each case the induced rubric pins the failure to a specific named criterion\. Inairline\|\|RSEA\|\|2the agent narrates a savings breakdown from downgrades but never commits the update;RubricForge’s*Request Fulfillment*criterion fires \(“did not confirm … or ensure the downgrades were completed”\)\. Inairline\|\|RSEA\|\|38the agent assures the user a certificate is “immediately available” though the order was never verified;*Order Verification*fires\. Inairline\|\|ReAct\|\|45thebook\_reservationcall returns “payment amount does not add up,” yet the agent reports “successfully booked”;*Successful Authentication*\(and, in the breakdown, Order Verification and Request Fulfillment\) fire\. In all three,G\-Evalrewards the confident final message;RubricForgereads the trajectory evidence the rubric tells it to look for\. This is the interpretability dividend of inducing*text*: the metric does not merely disagree, it says*which criterion*the trajectory violated\.

TABLE V:The only threeτ\\tau\-benchjudge disagreements \(allr⋆=r^\{\\star\}=fail\)\. In every caseG\-Eval=pass\(over\-credit\) andRubricForge=fail\(correct\), with the induced criterion that fired\.
### VII\-CPer\-Criterion Leave\-One\-Out Ablation

Which criteria carry the faithfulness? We re\-run the frozen judge with each of the five induced criteria removed in turn and re\-measure accuracy,κ\\kappa, and the false\-pass rate onτ\\tau\-benchtest \([Figure˜5](https://arxiv.org/html/2608.13564#S7.F5)\)\. Removing*Request Fulfillment*is the only ablation that clearly*degrades*faithfulness: accuracy falls0\.774→0\.7580\.774\\\!\\to\\\!0\.758,κ\\kappafalls\+0\.092→\+0\.068\+0\.092\\\!\\to\\\!\+0\.068, and the false\-pass rate*worsens*0\.115→0\.1350\.115\\\!\\to\\\!0\.135—this criterion, which demands explicit user confirmation that the requested action was completed, is what most resists over\-crediting\. Removing*Tool Usage*is mildly redundant: accuracy nudges to0\.7900\.790,κ\\kappato\+0\.118\+0\.118, and false\-pass to0\.0960\.096\(the tool\-error signal it captures is partly subsumed by the other criteria\)\. Removing Authentication, Order Verification, or User Communication leaves accuracy,κ\\kappa, and false\-pass exactly unchanged on this test set\. The rubric is thus robust to single\-criterion removal, with a clear faithfulness driver and one mildly redundant criterion—an interpretability statement no opaque judge can make\.

![Refer to caption](https://arxiv.org/html/2608.13564v1/x4.png)Figure 5:Per\-criterion leave\-one\-out onτ\\tau\-benchtest\. Removing*Request Fulfillment*worsens every faithfulness metric \(false\-pass0\.115→0\.1350\.115\\\!\\to\\\!0\.135,κ\\kappa\+0\.092→\+0\.068\+0\.092\\\!\\to\\\!\+0\.068\): it most drives faithfulness\. Removing*Tool Usage*is mildly redundant; the remaining criteria leave the test metrics unchanged\.
### VII\-DRanking versus Calibration

On gradedWebShopwe separate two often\-conflated notions of score quality:*ranking*\(does a higher judge score imply a higher true reward?\) and*absolute calibration*\(is the score numerically close to the reward?\)\.[Figure˜6](https://arxiv.org/html/2608.13564#S7.F6)shows they come apart, and honesty requires reporting both directions\.RubricForge*wins ranking*: Spearman0\.4100\.410\(Kendall0\.3470\.347\) versusG\-Eval’s0\.3700\.370\(Kendall0\.3140\.314\)—its scores order graded outcomes more faithfully, which is what one needs to compare or select agents\. ButG\-Evalis marginally better*calibrated*in absolute error: mean\|s^−r⋆\|\|\\hat\{s\}\-r^\{\\star\}\|is0\.2880\.288forG\-Evalversus0\.3360\.336forRubricForge, and a paired bootstrap on absolute error favorsG\-Evalby0\.0480\.048\(95%95\\%CI\[0\.018,0\.083\]\[0\.018,0\.083\],p=2×10−4p=2\\\!\\times\\\!10^\{\-4\}\)\. We do not paper over this:RubricForge’s evolved rubric tends toward more decisive\{0\.2,1\.0\}\\\{0\.2,1\.0\\\}\-style scores, which sharpens ranking but inflates absolute error against the smooth graded reward\. The two metrics measure different things, and a reward\-free evaluator used for*agent comparison*should be judged on ranking—whereRubricForgewins—while one used as an absolute score regressor should not\.

![Refer to caption](https://arxiv.org/html/2608.13564v1/x5.png)Figure 6:Ranking versus calibration on gradedWebShop\(n=56n=56\)\.RubricForgeranks graded outcomes better \(Spearman0\.4100\.410vs\.0\.3700\.370\) butG\-Evalis marginally better calibrated in absolute error \(mean\|s^−r⋆\|\|\\hat\{s\}\-r^\{\\star\}\|0\.2880\.288vs\.0\.3360\.336; paired\-bootstrapΔ=−0\.048\\Delta=\-0\.048,p=2×10−4p=2\\\!\\times\\\!10^\{\-4\}\)\. Ranking faithfulness and absolute calibration are distinct, and we report both honestly\.
### VII\-EStratification: Where the Advantage Lives

Finally we ask*where*RubricForge’s over\-crediting advantage concentrates, since a uniform gap and a localized one have different implications\. Stratifying theτ\\tau\-benchfalse\-pass gap \([Table˜VI](https://arxiv.org/html/2608.13564#S7.T6)\) shows the advantage is not uniform\. It concentrates in the*harder airline*domain \(false\-pass gap\+0\.19\+0\.19:G\-Eval0\.4380\.438vs\.RubricForge0\.2500\.250\), and within it in the*fluent*self\-evolvingairline\|\|RSEApolicy \(gap\+0\.25\+0\.25:G\-Eval0\.3750\.375vs\.RubricForge0\.1250\.125\), whose elaborated, confident narration is exactly what fools a generic judge\. The easier retail domain shows no gap \(both0\.0560\.056\): there, fluent\-failure over\-crediting is rare for either judge, so there is little to win\. Critically, the advantage is*not*a length artifact: in the longest\-trajectory tertile the false\-pass gap is0\.000\.00\(both judges over\-credit nothing there\), while the gap is largest in shorter, surface\-fluent traces\.RubricForge’s faithfulness edge is therefore driven by surface\-fluency×\\timestask\-difficulty—precisely the regime where a reward\-free evaluator is most likely to be fooled—rather than by trajectory length\.

TABLE VI:τ\\tau\-benchfalse\-pass rate stratified\. The over\-crediting advantage concentrates in the hard*airline*domain and the fluentRSEApolicy,*not*in the longest trajectories\. Gap==G\-Eval−\-RubricForge\(higher==largerRubricForgeadvantage\)\.

## VIIIDiscussion

The false\-pass rate is the deployment\-relevant metric\.The through\-line of our results is that for a reward\-free evaluator, aggregate agreement is the wrong headline and the false\-pass rate is the right one\. A metric that ships broken agents \(high false\-pass\) is dangerous regardless of its accuracy on an imbalanced label; a metric that occasionally discards a good sample \(false\-fail\) is merely inefficient\.RubricForgedoes not significantly win the former\-irrelevant binary\-agreement contest, and we say so plainly—but it nearly halves the false\-pass rate, the quantity that actually governs whether one can trust a reported success number\. “Harder to fool than to agree” is not a slogan covering a weak result; it is the precise, honest description of*which*axis induction improves\.

Grounding the objective resists Goodhart\.Why does inducing the rubric against labels help where it matters? Because it grounds the metric’s objective in true outcomes\. An ungrounded judge optimizes an internal notion of “looks successful,” which a fluent agent can satisfy without succeeding—a Goodharted metric drifting toward form\[[8](https://arxiv.org/html/2608.13564#bib.bib132),[7](https://arxiv.org/html/2608.13564#bib.bib133)\]\.RubricForge’s reflection step is repeatedly shown trajectories that*looked*successful but were labeled failures, and is pushed to articulate the observable evidence that distinguishes them—which is exactly an anti\-over\-crediting pressure\. This is the*grounded dual*of the sibling negative result \([Section˜II\-E](https://arxiv.org/html/2608.13564#S2.SS5)\): where an ungrounded prompt\-space meta\-objective*collapses*across users, a grounded evaluation objective confers faithfulness, and just as grounding an agent’s planning in checkable feedback sharpens it,RubricForgegrounds metric induction in checkable labels\. Grounding the objective, on whichever axis, is the common lever\.

A reward\-free evaluation protocol\.Beyond the method, we advocate the evaluation*protocol*as the reusable contribution\. Reporting an oracle\-reward ceiling, the false\-pass rate alongside agreement, a ranking\-versus\-calibration split, per\-criterion attribution, and fluency/difficulty stratification gives a far more honest picture of a reward\-free judge than a single accuracy or correlation number\. We recommend it for anyone deploying an LLM judge over agent trajectories, independent ofRubricForge\.

When induction helps, and threats to validity\.Induction helps most exactly where generic judges are most dangerous: hard tasks with fluent, confident agents\. It helps least where over\-crediting is already rare \(easy retail\)\. Several threats temper the generality of our claims\. The 7B judge is near its binary\-agreement ceiling, so the agreement story is genuinely a tie, not a latent win; the positive class onτ\\tau\-benchis small \(∼\\sim16%16\\%\), which is intrinsic to a hard binary success label but inflates the variance ofF1F\_\{1\}andκ\\kappa; and theWebShopcalibration loss is real and reported\. We also note that the renderer’s faithfulness—surfacing the right observable signals without leaking the reward—is a load\-bearing assumption we audited but that any deployment must re\-audit for its own trajectory format\.

## IXConclusion

We presentedRubricForge, a method that*induces*the text of an agent\-evaluation rubric by reflective evolution against ground\-truth\-labeled trajectories, freezes it, and applies it as a reward\-free metric with one frozen\-model call and no environment access\. Our central finding is deliberately honest and contrarian: on a hard binary success label an induced rubric does*not*significantly out\-agree a generic LLM judge \(McNemarp=0\.248p=0\.248\), and it is marginally worse calibrated in absolute graded score\. Where it pays off is*faithfulness*: it over\-credits fluent\-but\-failed trajectories about half as often \(0\.1150\.115vs\.0\.1730\.173false\-pass onτ\\tau\-bench; three catches, zero reversals\), ranks graded outcomes better \(Spearman0\.4100\.410vs\.0\.3700\.370\), and—because the artifact is human\-readable text—attributes every verdict to the named criterion responsible\. For a reward\-free evaluator, where a false pass ships a broken agent and a false fail merely costs a retry, this is the metric that matters:RubricForgeis*harder to fool than to agree*\. By grounding the evaluation objective in labeled outcomes, induction resists the Goodhart drift toward fluency that an ungrounded judge silently suffers—the grounded dual of the sibling meta\-objective\-collapse result\. We release the induction method and a reward\-free evaluation protocol \(oracle ceiling, false\-pass probe, ranking–calibration split, per\-criterion attribution\) as a step toward trustworthy automatic evaluation of language\-model agents\.

Limitations\.\(i\)*Single frozen 7B judge\.*All results use one frozen Qwen2\.5\-7B model as both agent and judge; whether the faithfulness advantage transfers to a stronger or larger judge \(e\.g\. 30B or a frontier model\) is future work\. We expect it to transfer because the induced rubric is backbone\-agnostic*text*—it can be dropped into any judge—but we do not claim it here\. \(ii\)*Two benchmarks\.*We evaluate onτ\\tau\-benchandWebShop; broader validation on AgentBench, WebArena, GAIA, and code agents remains open\. \(iii\)*Small positive class / binary\-agreement tie\.*The∼\\sim16%16\\%τ\\tau\-benchpositive rate is intrinsic to a hard binary label and widens the CIs onF1F\_\{1\}/κ\\kappa; correspondingly the binary\-agreement edge overG\-Evalis directional and not significant, and we do not over\-claim it\. \(iv\)*Calibration\.*RubricForgeranks graded outcomes better but is marginally worse in absolute calibration, so it should be used for agent comparison, not as an absolute score regressor\. \(v\)*Induction supervision\.*Induction needs a modest pool of labeled trajectories; in the fully label\-free regime the method reduces to its generic seed\.

## Appendix AInduced Rubrics, Sweep, and Hyperparameters

Induction hyperparameters\.GEPA reflective evolution \(reused unchanged\[[1](https://arxiv.org/html/2608.13564#bib.bib17)\]\) with minibatch44and seed0; budget200200rollout calls onτ\\tau\-bench\(the run used208208judging calls and accepted a pool of66rubrics, best validation agreement0\.7690\.769at20/2620/26\) and150150onWebShop\(156156calls, pool of44, best validation0\.7500\.750at18/2418/24\)\. The seed candidate is the generic G\-Eval\-style rubric, so accepted rubrics must beat a competent judge\. Judge calls are deterministic \(temperature0, max220220tokens\) with a robust\-JSON parse and a conservativep^=0\\hat\{p\}\{=\}0fallback on any parse/network failure\.

Full validation curves\.[Table˜VII](https://arxiv.org/html/2608.13564#A1.T7)lists every rubric in the accepted pool with its validation agreement, and[Figure˜7](https://arxiv.org/html/2608.13564#A1.F7)plots the same trajectory\.τ\\tau\-bench: seed17/2617/26, then accepted children at19,19,18,20,1819,19,18,20,18\(/26/26\), best20/26=0\.76920/26=0\.769\.WebShop: seed16/2416/24, then17,14,1817,14,18\(/24/24\), best18/24=0\.75018/24=0\.750; the corresponding graded\-test ranking\-vs\-calibration summary is reported in[Table˜VIII](https://arxiv.org/html/2608.13564#A1.T8)\. In both domains every accepted child except one sits at or above the generic seed, and the selectedρ⋆\\rho^\{\\star\}improves the seed by\+11\.5\+11\.5and\+8\.3\+8\.3validation points respectively—confirming that induction beats a competent starting rubric rather than a strawman, while the modest absolute gains are consistent with the frozen 7B judge operating near its agreement ceiling\.

TABLE VII:Induction sweep: validation agreement of every rubric in the accepted pool \(counts out of the validation split;2626forτ\\tau\-bench,2424forWebShop\)\.ρ0\\rho\_\{0\}is the generic G\-Eval seed;ρ⋆\\rho^\{\\star\}\(the selected, frozen rubric\) is inbold\. Children are listed in acceptance order\.![Refer to caption](https://arxiv.org/html/2608.13564v1/x6.png)Figure 7:Validation\-agreement trajectory of rubric induction\. Grey is the generic G\-Eval seed \(ρ0\\rho\_\{0\}\); green bars are accepted rubrics in the pool; the selectedρ⋆\\rho^\{\\star\}\(outlined\) lifts validation agreement0\.654→0\.7690\.654\\\!\\to\\\!0\.769onτ\\tau\-benchand0\.667→0\.7500\.667\\\!\\to\\\!0\.750onWebShop\. The induced rubric must beat a competent seed, not a degenerate one\.TABLE VIII:WebShopgraded\-test summary \(n=56n=56\)\.RubricForgewins ranking;G\-Evalwins absolute calibration \(paired\-bootstrapΔ​\|err\|=−0\.048\\Delta\|\{\\rm err\}\|=\-0\.048,95%95\\%CI\[−0\.083,−0\.018\]\[\-0\.083,\-0\.018\],p=2×10−4p=2\\\!\\times\\\!10^\{\-4\}\)\.The frozenτ\\tau\-benchrubric\.The induced rubric applied at test time \(verbatim, lightly truncated\) is:

> 1\. Successful Authentication: the agent must successfully authenticate the user using at least one valid method; any errors or repeated failed attempts indicate failed authentication\. 2\. Order Verification: the agent must retrieve and verify the user’s order details; any “order not found” or repeated lookups indicate failure\. 3\. Request Fulfillment: the agent must correctly fulfill the request \(cancel, modify, return, exchange\) under policy, with explicit user confirmation; any unfulfilled request indicates failure\. 4\. Tool Usage: the agent must call the appropriate tools without errors; tool\-call errors reduce the score\. 5\. User Communication: the agent must maintain clear, relevant communication throughout the interaction\.

The frozenWebShoprubric \(summary\)\.Four criteria: \(1\) the search query broadly aligns with the instruction’s key attributes; \(2\) the purchased product’s attributes \(color, size, style\) match the request, allowing only explicitly\-justified minor variances; \(3\) the purchase respects the price limit; \(4\) the agent completesBuy Nowwith final options matching its selections\. Failure is indicated if any criterion is unmet\.

Reproducibility\.All verdicts are deterministic given the frozen backbone, the frozen rubric, and the fixed by\-task split \(seed0\)\. The agreement table, McNemar tests, bootstrap CIs, the over\-crediting probe, the leave\-one\-out ablation, the calibration split, and the stratification are computed from the persisted per\-item predictions on the held\-out test split\.

## References

- \[1\]L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang,et al\.\(2025\)GEPA: reflective prompt evolution can outperform reinforcement learning\.arXiv preprint arXiv:2507\.19457\.Cited by:[Appendix A](https://arxiv.org/html/2608.13564#A1.p1.16),[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p1.1),[§IV\-A](https://arxiv.org/html/2608.13564#S4.SS1.p1.2)\.
- \[2\]\(2016\)Concrete problems in AI safety\.arXiv preprint arXiv:1606\.06565\.Cited by:[§II\-E](https://arxiv.org/html/2608.13564#S2.SS5.p1.1)\.
- \[3\]A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. Hajishirzi\(2024\)Self\-RAG: learning to retrieve, generate, and critique through self\-reflection\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.11511Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[4\]Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon,et al\.\(2022\)Constitutional AI: harmlessness from AI feedback\.arXiv preprint\.Note:arXiv:2212\.08073Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[5\]C\. Fernando, D\. Banarse, H\. Michalewski, S\. Osindero, and T\. Rocktäschel\(2023\)Promptbreeder: self\-referential self\-improvement via prompt evolution\.arXiv preprint\.Note:arXiv:2309\.16797Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p1.1)\.
- \[6\]C\. Finn, P\. Abbeel, and S\. Levine\(2017\)Model\-agnostic meta\-learning for fast adaptation of deep networks\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:1703\.03400Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[7\]L\. Gao, J\. Schulman, and J\. Hilton\(2023\)Scaling laws for reward model overoptimization\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2210\.10760Cited by:[§II\-E](https://arxiv.org/html/2608.13564#S2.SS5.p1.1),[§VIII](https://arxiv.org/html/2608.13564#S8.p2.1)\.
- \[8\]C\. A\. E\. Goodhart\(1984\)Problems of monetary management: the UK experience\.Monetary Theory and Practice: The UK Experience,pp\. 91–121\.Cited by:[§II\-E](https://arxiv.org/html/2608.13564#S2.SS5.p1.1),[§VIII](https://arxiv.org/html/2608.13564#S8.p2.1)\.
- \[9\]Z\. Gou, Z\. Shao, Y\. Gong, Y\. Shen, Y\. Yang, N\. Duan, and W\. Chen\(2023\)CRITIC: large language models can self\-correct with tool\-interactive critiquing\.arXiv preprint\.Note:arXiv:2305\.11738Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[10\]J\. Gu, X\. Jiang, Z\. Shi, H\. Tan, X\. Zhai, C\. Xu, W\. Li, Y\. Shen, S\. Ma, H\. Liu, S\. Wang, K\. Zhang, Y\. Wang, W\. Gao, L\. Ni, and J\. Guo\(2024\)A survey on LLM\-as\-a\-judge\.arXiv preprint arXiv:2411\.15594\.Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[11\]T\. M\. Hospedales, A\. Antoniou, P\. Micaelli, and A\. J\. Storkey\(2022\)Meta\-learning in neural networks: a survey\.IEEE Transactions on Pattern Analysis and Machine Intelligence \(TPAMI\)44\(9\),pp\. 5149–5169\.Note:arXiv:2004\.05439Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[12\]J\. Huang, S\. S\. Gu, L\. Hou, Y\. Wu, X\. Wang, H\. Yu, and J\. Han\(2022\)Large language models can self\-improve\.arXiv preprint\.Note:arXiv:2210\.11610Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[13\]J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. W\. Yu, X\. Song, and D\. Zhou\(2024\)Large language models cannot self\-correct reasoning yet\.International Conference on Learning Representations \(ICLR\)\.Note:arXiv:2310\.01798Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[14\]C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. Narasimhan\(2024\)SWE\-bench: can language models resolve real\-world GitHub issues?\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.06770Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[15\]R\. Kamoi, Y\. Zhang, N\. Zhang, J\. Han, and R\. Zhang\(2024\)When can LLMs actually correct their own mistakes? a critical survey of self\-correction of LLMs\.Transactions of the Association for Computational Linguistics \(TACL\)12,pp\. 1417–1440\.Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[16\]V\. Karpukhin, B\. Oğuz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. Yih\(2020\)Dense passage retrieval for open\-domain question answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 6769–6781\.Note:arXiv:2004\.04906Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[17\]O\. Khattab, K\. Santhanam, X\. L\. Li, D\. Hall, P\. Liang, C\. Potts, and M\. Zaharia\(2022\)Demonstrate\-search\-predict: composing retrieval and language models for knowledge\-intensive NLP\.arXiv preprint\.Note:arXiv:2212\.14024Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p1.1)\.
- \[18\]O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang,et al\.\(2024\)DSPy: compiling declarative language model calls into self\-improving pipelines\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p1.1)\.
- \[19\]S\. Kim, J\. Shin, Y\. Cho, J\. Jang, S\. Longpre, H\. Lee, S\. Yun, S\. Shin, S\. Kim, J\. Thorne, and M\. Seo\(2024\)Prometheus: inducing fine\-grained evaluation capability in language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.08491Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[20\]S\. Kim, J\. Suk, S\. Longpre, B\. Y\. Lin, J\. Shin, S\. Welleck, G\. Neubig, M\. Lee, K\. Lee, and M\. Seo\(2024\)Prometheus 2: an open source language model specialized in evaluating other language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 4334–4353\.Note:arXiv:2405\.01535External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.248)Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[21\]N\. Lambert, V\. Pyatkin, J\. Morrison, L\. Miranda, B\. Y\. Lin, K\. Chandu, N\. Dziri, S\. Kumar, T\. Zick, Y\. Choi, N\. A\. Smith, and H\. Hajishirzi\(2024\)RewardBench: evaluating reward models for language modeling\.Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[22\]H\. Lee, S\. Phatale, H\. Mansoor, T\. Mesnard, J\. Ferret, K\. Lu, C\. Bishop, E\. Hall, V\. Carbune, A\. Rastogi, and S\. Prakash\(2024\)RLAIF vs\. RLHF: scaling reinforcement learning from human feedback with AI feedback\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Note:arXiv:2309\.00267Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[23\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela\(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2005\.11401Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[24\]D\. Li, B\. Jiang, L\. Huang, A\. Beigi, C\. Zhao, Z\. Tan, A\. Bhattacharjee, Y\. Jiang, C\. Chen, T\. Wu, K\. Shu, L\. Cheng, and H\. Liu\(2024\)From generation to judgment: opportunities and challenges of LLM\-as\-a\-judge\.arXiv preprint arXiv:2411\.16594\.Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[25\]H\. Li, Q\. Dong, J\. Chen, H\. Su, Y\. Zhou, Q\. Ai, Z\. Ye, and Y\. Liu\(2024\)LLMs\-as\-judges: a comprehensive survey on LLM\-based evaluation methods\.arXiv preprint arXiv:2412\.05579\.Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[26\]J\. Li, S\. Sun, W\. Yuan, R\. Fan, H\. Zhao, and P\. Liu\(2024\)Generative judge for evaluating alignment\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.05470\. The proposed evaluator is named Auto\-J\.Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[27\]X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang, S\. Zhang, X\. Deng, A\. Zeng, Z\. Du, C\. Zhang, S\. Shen, T\. Zhang, Y\. Su, H\. Sun, M\. Huang, Y\. Dong, and J\. Tang\(2024\)AgentBench: evaluating LLMs as agents\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2308\.03688Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[28\]Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu\(2023\)G\-Eval: NLG evaluation using GPT\-4 with better human alignment\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 2511–2522\.Note:arXiv:2303\.16634External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.153)Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§I](https://arxiv.org/html/2608.13564#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1),[§V](https://arxiv.org/html/2608.13564#S5.p4.2)\.
- \[29\]A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[30\]G\. Mialon, C\. Fourrier, C\. Swift, T\. Wolf, Y\. LeCun, and T\. Scialom\(2024\)GAIA: a benchmark for general AI assistants\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2311\.12983Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[31\]S\. Min, M\. Lewis, L\. Zettlemoyer, and H\. Hajishirzi\(2022\)MetaICL: learning to learn in context\.InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\),Note:arXiv:2110\.15943Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[32\]N\. Muennighoff, Z\. Yang, W\. Shi, X\. L\. Li, L\. Fei\-Fei, H\. Hajishirzi, L\. Zettlemoyer, P\. Liang, E\. Candès, and T\. Hashimoto\(2025\)S1: simple test\-time scaling\.arXiv preprint\.Note:arXiv:2501\.19393Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[33\]A\. Nichol, J\. Achiam, and J\. Schulman\(2018\)On first\-order meta\-learning algorithms\.arXiv preprint\.Note:arXiv:1803\.02999Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[34\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. Christiano, J\. Leike, and R\. Lowe\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2203\.02155Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[35\]C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez\(2023\)MemGPT: towards LLMs as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[36\]A\. Pan, K\. Bhatia, and J\. Steinhardt\(2022\)The effects of reward misspecification: mapping and mitigating misaligned models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2201\.03544Cited by:[§II\-E](https://arxiv.org/html/2608.13564#S2.SS5.p1.1)\.
- \[37\]A\. Panickssery, S\. R\. Bowman, and S\. Feng\(2024\)LLM evaluators recognize and favor their own generations\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.37\.Note:arXiv:2404\.13076Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p2.1),[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p2.1)\.
- \[38\]J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein\(2023\)Generative agents: interactive simulacra of human behavior\.InACM Symposium on User Interface Software and Technology \(UIST\),Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[39\]R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. Finn\(2023\)Direct preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2305\.18290Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[40\]S\. Saha, O\. Levy, A\. Celikyilmaz, M\. Bansal, J\. Weston, and X\. Li\(2023\)Branch\-solve\-merge improves large language model evaluation and generation\.arXiv preprint arXiv:2310\.15123\.Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[41\]V\. Samuel, H\. P\. Zou, Y\. Zhou, S\. Chaudhari, A\. Kalyan, T\. Rajpurohit, A\. Deshpande, K\. Narasimhan, and V\. Murahari\(2024\)PersonaGym: evaluating persona agents and LLMs\.arXiv preprint\.Note:arXiv:2407\.18416Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[42\]W\. Shi, S\. Min, M\. Yasunaga, M\. Seo, R\. James, M\. Lewis, L\. Zettlemoyer, and W\. Yih\(2024\)REPLUG: retrieval\-augmented black\-box language models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\),Note:arXiv:2301\.12652Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[43\]N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. Yao\(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[44\]J\. Skalse, N\. H\. R\. Howe, D\. Krasheninnikov, and D\. Krueger\(2022\)Defining and characterizing reward hacking\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2209\.13085Cited by:[§II\-E](https://arxiv.org/html/2608.13564#S2.SS5.p1.1)\.
- \[45\]C\. Snell, J\. Lee, K\. Xu, and A\. Kumar\(2024\)Scaling LLM test\-time compute optimally can be more effective than scaling model parameters\.arXiv preprint\.Note:arXiv:2408\.03314Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[46\]N\. Stiennon, L\. Ouyang, J\. Wu, D\. M\. Ziegler, R\. Lowe, C\. Voss, A\. Radford, D\. Amodei, and P\. Christiano\(2020\)Learning to summarize from human feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2009\.01325Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[47\]H\. Su, J\. Luo, C\. Liu, X\. Yang, Y\. Zhang, Y\. Dong, and J\. Zhu\(2025\)A survey on autonomy\-induced security risks in large model\-based agents\.arXiv preprint arXiv:2506\.23844\.Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[48\]Y\. Sun, X\. Wang, Z\. Liu, J\. Miller, A\. A\. Efros, and M\. Hardt\(2020\)Test\-time training with self\-supervision for generalization under distribution shifts\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:1909\.13231Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[49\]Q\. Tu, S\. Fan, Z\. Tian, and R\. Yan\(2024\)CharacterEval: a chinese benchmark for role\-playing conversational agent evaluation\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),Note:arXiv:2401\.01275Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[50\]P\. Verga, S\. Hofstatter, S\. Althammer, Y\. Su, A\. Piktus, A\. Arkhangorodsky, M\. Xu, N\. White, and P\. Lewis\(2024\)Replacing judges with juries: evaluating LLM generations with a panel of diverse models\.arXiv preprint arXiv:2404\.18796\.Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[51\]D\. Wang, E\. Shelhamer, S\. Liu, B\. Olshausen, and T\. Darrell\(2021\)Tent: fully test\-time adaptation by entropy minimization\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2006\.10726Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[52\]G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. Anandkumar\(2023\)Voyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[53\]P\. Wang, L\. Li, L\. Chen, Z\. Cai, D\. Zhu, B\. Lin, Y\. Cao, L\. Kong, Q\. Liu, T\. Liu, and Z\. Sui\(2024\)Large language models are not fair evaluators\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\), Volume 1: Long Papers,pp\. 9440–9450\.Note:arXiv:2305\.17926External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.511)Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p2.1),[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p2.1)\.
- \[54\]X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou\(2023\)Self\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2203\.11171Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p2.1)\.
- \[55\]Y\. Wang, Z\. Yu, Z\. Zeng, L\. Yang, C\. Wang, H\. Chen, C\. Jiang, R\. Xie, J\. Wang, X\. Xie, W\. Ye, S\. Zhang, and Y\. Zhang\(2024\)PandaLM: an automatic evaluation benchmark for LLM instruction tuning optimization\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2306\.05087Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[56\]Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. Neubig\(2024\)Agent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[57\]T\. Wu, W\. Yuan, O\. Golovneva, J\. Xu, Y\. Tian, J\. Jiao, J\. Weston, and S\. Sukhbaatar\(2024\)Meta\-rewarding language models: self\-improving alignment with LLM\-as\-a\-meta\-judge\.arXiv preprint\.Note:arXiv:2407\.19594Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[58\]C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. Chen\(2024\)Large language models as optimizers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p1.1)\.
- \[59\]S\. Yang, S\. C\. Han, Y\. Ding, S\. Wang, and E\. Hovy\(2026\)ToolTree: efficient LLM agent tool planning via dual\-feedback monte carlo tree search and bidirectional pruning\.Note:ICLR 2026External Links:2603\.12740Cited by:[§II\-E](https://arxiv.org/html/2608.13564#S2.SS5.p1.1)\.
- \[60\]S\. Yang, S\. C\. Han, S\. Wang, Y\. Li, Y\. Ding, and E\. Hovy\(2026\)Toward understanding misalignment in LLM agents: a survey of taxonomy, causes, mitigation, and evaluation\.Note:ACL ARR 2026External Links:[Link](https://openreview.net/forum?id=zzTEGP2BYa)Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p2.1)\.
- \[61\]X\. Yang, J\. Zou, R\. Pan, R\. Qiu, P\. Lu, S\. Diao, J\. Jiang, H\. Tong, T\. Zhang, M\. J\. Buehler, J\. He, and J\. Zou\(2026\)Recursive multi\-agent systems\.arXiv preprint arXiv:2604\.25917\.Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[62\]S\. Yao, H\. Chen, J\. Yang, and K\. Narasimhan\(2022\)WebShop: towards scalable real\-world web interaction with grounded language agents\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2207\.01206Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1),[§III](https://arxiv.org/html/2608.13564#S3.p2.5),[§V](https://arxiv.org/html/2608.13564#S5.p1.10)\.
- \[63\]S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhan\(2024\)τ\\tau\-bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.arXiv preprint arXiv:2406\.12045\.Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1),[§III](https://arxiv.org/html/2608.13564#S3.p2.5),[§V](https://arxiv.org/html/2608.13564#S5.p1.10)\.
- \[64\]S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. Narasimhan\(2023\)Tree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2305\.10601Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[65\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1),[§V](https://arxiv.org/html/2608.13564#S5.p1.10)\.
- \[66\]S\. Ye, D\. Kim, S\. Kim, H\. Hwang, S\. Kim, Y\. Jo, J\. Thorne, J\. Kim, and M\. Seo\(2024\)FLASK: fine\-grained language model evaluation based on alignment skill sets\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2307\.10928Cited by:[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[67\]W\. Yuan, R\. Y\. Pang, K\. Cho, S\. Sukhbaatar, J\. Xu, and J\. Weston\(2024\)Self\-rewarding language models\.arXiv preprint\.Note:arXiv:2401\.10020Cited by:[§II\-B](https://arxiv.org/html/2608.13564#S2.SS2.p1.1)\.
- \[68\]E\. Zelikman, Y\. Wu, J\. Mu, and N\. D\. Goodman\(2022\)STaR: bootstrapping reasoning with reasoning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[69\]A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. Huang\(2024\)ExpeL: LLM agents are experiential learners\.InAAAI Conference on Artificial Intelligence,Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[70\]S\. Zhao, J\. Dang, and A\. Grover\(2024\)Group preference optimization: few\-shot alignment of large language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.11523Cited by:[§II\-E](https://arxiv.org/html/2608.13564#S2.SS5.p1.1)\.
- \[71\]L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica\(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.InAdvances in Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track,Vol\.36\.Note:arXiv:2306\.05685Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p1.1),[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.
- \[72\]S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. Neubig\(2024\)WebArena: a realistic web environment for building autonomous agents\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2307\.13854Cited by:[§II\-C](https://arxiv.org/html/2608.13564#S2.SS3.p1.1)\.
- \[73\]Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. Ba\(2023\)Large language models are human\-level prompt engineers\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2211\.01910Cited by:[§II\-D](https://arxiv.org/html/2608.13564#S2.SS4.p1.1)\.
- \[74\]L\. Zhu, X\. Wang, and X\. Wang\(2025\)JudgeLM: fine\-tuned large language models are scalable judges\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.17631Cited by:[§I](https://arxiv.org/html/2608.13564#S1.p3.1),[§II\-A](https://arxiv.org/html/2608.13564#S2.SS1.p1.1)\.