Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents

# Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents
Source: [https://arxiv.org/html/2607.12790](https://arxiv.org/html/2607.12790)
Xing Zhang1, Guanghui Wang1, Yanwei Cui1, Ziyuan Li2, Wei Qiu2, Bing Zhu2, Peiyang He1

###### Abstract

Self\-evolving agent systems improve by creating, revising, and retiring their own skills, but every such loop rests on a hidden assumption: a reliable evaluation metric already exists\. In many real applications it does not\. We make three claims\. First, metrics can be*evolved*: our metric loop searches compositions of small drawback detectors under a full evolutionary lifecycle, trained to agree with a ten\-item anchored reference set, regularized by consensus over unlabeled outputs, and audited against a held\-out anchor it never reads, yielding a transparent, inspectable metric rather than an opaque judge\. Second, since no metric exists to beat, the yardstick is recovering what an accurate metric would have enabled, and*Double Ratchet*, our co\-evolution of the metric with a lifecycle\-managed skill loop, does so: across code generation \(MBPP\+\), enterprise text\-to\-SQL \(Spider 2\.0\-Snow\), and reference\-free report generation, it retains 88–110% of the held\-out lift achieved by the same skill loop driven by ground truth or the best available rubric\. Third, safety comes from anchor discipline plus outer audits: removing anchor guards collapses the metric into a vacuous detector while removing the lifecycle does not; and when evolved skills gamed the report rubric, an independent judge caught it, one detector repaired it, and a task\-aware judge then preferred the evolved outputs over the pre\-evolution baseline in 77% of decided pairs\. We argue this failure\-expecting architecture is the right default wherever no reliable automatic verifier exists\.

## Introduction

Agent self\-evolution is now a standard recipe: a frozen language model is wrapped in a loop that writes skills, retrieves them, measures their contribution, and retires the ones that hurt\(Zhang et al\.[2026b](https://arxiv.org/html/2607.12790#bib.bib44); Wang et al\.[2024a](https://arxiv.org/html/2607.12790#bib.bib32); Shinn et al\.[2023](https://arxiv.org/html/2607.12790#bib.bib30)\)\. The loop is simple and effective, but every round consumes one expensive ingredient: an evaluation signal that decides whether a task attempt passed or failed\. On code benchmarks this signal is free, since unit tests exist\. In deployed systems it is usually the bottleneck, and the industry’s own products document it: Claude’s managed agents have developers hand\-write outcome rubrics for a separate grader\(Anthropic[2026](https://arxiv.org/html/2607.12790#bib.bib2)\), and Codex’s Record & Replay has a human demonstrate each workflow once\(OpenAI[2026](https://arxiv.org/html/2607.12790#bib.bib25)\)\. Both are manual substitutes for a missing metric, one rubric or demonstration at a time\. This paper automates that step\.

The question we ask is direct: if skills can be evolved, why not the metric? One clarification up front, because it determines how every result should be read: in our setting no accurate metric exists, so there is no baseline metric to beat, and the correct yardstick is how much of the performance an accurate metric*would have enabled*a system can recover without one\. Matching a ground\-truth\-driven oracle is the win condition, not a consolation\. Our starting hypothesis comes from the skill side: the Ratchet skill\-evolution system\(Zhang et al\.[2026b](https://arxiv.org/html/2607.12790#bib.bib44),[a](https://arxiv.org/html/2607.12790#bib.bib43)\)found that unmanaged LLM\-authored skill libraries drift, and that lifecycle management is what makes skill evolution work at all\. We posit the same discipline is necessary for evaluators, and treat the evaluator as a first\-class evolving object under a full lifecycle \(Figure[1](https://arxiv.org/html/2607.12790#Sx1.F1)\)\. Concretely, the metric is an expression tree over a pool of small drawback detectors, each checking one failure class of one task family; detectors are synthesized from clustered failures, gated at birth, scored by marginal contribution, and retired when useless\. Selection never asks whether the agent scores well: it asks whether the metric agrees with a small anchored reference set and with detector consensus on a large unlabeled set\. Two ablations, judged on metric validity rather than task score, locate which of the two ingredients \(anchor guards versus detector lifecycle\) carries the load\.

The design rests on one stance about evaluation: we rarely know what good is, but given an output we can usually find drawbacks, so a clean verdict means no known drawback was found, not certified correctness\. This framing is what makes bootstrap possible, and composing deterministic detectors, which fail differently from the LLM being evaluated, resists the shared\-blind\-spot collusion a bare LLM judge invites\.

A metric that grades the same loop that produced it is an obvious Goodhart hazard\(Goodhart[1984](https://arxiv.org/html/2607.12790#bib.bib14); Gao, Schulman, and Hilton[2023](https://arxiv.org/html/2607.12790#bib.bib13)\)\. We adopt a strict anchor discipline: a tiny development set with golden references supplies soft labels and is the only supervised signal; a locked set is never read by any loop and purely reports transfer; and when metric and skill bank co\-evolve, the metric grades only the skill loop’s training tasks while its evaluation stays pinned to the locked anchor\. A deliberately unsafe ablation with the guards disabled collapses in the way the theory predicts\.

We validate on three task areas arranged around this yardstick\. MBPP\+ code generation and Spider 2\.0\-Snow text\-to\-SQL have exact ground truth, which lets us measure the gap directly: run the oracle loop with the true metric, run the co\-loop without it, and see how much lift survives\. Report generation is the intended target: no golden metric exists, and its rubric score \(RAQS\) is a partial workaround we treat as reference signal only, so retention there is measured against an imperfect reference\. There we add the layer a real deployment would keep: an independent, stronger LLM judge compares final outputs pairwise against the pre\-evolution baseline\. This audit caught a real Goodhart failure \(skills gaming the rubric’s tag counter\), one added detector repaired it, and auditing the judge itself exposed a second failure mode; the results report the episode as a2×22\\times 2\. Verifier\-free self\-improvement is feasible, but only inside an architecture that expects its own metric to be gamed and its own judge to be miscalibrated, and can catch both\.

![Refer to caption](https://arxiv.org/html/2607.12790v1/x1.png)Figure 1:Double Ratchet architecture\. The metric loop \(left\) evolves an expression over drawback detectors; the skill loop \(right\) evolves the agent’s skill bank, its training graded by the evolved metric during co\-evolution\. Anchors \(bottom\) are never trained on; an independent judge audits final outputs on reference\-free tasks\.Our contributions are as follows\.

- •We formulate evaluation\-metric evolution as anchored search over compositions of atomic drawback detectors under a full lifecycle, transferring the lifecycle\-is\-the\-key lesson from skill evolution to evaluators\.
- •We give a leakage\-safe three\-split protocol and a selection objective that combines tiny\-anchor agreement with unlabeled consensus; ablations show the anchor guards are what keep the evolved metric valid \(non\-vacuous\), while the detector lifecycle contributes efficiency\.
- •We co\-evolve the metric with a skill loop, yielding*Double Ratchet*, which recovers 88–110% of the oracle or rubric\-driven lift across code, SQL, and report writing while producing an inspectable metric as a by\-product\.
- •We document an end\-to\-end Goodhart detection and repair on the reference\-free task, using an independent final judge as the outer audit\.

## Related Work

#### Self\-evolving LLM agents\.

Voyager grows a skill library for embodied tasks\(Wang et al\.[2024a](https://arxiv.org/html/2607.12790#bib.bib32)\), Reflexion and Self\-Refine turn failure traces and self\-feedback into revision\(Shinn et al\.[2023](https://arxiv.org/html/2607.12790#bib.bib30); Madaan et al\.[2023](https://arxiv.org/html/2607.12790#bib.bib22)\), and later systems accumulate reusable experience as insights, manuals, memories, or workflows\(Zhao et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib46); Chen et al\.[2024b](https://arxiv.org/html/2607.12790#bib.bib6); Park et al\.[2023](https://arxiv.org/html/2607.12790#bib.bib28); Wang et al\.[2024c](https://arxiv.org/html/2607.12790#bib.bib35)\), a landscape two surveys map\(Zhang et al\.[2025](https://arxiv.org/html/2607.12790#bib.bib42); Wang et al\.[2024b](https://arxiv.org/html/2607.12790#bib.bib34)\)\. A recent wave evolves, analyzes, and benchmarks skill libraries\(Wu et al\.[2025](https://arxiv.org/html/2607.12790#bib.bib36); Yang et al\.[2026](https://arxiv.org/html/2607.12790#bib.bib39); Xia et al\.[2026](https://arxiv.org/html/2607.12790#bib.bib37); Ni et al\.[2026](https://arxiv.org/html/2607.12790#bib.bib24); Wang, Ren, and Zhang[2026](https://arxiv.org/html/2607.12790#bib.bib33); Zhang et al\.[2026c](https://arxiv.org/html/2607.12790#bib.bib45); Li et al\.[2026](https://arxiv.org/html/2607.12790#bib.bib19)\), and Ratchet\(Zhang et al\.[2026b](https://arxiv.org/html/2607.12790#bib.bib44),[a](https://arxiv.org/html/2607.12790#bib.bib43)\)showed that unmanaged libraries drift, with a minimal lifecycle of retirement, bounded capacity, and meta\-skill guidance restoring large gains\. Automated agent design searches over agent code itself\(Hu, Lu, and Clune[2025](https://arxiv.org/html/2607.12790#bib.bib16); Elsken, Metzen, and Hutter[2019](https://arxiv.org/html/2607.12790#bib.bib11)\), and DSPy, OPRO, Promptbreeder, and TextGrad optimize prompts and pipelines against a metric\(Khattab et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib17); Yang et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib38); Fernando et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib12); Yuksekgonul et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib41)\)\. All of these assume the evaluation signal is given; we evolve it, and test which part of the lifecycle discipline carries the safety load for evaluators\.

#### Learned and searched objectives\.

CSE\-Autoloss evolves loss functions from primitive operations\(Liu et al\.[2021](https://arxiv.org/html/2607.12790#bib.bib21)\), AutoML\-Zero evolves whole learning algorithms\(Real et al\.[2020](https://arxiv.org/html/2607.12790#bib.bib29)\), and Pruner\-Zero evolves symbolic pruning metrics\(Dong et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib9)\)\. Our search space is analogous, primitive checks composed into a program, but the fitness is different in kind: a loss is scored by downstream accuracy on a fixed metric, while an evaluator has no downstream metric to lean on, so we score it by agreement with a small anchor plus consensus regularization and certify it on a locked anchor\.

#### Reward modeling and its failure modes\.

Learned reward signals are notoriously fragile: reward design is delicate even in classical RL\(Ng, Harada, and Russell[1999](https://arxiv.org/html/2607.12790#bib.bib23)\), RLHF reward models are over\-optimized by the policies they train\(Ouyang et al\.[2022](https://arxiv.org/html/2607.12790#bib.bib26); Gao, Schulman, and Hilton[2023](https://arxiv.org/html/2607.12790#bib.bib13)\), misspecification produces systematic gaming\(Pan, Bhatia, and Steinhardt[2022](https://arxiv.org/html/2607.12790#bib.bib27); Skalse et al\.[2022](https://arxiv.org/html/2607.12790#bib.bib31); Amodei et al\.[2016](https://arxiv.org/html/2607.12790#bib.bib1)\), and AI\-feedback and self\-rewarding setups let the model judge itself\(Bai et al\.[2022](https://arxiv.org/html/2607.12790#bib.bib4); Yuan et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib40)\)\. Our setting shares the hazard but differs in structure: the evolved metric is optimized to predict an anchor it never trains on, not to reward the agent, and its agent\-facing use is confined to training\.

#### LLM\-as\-judge\.

LLM judges are widely used as evaluation infrastructure and widely audited for bias\(Zheng et al\.[2023](https://arxiv.org/html/2607.12790#bib.bib47); Dubois et al\.[2023](https://arxiv.org/html/2607.12790#bib.bib10); Gu et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib15); Chen et al\.[2024a](https://arxiv.org/html/2607.12790#bib.bib5); Zhuge et al\.[2024](https://arxiv.org/html/2607.12790#bib.bib48)\)\. We use LLM judgment in three narrow, guarded roles: a teacher produces soft labels on the tiny reference set, one detector class wraps a judge for semantic mismatch, and a stronger position\-debiased pairwise judge audits final report outputs outside all loops\. The metric itself stays mostly deterministic, which is what makes its failures diagnosable\.

#### Benchmarks\.

MBPP\+ provides unit\-tested Python tasks\(Austin et al\.[2021](https://arxiv.org/html/2607.12790#bib.bib3); Liu et al\.[2023](https://arxiv.org/html/2607.12790#bib.bib20)\), in the execution\-graded tradition of HumanEval\(Chen et al\.[2021](https://arxiv.org/html/2607.12790#bib.bib7)\)\. Spider 2\.0 provides enterprise text\-to\-SQL over real warehouses with execution\-based grading\(Lei et al\.[2025](https://arxiv.org/html/2607.12790#bib.bib18)\)\. We use both as hard anchors for certifying evolved metrics, not as leaderboard entries, since our solver is a single LLM call without tools or iteration\.

## Problem Setup

Let a task family come with tasksttand a frozen solver that mapsttto an outputyy\. A metricMMmaps\(t,y\)\(t,y\)to a verdict in\{pass,fail,abstain\}\\\{\\textsc\{pass\},\\textsc\{fail\},\\textsc\{abstain\}\\\}\. We use three disjoint splits, which play the usual train/dev/test roles with one twist each\.

- •train, large and unlabeled: real solver outputs here expose where candidate metrics disagree or abstain, and in co\-evolution the evolved metric grades these tasks for the skill loop\.
- •dev, tiny and anchored \(ten items in all experiments\): each task carries a golden reference, and a teacher compares candidate outputs against it to emit soft pass/fail labels, the only supervised signal any loop reads\. The references are not themselves a metric: they grade only the ten tasks that carry them, while the evolved metric must grade arbitrary outputs on tasks that have none\.
- •test, held out and locked: the strongest available reference lives here \(unit tests for MBPP\+, official execution comparison for Spider 2\.0, the rubric score for reports\), but no loop ever reads it; it only reports the metric’s transfer and the agent’s held\-out score\.

The dev set is deliberately small: sparse anchoring is the realistic regime, since fresh labels in deployed systems arrive as a trickle of demos and incident reviews\.

### Metrics as Compositions of Drawback Detectors

An*op*is an atomic drawback detectoro​\(t,y,c\)→\{drawback,clean,abstain\}o\(t,y,c\)\\rightarrow\\\{\\textsc\{drawback\},\\textsc\{clean\},\\textsc\{abstain\}\\\}whereccis a shared context\. Each op checks exactly one failure class and abstains when it has no opinion\. Each task family starts from a handful of such ops covering its obvious failure modes, cheap to write by hand or have an LLM draft from the task spec; the lifecycle grows the pool from there\. Ops come in three cost tiers\. Static ops parse and inspect the artifact\. Execution ops run it, against a sandbox for code or a live warehouse for SQL\. Judge ops ask an LLM one narrow question\. Every task family also has a fixed non\-evolving*root*that validates basic structure, for example that code parses or that a report has a title, and caches the parsed artifact all other ops read\.

A*metric expression*composes op verdicts with logical operators\. Writingv​\(e\)∈\{1,0,⊥\}v\(e\)\\in\\\{1,0,\\bot\\\}for the verdict of expressioneeon\(t,y\)\(t,y\), where11denotesdrawback,0denotesclean, and⊥\\botdenotesabstain, the grammar is

e::=o\|⋁i=1kei\|⋀i=1kei\|\[∑i=1kv\(ei\)≥K\],e::=o\\;\\big\|\\;\\textstyle\\bigvee\_\{i=1\}^\{k\}e\_\{i\}\\;\\big\|\\;\\textstyle\\bigwedge\_\{i=1\}^\{k\}e\_\{i\}\\;\\big\|\\;\\big\[\\textstyle\\sum\_\{i=1\}^\{k\}v\(e\_\{i\}\)\\geq K\\big\],\(1\)i\.e\., a leaf op, a disjunction \(a drawback if any child finds one\), a conjunction \(a drawback only if all children agree\), or aKK\-of\-kkvote\. Abstaining children \(v​\(ei\)=⊥v\(e\_\{i\}\)=\\bot\) are excluded from their combinator, and a node whose children all abstain abstains itself\. The output passes when the root finds no drawback\. The final verdict is always the fixed root conjoined with the evolved expression\. Ops are pure functions of\(t,y\)\(t,y\), so verdicts are cached and an expression evaluation is a set of table lookups\. A metric is fully reproducible from its expression string plus the registered op pool\.

This drawback\-first, mostly deterministic design is a safety choice: a bare LLM judge shares failure modes with the LLM solver it grades, so it can drift into agreement with the solver before any optimization pressure exists, while deterministic detectors fail differently\.

### The Metric Loop

Algorithm[1](https://arxiv.org/html/2607.12790#alg1)sketches one round\. Two signals feed synthesis:*misses*, dev items the current metric passes although their soft label is fail, and*gaps*, train outputs on which the op pool abstains entirely or splits\. Both are clustered by failure class under a closed task\-specific taxonomy, and a synthesizer authors a typed op spec for any recurring cluster\. A new op must pass a birth gate: fire on at least half its cluster and stay clean on known\-good outputs\. Ops born from anchored misses enter the pool*active*\(selectable into expressions\), since their birth evidence already ties them to the anchor; ops born only from unlabeled gaps enter as*shadows*, which run and record verdicts every round but cannot be selected, because nothing yet shows they track quality rather than consensus noise\. A shadow is promoted to active once its verdicts demonstrably raise dev agreement\.

Candidate expressions are proposed by an LLM composer that sees the op catalog, the current gaps, and an elite history of the best expressions with their shortcomings, plus structural mutations and crossovers of the elites\. Selection keeps the best\-so\-far candidate by

S​\(e\)=Adev​\(e\)⋅Atrain​\(e\)w−λ​C​\(e\),\\begin\{split\}S\(e\)\\;=\\;A\_\{\\mathrm\{dev\}\}\(e\)\\cdot A\_\{\\mathrm\{train\}\}\(e\)^\{w\}\\;\-\\;\\lambda\\,C\(e\),\\end\{split\}\(2\)whereAdev​\(e\)A\_\{\\mathrm\{dev\}\}\(e\)isee’s agreement with the dev soft labels,Atrain​\(e\)A\_\{\\mathrm\{train\}\}\(e\)its agreement with the majority verdict of the opining pool ops on unlabeled train outputs,C​\(e\)C\(e\)expression size, andw,λw,\\lambdasmall constants\. The consensus term is the regularizer: a metric that memorizes ten dev items but behaves erratically on the broad distribution loses, and a degenerate always\-fail metric loses on dev\. Two guards apply: selection fails closed \(a candidate with no usable dev opinion is unselectable\), and a validity gate drops candidates that pass, fail, or abstain on everything\. Per\-op fitness is the leave\-one\-out marginal of the incumbent expression; non\-positive ops retire after a grace period\. Best\-so\-far selection never regresses its own objective\.

Algorithm 1One round of metric evolution1:Sense\(where is the metric wrong?\)

2:misses

←\\leftarrowdev items the incumbent passes but the soft label fails

3:gaps

←\\leftarrowtrain outputs where the pool abstains or splits

4:Grow\(expand the op pool\)

5:cluster misses

∪\\cupgaps by failure class; author one typed op per recurring cluster

6:birth gate: fires on

≥\\geqhalf its cluster and clean on known\-good

⇒\\Rightarrowadmit \(active if anchored, else shadow\)

7:Select\(recombine, keep the best\)

8:candidates

←\\leftarrowLLM compositions over the pool, plusmutations and crossovers of elites

9:keep

arg⁡maxe⁡S​\(e\)\\arg\\max\_\{e\}S\(e\)over candidates and the incumbent, behind the validity gate \(Eq\.[2](https://arxiv.org/html/2607.12790#Sx3.E2)\)

10:Curate\(merit\-based lifecycle\)

11:retire ops with non\-positiveleave\-one\-out marginal; promote shadow ops that raise dev agreement

12:Audit\(measurement only, never a training signal\)

13:report agreement against the locked test set

### Co\-Evolution with the Skill Loop: Double Ratchet

The skill loop we adopt\(Zhang et al\.[2026b](https://arxiv.org/html/2607.12790#bib.bib44)\)needs a grader for its training tasks\. In co\-evolution we hand it the current best evolved metric; we call the paired system*Double Ratchet*\(Figure[1](https://arxiv.org/html/2607.12790#Sx1.F1)\)\. The two loops alternate under a fixed, front\-loaded curriculum, with metric phases longest early, when the skill loop most needs coverage, and shrinking as the metric matures: the evolved metric grades train attempts, and failed attempts become capsules whose error text feeds skill synthesis\. The skill loop’s held\-out evaluation and rollback anchor stay pinned to the locked test set, which the metric never reads, so a corrupted metric can degrade skill learning but cannot corrupt the measurement\. The comparison that matters is against the reference skill loop, the same machinery given the ground\-truth metric \(or best available rubric\) for free: the closer co\-evolution comes to it, the better the evolved metric served where it counts\.

### Two Ablations: Anchor Guards versus Lifecycle

The design has two separable safety ingredients, and we ablate each while holding the other fixed\. Thenaiveablation disables the anchor guards \(fail\-closed anchoring and the validity gate, plus skill\-loop rollback\) while the detector lifecycle stays fully on; this is the collusion\-prone configuration that unanchored agent\-judge co\-evolution implicitly runs\. Theno\-lifecycleablation does the opposite: anchor guards stay on, but the birth gate and merit\-based retirement are disabled, so every synthesized detector enters the pool active and nothing retires\. This is the metric\-side analog of the unmanaged skill library in the library\-drift study\.

### Independent Final Judge for Reference\-Free Tasks

For task families without an accurate metric, rubric scores are references, not truth, so we add a final audit outside all loops: a stronger LLM compares each final output pairwise against the pre\-evolution baseline, judged twice with positions swapped, a win counting only when both orders agree\. This mimics the human acceptance step a real deployment would keep, and it is what exposed a proxy\-gaming failure the loops could not see\. The judge is an audit, not a training signal\.

## Experimental Setup

#### Task areas\.

MBPP\+\(Austin et al\.[2021](https://arxiv.org/html/2607.12790#bib.bib3)\), Python function synthesis with hidden unit tests as the hard anchor, andSpider 2\.0\-Snow\(Lei et al\.[2025](https://arxiv.org/html/2607.12790#bib.bib18)\), enterprise text\-to\-SQL over real Snowflake warehouses anchored by official execution\-result comparison, are the validation tasks: their exact ground truth makes the oracle\-versus\-co\-loop match measurable\.Report generationis the target task: a realistic workload derived from deployment\-style report artifacts, in which a pipeline writes analyst\-report sections from audited evidence cards \(tasks, cards, and golden sections are in the supplementary material\)\. No golden metric exists for it: the available rubric score \(RAQS\) plus golden demonstration sections is a partial reference signal, not ground truth\. Task descriptions with examples are in the appendix; tables abbreviate the tasks as MBPP\+, Spider, and Report\.

#### Hard\-subset construction\.

For MBPP\+ and Spider 2\.0 we screen every task with five independent frozen\-solver samples and keep a hard subset: tasks the solver already passes on every sample are dropped \(no lift room\), and the rest are stratified by screen pass rate so each split covers a balanced mix of difficulty levels\. The train/dev/test splits are 60/10/40 for MBPP\+, 59/10/40 for Spider 2\.0, and 73/10/48 for report generation, whose tasks derive from golden demonstration reports and need no screening\. The dev set always holds ten items, five clear failures and five clean positives\. The solver is a single direct LLM call with no tools, iteration, or chain\-of\-thought scaffolding, so absolute numbers are not comparable to leaderboard agents\.

#### Roles and models\.

All experiments use Claude models: every role in every loop \(solver, skill synthesizers, teachers, judge ops\) is Claude Opus 4\.7; the single exception is the report task’s final judge, Claude Opus 4\.8, a stronger model kept outside all loops\. A same\-family teacher is not a circular self\-grader: it compares candidate outputs against golden references on dev rather than judging unaided, and the locked anchor never passes through it\. Teacher quality is held fixed throughout our experiments; its influence on anchor quality is a variable we leave to future work\. Skill loops run 100 rounds; co\-evolution interleaves metric phases of 15/8/5/2 rounds between four 25\-round skill phases, matching the oracle’s 100\-skill\-round budget\.

## Results

Each run is repeated with three seeds and reports mean and sample standard deviation; where we compare loops we also report the seed\-level difference with a 95% bootstrap confidence interval, noting that three seeds and 40–48\-item held\-out sets support only coarse conclusions\. Because absolute levels carry per\-round sampling noise near five points, we report peak and final\-rounds values \(mean of the last ten rounds\) against each run’s own round\-0 baseline\. In all tables,shadedcells mark our proposed configuration\.

### The Evolved Metric Lifts and Transfers

Figure[2](https://arxiv.org/html/2607.12790#Sx5.F2)shows the metric loop’s selection objective per round, one panel per task\. The objective climbs on every seed of every task and never degrades afterwards: all loops ran the full 100\-round budget with early stopping disabled, and the long flat tails are the stability that best\-so\-far selection promises \(the last improvement lands by round 63, so the figure crops at 70\)\. The loop converges to a composed expression, not a single check: on MBPP\+ three ops mixing execution and specification judges, e\.g\.\(any spec\_mismatch crash returns\_not\_print\_only\); on Spider 2\.0 and report generation two\-op compositions such as\(any missing\_group\_by spec\_sql\_mismatch\), with one seed per task swapping a leaf \(appendix\)\.

Each panel’s chip compares the evolved metric’s peak held\-out anchor agreement against the natural baseline of using the LLM judge op alone \(no ground truth exists to score a standalone judge on Report\)\. The composed metrics beat the single judge they contain, and unlike a bare judge the composition is inspectable, mostly deterministic, and diagnosable\. Transfer varies by task: strong lift on MBPP\+; at\-ceiling from round 0 on Report, where subset seeding places a near\-optimal composition in the initial candidate set; and low on Spider despite a robust objective climb, a regime measurement we return to under the detectability spectrum\. Imperfect agreement does not doom the system, because the metric’s job in co\-evolution is directional \(next subsection\)\. The ten\-item dev set is also not fragile: replaying report\-generation metric selection offline on 200 stratified subsamples per size gives held\-out agreement of0\.871±0\.0630\.871\\pm 0\.063\(four items\),0\.882±0\.0510\.882\\pm 0\.051\(six\), and0\.874±0\.0410\.874\\pm 0\.041\(eight\), versus 0\.854 at ten\.

![Refer to caption](https://arxiv.org/html/2607.12790v1/x2.png)Figure 2:Metric\-loop selection objective per round \(mean±\\pmstd, three seeds; first 70 of 100 rounds, flat afterwards\)\. Chips: peak held\-out anchor agreement versus the bare LLM judge op, where ground truth exists to score one\.![Refer to caption](https://arxiv.org/html/2607.12790v1/x3.png)Figure 3:Held\-out learning curves: the reference skill loop \(orange, trained on ground truth or the best\-available rubric\) versus Double Ratchet \(blue, trained on its own evolved metric\)\. Mean±\\pmstd over three seeds; dotted line = frozen baseline; exact scores in the appendix\.
### Co\-Evolution Matches the Oracle

Table 1:Held\-out results across the three tasks\.*skill*: the reference loop, trained on ground truth \(MBPP\+, Spider\) or the best\-available rubric \(Report\)\.*co*: Double Ratchet\.*co naive*: the anchor\-guards\-off ablation, whose metric is vacuous \(†; Table[3](https://arxiv.org/html/2607.12790#Sx5.T3)\), so its higher score reflects unfiltered practice, not a working grader\.*Retention*is co’s peak lift as a fraction of skill’s; theΔ\\Deltapeak CIs contain zero on both ground\-truth tasks \(co matches the oracle within noise\) and show only a small, rubric\-scored gap on Report, where the final judge \(Table[2](https://arxiv.org/html/2607.12790#Sx5.T2)\) is decisive\.*Improved*counts never\-pass tasks solved at least once, or \(Report\) sections ending above their round\-0 score\. Full per\-loop scores are in the appendix\.Table[1](https://arxiv.org/html/2607.12790#Sx5.T1)summarizes the held\-out peaks and head\-to\-head comparison; Figure[3](https://arxiv.org/html/2607.12790#Sx5.F3)shows the learning curves\. Matching, not beating, is the success criterion: the reference rows are what self\-improvement achieves with an accurate metric handed over for free, and the question is how much of that lift Double Ratchet retains while evolving its metric from ten reference examples\. It retains 106% on MBPP\+, 110% on Spider, and 88% on Report: within noise on the two ground\-truth tasks, and a small gap on the report task, whose reference is itself a*partial rubric*, so the independent final judge below is decisive there\. Double Ratchet recovers most of the supervised benefit while producing an inspectable metric\.

Two shapes in Figure[3](https://arxiv.org/html/2607.12790#Sx5.F3)deserve a note\. Spider’s absolute lift is smallest for*both*loops: the residual failures of a single\-call solver mostly require live exploration that no guidance\-text skill can express, so the ceiling binds oracle and Double Ratchet alike \(all arms also share a large prompt\-grounding gain, appendix\)\. The report curves spike early because one targeted skill restores most of the lift, naming the mechanical rubric behaviors the minimal solver prompt omits\.

Aggregates also hide a task\-level result: the loops solve held\-out problems the frozen solver*never*solves \(Table[1](https://arxiv.org/html/2607.12790#Sx5.T1),*Improved*rows\), and on both hard\-anchor tasks Double Ratchet converts*more*never\-pass tasks than the ground\-truth\-driven loop\. This also settles how good the evolved metric needs to be: not very\. The Spider metric agrees with ground truth only at0\.500±0\.0260\.500\\pm 0\.026, yet its co\-loop retains the full reference lift, because the metric’s role in training is*directional*, not absolute: it decides which attempts become failure capsules, and the capsules carry concrete error text whose value does not depend on the pass/fail label being perfect\. The co\-loop’s metric\-graded training accuracy stays in a realistic band \(0\.75–0\.83 on Report, 0\.41–0\.83 on Spider\), so the metric is doing real grading\. Held\-out evaluation never passes through the evolved metric, so a weak metric slows learning but cannot corrupt measurement\. Spider thus bounds the claim: the method’s added value is largest where failures are mechanically detectable \(next subsections\)\.

### The Goodhart Episode and the Final\-Judge 2×\\times2

The report task produced the paper’s most instructive sequence \(Table[2](https://arxiv.org/html/2607.12790#Sx5.T2)\)\. The rubric’s metric\-discipline dimension counts inline evidence tags, and the first skill runs raised the rubric score by\+0\.26\+0\.26partly by gaming exactly this: evolved skills wrote the tag*in place of*the number \(about 30% of tags at peak rounds had no value beside them\) and invented confident forecasts to satisfy a style dimension\. The independent judge preferred the baseline in 88% of decided pairs, describing unrendered placeholders and fabricated conviction\.

The repair required one detector: a vocabulary\-aware value\-erasure check added to the capsule gate \(a tag whose registered metric has a numeric value but no numeral nearby is a defect\), plus rewritten failure\-feedback hints teaching value\-plus\-tag\. The rerun cut erased tags from roughly 30% to about 1% while the rubric score*rose*to0\.850±0\.0100\.850\\pm 0\.010, so nothing was traded away\.

The judge itself then needed auditing\. Under the original generic rubric the repair barely moved the win rate \(0\.122 to 0\.126\): its loss rationales blamed the raw tag syntax, which is the pipeline’s required output format\. A task\-aware rubric stating the format contract, while still treating valueless tags and fabricated figures as defects, re\-judged the same stored pairs\. The 2×\\times2 separates the two layers\. Down the task\-aware column, the erasure fix moves the win rate from 0\.515 to 0\.770, so the fix is real content quality; across each row, the generic\-to\-task\-aware jump shows a convention\-blind judge mistaking the task’s format contract for a defect\. The deployment lesson runs both directions: self\-evolved metrics need an independent outer audit, and the audit needs the task contract\.

Table 2:Final\-judge win rate of evolved report outputs over their pre\-evolution baselines \(higher is better; ties excluded\)\. Rows isolate the erasure fix; columns isolate the judge’s convention\-blindness\.
### Which Guard Carries the Safety Load

The ablations are judged on*metric validity*, not held\-out task score, because the two dissociate\. Table[3](https://arxiv.org/html/2607.12790#Sx5.T3)contrasts the three arms of the report task’s metric loop, where the anchor is softest and collusion pressure highest\. Disabling the anchor guards \(naive\) collapses the metric on three of three seeds: selection latches onto a detector that almost never fires, fail\-open scoring reports a vacuous perfect objective, and the metric passes 0\.94–1\.00 of everything it grades\. Disabling the lifecycle instead \(no\-lifecycle\) does not: objectives and held\-out agreement stay within the anchored band and the op pool does not bloat, because unselected junk ops are simply never picked\.

Held\-out task score is deliberately not the criterion, and Table[1](https://arxiv.org/html/2607.12790#Sx5.T1)shows why: naive co\-loops score as well as anchored ones, because a vacuous metric that passes everything reduces training to unfiltered practice while measurement stays pinned to the untouched ground\-truth anchor\. Practice alone improves skills, so the task score survives; the metric does not, and a deployment using it for triage or gating would inherit a grader that passes everything\. The anchor guards make the metric*valid and deployable*; they are not what makes practice useful\.

This inverts the emphasis of the skill\-evolution literature: for skills, lifecycle management is the key finding; for evaluators under anchored selection, anchor discipline carries the safety load, and the metric\-side analog of library drift is anchor drift, not pool drift\. The structural reason: a junk skill gets*routed*into prompts and hurts, while a junk op matters only if selection picks it, and the anchor stops that\. The lifecycle still buys efficiency at scale\.

Table 3:Metric\-validity ablations on report generation \(three seeds\)\. Removing the anchor guards \(*naive*\) collapses the metric into a vacuous always\-pass grader; removing the lifecycle \(*no\-lifecycle*\) does not\.*Train*: fraction of training attempts the metric passes\.
### The Detectability Spectrum

The three tasks span a spectrum of how mechanically detectable the solver’s failures are, and metric transfer tracks it\. On MBPP\+, failures are crashes and wrong outputs that cheap ops see directly, and held\-out agreement lifts\+0\.21\+0\.21; on report generation, the anchor derives from golden demonstrations through a closed, checkable failure taxonomy \(appendix\), and transfer reaches 0\.84\. Spider 2\.0 gives an in\-task controlled contrast: under our first prompt most failures were compile errors and the evolved metric reached 0\.85 held\-out agreement; after grounding the prompt with full typed schemas, the solver’s baseline nearly doubled and its*residual*failures became semantically wrong values under clean execution, invisible to deterministic ops, so held\-out agreement fell to0\.500±0\.0260\.500\\pm 0\.026even as the objective still lifted\. Same method, same task, different failure distribution: metric evolution buys the most where failures are checkable, and a purely semantic failure regime shifts the burden onto judge ops and outer audits\.

Qualitatively, everything the loops evolve is legible: metric expressions name failure modes a practitioner would recognize \(missing GROUP BY, uncited claims, overclaimed weak evidence\), and skill banks converge to task idioms\. This end\-to\-end auditability is what made the Goodhart diagnosis and one\-detector repair possible; concrete artifacts are reproduced in the appendix\.

## Discussion and Limitations

#### The anchor cannot be manufactured\.

Evolution expands coverage; it never creates ground truth\. With no anchor the loop optimizes noise, so anchor*quality*matters as much as its existence\. Hardening soft anchors into checkable ones is the highest\-leverage practitioner investment\.

#### Where the method helps\.

Metric evolution buys the most where the solver’s failures are mechanically detectable, as the Spider 2\.0 within\-task contrast quantifies, and guidance\-text skills cannot substitute for capabilities requiring interaction, such as live schema exploration\.

#### Noise and scope\.

Held\-out sets of tens of binary tasks carry per\-round noise near five points, so we report multi\-round means\. The study covers three task families, one solver model, and rounds in the low hundreds; adaptive curricula and failure\-anticipating detectors in the spirit of metamorphic testing\(Chen, Cheung, and Yiu[1998](https://arxiv.org/html/2607.12790#bib.bib8)\)are built but unexercised here\.

## Conclusion

Evaluation metrics can be evolved with the lifecycle discipline that makes skill evolution safe, anchored to a reference set the loop predicts and a locked set it never touches\. Double Ratchet removes the assumption that a reliable grader pre\-exists, retaining 88–110% of the supervised lift with an inspectable metric; its load\-bearing guard is anchor discipline, not the pool lifecycle; and on the deployment\-style task it caught its metric being gamed, repaired it with one detector, then caught its judge misreading the format contract\. Our answer to the title question: an anchor the grader must predict, one it never sees, and an outside judge\.

## References

- Amodei et al\. \(2016\)Amodei, D\.; Olah, C\.; Steinhardt, J\.; Christiano, P\.; Schulman, J\.; and Mané, D\. 2016\.Concrete Problems in AI Safety\.*arXiv preprint arXiv:1606\.06565*\.
- Anthropic \(2026\)Anthropic\. 2026\.New in Claude Managed Agents: Dreaming, Outcomes, and Multiagent Orchestration\.[https://claude\.com/blog/new\-in\-claude\-managed\-agents](https://claude.com/blog/new-in-claude-managed-agents)\.Product announcement, May 19, 2026\.
- Austin et al\. \(2021\)Austin, J\.; Odena, A\.; Nye, M\.; Bosma, M\.; Michalewski, H\.; Dohan, D\.; Jiang, E\.; Cai, C\.; Terry, M\.; Le, Q\.; and Sutton, C\. 2021\.Program Synthesis with Large Language Models\.*arXiv preprint arXiv:2108\.07732*\.
- Bai et al\. \(2022\)Bai, Y\.; Kadavath, S\.; Kundu, S\.; Askell, A\.; Kernion, J\.; Jones, A\.; Chen, A\.; Goldie, A\.; Mirhoseini, A\.; McKinnon, C\.; et al\. 2022\.Constitutional AI: Harmlessness from AI Feedback\.*arXiv preprint arXiv:2212\.08073*\.
- Chen et al\. \(2024a\)Chen, G\. H\.; Chen, S\.; Liu, Z\.; Jiang, F\.; and Wang, B\. 2024a\.Humans or LLMs as the Judge? A Study on Judgement Bias\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*\.
- Chen et al\. \(2024b\)Chen, M\.; Li, Y\.; Yang, Y\.; Yu, S\.; Lin, B\.; and He, X\. 2024b\.AutoManual: Generating Instruction Manuals by LLM Agents via Interactive Environmental Learning\.In*Advances in Neural Information Processing Systems*, volume 37\.
- Chen et al\. \(2021\)Chen, M\.; Tworek, J\.; Jun, H\.; Yuan, Q\.; Pinto, H\. P\. d\. O\.; Kaplan, J\.; Edwards, H\.; Burda, Y\.; Joseph, N\.; Brockman, G\.; et al\. 2021\.Evaluating Large Language Models Trained on Code\.*arXiv preprint arXiv:2107\.03374*\.
- Chen, Cheung, and Yiu \(1998\)Chen, T\. Y\.; Cheung, S\. C\.; and Yiu, S\. M\. 1998\.Metamorphic Testing: A New Approach for Generating Next Test Cases\.Technical Report HKUST\-CS98\-01, Hong Kong University of Science and Technology\.
- Dong et al\. \(2024\)Dong, P\.; Li, L\.; Tang, Z\.; Liu, X\.; Pan, X\.; Wang, Q\.; and Chu, X\. 2024\.Pruner\-Zero: Evolving Symbolic Pruning Metric from Scratch for Large Language Models\.In*Proceedings of the 41st International Conference on Machine Learning \(ICML\)*\.
- Dubois et al\. \(2023\)Dubois, Y\.; Li, X\.; Taori, R\.; Zhang, T\.; Gulrajani, I\.; Ba, J\.; Guestrin, C\.; Liang, P\.; and Hashimoto, T\. B\. 2023\.AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback\.*Advances in Neural Information Processing Systems*, 36\.
- Elsken, Metzen, and Hutter \(2019\)Elsken, T\.; Metzen, J\. H\.; and Hutter, F\. 2019\.Neural Architecture Search: A Survey\.*Journal of Machine Learning Research*, 20\(55\): 1–21\.
- Fernando et al\. \(2024\)Fernando, C\.; Banarse, D\.; Michalewski, H\.; Osindero, S\.; and Rocktäschel, T\. 2024\.Promptbreeder: Self\-Referential Self\-Improvement via Prompt Evolution\.In*Proceedings of the 41st International Conference on Machine Learning \(ICML\)*\.
- Gao, Schulman, and Hilton \(2023\)Gao, L\.; Schulman, J\.; and Hilton, J\. 2023\.Scaling Laws for Reward Model Overoptimization\.In*Proceedings of the 40th International Conference on Machine Learning \(ICML\)*, 10835–10866\.
- Goodhart \(1984\)Goodhart, C\. A\. E\. 1984\.Problems of Monetary Management: The UK Experience\.*Monetary Theory and Practice: The UK Experience*, 91–121\.
- Gu et al\. \(2024\)Gu, J\.; Jiang, X\.; Shi, Z\.; Tan, H\.; Zhai, X\.; Xu, C\.; Li, W\.; Shen, Y\.; Ma, S\.; Liu, H\.; Wang, S\.; Zhang, K\.; Wang, Y\.; Gao, W\.; Ni, L\.; and Guo, J\. 2024\.A Survey on LLM\-as\-a\-Judge\.*arXiv preprint arXiv:2411\.15594*\.
- Hu, Lu, and Clune \(2025\)Hu, S\.; Lu, C\.; and Clune, J\. 2025\.Automated Design of Agentic Systems\.In*International Conference on Learning Representations \(ICLR\)*\.
- Khattab et al\. \(2024\)Khattab, O\.; Singhvi, A\.; Maheshwari, P\.; Zhang, Z\.; Santhanam, K\.; Vardhamanan, S\.; Haq, S\.; Sharma, A\.; Joshi, T\. T\.; Moazam, H\.; Miller, H\.; Zaharia, M\.; and Potts, C\. 2024\.DSPy: Compiling Declarative Language Model Calls into Self\-Improving Pipelines\.In*International Conference on Learning Representations \(ICLR\)*\.
- Lei et al\. \(2025\)Lei, F\.; Chen, J\.; Ye, Y\.; Cao, R\.; Shin, D\.; Su, H\.; Suo, Z\.; Gao, H\.; Hu, W\.; Yin, P\.; Zhong, V\.; Xiong, C\.; Sun, R\.; Liu, Q\.; Wang, S\.; and Yu, T\. 2025\.Spider 2\.0: Evaluating Language Models on Real\-World Enterprise Text\-to\-SQL Workflows\.In*International Conference on Learning Representations \(ICLR\)*\.
- Li et al\. \(2026\)Li, X\.; Chen, W\.; Liu, Y\.; Zheng, S\.; Chen, X\.; He, Y\.; Li, Y\.; You, B\.; Shen, H\.; Sun, J\.; et al\. 2026\.SkillsBench: Benchmarking how well agent skills work across diverse tasks\.*arXiv preprint arXiv:2602\.12670*\.
- Liu et al\. \(2023\)Liu, J\.; Xia, C\. S\.; Wang, Y\.; and Zhang, L\. 2023\.Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Liu et al\. \(2021\)Liu, P\.; Zhang, G\.; Wang, B\.; Xu, H\.; Liang, X\.; Jiang, Y\.; and Li, Z\. 2021\.Loss Function Discovery for Object Detection via Convergence\-Simulation Driven Search\.In*International Conference on Learning Representations \(ICLR\)*\.
- Madaan et al\. \(2023\)Madaan, A\.; Tandon, N\.; Gupta, P\.; Hallinan, S\.; Gao, L\.; Wiegreffe, S\.; Alon, U\.; Dziri, N\.; Prabhumoye, S\.; Yang, Y\.; et al\. 2023\.Self\-Refine: Iterative Refinement with Self\-Feedback\.*Advances in Neural Information Processing Systems*, 36\.
- Ng, Harada, and Russell \(1999\)Ng, A\. Y\.; Harada, D\.; and Russell, S\. 1999\.Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping\.In*Proceedings of the 16th International Conference on Machine Learning \(ICML\)*\.
- Ni et al\. \(2026\)Ni, J\.; Liu, Y\.; Liu, X\.; Sun, Y\.; Zhou, M\.; Cheng, P\.; Wang, D\.; Jiang, X\.; and Jiang, G\. 2026\.Trace2Skill: Parallel Inductive Skill Distillation for LLM Agents\.*arXiv preprint arXiv:2603\.25158*\.
- OpenAI \(2026\)OpenAI\. 2026\.Codex: Record & Replay\.[https://developers\.openai\.com/codex/record\-and\-replay](https://developers.openai.com/codex/record-and-replay)\.Developer documentation\.
- Ouyang et al\. \(2022\)Ouyang, L\.; Wu, J\.; Jiang, X\.; Almeida, D\.; Wainwright, C\.; Mishkin, P\.; Zhang, C\.; Agarwal, S\.; Slama, K\.; Ray, A\.; Schulman, J\.; Hilton, J\.; Kelton, F\.; Miller, L\.; Simens, M\.; Askell, A\.; Welinder, P\.; Christiano, P\.; Leike, J\.; and Lowe, R\. 2022\.Training Language Models to Follow Instructions with Human Feedback\.*Advances in Neural Information Processing Systems*, 35: 27730–27744\.
- Pan, Bhatia, and Steinhardt \(2022\)Pan, A\.; Bhatia, K\.; and Steinhardt, J\. 2022\.The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models\.In*International Conference on Learning Representations \(ICLR\)*\.
- Park et al\. \(2023\)Park, J\. S\.; O’Brien, J\. C\.; Cai, C\. J\.; Morris, M\. R\.; Liang, P\.; and Bernstein, M\. S\. 2023\.Generative Agents: Interactive Simulacra of Human Behavior\.In*Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology*\.
- Real et al\. \(2020\)Real, E\.; Liang, C\.; So, D\.; and Le, Q\. 2020\.AutoML\-Zero: Evolving Machine Learning Algorithms From Scratch\.In*Proceedings of the 37th International Conference on Machine Learning \(ICML\)*, 8007–8019\.
- Shinn et al\. \(2023\)Shinn, N\.; Cassano, F\.; Gopinath, A\.; Narasimhan, K\.; and Yao, S\. 2023\.Reflexion: Language Agents with Verbal Reinforcement Learning\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Skalse et al\. \(2022\)Skalse, J\.; Howe, N\.; Krasheninnikov, D\.; and Krueger, D\. 2022\.Defining and Characterizing Reward Gaming\.*Advances in Neural Information Processing Systems*, 35: 9460–9471\.
- Wang et al\. \(2024a\)Wang, G\.; Xie, Y\.; Jiang, Y\.; Mandlekar, A\.; Xiao, C\.; Zhu, Y\.; Fan, L\.; and Anandkumar, A\. 2024a\.Voyager: An Open\-Ended Embodied Agent with Large Language Models\.*Transactions on Machine Learning Research*\.
- Wang, Ren, and Zhang \(2026\)Wang, J\.; Ren, Y\.; and Zhang, H\. 2026\.From Procedural Skills to Strategy Genes: Towards Experience\-Driven Test\-Time Evolution\.*arXiv preprint arXiv:2604\.15097*\.
- Wang et al\. \(2024b\)Wang, L\.; Ma, C\.; Feng, X\.; Zhang, Z\.; Yang, H\.; Zhang, J\.; Chen, Z\.; Tang, J\.; Chen, X\.; Lin, Y\.; Zhao, W\. X\.; Wei, Z\.; and Wen, J\. 2024b\.A Comprehensive Survey of LLM\-Based Autonomous Agents\.*Frontiers of Computer Science*, 18\(6\)\.
- Wang et al\. \(2024c\)Wang, Z\. Z\.; Mao, J\.; Fried, D\.; and Neubig, G\. 2024c\.Agent Workflow Memory\.*arXiv preprint arXiv:2409\.07429*\.
- Wu et al\. \(2025\)Wu, R\.; Wang, X\.; Mei, J\.; Cai, P\.; Fu, D\.; Yang, C\.; Wen, L\.; Yang, X\.; Shen, Y\.; Wang, Y\.; et al\. 2025\.Self\-Evolving LLM Agents through an Experience\-Driven Lifecycle\.*arXiv preprint arXiv:2510\.16079*\.
- Xia et al\. \(2026\)Xia, P\.; Chen, J\.; Wang, H\.; Liu, J\.; Zeng, K\.; Wang, Y\.; Han, S\.; Zhou, Y\.; Zhao, X\.; Chen, H\.; et al\. 2026\.SkillRL: Evolving Agents via Recursive Skill\-Augmented Reinforcement Learning\.*arXiv preprint arXiv:2602\.08234*\.
- Yang et al\. \(2024\)Yang, C\.; Wang, X\.; Lu, Y\.; Liu, H\.; Le, Q\. V\.; Zhou, D\.; and Chen, X\. 2024\.Large Language Models as Optimizers\.In*International Conference on Learning Representations*\.
- Yang et al\. \(2026\)Yang, Y\.; Li, J\.; Pan, Q\.; Zhan, B\.; Cai, Y\.; Du, L\.; Zhou, J\.; Chen, K\.; Chen, Q\.; Li, X\.; et al\. 2026\.AutoSkill: Experience\-Driven Lifelong Learning via Skill Self\-Evolution\.*arXiv preprint arXiv:2603\.01145*\.
- Yuan et al\. \(2024\)Yuan, W\.; Pang, R\. Y\.; Cho, K\.; Li, X\.; Sukhbaatar, S\.; Xu, J\.; and Weston, J\. 2024\.Self\-Rewarding Language Models\.In*Proceedings of the 41st International Conference on Machine Learning \(ICML\)*\.
- Yuksekgonul et al\. \(2024\)Yuksekgonul, M\.; Bianchi, F\.; Boen, J\.; Liu, S\.; Huang, Z\.; Guestrin, C\.; and Zou, J\. 2024\.TextGrad: Automatic “Differentiation” via Text\.*arXiv preprint arXiv:2406\.07496*\.
- Zhang et al\. \(2025\)Zhang, J\.; Qu, C\.; Fang, J\.; et al\. 2025\.A Survey of Self\-Evolving Agents: On Path to Artificial Super Intelligence\.*arXiv preprint arXiv:2507\.21046*\.
- Zhang et al\. \(2026a\)Zhang, X\.; Cui, Y\.; Wang, G\.; Li, Z\.; Qiu, W\.; Zhu, B\.; and He, P\. 2026a\.Library Drift: Diagnosing and Fixing a Silent Failure Mode in Self\-Evolving LLM Skill Libraries\.In*ICML 2026 Workshop on Failure Modes in Agentic AI*\.
- Zhang et al\. \(2026b\)Zhang, X\.; Cui, Y\.; Wang, G\.; Li, Z\.; Qiu, W\.; Zhu, B\.; and He, P\. 2026b\.Ratchet: A Minimal Hygiene Recipe for Self\-Evolving LLM Agents\.*arXiv preprint arXiv:2605\.22148*\.
- Zhang et al\. \(2026c\)Zhang, X\.; Wang, G\.; Cui, Y\.; Qiu, W\.; Li, Z\.; Zhu, B\.; and He, P\. 2026c\.Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents\.*arXiv preprint arXiv:2604\.15877*\.
- Zhao et al\. \(2024\)Zhao, A\.; Huang, D\.; Xu, Q\.; Lin, M\.; Liu, Y\.\-J\.; and Huang, G\. 2024\.ExpeL: LLM Agents Are Experiential Learners\.In*Proceedings of the AAAI Conference on Artificial Intelligence*\.
- Zheng et al\. \(2023\)Zheng, L\.; Chiang, W\.\-L\.; Sheng, Y\.; Zhuang, S\.; Wu, Z\.; Zhuang, Y\.; Lin, Z\.; Li, Z\.; Li, D\.; Xing, E\. P\.; Zhang, H\.; Gonzalez, J\. E\.; and Stoica, I\. 2023\.Judging LLM\-as\-a\-Judge with MT\-Bench and Chatbot Arena\.*Advances in Neural Information Processing Systems*, 36\.
- Zhuge et al\. \(2024\)Zhuge, M\.; Zhao, C\.; Ashley, D\.; Wang, W\.; Khizbullin, D\.; Xiong, Y\.; Liu, Z\.; Chang, E\.; Krishnamoorthi, R\.; Tian, Y\.; Shi, Y\.; Chandra, V\.; and Schmidhuber, J\. 2024\.Agent\-as\-a\-Judge: Evaluate Agents with Agents\.*arXiv preprint arXiv:2410\.10934*\.

## Appendix AAppendix A: Task Descriptions

The full task data for all three families is in the supplementary material; the cards below give enough context to interpret the results\.

MBPP\+validation task⋅\\cdotanchor: hidden unit testsTask\.A one\-paragraph natural\-language programming problem plus one example call; the solver writes a Python function\.Anchor\.The EvalPlus hidden test suite, which adds many more tests per problem than the original MBPP and repairs faulty ones, giving reliable pass/fail\.Typical failures\.Crashes, mishandled edge cases, printing instead of returning, misread specifications, exactly the failure classes the op pool checks\.

Spider 2\.0\-Snowvalidation task⋅\\cdotanchor: execution matchTask\.An analyst question over a real hosted Snowflake warehouse: production\-scale databases with hundreds of tables, cryptic columns, sharded table families, and dialect\-specific SQL\. The solver writes one query from the question, a rendered schema catalog, and optional domain notes\.Anchor\.Execute the candidate and compare the fetched table to the stored gold result with the official comparator \(value\-based; column names and order are ignored\)\.Typical failures\.Unresolvable identifiers, type errors on date\-like columns, and, hardest, queries that run cleanly but compute the wrong thing\. Our single\-call solver passes about a third of screened tasks\.

Report generationtarget task⋅\\cdotno accurate metric existsTask\.Write one*section*of an analyst report \(e\.g\., “value\-chain map: control points from ASIC to optical engine”\) in markdown, given the inputs below\.Inputs\.\(i\) Section title and purpose\. \(ii\) A*caliber*contract fixing units, reference dates, and accounting rules \(e\.g\., “all amounts USD, calibrated to 2026 Q2, market sizes must not be summed across pools”\)\. \(iii\) Evidence cards\. \(iv\) A metric vocabulary registering the section’s key figures \(id, name, value, unit\)\.Evidence card\.An audited unit of evidence: idE0103, one\-sentence claim, verbatim source quote, source attribution and date, stance \(supporting / opposing / neutral\), and an auditor’s credibility verdict\.Requirements\.Ground every claim in the cards \(cite\[E\#\#\#\]\); state each key figure’s value with its inline tag binding it to the ledger \(rendered downstream as audit tooltips\); hedge weak or disputed evidence and present both sides of conflicts; include required structure such as a TL;DR\.Why no metric exists\.Section quality is holistic; even the pipeline’s human operators cannot reduce “a good section” to a checkable rule\. The RAQS rubric scores lexical and structural proxies of the requirements above and is treated as reference signal only\.

## Appendix BAppendix B: Setup Details

#### Splits and screening\.

For MBPP\+ and Spider 2\.0 we screen every task with five independent samples of the frozen single\-call solver, graded by the official anchor\. We drop tasks the solver always passes \(no room to improve\) and stratify the rest across pass levels for a balanced difficulty mix, then split 60/40 into train and test\. The dev set is ten items, five clear failures and five clean passes \(for report generation: five golden\-quality positives and five class\-stratified checkable failures\), so the metric is scored on both classes\. In the released code the dev and test splits are namedeval\_devandeval\_locked\.

#### Anchors\.

*MBPP\+*: hidden unit tests\.*Spider 2\.0*: execute the candidate on the live warehouse and compare the fetched table to the stored gold result with the official comparator, which matches values \(ignoring column names and order\)\. We reproduce the evaluator’s CSV round\-trip of the fetched frame, without which even gold queries fail on dtype\.*Report generation*: the metric loop’s anchor is a frozen demo\-delta soft label, where a teacher compares the frozen\-agent report against the golden section under a closed seven\-class miss taxonomy and passes iff at most one miss class is present\. Each of the 48 held\-out sections contributes two locked items, the frozen\-agent report \(which the strict anchor almost always fails\) and the golden section \(which passes by definition\), giving 96 items balanced 50/50, so a degenerate metric scores 0\.5\. The skill and co\-loops are evaluated by the official rubric evaluator \(RAQS, 0–100 normalized\), whose dimensions cover evidence grounding, metric\-tag discipline, claim calibration, structure, and style\.

#### Spider 2\.0 prompt grounding\.

The generation prompt grounds the solver with a compact typed schema catalog \(columns as name:TYPE, sharded tables collapsed, sample values on date\-like columns\) and Snowflake\-specific rules \(quoted identifiers, type\-respecting casts, value\-only grading\)\. We report this because it is a large, shared effect we control for, not a contribution: grounding roughly doubles the frozen baseline \(0\.211 to 0\.329\) but is applied identically to every arm, so all reported lifts sit on top of it\. It also sets up the detectability contrast: grounding pushes the solver’s residual failures from compile errors into clean\-executing wrong values\.

## Appendix CAppendix C: The Metric Language and Lifecycle

#### Language\.

Recapping the grammar of Eq\.[1](https://arxiv.org/html/2607.12790#Sx3.E1): a metric is an expression over ops \(pure functions of \(task, output\) returningdrawback/clean/abstain\), composed by⋁\\bigvee,⋀\\bigwedge, and theKK\-of\-kkvote, with abstaining children excluded from a combinator and an all\-abstain node abstaining\. It is fully reproducible from its expression string plus the registered op pool\.

#### Lifecycle\.

Each task starts from a small pool of seed detectors, the obvious failure checks a practitioner reads straight off the task description \(does the code crash, does the query group correctly, is a key figure missing\); drafting them, or having an LLM draft them from the task spec, is an afternoon’s work, and their exact contents barely matter because the lifecycle grows and prunes the pool from there\. Synthesis authors typed op specs from clustered misses under a closed per\-task taxonomy\. The birth gate requires a new op to fire on at least half its cluster and stay clean on known\-good outputs\. Ops born from anchored misses enter active; unlabeled\-gap ops enter shadow and are promoted only on demonstrated anchor agreement\. Per\-op fitness is the leave\-one\-out marginal of the incumbent expression, and non\-positive ops retire after a two\-round grace\.

#### Selection and guards\.

The objective multiplies dev agreement by consensus agreement on unlabeled train outputs \(exponent 1\.0 for MBPP\+ and Spider, 0\.25 for report generation, where op consensus is a weaker regularizer\), minus a small complexity penalty\. Fail\-closed anchoring makes candidates without a usable dev opinion unselectable, and a validity gate drops all\-pass/all\-fail/all\-abstain candidates\. The naive ablation disables exactly these two guards; the no\-lifecycle ablation disables exactly the birth gate, shadow tier, and retirement\.

## Appendix DAppendix D: Final\-Judge Protocol and Rubrics

#### Protocol\.

For each of the 48 held\-out sections we pair the stored round\-0 report against the stored peak\-round report, and separately against the final\-round report, for every arm and seed\. Pre\-repair this is 3 arms×\\times3 seeds×\\times2 rounds×\\times48 sections==864 pairs; the post\-repair rerun adds a fourth seed on the two co\-loop arms, giving 1,056\. Each pair is judged twice with positions swapped, and a win counts only when both orders agree; otherwise it is a tie\.

#### Rubrics\.

The generic rubric asks a senior\-editor question about holistic reader value with no knowledge of the pipeline’s conventions\. The task\-aware rubric additionally states the format contract: evidence citations and inline metric tags are required output that renders downstream; a tag with no value nearby is a defect; fabricated figures are disqualifying\. Both rubrics judge the same stored pairs, so the four cells of Table[2](https://arxiv.org/html/2607.12790#Sx5.T2)differ only in pass and rubric\.

## Appendix EAppendix E: What Evolution Actually Produced

Concrete final artifacts from the paper runs, verbatim except where marked as condensed, illustrating the central qualitative claim: everything the loops evolve is legible\.

Evolved metricsone final expression per task, verbatimMBPP\+Spider 2\.0ReportFlag a solution if the judge finds a spec mismatch, or it crashes on probe inputs, or it prints instead of returning:Flag a query if it aggregates without grouping, or the judge finds it does not answer the question \(selected independently on two of three seeds\):Flag a section if a registered key figure never appears, or a weak\-credibility card is cited without hedging \(the other two seeds swap the first leaf for an uncited\-claims detector\):\(anyspec\_mismatchcrashreturns\_not\_print\_only\)\(anymissing\_group\_byspec\_sql\_mismatch\)\(anyledger\_metric\_uncoveredlow\_status\_card\_overclaimed\)A synthesized opborn, gated, and selected \(Spider 2\.0\)The loop clustered dev misses where queries returned every column although the question named specific ones, and authored the typed spec\{kind: static, check: forbids\_select\_star, detects: select\_star\_specific\_columns\}\. It passed the birth gate \(fired on its cluster, clean on known\-good queries\) and the final selected co\-loop metric composed it with the grouping check, replacing the judge op on this seed\.

Grading one task: hidden tests vs\. evolved metricMBPP\+ task, final\-round trace, verbatimTask and solver outputGround\-truth anchorEvolved metric“Write a function to find the ratio of zeroes in an array of integers\.”def zero\_count\(nums\):zeros = sum\(1 for xin nums if x == 0\)non\_zeros = len\(nums\) \- zeros\.\.\.return zeros / non\_zerosThe code runs cleanly and returns a number; it computes the ratio to*non\-zeros*where the spec wants the ratio to*all*elements\. Only the meaning is wrong\.Hidden unit tests \(EvalPlus\)\.assert zero\_count\(\.\.\.\)→\\rightarrowAssertionErrorVerdict:fail\.The verdict is reliable but opaque: no indication of which requirement was misread\.Evolved composition\(any spec\_mismatch crash returns\_not\_print\_only\)\.crash: cleanreturns\_not\_print\_only: cleanspec\_mismatch\(judge op\):drawbackVerdict:fail, agreeing with the anchor, and*naming*the failure class: the specification was misread\.The named drawback is what turns this attempt into a usable failure capsule for skill synthesis; the anchor’s bare verdict could not have driven the same repair\.Same lesson, two loopsfunctionally matched skills, oracle vs\. co\-evolved, condensedEach pair below was synthesized independently, one under ground\-truth grading \(oracle\) and one under the evolved metric \(Double Ratchet\); neither loop saw the other\. Converging on the same task lesson is the qualitative face of the retention result\.MBPP\+Spider 2\.0Report*\(grader imports a literal identifier\)**\(empty result diagnosis\)**\(the anti\-erasure lesson\)**oracle*function\-name\-from\-assert: “the harness imports the symbol literally as written in the assert; lift the identifier verbatim including case, the prose is only a hint\.”*oracle*sql\-empty\-result\-diagnosis: “a 0\-row result is almost never correct when the gold has rows; incrementally relax predicates to locate which condition eliminates all rows\.”*oracle*metric\-tag\-annotates\-not\-replaces: “the scorer counts a figure only when a literal number sits adjacent to the brace tag; a tag with no rendered numeral reads as an unrendered placeholder\.”*co*copy\_callable\_identifier\_from\_assert: “the grader imports the name verbatim from the assert; reproduce that exact spelling letter for letter, including apparent typos\.”*co*sql\-empty\-result\-sanity\-probe: “an empty result almost always means one predicate matched nothing; validate each CTE’s row count in isolation before trusting the join\.”*co*metric\-tag\-inline\-after\-number: “the tag annotates the value; it never replaces the value, and a bare card cite does not satisfy it\.”
## Appendix FAppendix F: Lifecycle Statistics

Tables[4](https://arxiv.org/html/2607.12790#A6.T4)and[5](https://arxiv.org/html/2607.12790#A6.T5)recap what the two lifecycles did in the paper runs \(anchored arms\)\. For the metric loops, each task starts from a small pool of 9 to 11 hand\-authored seed detectors, gap\-driven synthesis then proposes new ones continuously, and the birth gate rejects the overwhelming majority; the final metric composes only one to three leaves, reflecting that anchored selection prefers precise compositions over op accumulation\. For the skill loops, synthesis is prolific \(roughly 1\.5 skills per round\), the bounded bank forces contribution\-based eviction of more than half of everything created, and rollback fires rarely; the report skill loop is the outlier, its targeted failure capsules producing skills so effective that a handful suffice \(one seed held its plateau with a bank of six\)\.

Table 4:Metric\-loop op lifecycle, anchored arm over 100 rounds: hand\-authored seed detectors, ops born during the run, birth\-gate rejections, final pool of seed plus born, and leaves in the final expression\. Three\-seed min–max; a bare number means all seeds agree\.Table 5:Skill\-loop lifecycle \(100 rounds; bank cap 50\): skills synthesized, final active bank, contribution\-evicted, and rollbacks\. Three\-seed min–max; a bare number means all seeds agree\.
## Appendix GAppendix G: Negative and Supporting Results

#### Full per\-loop scores\.

Table[6](https://arxiv.org/html/2607.12790#A7.T6)gives the complete held\-out init / peak / final\-rounds scores summarized in Table[1](https://arxiv.org/html/2607.12790#Sx5.T1); frozen single\-call baselines are≈\\approx0\.27 \(MBPP\+\), 0\.285 \(Spider 2\.0\), and 0\.56 \(report generation\)\.

Table 6:Held\-out evaluation per loop, mean±\\pmstd over three seeds\. End: the final\-rounds value, mean of each run’s last ten rounds\.
#### Naive co\-loops are not a free pass\.

Although naive co\-loops match the anchored arm on held\-out evaluation \(their measurement anchor is untouched\), their metric arms are degenerate where inspection matters: on report generation the naive metric grades 0\.94–1\.00 of train as passing \(versus 0\.75–0\.83 anchored\), and on Spider 2\.0 the naive objective barely moves on two of three seeds\.

#### Skill\-loop robustness to metric quality\.

Two of three Spider 2\.0 co\-evolution seeds selected a weak synthesized single\-op metric \(train agreement 0\.41–0\.44\) yet reached peak evaluations of 0\.45–0\.50, indistinguishable from the sibling with a two\-op metric\. Training signal quality degrades gracefully because failed attempts still carry concrete error text into skill synthesis even when pass/fail labels are noisy\.

## Appendix HAppendix H: Reproducibility

We release the code: all loops, op packs, split builders, screening scripts, and the judge protocols, sufficient to regenerate every experiment end to end\. Each run writes its round\-by\-round history, op pool, skill bank, and judge outputs as structured JSON/SQLite result files, and every number in this paper is read directly from those files\. Spider 2\.0 requires credentialed access to the benchmark’s hosted warehouse; all other experiments are self\-contained\.