Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents

# Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents
Source: [https://arxiv.org/html/2609.02889](https://arxiv.org/html/2609.02889)
Wei Chen Tan1Nurul Aisyah Hassan2Arvind Raman3Li Hua Lim1Ahmad Faiz Razak2Jia Hui Wong3 1Universiti Malaya2Universiti Sains Malaysia3Universiti Putra Malaysia 4Monash University Malaysia Kuala Lumpur / Penang / Serdang, Malaysia

###### Abstract

A growing body of work makes a frozen large language model \(LLM\) into a better agent by*evolving its harness*—the textual scaffolding \(persona, strategy, format rules, control heuristics\) wrapped around the model\. The dominant recipe, exemplified by reflective prompt evolution, treats the harness as a single flat string and optimizes it as a whole\. We ask a sharper question:*where, inside the harness, does the optimization value actually live?*We introduceHarnessEvo, which decomposes the harness into four named, separately\-evolvable slots \(role, task\-strategy, tool/format\-rules, reflection/control\) and tunes each one in turn with the same reflective optimizer under an iso\-budget, paired with a leave\-one\-in / leave\-one\-out \(loi/loo\) protocol that attributes the held\-out gain to individual slots\. We use this structured decomposition not as a leaderboard method but as a*microscope*\. On ALFWorld with a frozen 7B backbone, the full method*ties*both baselines on the binary success metric:HarnessEvoreaches0\.6570\.657versus0\.6420\.642for stock and0\.6420\.642for flat\-string evolution \(McNemarp=0\.617p=0\.617andp=0\.480p=0\.480, both ns\)\. The interesting result is what the microscope reveals\. \(i\)*Localization:*the entire gain is carried by one slot—reflection/control, whose leave\-one\-in gain is\+0\.119\+0\.119\(p=0\.0046p=0\.0046,2222vs66discordant tasks\), while role, strategy and format are individually null\. \(ii\)*A budget\-splitting trap:*the structured method is strongly sub\-additive \(full gain\+0\.015≪∑slots\+0\.015\\ll\\sum\_\{\\text\{slots\}\}leave\-one\-in\+0\.164\+0\.164\) because the reflective optimizer has an accept\-and\-rescore*floor*; splitting an iso\-budget of6464rollouts across four slots leaves1616per slot, below that floor, so every slot froze at its empty seed and the load\-bearing slot was never captured\. \(iii\)*A budget\-allocation prescription:*concentrating the same budget on the high\-credit slot recovers the discarded gain—a targeted single\-slot run reaches0\.7610\.761\(\+0\.119\+0\.119,p=0\.0046p=0\.0046\) at*half*the split’s budget, and an all\-to\-control run at the full budget reaches0\.7240\.724\(\+0\.082\+0\.082,p=0\.0725p=0\.0725, borderline; we report both honestly and do not overclaim at the larger budget\)\. \(iv\)*Task\-contingency:*the phenomenon is benchmark\-specific; on WebShop every slot froze empty and all methods tie \(0\.5180\.518/0\.5390\.539/0\.5450\.545, ns\), and a targeted control run there stays null—confirming the WebShop result is a genuine absence of recurrent, verbalizable control failures, not budget starvation\. The harness\-optimization value that*does*survive is environment\-grounded self\-correction \(“skip a repeated action that returned*Nothing happens*; open closed receptacles before taking; confirm the object is held before placing; stop when the goal is satisfied”\), not reward\-exploiting strategy\. We conclude that harness value is localized, that uniform budget\-splitting is actively harmful, and that credit assignment plus budget concentration should precede any structured agent\-evolution scheme\.

Keywords:large language model agents, self\-evolving agents, prompt optimization, agent scaffolding, credit assignment, leave\-one\-in ablation, budget allocation, sub\-additivity, ALFWorld, WebShop

![Refer to caption](https://arxiv.org/html/2609.02889v1/x1.png)Figure 1:Harness\-optimization value is concentrated in one component\.Per\-slot leave\-one\-in \(loi\) gain on held\-out ALFWorld \(n=134n\{=\}134, frozen 7B backbone\): evolving*only*the reflection/control slot \(c4c\_\{4\}\) yields\+0\.119\+0\.119\(p=0\.0046p\{=\}0\.0046\), while role \(c1c\_\{1\}\), strategy \(c2c\_\{2\}\) and format \(c3c\_\{3\}\) are individually indistinguishable from the stock harness\. Evolving all four slots under the*same*budget split four ways \(Δ=\+0\.015\\Delta\{=\}\+0\.015, ns\) and flat single\-string evolution \(Δ=\+0\.000\\Delta\{=\}\+0\.000, ns\) both tie the baselines: the uniform split*starves*the one slot that carries the gain\. Error bars are95%95\\%bootstrap intervals on the paired held\-out difference\.## 1Introduction

A frozen large language model can be turned into a markedly stronger agent without touching a single weight, simply by improving the text that surrounds it\. The agent’s*harness*—the system preamble, persona description, task strategy, action templates and loop\-control instructions injected around the model at every step—determines, to first order, how an off\-the\-shelf LLM behaves in an interactive environment\. A rapidly growing literature exploits this fact: rather than fine\-tune, it*optimizes the harness*, treating prompt text as a parameter and searching over it with reflective feedback, evolutionary mutation, or memory accumulation\[[66](https://arxiv.org/html/2609.02889#bib.bib76),[50](https://arxiv.org/html/2609.02889#bib.bib19),[7](https://arxiv.org/html/2609.02889#bib.bib77),[18](https://arxiv.org/html/2609.02889#bib.bib18),[1](https://arxiv.org/html/2609.02889#bib.bib17),[47](https://arxiv.org/html/2609.02889#bib.bib21),[61](https://arxiv.org/html/2609.02889#bib.bib22)\]\. The flagship of this family, reflective prompt evolution\[[1](https://arxiv.org/html/2609.02889#bib.bib17)\], reports that evolving a single instruction string can match or beat reinforcement learning at a fraction of the rollout cost\. These methods are attractive precisely because they are cheap, weight\-free, and require only black\-box access to the model\.

Almost all of this work, however, optimizes the harness as an*undifferentiated whole*: a single flat string that is mutated, scored, and selected in one piece\. This conflates components that play very different roles\. A persona line \(“you are a careful household assistant”\), a task\-decomposition strategy \(“first locate the object in the receptacles where it usually lives”\), an action\-format rule \(“emit one action per line using the templatetake <obj\> from <recep\>”\), and a control heuristic \(“never repeat an action that returned*Nothing happens*; stop once the goal is satisfied”\) are all swept into the same string and optimized jointly\. When such a method works, we learn*that*the harness can be improved; we do not learn*which part of it*mattered, and we therefore cannot say whether the next harness\-optimization effort should spend its budget on the persona, the strategy, the format, or the control logic\. The field has a leaderboard but no map\.

This paper asks the map\-making question directly:where, inside the harness, does the optimization value live?We answer it with a deliberately structured instrument\. We decompose the harness into four named, mutually exclusive, separately\-evolvable slots—c1c\_\{1\}role/persona,c2c\_\{2\}task\-strategy,c3c\_\{3\}tool/format\-rules,c4c\_\{4\}reflection/control\-heuristics—and evolve them one at a time by coordinate ascent, reusing an existing reflective prompt optimizer slot\-by\-slot under a fixed total rollout budget that exactly matches the budget given to flat single\-string evolution\. We pair this with a leave\-one\-in / leave\-one\-out \(loi/loo\) credit\-assignment protocol:loievolves a single slot alone and measures its marginal value over the stock harness, whilelooevolves everything except one slot and measures how much the full harness loses without it\.*We stress at the outset that the structured decomposition is the microscope, not the product\.*We do not claim that decomposing the harness yields a better agent; on our binary metric it does not\. We claim that it lets us see something the flat methods cannot\.

What it sees is a striking, and in our view consequential, asymmetry\. On ALFWorld\[[38](https://arxiv.org/html/2609.02889#bib.bib140)\]with a frozen 7B backbone, the full structured method ties both baselines—it reaches0\.6570\.657held\-out success against0\.6420\.642for the stock harness and0\.6420\.642for flat\-string reflective evolution, and neither difference is significant \(McNemarp=0\.617p=0\.617versus stock,p=0\.480p=0\.480versus flat\)\. A reader who saw only this number would conclude, reasonably, that structured harness evolution does nothing\. But theloimap shows that the gain is not absent—it is*localized*\. Evolving only the reflection/control slotc4c\_\{4\}lifts held\-out success by\+0\.119\+0\.119\(p=0\.0046p=0\.0046;2222of the2828tasks on which the two harnesses disagree flip in favour of the evolved control\), whereas evolving only the role, strategy, or format slot is individually null \(\+0\.030\+0\.030,\+0\.007\+0\.007,\+0\.007\+0\.007; all ns\)\. The value of harness optimization, at least on this benchmark, lives almost entirely in one of the four components\.

Why, then, does the full method tie? Because the structured method is strongly*sub\-additive*, and we identify the mechanism\. The full harness’s gain over stock is only\+0\.015\+0\.015, while the sum of the fourloigains is\+0\.164\+0\.164—an order of magnitude larger\. The cause is an interaction between coordinate\-ascent budget\-splitting and a structural property of the reflective optimizer: it has an accept\-and\-rescore*floor*\. Each optimizer iteration spends rollouts scoring a parent and a mutated child on a minibatch, and only when a child is accepted does it spend a further block of rollouts re\-scoring the accepted child on the full validation set\. Splitting an iso\-budget of6464rollouts evenly across four slots leaves1616per slot—below the floor needed to accept and freeze even one mutation—so the structured method froze*every*slot at its empty seed, producing a harness that is byte\-identical to stock and never capturedc4c\_\{4\}’s gain at all\. Naive structured evolution is thus*worse than useless*here: it is worse than a targeted single\-slot run, because decomposition dilutes the budget below the optimizer’s working point\. We call this the*budget\-splitting trap*\.

The trap immediately implies its own cure: under a fixed budget, one should not split it uniformly but instead*concentrate*it on the high\-credit slot\. We verify this in two ways and report both honestly\. A leave\-one\-in run on the control slot at*half*the split’s budget \(3232rollouts, all onc4c\_\{4\}\) reaches0\.7610\.761, a\+0\.119\+0\.119gain over stock that is significant \(p=0\.0046p=0\.0046\)\. An all\-to\-control run at the full budget \(6464rollouts\) reaches0\.7240\.724, a\+0\.082\+0\.082gain that is positive and well above the four\-way split’s0\.6570\.657but borderline by McNemar \(2121vs1010discordant,p=0\.0725p=0\.0725, ns\)\. The*direction*is robust—both targeted runs beat the split, and both clear it comfortably—but the exact magnitude is noisy atn=134n=134given evolution and decoding stochasticity, so we make the deliberately conservative claim that targeted single\-slot evolution recovers a\+0\.08\+0\.08to\+0\.12\+0\.12gain that uniform budget\-splitting discards, and we do*not*assert significance at the larger budget\.

Finally, the whole phenomenon is*task\-contingent*\. On WebShop\[[55](https://arxiv.org/html/2609.02889#bib.bib123)\]with the same backbone and protocol, every slot froze at its empty seed, the full method and both baselines tie \(0\.5180\.518/0\.5390\.539/0\.5450\.545dense score, all ns\), and—crucially—a targeted all\-to\-control run there*also*stays null, with the control slot again freezing empty\. This rules out budget starvation as the explanation on WebShop: the optimizer had the concentrated budget and still found nothing to accept\. The difference between the benchmarks is legible in the audited slot text\. The control rules that help on ALFWorld encode recurrent, verbalizable control failures—repeated dead\-end actions, closed receptacles, premature stopping—that recur across household tasks; WebShop’s purchasing task, with its dense attribute\-match reward, simply does not exhibit a control failure mode that a general heuristic can fix\. The value that harness optimization captures, where it captures anything, is environment\-grounded*self\-correction*, not reward\-exploiting strategy—a point we develop as a constructive counterpoint to concerns about optimizing agents against gameable environment rewards\.

#### Contributions\.

We make four contributions, each stated as an honest characterization rather than a leaderboard claim\.

1. 1\.A component\-decomposed harness and a leave\-one\-in credit\-assignment protocolthat localize*where*harness\-optimization value lives\. On ALFWorld the value is overwhelmingly in the reflection/control slot \(loi\+0\.119\+0\.119, McNemarp=0\.0046p=0\.0046,2222vs66\); role, strategy and format are individually null\. We do not claim a binary win—the full harness ties both stock and flat\-string evolution \(0\.6570\.657/0\.6420\.642/0\.6420\.642, ns\)\.
2. 2\.A sub\-additivity result with an identified mechanism\.The full gain \(\+0\.015\+0\.015\) is far below the sum of the four single\-slot gains \(\+0\.164\+0\.164\)\. The cause is an accept\-and\-rescore floor in coordinate\-ascent reflective evolution: an iso\-budget of6464rollouts split four ways gives1616per slot, below the floor, so the structured method froze every slot empty and never captured the load\-bearing gain\. Uniform structured evolution is worse than a targeted run because decomposition dilutes the budget below the optimizer’s working point\.
3. 3\.A budget\-allocation prescription\.Under a fixed budget, concentrating it on the high\-credit slot recovers the gain that uniform splitting discards: a control\-only run wins by\+0\.119\+0\.119\(p=0\.0046p=0\.0046\) at*half*the split’s budget, and an all\-to\-control run at the full budget confirms the direction \(0\.7240\.724,\+0\.082\+0\.082, borderline\)\. We report the significant \(@​32@32\) and borderline \(@​64@64\) replications side by side and do not overclaim the larger budget\.
4. 4\.A task\-contingency boundary\.Localization is benchmark\-specific\. On WebShop the method yields a full null—every slot froze empty \(0\.5180\.518/0\.5390\.539/0\.5450\.545, ns\)—and a targeted control run there stays null too, showing the WebShop result is a genuine absence of recurrent verbalizable control failures, not budget starvation\. We characterize*when*harness optimization helps \(tasks with recurrent, verbalizable control failures: dead\-end actions, ordering constraints, premature stopping\) via the audited control\-rule text\.

Figure[1](https://arxiv.org/html/2609.02889#S0.F1)previews the central finding\. Section[2](https://arxiv.org/html/2609.02889#S2)situates the work; Section[3](https://arxiv.org/html/2609.02889#S3)formalizes the harness\-optimization problem and the component decomposition; Section[4](https://arxiv.org/html/2609.02889#S4)presentsHarnessEvo, its architecture, the coordinate\-ascent algorithm, and theloi/looprotocol; Section[5](https://arxiv.org/html/2609.02889#S5)gives the experimental setup; Section[6](https://arxiv.org/html/2609.02889#S6)reports the main results; Section[7](https://arxiv.org/html/2609.02889#S7)analyzes credit assignment and the budget\-splitting trap; Sections[8](https://arxiv.org/html/2609.02889#S8)and[9](https://arxiv.org/html/2609.02889#S9)discuss implications and limitations\.

## 2Related Work

### 2\.1Prompt and agent self\-evolution

The premise that a frozen model can be improved by optimizing its surrounding text underlies a large and fast\-moving literature\. Early automatic prompt search treats the instruction as a discrete object to be proposed and selected: APE generates and ranks instructions with the model itself\[[66](https://arxiv.org/html/2609.02889#bib.bib76)\], OPRO casts the LLM as an optimizer that iteratively proposes higher\-scoring prompts from a trajectory of past attempts\[[50](https://arxiv.org/html/2609.02889#bib.bib19)\], and Promptbreeder evolves both task prompts and the mutation prompts that produce them\[[7](https://arxiv.org/html/2609.02889#bib.bib77)\]\. Program\-of\-prompts frameworks compile multi\-stage LLM pipelines and tune their prompts as parameters: DSP and DSPy formalize declarative LLM programs and optimize their instructions and demonstrations\[[17](https://arxiv.org/html/2609.02889#bib.bib48),[18](https://arxiv.org/html/2609.02889#bib.bib18)\]\. A related, test\-time strand adapts the prompt or scratchpad on the fly—dynamic cheatsheets that accumulate reusable hints across queries\[[43](https://arxiv.org/html/2609.02889#bib.bib23)\]and scaling test\-time compute by revising the model’s own intermediate text\[[40](https://arxiv.org/html/2609.02889#bib.bib85),[28](https://arxiv.org/html/2609.02889#bib.bib86)\]\. In all of these the optimized object is the prompt text, but it is optimized as one piece; none asks which span of that text carries the value\.

The work most directly relevant to ours, and the one we treat as the principal foil, is reflective prompt evolution—GEPA\[[1](https://arxiv.org/html/2609.02889#bib.bib17)\]—which evolves a prompt by reflecting on rollout traces, mutating a single string, gating candidates on a minibatch, and maintaining a Pareto frontier over instances\.Agrawalet al\.\[[1](https://arxiv.org/html/2609.02889#bib.bib17)\]show this flat\-string scheme can outperform reinforcement learning at far lower rollout cost\. Our method reuses exactly this optimizer, but applies it slot\-by\-slot rather than to one undifferentiated string; our finding is that the value such an optimizer extracts is concentrated in a single component, and that the structured iso\-budget variant ties and can underperform the flat one*unless*the budget is localized to the load\-bearing slot\. We thus inherit GEPA’s mechanism and ask a question it was not designed to answer: not how much the harness can improve, but which part of it carries the improvement\.

A complementary strand evolves*context*rather than instructions\. Agent Workflow Memory induces reusable workflows from experience and injects them as context\[[47](https://arxiv.org/html/2609.02889#bib.bib21)\]; Agentic Context Engineering grows and curates an evolving context for self\-improving models\[[61](https://arxiv.org/html/2609.02889#bib.bib22)\]; memory architectures such as MemGPT and MemoryBank manage long\-horizon agent state\[[30](https://arxiv.org/html/2609.02889#bib.bib24),[64](https://arxiv.org/html/2609.02889#bib.bib60)\]\. Skill\- and experience\-induction methods—Voyager’s skill library\[[45](https://arxiv.org/html/2609.02889#bib.bib27)\], ExpeL’s experiential learning\[[62](https://arxiv.org/html/2609.02889#bib.bib20)\], and the broader self\-improvement line in which a model bootstraps from its own outputs\[[13](https://arxiv.org/html/2609.02889#bib.bib78),[60](https://arxiv.org/html/2609.02889#bib.bib28),[28](https://arxiv.org/html/2609.02889#bib.bib86)\]—accumulate reusable knowledge across episodes\. A parallel line closes the loop through self\-generated preference signals: self\-rewarding and meta\-rewarding language models judge and improve their own responses\[[59](https://arxiv.org/html/2609.02889#bib.bib81),[48](https://arxiv.org/html/2609.02889#bib.bib82)\], and generative agents simulate believable behaviour from accumulated memory\[[33](https://arxiv.org/html/2609.02889#bib.bib26)\]\. Self\-evolving*tool\-use*optimization, which assigns blame to individual tool calls and mutates a tool\-use policy accordingly, is closely related in spirit to our credit\-assignment goal but operates at the granularity of tool invocations rather than harness components\[*cf\.*[52](https://arxiv.org/html/2609.02889#bib.bib1), whose blame\-aware mutation localizes value at the tool\-call level, where ourloimap localizes it at the harness\-component level\]\. Surveys of LLM\-agent autonomy and self\-improvement map this space\[[42](https://arxiv.org/html/2609.02889#bib.bib4),[6](https://arxiv.org/html/2609.02889#bib.bib67)\]\. Across all of these, the unit of optimization is a monolith—a string, a context, a skill set, a policy—and the question of*which sub\-part of the harness carries the value*is, to our knowledge, not posed\. Our contribution is to pose and answer it, and to show that the answer changes how a fixed optimization budget should be spent\.

### 2\.2LLM\-agent frameworks and interactive benchmarks

The harness we decompose is, concretely, the scaffolding of a reasoning\-and\-acting agent\. ReAct interleaves chain\-of\-thought reasoning with environment actions\[[58](https://arxiv.org/html/2609.02889#bib.bib25)\]and is the loop our agent runs; Reflexion adds verbal self\-reflection across attempts\[[37](https://arxiv.org/html/2609.02889#bib.bib16)\]; Tree\-of\-Thoughts and self\-consistency structure or aggregate the reasoning itself\[[57](https://arxiv.org/html/2609.02889#bib.bib80),[46](https://arxiv.org/html/2609.02889#bib.bib79)\]\. Self\-correction and self\-refinement methods let a model critique and revise its own outputs\[[25](https://arxiv.org/html/2609.02889#bib.bib15),[10](https://arxiv.org/html/2609.02889#bib.bib74),[14](https://arxiv.org/html/2609.02889#bib.bib83)\], though their reliability without external feedback is contested\[[16](https://arxiv.org/html/2609.02889#bib.bib84)\]; our control slot is, in effect, a frozen, distilled, environment\-grounded version of such self\-correction, evolved offline rather than invoked online\. Tool\-use agents extend this to external APIs\[[34](https://arxiv.org/html/2609.02889#bib.bib141)\], and structured tool\-exploration schemes organize the search over tool calls\[[51](https://arxiv.org/html/2609.02889#bib.bib2)\]; retrieval\-augmented generation similarly equips an LLM with external knowledge at inference\[[21](https://arxiv.org/html/2609.02889#bib.bib38),[12](https://arxiv.org/html/2609.02889#bib.bib40),[4](https://arxiv.org/html/2609.02889#bib.bib49)\], a context\-side complement to the prompt\-side scaffolding we study\.

Interactive benchmarks ground these agents\. ALFWorld aligns embodied household tasks with text\-based TextWorld counterparts, requiring multi\-step manipulation under partial observability\[[38](https://arxiv.org/html/2609.02889#bib.bib140)\]; WebShop poses instruction\-conditioned web shopping with a dense attribute\-match reward\[[55](https://arxiv.org/html/2609.02889#bib.bib123)\]; WebArena and GAIA push toward realistic web and general\-assistant tasks\[[65](https://arxiv.org/html/2609.02889#bib.bib126),[26](https://arxiv.org/html/2609.02889#bib.bib125)\]; AgentBench\[[23](https://arxiv.org/html/2609.02889#bib.bib124)\],τ\\tau\-bench\[[56](https://arxiv.org/html/2609.02889#bib.bib30)\], and SWE\-bench\[[15](https://arxiv.org/html/2609.02889#bib.bib127)\]broaden the evaluation surface\. We use ALFWorld and WebShop precisely because they differ in the property our analysis turns out to hinge on: ALFWorld exhibits recurrent, verbalizable control failures \(repeated dead\-end actions, closed receptacles, premature stopping\) that a general heuristic can fix, whereas WebShop’s purchasing task does not, which is exactly why harness optimization helps on the former and is null on the latter\.

### 2\.3Credit assignment, ablation, and component analysis

Our central instrument is credit assignment: attributing an aggregate gain to the parts of a system\. Ablation is the standard tool, but it is usually applied to architectural modules or training\-signal components rather than to the slots of an agent’s prompt harness\. Analyses of individual\-unit importance show that single directions or units can carry disproportionate functional weight in a network\[[27](https://arxiv.org/html/2609.02889#bib.bib142)\], a structural analogue of our finding that one of four harness slots carries essentially all the value\. Decomposition\-and\-recombination strategies such as branch\-solve\-merge\[[36](https://arxiv.org/html/2609.02889#bib.bib137)\]and the generate\-then\-rank pattern\[[22](https://arxiv.org/html/2609.02889#bib.bib121)\]restructure LLM computation into parts, but do not measure per\-part marginal value under a shared budget\. A separate question is what value the frozen model already realizes*internally*: work framing in\-context learning as implicit optimization\[[44](https://arxiv.org/html/2609.02889#bib.bib106),[2](https://arxiv.org/html/2609.02889#bib.bib105)\]helps explain why slots that merely restate capabilities the model already has \(persona, format\) add nothing in our map, while a slot that supplies missing control structure does\. Our leave\-one\-in / leave\-one\-out protocol is, deliberately, the simplest possible credit\-assignment design that isolates a slot’s marginal contribution \(loi\) and its removal cost \(loo\) on held\-out tasks with paired significance tests, and our sub\-additivity result—full gain far below the sum of single\-slot gains—is a quantitative statement about how badly per\-part value composes when the optimizer’s budget is split\.

### 2\.4Reward over\-optimization, Goodhart effects, and grounded value

Optimizing an agent against a fixed objective invites the failure mode in which the proxy is gamed rather than the goal achieved\. This is the classical Goodhart phenomenon\[[9](https://arxiv.org/html/2609.02889#bib.bib132)\], formalized for reward learning as reward over\-optimization and hacking: as optimization pressure increases, measured reward rises while true performance eventually falls\[[8](https://arxiv.org/html/2609.02889#bib.bib133),[39](https://arxiv.org/html/2609.02889#bib.bib134),[31](https://arxiv.org/html/2609.02889#bib.bib136),[3](https://arxiv.org/html/2609.02889#bib.bib135)\]\. This concern has shaped preference\-based fine\-tuning, from learning rewards for summarization\[[41](https://arxiv.org/html/2609.02889#bib.bib129)\]to direct preference optimization and its AI\-feedback variants\[[35](https://arxiv.org/html/2609.02889#bib.bib130),[20](https://arxiv.org/html/2609.02889#bib.bib119)\]\. The risk is acute when an agent is tuned against an automatically computed environment reward, and broad treatments of agent misalignment catalogue when such optimization induces specification\-gaming behaviour\[*e\.g\.*[53](https://arxiv.org/html/2609.02889#bib.bib3), which surveys the taxonomy and causes of optimizing agents against environment reward, and motivates why a*gameable*dense reward is exactly where a harness optimizer should decline to overfit\]\. Two aspects of our results speak to this concern as a constructive counterpoint\. First, the harness value that survives on ALFWorld is*environment\-grounded self\-correction*—rules that avoid repeated dead\-end actions and verify preconditions—rather than a strategy that exploits the binary success checker; it generalizes across held\-out task types because it fixes a real, recurrent control failure\. Second, on WebShop, whose dense reward is the more gameable of the two, the optimizer declines to fill any slot even when handed a concentrated budget: it finds no general control rule worth accepting, and the method returns a genuine null rather than an over\-fit strategy\. In both cases the optimizer captures grounded correction where it exists and abstains where it does not, which is the behaviour one would hope for from a harness optimizer used in the presence of an imperfect reward\. Related lines on constitutional and preference\-based alignment\[[5](https://arxiv.org/html/2609.02889#bib.bib75),[29](https://arxiv.org/html/2609.02889#bib.bib128)\]and on LLM\-as\-judge evaluation—both its use\[[63](https://arxiv.org/html/2609.02889#bib.bib111),[24](https://arxiv.org/html/2609.02889#bib.bib110),[19](https://arxiv.org/html/2609.02889#bib.bib112)\]and its known biases\[[11](https://arxiv.org/html/2609.02889#bib.bib120),[32](https://arxiv.org/html/2609.02889#bib.bib117)\]—address the reward side of this problem, since a harness optimizer is only as trustworthy as the signal it optimizes; our contribution is on the optimization side, characterizing what a black\-box harness optimizer does and does not extract from a given signal\. Because our ALFWorld reward is an exact environment verifier rather than a learned judge, the value we localize is not an artifact of a gameable scorer\.

### 2\.5Relation to recursive self\-evolution

A distinct axis of agent self\-improvement is*recursion*: agents \(or multi\-agent systems\) that revise their own improvement procedure, not merely their prompt or context\[[54](https://arxiv.org/html/2609.02889#bib.bib5),[67](https://arxiv.org/html/2609.02889#bib.bib6)\]\. Recursion is orthogonal to the question we study\. We hold the optimizer fixed and ask where, in a single non\-recursive pass, its value lands inside the harness; we do not iterate the meta\-procedure\. We make this distinction in prose to position the work, but our empirical claims concern only single\-pass, coordinate\-ascent harness optimization, and our budget\-splitting analysis would apply to any scheme—recursive or not—that distributes a fixed rollout budget across named harness components before the optimizer’s accept\-floor is met\.

## 3Problem Formulation

#### Frozen\-model agent\.

We study an interactive agent built from a*frozen*language modelMM\(no gradient updates\) acting in an environmentℰ\\mathcal\{E\}over discrete steps\. At each step the agent observes a textual state, emits an action, and receives a new observation; an episode terminates with an environment\-verified outcome\. The model’s behaviour is conditioned on a textual*harness*HHthat is injected at a single, fixed point in the agent’s prompt construction—the same point a ReAct agent uses for its strategy preamble\[[58](https://arxiv.org/html/2609.02889#bib.bib25)\]\. Crucially,HHenters only as text: it never alters the rollout*code path*\(decoding temperature, environment, action parser, verifier are all fixed\)\. Two harnesses that render to the same string produce byte\-identical rollouts\.

###### Definition 1\(Harness\-optimization problem\)\.

Letρ​\(H\)\\rho\(H\)denote the expected held\-out success of the frozen\-model agent when injected with harness textHH, estimated by rolling out a held\-out task set under a deterministic \(temperature=0\\text\{temperature\}\{=\}0\) policy\. Given a rollout budgetBB\(the number of environment rollouts the optimizer may spend during search\), the harness\-optimization problem is to findH⋆∈arg⁡maxH⁡ρ​\(H\)H^\{\\star\}\\in\\arg\\max\_\{H\}\\rho\(H\)using at mostBBrollouts, withHHselected on a held\-out validation set and reported on a disjoint test set\.

#### The stock harness\.

The neutral baseline is the*stock*harnessH0H\_\{0\}, which carries no task guidance and renders to the empty string\. InjectingH0H\_\{0\}is byte\-identical to running the agent with no preamble at all\. Every optimization method must*earn*the content ofHHthrough search starting from this empty seed; this makes flat single\-string evolution and our structured method fair to compare, since both begin fromρ​\(H0\)\\rho\(H\_\{0\}\)and spend the same budgetBB\.

#### Component decomposition\.

We decompose the harness into four named, mutually exclusive slots in a fixed render order,

H=\(c1,c2,c3,c4\),render​\(H\)=c1​‖c2‖​c3∥c4,H=\(c\_\{1\},c\_\{2\},c\_\{3\},c\_\{4\}\),\\qquad\\mathrm\{render\}\(H\)=c\_\{1\}\\,\\\|\\,c\_\{2\}\\,\\\|\\,c\_\{3\}\\,\\\|\\,c\_\{4\},\(1\)where∥\\\|concatenates non\-empty slots \(each under a short neutral section header\) and drops empty ones, so thatrender​\(H0\)=ε\\mathrm\{render\}\(H\_\{0\}\)=\\varepsilon\(the empty string\)\. The four slots, with disjoint semantic scope, are:

- •c1c\_\{1\}role / persona: who the agent is and the disposition it adopts \(careful, methodical, goal\-driven\)\. No action templates or stop rules\.
- •c2c\_\{2\}task\-strategy: the high\-level approach for decomposing and solving a task \(what to do first, how to search the environment\)\. No persona, syntax, or loop heuristics\.
- •c3c\_\{3\}tool / format\-rules: the exact action templates and output format the agent must emit, and any hard syntactic rules\. Copy\-usable; no strategy or control logic\.
- •c4c\_\{4\}reflection / control\-heuristics: loop discipline and self\-correction rules \(never repeat a failed action, verify before finishing, when to backtrack, when to stop\), optionally with a small structured control\-knob line\. No persona, strategy, or low\-level syntax\.

#### Credit\-assignment objective\.

Beyond findingH⋆H^\{\\star\}, our goal is to*attribute*the gainρ​\(H⋆\)−ρ​\(H0\)\\rho\(H^\{\\star\}\)\-\\rho\(H\_\{0\}\)to the four slots\. For a slotckc\_\{k\}we define two held\-out quantities, computed by evolving the relevant slot\(s\) and evaluating on the test set:

ΔkLOI\\displaystyle\\Delta^\{\\mathrm\{LOI\}\}\_\{k\}=ρ​\(LOIk\)−ρ​\(H0\),\\displaystyle=\\rho\\big\(\\mathrm\{LOI\}\_\{k\}\\big\)\-\\rho\(H\_\{0\}\),\(marginal value ofckc\_\{k\}alone over stock\)\(2\)ΔkLOO\\displaystyle\\Delta^\{\\mathrm\{LOO\}\}\_\{k\}=ρ​\(LOOk\)−ρ​\(Hfull\),\\displaystyle=\\rho\\big\(\\mathrm\{LOO\}\_\{k\}\\big\)\-\\rho\(H\_\{\\mathrm\{full\}\}\),\(loss whenckc\_\{k\}is removed from the full harness\)\(3\)whereLOIk\\mathrm\{LOI\}\_\{k\}keeps only slotckc\_\{k\}evolved \(others at the empty seed\) andLOOk\\mathrm\{LOO\}\_\{k\}evolves all slots exceptckc\_\{k\}\. A value that is*localized*to slotckc\_\{k\}exhibits a large positiveΔkLOI\\Delta^\{\\mathrm\{LOI\}\}\_\{k\}with the other slots’loigains near zero\.*Sub\-additivity*is the statementρ​\(Hfull\)−ρ​\(H0\)≪∑kΔkLOI\\rho\(H\_\{\\mathrm\{full\}\}\)\-\\rho\(H\_\{0\}\)\\;\\ll\\;\\sum\_\{k\}\\Delta^\{\\mathrm\{LOI\}\}\_\{k\}, i\.e\. the jointly optimized harness captures far less than the sum of the individually optimized slots\. Table[1](https://arxiv.org/html/2609.02889#S3.T1)collects the notation\.

Table 1:Notation\.

## 4Method:HarnessEvo

HarnessEvohas three parts: \(i\) the four\-slot harness of Section[3](https://arxiv.org/html/2609.02889#S3), \(ii\) a coordinate\-ascent procedure that evolves the slots one at a time with an existing reflective optimizer under an iso\-budget, and \(iii\) theloi/loocredit\-assignment protocol\. The architecture is shown in Figure[2](https://arxiv.org/html/2609.02889#S4.F2)\.

### 4\.1Architecture and the single injection point

The defining design choice is that the harness is the*only*thing that varies\. The four slots are rendered, per Eq\. \([1](https://arxiv.org/html/2609.02889#S3.E1)\), into a single string that is injected at the exact point every comparison method uses—the strategy preamble of the ReAct loop\. The backboneMM, the decoding temperature \(0\), the environment, the action parser, and the success verifier are identical across stock, flat\-string evolution, andHarnessEvo\. Consequently the stock harness \(all slots empty\) reproduces the stock ReAct preamble byte\-for\-byte, and any measured difference is attributable solely to injected text content and structure, never to a changed rollout path\. This is what licenses the credit\-assignment interpretation: a slot’sloieffect is a clean text intervention\.

c1c\_\{1\}rolec2c\_\{2\}strategyc3c\_\{3\}formatc4c\_\{4\}control4 evolvable harness slotsGEPAGEPAGEPAGEPAper\-slotreflective evolutionfreeze & rendersingle stringHHsingle fixedinjection pointfrozenMMReAct rollout\(held\-out test\)iso\-budgetBBsplit across slots:B/4B/4per slot<<accept\-floor⇒\\Rightarrowall freeze emptyloo: evolve all\-but\-one slot⇒\\Rightarrowremoval cost \(*necessity*\)loi: evolve one slot alone⇒\\Rightarrowmarginal value \(*sufficiency*\)credit\-assignment map: held\-out success, paired McNemarFigure 2:HarnessEvoarchitecture\.The harness is four named slots \(c1c\_\{1\}role,c2c\_\{2\}strategy,c3c\_\{3\}format,c4c\_\{4\}control\)\. Coordinate ascent evolves each slot in turn with the*same*reflective optimizer \(GEPA\), freezing earlier slots and seeding later ones \(dashed arrows\); the per\-slot budgets sum to the flat method’s budgetBB\(iso\-budget\)\. The selected slots are frozen, rendered into a single string, and injected at the*single*fixed injection point of the frozen\-model ReAct rollout\. The leave\-one\-in \(loi\) and leave\-one\-out \(loo\) probes evolve one slot alone or all\-but\-one to produce a held\-out credit\-assignment map\. The red annotation marks the failure analyzed in Section[7](https://arxiv.org/html/2609.02889#S7): a four\-way split putsB/4B/4below the optimizer’s accept\-and\-rescore floor, so every slot freezes empty\.
### 4\.2Coordinate\-ascent evolution under an iso\-budget

We evolve the four slots by coordinate ascent \(Algorithm[1](https://arxiv.org/html/2609.02889#algorithm1)\)\. The optimizer is the reflective prompt\-evolution loop of GEPA\[[1](https://arxiv.org/html/2609.02889#bib.bib17)\], reused unchanged: it maintains a pool of candidate strings, selects a parent from the instance Pareto frontier, mutates it by reflecting on recent rollout traces, accepts the child if it does not lose on a training minibatch, and re\-scores accepted children on the full validation set, returning the best\-validation candidate\. We run this optimizer once per slot\. When evolving slotckc\_\{k\}, all earlier slots are*frozen*at their already\-evolved text and all later slots are held at their empty seed; the optimizer’s internal empty seed candidate therefore corresponds exactly to “the current harness with slotckc\_\{k\}neutral,” the correct baseline for that coordinate step\. A slot\-scoped reflection meta\-prompt instructs the reflection model to improve*only*slotckc\_\{k\}and shows the other slots as fixed context, so the optimizer does not duplicate or fight content that lives in another slot\.

The total rollout budget is capped to exactly equal the budget given to flat single\-string evolution \(iso\-budget\): we splitBBacross the evolved slots, and because the optimizer counts rollouts internally and stops at its per\-slot cap, the sum of per\-slot budgets equalsBBby construction\. The rollout path is, again, byte\-identical to flat evolution; only the content and structure of the injected text differ\.

Input:train set

TT, validation set

VV, frozen model

MM, slot order

\(c1,…,cm\)\(c\_\{1\},\\dots,c\_\{m\}\), total budget

BB, reflective optimizerGepa

Output:evolved harness

HfullH\_\{\\mathrm\{full\}\}and a per\-slot history

H←H0H\\leftarrow H\_\{0\}

//stock: all slots empty

\(b1,…,bm\)←SplitEvenly​\(B,m\)\(b\_\{1\},\\dots,b\_\{m\}\)\\leftarrow\\textsc\{SplitEvenly\}\(B,m\)

//per\-slot budgets,∑kbk=B\\sum\_\{k\}b\_\{k\}=B

for*k←1k\\leftarrow 1tomm*do

ctx←\\textit\{ctx\}\\leftarrowrender of all slots

≠ck\\neq c\_\{k\}as fixed context

tmplk←\\textit\{tmpl\}\_\{k\}\\leftarrowslot\-scoped reflection meta\-prompt for

ckc\_\{k\}withctxbaked in

define

rollk​\(text,τ\):=rollout​\(M,render​\(H​\[ck←text\]\),τ\)\\mathrm\{roll\}\_\{k\}\(\\textit\{text\},\\tau\):=\\mathrm\{rollout\}\\big\(M,\\mathrm\{render\}\(H\[c\_\{k\}\{\\leftarrow\}\\textit\{text\}\]\),\\tau\\big\)
\(bestk,valk,infok\)←Gepa​\(T,V,M;budget​bk,rollk,tmplk\)\(\\textit\{best\}\_\{k\},\\,\\textit\{val\}\_\{k\},\\,\\textit\{info\}\_\{k\}\)\\leftarrow\\textsc\{Gepa\}\\big\(T,V,M;\\ \\text\{budget \}b\_\{k\},\\ \\mathrm\{roll\}\_\{k\},\\ \\textit\{tmpl\}\_\{k\}\\big\)
H←H​\[ck←bestk\]H\\leftarrow H\[c\_\{k\}\\leftarrow\\textit\{best\}\_\{k\}\]
//freeze val\-selected best into slotckc\_\{k\}

return

H,historyH,\\ \\text\{history\}

Algorithm 1HarnessEvocoordinate\-ascent harness evolution \(iso\-budget\)
### 4\.3The leave\-one\-in / leave\-one\-out protocol

To produce the credit\-assignment map, we instantiate the same evolution machinery in two restricted modes\. A*leave\-one\-in*run for slotckc\_\{k\}evolves only that slot \(all others fixed at the empty seed\) and measuresΔkLOI\\Delta^\{\\mathrm\{LOI\}\}\_\{k\}, the marginal value ofckc\_\{k\}alone over stock\. A*leave\-one\-out*run for slotckc\_\{k\}evolves all slots exceptckc\_\{k\}, at the same total budgetBBas the full method, and measuresΔkLOO\\Delta^\{\\mathrm\{LOO\}\}\_\{k\}, how much the full harness loses withoutckc\_\{k\}\. The two probes are complementary:loiasks “is this slot sufficient?” andlooasks “is this slot necessary?”\. Each evolved harness is frozen and evaluated once on the held\-out test set under the shared rollout; we report bootstrap confidence intervals on the test success rate and paired McNemar tests against the relevant reference \(stock forloi, full forloo\)\. Single\-slotloiruns use budgetB/2B/2by default—half the four\-way split’s total—which, as Section[7](https://arxiv.org/html/2609.02889#S7)shows, is already above the optimizer’s accept\-floor and is part of the reason the targeted runs succeed where the split fails\.

## 5Experimental Setup

#### Backbone\.

All experiments use a single*frozen*Qwen2\.5\-7B model\[[49](https://arxiv.org/html/2609.02889#bib.bib32)\]as both the task policy and the reflection model, with no weight updates anywhere \(the fairness iron law: the same model that acts also reflects\)\. Decoding is deterministic \(temperature=0\\text\{temperature\}\{=\}0\) for rollouts; the reflection step samples mutations at higher temperature, as in GEPA\[[1](https://arxiv.org/html/2609.02889#bib.bib17)\]\.

#### Benchmarks and splits\.

OnALFWorld\[[38](https://arxiv.org/html/2609.02889#bib.bib140)\]we draw the train and validation tasks from a balanced 30\-task development manifest and evaluate on a*disjoint*held\-out test manifest ofn=134n=134tasks \(zero leakage by construction\)\. The optimizer sees\|T\|=30\|T\|=30training and a small held\-out validation set for candidate selection; episodes run up to4040steps with environment\-verified binary success\. OnWebShop\[[55](https://arxiv.org/html/2609.02889#bib.bib123)\]we use a held\-out test set ofn=80n=80sessions with the benchmark’s dense\[0,1\]\[0,1\]attribute\-match score; train and validation sessions are disjoint\. Test sets are fixed across all methods\.

#### Budgets\.

The primary arm fixes an iso\-budget ofB=64B=64rollouts \(*B64*\): stock spends≈0\\approx 0, flat\-string evolution spends the budget on one string, andHarnessEvosplits it across four slots \(1616per slot\)\. A corroboration arm uses a higher budget,*B120*\(B=120B=120, with a smaller validation set and minibatch\), to check robustness of the direction\. For the budget\-allocation study we additionally run single\-slotloiatB/2=32B/2=32rollouts and an all\-to\-control run at the full6464rollouts\.

#### Methods\.

We compare:Stock\(empty harnessH0H\_\{0\}\);GEPA\-flat\(flat single\-string reflective evolution at budgetBB, the principal foil\);HarnessEvo\-full\(coordinate ascent over all four slots at budgetBB\); the fourloiruns \(one per slot\); the fourlooruns \(all\-but\-one per slot\); and the budget\-allocation arms \(all\-to\-c4at3232and6464\)\. All methods share the identical rollout path\.

#### Metrics and statistics\.

The primary metric is held\-out success rate \(ALFWorld, binary, single\-greedy\) or dense score \(WebShop\)\. We report10,00010\{,\}000\-resample bootstrap95%95\\%confidence intervals on each rate, and*paired*significance tests on the per\-task held\-out outcomes: McNemar’s test for the binary ALFWorld metric \(reporting discordant counts andpp\) and a paired bootstrap on the WebShop dense score\. Following the journal\-style presentation conventions we adopt, we report every number with an interval and a paired test, label non\-significant results “ns,” andbold the true best in each column even when it is a baseline\. We use∗,∗∗,∗∗∗\*,\*\*,\*\*\*forp<0\.05,0\.01,0\.001p<0\.05,0\.01,0\.001\.

## 6Results

### 6\.1Main comparison: the full method ties

Table[2](https://arxiv.org/html/2609.02889#S6.T2)reports the primary iso\-budget \(B​64B64\) comparison on ALFWorld and the cross\-benchmark WebShop arm\. On the binary success metric,HarnessEvo\-full reaches0\.6570\.657, against0\.6420\.642for both the stock harness and flat\-string evolution\. Neither difference is significant: McNemar givesΔ=\+0\.015\\Delta=\+0\.015,p=0\.617p=0\.617versus stock \(33vs11discordant\) andΔ=\+0\.015\\Delta=\+0\.015,p=0\.480p=0\.480versus GEPA\-flat \(22vs0discordant\); flat\-string evolution itself ties stock exactly \(Δ=0\.000\\Delta=0\.000,p=0\.617p=0\.617\)\.*All three methods tie\.*We bold the true best per column; because the three are statistically indistinguishable, the honest reading is a three\-way tie, and we mark the baselines as co\-best rather than manufacture a win for the structured method\. On WebShop the same pattern holds at the level of dense score:0\.5180\.518\(stock\) /0\.5390\.539\(GEPA\) /0\.5450\.545\(HarnessEvo\), withHarnessEvoversus stockΔ=\+0\.027\\Delta=\+0\.027, paired\-bootstrapp=0\.054p=0\.054\(ns\), and every slot frozen empty \(Section[7](https://arxiv.org/html/2609.02889#S7)\)\. A reader stopping here would conclude that structured harness evolution does nothing; the per\-component map shows why that conclusion is incomplete\.

Table 2:Main iso\-budget comparison \(the full method ties\)\.ALFWorld held\-out success \(n=134n\{=\}134, single\-greedy,1010k\-bootstrap95%95\\%CI\) at budgetB=64B\{=\}64, and the cross\-benchmark WebShop dense score \(n=80n\{=\}80\)\. Paired McNemar \(ALFWorld\) / paired bootstrap \(WebShop\) against*Stock*; ns==not significant\.Boldmarks the true best per column; the three methods are statistically indistinguishable on each benchmark, so the baselines are co\-best andHarnessEvodoes*not*win\.*Reading\.*On ALFWorld the three are a tie \(alsoHarnessEvovs GEPA\-flat:Δ=\+0\.015\\Delta\{=\}\+0\.015,p=0\.480p\{=\}0\.480ns\)\. On WebShopHarnessEvovs Stock isΔ=\+0\.027\\Delta\{=\}\+0\.027, paired\-bootstrapp=0\.054p\{=\}0\.054\(ns\)\. Bolding follows the honesty rule: the best is a tie, and on ALFWorld a baseline is \(co\-\)best\.

### 6\.2Per\-component credit\-assignment map

Table[3](https://arxiv.org/html/2609.02889#S6.T3)reports theloiandloomap on ALFWorld atB​64B64; it is the analytical heart of the paper and is visualized in Figures[1](https://arxiv.org/html/2609.02889#S0.F1)and[3](https://arxiv.org/html/2609.02889#S6.F3)\. The marginal value is overwhelmingly concentrated in the reflection/control slotc4c\_\{4\}: evolving it alone lifts held\-out success byΔc4LOI=\+0\.119\\Delta^\{\\mathrm\{LOI\}\}\_\{c\_\{4\}\}=\+0\.119\(p=0\.0046p=0\.0046,∗⁣∗\*\*;2222of the2828discordant tasks flip toward the evolved control\), reaching0\.7610\.761success\. The other three slots are individually null:Δc1LOI=\+0\.030\\Delta^\{\\mathrm\{LOI\}\}\_\{c\_\{1\}\}=\+0\.030\(ns\),Δc2LOI=\+0\.007\\Delta^\{\\mathrm\{LOI\}\}\_\{c\_\{2\}\}=\+0\.007\(ns\),Δc3LOI=\+0\.007\\Delta^\{\\mathrm\{LOI\}\}\_\{c\_\{3\}\}=\+0\.007\(ns\)\. The leave\-one\-out side is consistent with a single load\-bearing slot: removing any one slot from the \(near\-empty\) full harness changes held\-out success by at most0\.0670\.067and never significantly \(Δc1LOO=−0\.067\\Delta^\{\\mathrm\{LOO\}\}\_\{c\_\{1\}\}=\-0\.067,Δc2LOO=−0\.007\\Delta^\{\\mathrm\{LOO\}\}\_\{c\_\{2\}\}=\-0\.007,Δc3LOO=−0\.007\\Delta^\{\\mathrm\{LOO\}\}\_\{c\_\{3\}\}=\-0\.007,Δc4LOO=−0\.015\\Delta^\{\\mathrm\{LOO\}\}\_\{c\_\{4\}\}=\-0\.015, all ns\)—unsurprising, since the full harness froze almost entirely empty and there is little to remove\. The decisive quantity is the comparison of the full gain to the sum of single\-slot gains: the full harness improves over stock by only\+0\.015\+0\.015, while theloigains sum to\+0\.164\+0\.164\. The system is strongly*sub\-additive*; the structured method captures roughly one\-tenth of what its components individually offer\.

Table 3:Per\-component credit\-assignment map on ALFWorld\(B​64B64,n=134n\{=\}134,1010k\-bootstrap95%95\\%CI\)\.loi: evolve only this slot,Δ\\Deltavs*Stock*\.loo: evolve all\-but\-this\-slot,Δ\\Deltavs*HarnessEvo\-full*\. McNemarpp;∗∗p<0\.01\*\*\\,p\{<\}0\.01\. The value is localized toc4c\_\{4\}\(control\); the others are individually null\. Full gain \(\+0\.015\+0\.015\)≪∑loi\\ll\\sum\\\!\\textsc\{loi\}\(\+0\.164\+0\.164\): strongly*sub\-additive*\.![Refer to caption](https://arxiv.org/html/2609.02889v1/x2.png)Figure 3:Per\-component attribution\.Left: leave\-one\-in held\-out gain over stock per slot—onlyc4c\_\{4\}\(control\) is load\-bearing \(\+0\.119\+0\.119,∗⁣∗\*\*\)\. Right: leave\-one\-out change relative to the full harness—no single removal is significant, because the full harness froze nearly empty\. Bars are95%95\\%bootstrap intervals on the paired held\-out difference \(n=134n\{=\}134\)\.
### 6\.3Budget corroboration \(B120\) and budget allocation

The higher\-budget arm \(*B120*\) corroborates the direction while underscoring that magnitudes are noisy at this sample size\. There, stock measures higher \(0\.6720\.672\), flat evolution gives0\.6420\.642, andHarnessEvo\-full gives0\.6270\.627; all pairwise differences are ns\. The control slot remains the most positive single\-slot direction \(Δc4LOI=\+0\.052\\Delta^\{\\mathrm\{LOI\}\}\_\{c\_\{4\}\}=\+0\.052, ns here—non\-significant because B120 stock is higher and the evolved control validated weaker\), and the system is sub\-additive again \(full−0\.045\-0\.045vs∑loi=\+0\.052\\sum\\\!\\textsc\{loi\}=\+0\.052\)\. We treat B120 as a robustness check on the*ordering*of slots, not as an independent significant gain\.

The budget\-allocation result is the practical payoff and we report it with full honesty\. Table[4](https://arxiv.org/html/2609.02889#S6.T4)and Figure[4](https://arxiv.org/html/2609.02889#S6.F4)compare, at fixed total budget, the uniform four\-way split against concentrating the budget on the high\-credit control slot\. The four\-way split \(1616/slot\) reaches0\.6570\.657\(ns\) and flat evolution reaches0\.6420\.642\(ns\)\. Concentrating onc4c\_\{4\}recovers a large gain: the control\-onlyloiat*half*the split’s budget \(3232rollouts\) reaches0\.7610\.761, a\+0\.119\+0\.119gain over stock that is significant \(p=0\.0046p=0\.0046,2222vs66discordant\); the all\-to\-control run at the*full*budget \(6464rollouts\) reaches0\.7240\.724, a\+0\.082\+0\.082gain that is positive and clearly above the split’s0\.6570\.657but borderline by McNemar \(2121vs1010discordant,p=0\.0725p=0\.0725, ns\)\. We do not overclaim significance at6464: the direction is robust \(both targeted runs beat the split, both exceed it by a wide margin\), but the exact magnitude fluctuates with evolution and decoding stochasticity, so the defensible claim is that targeted single\-slot evolution recovers a\+0\.08\+0\.08to\+0\.12\+0\.12held\-out gain that the uniform split discards\.

Table 4:Budget allocation at fixed budget on ALFWorld\(n=134n\{=\}134,1010k\-bootstrap95%95\\%CI, McNemar vs*Stock*\)\. Concentrating the budget on the high\-credit control slot recovers the gain the uniform four\-way split discards\. We report*both*the significant half\-budget replication \(@32\) and the borderline full\-budget replication \(@64\);boldmarks the true best\.†\\daggerCI for all\-to\-c4c\_\{4\}@64 \(0\.7240\.724\) overlaps that of @32; the point estimate is lower and only borderline\-significant\. We deliberately do not claim significance at the full budget\.

![Refer to caption](https://arxiv.org/html/2609.02889v1/x3.png)Figure 4:Budget allocation\.Left \(ALFWorld,n=134n\{=\}134\): at the same budgetB=64B\{=\}64, concentrating on the control slot \(all\-to\-c4c\_\{4\}, point shown at the significant @32 replication,0\.7610\.761,∗⁣∗\*\*\) far exceeds the four\-way split \(0\.6570\.657, ns\) and flat evolution \(0\.6420\.642, ns\)\. Right \(WebShop,n=80n\{=\}80\): the null is*genuine*, not budget starvation—even all\-to\-c4c\_\{4\}@64 stays at0\.5350\.535\(ns\), because the control slot froze empty for lack of a recurrent, verbalizable control failure to fix\.

## 7Analysis: Credit Assignment and the Budget\-Splitting Trap

### 7\.1The value is localized to control

Theloimap \(Table[3](https://arxiv.org/html/2609.02889#S6.T3), Figure[3](https://arxiv.org/html/2609.02889#S6.F3)\) makes a clean statement: of the four harness components, only reflection/control carries optimization value on ALFWorld, and it carries essentially all of it \(Δc4LOI=\+0\.119\\Delta^\{\\mathrm\{LOI\}\}\_\{c\_\{4\}\}=\+0\.119, the sum over the other three is\+0\.044\+0\.044and individually each is ns\)\. This is not an artifact of the optimizer failing on the other slots—the role and strategy slots*were*explored and accepted candidates in the targeted setting \(their pools grew\), the candidates simply did not transfer to held\-out success\. The asymmetry is semantic: the frozen 7B agent already knows how to phrase actions \(so the format slotc3c\_\{3\}adds nothing\) and already adopts a reasonable persona and plan \(soc1c\_\{1\}andc2c\_\{2\}add nothing\), but it lacks*loop discipline*—it repeats dead\-end actions, fails to open closed receptacles before taking, and stops prematurely—and the control slot supplies exactly that\. The audited control text that produced the gain is: “*skip a repeated action that returned ‘Nothing happens’; open closed receptacles before taking; confirm the object is held before placing; stop once the goal is satisfied\.*” It is a compact, general, environment\-grounded correction, not a task\-specific trick\.

### 7\.2Sub\-additivity and the accept\-and\-rescore floor

Why does the full method capture only\+0\.015\+0\.015when its components offer\+0\.164\+0\.164? The mechanism is a structural property of the reflective optimizer interacting with budget\-splitting\. Each optimizer iteration spends2×minibatch2\\times\\text\{minibatch\}rollouts scoring a parent and a candidate child;*only on acceptance*does it spend an additional block of rollouts re\-scoring the accepted child on the full validation set before admitting it to the pool\. There is therefore a minimum budget—an*accept\-and\-rescore floor*—below which the optimizer cannot accept and freeze even a single mutation: it must afford at least one parent/child minibatch comparison*and*the subsequent full\-validation rescore\. With the B64 configuration \(minibatch33, validation≈10\\approx 10\), an accept\-bearing iteration costs about2×3\+10=162\{\\times\}3\+10=16rollouts—exactly the1616rollouts a four\-way split allots to each slot\. The split therefore lands the budget*at or below*the floor for every slot, and the empirical consequence is unambiguous: at1616/slot,*zero*of the four slots accepted any candidate and all four froze at the empty seed \(Table[5](https://arxiv.org/html/2609.02889#S7.T5)\)\. The structured harness that emerged was byte\-identical to stock, which is whyHarnessEvo\-full ties stock to within\+0\.015\+0\.015of noise\. By contrast, at3232/slot three of four slots accept a mutation but only one freezes non\-empty text \(the load\-bearing control slot\), and at6060/slot all four accept and freeze text\. The floor is real, and uniform splitting drives the per\-slot budget straight through it\.

Table 5:The accept\-and\-rescore floor\.Number of slots \(out of44\) that*accepted*a mutation and that*froze non\-empty text*, by per\-slot budget\. At1616/slot \(the B64 four\-way split\) every slot freezes empty; the floor sits between1616and3232rollouts per slot\.This yields the paper’s sharpest practical statement:*naive structured evolution can be worse than a targeted single\-slot run*, because decomposing the harness divides the budget below the optimizer’s working point\. The remedy is not more structure but better allocation\.

### 7\.3Budget concentration recovers the gain

If the floor explains the failure, then concentrating the budget should fix it, and it does \(Table[4](https://arxiv.org/html/2609.02889#S6.T4), Figure[4](https://arxiv.org/html/2609.02889#S6.F4)\)\. Spending the entire budget on the high\-credit control slot keeps every rollout above the floor: the control\-only run accepts and freezes a substantive rule, and held\-out success rises to0\.7610\.761at half the split’s budget \(\+0\.119\+0\.119,∗⁣∗\*\*\) and0\.7240\.724at the full budget \(\+0\.082\+0\.082, borderline\)\. The half\-budget run is, strikingly, the strongest configuration we observe—more budget is not the issue;*where*the budget goes is\. This is the prescription we draw: under a fixed budget, identify the high\-credit slot \(via a cheaploiprobe\) and concentrate the budget there rather than splitting it uniformly across named components\. We report the @32 \(significant\) and @64 \(borderline\) replications together to be candid that the magnitude is noisy; the ordering—targeted≫\\ggsplit≈\\approxflat≈\\approxstock—is what is robust\.

### 7\.4Task\-type breakdown: where the control rule pays off

Figure[5](https://arxiv.org/html/2609.02889#S7.F5)breaks the control\-slot gain down by ALFWorld task type \(at the significantB=32B\{=\}32setting\)\. The gains concentrate exactly where the control rule’s content predicts\. The largest improvement is onlook\_at\_obj\_in\_lighttasks \(Δ=\+0\.444\\Delta=\+0\.444,n=18n=18\): these require the agent not to give up and to use the desk lamp while holding the object, a classic premature\-stopping / loop failure that the control rule directly addresses\. The next largest is onclean\_then\_place\(Δ=\+0\.226\\Delta=\+0\.226,n=31n=31\), a multi\-step, closed\-receptacle type where “open before taking” and “confirm holding before placing” bite\. Simplepick\_and\_place\(Δ=\+0\.042\\Delta=\+0\.042,n=24n=24\) andheat\_then\_place\(Δ=\+0\.087\\Delta=\+0\.087,n=23n=23\) show small positive effects; the two types where the rule helps least or slightly hurts—cool\_then\_place\(Δ=−0\.048\\Delta=\-0\.048,n=21n=21\) andpick\_two\_obj\_and\_place\(Δ=−0\.059\\Delta=\-0\.059,n=17n=17\)—are where the generic heuristic occasionally over\-fires\. Of the2222net task wins from the control rule, half \(1111\) fall in the closed\-receptacle / multi\-step types it was written for\. The gain is mechanistic, not diffuse\.

![Refer to caption](https://arxiv.org/html/2609.02889v1/x4.png)Figure 5:Control\-slot gain by ALFWorld task type\(control\-onlyloiatB=32B\{=\}32\)\. Diamonds \(◆\\blacklozenge\) mark closed\-receptacle / multi\-step types the control rules target\. Gains concentrate in loop\-prone, multi\-step types \(look\_at,clean\_then\_place\); near\-saturated or short\-horizon types show little change\. The effect tracks the rule’s content\.
### 7\.5The WebShop null is genuine task\-contingency, not starvation

On WebShop every slot froze empty underHarnessEvo\-full, and the full method tied the baselines \(Table[2](https://arxiv.org/html/2609.02889#S6.T2)\)\. A natural worry is that this null is just another instance of the budget\-splitting trap—perhaps WebShop too has a load\-bearing slot, starved by the four\-way split\. We rule this out directly\. The all\-to\-control run on WebShop, which hands the*entire*budget to the control slot \(well above the accept floor\),*also*returns a null: dense score0\.5350\.535versus0\.5180\.518for stock \(Δ=\+0\.017\\Delta=\+0\.017, paired\-bootstrapp=0\.36p=0\.36, ns\), and the control slot froze empty even with the concentrated budget\. The optimizer had every opportunity to accept a control rule and found none worth accepting\. The WebShop result is therefore a genuine property of the task, not an artifact of allocation: WebShop’s instruction\-conditioned purchasing, with its dense attribute\-match reward, simply does not exhibit a recurrent, verbalizable control failure mode that a general heuristic can repair\. This is the boundary of the phenomenon—harness optimization captures value precisely when the task has recurrent, verbalizable control failures \(dead\-end actions, ordering constraints, premature stopping\), and is null otherwise\.

### 7\.6Grounded self\-correction, not reward\-hacking

The value that survives is, on inspection, environment\-grounded*self\-correction*rather than reward\-exploiting strategy\. The control rule fixes real control errors and generalizes across held\-out task types \(Figure[5](https://arxiv.org/html/2609.02889#S7.F5)\); it does not encode a shortcut that games the binary success checker\. Two observations reinforce this\. First, the gain transfers across task types it was not specifically tuned on—the mark of a genuine correction rather than an over\-fit\. Second, on the benchmark with the more gameable dense reward \(WebShop\), the optimizer declines to fill any slot rather than fabricate a reward\-chasing strategy\. A harness optimizer that captures grounded correction where it exists and abstains where it does not is the desired behaviour in the presence of an imperfect environment reward; we develop this as a constructive counterpoint to over\-optimization concerns in Section[2\.4](https://arxiv.org/html/2609.02889#S2.SS4)\. Validation\-to\-test gaps were small or negative throughout, indicating no over\-fitting to the selection set\.

## 8Discussion

#### Localize before you optimize\.

The headline practical lesson is that harness\-optimization value is not diffuse across the scaffold; on our tasks it is concentrated in a single component\. This changes how a fixed optimization budget should be spent\. Rather than evolve the harness as one string \(which cannot tell you where the value is\) or split a budget uniformly across named components \(which, as we show, can drive every component below the optimizer’s accept\-floor and capture nothing\), the budget should first be used cheaply to*locate*the high\-credit component—a single\-slotloiprobe suffices—and then concentrated there\. Structured decomposition is valuable as a*microscope*for this localization, not as a drop\-in method for a better agent\.

#### The budget\-splitting trap is a property of the optimizer, not the harness\.

The sub\-additivity we observe is mechanistic: it follows from the accept\-and\-rescore floor of the reflective optimizer\. Any optimizer with a minimum budget to accept and validate a candidate will exhibit some version of this trap when its budget is divided across components below that minimum\. The practical implication generalizes beyond our specific optimizer: when distributing a fixed search budget across parts of a system, account for the per\-part minimum the search procedure needs to make progress, or the division will silently waste the budget\.

#### When does harness optimization help?

Our two benchmarks bracket the answer\. Harness optimization helped on ALFWorld and was null on WebShop, and the audited slot text explains the difference: ALFWorld has recurrent, verbalizable control failures that a compact general rule can fix, whereas WebShop does not\. This is a usable predictor\. Before investing in harness optimization for a new task, ask whether the frozen agent’s failures are \(i\) recurrent across instances and \(ii\) describable as a general control heuristic\. If yes, a targeted control\-slot evolution is likely to pay off; if no, the effort will return a null, and—reassuringly—a well\-behaved optimizer will return that null rather than an over\-fit\.

#### The surviving value is the safe kind\.

It is worth emphasizing that the optimization value we localize is grounded self\-correction, which is exactly the kind of value one would want a black\-box harness optimizer to capture: it fixes real control errors, generalizes, and does not exploit the reward\. On the benchmark where the reward is most gameable, the method abstained\. This is a small but encouraging signal that careful, localized harness optimization need not amount to reward\-hacking\.

## 9Conclusion

We asked where, inside an LLM agent’s harness, optimization value lives, and we built a structured microscope to find out:HarnessEvo, a four\-slot decomposition evolved by coordinate ascent under an iso\-budget, paired with a leave\-one\-in / leave\-one\-out credit\-assignment protocol\. The honest binary result is a tie: the full structured method matches both the stock harness and flat single\-string evolution on held\-out success \(0\.6570\.657/0\.6420\.642/0\.6420\.642, ns\)\. The finding is*where*the value lives and*how budget destroys it*\. On ALFWorld the value is localized almost entirely to the reflection/control slot \(leave\-one\-in\+0\.119\+0\.119,p=0\.0046p=0\.0046; role, strategy, format individually null\)\. The structured method ties not because the value is absent but because it is strongly sub\-additive: splitting an iso\-budget of6464rollouts across four slots leaves1616per slot, below the optimizer’s accept\-and\-rescore floor, so every slot froze empty and the load\-bearing gain was never captured—the budget\-splitting trap\. Concentrating the same budget on the high\-credit slot recovers the discarded gain \(targeted control\-only:0\.7610\.761,\+0\.119\+0\.119,p=0\.0046p=0\.0046at half the budget;0\.7240\.724,\+0\.082\+0\.082, borderline at the full budget—reported together, with no overclaim at the larger budget\)\. The phenomenon is task\-contingent: on WebShop every slot froze empty, a targeted control run stayed null, and the absence is a genuine property of the task rather than starvation\. The value that does survive is environment\-grounded self\-correction, not reward\-exploitation\. We conclude that credit assignment and budget concentration should precede any structured agent\-evolution scheme:*localize first, then concentrate the budget on the one slot that matters\.*

#### Limitations\.

We name them plainly\.*\(1\) Single frozen backbone\.*All results use one frozen Qwen2\.5\-7B model; we do not run a larger backbone\. Whether the control\-slot localization persists, shifts to another slot, or vanishes at larger scale is the natural next study—we explicitly flag backbone\-scaling of the localization as future work and make*no*claim about it here\.*\(2\) Two benchmarks, one of them null\.*We study ALFWorld and WebShop; the WebShop null is reported as a task\-contingency boundary, not a weakness, but two environments cannot map the full space of when harness optimization helps\.*\(3\) One structured scheme\.*We evolve slots by coordinate ascent with a single reflective optimizer; other decompositions \(different slot taxonomies, joint rather than sequential evolution, a different base optimizer\) might split value differently, and our sub\-additivity mechanism is tied to this optimizer’s accept\-floor\.*\(4\) Small evolve/validation splits and sample sizes\.*The held\-out sets are modest \(n=134n\{=\}134ALFWorld,n=80n\{=\}80WebShop\) and the validation sets used for candidate selection are small; this is why the budget\-allocation magnitude is significant at@​32@32but only borderline at@​64@64, and why we make directional rather than precise\-magnitude claims there\.

#### Future work\.

Beyond backbone scaling, the most promising directions are a cheap automatic*slot\-credit estimator*that allocates budget proportional to estimated per\-slot value \(turning our manual prescription into a one\-pass allocator\), and a taxonomy of tasks by their control\-failure profile to predict, a priori, whether targeted harness optimization will pay off\.

## References

- \[1\]L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang,et al\.\(2025\)GEPA: reflective prompt evolution can outperform reinforcement learning\.arXiv preprint arXiv:2507\.19457\.Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p2.1),[§4\.2](https://arxiv.org/html/2609.02889#S4.SS2.p1.3),[§5](https://arxiv.org/html/2609.02889#S5.SS0.SSS0.Px1.p1.1)\.
- \[2\]E\. Akyürek, D\. Schuurmans, J\. Andreas, T\. Ma, and D\. Zhou\(2023\)What learning algorithm is in\-context learning? investigations with linear models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2211\.15661Cited by:[§2\.3](https://arxiv.org/html/2609.02889#S2.SS3.p1.1)\.
- \[3\]D\. Amodei, C\. Olah, J\. Steinhardt, P\. Christiano, J\. Schulman, and D\. Mané\(2016\)Concrete problems in AI safety\.arXiv preprint arXiv:1606\.06565\.Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[4\]A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. Hajishirzi\(2024\)Self\-RAG: learning to retrieve, generate, and critique through self\-reflection\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.11511Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[5\]Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon,et al\.\(2022\)Constitutional AI: harmlessness from AI feedback\.arXiv preprint\.Note:arXiv:2212\.08073Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[6\]J\. Chen, X\. Wang, R\. Xu, S\. Yuan, Y\. Zhang, W\. Shi, J\. Xie, S\. Li, R\. Yang, T\. Zhu, A\. Chen, N\. Li, L\. Chen, C\. Hu, S\. Wu, S\. Ren, Z\. Fu, and Y\. Xiao\(2024\)From persona to personalization: a survey on role\-playing language agents\.Transactions on Machine Learning Research \(TMLR\)\.Note:arXiv:2404\.18231Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[7\]C\. Fernando, D\. Banarse, H\. Michalewski, S\. Osindero, and T\. Rocktäschel\(2023\)Promptbreeder: self\-referential self\-improvement via prompt evolution\.arXiv preprint\.Note:arXiv:2309\.16797Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p1.1)\.
- \[8\]L\. Gao, J\. Schulman, and J\. Hilton\(2023\)Scaling laws for reward model overoptimization\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2210\.10760Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[9\]C\. A\. E\. Goodhart\(1984\)Problems of monetary management: the UK experience\.Monetary Theory and Practice: The UK Experience,pp\. 91–121\.Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[10\]Z\. Gou, Z\. Shao, Y\. Gong, Y\. Shen, Y\. Yang, N\. Duan, and W\. Chen\(2023\)CRITIC: large language models can self\-correct with tool\-interactive critiquing\.arXiv preprint\.Note:arXiv:2305\.11738Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[11\]J\. Gu, X\. Jiang, Z\. Shi, H\. Tan, X\. Zhai, C\. Xu, W\. Li, Y\. Shen, S\. Ma, H\. Liu, S\. Wang, K\. Zhang, Y\. Wang, W\. Gao, L\. Ni, and J\. Guo\(2024\)A survey on LLM\-as\-a\-judge\.arXiv preprint arXiv:2411\.15594\.Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[12\]K\. Guu, K\. Lee, Z\. Tung, P\. Pasupat, and M\. Chang\(2020\)REALM: retrieval\-augmented language model pre\-training\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2002\.08909Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[13\]J\. Huang, S\. S\. Gu, L\. Hou, Y\. Wu, X\. Wang, H\. Yu, and J\. Han\(2022\)Large language models can self\-improve\.arXiv preprint\.Note:arXiv:2210\.11610Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[14\]J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. W\. Yu, X\. Song, and D\. Zhou\(2024\)Large language models cannot self\-correct reasoning yet\.International Conference on Learning Representations \(ICLR\)\.Note:arXiv:2310\.01798Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[15\]C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. Narasimhan\(2024\)SWE\-bench: can language models resolve real\-world GitHub issues?\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.06770Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p2.1)\.
- \[16\]R\. Kamoi, Y\. Zhang, N\. Zhang, J\. Han, and R\. Zhang\(2024\)When can LLMs actually correct their own mistakes? a critical survey of self\-correction of LLMs\.Transactions of the Association for Computational Linguistics \(TACL\)12,pp\. 1417–1440\.Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[17\]O\. Khattab, K\. Santhanam, X\. L\. Li, D\. Hall, P\. Liang, C\. Potts, and M\. Zaharia\(2022\)Demonstrate\-search\-predict: composing retrieval and language models for knowledge\-intensive NLP\.arXiv preprint\.Note:arXiv:2212\.14024Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p1.1)\.
- \[18\]O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang,et al\.\(2024\)DSPy: compiling declarative language model calls into self\-improving pipelines\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p1.1)\.
- \[19\]S\. Kim, J\. Shin, Y\. Cho, J\. Jang, S\. Longpre, H\. Lee, S\. Yun, S\. Shin, S\. Kim, J\. Thorne, and M\. Seo\(2024\)Prometheus: inducing fine\-grained evaluation capability in language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.08491Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[20\]H\. Lee, S\. Phatale, H\. Mansoor, T\. Mesnard, J\. Ferret, K\. Lu, C\. Bishop, E\. Hall, V\. Carbune, A\. Rastogi, and S\. Prakash\(2024\)RLAIF vs\. RLHF: scaling reinforcement learning from human feedback with AI feedback\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Note:arXiv:2309\.00267Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[21\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela\(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2005\.11401Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[22\]D\. Li, B\. Jiang, L\. Huang, A\. Beigi, C\. Zhao, Z\. Tan, A\. Bhattacharjee, Y\. Jiang, C\. Chen, T\. Wu, K\. Shu, L\. Cheng, and H\. Liu\(2024\)From generation to judgment: opportunities and challenges of LLM\-as\-a\-judge\.arXiv preprint arXiv:2411\.16594\.Cited by:[§2\.3](https://arxiv.org/html/2609.02889#S2.SS3.p1.1)\.
- \[23\]X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang, S\. Zhang, X\. Deng, A\. Zeng, Z\. Du, C\. Zhang, S\. Shen, T\. Zhang, Y\. Su, H\. Sun, M\. Huang, Y\. Dong, and J\. Tang\(2024\)AgentBench: evaluating LLMs as agents\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2308\.03688Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p2.1)\.
- \[24\]Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu\(2023\)G\-Eval: NLG evaluation using GPT\-4 with better human alignment\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 2511–2522\.Note:arXiv:2303\.16634External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.153)Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[25\]A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[26\]G\. Mialon, C\. Fourrier, C\. Swift, T\. Wolf, Y\. LeCun, and T\. Scialom\(2024\)GAIA: a benchmark for general AI assistants\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2311\.12983Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p2.1)\.
- \[27\]A\. S\. Morcos, D\. G\.T\. Barrett, N\. C\. Rabinowitz, and M\. Botvinick\(2018\)On the importance of single directions for generalization\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:1803\.06959Cited by:[§2\.3](https://arxiv.org/html/2609.02889#S2.SS3.p1.1)\.
- \[28\]N\. Muennighoff, Z\. Yang, W\. Shi, X\. L\. Li, L\. Fei\-Fei, H\. Hajishirzi, L\. Zettlemoyer, P\. Liang, E\. Candès, and T\. Hashimoto\(2025\)S1: simple test\-time scaling\.arXiv preprint\.Note:arXiv:2501\.19393Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[29\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. Christiano, J\. Leike, and R\. Lowe\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2203\.02155Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[30\]C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez\(2023\)MemGPT: towards LLMs as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[31\]A\. Pan, K\. Bhatia, and J\. Steinhardt\(2022\)The effects of reward misspecification: mapping and mitigating misaligned models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2201\.03544Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[32\]A\. Panickssery, S\. R\. Bowman, and S\. Feng\(2024\)LLM evaluators recognize and favor their own generations\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.37\.Note:arXiv:2404\.13076Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[33\]J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein\(2023\)Generative agents: interactive simulacra of human behavior\.InACM Symposium on User Interface Software and Technology \(UIST\),Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[34\]Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian, S\. Zhao, L\. Hong, R\. Tian, R\. Xie, J\. Zhou, M\. Gerstein, D\. Li, Z\. Liu, and M\. Sun\(2024\)ToolLLM: facilitating large language models to master 16000\+ real\-world APIs\.InInternational Conference on Learning Representations \(ICLR\),Note:Spotlight; arXiv:2307\.16789Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[35\]R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. Finn\(2023\)Direct preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2305\.18290Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[36\]S\. Saha, O\. Levy, A\. Celikyilmaz, M\. Bansal, J\. Weston, and X\. Li\(2023\)Branch\-solve\-merge improves large language model evaluation and generation\.arXiv preprint arXiv:2310\.15123\.Cited by:[§2\.3](https://arxiv.org/html/2609.02889#S2.SS3.p1.1)\.
- \[37\]N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. Yao\(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[38\]M\. Shridhar, X\. Yuan, M\. Côté, Y\. Bisk, A\. Trischler, and M\. Hausknecht\(2021\)ALFWorld: aligning text and embodied environments for interactive learning\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2010\.03768Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p4.13),[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p2.1),[§5](https://arxiv.org/html/2609.02889#S5.SS0.SSS0.Px2.p1.5)\.
- \[39\]J\. Skalse, N\. H\. R\. Howe, D\. Krasheninnikov, and D\. Krueger\(2022\)Defining and characterizing reward hacking\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2209\.13085Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[40\]C\. Snell, J\. Lee, K\. Xu, and A\. Kumar\(2024\)Scaling LLM test\-time compute optimally can be more effective than scaling model parameters\.arXiv preprint\.Note:arXiv:2408\.03314Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p1.1)\.
- \[41\]N\. Stiennon, L\. Ouyang, J\. Wu, D\. M\. Ziegler, R\. Lowe, C\. Voss, A\. Radford, D\. Amodei, and P\. Christiano\(2020\)Learning to summarize from human feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2009\.01325Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[42\]H\. Su, J\. Luo, C\. Liu, X\. Yang, Y\. Zhang, Y\. Dong, and J\. Zhu\(2025\)A survey on autonomy\-induced security risks in large model\-based agents\.arXiv preprint arXiv:2506\.23844\.Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[43\]M\. Suzgun, M\. Yuksekgonul, F\. Bianchi, D\. Jurafsky, and J\. Zou\(2025\)Dynamic cheatsheet: test\-time learning with adaptive memory\.arXiv preprint arXiv:2504\.07952\.Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p1.1)\.
- \[44\]J\. von Oswald, E\. Niklasson, E\. Randazzo, J\. Sacramento, A\. Mordvintsev, A\. Zhmoginov, and M\. Vladymyrov\(2023\)Transformers learn in\-context by gradient descent\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2212\.07677Cited by:[§2\.3](https://arxiv.org/html/2609.02889#S2.SS3.p1.1)\.
- \[45\]G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. Anandkumar\(2023\)Voyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[46\]X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou\(2023\)Self\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2203\.11171Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[47\]Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. Neubig\(2024\)Agent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[48\]T\. Wu, W\. Yuan, O\. Golovneva, J\. Xu, Y\. Tian, J\. Jiao, J\. Weston, and S\. Sukhbaatar\(2024\)Meta\-rewarding language models: self\-improving alignment with LLM\-as\-a\-meta\-judge\.arXiv preprint\.Note:arXiv:2407\.19594Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[49\]A\. Yang, A\. Li, B\. Yang, B\. Zhang,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§5](https://arxiv.org/html/2609.02889#S5.SS0.SSS0.Px1.p1.1)\.
- \[50\]C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. Chen\(2024\)Large language models as optimizers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p1.1)\.
- \[51\]S\. Yang, S\. C\. Han, Y\. Ding, S\. Wang, and E\. Hovy\(2026\)ToolTree: efficient LLM agent tool planning via dual\-feedback monte carlo tree search and bidirectional pruning\.Note:ICLR 2026External Links:2603\.12740Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[52\]S\. Yang, S\. C\. Han, X\. Ma, Y\. Li, M\. R\. Ghasemi Madani, and E\. Hovy\(2026\)EvoTool: self\-evolving tool\-use policy optimization in LLM agents via blame\-aware mutation and diversity\-aware selection\.Note:ACL 2026External Links:2603\.04900Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[53\]S\. Yang, S\. C\. Han, S\. Wang, Y\. Li, Y\. Ding, and E\. Hovy\(2026\)Toward understanding misalignment in LLM agents: a survey of taxonomy, causes, mitigation, and evaluation\.Note:ACL ARR 2026External Links:[Link](https://openreview.net/forum?id=zzTEGP2BYa)Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[54\]X\. Yang, J\. Zou, R\. Pan, R\. Qiu, P\. Lu, S\. Diao, J\. Jiang, H\. Tong, T\. Zhang, M\. J\. Buehler, J\. He, and J\. Zou\(2026\)Recursive multi\-agent systems\.arXiv preprint arXiv:2604\.25917\.Cited by:[§2\.5](https://arxiv.org/html/2609.02889#S2.SS5.p1.1)\.
- \[55\]S\. Yao, H\. Chen, J\. Yang, and K\. Narasimhan\(2022\)WebShop: towards scalable real\-world web interaction with grounded language agents\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2207\.01206Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p7.3),[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p2.1),[§5](https://arxiv.org/html/2609.02889#S5.SS0.SSS0.Px2.p1.5)\.
- \[56\]S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhan\(2024\)τ\\tau\-bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.arXiv preprint arXiv:2406\.12045\.Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p2.1)\.
- \[57\]S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. Narasimhan\(2023\)Tree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2305\.10601Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1)\.
- \[58\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p1.1),[§3](https://arxiv.org/html/2609.02889#S3.SS0.SSS0.Px1.p1.4)\.
- \[59\]W\. Yuan, R\. Y\. Pang, K\. Cho, S\. Sukhbaatar, J\. Xu, and J\. Weston\(2024\)Self\-rewarding language models\.arXiv preprint\.Note:arXiv:2401\.10020Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[60\]E\. Zelikman, Y\. Wu, J\. Mu, and N\. D\. Goodman\(2022\)STaR: bootstrapping reasoning with reasoning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[61\]Q\. Zhang, C\. Hu,et al\.\(2025\)Agentic context engineering: evolving contexts for self\-improving language models\.arXiv preprint arXiv:2510\.04618\.Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[62\]A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. Huang\(2024\)ExpeL: LLM agents are experiential learners\.InAAAI Conference on Artificial Intelligence,Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[63\]L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica\(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.InAdvances in Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track,Vol\.36\.Note:arXiv:2306\.05685Cited by:[§2\.4](https://arxiv.org/html/2609.02889#S2.SS4.p1.1)\.
- \[64\]W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang\(2024\)MemoryBank: enhancing large language models with long\-term memory\.InProceedings of the AAAI Conference on Artificial Intelligence \(AAAI\),Note:arXiv:2305\.10250Cited by:[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p3.1)\.
- \[65\]S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. Neubig\(2024\)WebArena: a realistic web environment for building autonomous agents\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2307\.13854Cited by:[§2\.2](https://arxiv.org/html/2609.02889#S2.SS2.p2.1)\.
- \[66\]Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. Ba\(2023\)Large language models are human\-level prompt engineers\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2211\.01910Cited by:[§1](https://arxiv.org/html/2609.02889#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.02889#S2.SS1.p1.1)\.
- \[67\]J\. Zou, R\. Qiu, G\. Li, X\. Yang, K\. Tieu, P\. Lu, K\. Shen, H\. Tong, Y\. Choi, J\. He, J\. Zou, M\. Wang, and L\. Yang\(2025\)Latent collaboration in multi\-agent systems\.arXiv preprint arXiv:2511\.20639\.Note:ICML 2026 SpotlightCited by:[§2\.5](https://arxiv.org/html/2609.02889#S2.SS5.p1.1)\.

## Appendix AFrozen Slot Texts, Additional Arms, and Hyperparameters

### A\.1Audited frozen slot texts

The localization claim rests on auditable text\. We reproduce the control\-slot content that produced the significant ALFWorld gain \(theB=32B\{=\}32control\-only run,401401characters, frozen after validation selection\):

think: Before performing any action, ensure the previous step succeeded\. if action returned ‘‘Nothing happens’’ and is identical to the last action, skip it and try another location\. open any closed receptacle before taking from it\. confirm the object is held before attempting to place it\. stop the sequence once the task is completed successfully or all relevant locations are checked without success\.

The full\-budget all\-to\-control run \(B=64B\{=\}64\) froze a longer, pseudo\-code variant of the same rules \(10001000characters; anif/thenelaboration of “skip repeated dead\-end actions, open before taking, verify holding before placing, stop when satisfied”\)\. OnWebShop, by contrast, the control slot \(and every other slot, under both the four\-way split and the all\-to\-control run\) froze*empty*\(0characters\): the optimizer accepted no candidate\. This empty\-versus\-substantive contrast across benchmarks is the direct, auditable signature of the task\-contingency boundary in Section[7](https://arxiv.org/html/2609.02889#S7)\. The role, strategy, and format slots on ALFWorld also froze empty in the iso\-budget full run \(0characters each\), consistent with their nullloieffects\.

### A\.2B120 corroboration arm \(full table\)

Table[6](https://arxiv.org/html/2609.02889#A1.T6)gives the higher\-budget arm in full\. Stock measures higher at this configuration \(0\.6720\.672\), flat evolution gives0\.6420\.642, andHarnessEvo\-full gives0\.6270\.627; all pairwise McNemar tests are ns\. The control slot remains the most positive single\-slot direction \(Δc4LOI=\+0\.052\\Delta^\{\\mathrm\{LOI\}\}\_\{c\_\{4\}\}=\+0\.052, ns\), and the system is sub\-additive \(full−0\.045\-0\.045vs∑loi=\+0\.052\\sum\\\!\\textsc\{loi\}=\+0\.052\)\. We include this arm as a robustness check on the*ordering*of per\-slot value, not as an independent significant result\.

Table 6:B120 corroboration arm on ALFWorld \(n=134n\{=\}134,1010k\-bootstrap CI\)\. Main methods \(top\) and per\-componentloimap \(bottom\)\. All differences ns; the control direction remains the largest single\-slot gain and the system is sub\-additive\.Method / SlotSuccess /ΔLOI\\Delta^\{\\mathrm\{LOI\}\}\[95% CI\]McNemarppRolloutsStock0\.672 \[0\.590, 0\.746\]—0GEPA\-flat0\.642 \[0\.560, 0\.716\]0\.3430\.343ns120120HarnessEvo\-full0\.627 \[0\.545, 0\.709\]0\.3770\.377ns120120c1c\_\{1\}role\+0\.007\+0\.007\[−0\.02,\+0\.04\-0\.02,\+0\.04\]1\.000 ns6060c2c\_\{2\}strategy\+0\.015\+0\.015\[−0\.07,\+0\.10\-0\.07,\+0\.10\]0\.860 ns6060c3c\_\{3\}format−0\.022\-0\.022\[−0\.07,\+0\.02\-0\.07,\+0\.02\]0\.546 ns6060c4c\_\{4\}control\+0\.052\+0\.052\[−0\.02,\+0\.13\-0\.02,\+0\.13\]0\.265 ns6060
### A\.3Hyperparameters and protocol

- •Backbone:frozen Qwen2\.5\-7B, used as both task policy and reflection model; rollouts deterministic \(temperature=0\\text\{temperature\}\{=\}0\); reflection mutations sampled \(temperature0\.90\.9\), as in GEPA\.
- •ALFWorld:train\|T\|=30\|T\|\{=\}30\(balanced over task types from a 30\-task dev manifest\), validation≈10\\approx 10\(B64\) /88\(B120\), testn=134n\{=\}134\(disjoint held\-out manifest, zero leakage\); max4040steps/episode; binary environment\-verified success\.
- •WebShop:disjoint train/validation/test sessions, testn=80n\{=\}80; dense\[0,1\]\[0,1\]attribute\-match score\.
- •Optimizer \(GEPA\):instance\-Pareto parent selection over a growing candidate pool; reflective mutation on a train minibatch \(size33at B64,22at B120\); improvement\-or\-equal accept gate for the binary\-sparse reward; accepted children re\-scored on the full validation set; final==best\-validation candidate, frozen\. Iso\-budget: per\-slot budgets sum exactly to the flat method’s budget\.
- •Budgets:B=64B\{=\}64\(primary\),B=120B\{=\}120\(corroboration\); single\-slotloiatB/2B/2; all\-to\-control at3232and6464\.
- •Statistics:10,00010\{,\}000\-resample bootstrap95%95\\%CIs; paired McNemar \(ALFWorld binary\) / paired bootstrap \(WebShop dense\); ns labeled;∗⁣/⁣∗⁣∗⁣/⁣∗⁣∗⁣∗\*/\*\*/\*\*\*forp<0\.05/0\.01/0\.001p<0\.05/0\.01/0\.001\.
- •Accept\-floor diagnostic:an accept\-bearing GEPA iteration costs2×minibatch\+\|V\|2\\times\\text\{minibatch\}\+\|V\|rollouts; at B64 \(minibatch=3\\text\{minibatch\}\{=\}3,\|V\|≈10\|V\|\{\\approx\}10\) this is≈16\\approx 16, equal to the four\-way split’s1616/slot, which is why the split froze every slot empty \(Table[5](https://arxiv.org/html/2609.02889#S7.T5)\)\.