Self-Evolving Agents with Anytime-Valid Certificates

# Self-Evolving Agents with Anytime-Valid Certificates
Source: [https://arxiv.org/html/2607.00871](https://arxiv.org/html/2607.00871)
###### Abstract

Self\-evolving agents violate the assumption behind most learning\-theoretic guarantees: the data, evaluator, components, and hypothesis space are produced by the policy being updated\. We presentSEA, an architecture that confines self\-modification to a small steering adapter and a versioned harness around a*frozen*base model and admits each modification only through an anytime\-valid gate that emits an auditable certificate against a fixed error budget\. Five loop controllers compose published guarantees; because such gates can only*select*among behaviors the frozen base already produces, five verifier\-in\-the\-loop mechanisms—best\-of\-NN, micro\-step search, self\-authored reproduction oracles, search\-layer control, and self\-repair—supply the dense, grader\-free signal the gates require, computed from the issue text alone\. On a5252\-instance SWE\-bench Verified subset across four base models, base capability is the dominant, confound\-free effect, and on two strong base models a deliberate no\-op\-composite control isolates the suite’s contribution at\+4\+4and\+5\+5\(Glm5\.224→2824\\to 28;Gpt29→3429\\to 34, the65%65\\%best\), with event logs confirming that its mechanisms fire and prevent regressions\. Results are single\-run on expensive evaluations; confirming run\-to\-run variance and adapting the per\-task algorithm mix are future work\.

Disclaimer:This paper was prepared for informational purposes by the LLM Suite group of JP Morgan Chase and its affiliates \(‘JPMC’\) and is not a product of the Research Department of JP Morgan\. JP Morgan makes no representation, warranty or undertaking whatsoever and disclaims all liability for the completeness, accuracy or reliability of the information contained herein\. This document is not intended as investment research or investment advice, or a recommendation, offer or solicitation for the purchase or sale of any security, financial instrument, financial product or service, or to be used in any way for evaluating the merits of participating in any transaction, and shall not constitute a solicitation under any jurisdiction or to any person, if such solicitation under such jurisdiction or to such person would be unlawful\.

## 1Introduction

A self\-evolving agent improves its own future behavior using data, evaluations, components, and a hypothesis space that it itself produces—rewriting prompts and tools, distilling its outputs, learning its reward models, growing skill libraries\. The guarantees one would invoke for such systems, however, were proven for*exogenous*environments: continual\-learning forgetting bounds\(Farajtabaret al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib4); Chugget al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib2)\), convergence of preference optimization\(Tiapkinet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib10); Wanget al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib9)\), unbiasedness of policy\-gradient estimators\(Meulemanset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib14)\), safe policy improvement\(Thomaset al\.,[2015](https://arxiv.org/html/2607.00871#bib.bib19)\), and library\-learning optimality\(Bowerset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib25)\)each assume a task stream, evaluator, MDP, or program library fixed independently of the learner\. We call the violation the*endogenous\-loop failure mode*: the evolving policy generates the data it trains on, the evaluator it is judged by, the components it is built from, and the hypothesis space it searches\. The name is a shorthand, not a precise mathematical category, and taken literally it overstates the problem: violating a theorem’s hypotheses voids its certificate but does not negate its conclusion\. The guarantee simply ceases to be*certified*—the bound may still hold, may degrade gracefully, or may break, depending on the problem—and performative\-prediction theory shows the loop can in fact still contract when the policy\-induced distribution shift is small enough\(Perdomoet al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib1)\)\. We use the term to mark where classical guarantees stop applying, not to claim that learning provably fails\.

This paper develops an architecture and a set of concrete algorithms for this setting, together with an executable reference implementation from which all pseudo\-code in this paper is distilled\. Two principles organize the design\. First, every self\-modification passes through an*anytime\-valid gate*: each classical seed result is wrapped in exactly the machinery required for its guarantee to survive the closed loop—performative stability\(Perdomoet al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib1)\), anytime\-valid inference\(Ramdaset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib12); Howardet al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib22)\), dynamic\-regret online learning\(Cutkosky,[2020](https://arxiv.org/html/2607.00871#bib.bib16); Baby and Wang,[2022](https://arxiv.org/html/2607.00871#bib.bib17)\), and two\-timescale stochastic approximation\(Borkar,[2008](https://arxiv.org/html/2607.00871#bib.bib11)\)\. Second, gates can only*select*among behaviors the frozen base model already produces; when a base model’s failure is systematic rather than stochastic, and the reward arrives only at the end of an episode, there is nothing for a gate to select\. We therefore make the task verifier an active in\-loop control signal \(§[5](https://arxiv.org/html/2607.00871#S5)\)—inside the episode, across attempts, over the action space, and inside credit assignment—so that the controllers have both the signal and the variation they need to act on\.

#### Contributions\.

1. 1\.A four\-layer reference architecture\(§[3](https://arxiv.org/html/2607.00871#S3)\) that decomposes a self\-evolving LLM agent into a frozen base modelL0L\_\{0\}, a small steering adapterL1L\_\{1\}\(steered online; not weight\-fine\-tuned in any reported run\), a mutable, versioned harnessL2L\_\{2\}, and a loop controllerL3L\_\{3\}\(Figure[1](https://arxiv.org/html/2607.00871#S1.F1)\)\. BecauseL0L\_\{0\}is frozen andL1L\_\{1\}is low\-dimensional, policy deltas∥πt−πt−1∥\\lVert\\pi\_\{t\}\-\\pi\_\{t\-1\}\\rVertare measurable and can be trust\-regioned, which is what renders the performative\-sensitivity machinery applicable at all\.
2. 2\.Five loop controllers\(§[4](https://arxiv.org/html/2607.00871#S4)\), one per failure mode of the endogenous loop: stability–plasticity, self\-referential collapse, credit assignment, verifiable self\-modification, and hypothesis\-space expansion\. Each is given as a precise problem statement, an algorithmic solution with pseudo\-code \(Algorithms[1](https://arxiv.org/html/2607.00871#alg1)–[5](https://arxiv.org/html/2607.00871#alg5)\), and an explicit account of which published results it builds on and which guarantees remain open conjectures in the endogenous setting\.
3. 3\.Verifier\-in\-the\-loop mechanisms and a two\-loop design\(§[5](https://arxiv.org/html/2607.00871#S5)\): a graded verifier \(Eq\.[9](https://arxiv.org/html/2607.00871#S5.E9)\), closed\-loop test execution, an explore→\\toedit budget, process\-level reward, and a verifier\-gated refinement hill\-climb \(Alg\.[6](https://arxiv.org/html/2607.00871#alg6)\), building to a verified micro\-step search \(Alg\.[7](https://arxiv.org/html/2607.00871#alg7)\) and a search\-then\-distill two\-loop design\(Zelikmanet al\.,[2022](https://arxiv.org/html/2607.00871#bib.bib41); Gulcehreet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib42)\)into which the five controllers are re\-aimed \(Table[2](https://arxiv.org/html/2607.00871#S5.T2); four on the search layer, Alg\.[9](https://arxiv.org/html/2607.00871#alg9), validated by offline gate simulations\)\.
4. 4\.A self\-authored in\-loop verifier\(§[5\.2](https://arxiv.org/html/2607.00871#S5.SS2), Algorithm[8](https://arxiv.org/html/2607.00871#alg8)\): the search is steered by a reproduction\-oracle suite the model writes from the issue alone, admitted by a single rule \(an oracle must*fail on the unpatched base*\), while the held\-out grader is reserved for terminal measurement\. Where an oracle suite is admitted the search runs grader\-free, and the held\-out tests never steer it\.
5. 5\.Verified self\-repair of the harness\(§[5\.4](https://arxiv.org/html/2607.00871#S5.SS4), Algorithm[10](https://arxiv.org/html/2607.00871#alg10)\): a repertoire of harness\-repair primitives that the loop*selects by measured fix\-rate against the real environment*, not by human judgment—the same propose\-and\-gate discipline asAlg 4, applied to the agent’s own failure modes\.
6. 6\.A reusable anytime\-valid statistical core\(§[6](https://arxiv.org/html/2607.00871#S6)\): normal\-mixture confidence sequences, Hoeffding e\-processes with predictable plug\-in betting, a*horizon\-free, normalized*confirm\-triggered harmonic spending schedule, time\-uniform PAC\-Bayes penalties, parameter\-free coin\-betting oracles with drift\-triggered restarts, exact 1\-D Wasserstein computation, wild\-bootstrap trend tests, MAP\-Elites archives, and Stitch\-style MDL compression by antiunification with sound dominance pruning\.

L0L\_\{0\}— frozen base modelL1L\_\{1\}— steering adapterθ\\theta\(online\-steered; not fine\-tuned\)L2L\_\{2\}— harness: prompt⋅\\cdottools⋅\\cdotbudgets⋅\\cdotlibrarydeployed policyπt=L0∘L1\(t\)∘L2\(t\)\\pi\_\{t\}=L\_\{0\}\\circ L\_\{1\}^\{\(t\)\}\\circ L\_\{2\}^\{\(t\)\}Alg 4SGM\-CS:gated harness edits \(CS\+\+ε​W1\\varepsilon W\_\{1\}\+\+CTHS\)Alg 5SDC\-QD:grows the abstraction library \(MDL\+\+QD\)Alg 3PA\-COCOA:counterfactual credit→θ\\to\\thetaAlg 1PPB\-CL:forgetting gate\+\+trust region onθ\\thetaAlg 2PNMP\-A:anchors the learned reward modelqqL3L\_\{3\}— loop controllers \(composite two\-timescale scheduler\)edit \(slow\)grow \(slow\)train \(fast\)protect \(fast\)environment\+\+verifiertasks⋅\\cdottest execution⋅\\cdotrewardsR,R~∈\[0,1\]R,\\tilde\{R\}\\in\[0,1\]rewardmodelqqcertificate ledgeraccept/hold/nsf⋅\\cdotδ\\delta\-spenddeploy / actobs\.\+\+test feedbackrollouts⋅\\cdotrewardse\-value drift gatescores rolloutsFigure 1:How the four layers and the five controllers interact\. The deployed policyπt=L0∘L1\(t\)∘L2\(t\)\\pi\_\{t\}=L\_\{0\}\\circ L\_\{1\}^\{\(t\)\}\\circ L\_\{2\}^\{\(t\)\}composes a frozen base modelL0L\_\{0\}, a small steering adapterL1L\_\{1\}, and a mutable harnessL2L\_\{2\}; the fourth layer—theL3L\_\{3\}loop controllers—sits*outside*this forward pass\. Deployments induce the performative distribution𝒟​\(πt\)\\mathcal\{D\}\(\\pi\_\{t\}\)and the verifier returns bounded terminal and process rewards; theL3L\_\{3\}controllers consume these and act on their own layers through anytime\-valid gates—Alg 3/Alg 1update and protectL1L\_\{1\}each round \(fast timescale\),Alg 4/Alg 5edit and growL2L\_\{2\}everyKKrounds \(slow timescale\), andAlg 2guards any learned reward model with an e\-value drift gate\. Every decision is logged in the certificate ledger against the error budgetδ0\\delta\_\{0\}\. In all reported runsL1L\_\{1\}is*steered*—its distribution over discrete directives updated online byAlg 1/Alg 3—never weight\-fine\-tuned; weight\-level adapter training by the slow\-loop distillation of §[5\.3](https://arxiv.org/html/2607.00871#S5.SS3)is designed but not run here\.We are deliberate about epistemic status\. Every mathematical statement in this paper is drawn from a published result, cited at the point of use; we do not derive new bounds or rates here\. Where a classical guarantee is invoked, we state precisely what its source proves and treat its survival under the endogenous loop as an*open conjecture*, without positing a proof\. Likewise, whether any given mechanism lifts a given base model is not argued from anecdote: the protocol of §[8](https://arxiv.org/html/2607.00871#S8)is designed to answer it, and we defer that discussion to the results\.

## 2Related Work

SEA introduces no new learning theory; it reuses guarantees proved for*exogenous*settings and asks what each needs in order to survive a loop the learner closes on itself\. The prior work accordingly plays two roles\. A shared lens \(performativity\) and a shared gate \(anytime\-valid inference\) cut across all five controllers; four classical literatures then each supply the seed result for one controller, and a fifth thread—verifier\-guided search—is the engine that feeds the gates rather than a controller in its own right\.

#### Performativity is the lens\.

The common difficulty is that an agent’s own updates move the distribution it is then judged on\. Performative prediction makes this precise:Perdomoet al\.\([2020](https://arxiv.org/html/2607.00871#bib.bib1)\)define decision\-dependent distributions𝒟​\(π\)\\mathcal\{D\}\(\\pi\)through the sensitivity conditionW1​\(𝒟​\(π\),𝒟​\(π′\)\)≤ε​∥π−π′∥W\_\{1\}\(\\mathcal\{D\}\(\\pi\),\\mathcal\{D\}\(\\pi^\{\\prime\}\)\)\\leq\\varepsilon\\lVert\\pi\-\\pi^\{\\prime\}\\rVert\(their Def\. 3\.1\) and show that forβ\\beta\-smooth,γ\\gamma\-strongly\-convex losses repeated retraining contracts to a performatively stable point at a linear rate whenε<γ/β\\varepsilon<\\gamma/\\beta\(their Thm\. 3\.5\), a threshold shown tight in their Prop\. 3\.6\.Mandalet al\.\([2023](https://arxiv.org/html/2607.00871#bib.bib15)\)carry this into reinforcement learning, where reward and transition kernels are\(εr,εp\)\(\\varepsilon\_\{r\},\\varepsilon\_\{p\}\)\-sensitive in the occupancy measure and convergence is*restored by*sufficiently strong regularization \(their Thm\. 1\)—so under performativity, trust regions and regularization are necessary rather than optional\. We adopt this lens throughout: the deployed configuration is our performative variable, andε\\varepsilonenters both as a trust\-region coefficient \(Alg 1\) and as a confidence\-radius inflation \(Alg 4\)\.

#### Anytime\-valid inference is the gate\.

Because a self\-evolving agent inspects its own statistics every round, any test it relies on must stay valid under continuous peeking—fixed\-nninference is invalid by construction\. E\-values, e\-processes, and time\-uniform confidence sequences supply exactly this license for optional stopping\(Ramdaset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib12); Howardet al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib22); Ville,[1939](https://arxiv.org/html/2607.00871#bib.bib44)\)\. The Statistical Gödel Machine\(Wuet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib21)\), a statistical descendant of Schmidhuber’s proof\-based Gödel machine\(Schmidhuber,[2003](https://arxiv.org/html/2607.00871#bib.bib36)\), applies the idea to self\-modification: it gates each self\-edit with an e\-value and, because an accepted edit is an irreversible commit, controls familywise \(not false\-discovery\) error under a harmonic spending schedule\.Alg 4builds directly on this gate, adding a performative correction and a non\-stationarity test\(Chandaket al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib23)\)and inheriting the abstention semantics of high\-confidence, Seldonian policy improvement\(Thomaset al\.,[2015](https://arxiv.org/html/2607.00871#bib.bib19),[2019](https://arxiv.org/html/2607.00871#bib.bib20)\)—the source of our “no solution found” output\.

#### Four endogenous failure modes, four seeds\.

Each remaining controller takes a guarantee proved in an exogenous world and asks it to hold once the agent supplies its own data\.*Continual learning\.*Orthogonal gradient descent leaves earlier\-task predictions unchanged\(Farajtabaret al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib4)\), but its no\-forgetting guarantee is an NTK\-regime result that needs unbounded Jacobian memory\(Abbana Bennaniet al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib5), Thm\. 2\), while the time\-uniform PAC\-Bayes forgetting certificates ofFriedman and Meir \([2025](https://arxiv.org/html/2607.00871#bib.bib3), Thm\. 3\.1\)andChugget al\.\([2023](https://arxiv.org/html/2607.00871#bib.bib2), Thm\. 3\.1\)require a data\-independent prior; both assumptions break when the agent reuses its own evolving policy on a stream it generates, whichAlg 1confronts with a finite direction buffer inside a performative trust region\.*Self\-consuming loops and preference games\.*Pure self\-training collapses—tails vanish and variance shrinks\(Shumailovet al\.,[2024](https://arxiv.org/html/2607.00871#bib.bib8)\)—unless real data is retained: accumulating rather than replacing data bounds the error independently of iteration count\(Gerstgrasseret al\.,[2024](https://arxiv.org/html/2607.00871#bib.bib7)\), and a constant real\-data fractionα\\alphacaps the cumulative shift at2​M​\(1−\(1−α\)t\)​α−1​dTV​\(n\)2M\(1\-\(1\-\\alpha\)^\{t\}\)\\alpha^\{\-1\}d\_\{\\mathrm\{TV\}\}\(n\)\(Fuet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib6), Thm\. 1\)\. In parallel, preference optimization converges only in the last iterate and only for a*fixed*game—magnetic mirror descent reaches the regularized Nash equilibrium and recovers the true one by refreshing its magnet\(Wanget al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib9)\), and proximal\-point self\-play contracts geometrically up to a gradient\-residual floor\(Tiapkinet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib10)\)—while a learned reward model over\-optimizes, its proxy–gold gap growing withKL\\sqrt\{\\mathrm\{KL\}\}\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib29)\)\.Alg 2couples a real\-data anchor against collapse with last\-iterate self\-play, separated onto two timescales\(Borkar,[2008](https://arxiv.org/html/2607.00871#bib.bib11)\)\.*Credit assignment\.*Hindsight credit assignment reweights each reward by how much the action could have influenced it, dropping rewards it could not\(Harutyunyanet al\.,[2019](https://arxiv.org/html/2607.00871#bib.bib13)\); COCOA generalizes the conditioning from future states to rewarding outcomes and is unbiased under a fully\-predictive encoding with ground\-truth contribution coefficients \(their Def\. 2 and Thm\. 1\)—but only in a*fixed*MDP\(Meulemanset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib14)\)\.Alg 3folds the deploying policy into the outcome encoding and drives the update with parameter\-free, strongly\-adaptive online learning\(Cutkosky,[2020](https://arxiv.org/html/2607.00871#bib.bib16); Orabona and Pál,[2016](https://arxiv.org/html/2607.00871#bib.bib18); Baby and Wang,[2022](https://arxiv.org/html/2607.00871#bib.bib17)\), whose dynamic\-regret rates are stated against a drifting comparator\.*Library learning\.*DreamCoder grows a program library by wake\-sleep MDL learning\(Elliset al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib24)\)and Stitch makes the abstraction step exact via corpus\-guided top\-down synthesis with sound dominance pruning\(Bowerset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib25)\), but neither has a convergence or no\-collapse guarantee when the corpus is generated under the very library being learned;Alg 5therefore keeps a MAP\-Elites archive over behaviors\(Mouret and Clune,[2015](https://arxiv.org/html/2607.00871#bib.bib30); Cully and Demiris,[2018](https://arxiv.org/html/2607.00871#bib.bib28)\)rather than a single library, and certifies held\-out description length in the spirit of PAC\-Bayes lifelong learning\(Pentina and Lampert,[2014](https://arxiv.org/html/2607.00871#bib.bib26)\)\.

#### Verifier\-guided search is the engine\.

Because the controllers can only*select*among behaviors the frozen base already produces, a separate line of work supplies the variation and the dense signal they act on\. Coverage under repeated sampling grows along an approximate exponentiated power law, yet only an automatic verifier turns that coverage into solved instances—majority vote and reward\-model selection plateau as samples grow\(Brownet al\.,[2024](https://arxiv.org/html/2607.00871#bib.bib43)\)\. Closing the loop with the model’s*own*judgment, as Self\-Refine\(Madaanet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib37)\)and Reflexion\(Shinnet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib38)\)do, stalls exactly where self\-assessment fails\. Our search \(§[5](https://arxiv.org/html/2607.00871#S5)\) departs on two axes\. First, the feedback is*executed*verification rather than opinion: best\-of\-NNkeeps the candidate an executable verifier scores highest, and refinement is a strict verifier\-scored hill\-climb with backtracking\. Second, the operators are wired into the certificate\-gated controllers—best\-of\-NNisAlg 4’s accept\-gate pointed at patches, and process rewards feedAlg 3’s credit assignment\. Where model\-generated tests have been used for selection and self\-debugging\(Chenet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib39),[2024](https://arxiv.org/html/2607.00871#bib.bib40)\), our verifier \(§[5\.2](https://arxiv.org/html/2607.00871#S5.SS2)\) keeps a strict firewall between the self\-authored oracle that*steers*the search and the held\-out grader that only*measures*it, admitting an oracle only if it fails on the unpatched base\.

## 3Architecture: Four Layers Around a Frozen Model

The deployed agent at roundttis the composition

πt=L0∘L1\(t\)∘L2\(t\),\\pi\_\{t\}\\;=\\;L\_\{0\}\\circ L\_\{1\}^\{\(t\)\}\\circ L\_\{2\}^\{\(t\)\},\(1\)where the four layers, and the channels through which the five controllers act on them, are summarized in Figure[1](https://arxiv.org/html/2607.00871#S1.F1):

L0L\_\{0\}— Base model\.A frozen pre\-trained LLM, accessed as a functionLLM​\(prompt\)→text\\mathrm\{LLM\}\(\\text\{prompt\}\)\\to\\text\{text\}, optionally returning token log\-probabilities\. Never updated\.

L1L\_\{1\}— Adapter\.The trainable policy parameterθ\\theta\. With open weights this would be a low\-rank adapter\(Huet al\.,[2022](https://arxiv.org/html/2607.00871#bib.bib31)\), prefix\(Li and Liang,[2021](https://arxiv.org/html/2607.00871#bib.bib32)\), or soft prompt\(Lesteret al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib33)\); the provider\-agnostic realization is a*steering adapter*: a stochastic policy over a finite set ofkksteering directives, parameterized by logitsθ∈ℝk\\theta\\in\\mathbb\{R\}^\{k\}withp=softmax​\(θ\)p=\\mathrm\{softmax\}\(\\theta\)\(§[3\.1](https://arxiv.org/html/2607.00871#S3.SS1)\)\.

L2L\_\{2\}— Harness\.Mutable orchestration: system prompt, tool definitions, step and exploration budgets, memory, the grown abstraction library, and a*repair pipeline*of adopted self\-repair primitives \(§[5\.4](https://arxiv.org/html/2607.00871#S5.SS4)\)\. Edited byAlg 4; grown byAlg 5; self\-repaired online\. Every structural edit changes the harness’s identity, so the policy version is well defined at all times\.

L3L\_\{3\}— Loop controller\.One of the five algorithms \(or their composite\)\. Holds certificates, gates, budgets, and archives; never part of the agent’s forward pass\.

The performative distribution𝒟​\(πt\)\\mathcal\{D\}\(\\pi\_\{t\}\)\(Perdomoet al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib1)\)is the distribution of \(prompt, tool\-call, environment\-response\) tuples the agent generates in deployment\. Each round the controller emits a*certificate*—a structured audit record carrying the round’s decision \(accept/hold/reject/nsf\), the error budget spent, and algorithm\-specific metrics\. Certificates form the unified ledger by which a run can be audited after the fact\.

### 3\.1The policy substrate

The pseudo\-code in this paper is written against the following concrete objects\.

#### Steering adapter \(L1L\_\{1\}\)\.

The only trainable parameters are logitsθ∈ℝk\\theta\\in\\mathbb\{R\}^\{k\}overkksteering directives—short natural\-language strategy instructions appended to the system prompt\. Each round the agent samples one directive,i∼softmax​\(θ\)i\\sim\\mathrm\{softmax\}\(\\theta\), so the distribution over directives is an explicit softmax whose log\-probability is known exactly on any backend\.*We never differentiate through the frozen model\.*The only gradient any controller uses is the closed\-form score\-function \(REINFORCE\) gradient of this softmax,

∇θlog⁡pθ​\(i\)=𝟏i−softmax​\(θ\),\\nabla\_\{\\theta\}\\log p\_\{\\theta\}\(i\)=\\mathbf\{1\}\_\{i\}\-\\mathrm\{softmax\}\(\\theta\),\(2\)which needs only the sampled indexiiand a softmax—no backpropagation, no model internals—and is therefore exact even on a text\-only API\. This is the gradientAlg 1andAlg 3consume\. For the forgetting gate the sameθ\\thetais read as a diagonal\-Gaussian PAC\-Bayes posteriorQθ=𝒩​\(θ,diag​ev\)Q\_\{\\theta\}=\\mathcal\{N\}\(\\theta,\\mathrm\{diag\}\\,e^\{v\}\)with fixed log\-variancevv, soKL​\(Qθ∥Q0\)\\mathrm\{KL\}\(Q\_\{\\theta\}\\\|Q\_\{0\}\)is just a scaled squared distance to the prior mean; the same vector supplies theℓ2\\ell\_\{2\}distance and ball projection the trust region needs\. No model Jacobian is ever formed: the “Jacobian memory” of orthogonal\-gradient continual learning is approximated by a finite FIFO buffer of past closed\-form gradients \([2](https://arxiv.org/html/2607.00871#S3.E2)\) \(Alg 1\), and true open\-weights Jacobian buffers, though supported, are used in no reported run\. Every committed policy is thus identifiable and auditable after the fact\.

#### Policy identity and distance\.

A policy’s*version*is determined by\(L1,L2\)\(L\_\{1\},L\_\{2\}\);L0L\_\{0\}is fixed\. Policy distance is∥θ−θ′∥2\\lVert\\theta\-\\theta^\{\\prime\}\\rVert\_\{2\}plus a unit structural step if the harnesses differ—harness edits are discrete, and their performative effect is estimated from deployment rewards rather than from a norm\.

#### Deployment\.

Deploy​\(π,tasks\)\\textsc\{Deploy\}\(\\pi,\\text\{tasks\}\)runs all tasks concurrently and returns one rollout per task: the directive index used, the action, the tool trajectory, and a bounded rewardR∈\[0,1\]R\\in\[0,1\]from the environment\. Two paths exist behind one interface: a single\-call path \(one LLM generation per task\), and an*actor*path in which a multi\-step tool\-using agent \(§[7](https://arxiv.org/html/2607.00871#S7)\) produces the final action\. An attempt index varies the sampling seed and the working directory, so independent attempts of the same \(policy, task\) pair are diverse and isolated—the substrate for best\-of\-NNand refinement search \(§[5](https://arxiv.org/html/2607.00871#S5)\)\. Rollouts are appended to a persistent replay buffer, whichAlg 3andAlg 4consume off\-policy\.

#### Access levels\.

What a provider exposes determines what is implementable\.*API\-text*\(text only\) already suffices for every controller, because the trainable policy is theL1L\_\{1\}softmax and its gradient \([2](https://arxiv.org/html/2607.00871#S3.E2)\) needs only sampled directive indices:Alg 1,Alg 2, andAlg 3run even on local text\-only servers, andAlg 4andAlg 5edit onlyL2L\_\{2\}\.*API\-logprob*adds sequence\-level importance ratios;*open weights*would additionally allow gradients through the model itself and true Jacobian memory—neither of which is used here\.

## 4The Five Loop Controllers

Throughout,πt\\pi\_\{t\}is the deployed configuration at roundtt;𝒟​\(π\)\\mathcal\{D\}\(\\pi\)the induced data distribution;ε\\varepsilonthe performative sensitivity \(W1​\(𝒟​\(π\),𝒟​\(π′\)\)≤ε​∥π−π′∥W\_\{1\}\(\\mathcal\{D\}\(\\pi\),\\mathcal\{D\}\(\\pi^\{\\prime\}\)\)\\leq\\varepsilon\\lVert\\pi\-\\pi^\{\\prime\}\\rVert\);LLthe loss’s smoothness\-to\-curvature ratio \(the condition numberβ/γ\\beta/\\gammaof Thm\. 3\.5 ofPerdomoet al\.\([2020](https://arxiv.org/html/2607.00871#bib.bib1)\), so the contraction regime isε​L<1\\varepsilon L<1\);δ0\\delta\_\{0\}a global error budget; andnsf\(“no solution found”\) the safe abstention output\(Thomaset al\.,[2015](https://arxiv.org/html/2607.00871#bib.bib19),[2019](https://arxiv.org/html/2607.00871#bib.bib20)\)\. All rewards are bounded in\[0,1\]\[0,1\]; the per\-rollout loss isℓ=1−R\\ell=1\-R\.

#### Two tiers, ten algorithms\.

The five controllers in this section \(Alg 1–Alg 5\) are the conceptual core\. The reference implementation pairs them with five verifier\-in\-the\-loop / actor\-side mechanisms \(Alg 6–Alg 10, §[5](https://arxiv.org/html/2607.00871#S5)–§[5\.4](https://arxiv.org/html/2607.00871#S5.SS4)\) that supply the signal and variation the controllers act on: gates can only*select*among existing behaviors, so the engine that generates and verifies those behaviors carries its own numbering\. Table[1](https://arxiv.org/html/2607.00871#S4.T1)is the full inventory\. Pseudocode for all algorithms is collected in Appendix[A](https://arxiv.org/html/2607.00871#A1)\.

Table 1:The implementation’s ten numbered algorithms\.Alg 1–Alg 5are the scheduledL3L\_\{3\}controllers of this section;Alg 6–Alg 10are the verifier\-in\-the\-loop and actor\-side mechanisms of §[5](https://arxiv.org/html/2607.00871#S5)–§[5\.4](https://arxiv.org/html/2607.00871#S5.SS4)\. The five controllers remain the contribution;Alg 6–Alg 10are the engine that feeds them\.Alg 4is omitted from the live SWE stack for wall\-clock cost \(§[7](https://arxiv.org/html/2607.00871#S7)\) and best\-of\-22\(Alg 6\) is removed from it \(§[9\.1](https://arxiv.org/html/2607.00871#S9.SS1)\); both remain in the catalog\.
### 4\.1Alg 1: Performative PAC\-Bayes Continual Learning \(PPB\-CL\)

Problem\.The agent must acquire new skills without catastrophically forgetting old ones, on a stream it generates itself\. The two natural seed results do not cover this setting: OGD’s forgetting guarantee assumes an exogenously indexed task sequence in the infinite\-width regime\(Farajtabaret al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib4); Abbana Bennaniet al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib5)\), and time\-uniform PAC\-Bayes forgetting certificates\(Chugget al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib2); Friedman and Meir,[2025](https://arxiv.org/html/2607.00871#bib.bib3)\)require a data\-independent prior—which fails the moment the agent reuses its own evolving policy as a prior\. The stream, the loss, and the prior are all endogenous here\.

Solution \(Algorithm[1](https://arxiv.org/html/2607.00871#alg1)\)\.*Intuition\.*Damp each update by how much it would degrade a small anchor set of past tasks, and cap its size by the remaining forgetting budget\. Concretely, keep the PAC\-Bayes object small and the prior frozen—the posterior is a mean\-parameterized diagonal GaussianQθQ\_\{\\theta\}over the steering adapter, with a frozen data\-independent priorQ0Q\_\{0\}\. Each round, a performative batch is deployed and a REINFORCE gradient of the expected loss is formed from the sampled directive indices, with the baseline pooled over the recent replay window \(a within\-round baseline is identically zero at batch size one\)\. The gradient is projected orthogonally off a FIFO buffer of the lastmmcommitted update directions \(the OGD guard\), and a candidate adapter is proposed by a single descent step\. The candidate must pass three gates in sequence\. \(i\)*Forgetting gate*: the candidate is deployed on a small*anchor set*of past tasks, each stored with its best historical rewardRj∗R^\{\*\}\_\{j\}\(withM=1M\{=\}1for rewards in\[0,1\]\[0,1\]; until anchors accumulate the gate passes vacuously\); the empirical backward transferbt^=1\|𝒜\|​∑j\[\(1−Rj\)−\(1−Rj∗\)\]\\widehat\{\\mathrm\{bt\}\}=\\frac\{1\}\{\|\\mathcal\{A\}\|\}\\sum\_\{j\}\[\(1\-R\_\{j\}\)\-\(1\-R^\{\*\}\_\{j\}\)\]enters the Donsker–Varadhan bound

Bfgt​\(θ\)=bt^​\(θ\)\+KL​\(Qθ∥Q0\)\+log⁡\(2​t/δ\)λ\+λ​M28​\|𝒜\|,B\_\{\\mathrm\{fgt\}\}\(\\theta\)\\;=\\;\\widehat\{\\mathrm\{bt\}\}\(\\theta\)\+\\frac\{\\mathrm\{KL\}\(Q\_\{\\theta\}\\\|Q\_\{0\}\)\+\\log\(2\\sqrt\{t\}/\\delta\)\}\{\\lambda\}\+\\frac\{\\lambda M^\{2\}\}\{8\|\\mathcal\{A\}\|\},\(3\)an instance of the anytime PAC\-Bayes recipe ofChugget al\.\([2023](https://arxiv.org/html/2607.00871#bib.bib2), Thm\. 3\.1\)—which holds for precisely the adapted posterior sequences a self\-evolving agent produces—via Ville’s inequality; the temperature is set per evaluation to its optimizerλ∗=8​\|𝒜\|​At\\lambda^\{\*\}=\\sqrt\{8\|\\mathcal\{A\}\|A\_\{t\}\}withAt=KL​\(Qθ∥Q0\)\+log⁡\(2​t/δ\)A\_\{t\}=\\mathrm\{KL\}\(Q\_\{\\theta\}\\\|Q\_\{0\}\)\+\\log\(2\\sqrt\{t\}/\\delta\), collapsing the last two terms of \([3](https://arxiv.org/html/2607.00871#S4.E3)\) toAt/\(2​\|𝒜\|\)\\sqrt\{A\_\{t\}/\(2\|\\mathcal\{A\}\|\)\}\(a fixedλ\\lambdaleft the bound an order of magnitude above any usable threshold\)\. IfBfgt\>τforgetB\_\{\\mathrm\{fgt\}\}\>\\tau\_\{\\mathrm\{forget\}\}, the step is damped geodesically toward the incumbent by factorρ\\rhoand re\-tested, up to five times\. \(ii\)*Performative trust region*: the surviving slackτforget−Bfgt\\tau\_\{\\mathrm\{forget\}\}\-B\_\{\\mathrm\{fgt\}\}, divided byε\\varepsilon, caps the adapter step inℓ2\\ell\_\{2\}; a violating candidate is projected onto the ball and the bound re\-evaluated\. This is what keeps the induced shift inside the contraction regime ofPerdomoet al\.\([2020](https://arxiv.org/html/2607.00871#bib.bib1)\)\. \(iii\)*Commit gate*: the candidate commits only ifBfgt≤τforgetB\_\{\\mathrm\{fgt\}\}\\leq\\tau\_\{\\mathrm\{forget\}\}andε​L<1\\varepsilon L<1; on commit, the projected gradient joins the buffer and the anchors absorb the round’s tasks with their best rewards\. The certificate reports the empirical risk,KL\\mathrm\{KL\}, the time\-uniform penalty\(KL\+log⁡\(2​t/δ\)\)/2​nt\\sqrt\{\(\\mathrm\{KL\}\+\\log\(2\\sqrt\{t\}/\\delta\)\)/2n\_\{t\}\}and risk bound, the forgetting bound, trust radius, and damping count\.

The pieces are published; their composition is an empirical construct we do not prove correct under the endogenous loop\. Each ingredient is sound on its own—the time\-uniform PAC\-Bayes penalty ofChugget al\.\([2023](https://arxiv.org/html/2607.00871#bib.bib2), Thm\. 3\.1\), the Donsker–Varadhan backward\-transfer forgetting bound ofFriedman and Meir \([2025](https://arxiv.org/html/2607.00871#bib.bib3), Thm\. 3\.1\), and the performative contraction regimeε<γ/β\\varepsilon<\\gamma/\\betaofPerdomoet al\.\([2020](https://arxiv.org/html/2607.00871#bib.bib1), Thm\. 3\.5\)—but whether they combine into an anytime\-valid forgetting guarantee that survives the loop is a conjecture, not a theorem we establish\. The capped direction buffer in particular forfeits the exact no\-forgetting property ofAbbana Bennaniet al\.\([2020](https://arxiv.org/html/2607.00871#bib.bib5), Thm\. 2\), which assumes unbounded Jacobian memory; and the anytime floor of \([3](https://arxiv.org/html/2607.00871#S4.E3)\) grows withtt, soτforget\\tau\_\{\\mathrm\{forget\}\}must sit above it or the gate freezes the learner\.

### 4\.2Alg 2: Performative Nash\-MP with Real\-Data Anchoring \(PNMP\-A\)

Problem\.Recursive self\-training and self\-rewarding loops drift to degenerate fixed points: the policy exploits the reward model’s over\-predictions, and the reward model is retrained on the policy’s own outputs\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib29); Shumailovet al\.,[2024](https://arxiv.org/html/2607.00871#bib.bib8)\)\. The seeds each cover half the problem: constant\-fraction real\-data anchoring prevents data collapse\(Fuet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib6)\)but says nothing about reward\-model drift, while last\-iterate Nash convergence\(Tiapkinet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib10); Wanget al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib9)\)assumes a*fixed*preference game\. The loop must be protected against both simultaneously\.

Solution \(Algorithm[2](https://arxiv.org/html/2607.00871#alg2)\)\.*Intuition\.*Mix a fixed reference anchor into every reward\-model update and gate any update that drifts from it, while letting the policy chase the reward model on a slower timescale than the model settles\. Two ideas do the work: separate the two risks onto two timescales\(Borkar,[2008](https://arxiv.org/html/2607.00871#bib.bib11)\), and gate the slow one\. The preference model is a Bradley–Terry score vectorq∈ℝkq\\in\\mathbb\{R\}^\{k\}over directives,P​\(i≻j\)=σ​\(qi−qj\)P\(i\\succ j\)=\\sigma\(q\_\{i\}\-q\_\{j\}\), with a frozen anchorqrealq\_\{\\mathrm\{real\}\}\(absent a held\-out real\-preference set the anchor defaults to the neutral zero vector, so in the SWE runs it prevents drift from neutrality rather than from human preferences\)\. Synthetic preference pairs come from directives pooled over the recent replay window, the higher\-mean directive winning\. The*slow*timescale blends the synthetic Bradley–Terry gradient with a pull toward the anchor,

qcand=q\+atslow​\[α​\(qreal−q\)\+\(1−α\)​∇qℒ^BT​\(q\)\],atslow=12​\(t\+1\)−1,q\_\{\\mathrm\{cand\}\}=q\+a^\{\\mathrm\{slow\}\}\_\{t\}\\big\[\\alpha\\,\(q\_\{\\mathrm\{real\}\}\-q\)\+\(1\-\\alpha\)\\,\\nabla\_\{q\}\\widehat\{\\mathcal\{L\}\}\_\{\\mathrm\{BT\}\}\(q\)\\big\],\\qquad a^\{\\mathrm\{slow\}\}\_\{t\}=\\tfrac\{1\}\{2\}\(t\{\+\}1\)^\{\-1\},\(4\)and is adopted only if it clears a drift gate: the per\-pair deviations from the anchor,Xi​j=\|σ​\(qicand−qjcand\)−σ​\(qireal−qjreal\)\|X\_\{ij\}=\|\\sigma\(q^\{\\mathrm\{cand\}\}\_\{i\}\-q^\{\\mathrm\{cand\}\}\_\{j\}\)\-\\sigma\(q^\{\\mathrm\{real\}\}\_\{i\}\-q^\{\\mathrm\{real\}\}\_\{j\}\)\|, feed a Hoeffding e\-process testingH0:𝔼​\[X\]≤τH\_\{0\}\\\!:\\mathbb\{E\}\[X\]\\leq\\tau, and rejection at the round’s CTHS levelδk\\delta\_\{k\}keeps the last safe model\. The*fast*timescale moves the policy by a regularized Nash mirror\-prox \(extragradient\) step on the logits,

θt=θt−1\+atfast​\(θMP−θt−1\),atfast=12​\(t\+1\)−0\.7,\\theta\_\{t\}=\\theta\_\{t\-1\}\+a^\{\\mathrm\{fast\}\}\_\{t\}\\,\(\\theta\_\{\\mathrm\{MP\}\}\-\\theta\_\{t\-1\}\),\\qquad a^\{\\mathrm\{fast\}\}\_\{t\}=\\tfrac\{1\}\{2\}\(t\{\+\}1\)^\{\-0\.7\},\(5\)whose game gradient is the self\-play advantageadvi​\(p\)=∑jpj​σ​\(qi−qj\)\\mathrm\{adv\}\_\{i\}\(p\)=\\sum\_\{j\}p\_\{j\}\\,\\sigma\(q\_\{i\}\-q\_\{j\}\)and whose prox operator folds in the KL to the previous iterate andβ\\betatimes the KL to the reference policy \(the two extragradient half\-steps are spelled out in Algorithm[2](https://arxiv.org/html/2607.00871#alg2)\)\. Becauseatslow/atfast→0a^\{\\mathrm\{slow\}\}\_\{t\}/a^\{\\mathrm\{fast\}\}\_\{t\}\\to 0, the model settles faster than the policy chases it—the required timescale separation\. A magnetzzis refreshed everyKKrounds andKL​\(πt∥z\)\\mathrm\{KL\}\(\\pi\_\{t\}\\\|z\)reported on the certificate; once the budgetδ0\\delta\_\{0\}is spent the preference model freezes and only policy updates continue\.

Here too the components are published and the combination is an unproven, empirical construct\. We borrow the constant\-α\\alphabound on cumulative distribution shift in self\-consuming training ofFuet al\.\([2025](https://arxiv.org/html/2607.00871#bib.bib6), Thm\. 1\), the geometric last\-iterate self\-play contraction \(up to a residual floor\) ofTiapkinet al\.\([2025](https://arxiv.org/html/2607.00871#bib.bib10), Prop\. 1\), and the anytime\-valid familywise control of e\-processes under harmonic spending\(Ramdaset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib12); Wuet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib21)\)\. Whether their two\-timescale coupling converges to a performative equilibrium without collapse is a conjecture, not a result we prove; we claim no rate, and note that performative equilibria need not be unique and that the shift bound degrades asα→0\\alpha\\to 0\.

### 4\.3Alg 3: Performative\-Aware COCOA \(PA\-COCOA\)

Problem\.Long\-horizon, sparse outcomes must be attributed across a multi\-component agent stack while the transition and reward kernels move with the deployed policy\(Mandalet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib15)\)\. COCOA’s unbiasedness\(Meulemanset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib14)\)holds under a fully\-predictive outcome encoding—pπ​\(Rk=r∣S0,A0,Uk=u\)=pπ​\(R=r∣U=u\)p^\{\\pi\}\(R\_\{k\}\{=\}r\\mid S\_\{0\},A\_\{0\},U\_\{k\}\{=\}u\)=p^\{\\pi\}\(R\{=\}r\\mid U\{=\}u\)for allkk\(their Def\. 2\)—with ground\-truth contribution coefficients \(their Thm\. 1\),*in a fixed MDP*; under endogenous drift the policy\-induced shift is a hidden confounder, and the high\-variance importance ratios of REINFORCE compound over the horizon\.

Solution \(Algorithm[3](https://arxiv.org/html/2607.00871#alg3)\)\.*Intuition\.*Replace “reward×\\timeslog\-prob” with a counterfactual per\-action contribution, condition on the deploying policy so policy drift stops confounding, and feed the result to a parameter\-free learner that restarts on detected drift\. Two moves realize this\. First,*augment*the outcome encoding with the deploying policy,U~=\(U,πt\)\\tilde\{U\}=\(U,\\pi\_\{t\}\), so the shift enters the conditioning set and stops confounding; concretely, every replay rollout carries its policy version and directive index, and the contribution model is estimated per directive over a recency\-weighted replay window, with the policy\-induced drift absorbed by the recency weighting rather than by explicit conditioning on the version \(a deliberate simplification of the full\(U,πt\)\(U,\\pi\_\{t\}\)encoder\)\. The contribution model is thus a recency\-weighted per\-directive mean of episode rewards over the recent window \(half\-lifehh; weight2−age/h2^\{\-\\mathrm\{age\}/h\}; default12\\tfrac\{1\}\{2\}for unseen directives\)—where the episode reward is the*process reward*R~\\tilde\{R\}of §[5](https://arxiv.org/html/2607.00871#S5)when available, i\.e\. the best verifier state the episode reached rather than only its terminal submit\. The augmented COCOA gradient then sums counterfactual contributions over*all*actions, eliminating the importance ratio; for the softmax steering adapter it has the closed form

g=∑a′w^​\(a′\)​∇θpθ​\(a′\)=\(w^⊙p\)−p​⟨w^,p⟩,p=softmax​\(θ\),g\\;=\\;\\sum\_\{a^\{\\prime\}\}\\widehat\{w\}\(a^\{\\prime\}\)\\,\\nabla\_\{\\theta\}\\,p\_\{\\theta\}\(a^\{\\prime\}\)\\;=\\;\\big\(\\widehat\{w\}\\odot p\\big\)\-p\\,\\langle\\widehat\{w\},p\\rangle,\\qquad p=\\mathrm\{softmax\}\(\\theta\),\(6\)a simplified instance of the COCOA estimator in which credit attaches to the episode outcome rather than to per\-step rewards\. A mean\-reverting exploration floor addsφ​\(𝟏/k−p\)\\varphi\\,\(\\mathbf\{1\}/k\-p\)to the gradient, keeping every directive sampled so its credit estimate stays fresh instead of collapsing onto one arm of an inflated signal\. Second, hand the update to a learner that needs no learning rate and tracks drift: per\-coordinate Krichevsky–Trofimov coin betting\(Orabona and Pál,[2016](https://arxiv.org/html/2607.00871#bib.bib18)\)\(the iterate isx=x0\+βt​Wtx=x\_\{0\}\+\\beta\_\{t\}\\mathrm\{W\}\_\{t\}with clipped betting fractionβt\\beta\_\{t\}and per\-coordinate wealthWt\\mathrm\{W\}\_\{t\}\), wrapped with drift\-triggered restarts\. A seeded wild\-bootstrap trend test \(§[6](https://arxiv.org/html/2607.00871#S6)\) on the mean\-reward stream signals non\-stationarity and the bettor re\-anchors at the current iterate—a restart\-based surrogate for the strongly\-adaptive guarantees that yield path\-length dynamic regretO~​\(T​VT\)\\widetilde\{O\}\(\\sqrt\{TV\_\{T\}\}\)on every interval\(Cutkosky,[2020](https://arxiv.org/html/2607.00871#bib.bib16)\), and the optimalO~​\(T1/3​VT2/3\)\\widetilde\{O\}\(T^\{1/3\}V\_\{T\}^\{2/3\}\)rate for strongly convex losses\(Baby and Wang,[2022](https://arxiv.org/html/2607.00871#bib.bib17)\)\.

As before, every component is published and the composition is empirical, with no correctness proof in the performative setting\. The unbiasedness of the counterfactual\-contribution estimator under a fully\-predictive encoding isMeulemanset al\.\([2023](https://arxiv.org/html/2607.00871#bib.bib14), Thm\. 1\), and Proposition 5 there already shows the hindsight model is invariant to conditioning on the policy logits within a deployment; the parameter\-free dynamic\-regret rates areCutkosky \([2020](https://arxiv.org/html/2607.00871#bib.bib16)\)\(path\-lengthO~​\(T​VT\)\\widetilde\{O\}\(\\sqrt\{TV\_\{T\}\}\)on every interval, withVTV\_\{T\}the comparator path length\) andBaby and Wang \([2022](https://arxiv.org/html/2607.00871#bib.bib17)\)\(O~​\(T1/3​VT2/3\)\\widetilde\{O\}\(T^\{1/3\}V\_\{T\}^\{2/3\}\)for strongly convex losses\)\. Whether unbiasedness and these rates carry to the performative MDP\(Mandalet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib15)\)across rounds is a conjecture we do not prove: a fully\-predictiveU~\\tilde\{U\}must be learnable, the recency\-weighted contribution estimate reintroduces a bias their theorem does not cover, and the regret statement is vacuous once path variationVTV\_\{T\}grows linearly\.

### 4\.4Alg 4: SGM Gated by Anytime\-Valid Confidence Sequences \(SGM\-CS\)

Problem\.A self\-modification must be admitted only if it statistically improves the agent, with risk bounded across an*unbounded*sequence of self\-proposed edits, where the proposal process is endogenous \(it may exploit residual statistical slack\) and a deployed edit changes the very distribution in which it is judged\.

Solution \(Algorithm[4](https://arxiv.org/html/2607.00871#alg4)\)\.*Intuition\.*Admit a self\-edit only through a sequential, peek\-safe paired test that spends a shrinking per\-edit slice of a global error budget—so familywise error stays bounded over an unbounded edit stream—and correct that test for the distribution shift the edit itself induces; abstain when in doubt\.Alg 4runs on API\-text access, edits onlyL2L\_\{2\}, and is the governance layer to deploy first\. The proposer is the frozen LLM acting on its own harness, cycling the three knobs of the explore/edit economy \(§[5](https://arxiv.org/html/2607.00871#S5)\): the forced\-edit threshold, the step budget, and the strategy guidance\. Baselineπt−1\\pi\_\{t\-1\}and candidateπcand\\pi\_\{\\mathrm\{cand\}\}are deployed*concurrently on the same task batch*under common random numbers \(aligned per\-task seeds, isolated working copies\), and the decision is a lower bound on the paired*process*\-reward \(Eq\.[10](https://arxiv.org/html/2607.00871#S5.E10)\) gain, corrected for performativity:

LCB=CSδklower​\(Rcand−Rbase\)−ε⋅W1​\(Rcand,Rbase\),admit iff​LCB≥−ϵtol\.\\mathrm\{LCB\}=\\mathrm\{CS\}^\{\\mathrm\{lower\}\}\_\{\\delta\_\{k\}\}\\\!\\big\(R\_\{\\mathrm\{cand\}\}\-R\_\{\\mathrm\{base\}\}\\big\)\-\\varepsilon\\cdot W\_\{1\}\(R\_\{\\mathrm\{cand\}\},R\_\{\\mathrm\{base\}\}\),\\qquad\\text\{admit iff \}\\mathrm\{LCB\}\\geq\-\\epsilon\_\{\\mathrm\{tol\}\}\.\(7\)The paired\-difference confidence sequence is taken at the round’s CTHS levelδk=δ0/\(Z​k​log2⁡\(k\+1\)\)\\delta\_\{k\}=\\delta\_\{0\}/\(Z\\,k\\log^\{2\}\(k\{\+\}1\)\)\(§[6](https://arxiv.org/html/2607.00871#S6)\); theW1W\_\{1\}term, computed exactly in 1\-D from the samples, inflates the bound by the shift the edit induces; and per\-version normal\-mixture sequences\(Howardet al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib22)\)accumulate value evidence across rounds\. Three details keep the test honest and cheap\. \(i\) Two free guards spend nothing: an exhausted budget or a no\-op proposal simply holds\. \(ii\) A*pre\-gate pilot*on the round’s best\-evidenced \(known\-passable\) task rejects \(nsf\) a candidate that shows no promise there, while an uninformative zero\-reward pilot falls through to the full evaluation; the remaining tasks are then ranked so the comparison falls where evidence is weakest\. \(iii\) A seeded wild\-bootstrap trend test\(Chandaket al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib23)\), fed only by rounds that actually re\-evaluated the baseline, widens the bound by one radius when the baseline\-value stream is non\-stationary\. Otherwise the round returnsnsfand the baseline is kept\.

The statistical components are published; their composition under an endogenous proposer is an empirical construct we do not prove safe\. We reuse anytime\-valid e\-processes with familywise control under Ville’s inequality\(Ramdaset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib12)\), the confirm\-triggered harmonic spending of the Statistical Gödel Machine\(Wuet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib21)\), the time\-uniform confidence sequence ofHowardet al\.\([2021](https://arxiv.org/html/2607.00871#bib.bib22)\), and the abstention semantics of safe policy improvement\(Thomaset al\.,[2015](https://arxiv.org/html/2607.00871#bib.bib19),[2019](https://arxiv.org/html/2607.00871#bib.bib20)\)\. Whether familywise safety survives the endogenous proposal process and the edit\-induced distribution shift is open: the performative correction rests on a knowable bound forε\\varepsilon, and the wild\-bootstrap widening\(Chandaket al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib23)\)was derived for*exogenous*non\-stationarity, so under the endogenous loop it is a conservative heuristic\. By construction the protocol favors*safety over progress*—it may rationally abstain \(nsf\) indefinitely\. So that this conservatism never discards a good*solution*\(as opposed to a good harness*edit*\), the SWE instantiation keeps a*shadow\-best*: the highest\-reward candidate patch seen across rounds is retained even when the harness\-edit gate abstains, decoupling “which harness to deploy” \(gated\) from “which patch to submit” \(best\-so\-far\); and a rejected harness that nonetheless produced a strong patch is*requeued*for a later round, with eligibility keyed on an absolute reward threshold and a bounded per\-harness retry count, and replayed only in a round whose active repositories include its own\.

### 4\.5Alg 5: Stitch\-in\-DreamCoder with Quality\-Diversity Acceptance \(SDC\-QD\)

Problem\.The agent must grow its own hypothesis space—invent reusable abstractions—while the corpus those abstractions are mined from is generated under the very library being learned\. DreamCoder’s endogenous loop has no convergence or no\-collapse theorem\(Elliset al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib24)\); Stitch’s MDL\-optimal abstraction step is one\-shot\(Bowerset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib25)\)\. A single converging library is the wrong target when corpus generation is endogenous\.

Solution \(Algorithm[5](https://arxiv.org/html/2607.00871#alg5)\)\.*Intuition\.*Keep a quality\-diversity archive of behaviorally distinct libraries rather than one “best” library, and add an abstraction only when it both reduces description length and occupies a new niche; each addition is the exact most\-compressive one on the current corpus\. Concretely, replace the single library with an*archive over a frontier*, and make each growth step exact\. Programs are S\-expressions with description length\|ρ\|\|\\rho\|equal to the node count, and the objective is the node\-count MDL surrogateJ​\(L;C\)=\|L\|\+∑ρ∈C\|ρ\|J\(L;C\)=\|L\|\+\\sum\_\{\\rho\\in C\}\|\\rho\|of DreamCoder’s description\-length posterior over libraries\(Elliset al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib24)\)\. Each round, a*wake*phase asks the frozen LLM to synthesize one program per task under the current library \(shown in the prompt; output is parsed defensively, since LLM output may be prose\); programs whose environment score reaches the solve threshold join the corpus\. The*sleep*phase runs Stitch\-style compression: candidate patterns are all corpus subtrees plus all pairwise antiunifications of structurally compatible subtrees \(differing positions become numbered holes\), and the net utility of a pattern with body sizebb, arityaa, andmmnon\-overlapping matches isu=m​\(b−1−a\)−bu=m\\,\(b\-1\-a\)\-b\(each match is rewritten as an application of size1\+a1\+a; the definition is paid once\)\. Candidates are scanned in descending order of utility with an early dominance break, so the maximizer over the candidate set is exact \(the soundness argument behind Stitch’s dominance pruning, Lemma 1 ofBowerset al\.\([2023](https://arxiv.org/html/2607.00871#bib.bib25)\)\)\. The best abstractionA∗A^\{\*\}is accepted only if \(i\)Δ​J=u​\(A∗\)≥umin\\Delta J=u\(A^\{\*\}\)\\geq u\_\{\\min\}, the minimum\-utility bar \(so a marginally compressive pattern does not enter the library\), and \(ii\) the resulting library is*novel or improving*in a MAP\-Elites archive keyed by the behavior descriptorϕ=\(library size,mean solved\-program size\)\\phi=\(\\text\{library size\},\\text\{mean solved\-program size\}\)with fitness the post\-compression MDL—so the run illuminates a \(compression, coverage\) frontier rather than converging to one point\(Mouret and Clune,[2015](https://arxiv.org/html/2607.00871#bib.bib30); Cully and Demiris,[2018](https://arxiv.org/html/2607.00871#bib.bib28)\)\. Accepted abstractions are surfaced asL2L\_\{2\}harness entries the agent can call\. A Hoeffding\-type per\-task description\-length certificate, in the spirit of the PAC\-Bayes lifelong\-learning bound ofPentina and Lampert \([2014](https://arxiv.org/html/2607.00871#bib.bib26)\), bounds expected per\-task description length on held\-out tasks:

𝔼new task​\[DL\]≤J​\(Lt;Ct\)\|Ct\|\+\|Lt\|\|Ct\|\+log⁡\(1/δ\)2​\|Ct\|\.\\mathbb\{E\}\_\{\\text\{new task\}\}\[\\mathrm\{DL\}\]\\;\\leq\\;\\frac\{J\(L\_\{t\};C\_\{t\}\)\}\{\|C\_\{t\}\|\}\+\\frac\{\|L\_\{t\}\|\}\{\|C\_\{t\}\|\}\+\\sqrt\{\\frac\{\\log\(1/\\delta\)\}\{2\|C\_\{t\}\|\}\}\.\(8\)On the SWE agent \(§[7](https://arxiv.org/html/2607.00871#S7)\), the wake phase is replaced by the agent’s own behavior: successful trajectories, encoded as\(tool,target\)\(\\text\{tool\},\\text\{target\}\)operation sequences, become the corpus, and the mined macros become callable harness entries\.

The per\-round compression step is exact and published; the loop around it is an empirical construct without a convergence proof\. The single step is exact in the published sense: Lemma 1 ofBowerset al\.\([2023](https://arxiv.org/html/2607.00871#bib.bib25)\)shows that dominance pruning never discards the optimal abstraction, so the accepted macro is the most\-compressive one on the current corpus \(over the implemented candidate set\), and MDL is monotone non\-increasing under a fixed corpus\. The held\-out description\-length certificate \(Eq\.[8](https://arxiv.org/html/2607.00871#S4.E8)\) is Hoeffding/Occam\-type—it carries noKL\\mathrm\{KL\}term and so is not itself a PAC\-Bayes bound, but shares the Hoeffding\-lemma core of the lifelong PAC\-Bayes bound ofPentina and Lampert \([2014](https://arxiv.org/html/2607.00871#bib.bib26)\)—and assumes an exchangeable task distribution that endogenous generation may violate\. Convergence of the quality\-diversity archive to a meaningful frontier is*not*a published result—no QD framework, includingCully and Demiris \([2018](https://arxiv.org/html/2607.00871#bib.bib28)\), proves it—so we make no such claim; it is at most a conjecture, and the global limit need not be meaningful when corpus generation is endogenous\. Every abstraction is a composition within a fixed typedλ\\lambda\-calculus; expanding the type system is beyond this algorithm\.

## 5The Verifier as an In\-Loop Control Signal

Problem\.The five controllers gate, select, and reshape: they act on the distribution of behaviors the frozen base model already produces\. On hard agentic tasks that distribution offers a passive learner little\. \(i\) The natural reward is*terminal and binary*—an instance is resolved or it is not—so a base that never fully resolves yields an identically zero signal and the loop has no gradient\. \(ii\) Some failures are*systematic, not stochastic*: on certain instances a base re\-runs the*same*attempt under temperature, returning identical partial patches and never the occasional good one, so selection over samples has no variance to exploit \(an instance\-level phenomenon, not a property of a whole model\)\. \(iii\) Yet the same base*is*reliable at the*micro\-step*—it can localize and emit an applying one\-line edit—even when it gets the whole\-patch*content*wrong; reliability lives at a finer granularity than the unit the loop scores\. \(iv\) The interventions that helped were*control\-flow*, not capability: forcing an edit after an exploration budget changes what the model does next, whereas coaxing latent quality does not\. \(v\) The verifier—the one component that knows ground truth—was consulted once, after the episode, when nothing could be done about what it reported; this is exactly the regime in which selection, not generation, is the bottleneck\(Brownet al\.,[2024](https://arxiv.org/html/2607.00871#bib.bib43)\)\. The problem is to convert the verifier from a passive end\-of\-episode score into a control signal that acts*inside*the episode,*across*attempts, over the*action space*, and inside*credit assignment*, and to relocate both the search and the reward to the micro\-step granularity where the base is reliable—without ever leaking gold labels\.

Solution\.A graded verifier and a family of search operators, all derived from test execution alone and all composable with every controller through the actor interface \(the attempt\-indexed deployment substrate of §[3\.1](https://arxiv.org/html/2607.00871#S3.SS1)\), building up to a verified micro\-step search \(§[5\.1](https://arxiv.org/html/2607.00871#S5.SS1)\)\.

#### Graded verifier\.

A shaped milestone reward first decomposes the path to a resolve into rungs derived only from test execution:0for no parseable diff,0\.20\.2for a diff that does not apply, and0\.4\+0\.6⋅fracf2p0\.4\+0\.6\\cdot\\mathrm\{frac\}\_\{\\mathrm\{f2p\}\}once tests run \(halved and capped at0\.550\.55when pass\-to\-pass regressions appear\), capped at11\. Because a whole batch of edits can sit at the same rung \(every one applies but none passes a test\), this is still too coarse to climb; the*graded*verifier adds two finer signals on top of the milestone scoresshapeds\_\{\\mathrm\{shaped\}\}:

V=\{1resolvedmin⁡\(0\.99,sshaped\+𝟙​\[sshaped≥0\.4\]​\(wp​1​\[progressed\]\+wj​j\)\)otherwise,V\\;=\\;\\begin\{cases\}1&\\text\{resolved\}\\\\\[2\.0pt\] \\min\\\!\\big\(0\.99,\\ s\_\{\\mathrm\{shaped\}\}\+\\mathbb\{1\}\[s\_\{\\mathrm\{shaped\}\}\\geq 0\.4\]\\,\(w\_\{p\}\\,\\mathbb\{1\}\[\\text\{progressed\}\]\+w\_\{j\}\\,j\)\\big\)&\\text\{otherwise,\}\\end\{cases\}\(9\)where the bonuses apply only once a patch has cleared the apply\-and\-run rung \(sshaped≥0\.4s\_\{\\mathrm\{shaped\}\}\\geq 0\.4\),*progressed*is true when the test’s failure*signature*moves \(the lastpytesterror line, with volatile numbers and addresses masked, differs from the seed’s—so the edit got further before failing\),j∈\[0,1\]j\\in\[0,1\]is an auxiliary LLM judge, and the bonuseswp,wjw\_\{p\},w\_\{j\}are kept small \(0\.080\.08each\) so an actual sub\-test pass always outranks the heuristics and a non\-resolved patch never reaches11\. Error\-progression is the key addition: it turns a plateau of equally\-applying patches into a gradient\. The fail\-to\-pass fractionfracf2p\\mathrm\{frac\}\_\{\\mathrm\{f2p\}\}here is computed from each instance’s held\-out gold tests, so this graded score is reserved for*terminal measurement*\(and offline development\), exactly as a test set should be—never to steer the search\. The dense signal the search actually climbs is supplied by a self\-authored verifier that reads only the issue \(§[5\.2](https://arxiv.org/html/2607.00871#S5.SS2)\)\.

#### Closed\-loop test execution\.

Arun\_teststool lets the agent run the real target tests on its*current*edits mid\-episode and read back actionable feedback: pass counts and the tail of the failure log\. A short,*typed repair instruction*keyed to the failure class \(a name or import error, an assertion mismatch, a syntax error, or a no\-patch episode each map to a specific corrective hint\) is injected into the guidance of the next*refinement attempt*\(Algorithm[6](https://arxiv.org/html/2607.00871#alg6)\), pairing each recorded failure with a concrete next step\. The tool is stateful \(its result depends on the current edits\), so it is exempt from the agent’s duplicate\-action guard\. This changes the task from one\-shot patch guessing to closed\-loop debugging—the agent can edit, test, read a failure paired with a concrete next step, and iterate before submitting\.

#### Process reward\.

With the verifier callable mid\-episode, credit assignment need not wait for the terminal submit\. The process reward of an episode is the best verifier state it reached,

R~=max⁡\(Rterminal,maxj⁡sj\),\\tilde\{R\}\\;=\\;\\max\\\!\\big\(R\_\{\\mathrm\{terminal\}\},\\ \\max\_\{j\}s\_\{j\}\\big\),\(10\)wheresjs\_\{j\}is the shaped score at the agent’sjj\-thrun\_testscall\. A trajectory that reached a good state and then regressed still carries credit;Alg 3’s contribution model \(Algorithm[3](https://arxiv.org/html/2607.00871#alg3)\) consumesR~\\tilde\{R\}in place of the terminal reward\.

#### Alg 6: Verifier\-gated best\-of\-NNand refinement search \(Algorithm[6](https://arxiv.org/html/2607.00871#alg6)\)\.

Across attempts, two search operators sit on top of any actor\.BestOfNrunsnnindependent, diverse attempts \(the attempt index varies the sampling seed and the working directory\) and keeps the one the verifier scores highest: under independent attempts, a per\-attempt resolve probabilityppbecomes1−\(1−p\)n1\-\(1\-p\)^\{n\}—a strict lift wheneverp\>0p\>0\. Empirically, coverage under repeated sampling grows along an approximate exponentiated power law innn, and a ground\-truth verifier is exactly the selector that converts coverage into solved instances\(Brownet al\.,[2024](https://arxiv.org/html/2607.00871#bib.bib43)\)\.Refineis a sequential hill\-climb: each attempt after the first is seeded with the best patch so far*and the verifier’s explanation of why it failed*, the new candidate is scored on the original task, and it replaces the incumbent only if strictly better—otherwise the search backtracks to the best known patch\. Refinement stops early on a full resolve\. We*removed*best\-of\-22from the live stack: the run logs show it never produced a second attempt \(the attempt count stays11\) and, run in isolation, degraded the apply\-rate, so it is reported here as a designed operator rather than a live component \(§[9\.1](https://arxiv.org/html/2607.00871#S9.SS1)\)\. Selecting by the verifier isAlg 4’s accept\-gate pointed at patches; keeping the best over diverse samples is the selection principle ofAlg 5—here applied at test time, within a single task\.

#### Enforced explore→\\toedit budget\.

Over the action space, the harness gains a tunable thresholdbexploreb\_\{\\mathrm\{explore\}\}: once the agent has spentbexploreb\_\{\\mathrm\{explore\}\}read\-only steps \(search/read/list\) without landing an edit, it is first warned to edit and then, after a short grace window of a few further explores under escalating pressure, the exploration tools are disabled until an edit is made\. The grace window is not cosmetic: a hard cap with no grace deadlocked a mid\-localization agent into a no\-op read loop that produced an empty patch, whereas the graded pressure let the same agent land its edit \(in one live case turning an empty patch into a correctly localized fix\)\. This converts the diagnosed over\-exploration failure from a property of the model into a property of the harness—and the harness is governed:bexploreb\_\{\\mathrm\{explore\}\}is a knob inAlg 4’s proposer cycle \(Algorithm[4](https://arxiv.org/html/2607.00871#alg4)\), so tightening it must survive the confidence\-sequence gate like any other self\-edit\.

### 5\.1Alg 7: Verified micro\-step search \(the fast loop\)

Best\-of\-NNand refinement search over*whole patches*, where a weak base is unreliable and near\-deterministic\. The fast loop \(Algorithm[7](https://arxiv.org/html/2607.00871#alg7)\) instead searches over*verified micro\-steps*: from the current best patch and the verifier’s failure feedback, a*reasoning\-first*generator \(first name the root cause, class, and method; then emit edits\) proposeskkmaterially different one\-line hypotheses, diversity forced by the prompt against a working memory of past failures rather than by sampling temperature\. Each hypothesis is composed onto the current best and scored only if it applies\. Three components keep the search efficient: a beam of widthbbover verified partial patches; the working memory of tried edits and distinct failure signatures, rendered into the prompt to suppress repeats; and a cheap\-to\-expensive verification cascade \(a parse/apply check, then the native fail\-to\-pass run only on survivors\) that confines the minute\-long test run to compiling patches\. The search halts on a pass and gives up when a full round of fresh hypotheses moves nothing\. Hypotheses may be pooled across several base models into one verified beam\.

### 5\.2Alg 8: The in\-loop verifier — self\-authored reproduction oracles

Problem\.The search needs a dense, per\-candidate signal to climb, but the held\-out gold tests must be reserved for unbiased terminal measurement—a learner that is allowed to consult its own test set will report progress that does not generalize\. So the in\-loop verifier must be computable from the*issue alone*, using no information from the gold tests, the test patch, or the fail\-to\-pass/pass\-to\-pass lists, while still being informative enough to tell a real fix from a near\-miss\.

Solution \(Algorithm[8](https://arxiv.org/html/2607.00871#alg8)\)\.*Intuition\.*Reproducing a described bug is easier than fixing it, so the model writes tests from the issue, keeps only those that fail on the unpatched code, and debugs against them; the held\-out tests are touched only at the end, to report whether the self\-authored verifier was right\. Concretely, a verifier model reads only the issue text \(and, optionally, repository source\) and writes minimal scripts that assert the*correct*expected behavior, so each script should fail on the buggy code and pass once the bug is fixed\. The leverage is asymmetric: reproducing a described bug is a higher\-success task than fixing it—the issue states the symptom, the model need only encode it—so a model unreliable at fixing can still author a usable verifier\.

*Admission\.*An oracle is kept only if it*fails on the unpatched base*\(a check that already passes on the buggy code captures nothing\), with two cheap rejections: timeouts, and failures that are unambiguously the script’s own fault, i\.e\. a syntax or indentation error\. An import or name error is deliberately*not*auto\-rejected—a bug whose very symptom is a broken import must remain admissible—and a symptom judge then demotes any oracle whose base failure does not match the issue’s stated symptom\. To improve*recall*\(whether a usable oracle gets built at all\), oracles are pooled across an*ensemble*of verifier models, so a verifier exists whenever*any*model authors one; this only trades precision in a safe direction, since a spurious oracle can make the score*under*\-claim \(a correct patch fails to flip an over\-strict check\) but can never mark a wrong patch correct\.

*Extracting the signal\.*LetAAbe the admitted oracle suite\. A candidate patchρ\\rhois run once \(prepared a single time, then every oracle executed against it\), and its scalar score is the fraction of admitted oracles it flips from failing to passing, voided if any previously\-passing check regresses:

Vself​\(ρ\)=\|\{o∈A:o​fails on base,o​passes on​ρ\}\|\|A\|if no green check regresses, else​0\.V\_\{\\mathrm\{self\}\}\(\\rho\)\\;=\\;\\frac\{\\big\|\\\{\\,o\\in A:o\\text\{ fails on base\},\\ o\\text\{ passes on \}\\rho\\,\\\}\\big\|\}\{\|A\|\}\\quad\\text\{if no green check regresses, else \}0\.\(11\)A patch is*promising*\(we never say “resolved”\) when every admitted oracle flips and nothing regresses\. HereVselfV\_\{\\mathrm\{self\}\}is the flip fraction \(the regression\-voiding term is optional\), and the held\-out grader runs the fail\-to\-pass directives, so “the grader agrees” means the gold target tests flip\. The searcher is the multi\-step tool\-using agent, whoserun\_teststool returnsVselfV\_\{\\mathrm\{self\}\}rather than the grader, so it debugs against a signal it authored; a robust search\-and\-replace applier \(an edit’s anchor must match exactly once, with a unique whitespace\-normalized fuzzy fallback and a nearest\-match hint on a miss\) turns the model’s edits into a clean diff so even a weak solver emits applying patches without silent wrong\-site edits\. The held\-out grader is invoked only once, on the finalized patch, to measure whetherVselfV\_\{\\mathrm\{self\}\}was right\. Around the single admission rule the implementation adds three safeguards: rejected oracles are*refined*for a bounded number of rounds \(each re\-prompt carries the reason for rejection, stopping at the first admission\); a quality gate demotes weak oracles whose assertions are negative\-only; and oracle code must ground itself in symbols actually extracted from the issue text, with a stoplist rejecting generic tokens\.

#### The no\-oracle path\.

When no oracle is admitted, the only gold\-free evidence is that related pre\-existing tests still pass—necessary, not sufficient\. This path is hardened in three ways\. A fully green but unverified verdict is*capped*at0\.90\.9with an explicit “nothing verified the issue itself is fixed” note, so best\-of\-NN, refinement, verify\-react, and the cross\-round memory stay armed exactly where verification signal is weakest\. Candidates that tie at the cap are split by a gold\-free*issue\-fidelity judge*\(issue text plus the two patches, one\-token verdict\)\. A regression veto keyed on the failing phrase prevents an oracle pass from masking a broken pre\-existing test, and a collection\-error parser closes the remaining false\-green channel \(a test run that dies during collection no longer reads as green\)\.

Why measurement stays honest\.BecauseVselfV\_\{\\mathrm\{self\}\}is a function only of the issue text and the repository’s own behavior, the held\-out tests play no role in steering the search; their single terminal call measures the result without having shaped it\. The price is thatVselfV\_\{\\mathrm\{self\}\}is*fallible*—it can call a patch promising that the grader rejects, or miss a fix its oracles did not cover—which is exactly why a patch is reported as “promising,” never “resolved,” and why*agreement*between the self\-oracle verdict and the independent terminal grader \(§[9](https://arxiv.org/html/2607.00871#S9)\) is the quantity we report\.

### 5\.3From search to weights: the slow loop and re\-aimed controllers

The fast loop raises quality at test time; a*slow loop*would amortize it into weights by collecting verified micro\-step trajectories \(label\-free positive data—they passed the tests\) and distilling them into a low\-rank adapter by reject sampling\(Zelikmanet al\.,[2022](https://arxiv.org/html/2607.00871#bib.bib41); Gulcehreet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib42)\)\.*The slow loop is designed but not trained in any reported run*\(§[10](https://arxiv.org/html/2607.00871#S10)\)\. The five controllers are*re\-aimed*from the single\-pass actor onto this scaffold \(Table[2](https://arxiv.org/html/2607.00871#S5.T2)\); four act on the search layer, reuse the primitives of §[6](https://arxiv.org/html/2607.00871#S6), wrap the beam search of Algorithm[7](https://arxiv.org/html/2607.00871#alg7)and the continual stream of Algorithm[9](https://arxiv.org/html/2607.00871#alg9), and are validated*only*by deterministic offline gate simulations, not live:

- •Alg 4becomes a*compute\-allocation*controller: an anytime\-valid confidence sequence on the per\-round marginal graded gain stops a branch once even its optimistic bound clears no further expected gain \(a rigorous campaign\-level allocator that bounds false\-abandonment, plus a cheap per\-instance plateau fast\-path\), reclaiming the expensive verifier budget for productive instances\.
- •Alg 5becomes the*branch diversity generator*: a MAP\-Elites archive over a behavior descriptor keeps one elite per behavior cell, so the beam spends its width on behaviorally distinct partial patches rather thanbbminor variants of one edit—the direct cure for the no\-variance failure\.
- •Alg 3becomes*step\-level credit*: a recency\-weighted mean of*verified*reward per branch class \(the edited file\) orders which step macro to try first, so the search stops re\-discovering the same productive edit shape on every problem\.
- •Alg 1becomes an anchored*forgetting gate*: an update to the evolved search/distilled policy is accepted only if a re\-evaluation of frozen earlier\-repo tasks does not regress\.

The fifth,Alg 2, guards the*slow loop*: it is the mode\-collapse defense for the distillation step \(an e\-value drift gate on trajectory diversity\), and is realized only once the slow loop trains—which the present system does not yet do, soAlg 2at the search layer remains design\.

Table 2:The five controllers re\-aimed from the single\-pass actor onto the two\-loop \(search\-then\-distill\) scaffold\. Four are built on the search layer and validated by deterministic offline gate simulations; the slow\-loop anti\-collapse guard awaits the distillation training step\.
### 5\.4Alg 10: Verified self\-repair of the harness

Problem\.A search is only as good as its operators\. In the fast loop, a recurring*harness*failure—roughly half of the pooled ensemble’s candidate edits failed to apply, because the model copiedsearchtext from the issue’s*diff*\(lines marked\+\+/−\-\) rather than from the current file—starves the search of valid candidates\. This is exactly the kind of systematic failure a self\-evolving agent should fix*itself*\. The tempting response is a human hand\-patch \(“strip the diff markers fromsearch”\); but a hand\-patch is an unverified belief about the failure’s cause, and acting on it is precisely the trust the rest of the paper forbids\.

Solution \(Algorithm[10](https://arxiv.org/html/2607.00871#alg10)\)\.The harness carries an evolvable*repair pipeline*—a list of adopted repair primitives, anL2L\_\{2\}component likeAlg 5’s grown library—applied to every hypothesis before it is composed\. Providing a*repertoire*of small, composable primitives \(de\-marking a diff into verbatim search/replace, stripping code fences, whitespace\-fuzzy matching against the actual file\) is not hand\-patching: the loop*selects and verifies*which to activate\. The gate \(ProposeRepairs\) takes the candidates that just failed to compose and, for each library primitive,*measures*the fraction it makes actually apply against the real composer, adopting only those clearing a threshold, best\-first; a repair is credited only when it genuinely changes the hypothesis, so an inert primitive earns nothing\. A second, generation\-level gate verifies prompt\-suffix repairs by*re\-generating*and re\-composing, adopting a suffix only if it beats the un\-amended apply\-rate by a margin\. Both gates ground adoption in execution, never in a prior\. The grown repertoire is the natural target ofAlg 5\(which can add primitives\) and the per\-instance selection is the natural target ofAlg 4’s confidence\-gated acceptance\.

The mechanism earned its keep by*refusing*a fix: run over the real recorded failures, the diff\-de\-marking primitive cleared the gate on0of3434candidates—the dominant failure was not naive diff prefixes butsearchtext targeting code the seed patch had already replaced, which no text rewrite can recover—so the verification gate rejected the hand\-patch a human would have shipped, and instead adopted a generation\-level repair where re\-generation raised the measured apply\-rate \(33%→50%33\\%\\to 50\\%on one anchor\) and declined it where re\-measurement showed no gain \(83%→83%83\\%\\to 83\\%on another\)\. These are single\-instance observations, not a controlled evaluation\.

Whether each mechanism lifts a given base model—and which failures are stochastic \(where best\-of\-NNand refinement have variation to exploit\) versus systematic \(where the budget enforcement, micro\-step relocation, and self\-repair bind\)—is what the protocol of §[8](https://arxiv.org/html/2607.00871#S8)measures; §[9](https://arxiv.org/html/2607.00871#S9)reports the study that has concluded\.

## 6The Anytime\-Valid Statistical Core

Every controller monitors its own running statistics and may stop, accept, or restart at any round\. The shared danger is therefore the same throughout: a fixed\-sample test is invalid the moment a controller peeks at its own numbers\. The primitives below are the standard tools that stay valid under such continuous peeking; we lead each with what it buys the loop, then give the formula\.

#### Normal\-mixture confidence sequences—an error bar valid at every round\.

A confidence sequence is an interval that holds for*all*nnat once, so a controller can read it after every round without inflating its error rate\. Forσ\\sigma\-sub\-Gaussian increments with partial sumSnS\_\{n\}and intrinsic timeVn=n​σ2V\_\{n\}=n\\sigma^\{2\}, the mixture supermartingaleMn=ρ/\(ρ\+Vn\)​exp⁡\{Sn2/2​\(ρ\+Vn\)\}M\_\{n\}=\\sqrt\{\\rho/\(\\rho\+V\_\{n\}\)\}\\exp\\\{S\_\{n\}^\{2\}/2\(\\rho\+V\_\{n\}\)\\\}gives, through Ville’s inequality, the time\-uniform radius

\|Sn\|n≤1n​\(ρ\+Vn\)​\(2​log⁡2α\+log⁡ρ\+Vnρ\)simultaneously for all​n,\\frac\{\|S\_\{n\}\|\}\{n\}\\;\\leq\\;\\frac\{1\}\{n\}\\sqrt\{\(\\rho\+V\_\{n\}\)\\Big\(2\\log\\tfrac\{2\}\{\\alpha\}\+\\log\\tfrac\{\\rho\+V\_\{n\}\}\{\\rho\}\\Big\)\}\\qquad\\text\{simultaneously for all \}n,\(12\)withρ\\rhotuned to tighten the boundary near a target sample size\(Howardet al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib22)\)\. \(The2​log⁡\(2/α\)2\\log\(2/\\alpha\)is a conservative union of two one\-sided boundaries; the exact two\-sided boundary ofHowardet al\.\([2021](https://arxiv.org/html/2607.00871#bib.bib22), Eq\. 14\)carries2​log⁡\(1/α\)2\\log\(1/\\alpha\)\.\) This is the value evidenceAlg 4accumulates per policy version\.

#### Hoeffding e\-processes—a bankroll that detects drift\.

An e\-process is a wealth process that grows in expectation only when the null is false, so a controller can bet against “no drift” and call drift the moment the wealth crosses1/δ1/\\delta, safely at any stopping time\. Concretely,∏iexp⁡\(λi​Xi−λi2/8\)\\prod\_\{i\}\\exp\(\\lambda\_\{i\}X\_\{i\}\-\\lambda\_\{i\}^\{2\}/8\)with predictableλi\\lambda\_\{i\}is an e\-process and rejection atEt≥1/δE\_\{t\}\\geq 1/\\deltais anytime\-valid\(Ramdaset al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib12)\); the bet is a GROW\-style plug\-in that tiltsλ\\lambdatoward the observed mean excess using only past data, so it grows fast when drift is real\. This isAlg 2’s reward\-model drift gate\.

#### Horizon\-free confirm\-triggered harmonic spending \(CTHS\)—one error budget split over unboundedly many edits\.

To keep familywise error belowδ0\\delta\_\{0\}across an open\-ended stream of accepted edits, the per\-edit budgets must sum toδ0\\delta\_\{0\}\. The SGM scheduleδt=δ0/\(t​HB\)\\delta\_\{t\}=\\delta\_\{0\}/\(t\\,H\_\{B\}\),HB=∑i≤B1/iH\_\{B\}=\\sum\_\{i\\leq B\}1/i, achieves this but pre\-commits a finite horizonBB\(Wuet al\.,[2025](https://arxiv.org/html/2607.00871#bib.bib21)\), which a self\-evolving agent lacks\. A horizon\-free schedule must stay summable on its own: the obvious choiceδ0/\(2​k​log2⁡\(k\+1\)\)\\delta\_\{0\}/\(2k\\log^\{2\}\(k\+1\)\)in fact over\-spends \(∑k≈1\.69​δ0\\sum\_\{k\}\\approx 1\.69\\,\\delta\_\{0\}\), silently breaking validity, so thekk\-th confirmation instead spends

δk=δ0Z​k​log2⁡\(k\+1\),Z=∑j≥11j​log2⁡\(j\+1\)≈3\.39,\\delta\_\{k\}=\\frac\{\\delta\_\{0\}\}\{Z\\,k\\log^\{2\}\(k\+1\)\},\\qquad Z=\\sum\_\{j\\geq 1\}\\frac\{1\}\{j\\log^\{2\}\(j\+1\)\}\\approx 3\.39,\(13\)withZZevaluated by an analytic tail correction so that∑kδk=δ0\\sum\_\{k\}\\delta\_\{k\}=\\delta\_\{0\}exactly—familywise validity with no horizon\.

#### Parameter\-free coin betting with restarts—an optimizer with no learning rate\.

Alg 3must track a moving optimum without a tuned step size, so it casts each update as a betting game\. A per\-coordinate KT bettor\(Orabona and Pál,[2016](https://arxiv.org/html/2607.00871#bib.bib18)\)keeps wealthWt\\mathrm\{W\}\_\{t\}and betting fractionβt=1t​∑i<tci\\beta\_\{t\}=\\frac\{1\}\{t\}\\sum\_\{i<t\}c\_\{i\}\(clipped to±12\\pm\\tfrac\{1\}\{2\}\) with normalized reward directionci=−gi/Gic\_\{i\}=\-g\_\{i\}/G\_\{i\}\(GiG\_\{i\}a running Lipschitz estimate\), playingxt=x0\+βt​Wtx\_\{t\}=x\_\{0\}\+\\beta\_\{t\}\\mathrm\{W\}\_\{t\}; this gives comparator\-adaptiveO​\(∥u∥​T​log⁡T\)O\(\\lVert u\\rVert\\sqrt\{T\\log T\}\)regret with no learning rate to set\. Drift re\-anchorsx0x\_\{0\}at the current iterate—a restart surrogate for the strongly\-adaptive methods that attain path\-length dynamic regret on every interval\(Cutkosky,[2020](https://arxiv.org/html/2607.00871#bib.bib16)\)and the optimalO~​\(T1/3​VT2/3\)\\widetilde\{O\}\(T^\{1/3\}V\_\{T\}^\{2/3\}\)rate for strongly convex losses\(Baby and Wang,[2022](https://arxiv.org/html/2607.00871#bib.bib17)\)\.

#### Exact 1\-D Wasserstein and sensitivity estimation—how far an edit moved the distribution\.

The performative correction \(Alg 4\) needs the distance between the reward distributions before and after an edit; in one dimension thisW1W\_\{1\}is exact, computed by integrating the gap between empirical CDFs over the merged support\. The sensitivityε\\varepsilonis then the non\-negative through\-origin slope ofW1W\_\{1\}shifts against adapter\-norm deltas on probe pairs, and the contraction checkε^​L<1\\widehat\{\\varepsilon\}L<1is exposed to the controllers\.

#### Wild\-bootstrap trend test—is the baseline drifting under us?

Before trusting a logged value stream, the loop tests it for a trend by re\-signing the OLS residuals \(around a no\-trend fit\) with Rademacher±1\\pm 1multipliers\(Chandaket al\.,[2020](https://arxiv.org/html/2607.00871#bib.bib23)\); the test is robust to heteroskedasticity and heavy tails and is seeded for reproducibility\.Alg 3restarts andAlg 4widens its bound when it fires\.

#### Weighted importance sampling—reusing old rollouts\.

To value a new policy from rollouts logged under older ones, self\-normalized weighted IS with clipped log\-ratios and an effective\-sample\-size diagnostic reweights the history, and per\-trajectory IS\-weighted returns are shaped so a confidence sequence wraps them directly\.

#### MDL compression and QD archives\.

Finally,Alg 5draws on Stitch\-style antiunification compression \(§[4\.5](https://arxiv.org/html/2607.00871#S4.SS5)\) and a MAP\-Elites grid archive with coverage, best\-elite, and Pareto\-frontier queries\.

## 7Composite Two\-Timescale Control and the Self\-Evolving SWE Agent

#### Actor protocol\.

The controllers are written against a single\-deployment interface; an*actor*protocol lets the same controllers drive a multi\-step, tool\-using agent: the actor turns \(policy, task, attempt\) into a final action plus the full tool trajectory and the episode’s process reward, recording whichL1L\_\{1\}strategy directive was sampled \(controllers may force a fixed directive—Alg 4compares harnesses under directive0—or let the actor sample on\-policy, asAlg 3requires\)\. Every controller then works unchanged with the multi\-step agent, and the search operators of §[5](https://arxiv.org/html/2607.00871#S5)wrap any actor without modification\.

#### The SWE agent\.

The evolvable actor is a multi\-step, tool\-using agent \(ReAct\-style\(Yaoet al\.,[2023](https://arxiv.org/html/2607.00871#bib.bib35)\)\) over a structured action vocabulary \(Appendix[E](https://arxiv.org/html/2607.00871#A5)\):list,search,read,edit\(unique search\-and\-replace with a whitespace\-fuzzy fallback\),run\_tests,submit, executed against a working copy at the pre\-fix revision with capped observations\. Accumulated edits form the candidate patch, so it applies by construction\. It reads strategy guidance, the grown macro library, step budget, and exploration threshold from the harness, with the sampledL1L\_\{1\}directive composed into the system prompt; each attempt’s working copy and best\-result archive are keyed by \(policy version, directive, attempt\), soAlg 4’s concurrent baseline/candidate arms and best\-of\-NNattempts stay isolated\.

#### Composite controller \(Algorithm[11](https://arxiv.org/html/2607.00871#alg11)\)\.

A scheduler runs any subset of the five over*one shared evolving policy*: each sub\-controller is wrapped as \(name, controller, period\) and runs whentmodperiod=0t\\bmod\\text\{period\}=0; period0disables it—the ablation mechanism\. Before a sub\-controller steps, it receives the current shared policy; after, its \(possibly edited\) policy is propagated to the next\. Each algorithm edits only its own layer through operations that preserve the other layer, so the edits compose\. Sub\-certificates are merged into one ledger row: metrics are prefixed per algorithm, error spends are summed, and the round’s decision is the highest\-precedence sub\-decision \(accept≻\\succnsf≻\\succreject≻\\succhold\)\. Default periods realize the two\-timescale split:L1L\_\{1\}controllers \(Alg 3orAlg 1\) every round;L2L\_\{2\}controllers \(Alg 4,Alg 5\) every 2 rounds;Alg 2every 3\.Alg 1andAlg 3both trainL1L\_\{1\}, so enabling both together is discouraged\. An optional*cost\-aware*mode makes the scheduler itself a compute allocator: it tracks a per\-controller no\-gain counter, reset only on*committed*output \(an accepted edit, a shadow\-accepted patch, a committed macro, or mined anti\-macros—raw activity such as solve counts or uncommitted compression gain never trips it\), and temporarily skips a slowL2L\_\{2\}controller that has shown no committed gain for a few consecutive rounds\. A skipped controller is left one round short of the threshold, so a dead controller settles into a probe/skip alternation—a scheduler\-level echo ofAlg 4’s per\-search compute\-allocation stop\.

#### Division of labor\.

On the SWE agent the controllers specialize\.Alg 4proposes and gates harness edits \(force\-edit threshold, step budget, strategy guidance\) against the paired process reward \(Eq\.[10](https://arxiv.org/html/2607.00871#S5.E10)\) at CTHS levels, with the pre\-gate pilot, common\-random\-number pairing, and shadow requeue of §[4\.4](https://arxiv.org/html/2607.00871#S4.SS4)\.Alg 5mines recurring\(tool,target\)\(\\text\{tool\},\\text\{target\}\)sub\-sequences from high\-reward trajectories into macros \(and failure signatures into anti\-macros\), admitting a macro only if it clears a*downstream\-utility*gate—positive newer\-vs\-older reward lift on its context—not merely an MDL gain; the library is capped per \(repository, status\) context and self\-prunes by a trailing\-window retirement pass, so the certificate reports compression gain only for committed macros\.Alg 3performs counterfactual credit assignment over rollouts to steer the directive adapter \(directives such as*explore broadly*,*localize and edit early*,*smallest diff*\), weighting credit per active repository family and excludingAlg 4’s rejected\-candidate arms\.Alg 1protects the adapter with the forgetting gate;Alg 2anchors any preference model\. The cost\-aware skip \(above\) matters here because a scheduledAlg 4round costs∼\\sim90 vs\.∼\\sim7 min for the rest of the composite combined \(three concurrent multi\-step deploys\)\.

## 8Experimental Setup

#### Benchmark and grading\.

SWE\-bench Verified—the 500\-instance human\-validated subset\(Chowdhuryet al\.,[2024](https://arxiv.org/html/2607.00871#bib.bib27)\)of SWE\-bench\(Jiménezet al\.,[2024](https://arxiv.org/html/2607.00871#bib.bib34)\)—evaluated with the official execution\-based harness: for each instance the repository is restored to the state preceding the fix, the candidate patch is applied, and the project’s real test suite is run\.*Resolved*requires every fail\-to\-pass test to pass and every pass\-to\-pass test to keep passing\. The working sample is a seeded random draw of5252instances \(2424Django,2727Matplotlib,11Flask\)\.

#### Rewards\.

Three bounded rewards are available to the loop\.*Proxy*: file\-overlap F1 between predicted and gold patches \(inner\-loop development only, never for claims\)\.*Native*: real test execution, the fraction of fail\-to\-pass tests passing\.*Shaped*: the dense milestone reward of §[5](https://arxiv.org/html/2607.00871#S5)\. The signal live runs actually climb is the self\-authored verifier of §[5\.2](https://arxiv.org/html/2607.00871#S5.SS2); the gold tests are reserved for terminal grading\.

#### Base models and backends\.

The study crosses*four*frozen base models, accessed only asL0L\_\{0\}; none is fine\-tuned\. Two are open\-weight checkpoints served locally with their separate reasoning channel disabled for structured output:Gemma\(gemma4:31b\-mlx, a∼\\sim31B model atnvfp4quantization\) andQwen\(qwen3\.6:27b,∼\\sim27B\)\. Two are frontier reasoning models behind the same interface:Gpt\-mini\(gpt\-5\.4\-mini, low reasoning effort\) andGpt\(gpt\-5\.5, reasoning disabled—the strongest base in the set\), each with a44k\-token completion cap\. A fifth model,Glm5\.2, is run with the no\-op control and the full suite \(Table[3](https://arxiv.org/html/2607.00871#S9.T3)\); its single\-pass baseline was not run, so its “off” cell is the control\. Adapter log\-probabilities are exact on every backend because the steered policy is theL1L\_\{1\}directive layer\.

#### The4×24\\times 2: four models×\\timesalgorithms off/on\.

Each base model is run with the algorithm stack*off*and*on*, on all5252instances, under the identical harness and the same official execution\-based grader—eight cells, one grading protocol\.*No algorithms*is the bare multi\-step actor of §[7](https://arxiv.org/html/2607.00871#S7)alone—no controllers, no search, no in\-loop test runner—one episode per instance under a1010\-step budget\.*Algorithms–A*\(the full composite\) enables eight of the ten algorithms of Table[1](https://arxiv.org/html/2607.00871#S4.T1):Alg 1/Alg 2/Alg 3/Alg 5scheduled as certificate controllers and the verifier\-tier mechanismsAlg 7–Alg 10active—verified micro\-step search \(Alg 7\), the self\-authored oracles of §[5\.2](https://arxiv.org/html/2607.00871#S5.SS2)at two samples \(Alg 8\) with the unverified\-green cap and tie\-break judge, search\-layer step\-credit \(Alg 9\), and verified self\-repair \(Alg 10\)—withAlg 4disabled for wall\-clock cost \(§[7](https://arxiv.org/html/2607.00871#S7)\) and best\-of\-22\(Alg 6\) removed \(§[9\.1](https://arxiv.org/html/2607.00871#S9.SS1)\)\. Each instance is solved once \(re\-touches return the cached result\)\. The harness, sample, step budget, and grader are held fixed across all eight cells, so the only varying factors are the base model and the stack\.

## 9Results and Discussion

Two results stand out\. First, base capability is a large, confound\-free effect: single\-pass baselines on the identical harness scale cleanly—Gemma18<18<Qwen24<24<Gpt\-mini25<25<Gpt2828\(Table[3](https://arxiv.org/html/2607.00871#S9.T3)\)—and the ordering is preserved with the full stack \(22,25,29,3422,25,29,34\)\. Second, the full stack improves every base \(on−\-off\+1\+1to\+6\+6\), and where we deconfounded it with a no\-op control on two strong models the suite’s contribution is\+5\+5\(Gpt\) and\+4\+4\(Glm5\.2\)—attributable to the algorithms, not to scaffolding \(§[9\.1](https://arxiv.org/html/2607.00871#S9.SS1)\); the best configuration isGpt\+\+*Algorithms–A*at34/5234/52\(65%65\\%\)\.

Table 3:Model×\\timesstack on a fixed5252\-instance SWE\-bench Verified subset \(identical harness, official execution\-based grader; resolved counts\)\. The full stack improves every base\. For the first four models “off” is the single\-pass baseline, so the on−\-offΔ\\Deltaalso includes a small scaffolding/directive effect \(a no\-op control onGptisolates the algorithm contribution at\+5\+5, §[9\.1](https://arxiv.org/html/2607.00871#S9.SS1)\); forGlm5\.2 the “off” cell*is*that no\-op control \(its single\-pass baseline was not run\), so its\+4\+4is already scaffolding\-free\.⋆Glm5\.2 “off”==no\-op control \(scaffolding on, algorithms off\)\.

01020304050%1822\+4Gemma2425\+1Qwen2529\+4Gpt\-mini2834\+6Gpt2428\+4Glmresolved / 52off*Algorithms–A*Figure 2:Across models\.Resolved instances \(of5252\) per base model with the stack off \(grey\) and on \(green\); models ordered by baseline\. Absolute resolution scales with base capability in both conditions\. The on−\-off difference \(annotated\+N\+N\) includes a small scaffolding/directive effect, except forGlm5\.2 whose “off” bar is the no\-op control \(§[9\.1](https://arxiv.org/html/2607.00871#S9.SS1)\), making its\+4\+4already scaffolding\-free\.### 9\.1Deconfounded single\-base ablation

To separate the algorithms from the scaffolding they run inside, we ran a deliberate*no\-op composite*control onGpt—the full composite scaffolding with strategy directives on but every algorithm disabled\. It scores2929against the single\-pass baseline2828, so the directive/scaffolding effect is only\+1\+1; the full suite reaches3434, a\+5\+5gain over the proper control, attributable to the algorithms themselves\. A second strong model,Glm5\.2, gives the same picture \(Table[3](https://arxiv.org/html/2607.00871#S9.T3), Fig\.[2](https://arxiv.org/html/2607.00871#S9.F2)\):2424with the algorithms off and2828with them on, a\+4\+4gain\. The deconfounded contribution of the suite is thus positive on both models we controlled \(\+5\+5and\+4\+4\)—a single run per cell, but a consistent direction across two models\.

Decomposing theGptsuite into single\-algorithm runs \(Table[4](https://arxiv.org/html/2607.00871#S9.T4), Fig\.[3](https://arxiv.org/html/2607.00871#S9.F3)\) localizes the contribution\. Re\-anchored to the2929control, several algorithms contribute alone:Alg 2\+5\+5\(the strategy\-directive learner\),Alg 7/Alg 8\+3\+3\(micro\-step search, self\-oracles\),Alg 3\+2\+2; the full suite reaches the best single component, consistent with overlapping rather than additive gains\. Two honest reads:Alg 6\(best\-of\-22\) is net\-negative \(2626\) and, per the run logs, never produced a second attempt while adding patch\-apply failures, so we*remove*it from the live stack; andAlg 4’s3636is*not*an algorithm effect—its confidence\-sequence gate accepted zero edits, so it ran the control configuration and the count is a high draw, which the event\-log attribution \(not the raw count\) caught\. These are single\-run figures \(§[10](https://arxiv.org/html/2607.00871#S10)collects the limitations and next steps\)\.

Table 4:Gpt\(gpt\-5\.5\) single\-algorithm ablation, one run per cell, same5252instances and grader\. The no\-op composite \(directives on, all algorithms off\) is the deconfounded control; the full suite reaches3434,\+5\+5over it\.Alg 4’s3636is not an algorithm effect—its gate accepted0edits, so it ran the control configuration \(caught by the event log\)\.0102030control2929baseline28control29Alg 130Alg 234Alg 331Alg 436Alg 528Alg 626Alg 732Alg 832Alg 929Alg 1030full34resolved / 52Figure 3:Single\-base ablation \(Gpt,gpt\-5\.5\)\.Resolved instances for each single\-algorithm config, the no\-op composite*control*\(2929, dashed\), the single\-pass baseline \(2828\), and the full suite \(3434\); one run per cell \(Table[4](https://arxiv.org/html/2607.00871#S9.T4)\)\. Most configs sit near the control; the full suite andAlg 2are highest \(green\)\.Alg 4’s3636is an artifact—its gate accepted0edits, so it ran the control—andAlg 6\(best\-of\-22, removed\) is below the control \(both red\)\. Single runs on expensive evaluations; per\-cell deltas are not separated from run\-to\-run variance\.
### 9\.2What the mechanisms verifiably do

The variance\-immune evidence is in the event logs \(decisions, accepts, oracle admissions, veto and react/refine firings\), which pass\-count noise does not touch\. Log analysis suggests the measurable lever inside the stack is*more closed\-loop debugging*, driven by the self\-oracle \(Alg 8\) and reflected in the full suite: relative to the control,run\_testscalls rise by∼\\sim50% and mean episode length by∼\\sim1\.3 steps—the agent iterates against its self\-authored tests rather than guessing once\. Among the certificate controllers, the ones that fire every round are the directive learnersAlg 2andAlg 3\(5252policy updates each;Alg 2is the top single\-algorithm cell at3434\); the gated, library, and forgetting controllersAlg 4,Alg 5,Alg 1contribute less\. On this evidence the suite’s live value is carried byAlg 2/Alg 3\(directive shaping\) andAlg 7/Alg 8\(verified search and self\-oracle\)\. Consistent with selection\-not\-creation, the stack does not destabilize a base—added regressions are few \(full suite44p2p\-flagged instances vs\.33for the control\)—and a consensus\-flip analysis separates signal from noise: a robust subset ofGptinstances flips under several distinct configs while single\-config flips are noise\-like;17/5217/52are solved by every config and1010by none\.

Position\.Two levers move the metric, and they are not interchangeable\. Within a fixed base the lever is the algorithm suite, whose deconfounded gain is carried byAlg 2/Alg 3\(directive shaping\) andAlg 7/Alg 8\(verified search and self\-oracle\)\. For*absolute*resolution the lever is a stronger base: the1010instances solved by no configuration—mostly Matplotlib—are a capability/harness wall that more search does not move\.

## 10Limitations

The endogenous\-loop guarantees remain open conjectures: each controller’s statistical primitives are individually published and sound, but their compositions—the two\-timescale coupling inAlg 2, the backward\-transfer reduction inAlg 1, the performative corrections inAlg 1andAlg 4—are not proven here, and several rest on assumptions the endogenous loop itself erodes \(a stationary held\-out real set forAlg 2; a learnable fully\-predictive augmented outcome and bounded comparator path variation forAlg 3; exchangeable task generation forAlg 5\)\. The performative sensitivityε\\varepsilonis a behavioral constant taken as a hyperparameter; it is not estimable online with its own validity guarantee, and the contraction conditionε​L<1\\varepsilon L<1may fail at LLM scale\. The anytime\-validity that makes the gates sound also makes them conservative: PAC\-Bayes terms grow vacuous for high\-dimensional posteriors, the DV forgetting floor rises withtt, andAlg 4guarantees safety, not progress\. A boundary we state up front: controllers and verifier\-guided search select, gate, and reshape the behavior of the frozen base; they cannot manufacture capability it lacks\. This bounds the absolute*ceiling*, which the base sets; within that ceiling the suite’s gains \(§[9\.1](https://arxiv.org/html/2607.00871#S9.SS1), §[9\.2](https://arxiv.org/html/2607.00871#S9.SS2)\) are real but measured as single runs on expensive evaluations, so we report magnitudes rather than significance\. The self\-oracle is fallible by design—it can under\-claim \(an over\-strict oracle fails to flip a correct patch\) and over\-claim \(a false\-green pass\)—so a patch is reported as “promising,” never “resolved,” the terminal grader being authoritative\. A proper ablation across diverse harnesses and base models is also outstanding and, we think, the most informative next step: with one harness and a single sample per cell we can say the suite as a whole helps, but not which algorithm is advantageous for which*type*of problem—the matching of controllers to problem structure \(stochastic vs\. systematic failures, short vs\. long horizons, repository families\) can only be settled by a factorial study that varies harness and model together\. Finally, each cell is a single run \(no per\-controller multi\-seed isolation\) and the slow\-loop distillation is designed but not trained; repeated runs, a stronger\-base confirmation, the per\-task algorithm–problem ablation, and the distillation step are the remaining experiments\.

## 11Conclusion

SEA confines self\-evolution of an LLM agent to a frozen base plus a small steering adapter and a versioned harness, and admits each self\-modification through an anytime\-valid gate that certifies it against a budgeted error ledger\. Five controllers compose published guarantees, and five verifier\-in\-the\-loop mechanisms—including a self\-authored reproduction\-oracle verifier computed from the issue alone—supply the dense, grader\-free signal those gates need\. On a5252\-instance SWE\-bench Verified subset across four bases, base capability is the dominant, confound\-free effect, and on two strong base models a deliberate no\-op\-composite control isolates the suite’s contribution at\+5\+5and\+4\+4\(Gpt29→3429\\to 34,65%65\\%;Glm5\.224→2824\\to 28\), with the event logs verifying that its mechanisms fire and prevent regressions\. Because these single\-run evaluations are expensive, confirming run\-to\-run variance and adapting the per\-task algorithm mix—then the distillation step that turns verified traces into weights—are the outstanding work\.

## References

- M\. Abbana Bennani, T\. Doan, and M\. Sugiyama \(2020\)Generalisation guarantees for continual learning with orthogonal gradient descent\.In4th Lifelong Machine Learning Workshop at ICML,Note:arXiv:2006\.11942Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p3.3)\.
- D\. Baby and Y\. Wang \(2022\)Optimal dynamic regret in proper online learning with strongly convex losses and beyond\.InProceedings of the 25th International Conference on Artificial Intelligence and Statistics \(AISTATS\),PMLR, Vol\.151,pp\. 1805–1845\.Note:arXiv:2201\.08905Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p2.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p2.13),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p3.5),[§6](https://arxiv.org/html/2607.00871#S6.SS0.SSS0.Px4.p1.9)\.
- V\. S\. Borkar \(2008\)Stochastic approximation: a dynamical systems viewpoint\.Cambridge University Press\.Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p2.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p2.3)\.
- M\. Bowers, T\. X\. Olausson, L\. Wong, G\. Grand, J\. B\. Tenenbaum, K\. Ellis, and A\. Solar\-Lezama \(2023\)Top\-down synthesis for library learning\.Proceedings of the ACM on Programming Languages7\(POPL\),pp\. 41:1–41:32\.Note:arXiv:2211\.16605Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p1.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p1.1),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p2.11),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p3.2)\.
- B\. Brown, J\. Juravsky, R\. Ehrlich, R\. Clark, Q\. V\. Le, C\. Ré, and A\. Mirhoseini \(2024\)Large language monkeys: scaling inference compute with repeated sampling\.arXiv preprint arXiv:2407\.21787\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px4.p1.2),[§5](https://arxiv.org/html/2607.00871#S5.SS0.SSS0.Px4.p1.7),[§5](https://arxiv.org/html/2607.00871#S5.p1.1)\.
- Y\. Chandak, S\. M\. Jordan, G\. Theocharous, M\. White, and P\. S\. Thomas \(2020\)Towards safe policy improvement for non\-stationary MDPs\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2010\.12645Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2607.00871#S4.SS4.p2.5),[§4\.4](https://arxiv.org/html/2607.00871#S4.SS4.p3.1),[§6](https://arxiv.org/html/2607.00871#S6.SS0.SSS0.Px6.p1.1),[14](https://arxiv.org/html/2607.00871#alg4.l14.1)\.
- B\. Chen, F\. Zhang, A\. Nguyen, D\. Zan, Z\. Lin, J\. Lou, and W\. Chen \(2023\)CodeT: code generation with generated tests\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2207\.10397Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px4.p1.2)\.
- X\. Chen, M\. Lin, N\. Schärli, and D\. Zhou \(2024\)Teaching large language models to self\-debug\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2304\.05128Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px4.p1.2)\.
- N\. Chowdhury, J\. Aung, C\. J\. Shern, O\. Jaffe, D\. Sherburn, G\. Starace, E\. Mays, R\. Dias, M\. Aljubeh, M\. Glaese, C\. E\. Jimenez, J\. Yang, K\. Liu, and A\. Madry \(2024\)Introducing SWE\-bench Verified\.Note:OpenAIExternal Links:[Link](https://openai.com/index/introducing-swe-bench-verified/)Cited by:[§8](https://arxiv.org/html/2607.00871#S8.SS0.SSS0.Px1.p1.4)\.
- B\. Chugg, H\. Wang, and A\. Ramdas \(2023\)A unified recipe for deriving \(time\-uniform\) PAC\-Bayes bounds\.Journal of Machine Learning Research24\(372\),pp\. 1–61\.Note:arXiv:2302\.03421Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p1.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p2.20),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p3.3)\.
- A\. Cully and Y\. Demiris \(2018\)Quality and diversity optimization: a unifying modular framework\.IEEE Transactions on Evolutionary Computation22\(2\),pp\. 245–259\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p2.11),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p3.2)\.
- A\. Cutkosky \(2020\)Parameter\-free, dynamic, and strongly\-adaptive online learning\.InProceedings of the 37th International Conference on Machine Learning \(ICML\),PMLR, Vol\.119\.Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p2.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p2.13),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p3.5),[§6](https://arxiv.org/html/2607.00871#S6.SS0.SSS0.Px4.p1.9)\.
- K\. Ellis, C\. Wong, M\. Nye, M\. Sablé\-Meyer, L\. Morales, L\. Hewitt, L\. Cary, A\. Solar\-Lezama, and J\. B\. Tenenbaum \(2021\)DreamCoder: bootstrapping inductive program synthesis with wake\-sleep library learning\.InProceedings of the 42nd ACM SIGPLAN Conference on Programming Language Design and Implementation \(PLDI\),Note:arXiv:2006\.08381Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p1.1),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p2.11)\.
- M\. Farajtabar, N\. Azizan, A\. Mott, and A\. Li \(2020\)Orthogonal gradient descent for continual learning\.InProceedings of the 23rd International Conference on Artificial Intelligence and Statistics \(AISTATS\),PMLR, Vol\.108,pp\. 3762–3773\.Note:arXiv:1910\.07104Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p1.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p1.1)\.
- L\. Friedman and R\. Meir \(2025\)Data\-dependent and oracle bounds on forgetting in continual learning\.InConference on Lifelong Learning Agents \(CoLLAs\),Note:arXiv:2406\.09370Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p3.3)\.
- S\. Fu, Y\. Wang, Y\. Chen, X\. Tian, and D\. Tao \(2025\)A theoretical perspective: how to prevent model collapse in self\-consuming training loops\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2502\.18865Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p1.1),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p3.2)\.
- L\. Gao, J\. Schulman, and J\. Hilton \(2023\)Scaling laws for reward model overoptimization\.InProceedings of the 40th International Conference on Machine Learning \(ICML\),PMLR, Vol\.202,pp\. 10835–10866\.Note:arXiv:2210\.10760Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p1.1)\.
- M\. Gerstgrasser, R\. Schaeffer, A\. Dey, R\. Rafailov,et al\.\(2024\)Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data\.InConference on Language Modeling \(COLM\),Note:arXiv:2404\.01413Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3)\.
- C\. Gulcehre, T\. L\. Paine, S\. Srinivasan,et al\.\(2023\)Reinforced self\-training \(ReST\) for language modeling\.arXiv preprint arXiv:2308\.08998\.Cited by:[item 3](https://arxiv.org/html/2607.00871#S1.I1.i3.p1.1),[§5\.3](https://arxiv.org/html/2607.00871#S5.SS3.p1.1)\.
- A\. Harutyunyan, W\. Dabney, T\. Mesnard, M\. G\. Azar, B\. Piot, N\. Heess, H\. van Hasselt, G\. Wayne, S\. Singh, D\. Precup, and R\. Munos \(2019\)Hindsight credit assignment\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3)\.
- S\. R\. Howard, A\. Ramdas, J\. McAuliffe, and J\. Sekhon \(2021\)Time\-uniform, nonparametric, nonasymptotic confidence sequences\.The Annals of Statistics49\(2\),pp\. 1055–1080\.Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p2.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2607.00871#S4.SS4.p2.5),[§4\.4](https://arxiv.org/html/2607.00871#S4.SS4.p3.1),[§6](https://arxiv.org/html/2607.00871#S6.SS0.SSS0.Px1.p1.8),[11](https://arxiv.org/html/2607.00871#alg4.l11)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2106\.09685Cited by:[itemL1L\_\{1\}— Adapter\.](https://arxiv.org/html/2607.00871#S3.I1.ix2.p1.4)\.
- C\. E\. Jiménez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. Narasimhan \(2024\)SWE\-bench: can language models resolve real\-world GitHub issues?\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.06770Cited by:[§8](https://arxiv.org/html/2607.00871#S8.SS0.SSS0.Px1.p1.4)\.
- B\. Lester, R\. Al\-Rfou, and N\. Constant \(2021\)The power of scale for parameter\-efficient prompt tuning\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Note:arXiv:2104\.08691Cited by:[itemL1L\_\{1\}— Adapter\.](https://arxiv.org/html/2607.00871#S3.I1.ix2.p1.4)\.
- X\. L\. Li and P\. Liang \(2021\)Prefix\-tuning: optimizing continuous prompts for generation\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics \(ACL\),Note:arXiv:2101\.00190Cited by:[itemL1L\_\{1\}— Adapter\.](https://arxiv.org/html/2607.00871#S3.I1.ix2.p1.4)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px4.p1.2)\.
- D\. Mandal, S\. Triantafyllou, and G\. Radanovic \(2023\)Performative reinforcement learning\.InProceedings of the 40th International Conference on Machine Learning \(ICML\),PMLR, Vol\.202,pp\. 23642–23680\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px1.p1.7),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p1.2),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p3.5)\.
- A\. Meulemans, S\. Schug, S\. Kobayashi, N\. Daw, and G\. Wayne \(2023\)Would I have gotten that reward? long\-term credit assignment by counterfactual contribution analysis\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p1.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p1.2),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p3.5)\.
- J\. Mouret and J\. Clune \(2015\)Illuminating search spaces by mapping elites\.arXiv preprint arXiv:1504\.04909\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p2.11)\.
- F\. Orabona and D\. Pál \(2016\)Coin betting and parameter\-free online learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.3](https://arxiv.org/html/2607.00871#S4.SS3.p2.13),[§6](https://arxiv.org/html/2607.00871#S6.SS0.SSS0.Px4.p1.9)\.
- A\. Pentina and C\. H\. Lampert \(2014\)A PAC\-Bayesian bound for lifelong learning\.InProceedings of the 31st International Conference on Machine Learning \(ICML\),PMLR, Vol\.32,pp\. 991–999\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p2.11),[§4\.5](https://arxiv.org/html/2607.00871#S4.SS5.p3.2)\.
- J\. C\. Perdomo, T\. Zrnic, C\. Mendler\-Dünner, and M\. Hardt \(2020\)Performative prediction\.InProceedings of the 37th International Conference on Machine Learning \(ICML\),PMLR, Vol\.119,pp\. 7599–7609\.Note:arXiv:2002\.06673Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p1.1),[§1](https://arxiv.org/html/2607.00871#S1.p2.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px1.p1.7),[§3](https://arxiv.org/html/2607.00871#S3.p2.1),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p2.20),[§4\.1](https://arxiv.org/html/2607.00871#S4.SS1.p3.3),[§4](https://arxiv.org/html/2607.00871#S4.p1.11)\.
- A\. Ramdas, P\. Grünwald, V\. Vovk, and G\. Shafer \(2023\)Game\-theoretic statistics and safe anytime\-valid inference\.Statistical Science38\(4\),pp\. 576–601\.Note:arXiv:2210\.01948Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p2.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p3.2),[§4\.4](https://arxiv.org/html/2607.00871#S4.SS4.p3.1),[§6](https://arxiv.org/html/2607.00871#S6.SS0.SSS0.Px2.p1.5)\.
- J\. Schmidhuber \(2003\)Gödel machines: self\-referential universal problem solvers making provably optimal self\-improvements\.arXiv preprint cs/0309048\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px2.p1.1)\.
- N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px4.p1.2)\.
- I\. Shumailov, Z\. Shumaylov, Y\. Zhao, N\. Papernot, R\. Anderson, and Y\. Gal \(2024\)AI models collapse when trained on recursively generated data\.Nature631\(8022\),pp\. 755–759\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p1.1)\.
- P\. S\. Thomas, B\. Castro da Silva, A\. G\. Barto, S\. Giguere, Y\. Brun, and E\. Brunskill \(2019\)Preventing undesirable behavior of intelligent machines\.Science366\(6468\),pp\. 999–1004\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2607.00871#S4.SS4.p3.1),[§4](https://arxiv.org/html/2607.00871#S4.p1.11)\.
- P\. S\. Thomas, G\. Theocharous, and M\. Ghavamzadeh \(2015\)High confidence policy improvement\.InProceedings of the 32nd International Conference on Machine Learning \(ICML\),PMLR, Vol\.37,pp\. 2380–2388\.Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p1.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2607.00871#S4.SS4.p3.1),[§4](https://arxiv.org/html/2607.00871#S4.p1.11)\.
- D\. Tiapkin, D\. Calandriello, D\. Belomestny, E\. Moulines, A\. Naumov, K\. Rasul, M\. Valko, and P\. Ménard \(2025\)Proximal point nash learning from human feedback\.arXiv preprint arXiv:2505\.19731\.Note:v1 titled “Accelerating Nash Learning from Human Feedback via Mirror Prox \(Nash\-MP\)”Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p1.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p1.1),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p3.2)\.
- J\. Ville \(1939\)Étude critique de la notion de collectif\.Gauthier\-Villars, Paris\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Wang, C\. Ma, Q\. Chen, L\. Meng, Y\. Han, J\. Xiao, Z\. Zhang, J\. Huo, W\. J\. Su, and Y\. Yang \(2025\)Magnetic preference optimization: achieving last\-iterate convergence for language model alignment\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2410\.16714Cited by:[§1](https://arxiv.org/html/2607.00871#S1.p1.1),[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px3.p1.3),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p1.1)\.
- X\. Wu, S\. Yin, Y\. Kang, X\. Zhang, Q\. Xu, Z\. Chen, and W\. Zhang \(2025\)SGM: a statistical Gödel machine for risk\-controlled recursive self\-modification\.arXiv preprint arXiv:2510\.10232\.Cited by:[§2](https://arxiv.org/html/2607.00871#S2.SS0.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2607.00871#S4.SS2.p3.2),[§4\.4](https://arxiv.org/html/2607.00871#S4.SS4.p3.1),[§6](https://arxiv.org/html/2607.00871#S6.SS0.SSS0.Px3.p1.8)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§7](https://arxiv.org/html/2607.00871#S7.SS0.SSS0.Px2.p1.2)\.
- E\. Zelikman, Y\. Wu, J\. Mu, and N\. D\. Goodman \(2022\)STaR: bootstrapping reasoning with reasoning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2203\.14465Cited by:[item 3](https://arxiv.org/html/2607.00871#S1.I1.i3.p1.1),[§5\.3](https://arxiv.org/html/2607.00871#S5.SS3.p1.1)\.

## Appendix AAlgorithm Pseudocode

This appendix collects the pseudocode for every algorithm in Table[1](https://arxiv.org/html/2607.00871#S4.T1); each box is referenced from the main text\.

Algorithm 1PPB\-CL: forgetting\-gated, trust\-regioned continual adapter learning1:initfrozen prior

Q0=𝒩​\(θ0,diag​ev\)Q\_\{0\}=\\mathcal\{N\}\(\\theta\_\{0\},\\mathrm\{diag\}\\,e^\{v\}\); update\-direction buffer

B←∅B\\leftarrow\\emptyset\(FIFO, cap

mm\); anchors

𝒜←∅\\mathcal\{A\}\\leftarrow\\emptyset\(cap

AA, each task with best historical reward\)

2:for

t=1,2,…t=1,2,\\dotsdo

3:rollouts

←Deploy​\(πt−1,nt​tasks\)\\leftarrow\\textsc\{Deploy\}\(\\pi\_\{t\-1\},n\_\{t\}\\text\{ tasks\}\); record rewards

RiR\_\{i\}, directive indices

aia\_\{i\}⊳\\trianglerightSt∼𝒟​\(πt−1\)S\_\{t\}\\sim\\mathcal\{D\}\(\\pi\_\{t\-1\}\)

4:

g^←1nt​∑i\(ℓi−ℓ¯pool\)​∇θlog⁡pθ​\(ai\)\\hat\{g\}\\leftarrow\\frac\{1\}\{n\_\{t\}\}\\sum\_\{i\}\\big\(\\ell\_\{i\}\-\\bar\{\\ell\}\_\{\\mathrm\{pool\}\}\\big\)\\,\\nabla\_\{\\theta\}\\log p\_\{\\theta\}\(a\_\{i\}\),

ℓi=1−Ri\\ell\_\{i\}=1\-R\_\{i\}⊳\\trianglerightbaseline pooled over the recent replay window

5:

g^⟂←g^−∑J∈B⟨g^,J⟩⟨J,J⟩​J\\hat\{g\}\_\{\\perp\}\\leftarrow\\hat\{g\}\-\\sum\_\{J\\in B\}\\frac\{\\langle\\hat\{g\},J\\rangle\}\{\\langle J,J\\rangle\}J⊳\\trianglerightOGD projection off stored directions

6:

θcand←θt−1−η​g^⟂\\theta\_\{\\mathrm\{cand\}\}\\leftarrow\\theta\_\{t\-1\}\-\\eta\\,\\hat\{g\}\_\{\\perp\};

Bfgt←DVBound​\(θcand,t\)B\_\{\\mathrm\{fgt\}\}\\leftarrow\\textsc\{DVBound\}\(\\theta\_\{\\mathrm\{cand\}\},t\)⊳\\trianglerightEq\. \([3](https://arxiv.org/html/2607.00871#S4.E3)\)

7:

d←0d\\leftarrow 0
8:while

Bfgt\>τforgetB\_\{\\mathrm\{fgt\}\}\>\\tau\_\{\\mathrm\{forget\}\}and

d<5d<5do⊳\\trianglerightdamped re\-test

9:

θcand←θt−1\+ρ​\(θcand−θt−1\)\\theta\_\{\\mathrm\{cand\}\}\\leftarrow\\theta\_\{t\-1\}\+\\rho\\,\(\\theta\_\{\\mathrm\{cand\}\}\-\\theta\_\{t\-1\}\);

Bfgt←DVBound​\(θcand,t\)B\_\{\\mathrm\{fgt\}\}\\leftarrow\\textsc\{DVBound\}\(\\theta\_\{\\mathrm\{cand\}\},t\);

d←d\+1d\\leftarrow d\+1
10:endwhile

11:

r←max⁡\(τforget−Bfgt,0\)/εr\\leftarrow\\max\(\\tau\_\{\\mathrm\{forget\}\}\-B\_\{\\mathrm\{fgt\}\},\\,0\)/\\varepsilon⊳\\trianglerightperformative trust region

12:if

∥θcand−θt−1∥2\>r\\lVert\\theta\_\{\\mathrm\{cand\}\}\-\\theta\_\{t\-1\}\\rVert\_\{2\}\>rthen

13:

θcand←Πℬ​\(θt−1,r\)​\(θcand\)\\theta\_\{\\mathrm\{cand\}\}\\leftarrow\\Pi\_\{\\mathcal\{B\}\(\\theta\_\{t\-1\},r\)\}\(\\theta\_\{\\mathrm\{cand\}\}\); recompute

BfgtB\_\{\\mathrm\{fgt\}\}and

rr
14:endif

15:if

Bfgt≤τforgetB\_\{\\mathrm\{fgt\}\}\\leq\\tau\_\{\\mathrm\{forget\}\}and

ε​L<1\\varepsilon L<1then⊳\\trianglerightaccept

16:

θt←θcand\\theta\_\{t\}\\leftarrow\\theta\_\{\\mathrm\{cand\}\};

B←\(B∪\{g^⟂\}\)\[−m:\]B\\leftarrow\(B\\cup\\\{\\hat\{g\}\_\{\\perp\}\\\}\)\[\-m\{:\}\]; update

𝒜\\mathcal\{A\}with this round’s tasks/best rewards

17:else

θt←θt−1\\theta\_\{t\}\\leftarrow\\theta\_\{t\-1\}⊳\\trianglerighthold

18:endif

19:emit certificate

\(t,Bfgt,r,KL​\(Qθt∥Q0\),PAC\-Bayes penalty,d\)\(t,\\,B\_\{\\mathrm\{fgt\}\},\\,r,\\,\\mathrm\{KL\}\(Q\_\{\\theta\_\{t\}\}\\\|Q\_\{0\}\),\\,\\text\{PAC\-Bayes penalty\},\\,d\)
20:endfor

21:

22:functionDVBound\(

θ\\theta,

tt\)⊳\\trianglerightanytime\-valid backward transfer, Eq\. \([3](https://arxiv.org/html/2607.00871#S4.E3)\)

23:deploy

π​\(θ\)\\pi\(\\theta\)on anchor tasks;

bt^←1\|𝒜\|​∑j\[\(1−Rj\)−\(1−Rj∗\)\]\\widehat\{\\mathrm\{bt\}\}\\leftarrow\\frac\{1\}\{\|\\mathcal\{A\}\|\}\\sum\_\{j\}\\big\[\(1\-R\_\{j\}\)\-\(1\-R^\{\*\}\_\{j\}\)\\big\]
24:return

bt^\+At/λ∗\+λ∗/\(8​\|𝒜\|\)\\widehat\{\\mathrm\{bt\}\}\+A\_\{t\}/\\lambda^\{\*\}\+\\lambda^\{\*\}/\(8\|\\mathcal\{A\}\|\)⊳\\trianglerightAt=KL\+log⁡\(2​t/δ\)A\_\{t\}=\\mathrm\{KL\}\+\\log\(2\\sqrt\{t\}/\\delta\);λ∗=8​\|𝒜\|​At\\lambda^\{\*\}=\\sqrt\{8\|\\mathcal\{A\}\|A\_\{t\}\}, the DV\-optimal temperature

25:endfunction

Algorithm 2PNMP\-A: two\-timescale anchored preference learning with an e\-value drift gate1:init

q←𝟎q\\leftarrow\\mathbf\{0\}; frozen anchor

qrealq\_\{\\mathrm\{real\}\}; magnet

z←θ0z\\leftarrow\\theta\_\{0\}; reference

log⁡πref←log⁡softmax​\(θ0\)\\log\\pi\_\{\\mathrm\{ref\}\}\\leftarrow\\log\\mathrm\{softmax\}\(\\theta\_\{0\}\); CTHS budget\(

δ0\\delta\_\{0\}\); rates

atslow=12​\(t\+1\)−1a^\{\\mathrm\{slow\}\}\_\{t\}=\\tfrac\{1\}\{2\}\(t\{\+\}1\)^\{\-1\},

atfast=12​\(t\+1\)−0\.7a^\{\\mathrm\{fast\}\}\_\{t\}=\\tfrac\{1\}\{2\}\(t\{\+\}1\)^\{\-0\.7\}
2:for

t=1,2,…t=1,2,\\dotsdo

3:rollouts

←Deploy​\(πt−1\)\\leftarrow\\textsc\{Deploy\}\(\\pi\_\{t\-1\}\); pairs

←\\leftarrow\{

\(i,j,winner\)\(i,j,\\text\{winner\}\): winner has higher mean reward over the recent replay window\}

4:ifnotbudget\.exhaustedthen⊳\\trianglerightslow: anchored preference update

5:

gsynth←∇qℒ^BT​\(q;pairs\)g\_\{\\mathrm\{synth\}\}\\leftarrow\\nabla\_\{q\}\\,\\widehat\{\\mathcal\{L\}\}\_\{\\mathrm\{BT\}\}\(q;\\text\{pairs\}\);

qcand←q\+atslow​\[α​\(qreal−q\)\+\(1−α\)​gsynth\]q\_\{\\mathrm\{cand\}\}\\leftarrow q\+a^\{\\mathrm\{slow\}\}\_\{t\}\\big\[\\alpha\\,\(q\_\{\\mathrm\{real\}\}\-q\)\+\(1\-\\alpha\)\\,g\_\{\\mathrm\{synth\}\}\\big\]
6:

X←\{\|σ​\(qicand−qjcand\)−σ​\(qireal−qjreal\)\|\}i<jX\\leftarrow\\big\\\{\\,\\lvert\\sigma\(q^\{\\mathrm\{cand\}\}\_\{i\}\{\-\}q^\{\\mathrm\{cand\}\}\_\{j\}\)\-\\sigma\(q^\{\\mathrm\{real\}\}\_\{i\}\{\-\}q^\{\\mathrm\{real\}\}\_\{j\}\)\\rvert\\,\\big\\\}\_\{i<j\}⊳\\trianglerightper\-pair drift vs\. frozen anchor

7:

E←EProcess\(X;H0:𝔼\[X\]≤τ\)E\\leftarrow\\textsc\{EProcess\}\(X;\\,H\_\{0\}\\\!:\\mathbb\{E\}\[X\]\\leq\\tau\)
8:

k←k\+1k\\leftarrow k\+1;

δk←δ0/\(Z​k​log2⁡\(k\+1\)\)\\delta\_\{k\}\\leftarrow\\delta\_\{0\}/\\big\(Z\\,k\\log^\{2\}\(k\{\+\}1\)\\big\)⊳\\trianglerightnormalized CTHS spend

9:if

E≥1/δkE\\geq 1/\\delta\_\{k\}thenreject: keep

qq⊳\\trianglerightanytime\-valid evidence of unsafe drift

10:else

q←qcandq\\leftarrow q\_\{\\mathrm\{cand\}\}⊳\\trianglerightaccept

11:endif

12:endif

13:

advi​\(p\)←∑jpj​σ​\(qi−qj\)\\mathrm\{adv\}\_\{i\}\(p\)\\leftarrow\\sum\_\{j\}p\_\{j\}\\,\\sigma\(q\_\{i\}\-q\_\{j\}\);

prox​\(g\)←softmax​\(log⁡pt−1\+η​g\+β​log⁡πref1\+β\)\\mathrm\{prox\}\(g\)\\leftarrow\\mathrm\{softmax\}\\\!\\big\(\\tfrac\{\\log p\_\{t\-1\}\+\\eta g\+\\beta\\log\\pi\_\{\\mathrm\{ref\}\}\}\{1\+\\beta\}\\big\)⊳\\trianglerightfast

14:

θ1/2←log⁡prox​\(adv​\(softmax​\(θt−1\)\)\)\\theta\_\{1/2\}\\leftarrow\\log\\mathrm\{prox\}\\big\(\\mathrm\{adv\}\(\\mathrm\{softmax\}\(\\theta\_\{t\-1\}\)\)\\big\);

θMP←log⁡prox​\(adv​\(softmax​\(θ1/2\)\)\)\\theta\_\{\\mathrm\{MP\}\}\\leftarrow\\log\\mathrm\{prox\}\\big\(\\mathrm\{adv\}\(\\mathrm\{softmax\}\(\\theta\_\{1/2\}\)\)\\big\)
15:

θt←θt−1\+atfast​\(θMP−θt−1\)\\theta\_\{t\}\\leftarrow\\theta\_\{t\-1\}\+a^\{\\mathrm\{fast\}\}\_\{t\}\\,\(\\theta\_\{\\mathrm\{MP\}\}\-\\theta\_\{t\-1\}\)⊳\\trianglerightextragradient step

16:if

tmodK=0t\\bmod K=0then

z←θtz\\leftarrow\\theta\_\{t\}⊳\\trianglerightmagnet refresh

17:endif

18:emit certificate

\(t,accept/hold,δk,E,KL​\(πt∥z\),pref\. drift\)\(t,\\,\\textsc\{accept\}/\\textsc\{hold\},\\,\\delta\_\{k\},\\,E,\\,\\mathrm\{KL\}\(\\pi\_\{t\}\\\|z\),\\,\\text\{pref\.\\ drift\}\)
19:endfor

Algorithm 3PA\-COCOA: counterfactual credit assignment driving a coin\-betting oracle1:initoracle

𝒪←\\mathcal\{O\}\\leftarrowper\-coordinate KT coin betting \(dim

kk\);

θ0←𝒪\.Predict​\(\)\\theta\_\{0\}\\leftarrow\\mathcal\{O\}\.\\textsc\{Predict\}\(\); reward stream

V←\[\]V\\leftarrow\[\\,\];

VT←0V\_\{T\}\\leftarrow 0
2:for

t=1,2,…t=1,2,\\dotsdo

3:rollouts

←Deploy​\(πt\)\\leftarrow\\textsc\{Deploy\}\(\\pi\_\{t\}\)⊳\\trianglerightactor samples directivea∼softmax​\(θt\)a\\sim\\mathrm\{softmax\}\(\\theta\_\{t\}\)per task

4:append masked mean process reward to

VV\(environment\-errored rollouts zeroed\); extend replay buffer⊳\\trianglerightaugmented outcome: rollouts carry\(a,πt\)\(a,\\pi\_\{t\}\)

5:foreach directive

a∈\{1\.\.k\}a\\in\\\{1\.\.k\\\}do⊳\\trianglerightcontribution model, off\-policy on replay

6:

w^​\(a\)←∑r∈recent​\(4​h\):ar=aωr​2−age​\(r\)/h​R~r∑r:ar=aωr​2−age​\(r\)/h\\widehat\{w\}\(a\)\\leftarrow\\dfrac\{\\sum\_\{r\\in\\mathrm\{recent\}\(4h\):\\,a\_\{r\}=a\}\\omega\_\{r\}\\,2^\{\-\\mathrm\{age\}\(r\)/h\}\\,\\tilde\{R\}\_\{r\}\}\{\\sum\_\{r:\\,a\_\{r\}=a\}\\omega\_\{r\}\\,2^\{\-\\mathrm\{age\}\(r\)/h\}\}\(default

12\\tfrac\{1\}\{2\}if no data\)⊳\\trianglerightR~\\tilde\{R\}: process reward, Eq\. \([10](https://arxiv.org/html/2607.00871#S5.E10)\);ωr\\omega\_\{r\}folds in the env\-error mask and repo\-family weight, and rejected\-candidate rollouts are skipped

7:endfor

8:

p←softmax​\(θt\)p\\leftarrow\\mathrm\{softmax\}\(\\theta\_\{t\}\);

g←\(w^⊙p\)−p​⟨w^,p⟩\+φ​\(𝟏/k−p\)g\\leftarrow\(\\widehat\{w\}\\odot p\)\-p\\,\\langle\\widehat\{w\},p\\rangle\+\\varphi\\,\(\\mathbf\{1\}/k\-p\)⊳\\trianglerightEq\. \([6](https://arxiv.org/html/2607.00871#S4.E6)\)\+\+exploration floor; no importance ratio

9:

drift←WildBootstrapTrend​\(V;α=0\.01\)\\mathrm\{drift\}\\leftarrow\\textsc\{WildBootstrapTrend\}\(V;\\,\\alpha\{=\}0\.01\)if

\|V\|≥12\|V\|\\geq 12elsefalse

10:ifdriftthen

𝒪\.Restart​\(\)\\mathcal\{O\}\.\\textsc\{Restart\}\(\)⊳\\trianglerightanchor a new comparator segment

11:

θt\+1←𝒪\.Update​\(−g\)\\theta\_\{t\+1\}\\leftarrow\\mathcal\{O\}\.\\textsc\{Update\}\(\-g\)⊳\\trianglerightoracle minimizes loss; feed−g\-g

12:

VT←VT\+∥θt\+1−θt∥2V\_\{T\}\\leftarrow V\_\{T\}\+\\lVert\\theta\_\{t\+1\}\-\\theta\_\{t\}\\rVert\_\{2\}
13:emit certificate

\(t,∥g∥,∥θt\+1−θt∥,VT,\#​restarts,maxa⁡w^−mina⁡w^\)\(t,\\,\\lVert g\\rVert,\\,\\lVert\\theta\_\{t\+1\}\-\\theta\_\{t\}\\rVert,\\,V\_\{T\},\\,\\\#\\text\{restarts\},\\,\\max\_\{a\}\\widehat\{w\}\-\\min\_\{a\}\\widehat\{w\}\)
14:endfor

Algorithm 4SGM\-CS: self\-edit admission with anytime familywise risk control1:initdeployed policy

π0\\pi\_\{0\}; confirmation count

k←0k\\leftarrow 0; CTHS budget\(

δ0\\delta\_\{0\}\); per\-version CS map

2:for

t=1,2,…t=1,2,\\dotsdo

3:ifbudget\.exhaustedthenemithold;continue⊳\\trianglerightharness frozen

4:

πcand←\\pi\_\{\\mathrm\{cand\}\}\\leftarrowpop a requeued promising*same\-family*harness \(its repo among the round’s\), else

ProposeEdit​\(πt−1,t\)\\textsc\{ProposeEdit\}\(\\pi\_\{t\-1\},t\)⊳\\trianglerightshadow requeue / force\-edit / budget / guidance

5:if

version​\(πcand\)=version​\(πt−1\)\\mathrm\{version\}\(\\pi\_\{\\mathrm\{cand\}\}\)=\\mathrm\{version\}\(\\pi\_\{t\-1\}\)thenemithold;continue⊳\\trianglerightno\-op edit; no spend

6:

tasks←\\text\{tasks\}\\leftarrowrank batch by*weak recent evidence*\(unseen / low\-reward first\); move the*best\-evidenced*task to the front as pilot⊳\\trianglerighthigher\-signal, index\-aligned pairing

7:pre\-gate:run

πcand\\pi\_\{\\mathrm\{cand\}\}on the pilot;ifbest pilot reward

<<pregateMinandpilot task is*known passable*thenspend

δk\\delta\_\{k\}\(

k\+\+k\{\+\}\{\+\}\); requeue a promoted shadow; emitnsf;continue⊳\\trianglerightevidence\-gated; cold start falls through

8:deploy

πt−1\\pi\_\{t\-1\}and

πcand\\pi\_\{\\mathrm\{cand\}\}concurrently on the*same*task batch \(isolated working copies; common random numbers—aligned per\-task seeds\)

9:ifno learnable pairing survives \(e\.g\. every rollout env\-errored\)thenemithold;continue⊳\\trianglerightno spend

10:

k←k\+1k\\leftarrow k\+1;

δk←δ0/\(Z​k​log2⁡\(k\+1\)\)\\delta\_\{k\}\\leftarrow\\delta\_\{0\}/\\big\(Z\\,k\\log^\{2\}\(k\+1\)\\big\)⊳\\trianglerightnormalized CTHS spend, committed on decision

11:update per\-version CSs\[Howardet al\.,[2021](https://arxiv.org/html/2607.00871#bib.bib22)\]with

RbaseR\_\{\\mathrm\{base\}\},

RcandR\_\{\\mathrm\{cand\}\}
12:

LCB←CSδklower​\(Rcand−Rbase\)−ε⋅W1​\(Rcand,Rbase\)\\mathrm\{LCB\}\\leftarrow\\mathrm\{CS\}\_\{\\delta\_\{k\}\}^\{\\mathrm\{lower\}\}\\big\(R\_\{\\mathrm\{cand\}\}\-R\_\{\\mathrm\{base\}\}\\big\)\-\\varepsilon\\cdot W\_\{1\}\(R\_\{\\mathrm\{cand\}\},R\_\{\\mathrm\{base\}\}\)⊳\\trianglerightperformative correction, exact 1\-DW1W\_\{1\}

13:ifwild\-bootstrap trend test rejects stationarity of the

vbasev\_\{\\mathrm\{base\}\}stream from baseline\-evaluated roundsthen

14:

LCB←LCB−radius\\mathrm\{LCB\}\\leftarrow\\mathrm\{LCB\}\-\\mathrm\{radius\}⊳\\trianglerightwiden;Chandaket al\.\[[2020](https://arxiv.org/html/2607.00871#bib.bib23)\]

15:endif

16:if

LCB≥−ϵtol\\mathrm\{LCB\}\\geq\-\\epsilon\_\{\\mathrm\{tol\}\}then

17:

πt←πcand\\pi\_\{t\}\\leftarrow\\pi\_\{\\mathrm\{cand\}\}⊳\\trianglerightaccept

18:else

πt←πt−1\\pi\_\{t\}\\leftarrow\\pi\_\{t\-1\};if

πcand\\pi\_\{\\mathrm\{cand\}\}produced a strong patchthenrequeue it \(bounded retries per harness\)⊳\\trianglerightnsf, but retry a promising harness later

19:endif

20:emit certificate

\(t,accept/nsf,LCB,δk,ε​W1,widened?,shadow\)\(t,\\ \\textsc\{accept\}/\\textsc\{nsf\},\\ \\mathrm\{LCB\},\\ \\delta\_\{k\},\\ \\varepsilon W\_\{1\},\\ \\text\{widened?\},\\ \\text\{shadow\}\)
21:endfor

Algorithm 5SDC\-QD: library growth with MDL compression and quality\-diversity acceptance1:initcorpus

C←∅C\\leftarrow\\emptyset; MAP\-Elites archive

ℳ\\mathcal\{M\}over descriptor

ϕ\\phi; library

L←L0L\\leftarrow L\_\{0\}; solve threshold

s∗s^\{\*\}
2:for

t=1,2,…t=1,2,\\dotsdo

3:foreach task in batchdo⊳\\trianglerightwake: LLM\-guided program search

4:

ρ←ParseSExpr​\(LLM​\(task,Lt\)\)\\rho\\leftarrow\\textsc\{ParseSExpr\}\\big\(\\mathrm\{LLM\}\(\\text\{task\},\\,L\_\{t\}\)\\big\)⊳\\trianglerightuntrusted output;Noneif not well\-formed

5:if

ρ≠\\rho\\neqNone and

env\.score​\(ρ\)≥s∗\\mathrm\{env\.score\}\(\\rho\)\\geq s^\{\*\}then

C←C∪\{ρ\}C\\leftarrow C\\cup\\\{\\rho\\\}
6:endfor

7:

Jbefore←\|Lt\|\+∑ρ∈C\|ρ\|J\_\{\\mathrm\{before\}\}\\leftarrow\|L\_\{t\}\|\+\\sum\_\{\\rho\\in C\}\|\\rho\|
8:cands

←\\leftarrowsubtrees

\(C\)∪\(C\)\\;\\cuppairwise antiunifications of compatible subtrees⊳\\trianglerightsleep: Stitch

9:

A∗←arg⁡maxA∈cands⁡u​\(A\)A^\{\*\}\\leftarrow\\arg\\max\_\{A\\in\\text\{cands\}\}u\(A\),

u​\(A\)=mA​\(bA−1−aA\)−bAu\(A\)=m\_\{A\}\\,\(b\_\{A\}\-1\-a\_\{A\}\)\-b\_\{A\}⊳\\trianglerightdescending scan, sound dominance break

10:if

A∗≠A^\{\*\}\\neqNone and

Δ​J≔u​\(A∗\)≥umin\\Delta J\\coloneqq u\(A^\{\*\}\)\\geq u\_\{\\min\}then⊳\\trianglerightquality\-diversity acceptance, minimum\-utility bar

11:

b←ϕ​\(Lt∪\{A∗\}\)=\(\|Lt\|\+1,meanρ∈C​\|ρ\|\)b\\leftarrow\\phi\(L\_\{t\}\\cup\\\{A^\{\*\}\\\}\)=\(\\,\|L\_\{t\}\|\{\+\}1,\\ \\mathrm\{mean\}\_\{\\rho\\in C\}\|\\rho\|\\,\);

f←Jbefore−Δ​Jf\\leftarrow J\_\{\\mathrm\{before\}\}\-\\Delta J⊳\\trianglerightlower fitness is better

12:if

ℳ\.IsNovel​\(b,f\)\\mathcal\{M\}\.\\textsc\{IsNovel\}\(b,f\)then⊳\\trianglerightempty cell or improves incumbent

13:

ℳ\.Add​\(Lt∪\{A∗\},b,f\)\\mathcal\{M\}\.\\textsc\{Add\}\(L\_\{t\}\\cup\\\{A^\{\*\}\\\},\\,b,\\,f\);

Lt\+1←Lt∪\{A∗\}L\_\{t\+1\}\\leftarrow L\_\{t\}\\cup\\\{A^\{\*\}\\\}; surface

A∗A^\{\*\}in harness

L2L\_\{2\}⊳\\trianglerightaccept

14:else

Lt\+1←LtL\_\{t\+1\}\\leftarrow L\_\{t\}⊳\\trianglerightreject: no novelty

15:endif

16:else

Lt\+1←LtL\_\{t\+1\}\\leftarrow L\_\{t\}⊳\\trianglerightreject: no MDL gain

17:endif

18:cert

←\\leftarrowdescription\-length certificate, Eq\. \([8](https://arxiv.org/html/2607.00871#S4.E8)\)

19:emit certificate

\(t,\#solved,\|Lt\+1\|,ΔJ,ℳ\.coverage,\|ℳ\.ParetoFrontier\(\)\|,cert\)\(t,\\,\\\#\\text\{solved\},\\,\|L\_\{t\+1\}\|,\\,\\Delta J,\\,\\mathcal\{M\}\.\\mathrm\{coverage\},\\,\|\\mathcal\{M\}\.\\mathrm\{ParetoFrontier\}\(\)\|,\\,\\text\{cert\}\)
20:endfor

Algorithm 6\(Alg 6\) Verifier\-in\-the\-loop search: best\-of\-NNselection and refinement with backtracking1:functionBestOfN\(

π\\pi, task

xx;

nn\)⊳\\trianglerightindependent diverse attempts; verifier selects

2:for

i=0,…,n−1i=0,\\dots,n\-1do

yi←Act​\(π,x,attempt=i\)y\_\{i\}\\leftarrow\\textsc\{Act\}\(\\pi,x,\\,\\mathrm\{attempt\}\{=\}i\)⊳\\trianglerightattempt\-indexed seed \+ isolated workspace; live:n=2n\{=\}2, attempt22only on a failing verdict, prompted to differ structurally

3:endfor

4:return

yi∗y\_\{i^\{\*\}\},

i∗=arg⁡maxi⁡V​\(x,yi\)i^\{\*\}=\\arg\\max\_\{i\}V\(x,y\_\{i\}\)⊳\\trianglerightVV: shaped native\-test verifier;p→1−\(1−p\)np\\to 1\-\(1\-p\)^\{n\}

5:endfunction

6:

7:functionRefine\(

π\\pi, task

xx; depth

dd\)⊳\\trianglerightverifier\-guided hill\-climb over patches

8:

y∗←Act​\(π,x,attempt=0\)y^\{\*\}\\leftarrow\\textsc\{Act\}\(\\pi,x,\\,\\mathrm\{attempt\}\{=\}0\);

v∗←V​\(x,y∗\)v^\{\*\}\\leftarrow V\(x,y^\{\*\}\)
9:for

i=1,…,d−1i=1,\\dots,d\-1do

10:if

v∗≥1v^\{\*\}\\geq 1then break⊳\\trianglerightresolved; stop early

11:

x′←x⊕\(y∗,Feedback​\(x,y∗\)\)x^\{\\prime\}\\leftarrow x\\oplus\\big\(y^\{\*\},\\ \\textsc\{Feedback\}\(x,y^\{\*\}\)\\big\)⊳\\trianglerightprior best patch \+ why the verifier rejected it

12:

y←Act​\(π,x′,attempt=i\)y\\leftarrow\\textsc\{Act\}\(\\pi,x^\{\\prime\},\\,\\mathrm\{attempt\}\{=\}i\);

v←V​\(x,y\)v\\leftarrow V\(x,y\)⊳\\trianglerightscored on the*original*task

13:if

v\>v∗v\>v^\{\*\}then

\(y∗,v∗\)←\(y,v\)\(y^\{\*\},v^\{\*\}\)\\leftarrow\(y,v\)⊳\\trianglerightclimb; else backtrack \(keep best\)

14:endfor

15:return

y∗y^\{\*\}
16:endfunction

Algorithm 7\(Alg 7\) Verified micro\-step search \(fast loop\): beam search with memory and a verifier cascade1:initbeam

←\{\\leftarrow\\\{seed patch

\}\\\}; memory

ℳ←∅\\mathcal\{M\}\\leftarrow\\emptyset; best

←\\leftarrowseed; depth

←0\\leftarrow 0
2:whiledepth

<<maxDepthand

V​\(best\)<1V\(\\text\{best\}\)<1do

3:children

←\[\]\\leftarrow\[\\,\]
4:fornode in beamdo

5:

H←Generate​\(node,ℳ,k\)H\\leftarrow\\textsc\{Generate\}\(\\text\{node\},\\,\\mathcal\{M\},\\,k\)⊳\\trianglerightreasoning\-first: name class/method, thenkkdistinct one\-line edits; capped prompt \(issue≤\\leq1\.8k chars, JSON\-array reply\)

6:for

h∈Hh\\in Hwith

h∉ℳ\.triedh\\notin\\mathcal\{M\}\.\\text\{tried\}do

7:

ρ←Compose​\(node\.patch,h\)\\rho\\leftarrow\\textsc\{Compose\}\(\\text\{node\.patch\},\\,h\);if

ρ=⊥\\rho=\\botthen

ℳ\.tried\+=h\\mathcal\{M\}\.\\text\{tried\}\\\!\\mathrel\{\+\}=h;continue⊳\\trianglerightdid not apply

8:

\(v,fb\)←Cascade​\(ρ\)\(v,\\,\\text\{fb\}\)\\leftarrow\\textsc\{Cascade\}\(\\rho\)⊳\\trianglerightcheap parse/apply check, then native suite only on survivors

9:

ℳ\.Record​\(h,v,fb\)\\mathcal\{M\}\.\\textsc\{Record\}\(h,v,\\text\{fb\}\); children\.append

\(ρ,v\)\(\\rho,v\);if

v≥1v\\geq 1then break

10:endfor

11:endfor

12:depth

←\\leftarrowdepth

\+1\+1;ifchildren

=∅=\\emptysetthen break⊳\\trianglerighta whole diverse round moved nothing

13:beam

←\\leftarrowtop\-

bbof \(beam

∪\\cupchildren\) by

VV; best

←arg⁡maxV\\leftarrow\\arg\\max\_\{V\}over all scored candidates⊳\\trianglerightparents kept; best tracked globally, independent of beam policy

14:endwhile

15:returnbest, its verified trace, and \# expensive verifier calls

Algorithm 8\(Alg 8\) Self\-authored reproduction oracles: grader\-free in\-loop verification1:givenissue text \(no gold tests, no test patch, no f2p/p2p lists\); a verifier model; a solver agent

2:

O←Synthesize​\(issue,k\)O\\leftarrow\\textsc\{Synthesize\}\(\\text\{issue\},\\,k\)over an*ensemble*of models⊳\\trianglerightany model may author a usable oracle \(recall\)

3:

A←\{o∈O:RunOnBase​\(o\)​fails, not a timeout, not asyntacticself\-error\}A\\leftarrow\\\{\\,o\\in O:\\textsc\{RunOnBase\}\(o\)\\text\{ fails, not a timeout, not a \\emph\{syntactic\} self\-error\}\\,\\\}⊳\\trianglerightfails\-on\-base admission—no ground truth

4:

A←\{o∈A:SymptomJudge​\(o\)​matches the issue\}A\\leftarrow\\\{\\,o\\in A:\\textsc\{SymptomJudge\}\(o\)\\text\{ matches the issue\}\\,\\\}⊳\\trianglerightdenoise; separates real import bugs from hallucinated APIs

5:the agent debugs withrun\_tests

=\\\!=\\\!self\-score over

AA:⊳\\trianglerightthe in\-loop gradient isAA, never the grader

6:

Vself​\(ρ\)=\#​\{o∈A:ρ​flips​o​fail→pass\}\|A\|V\_\{\\mathrm\{self\}\}\(\\rho\)=\\dfrac\{\\\#\\\{o\\in A:\\ \\rho\\text\{ flips \}o\\text\{ fail\}\\to\\text\{pass\}\\\}\}\{\|A\|\}, set to

0if any green check regresses;*promising*iff all flip

7:

ρ⋆←\\rho^\{\\star\}\\leftarrowthe agent’s submitted patch

8:measure once\(terminal, held out from the loop\): held\-out grader on

ρ⋆\\rho^\{\\star\}⊳\\trianglerightused only to report, never to steer

Algorithm 9\(Alg 9\) Efficient continual search: search\-layer controllers \(compute\-allocation\+\+QD diversity; step\-credit\+\+forgetting gate\)1:initstep\-credit

𝒞\\mathcal\{C\}\(Alg 3\); forgetting gate

𝒢\\mathcal\{G\}\(Alg 1\); earlier\-repo anchor

AA
2:foreach round over the problem streamdo

3:foreach problem

xxdo

4:order branch classes for

xxby learned credit

𝒞\\mathcal\{C\}⊳\\trianglerightAlg 3: productive step macros first

5:

BeamSearch​\(x\)\\textsc\{BeamSearch\}\(x\)with two controllers active:⊳\\trianglerightAlgorithm[7](https://arxiv.org/html/2607.00871#alg7)

6:Alg 4*budget*: record per\-depth marginal gain; stop when the CS upper bound

≤\\leqminGain

7:Alg 5*diversify*: pick the next beam by MAP\-Elites behavior cells, not raw top\-

bbscore

8:credit the solving branch

11, the branches tried before it

0⊳\\trianglerightAlg 3update; wasted shapes demoted

9:endfor

10:propose policy update;accept iff

𝒢\\mathcal\{G\}: re\-eval of anchor

AAdoes not regress⊳\\trianglerightAlg 1forgetting gate

11:record \(mean depth, resolved\-rate, forgetting\) for the round

12:endfor

Algorithm 10\(Alg 10\) Verified self\-repair: adopt a harness repair only by measured fix\-rate1:functionProposeRepairs\(failures

FF\(base, hypothesis pairs that did not apply\); repertoire

ℛ\\mathcal\{R\}; threshold

τ\\tau\)

2:adopted

←\[\]\\leftarrow\[\\,\]
3:foreach primitive

r∈ℛr\\in\\mathcal\{R\}do

4:fixed

←\|\{\(b,h\)∈F:r​\(h\)≠h∧Composes​\(b,r​\(h\)\)\}\|\\leftarrow\\big\|\\\{\(b,h\)\\in F:r\(h\)\\neq h\\ \\wedge\\ \\textsc\{Composes\}\(b,\\,r\(h\)\)\\\}\\big\|⊳\\trianglerightre\-test against the*real*composer

5:iffixed

/\|F\|≥τ/\|F\|\\geq\\tauthenadopted\.append

\(r,fixed/\|F\|\)\\big\(r,\\ \\text\{fixed\}/\|F\|\\big\)⊳\\trianglerightcredited only if it changedhhand applied

6:endfor

7:returnadopted sorted by fix\-rate, best\-first⊳\\trianglerightappended to the harness repair pipeline \(L2L\_\{2\}\)

8:endfunction

9:

10:functionProposeGenerationRepair\(suffix repertoire

𝒢\\mathcal\{G\}; baseline apply\-rate

ρ0\\rho\_\{0\}; margin

mm\)

11:foreach suffix

g∈𝒢g\\in\\mathcal\{G\}do

12:

ρ←RegenApplyRate​\(g\)\\rho\\leftarrow\\textsc\{RegenApplyRate\}\(g\)⊳\\trianglerightre\-generate hypotheses withggappended; fraction that compose

13:if

ρ−ρ0≥m\\rho\-\\rho\_\{0\}\\geq mthenadopt

gg⊳\\trianglerightverified at the source: fixes*future*candidates

14:endfor

15:endfunction

Algorithm 11Composite controller: scheduled co\-evolution over one shared policy1:initshared policy

π\\pi; schedule

\{\(namei,𝒞i,Ki\)\}\\\{\(\\text\{name\}\_\{i\},\\,\\mathcal\{C\}\_\{i\},\\,K\_\{i\}\)\\\}⊳\\trianglerightKi=0K\_\{i\}=0disables𝒞i\\mathcal\{C\}\_\{i\}\(ablation\)

2:for

t=1,2,…t=1,2,\\dotsdo

3:ran

←\[\]\\leftarrow\[\\,\]
4:foreach

\(namei,𝒞i,Ki\)\(\\text\{name\}\_\{i\},\\mathcal\{C\}\_\{i\},K\_\{i\}\)with

Ki\>0K\_\{i\}\>0and

tmodKi=0t\\bmod K\_\{i\}=0do⊳\\trianglerightin schedule order

5:ifcost\-awareand

𝒞i\\mathcal\{C\}\_\{i\}is a slow

L2L\_\{2\}controller with no*committed*gain for the lastskipAfterroundsthenskip \(hold, reclaim its deploys\); set its counter to

skipAfter−1\\textsc\{skipAfter\}\-1\(probe next round\);continue

6:

𝒞i\.π←π\\mathcal\{C\}\_\{i\}\.\\pi\\leftarrow\\pi;

ci←𝒞i\.Step​\(t\)c\_\{i\}\\leftarrow\\mathcal\{C\}\_\{i\}\.\\textsc\{Step\}\(t\);

π←𝒞i\.π\\pi\\leftarrow\\mathcal\{C\}\_\{i\}\.\\pi⊳\\trianglerightpropagate this layer’s edit

7:update

𝒞i\\mathcal\{C\}\_\{i\}’s no\-gain counter from

cic\_\{i\}\(reset on committed gain: accept / shadow\-accept / committed macro / anti\-macros\); ran\.append

\(\(namei,ci\)\)\\big\(\(\\text\{name\}\_\{i\},c\_\{i\}\)\\big\)
8:endfor

9:emit merged certificate: metrics prefixed

namei\.∗\\text\{name\}\_\{i\}\.\\ast; spends summed;

10:decision

←\\leftarrowfirst of \(accept,nsf,reject\) present among

\{ci\}\\\{c\_\{i\}\\\}, elsehold

11:endfor

## Appendix BThe Full Self\-Evolution Loop: HowAlg 6–Alg 10Plug In

Figure[1](https://arxiv.org/html/2607.00871#S1.F1)shows the four layers and the five scheduled controllers \(Alg 1–Alg 5\), which can only*select*among behaviors the frozen base already produces\. Figure[4](https://arxiv.org/html/2607.00871#A2.F4)extends it with the verifier\-tier mechanismsAlg 6–Alg 10\(Table[1](https://arxiv.org/html/2607.00871#S4.T1)\) that*generate and verify*those behaviors, supplying the variation and the dense, grader\-free signal the controllers consume\. The framework is one closed loop over three stages\.\(1\) Policy\.The deployed policyπt=L0∘L1\(t\)∘L2\(t\)\\pi\_\{t\}=L\_\{0\}\\circ L\_\{1\}^\{\(t\)\}\\circ L\_\{2\}^\{\(t\)\}is rolled out\.\(2\) Engine \(Alg 6–Alg 10\)\.An actor\-and\-search engine manufactures diverse candidate patches and scores them against a self\-authored verifier: best\-of\-NN/refinement varies attempts \(Alg 6\), micro\-step search relocates the search to one\-line edits where even a weak base is reliable \(Alg 7\), self\-authored reproduction oracles supply the in\-loop rewardVselfV\_\{\\mathrm\{self\}\}without touching the held\-out grader \(Alg 8\), the search\-layer controllers govern that search \(Alg 9\), and verified self\-repair fixes the harness’s own edit/compose operators \(Alg 10\)\.\(3\) Controllers \(Alg 1–Alg 5\)\.The*verified*rollouts and process rewards the engine emits feed theL3L\_\{3\}controllers, which gate every self\-modification, write the certificate ledger, and updateπt\\pi\_\{t\}for the next round\. The held\-out grader sits*outside*the loop: it measures the finalized patch once and never steers the search\. The division of labor is the figure’s point—Alg 6–Alg 10create and verify variation;Alg 1–Alg 5gate and select among it;Alg 9is itselfAlg 1/Alg 3/Alg 4/Alg 5re\-aimed onto the search layer \(Table[2](https://arxiv.org/html/2607.00871#S5.T2)\)\.

L0L\_\{0\}— frozen base modelL1L\_\{1\}— steering adapterθ\\theta\(online\-steered\)L2L\_\{2\}— harness: prompt⋅\\cdottools⋅\\cdotbudgets⋅\\cdotlibrary⋅\\cdotrepairdeployed policyπt=L0∘L1\(t\)∘L2\(t\)\\pi\_\{t\}=L\_\{0\}\\circ L\_\{1\}^\{\(t\)\}\\circ L\_\{2\}^\{\(t\)\}actor— multi\-stepReActagent over the structured tool vocabulary \(§[7](https://arxiv.org/html/2607.00871#S7)\)Alg 6best\-of\-NN/ refinement— designed operator; off in the live stackAlg 7verified micro\-step search— beam over one\-line edits where the base is reliableAlg 8self\-authored oracles⇒Vself\\Rightarrow V\_\{\\mathrm\{self\}\}— dense, grader\-free in\-loop rewardAlg 9search\-layer controllers— compute\-stop⋅\\cdotQD diversify⋅\\cdotstep\-credit⋅\\cdotforgetting gateAlg 10verified self\-repair— adopts edit/compose fixes by measured fix\-rate \(→L2\\to L\_\{2\}\)actor & verified\-search engine \(Alg 6–Alg 10\)*generates & verifies*candidate behaviorsAlg 4SGM\-CS — gated harness edits \(L2L\_\{2\}\)Alg 5SDC\-QD — grows the abstraction library \(L2L\_\{2\}\)Alg 3PA\-COCOA — counterfactual credit→θ\\to\\theta\(L1L\_\{1\}\)Alg 1PPB\-CL — forgetting gate\+\+trust region \(L1L\_\{1\}\)Alg 2PNMP\-A — anchors the reward modelqqL3L\_\{3\}scheduled controllers \(Alg 1–Alg 5\)*gate & select*self\-modificationsenvironment\+\+held\-out graderterminal measurement only — never steers the searchrewardmodelqqcertificate ledgeraccept/hold/nsf⋅\\cdotδ\\delta\-spenddeployπt\\pi\_\{t\}/ actverified rollouts\+\+process rewardR~\\tilde\{R\}finalized patche\-value drift gateqqscores rolloutsgated self\-modifications⇒πt\+1\\Rightarrow\\pi\_\{t\+1\}Alg 3/Alg 1→L1\\to L\_\{1\}Alg 4/Alg 5→L2\\to L\_\{2\}Alg 2→q\\to q

Figure 4:The full self\-evolution loop \(extends Figure[1](https://arxiv.org/html/2607.00871#S1.F1)\)\.The five scheduled controllers of Figure[1](https://arxiv.org/html/2607.00871#S1.F1)\(right,Alg 1–Alg 5\) can only select among behaviors the frozen policy stack \(left,πt\\pi\_\{t\}\) already produces\. The verifier\-tier mechanismsAlg 6–Alg 10\(centre, green dashed\) are the engine that supplies them: the actor and its search operators \(Alg 6/Alg 7\) manufacture diverse candidate patches; self\-authored reproduction oracles \(Alg 8\) score each candidate with a dense, grader\-free rewardVselfV\_\{\\mathrm\{self\}\}; the search\-layer controllers \(Alg 9,=Alg 1/Alg 3/Alg 4/Alg 5=\\textsc\{Alg~1\}/\\textsc\{Alg~3\}/\\textsc\{Alg~4\}/\\textsc\{Alg~5\}re\-aimed\) govern the search; and verified self\-repair \(Alg 10\) fixes the harness’s own edit operators\. The engine emits*verified*rollouts and process rewardsR~\\tilde\{R\}that feed theL3L\_\{3\}controllers, which gate each self\-modification, write the certificate ledger, and close the loop by updatingπt\\pi\_\{t\}\(Alg 3/Alg 1onL1L\_\{1\},Alg 4/Alg 5onL2L\_\{2\},Alg 2on the reward modelqq\)\. The held\-out grader sits outside the loop, measuring the finalized patch once and never steering it\.
## Appendix CCertificate Schema

Each round of every controller emits an immutable, structured certificate with fields:algorithm,round,decision∈\{accept,hold,reject,nsf\}\\in\\\{\\textsc\{accept\},\\textsc\{hold\},\\textsc\{reject\},\\textsc\{nsf\}\\\},delta\_spent\(this round’s error spend\),cumulative\_delta, a metrics map, and a free\-text note\. Controller\-specific metrics include:Alg 1empirical risk, KL to prior, PAC\-Bayes penalty and risk bound, forgetting bound, trust radius, gradient norm, damping count,ε​L\\varepsilon L;Alg 2mean reward, e\-value, KL to magnet, slow/fast rates, preference drift;Alg 3gradient and step norms, path variation, restarts, contribution spread;Alg 4corrected LCB, per\-version value bounds, performative shift, widening flag, baseline\-evaluated flag \(so the stationarity stream can exclude rounds without a fresh baseline value\);Alg 5solved count, library size,Δ​J\\Delta J\(the SWE variant reports it only when the macro committed, alongside a committed\-macro flag; the generic controller reports it unconditionally\), archive coverage, frontier size, description\-length bound\. The composite controller merges sub\-certificates with per\-algorithm metric prefixes into one ledger row per round\. The search\-layer controllers emit a finer\-grained, per\-*decision*audit log: an opt\-in, decision\-neutral JSONL sink writes one flushed row per controller decision \(the marginal gain and confidence interval forAlg 4, the collapsed behavior cells forAlg 5, the per\-branch verified reward forAlg 3, the measured forgetting forAlg 1\), so a crashed or killed run still leaves a complete, inspectable trail\. This per\-decision trail is what attributes each flip and regression to a named mechanism in §[9](https://arxiv.org/html/2607.00871#S9)\.

## Appendix DDefault Hyperparameters

Defaults in the reference implementation:Alg 1:δ=0\.05\\delta=0\.05,τforget=0\.1\\tau\_\{\\mathrm\{forget\}\}=0\.1\(raised to1\.11\.1above the anytime floor for live multi\-step SWE runs\),ε=0\.2\\varepsilon=0\.2,L=1L=1, buffer capacitym=16m=16, dampingρ=0\.5\\rho=0\.5,λ\\lambdaset per evaluation to the DV\-optimalλ∗=8​\|𝒜\|​At\\lambda^\{\*\}=\\sqrt\{8\|\\mathcal\{A\}\|A\_\{t\}\}\(λ=4\\lambda=4fallback forAt≤0A\_\{t\}\\leq 0\), REINFORCE baseline pooled over a replay window of8×8\\timesbatch size, learning rateη=0\.5\\eta=0\.5, anchor capacityA=4A=4\.Alg 2:α=0\.3\\alpha=0\.3,β=0\.1\\beta=0\.1,η=0\.5\\eta=0\.5,τ=0\.1\\tau=0\.1,δ0=0\.05\\delta\_\{0\}=0\.05, magnet periodK=5K=5, slow/fast step exponents1\.0/0\.71\.0/0\.7witha0=0\.5a\_\{0\}=0\.5, preference pairs pooled over the recent replay window\.Alg 3: contribution half\-lifeh=64h=64over a window of4​h4hrecent rollouts, drift test from1212observations at level0\.010\.01, exploration floorφ=0\.05\\varphi=0\.05\.Alg 4:δ0=0\.05\\delta\_\{0\}=0\.05,ϵtol=0\.02\\epsilon\_\{\\mathrm\{tol\}\}=0\.02,ε=0\.1\\varepsilon=0\.1, CSα=0\.05\\alpha=0\.05withσ=0\.5\\sigma=0\.5\(Hoeffding scale for\[0,1\]\[0,1\]rewards\); SWE proposer cycle: force\-edit threshold decrements of22down to a floor of33\(initialized at half the step budget\), step\-budget increments of66capped at3636, guidance rewrites under 60 words; pre\-gate pilot of11task with minimum reward0\.050\.05\(disabled on the first round, when no evidence exists\), shadow threshold1\.01\.0with retry cap of22per harness; the paired\-difference CS runs at scaleσ=1\.0\\sigma=1\.0\.Alg 5:δ=0\.05\\delta=0\.05, solve threshold1\.01\.0, minimum utility1\.01\.0; SWE macro\-reward bar equal to the corpus bar, downstream lift measured once the contextual pool holds≥4\\geq 4rewards; generic archive16×1616\\times 16over \(library size, mean program size\) in\[0,32\]2\[0,32\]^\{2\}; SWE archive8×168\\times 16over \(arity, pattern size\) in\[0,8\]×\[0,16\]\[0,8\]\\times\[0,16\], operation windows of length22–33, corpus bar0\.40\.4on the process reward, minimum downstream lift0\.050\.05\(strictly required; path\-literal and single\-op macros refused\), library capped at33entries per \(repository, status\) context and1212total, retirement over a2424\-rollout window at bad\-rate0\.50\.5\. Verifier mechanisms: best\-of\-NNdefaultn=4n=4, refinement depth33\(best\-of\-NN/Alg 6is removed from the live composite, §[9\.1](https://arxiv.org/html/2607.00871#S9.SS1); refinement depth11so verify\-react stays active; unverified\-green cap0\.90\.9; tie\-break judge at score≥0\.85\\geq 0\.85with zero admitted oracles; micro\-search issue cap1\.81\.8k characters; fuzzy\-edit hint threshold0\.60\.6\);run\_testsfeedback carries the last14001400characters of the failure log\. Composite default periods:Alg 1/Alg 3every round,Alg 4/Alg 5every22,Alg 2every33; cost\-aware skip after22gainless scheduled rounds\.

## Appendix EThe SWE Agent Tool Protocol

The agent replies with exactly one structured action per turn, drawn from the vocabulary

list​\(d\)\\texttt\{list\}\(d\)search​\(r\)\\texttt\{search\}\(r\)read\(f,i:j\)\\texttt\{read\}\(f,\\,i\{:\}j\)edit​\(f,old→new\)\\texttt\{edit\}\(f,\\,\\textit\{old\}\\\!\\to\\\!\\textit\{new\}\)run\_tests​\(\)\\texttt\{run\\\_tests\}\(\)submit​\(\)\\texttt\{submit\}\(\)

over directoriesdd, regular expressionsrr, filesff, line rangesi:ji\{:\}j, and exact text replacements\.

Edits require the search text to match exactly once; a miss falls back to a unique whitespace\-normalized fuzzy match, and a true miss returns the nearest similar line as a hint \(ambiguous matches are refused\)\. Search is regular\-expression matching with capped output \(at most6060matches\); read windows report the file’s true line total; observations are truncated to a fixed character budget; malformed replies receive a corrective error message and consume a step\.run\_testsis advertised only when a test runner is configured; it applies the current edits, runs the instance’s target tests, and returns pass counts plus the tail of the failure log \(“submit now” on a full resolve\)\. When a test runner is available,submitis blocked untilrun\_testshas been run on the latest edits, so the agent cannot submit blind; and an episode that would otherwise end with an empty diff triggers a final no\-patch recovery pass \(up to two forced\-edit attempts at temperature0\)\. When the explore→\\toedit budget binds,search/read/listreturn a corrective message until an edit lands\. The final patch is the accumulated edit set, scored by the native evaluator\.