The Harness as the Only Mutable Surface: Compliance-Bounded Self-Evolution of LLM Agents in Credit Pipelines, with a Measured Admission Gate

# The Harness as the Only Mutable Surface:Compliance-Bounded Self-Evolution of LLM Agentsin Credit Pipelines, with a Measured Admission Gate
Source: [https://arxiv.org/html/2610.10629](https://arxiv.org/html/2610.10629)
ORCID 0000\-0002\-0783\-8573

###### Abstract

Self\-improving large language model \(LLM\) agents can adapt a credit pipeline to a changed rule, but an agent that rewrites itself destroys the artefact a supervisor reviews: a named change, a recorded test, an approval\. We argue that self\-evolution is reviewable only if it is confined to the runtime harness—instruction text, tool\-call logic and primitive composition—while model weights stay fixed, so that every adaptation is a diff with a cause and a test attached\. We give a dual\-loop engine built on that bound, in which a local loop edits one primitive and a global loop replaces primitives from a typed library, and both submit to one admission gate that writes a hash\-chained record before deployment\. We then measure the gate in simulation, with a simulated agent and a seeded\-search proposer rather than language models\. Across three families of supervisory re\-interpretation at three severities, 10 seeds each, the gated dual loop admitted 144 of 7,449 candidate changes, none of which worsened error on held\-out history, and restored the false\-positive rate to the oracle level without raising missed flags in all low\- and mid\-severity cells\. With the gate replaced by the check an unbounded system applies—fewer errors visible in recent traces—the same loops admitted 309 harmful changes and left missed flags above 10% in 49 of 90 runs: false positives fell because the screen was loosened\. Evaluated on pre\-shift labels, the gate rejected every candidate, so a re\-interpretation must be encoded as a rule that relabels history\. Parametric and scope shifts were repaired locally, a structural one only by primitive replacement; at the highest structural severity the gate’s fixed tolerance blocked the correct replacement in half the seeds\. We map the mechanisms to the EU AI Act’s provisions for high\-risk credit scoring and note that the April 2026 US model\-risk guidance excludes agentic AI from its scope\.

Keywords:LLM agents; self\-evolving agents; credit risk; model risk management; EU AI Act; auditability; RegTech; benchmark

## 1Introduction

Credit underwriting is moving from monolithic scoring models to pipelines of specialised LLM agents that query registries, screen counterparties and draft decisions\. When a statute or a supervisory interpretation changes, such a pipeline must change too\. Work on self\-evolving agents promises to automate that adaptation\[[14](https://arxiv.org/html/2610.10629#bib.bib14),[15](https://arxiv.org/html/2610.10629#bib.bib15)\], but it was built to raise a task metric on open benchmarks, not to leave behind what a supervisor reviews\.

The difficulty is not that self\-evolution is unsafe in general\. It is that model\-risk governance presumes a versioned object with an owner, a test result and an approval\[[3](https://arxiv.org/html/2610.10629#bib.bib3),[1](https://arxiv.org/html/2610.10629#bib.bib1)\], and continuous self\-modification dissolves all three\. The question is therefore not whether to bound self\-evolution but which bound preserves the reviewable artefact\.

Our answer is that the runtime harness is the only legitimate mutable surface,*because*it is the auditable one\. Weights cannot be diffed into named changes; a harness can\. If the engine may rewrite instruction text, tool\-call logic and the composition of typed primitives, and nothing else, then “the agent changed” stops being an event that requires re\-validating a model and becomes a text diff with a cause and a test attached\. The self\-evolution literature treats the harness as one surface among several to optimise over\[[5](https://arxiv.org/html/2610.10629#bib.bib5),[6](https://arxiv.org/html/2610.10629#bib.bib6),[7](https://arxiv.org/html/2610.10629#bib.bib7)\]; we treat it as the only one an institution may let evolve\.

A bound is only as good as the control that enforces it\. In an earlier presentation of this design the admission gate—the mechanism that distinguishes it from unbounded methods—was specified but not measured, and three referees said so\. This paper is the measurement\. We claim:

1. 1\.A position and an architecture\(Sections[4](https://arxiv.org/html/2610.10629#S4)–[5](https://arxiv.org/html/2610.10629#S5)\)\. Harness\-only evolution with fixed, digest\-pinned weights \(a design commitment: no model is pinned in our simulation\); a local and a global loop sharing one retrospective pool and one gate; an admission rule with a value for every threshold \(Table[2](https://arxiv.org/html/2610.10629#S5.T2)\); a hash\-chained log written before deployment\. We separate what the log makes auditable \(the configuration change\) from what it does not make reproducible \(downstream behaviour of a hosted model\)\.
2. 2\.A measured gate\(Sections[6](https://arxiv.org/html/2610.10629#S6)–[7](https://arxiv.org/html/2610.10629#S7)\)\. A regulatory\-shift track with three families and three severities of re\-interpretation, 10 seeds per cell and ten arms, including a no\-gate arm, a gate on stale labels, oracle and manual updates and a compute\-matched local\-only arm\. We report missed\-flag rates on every row, candidate counts, false admissions audited on held\-out data, one worked rejection, and the gate’s own failure mode\.
3. 3\.An answer to where the labels come from\(Section[6\.2](https://arxiv.org/html/2610.10629#S6.SS2)\)\. After a re\-interpretation the retrospective pool must be relabelled by the new rule, or a correct adaptation is indistinguishable from a regression\. This makes the adaptation supervised at the level of the rule and autonomous only at the level of the harness, and we position the autonomy claim accordingly\.
4. 4\.A regulatory mapping\(Section[8](https://arxiv.org/html/2610.10629#S8)\) onto the EU AI Act’s provisions on predetermined changes, logging, human oversight, robustness and change management, and an account of the gap the April 2026 US guidance leaves for agentic systems\.
5. 5\.A released benchmark\(Section[9](https://arxiv.org/html/2610.10629#S9)\), CACE\-Bench, whose reference run on decision availability we now report with missed\-flag rates and 5 seeds\.

What we do*not*claim is equally specific\. The proposer in our experiments is a seeded search over a stated edit space, not a language model, and the typed library contains the correct replacement primitive by construction\. We therefore measure whether the bound and the gate select correct changes and reject harmful ones, not whether an LLM can invent the changes\. All data are synthetic\. No supervisor has reviewed the mapping\.

## 2Related work

#### Self\-evolving agents\.

Agents that improve themselves now span memory\[[9](https://arxiv.org/html/2610.10629#bib.bib9)\], prompts\[[8](https://arxiv.org/html/2610.10629#bib.bib8),[7](https://arxiv.org/html/2610.10629#bib.bib7)\], harnesses\[[5](https://arxiv.org/html/2610.10629#bib.bib5)\], typed harness primitives co\-evolved with model parameters\[[6](https://arxiv.org/html/2610.10629#bib.bib6)\], workflows\[[13](https://arxiv.org/html/2610.10629#bib.bib13)\], and code or architecture\[[10](https://arxiv.org/html/2610.10629#bib.bib10),[11](https://arxiv.org/html/2610.10629#bib.bib11),[12](https://arxiv.org/html/2610.10629#bib.bib12)\]; two surveys organise the field by what evolves and where the loop closes\[[14](https://arxiv.org/html/2610.10629#bib.bib14),[15](https://arxiv.org/html/2610.10629#bib.bib15)\]\. Our local loop follows the weakness\-mining, proposal and validation cycle of Self\-Harness\[[5](https://arxiv.org/html/2610.10629#bib.bib5)\]; our global loop follows the primitive replacement of HarnessX\[[6](https://arxiv.org/html/2610.10629#bib.bib6)\], without HarnessX’s co\-evolution of model parameters, which is precisely what our bound excludes\. We do not claim either loop as new\. Table[1](https://arxiv.org/html/2610.10629#S2.T1)places the work on the axes a regulated deployment cares about\. The field closes its loops on a task metric and produces no change record intended for a third party; that is a governance gap, not a capability gap, and we make no claim that our engine optimises better than any of these systems\.

Table 1:Positioning against representative self\-evolution systems\.*Gate*: is a change admitted only after an explicit, recorded test?*Record*: is a change artefact produced for an external reviewer?*Gate measured*: are admission and rejection statistics reported?∼\\sim: a validation step exists but is neither recorded for review nor reported as a control\.
#### Evaluation of agents in finance\.

We build on a three\-level quality\-metric system for LLM agents in the credit pipeline introduced in prior work\[[16](https://arxiv.org/html/2610.10629#bib.bib16)\]; it is not a contribution of this paper\. Every number reported here is computed against deterministic labels; no LLM judge is involved anywhere in the results\.

#### Model\-risk governance\.

US supervisory model\-risk guidance was revised on 17 April 2026 \(SR 26\-2; OCC Bulletin 2026\-13; FDIC FIL\-15\-2026\), rescinding SR 11\-7 and stating that generative and agentic AI models are outside its scope\[[3](https://arxiv.org/html/2610.10629#bib.bib3),[4](https://arxiv.org/html/2610.10629#bib.bib4)\]\. The EU AI Act classifies creditworthiness assessment of natural persons as high\-risk\[[1](https://arxiv.org/html/2610.10629#bib.bib1), Annex III, point 5\(b\)\]and regulates change in systems that continue to learn\[[1](https://arxiv.org/html/2610.10629#bib.bib1), Art\. 43\(4\)\]\. Section[8](https://arxiv.org/html/2610.10629#S8)develops both\.

## 3Problem, threat model and requirements

We consider a multi\-agent credit pipeline—client assistant, underwriter, compliance and legal agents—whose agents execute*typed primitives*: named, individually testable units of behaviour such as registration verification or the beneficial\-owner check\. A primitive is defined by its instruction text, its tool\-call contract and its output schema\. The set of primitives available to an agent, with their instruction texts and parameters, is that agent’s*runtime harness*\. It is the only object the engine may modify and the granularity at which every audit record is written\. The benchmark is jurisdiction\-parametrised; its reference run \(Section[9](https://arxiv.org/html/2610.10629#S9)\) covers seven Latin American jurisdictions, and nothing in the engine depends on a particular national statute\.

#### Threat model\.

We design against five failure modes\.F1 Silent drift: a change improves the target metric while degrading an unmeasured one; mitigated by the non\-regression clause, and only to the extent that the degraded dimension is measured\.F2 Retro\-test overfitting: a candidate is tuned until it passes the pool; mitigated by a pool split the gate never sees \(HvalH\_\{\\mathrm\{val\}\}\) and a held\-out post\-shift window\.F3 Untraceable change: mitigated by making the log write a precondition of deployment\.F4 Objective capture: evolution objective and audit evidence from the same judge; mitigated by computing both on deterministic labels only\.F5 Exogenous model change: the provider updates a hosted model beneath a fixed harness;*not*mitigated by our design \(Section[4](https://arxiv.org/html/2610.10629#S4)\)\.

#### Requirements\.

\(R1\)*Measurability*: every step yields signals localisable to a primitive\. \(R2\)*Auditability*: each change carries a cause, a test result and a deployment record in an append\-only log\. \(R3\)*Bounded autonomy*: adaptation must not degrade previously correct behaviour beyond a stated tolerance, and weights must not change\. \(R4\)*Regulatory correspondence*: controls must correspond to identifiable obligations \(Section[8](https://arxiv.org/html/2610.10629#S8)\)\.

## 4The harness as the governance primitive

The engine may rewrite instruction text, tool\-call logic and primitive composition; it may not fine\-tune, and the served model is pinned by digest\. This preserves the object a supervisor knows how to review\. A change under model\-risk governance is expected to have a cause, a test, an approver and a rollback path\. A weight update supplies none of these naturally: it is continuous, not decomposable into named changes, and cannot be diffed\. A harness edit supplies all of them by construction: it is a diff against a named primitive, reproducible from the log, and revertible by writing the previous version back\.

Two properties must be kept apart, because a referee rightly observed that we had conflated them\.*Auditability of the change*is delivered: every harness version is a canonical serialisation with a SHA\-256 digest, every gate decision names the digests before and after, and the log is hash\-chained so that removing or reordering a record breaks verification \(all 540 of the 540 logs written in our experiments, 43,814 records in all, verify\)\. The record names a deploying identity; in the simulation it is the gate’s service identity, and a human approval step before deployment is a deployment choice we do not simulate\.*Reproducibility of behaviour*is not delivered by the same mechanism: identical harnesses on a hosted model can behave differently if the provider changes the model, the decoding settings drift, a tool returns different data, or retrieval state moves\. Weight attestation binds our changes and not the provider’s \(F5\)\. For a deterministic pipeline—as in our benchmark—the two coincide; for a hosted LLM, behaviour reproducibility additionally requires pinned model versions and decoding parameters, recorded tool responses, and canary probes on a fixed prompt set\.

The cost of the bound is real: the system cannot learn anything that instruction text and primitive composition cannot express\. A task the base model cannot do falls outside the loop and remains engineering work\. We regard that as the right trade for regulated deployment, not as a free one\.

## 5The engine

### 5\.1Admission rule

Lethhbe the deployed harness,h′h^\{\\prime\}a candidate,HselH\_\{\\mathrm\{sel\}\}the half of the retrospective pool the gate may use, labelled under the*current*rule \(Section[6\.2](https://arxiv.org/html/2610.10629#S6.SS2)\),C⊆HselC\\subseteq H\_\{\\mathrm\{sel\}\}the caseshhanswers correctly, andk∈\{FP,miss\}k\\in\\\{\\mathrm\{FP\},\\mathrm\{miss\}\\\}the error kind of the cluster that caused the proposal\. Over the cases inHselH\_\{\\mathrm\{sel\}\}on which an error of kindkkis possible \(trueclearfor FP, trueflagfor misses\), letbbcount cases wherehherrs andh′h^\{\\prime\}does not,ccthe reverse, andnkn\_\{k\}their number\. The paired reduction isg^=\(b−c\)/nk\\hat\{g\}=\(b\-c\)/n\_\{k\}with variance estimatevar^=\(b\+c\)/nk2−\(b−c\)2/nk3\\widehat\{\\mathrm\{var\}\}=\(b\+c\)/n\_\{k\}^\{2\}\-\(b\-c\)^\{2\}/n\_\{k\}^\{3\}\. The candidate is admitted iff

g^−z0\.975​var^\>0⏟significant improvement∧\|\{x∈C:h′​\(x\)≠y⁡\(x\)\}\|\|C\|≤ϵ⏟non\-regression on​C\.\\underbrace\{\\hat\{g\}\-z\_\{0\.975\}\\sqrt\{\\widehat\{\\mathrm\{var\}\}\}\>0\}\_\{\\text\{significant improvement\}\}\\quad\\wedge\\quad\\underbrace\{\\frac\{\|\\\{x\\in C:h^\{\\prime\}\(x\)\\neq y\(x\)\\\}\|\}\{\|C\|\}\\leq\\epsilon\}\_\{\\text\{non\-regression on \}C\}\.\(1\)The improvement margin is statistical rather than a fixedδ\\delta, so 0\.1 and 5 percentage points are no longer alike to the gate\. The toleranceϵ=0\.005\\epsilon=0\.005is not zero because execution noise makes strict non\-regression unsatisfiable: any change that alters the rule’s output on a case can meet that case’s noise draw and turn a correct answer wrong\. Section[7\.6](https://arxiv.org/html/2610.10629#S7.SS6)measures what this choice costs\.

Admission is a write, not only a decision\. Before a candidate reaches the runtime the gate appends⟨\\langlesequence, stream position, loop, cause, diff, digest ofhh, digest ofh′h^\{\\prime\}, rendered text ofh′h^\{\\prime\},bb,cc,nkn\_\{k\},g^\\hat\{g\}and its interval,\|C\|\|C\|, regressions onCC, decision, deploying identity, hash of the previous record⟩\\rangleand the record’s own hash\. Rejections are written with the same schema\.

### 5\.2Loops

Operational feedback arrives in windows of 500 cases\. Every flagged case is reviewed by an analyst, so its false positive becomes visible; cleared cases are audited at 5%, so most misses stay invisible\. This asymmetry is deliberate and realistic, and it is what makes an ungated loop dangerous: a proposer guided by visible failures sees false positives and almost no misses\.

Algorithm 1Local loop, per window1:window

WW, deployed harness

hh, gate pool

HselH\_\{\\mathrm\{sel\}\}, budget

Bℓ=3B\_\{\\ell\}=3, clusters

K=3K=3, escalation threshold

N=3N=3
2:

F←F\\leftarrowvisible errors in

WWclustered by \(hit type, error kind\); keep the

KKmost frequent

3:foreach cluster

f∈Ff\\in Fdo

4:

P←P\\leftarrowone\-step edits to the deployed primitive \(Table[2](https://arxiv.org/html/2610.10629#S5.T2)\), shuffled; keep

BℓB\_\{\\ell\}
5:foreach

h′∈Ph^\{\\prime\}\\in Pdo

6:write record;if

h′h^\{\\prime\}satisfies Eq\. \([1](https://arxiv.org/html/2610.10629#S5.E1)\) on

HselH\_\{\\mathrm\{sel\}\}then

h←h′h\\leftarrow h^\{\\prime\};break

7:endfor

8:

𝑠𝑡𝑎𝑙𝑙⁡\[f\]←0\\mathit\{stall\}\[f\]\\leftarrow 0if admitted, else

𝑠𝑡𝑎𝑙𝑙⁡\[f\]\+1\\mathit\{stall\}\[f\]\+1
9:if

𝑠𝑡𝑎𝑙𝑙⁡\[f\]≥N\\mathit\{stall\}\[f\]\\geq NthenEscalate\(f\)\(f\)to Algorithm[2](https://arxiv.org/html/2610.10629#alg2);

𝑠𝑡𝑎𝑙𝑙⁡\[f\]←0\\mathit\{stall\}\[f\]\\leftarrow 0
10:endif

11:endfor

Algorithm 2Global loop, on escalation \(dual loop\) or on threshold breach \(global\-only arm\)1:cluster

ff, deployed harness

hh, typed library

ℒ\\mathcal\{L\}, budget

Bg=6B\_\{g\}=6
2:

P←\{\(p,θ\):p∈ℒ∖\{ph\},θ∈Θ\}P\\leftarrow\\\{\(p,\\theta\):p\\in\\mathcal\{L\}\\setminus\\\{p\_\{h\}\\\},\\ \\theta\\in\\Theta\\\}with

hh’s scope, shuffled; keep

BgB\_\{g\}
3:foreach

h′∈Ph^\{\\prime\}\\in Pdo

4:write record;if

h′h^\{\\prime\}satisfies Eq\. \([1](https://arxiv.org/html/2610.10629#S5.E1)\) on

HselH\_\{\\mathrm\{sel\}\}then

h←h′h\\leftarrow h^\{\\prime\};return

5:endfor

The*local loop*\(Algorithm[1](https://arxiv.org/html/2610.10629#alg1)\) may change one parameter or one clause of the deployed primitive per candidate\. The*global loop*\(Algorithm[2](https://arxiv.org/html/2610.10629#alg2)\) may replace the primitive with another from a typed library—here, a check on the stake held through the first\-layer vehicle versus a check on the effective stake along the ownership chain—so a replacement is still a discrete, nameable change\. It has no privileged path to deployment: the same gate applies\. In the global\-only arm the loop is triggered when the visible false\-positive share of reviewed flags exceeds its pre\-shift level byτ=5\\tau=5percentage points for two consecutive windows\. Table[2](https://arxiv.org/html/2610.10629#S5.T2)lists every constant\.

Table 2:Every free parameter of the engine and the experiment, with the value used\. All are constants inevolve\.py\.

## 6The regulatory\-shift track

### 6\.1World and rules

The population is the screening\-alert queue: each case carries a name hit of one of five types \(formal designation, high\- and low\-grade adverse media, foreign and domestic politically exposed person\) and a beneficial\-ownership chain of depth one to three\. The compliance primitive returnsflagorclear\. Ground truth is a versioned rule\. BeforeTT\(rule v1\): flag when the hit type is in scope and the stake held through the first\-layer vehicle is at least 25%\. FromTT\(rule v2\) one of three re\-interpretations applies:

- •Thr\(parametric\): the ownership threshold rises to 30, 40 or 50%\.
- •Scope\(clause\): low\-grade adverse media; then all adverse media; then also domestic PEP status no longer qualify on their own\.
- •Struct\(structural\): the threshold applies to the*effective*stake, the product of stakes along the chain, with multi\-layer chains in 30, 50 or 70% of cases\. No threshold or scope edit to the first\-layer check can express this rule\.

All three make the v1 harness over\-flag, so the observable symptom is the same—a rise in false positives—while the correct repair differs in kind\.

### 6\.2Where the labels come from after a re\-interpretation

A referee observed that, as we had described it, the retrospective test was circular: the pre\-shift pool carries labels under the interpretation the shift replaced, so a correct adaptation must look like a regression on it\. The objection is correct, and the answer is a design commitment rather than a detail\. AtTTthe pool isrelabelled by applying rule v2 to the stored case facts\. In a deployment this is the compliance function encoding the supervisor’s re\-interpretation as an executable rule—nobody relabels cases by hand, but a person authors the rule\. The engine therefore searches for a harness edit autonomously while the target of the search is set by people: the adaptation is*supervised at the level of the rule and autonomous at the level of the harness*\. We position the autonomy claim accordingly\. The “gate on stale labels” arm, which keeps the v1 labels, shows empirically what happens otherwise \(Section[7\.2](https://arxiv.org/html/2610.10629#S7.SS2)\)\.

The relabelling assumes the rule encodes the re\-interpretation correctly; a mis\-encoded rule would be faithfully optimised towards\. That is the right place for the risk to sit—the rule is a short, reviewable artefact authored by an accountable function—but it is a risk\.

### 6\.3Arms

All arms see the same cases, the same noise draws and the same feedback model; what each observes depends on its own harness\.*Healthy*: no shift, v1 harness\.*Degraded*: shift, no adaptation\.*Oracle*: the correct v2 harness fromTT\.*Manual*: the correct harness after a 2,000\-case lag, standing in for an engineer who rewrites the prompt after an alert\.*No gate*: the dual loop with Eq\. \([1](https://arxiv.org/html/2610.10629#S5.E1)\) replaced by the check an unbounded self\-evolution system applies—admit if the candidate makes fewer errors visible in the current window\.*Gate on stale labels*: the dual loop withHselH\_\{\\mathrm\{sel\}\}under v1 labels\.*Global only*,*local only*, and*local only, compute\-matched*, whose per\-cycle budget is multiplied, per run, by the ratio of the dual loop’s proposals to the local\-only arm’s \(rounded up; between 2 and 2 in our runs\)\.*Dual loop*: the full engine\.

### 6\.4Metrics

On the held\-out window, withflagas the positive class: false\-positive rate=FP/\(FP\+TN\)=\\mathrm\{FP\}/\(\\mathrm\{FP\}\+\\mathrm\{TN\}\); missed\-flag rate=FN/\(FN\+TP\)=\\mathrm\{FN\}/\(\\mathrm\{FN\}\+\\mathrm\{TP\}\)and recall=1−=1\-missed\-flag rate;*regression rate*==the share of held\-out cases answered correctly by the degraded harness that the final harness answers wrongly \(a paired, case\-level measure\)\. During adaptation:*excess errors*==true errors in\[T,85%\)\[T,85\\%\)minus those of the oracle arm;*recovery window*==the first window from which the arm’s error rate stays within one percentage point of the oracle’s \(7 means never\)\. For the gate: candidates proposed, admitted and rejected by reason;*false admissions*, admitted changes that increased true error onHvalH\_\{\\mathrm\{val\}\}, which the gate never sees; and*oracle rejections*, rejections of the exactly correct harness\. Means are over seeds with 95%tt\-intervals, clipped to\[0,100\]\[0,100\]\. The held\-out window holds 3,450 alerts per run; its numbers of trueclearand trueflagcases under the post\-shift rule, which are the denominators of the two rates, are given per cell in Table[4](https://arxiv.org/html/2610.10629#S7.T4)\.

## 7Results

Table[3](https://arxiv.org/html/2610.10629#S7.T3)reports the mid\-severity cell of each family for all ten arms; Table[4](https://arxiv.org/html/2610.10629#S7.T4)the whole grid for six arms; Table[5](https://arxiv.org/html/2610.10629#S7.T5)the gate statistics over all 90 runs\.

Table 3:Mid severity, held\-out window, mean over seeds with 95%tt\-intervals\. FP and missed\-flag rates in %; intervals clipped to\[0,100\]\[0,100\]\. For the no\-gate arm the missed\-flag interval is replaced by the range \{min–max\} across seeds because its distribution is bimodal\. Regression on held\-out cases previously answered correctly, in %\. Gate: candidates proposed / admitted / false admissions audited onHvalH\_\{\\mathrm\{val\}\}, summed over seeds\.ArmFPMissed flagRegr\.Excess err\.Recov\. win\.Gate: prop\. / adm\. / falseThr shift, mid severity, 10 seedsHealthy \(no shift\)2\.0 \[1\.6, 2\.3\]1\.9 \[1\.6, 2\.1\]0\.0–––Degraded \(no adaptation\)28\.0 \[27\.3, 28\.7\]1\.9 \[1\.6, 2\.2\]0\.04987\.0–Oracle update1\.9 \[1\.6, 2\.2\]1\.9 \[1\.6, 2\.2\]0\.300\.0–Manual update \(lag 2,000\)1\.9 \[1\.6, 2\.2\]1\.9 \[1\.6, 2\.2\]0\.32954\.0–No gate \(trace check only\)1\.9 \[1\.6, 2\.2\]1\.9 \{1\.2–2\.5\}0\.31392\.5784 / 30 / 6Gate on stale labels28\.0 \[27\.3, 28\.7\]1\.9 \[1\.6, 2\.2\]0\.04987\.0948 / 0 / 0Global loop only28\.0 \[27\.3, 28\.7\]1\.9 \[1\.6, 2\.2\]0\.04987\.0360 / 0 / 0Local loop only1\.9 \[1\.6, 2\.2\]1\.9 \[1\.6, 2\.2\]0\.3811\.3566 / 18 / 0Local only, compute\-matched1\.9 \[1\.6, 2\.2\]1\.9 \[1\.6, 2\.2\]0\.3741\.01519 / 20 / 0Dual loop \(gated\)1\.9 \[1\.6, 2\.2\]1\.9 \[1\.6, 2\.2\]0\.3821\.3834 / 21 / 0Scope shift, mid severity, 10 seedsHealthy \(no shift\)2\.0 \[1\.6, 2\.3\]1\.9 \[1\.6, 2\.1\]0\.0–––Degraded \(no adaptation\)36\.9 \[36\.1, 37\.7\]1\.8 \[1\.6, 2\.1\]0\.07807\.0–Oracle update2\.0 \[1\.7, 2\.2\]1\.8 \[1\.6, 2\.1\]0\.600\.0–Manual update \(lag 2,000\)2\.0 \[1\.7, 2\.2\]1\.8 \[1\.6, 2\.1\]0\.64554\.0–No gate \(trace check only\)2\.0 \[1\.7, 2\.2\]79\.7 \{2\.1–98\.7\}38\.19406\.9536 / 68 / 50Gate on stale labels36\.9 \[36\.1, 37\.7\]1\.8 \[1\.6, 2\.1\]0\.07807\.0921 / 0 / 0Global loop only36\.9 \[36\.1, 37\.7\]1\.8 \[1\.6, 2\.1\]0\.07807\.0360 / 0 / 0Local loop only3\.7 \[0\.0, 7\.7\]1\.8 \[1\.6, 2\.1\]0\.62663\.0590 / 19 / 0Local only, compute\-matched2\.0 \[1\.7, 2\.2\]1\.8 \[1\.6, 2\.1\]0\.61111\.01306 / 20 / 0Dual loop \(gated\)2\.0 \[1\.7, 2\.2\]1\.8 \[1\.6, 2\.1\]0\.62362\.8789 / 20 / 0Struct shift, mid severity, 10 seedsHealthy \(no shift\)2\.1 \[1\.7, 2\.5\]1\.9 \[1\.7, 2\.2\]0\.0–––Degraded \(no adaptation\)27\.4 \[26\.8, 28\.0\]1\.9 \[1\.6, 2\.2\]0\.04987\.0–Oracle update2\.0 \[1\.7, 2\.3\]1\.9 \[1\.6, 2\.2\]0\.300\.0–Manual update \(lag 2,000\)2\.0 \[1\.7, 2\.3\]1\.9 \[1\.6, 2\.2\]0\.32934\.0–No gate \(trace check only\)2\.0 \[1\.7, 2\.3\]32\.7 \{1\.2–98\.1\}16\.911266\.8478 / 88 / 63Gate on stale labels27\.4 \[26\.8, 28\.0\]1\.9 \[1\.6, 2\.2\]0\.04987\.0954 / 0 / 0Global loop only2\.0 \[1\.7, 2\.3\]1\.9 \[1\.6, 2\.2\]0\.31562\.143 / 10 / 0Local loop only27\.4 \[26\.8, 28\.0\]1\.9 \[1\.6, 2\.2\]0\.04987\.0630 / 0 / 0Local only, compute\-matched27\.4 \[26\.8, 28\.0\]1\.9 \[1\.6, 2\.2\]0\.04987\.02133 / 0 / 0Dual loop \(gated\)2\.0 \[1\.7, 2\.3\]1\.9 \[1\.6, 2\.2\]0\.32283\.1878 / 10 / 0Table 4:Full grid, held\-out window: FP / missed\-flag rate in %, mean over 10 seeds\.nn: trueclear/ trueflagcases per seed under the post\-shift rule, the denominators of the two rates\.Table 5:Gate statistics over all 90 runs \(9 cells×\\times10 seeds\)\. Rejections are counted by the first clause of Eq\. \([1](https://arxiv.org/html/2610.10629#S5.E1)\) that fails \(improvement is checked first\): no significant improvement / regression onCC\.†No\-gate rejections are by the trace check\. False adm\.: admitted changes that increased true error onHvalH\_\{\\mathrm\{val\}\}\. Oracle rej\.: rejections of the exactly correct harness\. Miss\>\>10%: runs whose final missed\-flag rate exceeds 10%\. Max adm\.: largest number of admissions in one run\.### 7\.1The gate is what keeps the screen from being loosened

Every arm that recovers brings the false\-positive rate down to the oracle’s\. What separates them is the missed\-flag rate, which the earlier presentation did not report\. With the gate, the dual loop ends at the oracle’s missed\-flag rate in all three families \(Table[3](https://arxiv.org/html/2610.10629#S7.T3); 1\.9%, 1\.8% and 1\.9% against the oracle’s 1\.9%, 1\.8% and 1\.9%\), with 0 false admissions over 144 admissions in 90 runs \(Table[5](https://arxiv.org/html/2610.10629#S7.T5)\)\. Without it, the same proposers, fed the same windows, admitted 309 changes that increased true error onHvalH\_\{\\mathrm\{val\}\}; the final missed\-flag rate exceeded 10% in 49 of 90 runs, reaching a mean of 79\.7% in the mid scope cell while its false\-positive rate matched the oracle’s\. This is the failure referees anticipated: under scope and structural re\-interpretations, false positives fall because flagging stops\. Under the threshold family the visible\-failure check happens to point in the right direction—raising the threshold is the repair—and the no\-gate arm’s missed\-flag rate stays near the oracle’s\. It is not hypothetical here; it is what a loop closed on visible failures does when misses are mostly invisible, and Eq\. \([1](https://arxiv.org/html/2610.10629#S5.E1)\), evaluated on fully labelled history, is what stops it\. The no\-gate arm also churns: up to 13 admissions in a run against at most 4 for the dual loop, and more excess errors during adaptation than leaving the pipeline degraded \(940 against 780 in the mid scope cell\)\.

### 7\.2Stale labels make the gate refuse every correct change

The gate on v1 labels rejected all 8,490 of its candidates, every one for lack of significant improvement—including the exactly correct harness 470 times; every one of the 8,490 also failed the non\-regression clause— and left the pipeline at the degraded false\-positive rate in every cell\. Under the old interpretation a correct repair does not reduce errors; it creates them\. The circularity the referee identified is therefore real, and relabelling the pool by the new rule \(Section[6\.2](https://arxiv.org/html/2610.10629#S6.SS2)\) is a necessary part of the design, not an implementation choice\.

### 7\.3Division of labour between the loops

The ablation separates the loops cleanly by the kind of re\-interpretation \(Table[4](https://arxiv.org/html/2610.10629#S7.T4)\)\. Parametric \(*thr*\) and clause \(*scope*\) shifts are repaired by the local loop alone, and the global\-only arm, whose library offers only primitive replacements, cannot repair them\. The structural shift is the reverse: the local\-only arm proposed 1,890 candidates across the three structural cells and admitted 0; 1,890 were rejected for regression, because no threshold on the first\-layer stake reproduces a rule on the effective stake; the local\-only arm stays at the degraded rate, and the compute\-matched local\-only arm, with its budget scaled to the dual loop’s proposal count, does no better \(27\.4% against 27\.4% in the mid cell\)\. Repair comes only through primitive replacement\. Local\-only repair of the scope family is not perfect either: in the mid cell it recovered in 9/10 seeds\. The dual loop repairs all three families because escalation routes each problem to the loop that can express its fix\. This division of labour is, in part, true by construction—the local edit space cannot express a rule on the effective stake, and the library contains one—so what the experiment adds is that escalation finds the right loop without being told the family, and that the gate rejects the inexpressible local candidates rather than admitting the least\-bad of them\. The dual loop pays for routing in two ways\. It recovers more slowly than the right single loop on the structural family, because the local loop must stall forNNcycles before escalating \(excess errors 228 against 156 for global\-only in the mid cell\)\. And it makes extra proposals after recovery: residual noise still forms clusters, their candidates are proposed and rejected, and some escalate; none of these was admitted\.

The earlier version of this paper reported this division of labour from a single run on different metrics\. A referee suggested it was the cleaner finding; the measured version confirms the structure and replaces the anecdote\.

### 7\.4Recovery speed against a human engineer

Under our assumed 2,000\-case lag, the manual update accumulates more excess errors during adaptation than the dual loop in every family at mid severity \(Table[3](https://arxiv.org/html/2610.10629#S7.T3)\), but not everywhere: in the high\-severity structural cell, where the dual loop recovers in only half the seeds, it accumulates 500 against the manual update’s 408\. The comparison depends entirely on that lag, which is an assumption; its value is to show where a gated automatic loop helps—time to repair—not to claim the automatic loop is better than a person\.

### 7\.5Worked rejection

In the mid structural cell, seed 0, the largest visible error cluster after the shift is false positives on formal designations, and at case 16099 the local loop’s first candidate is*scope \-designation*: stop treating a designation as a qualifying hit\. It is the kind of edit a loop guided only by visible failures finds attractive, since flags it no longer raises can no longer be reviewed as false\. OnHselH\_\{\\mathrm\{sel\}\}under v2 labels it removes 287 false positives and introduces 4 among 4,561 trueclearcases, a reduction of 6\.2 points \(95% interval \[5\.5, 6\.9\]\), so the improvement clause passes\. But 864 of the 6,696 cases the deployed harness answered correctly \(12\.9%\) would turn wrong—designated owners above the threshold, now cleared\. The record, sequence 0 with hash prefix0f6978c25079chained to the genesis value, is written as*rejected: regression onCC*\. Threshold edits to the first\-layer check fail the same clause, because none expresses a rule on the effective stake\. After 21 local rejections the cluster escalates; the global loop’s first 4 candidates, effective\-stake checks at thresholds 0\.20, 0\.40, 0\.15, 0\.50, are rejected, and at case 17099 it proposes*primitive direct\_stake\_check→\\rightarroweffective\_stake\_check*, which reduces false positives by 26\.3 points \(lower bound 25\.0\) with 21 regressions onCC\(0\.31%, attributable to execution noise\), and is admitted as record 25\. Both records, and every other, are in the released results\.

### 7\.6What the tolerance costs

The gate has a second failure mode, and we report it\. In the high\-severity structural cell, replacing the primitive changes the rule’s output on a large share of cases, and the shared execution noise then turns enough of them wrong that the correct replacement exceedsϵ=0\.005\\epsilon=0\.005\. The dual loop recovered in 5/10 seeds, rejecting the exactly correct harness 24 times, and its mean false\-positive rate stayed at 18\.1% \(Table[6](https://arxiv.org/html/2610.10629#S7.T6)\)\. Doubling the tolerance toϵ=0\.010\\epsilon=0\.010, which we tested on the structural family only, recovers 10/10 seeds without any false admission\. A fixedϵ\\epsilonconflates noise with regression; a tolerance scaled to the expected noise on the cases a candidate changes would separate them, and is the next revision of the rule\. Until then,ϵ\\epsilonis a policy parameter: lower values guard more strictly against regression and, as measured here, reject more large correct changes\. No harmful change was admitted at either value, so the protective side of the trade\-off is not measured by this track\.

Table 6:Sensitivity to the non\-regression tolerance, structural family, dual loop, held\-out window\. Recovered: seeds whose final harness is exactly the correct one\. Oracle rej\.: rejections of the exactly correct harness, summed over seeds\.
### 7\.7Harness growth

A referee asked whether repeated repair accumulates special\-case rules\. In this track the edit space is parametric, so the harness’s length is bounded \(its rendered text ranged from 151 to 253 characters across all runs\); what can grow is churn\. The gated dual loop made at most 4 admissions in a run, the no\-gate arm up to 13\. With an LLM proposer that writes free text, length and rule count become the quantities to bound, and the log already records each version’s rendered text for that purpose\.

## 8Regulatory correspondence

#### The EU AI Act\.

AI systems intended to evaluate the creditworthiness of natural persons or establish their credit score are high\-risk\[[1](https://arxiv.org/html/2610.10629#bib.bib1), Annex III, point 5\(b\)\]; after the Digital Omnibus on AI the corresponding obligations apply from 2 December 2027\[[2](https://arxiv.org/html/2610.10629#bib.bib2)\]\. The provision that matters most for self\-evolution is Article 43\(4\): for high\-risk systems that continue to learn, changes*pre\-determined*by the provider at the initial conformity assessment and described in the technical documentation do not constitute a substantial modification, whereas a change not foreseen in that assessment that affects compliance does\[[1](https://arxiv.org/html/2610.10629#bib.bib1), Arts\. 3\(23\), 43\(4\)\]\. A harness\-only engine with a fixed edit space, a fixed library and a fixed admission rule is a way of making change*pre\-determinable*: the set of reachable harnesses and the rule that admits them can be written into the technical documentation before deployment\. Table[7](https://arxiv.org/html/2610.10629#S8.T7)maps the remaining obligations\.

Table 7:Obligation, mechanism, artefact, and the status of the evidence in this paper\. This is a design argument, not a compliance opinion\.
#### The US gap\.

The interagency guidance of 17 April 2026 replaced SR 11\-7 and OCC Bulletin 2011\-12, the framework our earlier presentation leaned on, and states that generative and agentic AI models are not within its scope; the agencies announced a forthcoming request for information on AI use, including generative and agentic AI\[[3](https://arxiv.org/html/2610.10629#bib.bib3),[4](https://arxiv.org/html/2610.10629#bib.bib4)\]\. We read this as strengthening rather than weakening the case for the design\. There is, as of writing, no US supervisory template for change control in agentic systems, and the guidance that does exist addresses models whose changes are versioned objects\. An architecture that turns agentic adaptation back into versioned, tested, recorded changes is a candidate answer to the question the agencies have left open\. We claim no more than that: the guidance does not endorse the design, and our mapping is to the EU text\.

Two further limits belong in this section\. Bias monitoring under Article 10 has no mechanism in our design and no evidence in our experiments, because the benchmark carries no protected attributes\. And no examiner has reviewed any artefact; “a gate a regulator could accept” is a design goal, not an obtained acceptance\.

## 9The reference run: deciding without evidence

CACE\-Bench also measures a failure mode that accuracy\-style metrics cannot see\. Each case requires several classes of evidence, each resolved through a per\-jurisdiction provider chain in which a provider may not respond; when the sources needed for the compliance conclusion did not all answer, escalation is the correct outcome\. The*silent\-decision rate*is the share of such undecidable cases on which the pipeline decided anyway\. Table[8](https://arxiv.org/html/2610.10629#S9.T8)reports the reference run \(seed 0, 23,000 cases, seven Latin American jurisdictions, provider registry 2026\-08\-03\) with the missed\-flag rate now reported alongside false positives, and the range over 5 seeds\.

Table 8:Reference run, CACE\-Bench v0\.6\.0\. Blind: availability\-blind single pass\. Judge: with the judge\-and\-recovery layer\. Seed 0 with 95% Wilson intervals\[[17](https://arxiv.org/html/2610.10629#bib.bib17)\]; denominatornn; range over seeds 0–4\.Undecidable cases are 15\.6% of the stream \(15\.6–16\.2% across seeds\), and an availability\-blind pipeline decides 55\.1% of them\. The judge\-and\-recovery layer brings that to 6\.5%\. False positives fall from 22\.5% to 5\.0%, and missed flags fall too, from 15\.3% to 3\.5%, so the reduction is not bought by flagging less\. The recovery layer, however, is a documented parametric model—each first\-pass error is detected with probability 0\.85 and corrected with probability 0\.92—so this run establishes where the failure concentrates, not how well any particular implementation repairs it\.

## 10Threats to validity

Neither the agent nor the proposer is a language model\.The agent is a deterministic rule that applies the harness, with a 2% outcome flip standing in for execution error; the proposer is a seeded search over a stated edit space, and the library contains the correct primitive\. The track measures the bound and the gate, not proposal quality; an LLM proposer will produce candidates outside this space, including free\-text edits whose regression surface is larger\. The interface accepts one, and that is the next experiment\.

Synthetic throughout\.Distributions, shift families and the feedback model \(all flags reviewed, 5% of clears audited\) are ours\. The no\-gate failure depends on misses being less visible than false positives; we believe that asymmetry is realistic, but its size is an assumption\.

Execution noise is modelled as common random numbers\.Sharing each case’s noise draw across harness versions makes paired comparison clean and isolates the gate’s response to rule changes\. Real LLM noise is not shared across versions, which would raise the noise floor of the regression clause and strengthen the case for a noise\-scaledϵ\\epsilon\(Section[7\.6](https://arxiv.org/html/2610.10629#S7.SS6)\)\.

The false\-admission audit is weak by construction\.HselH\_\{\\mathrm\{sel\}\}andHvalH\_\{\\mathrm\{val\}\}are interleaved halves of the same pre\-shift stream labelled by the same deterministic rule, so a rule\-level edit that passes on one half will almost always pass on the other\. Zero false admissions onHvalH\_\{\\mathrm\{val\}\}is therefore expected for any gated arm; the informative contrasts are against the no\-gate arm and on the post\-shift held\-out window\.

Baselines not run\.We did not run a static prompt\-optimisation baseline or direct implementations of the cited self\-evolution methods; the no\-gate arm stands in for the latter’s admission logic only\.

One primitive, one shift at a time\.Interacting primitives and concurrent shifts are not tested\.

Label source\.The relabelled pool is only as correct as the rule that relabels it\.

Fairness\.No protected attributes; no fairness claim\.

## 11Conclusion

Self\-evolution in regulated lending is acceptable only if what evolves is something a supervisor can review\. We argued that the runtime harness is that thing and the only thing, built an engine that respects the bound, and measured the control that enforces it\. On a synthetic regulatory\-shift track the gate admitted 144 of 7,449 candidates with 0 harmful admissions and recovered every low\- and mid\-severity cell, while the same loops without it admitted 550 changes, 309 of them harmful, and under scope and structural re\-interpretations loosened the screen until missed flags exceeded 10% in 49 of 90 runs\. The gate needs labels under the new rule, which makes adaptation supervised at the level of the rule; it divides work between a local and a global loop by the kind of re\-interpretation; and its fixed tolerance rejects some large correct changes, which a noise\-scaled tolerance should fix\. The proposer in these experiments is not a language model, and replacing it with one is the experiment that will show whether the bound survives contact with free text\.

## Declarations

#### Availability of data and materials\.

All code, configuration and per\-run outputs, including every gate log, are available in the CACE\-Bench repository at[https://github\.com/rav11l/cace\-bench](https://github.com/rav11l/cace-bench), release v0\.6\.0, archived on Zenodo under concept DOI[10\.5281/zenodo\.21394049](https://doi.org/10.5281/zenodo.21394049)\. The data are fully synthetic; no personal or production data were used\.

#### Use of generative AI\.

A generative AI assistant \(Anthropic Claude\) was used in preparing this work: to write and debug parts of the simulation and analysis code, to run the experiments, to draft and edit text, and to check the manuscript against the result files\. All research questions, the position argued, the experimental design decisions and the final text are the author’s responsibility\. Every number in the paper is generated by a script from the released result files rather than typed, and the numbers were independently recomputed from the raw outputs before submission\. No generative AI system is a component of the experiments reported: the agent and the proposer in the simulation are deterministic or seeded programs\.

#### Competing interests\.

The author declares no competing interests\.

#### Funding\.

This research received no external funding\.

#### Authors’ contributions\.

R\.A\. conceived the study, wrote the software, ran the experiments and wrote the manuscript\.

#### Preprint\.

This preprint has not been peer reviewed\.

## References

- \[1\]European Parliament and Council\. Regulation \(EU\) 2024/1689 laying down harmonised rules on artificial intelligence \(Artificial Intelligence Act\)\. Official Journal of the European Union, L, 12 July 2024\.
- \[2\]European Parliament and Council\. Regulation \(EU\) 2026/1744 amending Regulation \(EU\) 2024/1689 as regards the simplification of the implementation of harmonised rules on artificial intelligence \(Digital Omnibus on AI\)\. Official Journal of the European Union, L, 24 July 2026\.
- \[3\]Board of Governors of the Federal Reserve System\. SR 26\-2: Revised Guidance on Model Risk Management, with attachment “Supervisory Guidance on Model Risk Management” \(jointly with the FDIC and OCC\), 17 April 2026\.
- \[4\]Office of the Comptroller of the Currency\. Bulletin 2026\-13, Model Risk Management: Revised Interagency Guidance, and news release NR\-OCC\-2026\-29, 17 April 2026; Federal Deposit Insurance Corporation, FIL\-15\-2026, 17 April 2026\.
- \[5\]H\. Zhang et al\. Self\-Harness: Harnesses that improve themselves\. arXiv:2606\.09498, 2026\.
- \[6\]Darwin Agent Team\. HarnessX: A composable, adaptive, and evolvable agent harness foundry\. arXiv:2606\.14249, 2026\.
- \[7\]L\. A\. Agrawal et al\. GEPA: Reflective prompt evolution can outperform reinforcement learning\. arXiv:2507\.19457, 2025\.
- \[8\]C\. Fernando, D\. Banarse, H\. Michalewski, S\. Osindero, T\. Rocktäschel\. Promptbreeder: Self\-referential self\-improvement via prompt evolution\. arXiv:2309\.16797, 2023\.
- \[9\]N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, S\. Yao\. Reflexion: Language agents with verbal reinforcement learning\. In Advances in Neural Information Processing Systems, 2023\.
- \[10\]S\. Hu, C\. Lu, J\. Clune\. Automated design of agentic systems\. In International Conference on Learning Representations, 2025\.
- \[11\]X\. Yin et al\. Gödel Agent: A self\-referential agent framework for recursive self\-improvement\. arXiv:2410\.04444, 2024\.
- \[12\]J\. Zhang et al\. Darwin Gödel Machine: Open\-ended evolution of self\-improving agents\. arXiv:2505\.22954, 2025\.
- \[13\]Y\. Wang et al\. EvoAgentX: An automated framework for evolving agentic workflows\. In Proceedings of EMNLP 2025: System Demonstrations, 2025\. arXiv:2507\.03616\.
- \[14\]J\. Fang et al\. A comprehensive survey of self\-evolving AI agents: A new paradigm bridging foundation models and lifelong agentic systems\. arXiv:2508\.07407, 2025\.
- \[15\]H\. Gao et al\. A survey of self\-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence\. arXiv:2507\.21046, 2025\.
- \[16\]R\. Akhtyamov\. A three\-level quality\-metric system for LLM agents in the credit pipeline\. Vestnik Finansovykh Tekhnologiy, 2026\.
- \[17\]E\. B\. Wilson\. Probable inference, the law of succession, and statistical inference\. Journal of the American Statistical Association, 22\(158\):209–212, 1927\.