面向长时程终端Agent的Harness自进化

# Harness Self-Evolution for Long-Horizon Terminal Agents
Source: [https://arxiv.org/html/2610.27435](https://arxiv.org/html/2610.27435)
1\]Tencent HY LLM Frontier 2\]Indiana University 3\]University of Maryland, College Park 4\]University of Georgia 5\]National University of Singapore\\contribution\[∗\]This work was conducted during an internship at Tencent\.\\contribution\[†\]Internship mentor\.

Yucheng ShiZongxia LiJunyao YangZhongzhi LiYue YuHaitao MiDongruo ZhouLeoweiliangAffiliation:\[Affiliation:\[Affiliation:\[Affiliation:\[Affiliation:\[

October 19, 2026

###### Abstract

Long\-horizon terminal agents may complete a task in one run yet fail to do so consistently across repeated runs\. Stochastic generation can introduce local errors that propagate through subsequent actions and context, creating a gap between task\-solving ability and reliable execution\. The harness controls command execution, feedback, context management, and completion decisions, providing a direct way to address this gap\. The effectiveness of these mechanisms depends on the model’s behavior\. We presentRiseHarness, a framework for iteratively improving agent harnesses while keeping model weights fixed\.RiseHarnessfocuses on*flipping tasks*, which have both successful and failed runs under the same configuration\. Successful runs establish that the current system can complete the task and provide references for analyzing failures\. The analysis combines agent trajectories with internal harness traces collected by a runtime monitor, examining both the model’s actions and how the harness executes them, presents feedback, and controls the interaction\. A Harness Handbook connects this evidence to specific harness behaviors and their source code, helping identify recurring problems in mechanisms shared across tasks\. To address these problems,RiseHarnesssearches other harness frameworks for relevant implementations and uses them as design references, taking into account the current execution flow, interaction protocol, available signals, and model behavior\. Candidate changes undergo behavioral checks and repeated task evaluation; versions with measured gains become the basis for the next iteration\. This loop progressively integrates suitable mechanisms from different frameworks into a single harness and uses new execution evidence to guide further changes\. We evaluate task success and reliability across models, use ablations to examine the contributions of harness execution evidence and external research, and test transfer to unseen benchmarks with the same model\. The framework treats a specific model as the unit of harness adaptation, studying how changes to shared interaction mechanisms can make existing capabilities more reliable across tasks\.

## 1Introduction

A model that solves a long\-horizon terminal task once may fail to do so consistently\. Across many interactions, an incorrect command or a missed observation can change the environment and the context available for later decisions\. Stochastic generation can therefore lead to different outcomes even when the task and model remain unchanged\. Long\-horizon terminal benchmarks examine workflows requiring hundreds of interactions and sustained progress\[[1](https://arxiv.org/html/2610.27435#bib.bib1),[2](https://arxiv.org/html/2610.27435#bib.bib2)\]\. This distinction between reaching a successful outcome and succeeding reliably across repeated runs is central to agent evaluation\[[3](https://arxiv.org/html/2610.27435#bib.bib3),[4](https://arxiv.org/html/2610.27435#bib.bib4)\]\. We study how to narrow this gap by improving the harness around a fixed model\.

Training\-based approaches improve terminal agents through reinforcement learning and verified task synthesis\[[5](https://arxiv.org/html/2610.27435#bib.bib5),[6](https://arxiv.org/html/2610.27435#bib.bib6)\]\. The harness controls command execution, observation feedback, context management, and completion decisions\. Interface design can affect agent performance without updating model weights\[[7](https://arxiv.org/html/2610.27435#bib.bib7)\], and the harness itself has become a target for automatic improvement\. Execution traces and evaluation feedback can guide changes to harness code and support the evaluation of their effects\[[8](https://arxiv.org/html/2610.27435#bib.bib8),[9](https://arxiv.org/html/2610.27435#bib.bib9),[10](https://arxiv.org/html/2610.27435#bib.bib10)\]\. Failure\-informed runtime interventions have also been studied as a way to improve repeated success without changing model weights\[[11](https://arxiv.org/html/2610.27435#bib.bib11)\]\.

Our focus is how to connect differences between successful and failed runs to actual internal harness behavior, identify shared mechanisms to improve, and use implementations from other frameworks as references for changes suited to the current harness and model\. A difference between two trajectories does not by itself establish that a harness behavior is missing: the harness may already provide the relevant feedback, or the information needed for a proposed intervention may not be available\. Likewise, an effective design in another framework may depend on different interfaces or execution conditions\. We introduceRiseHarness, which uses execution evidence to check these conditions and to design and validate changes\. With the evaluated model held fixed, its evolution loop combines comparisons of successful and failed runs, internal execution analysis, and research into existing implementations\.

We begin with*flipping tasks*, which have both successful and failed runs under the same model and harness configuration\. Successful runs show that the current system can solve the task and provide references for analyzing failures\. Same\-task trajectory contrast and cross\-task aggregation help identify recurring behavioral problems\[[12](https://arxiv.org/html/2610.27435#bib.bib12)\]\. InRiseHarness, the contrast first identifies a behavioral need; the analysis then checks whether the current harness can support that behavior and whether it already performs it\. A runtime monitor records internal execution and aligns these harness traces with agent trajectories, exposing how proposed actions are dispatched and how feedback reaches the model\. We use the Harness Handbook\[[13](https://arxiv.org/html/2610.27435#bib.bib13)\]to connect these records to behavioral responsibilities and implementing code\. This joint analysis helps distinguish an agent’s unsuccessful response from a missing or ineffective harness mechanism and groups supported findings into targets for shared changes\.

External documentation can inform the implementation of agent capabilities\[[14](https://arxiv.org/html/2610.27435#bib.bib14)\]\.RiseHarnesssearches other frameworks, source code, and technical documentation for implementations relevant to the behavioral problems identified through execution analysis\. For completion decisions, for example, it can examine how other frameworks track unfinished commands or verify results\. The system studies the signals and interfaces these designs require, then considers the current harness’s execution flow, available information, and model behavior to decide which ideas are applicable and how to modify existing mechanisms\. The goal is to draw on useful designs from different frameworks to develop a harness whose mechanisms work together for the current model\.

Behavioral checks assess whether a candidate implements the intended change, while repeated task evaluations measure its effect on performance\. Accepted changes provide the basis for subsequent rounds; unsuccessful candidates supply execution evidence for revising the implementation or proposed mechanism\. Model weights remain fixed throughout\.

RiseHarnessevolves the harness for a specific model based on its execution behavior\. Models may require different forms of interaction support\[[15](https://arxiv.org/html/2610.27435#bib.bib15),[16](https://arxiv.org/html/2610.27435#bib.bib16)\]; we therefore conduct evolution separately for each model and examine the resulting behavioral changes\. These changes affect shared mechanisms for command execution, observation feedback, and completion\. Their benefits beyond the evolution tasks require separate validation\. Repeated evaluations and ablations assess the effects on task success and execution reliability, as well as the contributions of dual\-trace evidence and research into external implementations\. We then hold both the model and the evolved harness fixed and evaluate on benchmarks not used for evolution to test whether these behavioral improvements support a broader range of tasks\.

Our main contributions are:

- •An evidence\-driven harness evolution framework\.We introduceRiseHarness, which uses successful runs as references for failed runs and iteratively retains harness changes that improve measured performance without updating model weights\.
- •Behavioral diagnosis from paired runs and internal execution\.We combine agent and harness traces with the Harness Handbook to check proposed behavioral needs against actual execution and locate shared mechanisms for modification\.
- •Harness modification informed by external implementations\.We retrieve relevant implementations from other frameworks, examine their requirements, and use the current harness’s structure, runtime signals, and model behavior to design specific changes\.
- •Reliability and generalization\.Repeated runs and ablations assess harness evidence and external research; we compare changes across models and test cross\-benchmark transfer with the model held fixed\.

![Refer to caption](https://arxiv.org/html/2610.27435v1/figure01_workflow.png)Figure 1:Operations within a discovery round ofRiseHarness: record dual traces, identify shared behavioral problems, research and adapt mechanisms, and implement and evaluate changes\. The Handbook connects execution evidence to harness behavior and code\.
## 2Related Work

##### Long\-horizon execution and success consistency\.

Long\-horizon agents must coordinate reasoning, actions, and execution feedback across many interactions\[[17](https://arxiv.org/html/2610.27435#bib.bib17),[18](https://arxiv.org/html/2610.27435#bib.bib18)\]\. Long\-Horizon\-Terminal\-Bench and its 2\.0 extension use intermediate grading to measure progress through extended workflows\[[1](https://arxiv.org/html/2610.27435#bib.bib1),[2](https://arxiv.org/html/2610.27435#bib.bib2)\]\. Local errors can alter the environment and subsequent context, leading to different outcomes for the same model and task\. Average success alone therefore does not fully capture execution reliability\.τ\\tau\-bench measures consistency throughpassk\\mathrm\{pass\}^\{k\}, the proportion of tasks solved in allkkrepeated trials\[[3](https://arxiv.org/html/2610.27435#bib.bib3)\]; broader reliability research also examines robustness, predictability, and safety\[[4](https://arxiv.org/html/2610.27435#bib.bib4)\]\. AgentDebug diagnoses errors and supplies corrective feedback for recovery\[[19](https://arxiv.org/html/2610.27435#bib.bib19)\]; AgentRx localizes failures using constraints and evidence\-bearing validation logs\[[20](https://arxiv.org/html/2610.27435#bib.bib20)\]\. FIRE studies failure\-informed runtime interventions that improve repeated success without updating model weights\[[11](https://arxiv.org/html/2610.27435#bib.bib11)\]\. These studies motivate examining whether an agent system can consistently support successful execution, beyond whether its model can solve a task at all\. We study tasks with both successful and failed runs to assess whether harness changes can narrow this gap\.

##### Agent harnesses\.

The harness is the runtime layer that organizes model–environment interaction, including command execution, observation construction, context management, and completion decisions\. SWE\-agent shows that interface design affects task performance with a fixed model\[[7](https://arxiv.org/html/2610.27435#bib.bib7)\]\. OpenHands provides composable tools, memory management, sandboxed execution, and lifecycle control, illustrating the components of this layer\[[21](https://arxiv.org/html/2610.27435#bib.bib21)\]\. Memory, planning, and action modules also appear in agentic recommender architectures\[[22](https://arxiv.org/html/2610.27435#bib.bib22)\]\. Understanding how a harness affects execution requires both runtime observations and knowledge of its implementation\. AgentTrace records runtime activity and model interactions\[[23](https://arxiv.org/html/2610.27435#bib.bib23)\], while the Harness Handbook links behavioral responsibilities to implementing code, providing a behavior\-oriented basis for analysis and modification\[[13](https://arxiv.org/html/2610.27435#bib.bib13)\]\. HarnessBank and HarnessDev further show that the effects of harness changes can depend on the runtime model\[[15](https://arxiv.org/html/2610.27435#bib.bib15),[16](https://arxiv.org/html/2610.27435#bib.bib16)\]\. Harness design thus requires understanding both its internal mechanisms and how the model responds to them\.

Algorithm 1RiseHarness: harness self\-evolution1:Fixed model

MθM\_\{\\theta\}, initial

\(H0,K0,R0\)\(H\_\{0\},K\_\{0\},R\_\{0\}\), development tasks

𝒟\\mathcal\{D\}, round limit

TT, policy

Π\\Pi
2:Accepted harness, its Handbook, and reference evidence

3:

Next⁡\(∅\)=discover\\operatorname\{Next\}\(\\varnothing\)=\\mathrm\{discover\}\# First\-round default

4:

t←0t\\leftarrow 0;

ℒ←∅\\mathcal\{L\}\\leftarrow\\varnothing\#R0R\_\{0\}: outcomes and dual traces

5:while

t<Tt<Tand

Continue⁡\(ℒ,Π\)\\operatorname\{Continue\}\(\\mathcal\{L\};\\Pi\)do

6:if

Next⁡\(ℒ\)=discover\\operatorname\{Next\}\(\\mathcal\{L\}\)=\\mathrm\{discover\}then

7:

ℱt←Flip⁡\(Rt\)\\mathcal\{F\}\_\{t\}\\leftarrow\\operatorname\{Flip\}\(R\_\{t\}\)
8:

𝒜←⋃p∈Pair⁡\(Rt,ℱt\)Revise⁡\(Wish⁡\(Compare⁡\(p,Kt\),pa\),ph\)\\mathcal\{A\}\\leftarrow\\bigcup\\nolimits\_\{p\\in\\operatorname\{Pair\}\(R\_\{t\},\\mathcal\{F\}\_\{t\}\)\}\\operatorname\{Revise\}\(\\operatorname\{Wish\}\(\\operatorname\{Compare\}\(p,K\_\{t\}\),p^\{a\}\),p^\{h\}\)\# Check behavioral needs

9:

𝒢t←Group⁡\(𝒜,ℒ,Π\)\\mathcal\{G\}\_\{t\}\\leftarrow\\operatorname\{Group\}\(\\mathcal\{A\},\\mathcal\{L\};\\Pi\);

𝒫t←Propose⁡\(𝒢t,Ht,Kt,Rt\)\\mathcal\{P\}\_\{t\}\\leftarrow\\operatorname\{Propose\}\(\\mathcal\{G\}\_\{t\},H\_\{t\},K\_\{t\},R\_\{t\}\)\# Group and research

10:else

11:

𝒫t←Revisit⁡\(ℒ\)\\mathcal\{P\}\_\{t\}\\leftarrow\\operatorname\{Revisit\}\(\\mathcal\{L\}\)
12:endif

13:

\(Ct,KtC\)←Implement⁡\(Ht,𝒫t,ℒ,Π\)\(C\_\{t\},K\_\{t\}^\{C\}\)\\leftarrow\\operatorname\{Implement\}\(H\_\{t\},\\mathcal\{P\}\_\{t\},\\mathcal\{L\};\\Pi\);

Et←∅E\_\{t\}\\leftarrow\\varnothing\# Build and check

14:if

Ct≠⊥C\_\{t\}\\neq\\botthen

Ct←Screen⁡\(Mθ,Ct,𝒫t,Π\)C\_\{t\}\\leftarrow\\operatorname\{Screen\}\(M\_\{\\theta\},C\_\{t\},\\mathcal\{P\}\_\{t\};\\Pi\)\# Targeted trials; record dual traces

15:if

Ct≠⊥C\_\{t\}\\neq\\botthen

Et←Evaluate⁡\(Mθ,Ct,𝒟,Π\)E\_\{t\}\\leftarrow\\operatorname\{Evaluate\}\(M\_\{\\theta\},C\_\{t\},\\mathcal\{D\};\\Pi\)\# Full repeats; record dual traces

16:

\(Ht\+1,Kt\+1,Rt\+1\)←\(Ht,Kt,Rt\)\(H\_\{t\+1\},K\_\{t\+1\},R\_\{t\+1\}\)\\leftarrow\(H\_\{t\},K\_\{t\},R\_\{t\}\)
17:if

ValidComplete⁡\(Et\)\\operatorname\{ValidComplete\}\(E\_\{t\}\)and

p^​\(Et\)\>p^Bconfirm​\(𝒟\)\\widehat\{p\}\(E\_\{t\}\)\>\\widehat\{p\}\_\{B\}^\{\\,\\mathrm\{confirm\}\}\(\\mathcal\{D\}\)then

18:

\(Ht\+1,Kt\+1,Rt\+1\)←Publish⁡\(Ct,KtC,Et\)\(H\_\{t\+1\},K\_\{t\+1\},R\_\{t\+1\}\)\\leftarrow\\operatorname\{Publish\}\(C\_\{t\},K\_\{t\}^\{C\},E\_\{t\}\)\# Adopt harness and evidence

19:endif

20:

ℒ←Update⁡\(ℒ,t\)\\mathcal\{L\}\\leftarrow\\operatorname\{Update\}\(\\mathcal\{L\};t\);

t←t\+1t\\leftarrow t\+1
21:endwhile

22:return

\(Ht,Kt,Rt\)\(H\_\{t\},K\_\{t\},R\_\{t\}\)

##### Harness self\-evolution\.

Automated agent design optimizes prompts, workflows, and executable agents\[[24](https://arxiv.org/html/2610.27435#bib.bib24),[25](https://arxiv.org/html/2610.27435#bib.bib25),[26](https://arxiv.org/html/2610.27435#bib.bib26),[27](https://arxiv.org/html/2610.27435#bib.bib27),[28](https://arxiv.org/html/2610.27435#bib.bib28)\]\. Iterative response refinement is also studied in federated in\-context learning and uncertainty\-aware reasoning\[[29](https://arxiv.org/html/2610.27435#bib.bib29),[30](https://arxiv.org/html/2610.27435#bib.bib30)\]\. Verified trajectory rewriting and recursive policy refinement improve terminal models through training\[[31](https://arxiv.org/html/2610.27435#bib.bib31),[32](https://arxiv.org/html/2610.27435#bib.bib32)\]\. AHE, HarnessFix, and Meta\-Harness use execution evidence and evaluation feedback to guide harness updates\[[9](https://arxiv.org/html/2610.27435#bib.bib9),[10](https://arxiv.org/html/2610.27435#bib.bib10),[8](https://arxiv.org/html/2610.27435#bib.bib8)\]\. ModularRSI contrasts successful and failed runs, aggregates findings across tasks, and evolves modular components\[[12](https://arxiv.org/html/2610.27435#bib.bib12)\]\. Self\-Harness analyzes failures and retains fixed\-model edits through regression tests\[[33](https://arxiv.org/html/2610.27435#bib.bib33)\]\. HarnessEvolve also uses successful references for failure analysis, but generates and verifies them with access to ground\-truth answers\[[34](https://arxiv.org/html/2610.27435#bib.bib34)\]\. Related work explores design reuse\[[35](https://arxiv.org/html/2610.27435#bib.bib35)\], external documentation\[[14](https://arxiv.org/html/2610.27435#bib.bib14)\], and controls against benchmark overfitting\[[36](https://arxiv.org/html/2610.27435#bib.bib36)\]\. Evaluation studies also emphasize matched\-budget search baselines and held\-out tests\[[37](https://arxiv.org/html/2610.27435#bib.bib37)\]\. Our framework combines same\-configuration successes, dual traces, and the Handbook to check behavioral needs, then consults external implementations to design local changes\. Candidates undergo implementation checks and repeated evaluation; transfer is assessed separately with the harness held fixed\.

## 3Harness Self\-Evolution

RiseHarnessis a harness self\-evolution framework for long\-horizon agents\. It uses execution evidence and external design knowledge to iteratively improve execution reliability while keeping model weights fixed\. We first introduce the basic concepts and notation in the preliminaries\. We then describe the four components of the method—execution recording and behavior localization, problem discovery and grouping, mechanism research and adaptation, and implementation and validation—with reference to Figure[1](https://arxiv.org/html/2610.27435#S1.F1)and Algorithm[1](https://arxiv.org/html/2610.27435#alg1)\.

### 3\.1Preliminaries

LetMθM\_\{\\theta\}denote the evaluated model and𝒟\\mathcal\{D\}the development task set\. Both remain fixed throughout evolution, as do the task protocol, verifier, and acceptance rules\. Modifications are confined to harness prompts and executable interaction logic; the model proposing them may differ fromMθM\_\{\\theta\}\. At roundtt, the current harnessHtH\_\{t\}has a corresponding HandbookKtK\_\{t\}and a reference corpusRtR\_\{t\}containing outcomes and dual traces fromrt≥2r\_\{t\}\\geq 2complete repetitions per task\. The initial corpusR0R\_\{0\}is collected by repeatedly running the seed harnessH0H\_\{0\}on𝒟\\mathcal\{D\}\.

The basic analysis unit is a pairppof successful and failed runs of the same task under the same model and harness configuration\. The paired agent and harness traces are denoted bypap^\{a\}andphp^\{h\}, respectively\. A*wish*specifies the behavior expected from the harness under an observable condition\. A*finding*consists of a revised wish, assessments of whether the required information was available and the behavior had already been performed, and supporting execution evidence\. The finding set𝒜\\mathcal\{A\}is partitioned into behavioral groups𝒢\\mathcal\{G\}: findings in one group could be addressed by a shared harness change\. Each finding belongs to one group, while a single run pair may yield several findings assigned to different groups\. The selected groups are denoted by𝒢t⊆𝒢\\mathcal\{G\}\_\{t\}\\subseteq\\mathcal\{G\}\.

Let𝒫t\\mathcal\{P\}\_\{t\}denote the proposals for roundtt,CtC\_\{t\}an implemented candidate harness, andKtCK\_\{t\}^\{C\}its updated Handbook\. The candidate’s full evaluation outcomes and traces are denoted byEtE\_\{t\}\. The historyℒ\\mathcal\{L\}records proposals, results, and feedback across rounds; the policyΠ\\Pispecifies checking, screening, baseline, and stopping rules\. The aggregate pass rate of the candidate is denoted byp^​\(Et\)\\widehat\{p\}\(E\_\{t\}\), andp^Bconfirm​\(𝒟\)\\widehat\{p\}\_\{B\}^\{\\,\\mathrm\{confirm\}\}\(\\mathcal\{D\}\)denotes the confirmation baseline specified byΠ\\Pi\. These quantities determine whether a candidate is adopted, as detailed in Section[3\.5](https://arxiv.org/html/2610.27435#S3.SS5)\.

### 3\.2Execution Evidence

Improving a harness first requires distinguishing the model’s requested actions from the system’s actual behavior\. An agent trajectoryτagent\\tau^\{\\mathrm\{agent\}\}records the information the model receives, its responses, and tool interactions, but does not fully expose internal harness processing\. A requested command, for example, may not be dispatched, and terminal output may not be returned to the model in full\. The agent trajectory alone may therefore be insufficient to locate the problem\.

RiseHarnessuses a runtime monitor to record calling relationships, arguments, return values, exceptions, and source locations at selected harness function boundaries, producing a harness traceτharness\\tau^\{\\mathrm\{harness\}\}\. Run identifiers and episode indices link the two traces, allowing the analysis to follow a model request through its processing and the feedback ultimately returned\. Both traces, together with task outcomes and execution identities, are retained in the reference corpusRtR\_\{t\}\.

Function\-level records must also be connected to behavior\. Because a single harness behavior may span several functions,RiseHarnessuses a HandbookKtK\_\{t\}aligned with the current source to describe behavioral responsibilities, triggers, implementing functions, and exit conditions\[[13](https://arxiv.org/html/2610.27435#bib.bib13)\]\. These mappings connect individual calls to command execution, observation handling, context management, and completion checks\. They explain each call’s role in the overall mechanism and help locate the relevant code\.

The traces thus establish what happened, while the Handbook identifies the behavior involved and its implementation\. For a pairpp, the paired agent and harness tracespap^\{a\}andphp^\{h\}provide execution evidence\. The HandbookKtK\_\{t\}connects these records to harness behavior during analysis \(line[8](https://arxiv.org/html/2610.27435#alg1.l8)\) and supports mechanism adaptation during proposal generation \(line[9](https://arxiv.org/html/2610.27435#alg1.l9)\)\. Initial seed runs supplyR0R\_\{0\}; candidate screening and full evaluation use the same recording process \(lines[14](https://arxiv.org/html/2610.27435#alg1.l14)–[15](https://arxiv.org/html/2610.27435#alg1.l15)\), supplying new execution evidence for subsequent iterations\.

### 3\.3Behavioral Analysis

This stage identifies behavioral problems in repeated runs and groups related findings into targets for harness improvement\. We focus on flipping tasks, which have both successful and failed runs under the same model and harness configuration\. Successful runs show that the current system can complete the task and provide concrete references for examining failures\. Letyi,j​\(Ht\)∈\{0,1\}y\_\{i,j\}\(H\_\{t\}\)\\in\\\{0,1\\\}indicate whether taskiisucceeds in repetitionjjunder harnessHtH\_\{t\}\. The flipping task set inRtR\_\{t\}is

ℱt=\{i:0<∑j=1rtyi,j​\(Ht\)<rt\}\.\\mathcal\{F\}\_\{t\}=\\left\\\{i:\\ 0<\\sum\_\{j=1\}^\{r\_\{t\}\}y\_\{i,j\}\(H\_\{t\}\)<r\_\{t\}\\right\\\}\.\(1\)
For each flipping task, the system pairs successful and failed runs, examines relevant interactions, and uses their execution differences to formulate behavioral wishes\. Each wish is checked against the information available to the harness and the mechanisms it actually executed at the time\. Findings from individual pairs are then pooled to identify shared behavioral needs across tasks\. Four operations implement this process \(Algorithm[1](https://arxiv.org/html/2610.27435#alg1), lines[8](https://arxiv.org/html/2610.27435#alg1.l8)–[9](https://arxiv.org/html/2610.27435#alg1.l9)\):

The first three operations produce the finding set𝒜\\mathcal\{A\}, retaining revised wishes, assessments, and supporting evidence\. A single run pair may expose several problems and therefore yield findings assigned to different groups\.

After grouping, an independent review checks whether each common wish applies to all members and whether the groups correspond to distinct modification needs\. Review feedback guides regrouping\. Groups that remain disputed or are excluded by the historyℒ\\mathcal\{L\}do not proceed to proposal generation\. The remaining groups are ranked by distinct\-task coverage, distinct failing runs, and finding count to select𝒢t\\mathcal\{G\}\_\{t\}\. Each selected group carries its common wish and complete member evidence into the next stage of mechanism research and design\.

### 3\.4Design Exploration

This stage turns shared behavioral wishes into concrete harness modification proposals\. Given the selected groups𝒢t\\mathcal\{G\}\_\{t\}, current harnessHtH\_\{t\}, HandbookKtK\_\{t\}, and reference corpusRtR\_\{t\},Propose\\mathop\{\\text\{Propose\}\}\\nolimitsexamines implementations in other frameworks to generate proposals𝒫t\\mathcal\{P\}\_\{t\}\(line[9](https://arxiv.org/html/2610.27435#alg1.l9)\)\.

The system first uses group evidence, the Handbook, and source code to identify the behavior to improve, its current implementation, and the available runtime information\. It then translates the common wish into search questions\. The search examines how other harnesses handle similar problems, including their execution flow, maintained state, and feedback to the model\. For information loss in long command outputs, for example, it can investigate how full outputs are saved, key information is presented, and later retrieval is supported\. Retrieved implementations and their sources are retained as references for designing local changes\.

The system then designs changes suited to the current harness’s structure, interaction protocol, and model behavior observed in the traces\. Each design specifies how the borrowed ideas would affect existing mechanisms, where the required information would come from, and whether additional state or interface support is needed\. The system also examines traces from tasks that succeeded in every recorded repetition to assess whether the proposed intervention could disrupt successful execution and to refine its triggers and scope\.

The resulting proposals specify behavioral changes, implementation sites, dependencies, and additional interaction costs, along with expected outcomes and observations for testing them\. Each proposal links its execution evidence and external design sources for subsequent implementation and validation\. If no suitable external precedent is found, the system retains the search record and may propose an explicitly identified local design\.

### 3\.5Validation and Evolution

Once proposals are generated,Implement\\mathop\{\\text\{Implement\}\}\\nolimitsapplies them to the current harnessHtH\_\{t\}to build a candidateCtC\_\{t\}, together with its updated HandbookKtCK\_\{t\}^\{C\}and trace recording configuration \(Algorithm[1](https://arxiv.org/html/2610.27435#alg1), line[13](https://arxiv.org/html/2610.27435#alg1.l13)\)\. The system checks that the candidate can run and record traces\. When enabled, independent behavioral checks \(Appendix[A\.2](https://arxiv.org/html/2610.27435#A1.SS2)\) test whether each change produces the expected behavior under the specified conditions\. Problems found during checking are fed back for repair within a predefined attempt limit\. Only candidates that pass these checks proceed to task evaluation\.

Evaluation proceeds through screening followed by full validation\.Screen\\mathop\{\\text\{Screen\}\}\\nolimitsexamines task outcomes and mechanism activation on proposal\-related tasks to select a configuration for validation\.Evaluate\\mathop\{\\text\{Evaluate\}\}\\nolimitsthen collects fresh repeated runs on the development set𝒟\\mathcal\{D\}, producing outcomes and dual tracesEtE\_\{t\}\(lines[14](https://arxiv.org/html/2610.27435#alg1.l14)–[15](https://arxiv.org/html/2610.27435#alg1.l15)\)\. Adoption depends on this full evaluation: the records must be valid, the required run set complete, and the aggregate pass rate higher than the declared comparison baseline \(line[17](https://arxiv.org/html/2610.27435#alg1.l17)\)\.

On adoption,Publish\\mathop\{\\text\{Publish\}\}\\nolimitssaves the candidate and its Handbook asHt\+1H\_\{t\+1\}andKt\+1K\_\{t\+1\}, and usesEtE\_\{t\}as the next reference corpusRt\+1R\_\{t\+1\}\(line[18](https://arxiv.org/html/2610.27435#alg1.l18)\)\. Source code, mechanism settings, and recording configuration are saved with the version, keeping the implementation, behavioral description, and execution evidence aligned for the next round\. If the candidate is not adopted, the current version and its reference data are retained\.

Checking and evaluation records from each round are added to the historyℒ\\mathcal\{L\}to guide subsequent search \(line[20](https://arxiv.org/html/2610.27435#alg1.l20)\)\. Based on this evidence, the system decides whether to repair the implementation, revise proposal requirements, or try other proposals\.Revisit\\mathop\{\\text\{Revisit\}\}\\nolimitsreturns existing proposals to the same implementation and validation process \(line[11](https://arxiv.org/html/2610.27435#alg1.l11)\)\. After a new version is adopted, discovery resumes with the updated execution evidence\. Iteration ends at the round limit or another configured stopping condition\.

## 4Experiments

We evaluateRiseHarnessacross models to examine its effects on task success and execution reliability in long\-horizon terminal tasks\. We use ablations to assess the contributions of key components and evaluate evolved harnesses on benchmarks not used for evolution to examine transfer\. Section[4\.1](https://arxiv.org/html/2610.27435#S4.SS1.SSS0.Px1)describes the models, benchmarks, baselines, and evaluation protocol\. The remaining subsections address the following four research questions:

- •RQ1:Compared with the seed harness and existing baselines, canRiseHarnessimprove task success and repeated\-run reliability across models? \(Section[4\.2](https://arxiv.org/html/2610.27435#S4.SS2.SSS0.Px1)\)
- •RQ2:How does removing harness traces and the Handbook, or omitting external design research, affect harness evolution? \(Section[4\.3](https://arxiv.org/html/2610.27435#S4.SS3)\)
- •RQ3:How do retained harness changes vary across models in their behavioral targets and intervention mechanisms? \(Section[4\.4](https://arxiv.org/html/2610.27435#S4.SS4)\)
- •RQ4:With the model and evolved harness fixed, do gains in task success and reliability transfer to benchmarks not used for evolution? \(Section[4\.4](https://arxiv.org/html/2610.27435#S4.SS4)\)

### 4\.1Experimental Setup

##### Models and benchmarks\.

We evaluateRiseHarnesson GPT\-6\-sol and DeepSeek\-V4\-Flash, keeping model weights fixed throughout harness evolution\. For each model, we start from the unmodified Terminus\-2 harness and evolve it on Terminal\-Bench Hard \(TBH\), a benchmark of 100 challenging terminal tasks described by[Li et al\. \[31\]](https://arxiv.org/html/2610.27435#bib.bib31)\. Repeated runs on TBH provide both execution trajectories for behavioral analysis and task outcomes for evaluating changes and selecting harness versions\. To assess cross\-benchmark generalization, we freeze the evolved harnesses and test them on Terminal\-Bench 2\.1 \(TB2\.1\), which contains 89 tasks\[[38](https://arxiv.org/html/2610.27435#bib.bib38)\]\. TB2\.1 is used only for transfer evaluation and does not inform harness modifications\.

##### Baselines and evaluation protocol\.

We compareRiseHarnesswith the seed Terminus\-2 harness, ModularRSI\[[12](https://arxiv.org/html/2610.27435#bib.bib12)\], and AHE Harness\[[9](https://arxiv.org/html/2610.27435#bib.bib9)\]\. We also include two ablation baselines to assess the contributions of key components:

- •RiseHarnessw/o Harness Evidence:analysis uses only agent traces and source code, without harness traces or the Harness Handbook\.
- •RiseHarnessw/o External Research:analysis retains dual traces and the Handbook, but proposals are generated from local information without external research\.

FullRiseHarnessand both ablation baselines start from the same seed harness and reference runs, with matched models, editable scope, and predefined budgets\. Apart from the removed components, their procedures are identical \(Section[4\.3](https://arxiv.org/html/2610.27435#S4.SS3)\)\. We evaluate the publicly released ModularRSI harness without re\-evolution\. Each selected harness is frozen before independent repeated evaluation\. The reference\-baseline and version\-selection rules used during evolution are specified in Appendices[C\.2](https://arxiv.org/html/2610.27435#A3.SS2)and[C\.3](https://arxiv.org/html/2610.27435#A3.SS3)\.

##### Metrics and reporting\.

We measure both average task performance and repeated\-run reliability\. Unless otherwise stated, each task is evaluated inr=4r=4repeated runs\. For taskiiand repetitionjj, letyi,j​\(H\)=1y\_\{i,j\}\(H\)=1denote a pass under harnessHHandyi,j​\(H\)=0y\_\{i,j\}\(H\)=0a scored failure\. Scoring and completeness rules are specified in Appendix[C\.1](https://arxiv.org/html/2610.27435#A3.SS1)\.

For a task setSS, we report four metrics, distinguishing average success from reliability across repeated runs\[[3](https://arxiv.org/html/2610.27435#bib.bib3)\]:

- •Pass@1:the average pass rate over all tasks and repetitions, measuring performance in a single run\.
- •Passr:the fraction of tasks that pass in allrrruns, measuring repeated\-run success reliability\.
- •Pass@rr:the fraction of tasks that pass at least once inrrruns, measuring observed task coverage within a limited number of attempts\.
- •Flip:the fraction of tasks with both successful and failed runs, measuring outcome inconsistency\.

On a complete trial grid, letN=\|S\|N=\|S\|andki=∑j=1ryi,j​\(H\)k\_\{i\}=\\sum\_\{j=1\}^\{r\}y\_\{i,j\}\(H\)be the number of successful runs for taskii\. Suppressing the common harness argument, the metrics are

Pass​@​1\\displaystyle\\mathrm\{Pass@1\}=1N​r​∑i∈Ski,\\displaystyle=\\frac\{1\}\{Nr\}\\sum\_\{i\\in S\}k\_\{i\},Passr\\displaystyle\\qquad\\mathrm\{Pass\}^\{r\}=1N∑i∈S𝕀\[ki=r\],\\displaystyle=\\frac\{1\}\{N\}\\sum\_\{i\\in S\}\\mathbb\{I\}\[k\_\{i\}=r\],\(2\)Pass​@​r\\displaystyle\\mathrm\{Pass@\}r=1N∑i∈S𝕀\[ki\>0\],\\displaystyle=\\frac\{1\}\{N\}\\sum\_\{i\\in S\}\\mathbb\{I\}\[k\_\{i\}\>0\],Flip\\displaystyle\\qquad\\mathrm\{Flip\}=Pass​@​r−Passr\.\\displaystyle=\\mathrm\{Pass@\}r\-\\mathrm\{Pass\}^\{r\}\.Changes denote absolute differences between the reported rates\. Pass@1 corresponds to the aggregate pass ratep^​\(H,S,r\)\\widehat\{p\}\(H;S,r\)used for version selection\. A lower Flip alone does not imply more reliable success: a flipping task may instead become an all\-failure task\. We therefore interpret Flip together with Passr\. Aggregate category changes do not establish which initially stable tasks have regressed; that requires per\-task comparisons\. During evolution, aggregate pass rate determines whether a change is adopted; reliability is reported separately\.

### 4\.2Performance and Reliability

##### Overall performance\.

Table[1](https://arxiv.org/html/2610.27435#S4.T1)summarizes task performance forRiseHarnessand the comparison baselines\. Relative to the seed Terminus\-2 harness,RiseHarnessimproves both average success and repeated\-run reliability across the two models, with higher Pass@1 and Pass4\.

For GPT\-6\-sol,RiseHarnessimproves Pass@1 by 5\.25% and 3\.09% on TBH and TB2\.1, respectively, while increasing the all\-success task fraction on both benchmarks\. For DeepSeek\-V4\-Flash,RiseHarnessimproves TBH Pass@1 by 7\.25% and Pass4by 14%\. Pass@4 increases only modestly in comparison, suggesting that the main gain is in repeated\-run success reliability rather than a large expansion in observed task coverage\.

Table 1:Performance and reliability \(r=4r=4; %\)\. Coral annotations show absolute changes from Terminus\-2 forRiseHarnesswithin each model and benchmark\. Higher pass metrics and lower Flip are better\. Dashes denote unreported results\. DeepSeek uses the final four\-modification version on TBH and the best observed three\-modification version on TB2\.1\. GPT\-6\-sol trial counts appear in Appendix[D\.1](https://arxiv.org/html/2610.27435#A4.SS1)\.Figure 2:DeepSeek task outcomes; labels rounded\.Figure 3:DeepSeek\-V4\-Flash evolution; stars mark peaks\. Y\-axis ranges differ\.
Figure 4:Model\-specific interventions mapped to Handbook behaviors\. GPT\-6\-sol modifies session recovery, observation construction, and completion context; DeepSeek\-V4\-Flash adds completion\-evidence checks\. Details appear in Appendix[B](https://arxiv.org/html/2610.27435#A2)\.
##### Performance across iterations\.

Figures[3](https://arxiv.org/html/2610.27435#S4.F3)and[3](https://arxiv.org/html/2610.27435#S4.F3)show task outcome distributions and pass rates, respectively, as four mechanisms for completion confirmation accumulate for DeepSeek\-V4\-Flash\. Each version retains the preceding changes and adds one mechanism; their definitions and triggers appear in Appendix[B](https://arxiv.org/html/2610.27435#A2)\. On TBH, despite fluctuations among intermediate versions, the final version achieves the highest average pass rate and all\-success task fraction, while reducing both flipping and all\-failure tasks relative to the seed harness\. These results show that changes to shared interaction behaviors can yield improvements in performance and reliability when combined and evaluated iteratively\.

The gains also extend to TB2\.1, which was not used for evolution\. Directly applying the final version selected on TBH improves Pass@1 by 3\.09% and Pass4by 6\.74% over Terminus\-2, showing that the benefits are not confined to the tasks that produced the proposals\. Version rankings differ between benchmarks: the three\-mechanism version achieves higher observed performance on TB2\.1, whereas the four\-mechanism final version performs best on TBH\. Overall, the results show thatRiseHarnessimproves success and reliability on the evolution tasks and also benefits the same model on a benchmark not used for evolution\.

Figure 5:DeepSeek ablations on TBH\. HE: harness evidence; ER: external research\. Separate y\-axis ranges\.Figure 6:TB2\.1 performance \(r=4r=4\)\. DeepSeek uses its best observed \+ OO version\. Separate y\-axis ranges\.

### 4\.3Component Contributions

We use two ablations to examine the contributions of harness execution evidence and external design research to harness evolution for DeepSeek\-V4\-Flash\. Apart from the removed component, the analysis, implementation, and evaluation procedures remain unchanged\.

##### Execution evidence\.

Figure[6](https://arxiv.org/html/2610.27435#S4.F6)compares fullRiseHarnesswith the variant without harness traces or the Handbook\. The ablated variant retains access to agent traces and source code and follows the same external research procedure\. On TBH, the full method achieves Pass@1 and Pass4gains of 7% and 8%, respectively, indicating that harness traces and the Handbook provide useful information beyond agent traces and source code\. On TB2\.1, the full method also outperforms the ablated variant on both metrics, supporting the joint contribution of these two sources of information to the evolved harness\.

##### External research\.

Figure[6](https://arxiv.org/html/2610.27435#S4.F6)compares fullRiseHarnesswith the variant that generates proposals using only local information\. Both variants use dual traces and the Handbook; they differ in whether they search for external designs and implementations\. On TBH, the full method improves Pass@1 and Pass4by 5\.50% and 6%, respectively, while achieving higher Pass@4 and lower Flip\. On TB2\.1, the full method also maintains higher average success and all\-success task fractions\. These results suggest that external research usefully complements local execution analysis, with benefits on both the evolution tasks and tasks not used for evolution\.

### 4\.4Adaptation and Generalization

##### Model\-specific behavioral requirements\.

The retained changes differ across models\. DeepSeek\-V4\-Flash strengthens the evidence required for completion, whereas GPT\-6\-sol emphasizes execution continuity and feedback through session recovery, workspace diffs, and prior\-work evidence\. Figure[4](https://arxiv.org/html/2610.27435#S4.F4)locates these changes; Appendix[B](https://arxiv.org/html/2610.27435#A2)describes their mechanisms\.

Figure 7:Model\-adapted vs\. swapped final harnesses \(r=4r=4\)\. GPT: GPT\-6\-sol; DeepSeek: DeepSeek\-V4\-Flash, using its final \+ IR harness\. Y\-axes start above zero\.Figure[7](https://arxiv.org/html/2610.27435#S4.F7)compares each model’s final harness with the harness evolved for the other model\. Both models perform better with their own harnesses on both benchmarks \(full results in Appendix[D\.4](https://arxiv.org/html/2610.27435#A4.SS4)\)\.

##### Cross\-benchmark generalization\.

We freeze the harnesses evolved on TBH and evaluate them on TB2\.1 with the execution model unchanged\. Figure[6](https://arxiv.org/html/2610.27435#S4.F6)compares performance with Terminus\-2\. Both models retain gains in average success and all\-success task fraction on TB2\.1; the direct transfer of DeepSeek’s final \+ IR version is reported in Appendix[D\.2](https://arxiv.org/html/2610.27435#A4.SS2)\. Together, the transfer experiments show that harness improvements can extend to unseen tasks while remaining sensitive to the model executing them\.

## 5Conclusion

We presentedRiseHarness, a framework that iteratively improves the harness to increase task success and execution reliability for long\-horizon terminal agents while keeping model weights fixed\. Starting from flipping tasks, it contrasts successful and failed runs under the same configuration and combines agent trajectories, internal harness traces, and a Harness Handbook to connect execution problems to specific interaction behaviors and their implementations\. It then searches other harness frameworks for relevant designs and develops proposals suited to the current execution flow, available signals, and model behavior\. Implementation checks and repeated evaluation determine whether a change is retained\. Accepted versions, together with their updated Handbooks and execution evidence, become the basis for subsequent iterations\.

Our experiments show that harness evolution can improve average task performance and increase the fraction of tasks solved consistently across repeated runs\. Ablations further support the contributions of harness execution evidence and external design research: the former exposes internal processing that agent trajectories alone may not reveal, while the latter provides design options beyond local analysis\. The changes retained for different models target different behaviors and use different interventions\. Swapping the evolved harnesses between models reduces performance, supporting model\-specific harness adaptation\. At the same time, frozen harnesses retain gains on a benchmark not used for evolution, demonstrating the transfer value of improvements to interaction mechanisms shared across tasks\.

These findings show that systematic harness analysis and evolution can help models use their existing capabilities more reliably, alongside improvements to the models themselves\. Future work will examine the approach across more models, task distributions, and agent frameworks, and explore ways to reduce the evaluation cost of harness evolution\.

## References

- \[1\]Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang, Junyao Yang, Zhichao Liu, Xiyang Wu, Anhao Li, Chengsong Huang, Ergan Shang, Yue Yu, Ninghao Liu, Lichao Sun, Haotao Mi, and Leowei Liang\.Long\-Horizon\-Terminal\-Bench: Testing the Limits of Agents on Long\-Horizon Terminal Tasks with Dense Reward\-Based Grading\.*arXiv preprint arXiv:2607\.08964*, 2026a\.URL[https://arxiv\.org/abs/2607\.08964](https://arxiv.org/abs/2607.08964)\.
- \[2\]Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang, Junyao Yang, Chengsong Huang, Zhichao Liu, Xiyang Wu, Anhao Li, Yue Yu, Ergan Shang, Fuxiao Liu, Tong Zheng, Ninghao Liu, Lichao Sun, Haitao Mi, and Leowei Liang\.Long\-Horizon\-Terminal\-Bench 2\.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress\.*arXiv preprint arXiv:2610\.14146*, 2026b\.URL[https://arxiv\.org/abs/2610\.14146](https://arxiv.org/abs/2610.14146)\.
- \[3\]Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan\.τ\\tau\-bench: A benchmark for tool\-agent\-user interaction in real\-world domains\.*arXiv preprint arXiv:2406\.12045*, 2024\.URL[https://arxiv\.org/abs/2406\.12045](https://arxiv.org/abs/2406.12045)\.
- \[4\]Stephan Rabanser, Sayash Kapoor, Peter Kirgis, Kangheng Liu, Saiteja Utpala, and Arvind Narayanan\.Towards a science of AI agent reliability\.*arXiv preprint arXiv:2602\.16666*, 2026\.URL[https://arxiv\.org/abs/2602\.16666](https://arxiv.org/abs/2602.16666)\.
- \[5\]Junyao Yang, Yucheng Shi, Zhongzhi Li, Ruhan Wang, Zongxia Li, Haitao Mi, and Leowei Liang\.T1: Terminal Agent Reinforcement Learning for Long\-Horizon Tasks\.*arXiv preprint arXiv:2609\.11042*, 2026a\.URL[https://arxiv\.org/abs/2609\.11042](https://arxiv.org/abs/2609.11042)\.
- \[6\]Zhongzhi Li, Yucheng Shi, Zongxia Li, Ruhan Wang, Anhao Li, Zixun Huang, Junyao Yang, Lei Ke, Ninghao Liu, Haitao Mi, and Leowei Liang\.Recursive Synthesis for Long\-Horizon Terminal Tasks\.*arXiv preprint arXiv:2608\.05466*, 2026c\.URL[https://arxiv\.org/abs/2608\.05466](https://arxiv.org/abs/2608.05466)\.
- \[7\]John Yang, Carlos E\. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press\.SWE\-agent: Agent\-computer interfaces enable automated software engineering\.*arXiv preprint arXiv:2405\.15793*, 2024\.URL[https://arxiv\.org/abs/2405\.15793](https://arxiv.org/abs/2405.15793)\.
- \[8\]Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn\.Meta\-Harness: End\-to\-end optimization of model harnesses\.*arXiv preprint arXiv:2603\.28052*, 2026\.URL[https://arxiv\.org/abs/2603\.28052](https://arxiv.org/abs/2603.28052)\.
- \[9\]Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Zhiheng Xi, Xuanjing Huang, Hang Yan, Zhenhua Han, Tao Gui, and Yu\-Gang Jiang\.Agentic harness engineering: Observability\-driven automatic evolution of coding\-agent harnesses\.*arXiv preprint arXiv:2604\.25850*, 2026\.URL[https://arxiv\.org/abs/2604\.25850](https://arxiv.org/abs/2604.25850)\.
- \[10\]Mengzhuo Chen, Junjie Wang, Zhe Liu, Yawen Wang, Haiming Zheng, and Qing Wang\.From failed trajectories to reliable LLM agents: Diagnosing and repairing harness flaws\.*arXiv preprint arXiv:2606\.06324*, 2026\.URL[https://arxiv\.org/abs/2606\.06324](https://arxiv.org/abs/2606.06324)\.
- \[11\]Nikita Agarwal and Nivedit Jain\.FIRE: Failure\-informed runtime engineering for reliable language\-model agents\.*arXiv preprint arXiv:2609\.26048*, 2026\.URL[https://arxiv\.org/abs/2609\.26048](https://arxiv.org/abs/2609.26048)\.
- \[12\]Siwei Wu, Jincheng Ren, Yizhi Li, Haau\-Sing Li, Chengran Yang, Xudong Ma, Yuxuan Zhang, Weicheng Gu, Jian Yang, Riza Batista\-Navarro, Chuanyi Zhang, Xianglong Liu, Ming Zhou, Bryan Dai, and Chenghua Lin\.ModularRSI: Modular and generalizable recursive harness self\-improvement\.*arXiv preprint arXiv:2609\.14857*, 2026a\.URL[https://arxiv\.org/abs/2609\.14857](https://arxiv.org/abs/2609.14857)\.
- \[13\]Ruhan Wang, Yucheng Shi, Zongxia Li, Zhongzhi Li, Yue Yu, Junyao Yang, Kishan Panaganti, Haitao Mi, Dongruo Zhou, and Leoweiliang\.Harness handbook: Making evolving agent harnesses readable, navigable, and editable\.*arXiv preprint arXiv:2607\.13285*, 2026a\.URL[https://arxiv\.org/abs/2607\.13285](https://arxiv.org/abs/2607.13285)\.
- \[14\]Zihao Cheng, Zeming Liu, Yingyu Shan, Xinyi Wang, Xiangrong Zhu, Yunpu Ma, Hongru Wang, Yuhang Guo, Wei Lin, and Yunhong Wang\.Mem2Evolve: Towards self\-evolving agents via co\-evolutionary capability expansion and experience distillation\.*arXiv preprint arXiv:2604\.10923*, 2026\.URL[https://arxiv\.org/abs/2604\.10923](https://arxiv.org/abs/2604.10923)\.
- \[15\]Xiaotian Luo, Dizhan Xue, Fengxingyu Wang, Chuanrui Hu, and Yafeng Deng\.HarnessBank: Semantic gene\-bank search with gated verification for agent\-harness self\-evolution\.*arXiv preprint arXiv:2607\.13683*, 2026\.URL[https://arxiv\.org/abs/2607\.13683](https://arxiv.org/abs/2607.13683)\.
- \[16\]Yuhao Wu, Jingyuan Zhang, Jiajun Shi, Xinping Lei, Qingshui Gu, Yuxuan Zhang, Zexuan Wang, Chen He, Chen Huang, Maojia Song, Zhiyuan Zeng, Shaowen Wang, Jinkai Liu, Yunfeng Shi, Jiaheng Liu, Shen Yan, Wenhao Huang, Ge Zhang, and Wenxuan Zhang\.HarnessDev: Can LLMs create and evolve their own agent harness?*arXiv preprint arXiv:2609\.01437*, 2026b\.URL[https://arxiv\.org/abs/2609\.01437](https://arxiv.org/abs/2609.01437)\.
- \[17\]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao\.ReAct: Synergizing reasoning and acting in language models\.*arXiv preprint arXiv:2210\.03629*, 2022\.URL[https://arxiv\.org/abs/2210\.03629](https://arxiv.org/abs/2210.03629)\.
- \[18\]Mike A\. Merrill, Alexander G\. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E\. Kelly Buchanan, et al\.Terminal\-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces\.*arXiv preprint arXiv:2601\.11868*, 2026\.URL[https://arxiv\.org/abs/2601\.11868](https://arxiv.org/abs/2601.11868)\.
- \[19\]Kunlun Zhu, Zijia Liu, Bingxuan Li, Muxin Tian, Yingxuan Yang, Jiaxun Zhang, Pengrui Han, Qipeng Xie, Fuyang Cui, Weijia Zhang, Xiaoteng Ma, Xiaodong Yu, Gowtham Ramesh, Jialian Wu, Zicheng Liu, Pan Lu, James Zou, and Jiaxuan You\.Where LLM agents fail and how they can learn from failures\.*arXiv preprint arXiv:2509\.25370*, 2025\.URL[https://arxiv\.org/abs/2509\.25370](https://arxiv.org/abs/2509.25370)\.
- \[20\]Shraddha Barke, Arnav Goyal, Alind Khare, Avaljot Singh, Suman Nath, and Chetan Bansal\.AgentRx: Diagnosing AI agent failures from execution trajectories\.*arXiv preprint arXiv:2602\.02475*, 2026\.URL[https://arxiv\.org/abs/2602\.02475](https://arxiv.org/abs/2602.02475)\.
- \[21\]Xingyao Wang, Simon Rosenberg, Juan Michelini, Calvin Smith, Hoang Tran, Engel Nyst, Rohit Malhotra, Xuhui Zhou, Valerie Chen, Robert Brennan, and Graham Neubig\.The OpenHands software agent SDK: A composable and extensible foundation for production agents\.*arXiv preprint arXiv:2511\.03690*, 2025a\.URL[https://arxiv\.org/abs/2511\.03690](https://arxiv.org/abs/2511.03690)\.
- \[22\]Chengkai Huang, Junda Wu, Yu Xia, Zixu Yu, Ruhan Wang, Tong Yu, Ruiyi Zhang, Ryan A\. Rossi, Branislav Kveton, Dongruo Zhou, Julian McAuley, and Lina Yao\.Towards Agentic Recommender Systems in the Era of Multimodal Large Language Models\.*arXiv preprint arXiv:2503\.16734*, 2025\.URL[https://arxiv\.org/abs/2503\.16734](https://arxiv.org/abs/2503.16734)\.
- \[23\]Adam AlSayyad, Kelvin Yuxiang Huang, and Richik Pal\.AgentTrace: A structured logging framework for agent system observability\.*arXiv preprint arXiv:2602\.10133*, 2026\.URL[https://arxiv\.org/abs/2602\.10133](https://arxiv.org/abs/2602.10133)\.
- \[24\]Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T\. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts\.DSPy: Compiling declarative language model calls into self\-improving pipelines\.*arXiv preprint arXiv:2310\.03714*, 2023\.URL[https://arxiv\.org/abs/2310\.03714](https://arxiv.org/abs/2310.03714)\.
- \[25\]Lakshya A\. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl\-Ong, Arnav Singhvi, Herumb Shandilya, Michael J\. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G\. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab\.GEPA: Reflective prompt evolution can outperform reinforcement learning\.*arXiv preprint arXiv:2507\.19457*, 2025\.URL[https://arxiv\.org/abs/2507\.19457](https://arxiv.org/abs/2507.19457)\.
- \[26\]Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu\.AFlow: Automating agentic workflow generation\.*arXiv preprint arXiv:2410\.10762*, 2024\.URL[https://arxiv\.org/abs/2410\.10762](https://arxiv.org/abs/2410.10762)\.
- \[27\]Shengran Hu, Cong Lu, and Jeff Clune\.Automated design of agentic systems\.*arXiv preprint arXiv:2408\.08435*, 2024\.URL[https://arxiv\.org/abs/2408\.08435](https://arxiv.org/abs/2408.08435)\.
- \[28\]Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune\.Darwin Gödel machine: Open\-ended evolution of self\-improving agents\.*arXiv preprint arXiv:2505\.22954*, 2025\.URL[https://arxiv\.org/abs/2505\.22954](https://arxiv.org/abs/2505.22954)\.
- \[29\]Ruhan Wang, Zhiyong Wang, Chengkai Huang, Rui Wang, Tong Yu, Lina Yao, John C\. S\. Lui, and Dongruo Zhou\.Federated In\-Context Learning: Iterative Refinement for Improved Answer Quality\.*arXiv preprint arXiv:2506\.07440*, 2025b\.URL[https://arxiv\.org/abs/2506\.07440](https://arxiv.org/abs/2506.07440)\.
- \[30\]Ruhan Wang, Chengkai Huang, Zhiyong Wang, Junda Wu, Rui Wang, Tong Yu, Julian McAuley, Lina Yao, and Dongruo Zhou\.FERA: Uncertainty\-Aware Federated Reasoning for Large Language Models\.*arXiv preprint arXiv:2605\.10082*, 2026b\.URL[https://arxiv\.org/abs/2605\.10082](https://arxiv.org/abs/2605.10082)\.
- \[31\]Zongxia Li, Yucheng Shi, Zhongzhi Li, Junyao Yang, Ruhan Wang, Chengsong Huang, Fuxiao Liu, Haitao Mi, Jordan Boyd\-Graber, and LeoweiLiang\.Scaling trajectories for complex tasks through recursive self\-rewrite\.*arXiv preprint arXiv:2610\.02826*, 2026d\.URL[https://arxiv\.org/abs/2610\.02826](https://arxiv.org/abs/2610.02826)\.
- \[32\]Junyao Yang, Zongxia Li, Yucheng Shi, Zhongzhi Li, Ruhan Wang, Ruiwen Zhou, Haitao Mi, Shuicheng Yan, and Leowei Liang\.Terminal\-RSI: Recursive Self\-Improving Terminal Agents via Recursive Self\-Rewrite and Policy\-Refine\.*arXiv preprint arXiv:2610\.22192*, 2026b\.URL[https://arxiv\.org/abs/2610\.22192](https://arxiv.org/abs/2610.22192)\.
- \[33\]Hangfan Zhang, Shao Zhang, Kangcong Li, Chen Zhang, Yang Chen, Yiqun Zhang, Lei Bai, and Shuyue Hu\.Self\-Harness: Harnesses that improve themselves\.*arXiv preprint arXiv:2606\.09498*, 2026\.URL[https://arxiv\.org/abs/2606\.09498](https://arxiv.org/abs/2606.09498)\.
- \[34\]Wen Jiang, Mingmin Chu, Yimeng Tian, Qianxin Zhang, Haofei Yang, Rui Yang, Yang Liu, Tao Lv, and Fangming Li\.HarnessEvolve: Learning from reference trajectories for reliable agent self\-evolution\.*arXiv preprint arXiv:2609\.00829*, 2026\.URL[https://arxiv\.org/abs/2609\.00829](https://arxiv.org/abs/2609.00829)\.
- \[35\]Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li\.AgentSquare: Automatic LLM agent search in modular design space\.*arXiv preprint arXiv:2410\.06153*, 2024\.URL[https://arxiv\.org/abs/2410\.06153](https://arxiv.org/abs/2410.06153)\.
- \[36\]Peng Xia, Rujun Han, Zifeng Wang, Yanfei Chen, Yufan Zhuang, Yoonho Lee, Chengsong Huang, Han Yu, Zhongying CuiZhu, Yifei Ming, Huaxiu Yao, Burak Gokturk, Tomas Pfister, and Chen\-Yu Lee\.RRSI: Regularized recursive self\-improvement of agent harnesses\.*arXiv preprint arXiv:2609\.24972*, 2026\.URL[https://arxiv\.org/abs/2609\.24972](https://arxiv.org/abs/2609.24972)\.
- \[37\]Yike Wang, Huaisheng Zhu, Zhengyu Hu, Yige Yuan, Zhengyu Chen, Shakti Senthil, Hannaneh Hajishirzi, Yulia Tsvetkov, Noah A\. Smith, Pradeep Dasigi, and Teng Xiao\.Rethinking the evaluation of harness evolution for agents\.*arXiv preprint arXiv:2607\.12227*, 2026c\.URL[https://arxiv\.org/abs/2607\.12227](https://arxiv.org/abs/2607.12227)\.
- \[38\]The Terminal\-Bench Team\.Terminal\-Bench 2\.1\.Official benchmark release announcement, 2026\.URL[https://www\.tbench\.ai/news/terminal\-bench\-2\-1](https://www.tbench.ai/news/terminal-bench-2-1)\.Accessed October 19, 2026\.
- \[39\]Ahmad Faiz, Sotaro Kaneda, Ruhan Wang, Rita Osi, Prateek Sharma, Fan Chen, and Lei Jiang\.LLMCarbon: Modeling the End\-to\-End Carbon Footprint of Large Language Models\.In*International Conference on Learning Representations*, 2024\.URL[https://arxiv\.org/abs/2309\.14393](https://arxiv.org/abs/2309.14393)\.
- \[40\]Junyao Yang, Yucheng Shi, Zongxia Li, Zhongzhi Li, Ruhan Wang, Xiangxin Zhou, Kishan Panaganti, Haitao Mi, and Leowei Liang\.Stale but Stable: Staleness\-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning\.*arXiv preprint arXiv:2607\.18722*, 2026c\.URL[https://arxiv\.org/abs/2607\.18722](https://arxiv.org/abs/2607.18722)\.

## Appendix Contents

## Appendix ASystem Implementation

This appendix supplements Sections[3\.2](https://arxiv.org/html/2610.27435#S3.SS2)–[3\.5](https://arxiv.org/html/2610.27435#S3.SS5)and Algorithm[1](https://arxiv.org/html/2610.27435#alg1)\. Model\-specific mechanisms, evaluation rules, and supplementary results follow in Appendices[B](https://arxiv.org/html/2610.27435#A2),[C](https://arxiv.org/html/2610.27435#A3), and[D](https://arxiv.org/html/2610.27435#A4), respectively\. Worked examples for both models and saved prompt excerpts appear in Sections[A\.4](https://arxiv.org/html/2610.27435#A1.SS4)and[A\.5](https://arxiv.org/html/2610.27435#A1.SS5)\.

### A\.1Evidence and Versioning

Each reference corpusRtR\_\{t\}contains task outcomes and paired agent and harness traces under the current harnessHtH\_\{t\}\. Run identifiers and episode indices connect model requests to the internal calls that process them and the observations returned to the model\. The monitor records selected function boundaries, including arguments, returns, exceptions, caller relationships, and source locations\. The HandbookKtK\_\{t\}maps these records to behavioral responsibilities and implementing functions\. The two sources are complementary: traces describe observed execution, while the Handbook supplies the behavioral structure used to interpret it\.

A candidate comprises source changes, mechanism settings, an updated Handbook, and a compatible recording configuration\. Smoke checks establish that it can run and record traces\. On adoption, these artifacts are saved together with the full evaluation outcomes and tracesEtE\_\{t\}\. They become the next harness, Handbook, and reference corpus, keeping subsequent analysis tied to the accepted implementation\. Rejection preserves the current version\.

### A\.2Behavioral Validation

When independent behavioral checking is enabled, each implemented mechanism has a contract describing its trigger conditions, observable effects, relevant hooks and state, and runtime setting\. Dependencies, replacements, and conflicts specify which mechanisms may be enabled together\. These checks concern the implementation of a proposed behavior; task evaluation measures its effect on performance\.

Contracts include positive and negative cases and recovery cases where applicable\. For an inputxx, leto−​\(x\)o^\{\-\}\(x\)denote the expected incumbent behavior ando\+​\(x\)o^\{\+\}\(x\)the expected modified behavior\. An independent observer first checks the incumbent expectation before the contract is frozen\. The implementation is then checked in three configurations:

O⁡\(Ht,x\)\\displaystyle O\(H\_\{t\},x\)=o−​\(x\),\\displaystyle=o^\{\-\}\(x\),\(3\)O⁡\(Ct,new mechanisms disabled,x\)\\displaystyle O\(C\_\{t\},\\text\{new mechanisms disabled\},x\)=o−​\(x\),\\displaystyle=o^\{\-\}\(x\),O⁡\(Ct,mechanism and dependencies enabled,x\)\\displaystyle O\(C\_\{t\},\\text\{mechanism and dependencies enabled\},x\)=o\+​\(x\)\.\\displaystyle=o^\{\+\}\(x\)\.The disabled configuration checks preservation of incumbent behavior, while the enabled configuration checks the intended intervention\. Observed results are compared with the frozen expectations; a process crash cannot establish successful handling of a negative case\. Failures are returned for repair within the configured attempt limit\. These finite cases do not guarantee correctness on all tasks or isolate a mechanism that executes with dependencies\.

### A\.3Iteration Management

Each round links proposals, checks, outcomes, and execution evidence to the candidate\. The historyℒ\\mathcal\{L\}guides implementation repair, proposal revision, alternative proposals, or renewed discovery after adoption\. The proposing model may differ from the fixed evaluated modelMθM\_\{\\theta\}\.

The round limitTTand policyΠ\\Pibound this process\. Candidates must pass the configured checks before evaluation; adoption requires a valid, complete full evaluation under the aggregate pass\-rate rule in Appendix[C\.2](https://arxiv.org/html/2610.27435#A3.SS2)\. Screening alone cannot promote a candidate\.

### A\.4Case Studies

The following examples reproduce fields from the saved GPT\-6\-sol and DeepSeek\-V4\-Flash records\. Each example follows one finding through its group, external research, proposal, implementation report, and evaluation receipts\. The displayed fields are copied in full, without rewriting their values; LaTeX escaping and line wrapping affect presentation only\. Other fields, including long repeated call indexes, are omitted\. File names and JSON pointers identify the excerpts\. Internal run numbers, join keys, and opaque task identifiers are omitted from the reading copy\. The underlying complete records and their SHA\-256 hashes are retained with the source package\. These are the pipeline’s contemporaneous outputs, including its hypotheses and limitations, rather than retrospective explanations of why a task failed\. Source\-local behavior codes and function names are preserved inside quotations\.

#### A\.4\.1GPT\-6\-sol

This example concerns an HTTP scheduling service\. A boundary\-time probe fails, but the agent later declares the changes verified\.

01 Agent and harness recordGPT\-6\-solSource:compare2\.json/episodes/failing~18/agent;/episodes/failing~18/harness\_calls/35Agent analysis \(failing/8\): Yes\. The fixes are in place and verified: UTC parsing works under a non\-UTC timezone, the exact 86,400\-second boundary is accepted, and ‘POST /schedule‘ returns ‘200 OK‘ with a Unix timestamp\. The service remains listening on 127\.0\.0\.1:8080\.Recorded harness call \(failing/8\):function:agents/terminus\_2/terminus\_2\.Terminus2\.\_execute\_commandsoutput: New Terminal Output:root@e56572f2\-a153\-412c\-b305\-210e6edf55b0:/app/bash\-http\-scheduler\#

02 Pair\-level wishGPT\-6\-solSource:readings/suggest2\.wishes\.json/wishes/1asks\_the\_agent:truestatementWhen asking the agent to confirm completion, the harness should surface an unresolved failed probe from earlier in the run alongside the confirmation question rather than presenting only an idle terminal prompt, so the earlier counterexample remains visible at the last reversible moment\.observableA reply has first asserted completion and the harness is about to issue its confirmation prompt; its earlier terminal captures include failed probe output not followed by a successful probe of the same input\.because1\.side: failing\|turn: 3\|quote: 24:00: date: invalid date ‘24:00’2\.side: failing\|turn: 8quote: Current terminal state: New Terminal Output:root@e56572f2\-a153\-412c\-b305\-210e6edf55b0:/app/bash\-http\-scheduler\#3\.side: passing\|turn: 5quote: Add a small form decoder in the HTTP script, normalize ‘24:00‘ in the processor, then test encoded midnight, ‘24:00‘, and a future UTC time\.

03 Revision against harness executionGPT\-6\-solSource:readings/suggest2\.revisions\.json/revisions/1can: no, the harness has no way to obtain what this needsalready: no\|at:foundThe earlier probe appears in terminal captures, but get\_incremental\_output only diffs pane content, \_limit\_output\_length only checks length, and parse\_response reads the JSON structure rather than the meaning of terminal failures\. At confirmation, \_execute\_commands returns an idle prompt and the agent receives a generic question\. No recorded call classifies a probe as failed on a required input or determines whether a later test resolved the same failure; the persistent pane text alone cannot establish that status\.statementAt the confirmation moment, the harness should bring an earlier unresolved failure of an explicitly required input back into view instead of showing only the idle terminal and a generic confirmation question\.observableThe agent first claims completion and the harness is about to ask for confirmation; identifying an unresolved failed probe would require a recorded relationship between an earlier input, its expected result, and a later replay\.

04 Shared behavioral requirementGPT\-6\-solSource:proposals/pi/message\.txt, line 4 \(JSON object\)name: Known failed comparisons are not tracked as unresolved until a matching successful rerunstatementThe harness should retain every failed independently grounded comparison as a blocker tied to the checked artifact and expectation, show unresolved failures at confirmation, and clear them only after a successful rerun against the changed artifact\.observableA recorded comparison or probe has a failed outcome and no later passing result for the same criterion on the current artifact version when completion is claimed\.rests\_on1\.because: No recorded call classifies a probe as failed on a required input or determines whether a later test resolved the same failurecovers: 2\|members: 2how\_it\_knows: It reads the failed check receipt and searches later receipts with matching criterion and a newer artifact version for an actual pass\.Only the supporting finding for this example is shown inrests\_on\.

05 Search planGPT\-6\-solSource:suggest2\.search\_plans\.json/search\_plans/0problemPassing run detected a direct\-output discrepancy then rewrote the artifact without a matching comparison rerun; failing run saw invalid boundary parsing and later accepted unrelated successes\. Terminal screen receipts are not typed or associated with artifact revisions\.trigger: A claimed completion after a failed probe or comparison with no authenticated matching successful receipt for current artifact\.target\_behaviorKeep an unresolved failure ledger and withhold completion until a same\-criterion check passes on the changed artifact; do not infer resolution from unrelated checks or success prose\.framework: Harbor Terminus2 through base\_terminus\.Agent subclass, with tmux dispatch and capture in sandbox and separate BaseEnvironment\.exec facility\.execution\_modelAgent returns JSON commands; \_execute\_commands sends each as tmux keystrokes nonblocking and captures a single incremental screen at the end; \_run\_agent\_loop branches on parsed completion and its two\-claim handshake\. An independent environment\.exec returns return\_code/stdout/stderr but cannot give the status of a previously tmux\-dispatched action\.allowed\_changesOnly candidate\_source subclass of Terminus2 and local helpers, with default\-off runtime switches; frozen Harbor, Trial, verifier and environment implementations cannot change\. Candidate may override inherited loop and use session\.environment\.exec, with finite rejection bound\.available\_signals1\.agent action channel: parsed keystrokes in \_handle\_llm\_interaction and a combined tmux terminal increment in \_execute\_commands2\.independent harness capability: session\.environment\.exec returns ExecResult with return\_code/stdout/stderr for a new out\-of\-band command, not the earlier tmux action3\.agent loop: is\_task\_complete, \_pending\_completion and original\_instruction in \_run\_agent\_loopmissing\_signals1\.No typed comparison identity, expected value, actual value or pass/fail from arbitrary tmux screen text2\.No artifact\-version fingerprints and no receipt associating test identity with checked artifact3\.No per\-command exit status for commands dispatched via tmux, and no way to attribute an out\-of\-band exec status to those commands4\.No hidden verifier expected values visible before completionqueries1\.query: site:github\.com/harbor\-framework harbor terminus\_2 tmux session send\_keys get\_incremental\_output command result completionpurpose: Check whether current framework provides typed per\-action receipts or only screen captures and locate an applicable completion interception boundary\.scope: framework2\.query: agent coding shell tmux screen capture no per\-command exit status append sentinel command exit code test receiptpurpose: Find systems sharing unstructured terminal dispatch boundary that shape action delivery to get attributable test status\.scope: transfer3\.query: site:docs\.pytest\.org cache last failed tests rerun only failures lastfailedpurpose: Find a concrete externally documented failed\-criterion persistence and rerun clearance mechanism, assessing whether it transfers to agent completion\.scope: transfer

06 External precedent and local adaptationGPT\-6\-solSource:suggest2\.precedents\.json/precedents/0subject: pytest’s cross\-invocation test\-runner cache, not an agent\-turn completion gate\.mechanism: Retain failed test identifiers between invocations, select them for rerun on –lf rather than letting successful unrelated tests substitute for them\.checkableCompare cached failed test node identifiers to node identifiers and outcomes produced in the new invocation; the documented cache/lastfailed mapping holds failed node ids\.adaptationstatus: requires\_extensionsignal\_mappingpytest test node id/outcome maps to a candidate\-defined agent\_authored\_spec\_check identity and result; unlike pytest, local tmux action channel exposes only combined incremental output and cannot infer an action exit status\. Candidate must dispatch a dedicated check with a tagged terminal receipt or invoke session\.environment\.exec out of band for a new check and record its own return\_code/stdout/stderr; that independent result is NOT attributed to a prior tmux action\. Artifact version must be obtained separately through out\-of\-band hashes of declared paths\. Completion flag is available in \_run\_agent\_loop\.required\_changes1\.Candidate subclass stores a per\-run ledger keyed by check identity and artifact hashes and resets it per run\.2\.Generate checks before or apart from candidate implementation using agent\_authored\_spec\_check from task instruction/inputs; collect actual receipt from new independent exec, never infer failed status from terminal prose\.3\.Override completion\-loop decision to show unresolved ledger and retry with finite bound, without hidden verifier metadata\.validation\_needs1\.Current terminus\-hooks\-v1 fixture does not exercise full loop ledger; extend independent checker in a later round or use benchmark receipts and recorder smoke\.2\.An agent\-authored check can share agent’s misreading; limit by spec\-first checks, literal requirements, or cross\-method expectations; do not claim equivalence to verifier\.sources\[0\]\(one source record\):url:[https://docs\.pytest\.org/en/stable/how\-to/cache\.html](https://docs.pytest.org/en/stable/how-to/cache.html)quoteThe plugin provides two command line options to rerun failures from the last ‘pytest‘ invocation:\* \[‘–lf, –last\-failed‘\]\(\.\./reference/reference\.html\#cmdoption\-last\-failed\) \- to only re\-run the failures\. \* \[‘–ff, –failed\-first‘\]\(\.\./reference/reference\.html\#cmdoption\-failed\-first\) \- to run the failures first and then the rest of the tests\.

07 Generated proposalGPT\-6\-solSource:suggest2\.proposals\.json/proposals/0kind: buildneeds1\.at: stage\-4\.2whatBefore implementation begins, use original\_instruction to construct a bounded set of independently runnable, explicitly identified agent\_authored\_spec\_check commands with expected outcomes and declared artifact paths; if construction fails, leave task on existing completion path rather than inventing a check\.recorded:true2\.at: stage\-4\.4whatCandidate subclass runs new independent check commands via environment\.exec, records stdout/stderr/return\_code and independently computed artifact fingerprints, and maintains per\-run unresolved check identities; never treats the existing merged tmux snapshot or successful send\_keys as command status\.recorded:true3\.at: stage\-4\.7whatCandidate subclass compares pending failed check identities and artifact versions against independent passing rerun receipts before permitting the second completion claim to exit; reset ledger per run and give bounded, actionable failure observation with an eventual fallback when time or turn budget runs out\.recorded:truecosts\_turns: 1insteadAt a completion claim with a recorded failed check, retain that check as unresolved even when other commands succeeded, independently rerun its same check against current artifacts and refuse confirmation while the matching check still fails or lacks a completed receipt\. Show its identity and bounded diagnostics; release only after a matching current\-artifact pass\. On no buildable check or no recorded failure, preserve the existing two\-claim confirmation exactly, and never recast an uninstrumented tmux output as a test result\.checkableCompare \(check id, expected outcome, declared artifact\-path fingerprints\) from spec\-first registry to \(same check id, actual return\_code, observed outputs, current path fingerprints\) from an independent environment\.exec receipt\. A failure remains pending until a receipt for the same id and current fingerprint meets its recorded expectation; unrelated successful executions and uncompleted tmux sends do not clear it\. These signals need new plumbing into stage\-4\.7 from candidate\-owned stage\-4\.2 and stage\-4\.4 state\.predictsOn the first failing pair, a spec\-first comparison that detects different adjacent grayscale frames will remain pending after unrelated randomized passes and prompt a matching rerun before exit; on the second pair, if a spec\-first boundary check can be generated, incomplete fixes produce an unresolved receipt\. If checks cannot be generated for either, its run remains baseline\-like rather than claiming a verified improvement\.refuted\_ifGenerated checks miss the disagreement, a nonmatching pass clears a failed id, or stable runs incur a repeated confirmation loop despite clean receipts\.fires\_on\_stablequoted\{"call": 609, "source": "calls\.1700865\.jsonl", "function": "agents/terminus\_2/terminus\_2\.Terminus2\.\_execute\_commands", "depth": 4, "parent": \{"call": 29, "function": "agents/terminus\_2/terminus\_2\.Terminus2\.\_run\_agent\_loop", "episode": "setup"\}fires:falsethen\_whatAt episode 9 the agent has zero commands and claims task\_complete after prior episode’s validation and benchmark output; no independently recorded failed check id exists in the baseline, so the proposed failure\-only rejection condition is not met\. If the new spec\-first check fails in a rerun despite this passing baseline, this condition would fire and consume at least one extra turn; inspect that risk experimentally rather than assuming this stable task remains safe\.

08 Implementation and evaluation recordsGPT\-6\-solSource:implementation\.report\.json;conformance\.json;experiments/confirm/decision\.jsonstatus: completeimplemented\_arms1\.spec\_check\_failure\_ledgerdeviations\[\]summaryRepaired missing Harbor trajectory, chat, path, and datetime imports so the candidate resolves its referenced names\. Parsed the module and embedded independent\-check scripts locally; benchmark and recorder were not run\.Independent\-check receipt:status: not\_runscope: behavior checks disabled; arms are judged by trial performanceFull\-confirmation receipt:status: promotion\_readyscored\_countsreferencepassed: 201\|failed: 199\|runtime\_failures: 15\|trials: 400candidatepassed: 222\|failed: 178\|runtime\_failures: 2\|trials: 400

#### A\.4\.2DeepSeek\-V4\-Flash

This example concerns a service built from an image\-based specification\. The paired runs differ in how they read and verify the model parameters\.

01 Agent and harness recordDeepSeek\-V4\-FlashSource:compare2\.json/episodes/failing~110/agent;/episodes/failing~115/harness\_calls/49Agent analysis \(failing/10\): We need to start implementing the pipeline\. The OCR has been manually verified enough to know the specs are hidden layers \[128, 84\], mean 3\.5, and std 0\.8\. I will create an OCR extraction helper that runs tesseract and parses the raw text, with fallbacks matching the verified values\.Recorded harness call \(failing/15\):function:base\_terminus\.Agent\.\_rsi\_gate\_confirming\_claim

02 Pair\-level wishDeepSeek\-V4\-FlashSource:readings/suggest2\.wishes\.json/wishes/0asks\_the\_agent:falsestatementWhen a value asserted in a run’s reasoning is drawn from a source that the harness can see was read more than once, the harness should treat repeated readings produced by an identical method as a single observation rather than corroboration\. It should withhold standing from a restated value whose supporting readings all share one method fingerprint, and it should expose that the value rests on a single uncorroborated extraction until a differently\-parameterised reading of the same source has been recorded\.observableThe harness holds two or more extraction attempts recorded against the same source path and can compare the command line and parser/crop parameters of each; they are byte\-identical, yet a value drawn from that source is restated as established\.because1\.side: failing\|turn: 10quote: The OCR has been manually verified enough to know the specs are hidden layers \[128, 84\], mean 3\.5, and std 0\.8\.2\.side: passing\|turn: 5quoteThe original low\-resolution OCR confusingly read 128,84 and 35/08\. We should confirm the exact values with tightly cropped OCR before generating data and building the model\.3\.side: failing\|turn: 13quoteBefore finalizing, I want to perform one last visual check of the digit glyphs in the spec image to ensure the chosen mean 3\.5 is correct, since that value affects the synthetic feature and stats endpoint\.

03 Revision against harness executionDeepSeek\-V4\-FlashSource:readings/suggest2\.revisions\.json/revisions/0can: yes, with what the harness already holds at that momentalready: no\|at: \_handle\_llm\_interactionfoundAt stage\-4\.2 the harness executes the agent’s command batch and holds its command line before any value is passed forward\. On the failing side that batch runs tesseract once with \[’tesseract’, path, ’stdout’, ’–psm’, ’3’\] and the agent then asserts ’The OCR has been manually verified enough to know the specs are hidden layers \[128, 84\]’\. On the passing side the batch crops three regions, upscales x12 LANCZOS, thresholds 55% and sweeps psm 6/7/8/13, yielding a different value\. Both command lines are already visible to the harness at the same call, so two readings of one source and whether they share a method fingerprint are in hand\. Nothing in the trace shows the harness comparing them or withholding standing from a value whose supporting readings all share one method\.statementWhen a value asserted in a run’s reasoning is drawn from a source the harness can see was read more than once, the harness should treat readings produced by an identical method fingerprint as a single observation rather than corroboration\. At the moment such a value is restated as established, the harness should withhold standing from it while every supporting reading shares one command line and one set of parser or region parameters, and expose that it rests on a single uncorroborated extraction until a differently\-parameterised reading of the same source has been recorded\.observableTwo or more extraction attempts against the same source path among the harness’s own executed command batches, whose command line and parser or region parameters are byte\-identical, with a value from that source restated in the turn’s reasoning as established\.

04 Shared behavioral requirementDeepSeek\-V4\-FlashSource:proposals/pi/message\.txt, line 4 \(JSON object\)name: A value the run treats as settled although its own single method produced it, or two readings of one source disagreestatementBefore it releases a run as done, the harness inspects its own record of the commands it executed and the outputs they produced, and withholds the release once with the conflicting readings returned to the run when the value the claim depends on was produced only by one method reapplied with the same parameters, or when two of its own recorded readings of the same input disagree and no later differently\-formed observation resolves them\.observableThe harness holds its own record of the command batches it executed with their outputs, the source or input each read, and the value the current claim depends on; the moment is that value having been produced only by one method reapplied with the same parameters, or two of those recorded outputs disagreeing on it with no later differently\-formed observation resolving them\.rests\_on1\.because: that batch runs tesseract once with \[’tesseract’, path, ’stdout’, ’–psm’, ’3’\]covers: 11\|members: 11Only the supporting finding for this example is shown inrests\_on\.

05 Search planDeepSeek\-V4\-FlashSource:suggest2\.search\_plans\.json/search\_plans/0problemIn the failing runs a value is read once by a single observation method, treated as settled, and carried to a confirming completion claim; where the same input was read twice by the same method with different results, or where the only reading that produced the claimed value came from one method signature, the second confirmation released anyway because \_pending\_completion is a bare boolean\.triggerThe confirming completion claim in \_run\_agent\_loop: is\_task\_complete is True and was\_pending\_completion is True, immediately before the inherited return\.target\_behaviorAt the confirming completion claim the harness inspects its own recorded per\-run history of dispatched commands and returned observations and withholds the completion once when the claimed value’s recorded readings all share one method signature, or when two recorded readings of one input carry different values and no differently\-formed reading produced the claimed value\.frameworkharbor Terminus2 agent \(terminus\_2\.py\) with an external subclass candidate at base\_terminus:Agent, driven by the RSI arm harness that sets agent\.\_rsi\_arm\_event\.execution\_modelTerminus2 drives a tmux session: each episode it calls the LLM, parses commands, dispatches keystrokes via \_execute\_commands, captures the terminal back, appends a Step to self\.\_trajectory\_steps, and passes the observation as the next prompt\. On the second consecutive task\_complete the loop returns\. Candidate arms are overrides on the subclass, dispatched at the confirming\-claim divergence\.allowed\_changescandidate\_source \(base\_terminus\.py\) only: overrides of Terminus2 methods and new helper modules\. harbor\_source is read\-only\. Switches must be frozen boolean runtime\_kwargs defaulting false\. An arm that withholds must state a finite bound and let the original behaviour proceed once reached\. \_rsi\_arm\_event is provided by the harness and must only be called by an enabled arm, exactly once per intervention\.available\_signals1\.raw reply text of the current turn \(self\.\_rsi\_gate\_raw\_reply, captured in \_handle\_llm\_interaction\)2\.the accumulated trajectory steps \(self\.\_trajectory\_steps\), each Step carrying its tool\_calls\[\]\.arguments\.keystrokes and its observation results content3\.the per\-run list of dispatched keystrokes accumulated in \_execute\_commands4\.terminal\_output for the current episode returned by \_execute\_commands5\.\_pending\_completion boolean and the confirming\-claim divergence in \_run\_agent\_loopmissing\_signals1\.no call currently parses a value out of the observation text or a keystroke to form an "equivalence class" of readings2\.no per\-run store of \(reading, produced\-value, method\-signature\) triples exists; only raw keystroke strings and raw observation text are retained3\.no notion of "method signature" is defined anywhere in the frozen or candidate sourcequeries1\.query: AI agent framework require independent verification of extracted value before accepting task completion tool diversitypurposeFind an established mechanism by which an agent framework decides a value is corroborated across different observation methods before accepting a terminal claim \(tool\-diversity / method\-variety confidence gating\)\.scope: framework2\.query: self\-consistency sample multiple readings disagreement unresolved gate before final answer agentpurposeFind precedents for detecting disagreement among repeated readings of the same observation and blocking a claim on unresolved disagreement \(self\-consistency / agreement checking as a gate\)\.scope: transfer3\.query: agent harness audit recorded tool call outputs against completion claim evidence grounding prior stepspurposeFind whether trace\-based audit of an agent’s own prior tool calls and outputs to check a claim against recorded evidence is an established harness capability \(evidence\-grounded completion gating\)\.scope: framework4\.query: LangGraph AutoGen verifier node inspect prior tool observations before final answer allow terminationpurposeFind concrete mechanism names in mainstream agent frameworks \(LangGraph, AutoGen, Claude Code, OpenAI Agents SDK\) for a verifier/gate that inspects prior observations before allowing terminal state\.scope: framework

06 External precedent and local adaptationDeepSeek\-V4\-FlashSource:suggest2\.precedents\.json/precedents/0subjectAn out\-of\-band audit/enforcement library for LLM agent pipelines \(audit\-fence\), plus an agent\-patterns catalogue entry \(self\-consistency\) and a Claude\-skill verification protocol\. The moment each describes is a claim/submission event in an agent’s loop — a claim record submitted for acceptance before it is treated as settled — so it is a turn in an agent loop, not a deployment gate\.mechanismA claim\-to\-source validation gate that refuses a completion/claim record unless the record’s evidence is tied to the harness’s own recorded prior observations: \(1\) a search/observation must have preceded the claim, and \(2\) the submitted evidence must be a verbatim substring of a recorded observation, with the record carrying the source tool identity and index that produced it\. On any failure the submission is blocked and must retry rather than being accepted on the assertion\.checkableFor the claimed value token, two sets a program compares: \(a\) the set of method signatures of recorded readings that produced that token, formed from the dispatching keystroke bytes paired with the observation content that followed, and \(b\) the set of distinct observed values recorded for that same input across the run\. The gate withholds when \(a\) has one member, or when \(b\) has more than one member and no member of \(a\) differs in signature from the signature of the reading that produced the claimed value\. In audit\-fence the analogous compared pair is explicit: each submission carries an ‘evidence‘ string and a ‘source\_tool‘/‘source\_index‘, and the gate checks the evidence is a verbatim substring of a recorded search result and that a search precedes the submission\.adaptationrequired\_changes1\.Add a per\-run extraction that, at the confirming claim, recovers candidate value tokens from recorded observations and from the claimed reply, and associates each with the dispatching keystroke string so a ’method signature’ can be compared\. This is new code living in helper functions in the candidate source\.2\.Record the per\-run set of \(method\-signature, produced\-value\) pairs for the input each reading consumed\. Today only raw keystroke strings and raw observation text are retained, so the pairing must be formed at read time\.3\.Decide, at the existing confirming\-claim divergence in the candidate’s \_run\_agent\_loop override, whether to withhold; the withhold is the existing ’re\-issue the confirmation message and continue’ action that both existing arms already use, bounded at one withhold per run\.4\.Extend the confirmation message for the withhold by one sentence, following the existing \_rsi\_designated\_withhold\_message / \_rsi\_observed\_withhold\_message pattern, so the message the model sees names the unreconciled reading rather than just repeating the base text\.signal\_mappingaudit\-fence’s ‘search history‘ maps to the harness’s ‘self\.\_trajectory\_steps‘ \(the accumulated record of dispatched keystrokes and returned observation content\)\. ‘submit\(…\)‘ maps to the agent’s completion claim \(‘is\_task\_complete‘ in the raw reply\)\. audit\-fence’s requirement that ’a search must precede every submission’ maps to the harness’s requirement that some recorded reading produced the value the claim rests on\. The ‘evidence‘ substring check maps to recovering the claimed value token as a verbatim substring of a recorded observation\. ClaimRecord\.‘source\_tool‘ \+ ‘source\_index‘ map to a per\-reading method signature formed from the dispatching keystroke \(the tool/parameters/transformation\), which is the harness’s own recorded signal and needs no new observation\. ‘blocked‘ maps to the harness’s existing withhold action \(re\-issue confirmation, keep ‘\_pending\_completion‘ set, bounded once\); ‘REJECTED \(must retry\)‘ maps to continuing the loop after re\-issuing the confirmation message\. Note: audit\-fence runs as an out\-of\-band observer with its own state; here the comparison must live inside a candidate subclass override, and the local harness supplies tools for exactly that \(‘\_rsi\_arm\_event‘ recorder, the confirming\-claim divergence in the copied completion loop\)\.status: requires\_extensionvalidation\_needs1\.A run whose only reading of a value came from one method signature and released anyway, to confirm the withhold fires \(the failing tool\-diversity case in this cohort\)\.2\.A run where two readings of one input disagreed with no differently\-formed re\-read producing the claimed value, to confirm the disagreement clause fires\.3\.A stable passing run where a differently\-formed re\-read did produce the claimed value, to confirm the arm releases and does not cost the task\.4\.Confirmation that the whole\-value extraction step recovers a value token from the recorded observation text in a real trial, since observations are byte\-capped and may elide the value\.sources\[0\]\(one source record\):url:[https://github\.com/M\-HuangX/audit\-fence](https://github.com/M-HuangX/audit-fence)quoteEach audit unit follows the same enforcement pattern — an automated process reads the document under review, then verifies each claim against the unit’s data sources

07 Generated proposalDeepSeek\-V4\-FlashSource:suggest2\.proposals\.json/proposals/0kind: buildneeds1\.at: stage\-4\.7whatA per\-run store associating each recorded reading with a method signature: the dispatched keystroke bytes of the episode that produced it, plus the observed value token recovered from that episode’s observation content\. Today ‘\_rsi\_dispatched\_keystrokes‘ holds only keystroke strings for the designated arm, and ‘\_trajectory\_steps‘ holds keystrokes and observations separately with nothing pairing a keystroke to a value token\. The store must be built by new helper functions in the candidate source, read at decision time from ‘\_trajectory\_steps‘ or accumulated in ‘\_execute\_commands‘ alongside the existing designated\-arm accumulation\.recorded:true2\.at: stage\-4\.7whatA normalization that turns one dispatched keystroke into a comparable method identity \(command word plus its parameters and input transformation\), so two invocations of the same method compare equal and a differently formed reading compares unequal\. No such normalization exists in the frozen dependency or the candidate; without it the comparison would be between raw keystroke strings, which differ in incidental text and would fire on nearly every run\.recorded:true3\.at: stage\-4\.7whatA value\-token extractor that recovers a candidate value from a raw reply and from an observation’s text, so the claimed value \(from ‘\_rsi\_gate\_raw\_reply‘, already captured at ‘\_handle\_llm\_interaction‘\) and each recorded observation can be compared\. No existing call reads a value out of observation text; ‘\_limit\_output\_length‘ only bounds its length\. This is new code in the candidate source\.recorded:true4\.at: stage\-4\.7whatThe arm’s per\-run withhold bound and reset, matching the existing pattern \(‘\_MAX\_WITHHOLDS = 1‘\) and the frozen ‘\_reset\_per\_run\_state‘ extension, so the arm withholds at most once per run and the original behaviour proceeds once the bound is reached, as the build protocol requires\.recorded:truecosts\_turns: 1insteadAt the confirming\-claim divergence, after the existing three arms each decide, a fourth independently enabled arm decides the release of the confirming claim from the run’s own recorded readings\. It withholds once \(reason ‘unreconciled\_reading‘\) when no recorded reading of the input the claim names produced a different value from a differently formed method, or when two recorded readings of that input carry different values and none of them was formed differently from the one the claim rests on; otherwise it releases\. The withhold re\-issues the confirmation message extended with one sentence naming the unreconciled reading, keeping ‘\_pending\_completion‘ set so one more consecutive claim ends the run exactly as it does today\. Every path that reaches a comparison reports exactly once through the harness\-provided recorder, with the reason naming the case that held, so the recorder carries the evidence the arm acted\.checkableTwo explicitly recorded sets a program compares at the confirming claim\. \(A\) For each recorded reading in the run, a method signature formed from the dispatched keystroke bytes of that episode \(the command text plus its parameters\) paired with the observed value token extracted from the observation content that followed it\. \(B\) The set of distinct observed value tokens recorded for the same input across the run\. The gate withholds when either set \(A\), restricted to readings that produced the claimed value, has exactly one distinct signature, or set \(B\) has more than one member and no reading whose signature differs from that of the claiming reading produced the claimed value\. Both source values are already on the harness at this moment: the keystrokes sit on each trajectory step’s tool\_calls arguments\.keystrokes and the observations on the step’s observation results content, both appended by stage\-4\.6 inside self\.\_trajectory\_steps, and the claimed value is read from self\.\_rsi\_gate\_raw\_reply, already captured at \_handle\_llm\_interaction\. No new observation of the environment is needed\. What does not exist today is the signature pairing and the value\-token extraction, which is why data would have to be plumbed in: those are computed from data already held, by new helpers, at the moment the gate runs\. The recorder call is the existing \_rsi\_report\(arm\_id, "triggered", reason=…\), the same call the six existing arm reasons use, and the arm reports nothing while its switch is false\.predictsOn a rerun with this arm enabled, a failing run in which the only recorded reading of the claimed value came from one method signature, or in which two recorded readings of one input carry different values without a differently formed re\-read resolving them, will emit an ‘unreconciled\_reading‘ reporter event at the confirming claim, receive the extended confirmation message, and continue one more episode instead of exiting; a passing run whose differently formed re\-read produced the claimed value will emit no withhold event from this arm and exit at the same confirming\-claim return as today\. With the switch false, the recorder carries no events from this arm and the loop is byte\-for\-byte the inherited one\.refuted\_ifThe rerun shows no ‘unreconciled\_reading‘ event on any failing run that released a single\-method or disagreeing claim, or shows the arm firing on stable passing runs and costing them the task, or shows the withhold sending the run into repeated identical confirmations rather than exiting on the next consecutive claim\.fires\_on\_stablefires:falsequoted"base\_terminus\.Agent\.\_get\_completion\_confirmation\_message": 4, "base\_terminus\.Agent\.\_rsi\_report\_handshake": 4, "base\_terminus\.Agent\.\_rsi\_report": 8, "base\_terminus\.Agent\.\_rsi\_gate\_confirming\_claim": 4,then\_whatThis task reached stage\-4\.7 four times and passed in every recorded repeat, with Agent\.\_rsi\_report called 8 times across 4 confirming claims \(two reports per confirming claim: the handshake report and the observed arm’s match report\)\. The condition the new arm needs is a disagreeing pair of observed value tokens for one input, or a single\-signature set for the claimed value; this run’s recorded calls show the confirming\-claim arms reaching their comparisons and reporting, and the run still exiting as a pass\. Because the switch defaults false the new arm does not run at all on the baseline, and even armed it withholds only when the computed signature pairing yields disagreement or a single signature, which this two\-report\-per\-claim record does not supply; so it does not fire and the run exits at the same confirming\-claim return as today\. The quoted counts are the one place the trace shows stage\-4\.7 being reached by a task that already passes on every repeat\.

08 Implementation and evaluation recordsDeepSeek\-V4\-FlashSource:implementation\.report\.json;conformance\.json;experiments/confirm/decision\.jsonstatus: completeimplemented\_arms1\.unreconciled\_reading\_gatedeviations\[\]summaryImplemented the frozen unreconciled\_reading\_gate arm in candidate\_source/base\_terminus\.py only, preserving the existing harness byte\-for\-byte when it is disabled\. The new runtime kwarg unreconciled\_reading\_gate defaults false and is bound in Agent\.\_\_init\_\_; \_switches\.armed reports the new id ’unreconciled\_reading\_gate’ in addition to the three retained ids \(observed\_operand\_release\_gate, require\_designated\_invocation, structured\_completion\_gate\)\. With all four keywords false, Agent\.\_run\_agent\_loop remains a single await super\(\); the new state is reset in Agent\.\_reset\_per\_run\_state\. When the new arm is armed, Agent\.\_handle\_llm\_interaction also captures self\.\_rsi\_gate\_raw\_reply and Agent\.\_execute\_commands also accumulates self\.\_rsi\_dispatched\_keystrokes\. At the confirming\-claim divergence \(is\_task\_complete and was\_pending\_completion\) the arm runs independently beside the three existing arms; the loop withholds if any arm says withhold, and the new arm’s withhold re\-issues the inherited confirmation message extended with one sentence naming the unreconciled reading while keeping \_pending\_completion set\. The decision builds, from state already held, set \(A\) of \(method signature, observed value token\) pairs and set \(B\) of distinct observed value tokens for the input, and withholds once \(reason unreconciled\_reading\) when the claimed value’s producing readings carry exactly one distinct signature \(single\_signature\) or when the input’s readings carry more than one value and none formed differently from the claiming reading produced the claimed value \(disagreeing\_readings\); otherwise it releases \(reading\_reconciled\)\. Missing information \(no\_claim\_value, no\_readings, no\_signature\) fails open with a reported release\. Every path that reaches the comparison reports exactly once through the harness\-provided \_rsi\_arm\_event recorder, and nothing is reported while the switch is false or the bound is spent\. The withhold is bounded once per run by \_rsi\_unreconciled\_withholds\_used capped at \_MAX\_WITHHOLDS=1; it runs no command, subprocess, or probe, so it adds no timeout\. Three new pure module\-level helpers \(\_normalize\_method\_signature, \_extract\_value\_tokens, \_input\_identity\) plus two reading\-extraction helpers implement the contract’s missing normalization/value\-token/input\-identity notions over data already held\. Selftests inside candidate\_source exercise the helpers, the disabled no\-op, the single\-signature withhold, the reconciled release, the unknown release paths, the once\-per\-run bound, the switches reader, and the raw\-reply capture; ’selftest ok’ passes\. This is an implementation report only: no conformance or performance claim is made\.Independent\-check receipt:status: not\_runscope: behavior checks disabled; arms are judged by trial performanceFull\-confirmation receipt:status: promotion\_readyscored\_countsreferencepassed: 230\|failed: 170\|runtime\_failures: 8\|trials: 400candidatepassed: 239\|failed: 161\|runtime\_failures: 7\|trials: 400

##### Reading these records\.

The pair\-level feasibility judgement and the group\-level proposal belong to separate stages and can differ; both are reproduced rather than merged\. The GPT\-6\-sol candidate recognizes four predefined instruction patterns, so its implementation is narrower than an arbitrary\-task check generator\. The DeepSeek candidate’s conflicting\-readings branch has the limitations in Appendix[B](https://arxiv.org/html/2610.27435#A2)\. Both examples record independent behavioral checks as not run\. DeepSeek’s 239/400 confirmation is the original adoption measurement, distinct from the later same\-code repeat reported in the main table\. The research audits flag missing planned\-query receipts for GPT\-6\-sol and one unfetched source for DeepSeek; the displayed external\-source record has fetched\-page provenance\. Saved source citations do not establish the correctness of a proposed adaptation\.

### A\.5Prompt Examples

The proposal messages for the two examples contain the same directives at lines 3, 5, and 6, reproduced below\. Their model\-specific group JSON at line 4 is shown with each example above\. Context inventories and absolute\-path checklists are retained in the full messages but omitted here\. The implementation prompts share an identical instruction array prefix; that prefix and the GPT\-6\-sol repair instructions are reproduced in full\. These are task\-level instructions, not the complete loaded skill or system prompt\.

Proposal generation / saved message line 3Source: Both model\-specificpi/message\.txtfiles, line 3Read ALL cohort members; no member cap applies\. Use this accepted group wish as a hypothesis, not a conclusion:

Proposal generation / saved message line 5Source: Both model\-specificpi/message\.txtfiles, line 5Read stable\-task material with suggest2\_stable\. It is counterexample material only: absent recorded calls do not establish non\-interference, aggregate counts do not establish a trigger, and repeat counts are not necessarily four\. Do not fabricate stable quotes\. Search actual precedents with the configured web tools; a network failure is not evidence of no precedents\. Record wish/revise/precedent/handbook/propose through the tools, then stop\. No implementation here\. A proposal whose mechanism needs a signal your search plan lists under missing\_signals must produce that signal itself within allowed\_changes, for example by having the agent run its own check; otherwise record no proposal for that wish\.

Proposal generation / saved message line 6Source: Both model\-specificpi/message\.txtfiles, line 6Before calling suggest2\_search\_plan \(not merely before external search\), use the local read tool to complete EVERY role in this checklist\. Successful nonempty read results must finish before the search\-plan tool call starts\. For reference/dependency, read a relevant source file from any listed root, selected from that root’s files inventory in the context; a directory listing is not a source read\. For handbook/protocol roles, read relevant passages from an exact listed file\. You need not read an entire handbook or every dependency root\. One successful read can satisfy multiple roles when paths overlap\. Only roles actually configured below are required; trace/cohort material is supplementary\.

Implementation / complete shared instructions\[0:14\]Source: Bothstages/implement/\.\.\./pi\_stage/prompt\.txtfiles,/instructions/0through/instructions/131\.You are one stage in a harness iteration pipeline\. Read the frozen inputs before acting\.2\.Do not edit frozen proposals, contracts, reference results/source, controller, adapters, or expectations\.3\.Read each imported proposal’s embedded research archives when present\. Resolve borrowed against research\.precedents\.document; research\.search\_plans\.document explains the current framework and missing signals\. Treat adaptation\.required\_changes and validation\_needs as guidance: where a signal they ask for cannot be produced, build on the closest signal the harness has and say so in the arm description or report; that is not a reason to defer\. These are research claims, not verified capabilities or permission to change frozen contracts\.4\.Write the required JSON artifact to product\_path\. Exit text is not an artifact or an acceptance verdict\.5\.Do not install packages, access credentials, call a benchmark/model runner, or start remote jobs\.6\.No performance improvement is established here\. Do not claim conformance or recorder validity yourself\.7\.Implement the frozen arms in candidate\_source only\. Preserve the existing harness when every new arm is disabled, including retained base arms\.8\.For Harbor, preserve the configured agent\_import\_path and construct each arm from its frozen runtime\_kwarg, default false\. Support every retained old flag even if replaced\. The configured arm\_reader\_attr must report the actual enabled switches using arm\_reader\_values \(arm\_ids by default, kwargs only when explicitly configured\); accepting a keyword via \*\*kwargs without implementing it is not support\. Do not modify the runtime mapping, observer, or frozen cases to fit your implementation\.9\.Keep trigger predicates and actual decisions consistent\. Every time an enabled arm intervenes, report it exactly once through the harness\-provided recorder, and report ’unknown’ when missing information stops the decision: record = getattr\(self, "\_rsi\_arm\_event", None\) if record is not None: record\("<arm\_id\>", "triggered", reason="<short, JSON\-serializable\>"\) Do not report checks that found nothing to do, never define or replace \_rsi\_arm\_event, and never report while the arm’s switch is false\. These reports are the only evidence that an arm acted in a real trial\.10\.Honor requires/replaces, bound intervention retries and resource use, and keep harness/infra failures distinct from model verdicts\. Do not widen completion gates or invent benchmark acceptance metadata\.11\.You may add selftests inside candidate\_source, but cannot edit the independently frozen cases, their adapters, or proposal scope\. Use existing project tools; no actual benchmark runs\.12\.Write \{status:’complete’,implemented\_arms:\[all frozen arm ids\],deviations:\[\],summary:’…’\} to product\_path\. If impossible, report status:’blocked’ and concrete reasons\. List any departure from the contract in deviations rather than hiding it; it does not end the round\.13\.Only candidate\_source code may change\. Your report and local notes may be written inside work\_dir\.14\.No conformance case exists this round: each arm is judged by measured trial performance and by its recorder reports alone\. Implement every arm’s full mechanism as its contract describes, leave the harness exactly as it was while the arm is disabled, and report every intervention\.

GPT\-6\-sol / saved implementation\-repair instructionsSource:stages/implement/2/pi\_stage/prompt\.txt,/instructions/14and/instructions/1515\.This is implementation repair 1 of 2: the previous candidate failed a build check\. Read failures\_path: check names it \(source\_names: the source reads names it never defines or imports; trace\_smoke: the trace recorder could not load or record the candidate\), and stderr\_tail holds the details, such as each unbound name with its lines or an import\-time exception\.16\.candidate\_source already holds the previous implementation; fix its code so the module imports, every name it reads is defined or imported, and every enabled arm runs\. Keep the frozen contract, proposals and each arm’s scope unchanged, and keep the harness exactly as before while arms are disabled\.

## Appendix BModel\-Specific Adaptation

This appendix describes the changes analyzed in Section[4\.4](https://arxiv.org/html/2610.27435#S4.SS4.SSS0.Px1), separating what each mechanism does from its measured effects\. The two main models are presented first; the earlier GPT\-5\.6\-Terra change is included separately\. Evaluation rules and numerical results follow in Appendices[C](https://arxiv.org/html/2610.27435#A3)and[D](https://arxiv.org/html/2610.27435#A4)\.

### B\.1GPT\-6\-sol

The three retained changes improve execution continuity and the evidence returned to the model\. Their behavioral locations follow the seed Handbook: session liveness is checked by the Interaction main loop; persisted changes are presented during Observation construction and output truncation; and completion\-specific context is supplied during the Completion confirmation handshake\.

##### Completion\-claim context\.

When the agent declares completion, the confirmation message includes evidence of the work performed earlier in the run, rather than only an idle terminal prompt\. This modification targets the Completion confirmation handshake, supported by observation construction and the recorded execution history\. Its implementation setting iscompletion\_claim\_context\.

##### Persisted workspace feedback\.

The harness presents changes actually written to files as a workspace diff\. The feedback is grounded in persisted file contents rather than terminal echo, giving the agent a view of what changed\. Its primary behavioral location is Observation construction and output truncation, with file inspection supplying the underlying evidence\. Its setting ispersisted\_workspace\_diff\.

##### Session\-loss recovery\.

When the terminal session disappears, the harness recovers the session and informs the agent of the resulting state loss\. This changes the Interaction main loop’s handling of session liveness, with tmux session operations and observation construction supporting recovery and its feedback\. Its setting issingle\_session\_loss\_recovery\.

### B\.2DeepSeek\-V4\-Flash

In the seed harness, the first completion declaration requests confirmation, and a second consecutive declaration permits termination\. The four adopted modifications add checks to this handshake\. In Figures[3](https://arxiv.org/html/2610.27435#S4.F3)and[3](https://arxiv.org/html/2610.27435#S4.F3), SE denotes structured evidence, DI designated invocation, OO observed operand, and IR independent reading\. The corresponding behaviors are detailed below\. Each check can withhold completion at most once per run, providing a bounded opportunity for correction rather than enforcing an indefinitely repeated verification requirement\.

##### Primary and supporting stages\.

The primary stage is where a modification changes a harness decision or action; supporting stages acquire its inputs or maintain the required state\. For example, designated\-invocation verification reads command history recorded during dispatch but changes the completion decision during the handshake\. All four modifications therefore target Completion confirmation handshake, with their supporting stages listed in Table[2](https://arxiv.org/html/2610.27435#A2.T2)\.

Table 2:Behavioral modifications for DeepSeek\-V4\-Flash and their Handbook mappings\.
##### Executable completion verification \(SE\)\.

The model supplies a verification command and its expected exit code in the completion declaration\. The harness executes the command and compares its exit code with the expected value\. A match permits this check to release completion; missing fields, execution failure, or a mismatch produce one request to continue working\. This makes the model’s completion evidence executable by the harness, while preserving the bounded withholding rule\.

##### Designated\-invocation verification \(DI\)\.

When a task explicitly specifies an invocation, the harness checks whether it appears in the dispatched\-command history\. It conservatively extracts the interpreter, script, and arguments from the task instruction, and accommodates absolute paths and relative script paths after a directory change through basename matching\. The check triggers only when an alternative invocation has been observed but the designated invocation has not\. It then requests that the model execute the specified invocation before confirming completion again\.

##### Completion\-outcome observation \(OO\)\.

On the second completion declaration, the harness executes the supplied verification command through the session’s environment interface, with a 30\-second probe limit\. The check compares an observed integer exit code with the expected value and records a successful match as well as a withholding decision\. Missing evidence, a non\-integer result, or a mismatch can withhold completion once\. If the sandbox call raises an exception, this check fails open, so the probe exception does not itself turn into a task failure\.

##### Independent\-reading corroboration \(IR\)\.

This check uses existing execution history rather than launching another command\. It relates dispatched commands, normalized reading\-method signatures, input objects, and values extracted from subsequent observations\. If a reading used to support completion is backed by only one method, the harness requests a different form of inspection or a second independent reading method\. Repeating the same method is not treated as independent corroboration\.

##### Implementation boundaries\.

Executable completion verification and completion\-outcome observation are separate checks and can execute the same verification command twice\. Their effects are therefore overlapping\. Independent\-reading corroboration primarily detects reliance on a single reading method: the current evidence\-set construction makes the separate conflicting\-readings branch largely unreachable\. We consequently do not describe it as a general consistency checker\. These are implementation boundaries, not estimates of the individual mechanisms’ causal contributions\. The full cumulative measurements and their evaluation provenance are reported in Appendix[D\.2](https://arxiv.org/html/2610.27435#A4.SS2)\.

### B\.3GPT\-5\.6\-Terra

The earlier GPT\-5\.6\-Terra modification targets Command dispatch and timeout handling\. A command can contain a literal tab as input data; if tmux interprets it as a special key event, the terminal receives different input from the model’s intended command\. Literal control\-character transport preserves the character as data during dispatch\. This changes command transport rather than completion confirmation\. Its results are reported separately in Appendix[D\.5](https://arxiv.org/html/2610.27435#A4.SS5)and do not establish the changes needed by GPT\-6\-sol\.

## Appendix CEvaluation Methodology

This appendix supplements Section[4\.1](https://arxiv.org/html/2610.27435#S4.SS1.SSS0.Px1)with scoring, baseline, and version\-reporting rules\. Numerical results follow in Appendix[D](https://arxiv.org/html/2610.27435#A4)\.

### C\.1Repeated Evaluation

The defaultr=4r=4gives 400 trial slots for TBH and 356 for TB2\.1\. The metrics in Section[4\.1](https://arxiv.org/html/2610.27435#S4.SS1.SSS0.Px3)require a complete grid\. Passes score11; task failures and completed runtime failures with valid records score00, retaining their harness, infrastructure, or unknown\-error classification\. Missing, cancelled, or identity\-mismatched measurements are⊥\\botand cannot be dropped to make an incomplete grid valid\. Blank table entries mean unreported results, not failures\.

Aggregate outcome proportions do not identify which initially stable tasks regressed; paired per\-task outcomes are needed\.

### C\.2Comparison Protocol

Before evaluating a candidate, the policy specifies the comparison baseline\. For stagesson task setSS, it selects either historical reference runs or a newly measured incumbent control:

p^Bs​\(S\)=\{p^​\(Rt\|S\),bs=historical,p^​\(Ht,S,rs\),bs=fresh\.\\widehat\{p\}\_\{B\}^\{\\,s\}\(S\)=\\begin\{cases\}\\widehat\{p\}\(R\_\{t\}\|\_\{S\}\),&b\_\{s\}=\\mathrm\{historical\},\\\\ \\widehat\{p\}\(H\_\{t\};S,r\_\{s\}\),&b\_\{s\}=\\mathrm\{fresh\}\.\\end\{cases\}\(4\)Herersr\_\{s\}is the stage’s repetition count\. Historical comparisons use their stored repetitions and corresponding denominator\. The controller defaults to historical reuse; fresh controls are optional\. Historical source identity does not establish unchanged service or environment conditions, so baseline provenance remains part of the evaluation record\.

Adoption requires a valid, complete full evaluation with aggregate pass rate above the declared baseline\. Previously stable tasks may regress despite an overall gain, so reliability is reported separately from the adoption score\.

### C\.3Result Reporting

The main comparison, cumulative curves, and transfer experiment refer to explicitly identified harness versions\. On DeepSeek\-V4\-Flash, TBH uses the final four\-mechanism version \(\+ IR\)\. Table[1](https://arxiv.org/html/2610.27435#S4.T1)and Figure[6](https://arxiv.org/html/2610.27435#S4.F6)report the best observed TB2\.1 three\-mechanism version \(\+ OO\), selected by its TB2\.1 score\. This is distinct from directly transferring the final version selected on TBH\. The full sequence in Appendix[D\.2](https://arxiv.org/html/2610.27435#A4.SS2)keeps both versions visible\. TB2\.1 does not inform harness modifications; selection by its observed score is nevertheless part of this reporting protocol\.

The four non\-base TBH versions use the original100×4100\\times 4grid\. The seed’s TBH result is 227/400 passes \(56\.75%\), with 37 all\-success, 34 flipping, and 29 all\-failure tasks\. For the first three non\-base TB2\.1 versions, controller\-timeout slots were rerun and merged with the other results; the final version uses a complete rerun\. These differences in evaluation assembly are retained when interpreting the cumulative sequence\. The curves measure combined harness versions rather than isolated causal effects of individual mechanisms\.

## Appendix DSupplementary Results

The following results extend the main experiments in order: main\-comparison trial counts, cumulative DeepSeek\-V4\-Flash versions, component ablations, and cross\-model transfer\. The earlier GPT\-5\.6\-Terra experiment follows separately\. Mechanisms and measurement rules are defined in Appendices[B](https://arxiv.org/html/2610.27435#A2)and[C](https://arxiv.org/html/2610.27435#A3)\.

### D\.1GPT\-6\-sol

Table[3](https://arxiv.org/html/2610.27435#A4.T3)gives the GPT\-6\-sol pass counts underlying Table[1](https://arxiv.org/html/2610.27435#S4.T1), retaining the full4​N4Ndenominator\. Its ablations remain unreported\. Cross\-model transfer measurements appear in Appendix[D\.4](https://arxiv.org/html/2610.27435#A4.SS4)\.

Table 3:GPT\-6\-sol passed / total trials\.
### D\.2Evolution Results

Table[4](https://arxiv.org/html/2610.27435#A4.T4)gives the values plotted in Figures[3](https://arxiv.org/html/2610.27435#S4.F3)and[3](https://arxiv.org/html/2610.27435#S4.F3)\. Each “\+” adds one of the mechanisms defined in Appendix[B](https://arxiv.org/html/2610.27435#A2): structured evidence \(SE\), designated invocation \(DI\), observed operand \(OO\), and independent reading \(IR\)\. These are cumulative versions, not controller round identifiers\.

Table 4:Cumulative DeepSeek\-V4\-Flash results \(r=4r=4; %\), matching Figures[3](https://arxiv.org/html/2610.27435#S4.F3)and[3](https://arxiv.org/html/2610.27435#S4.F3)\.The final \+ IR version achieves the highest TBH Pass@1 and Pass4\. On TB2\.1, the \+ OO version has the highest observed values, whereas direct transfer of the TBH\-selected \+ IR version improves Pass@1 and Pass4over the seed by 3\.09% and 6\.74%, respectively\. The version\-selection and rerun procedures are specified in Appendix[C\.3](https://arxiv.org/html/2610.27435#A3.SS3)\. The category proportions show aggregate distributions, not transitions of particular tasks\.

### D\.3Ablation Results

Table[5](https://arxiv.org/html/2610.27435#A4.T5)complements the TBH comparisons in Figure[6](https://arxiv.org/html/2610.27435#S4.F6)and the comparisons in Section[4\.3](https://arxiv.org/html/2610.27435#S4.SS3)\. HE denotes harness traces and the Handbook; ER denotes external design research\. Both ablations retain the remaining components\. The full method uses \+ IR on TBH and \+ OO on TB2\.1, following the version\-reporting protocol above\.

Table 5:DeepSeek\-V4\-Flash ablations \(r=4r=4; %\)\. These values support Section[4\.3](https://arxiv.org/html/2610.27435#S4.SS3)and are kept separate from the main comparison table\.
### D\.4Cross\-Model Transfer

We apply each model’s final evolved harness to the other model, keeping the transferred harness fixed, and evaluate each task four times on both benchmarks\. Table[6](https://arxiv.org/html/2610.27435#A4.T6)reports these cross\-model evaluations\. This differs from cross\-benchmark generalization, which keeps the execution model unchanged\.

Table 6:Cross\-model transfer of final evolved harnesses \(r=4r=4\)\. Pass metrics and Flip are in %; task counts use 100 tasks on TBH and 89 on TB2\.1\. Arrows indicate harness source model→\\rightarrowexecution model\.Figure[7](https://arxiv.org/html/2610.27435#S4.F7)uses the final \+ IR harness for DeepSeek\-V4\-Flash on both benchmarks, rather than the best\-observed \+ OO version in Table[1](https://arxiv.org/html/2610.27435#S4.T1)\. GPT\-6\-sol’s own\-adapted TB2\.1 reference is the 291/356RiseHarnessresult in Table[3](https://arxiv.org/html/2610.27435#A4.T3)\.

### D\.5GPT\-5\.6\-Terra

These earlier experiments use GPT\-5\.6\-Terra, a different model from GPT\-6\-sol in the main evaluation\. Its results are kept separate from the GPT\-6\-sol rows in Table[1](https://arxiv.org/html/2610.27435#S4.T1)\. The command\-transport modification is described in Appendix[B\.3](https://arxiv.org/html/2610.27435#A2.SS3)\.

Table 7:Historical GPT\-5\.6\-Terra results \(r=4r=4; %\), separate from GPT\-6\-sol\. Coral annotations show absolute changes from Terminus\-2\.

## Appendix EScope and Limitations

##### Execution evidence and analysis coverage\.

Successful runs provide references for failure analysis, but differences may also reflect sampling, environment variation, or task difficulty\. Discovery from traceable flipping tasks excludes consistently failing tasks and failures before recording\. The monitor covers selected boundaries, and the Handbook may omit behavior or become stale\. Neither source establishes causality\.

##### Candidate changes and regression risks\.

External designs may require unavailable interfaces or signals\. Behavioral checks cover finite cases, while combined\-candidate evaluation does not isolate individual mechanisms\. Aggregate\-score adoption can retain changes that harm previously stable tasks; aggregate outcome proportions cannot quantify these regressions without per\-task comparisons\.

##### Evaluation and transfer\.

Repeated development\-set selection can exploit noise, and historical baselines are sensitive to service or environment drift\. Retained\-score trends therefore require independent evaluation\. The reporting protocol in Appendix[C\.3](https://arxiv.org/html/2610.27435#A3.SS3)distinguishes target\-selected versions from frozen final\-version transfer\. Cross\-model swapping shows model dependence for the tested harnesses, without ruling out transfer of individual mechanisms between other models\.

##### Scope and resource costs\.

Other harnesses require compatible monitoring, behavioral mappings, and evaluation adapters\. Long runs, external research, and repeated evaluation incur inference and storage costs; configured concurrency is not measured speedup\. LLMCarbon motivates broader resource accounting\[[39](https://arxiv.org/html/2610.27435#bib.bib39)\], but emissions are not measured here\. The loop keeps model weights fixed and does not learn its outer controller\. Training\-side methods, including work on asynchronous policy mismatch\[[40](https://arxiv.org/html/2610.27435#bib.bib40)\], are complementary directions for future study\.