Self-Evolving Coding Agents via Harness-Policy Optimization

# Self-Evolving Coding Agents via Harness-Policy Optimization
Source: [https://arxiv.org/html/2610.22791](https://arxiv.org/html/2610.22791)
Zhepei Wei††thanks:Work done during internship at Microsoft Research\.Wenlin YaoAffiliation:Microsoft ResearchBaolin PengAffiliation:Microsoft ResearchDanqing WangAffiliation:Carnegie Mellon University[https://github\.com/MSR\-Orchard/Self\-HPO](https://github.com/MSR-Orchard/Self-HPO)![[Uncaptioned image]](https://arxiv.org/html/2610.22791v1/figures/hf-logo.png)[https://huggingface\.co/self\-hpo](https://huggingface.co/self-hpo)Qianhui WuAffiliation:Microsoft ResearchHao ChengAffiliation:Microsoft ResearchYu MengAffiliation:University of VirginiaJianfeng GaoAffiliation:Microsoft Research

###### Abstract

Large language model \(LLM\) agents have shown promising potential in coding tasks, and their performance is jointly determined by the policy model and the harness that mediates their interaction with the environment\. Indeed, harness and policy are mutually reinforcing and evolving only one component is suboptimal—a harness tuned for one checkpoint may not target the failure modes of an updated policy, while policy learning remains limited to the trajectories elicited by a fixed harness\. In light of this, we proposeSelf\-HPO, a recursive self\-improving framework for coding agents that alternates harness and policy optimization, using the evolving policy as its own harness proposer\. Specifically, in each round, the current policy first analyzes its own failures and proposes new harness candidates \(H\-step\)\. The policy is then updated through reinforcement learning \(RL\) under the selected harness \(P\-step\), and hard tasks identified during training form the next harness search set, keeping harness optimization targeted to the policy’s evolving failure modes\. Across three coding benchmarks,Self\-HPOconsistently outperforms harness\-only and policy\-only baselines, improving Qwen3\.5\-35B\-A3B’s average resolve rate from 57\.8% to 64\.8%, with gains generalizing to out\-of\-distribution settings\. Further analysis shows that both optimized components retain gains even when paired with their base counterparts, while their joint gain exceeds the sum of their individual gains\.Self\-HPOalso improves reliability \(pass@1\) and expands the capability frontier \(pass@kk\) over the base agent\. Additionally, we show that with a stronger external model \(e\.g\., Opus 4\.8\) as the harness proposer, the same framework yields a further performance boost\.

## 1Introduction

Large language model \(LLM\) agents have made rapid progress on complex, multi\-step tasks such as software engineering\([Jimenez et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib14);[Yang et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib15)\), web navigation\([Zhou et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib95);[He et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib96);[Wei et al\., 2025c](https://arxiv.org/html/2610.22791#bib.bib97)\), and computer use\([Xie et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib98);[Agashe et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib99)\)\. Such an agent consists of two components: the policy model and its harness, which is the external scaffold that decides what the model observes, how its actions are executed, what feedback it receives, and when an episode terminates\([Yao et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib23);[Shinn et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib53);[Yang et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib15);[He et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib96);[Lee et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib16);[Zhang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib18)\)\. Typically, the policy model defines the agent’s capability boundary, while the harness determines how these capabilities are elicited, hence the same model may succeed or fail on one problem depending on its harness\([Le et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib37);[Zhang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib18)\)\.

There have been two main lines of research on improving such agents\. Harness optimization holds the policy fixed and searches over prompts, workflows, or agent code\([Hu et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib42);[Lee et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib16);[Zhang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib18)\), whereas policy optimization holds the harness fixed and updates the model from successful trajectories or verifiable rewards\([Shao et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib31);[Guo et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib32);[Wei et al\., 2025a](https://arxiv.org/html/2610.22791#bib.bib88)\)\. However, harness\-only optimization cannot improve the underlying model, while policy\-only training can learn only from the trajectories that its fixed harness elicits\. Although recent works begin to optimize both components\([Kim et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib40);[Chen et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib8)\), they rely on an external harness proposer and face the challenge of a moving optimization target as the policy and its failure modes evolve, leaving self\-driven evolution for coding agents underexplored\.

In this work, we introduceSelf\-HPO, a framework for recursive self\-improvement driven by the agent’s own policy\. As illustrated in Figure[1](https://arxiv.org/html/2610.22791#S1.F1),Self\-HPOalternates harness optimization \(H\-step\) with policy optimization under the optimized harness \(P\-step\)\. Harness optimization exposes and compensates for the weaknesses of the current policy, while policy optimization internalizes the elicited behaviors into the model and shifts the failures that the next harness should target\. Specifically, in the H\-step, the current policy analyzes its failed trajectories and proposes harness revisions targeting its own failure modes, and selects the updated harness based on search\-set performance \(§[2\.2](https://arxiv.org/html/2610.22791#S2.SS2)\)\. In the P\-step, the policy is updated using reinforcement learning under the optimized harness \(§[2\.3](https://arxiv.org/html/2610.22791#S2.SS3)\)\. In the next H\-step, the updated policy serves as the harness proposer, with hard tasks identified from the preceding P\-step as the search set to keep harness search aligned with the policy’s evolving failure modes\.

![Refer to caption](https://arxiv.org/html/2610.22791v1/teaser_overview.png)

![Refer to caption](https://arxiv.org/html/2610.22791v1/teaser_curve.png)

Figure 1:Self\-HPOrecursively improves the harness and policy using the agent itself\.Left:In each H\-step, the current policy proposes an improved harness, and each P\-step then updates the policy through reinforcement learning with the optimized harness\.Right:Average resolve rate over SWE\-Bench Verified, SWE\-Bench Multilingual, and SWE\-Bench Pro along the H\- and P\-steps\. Policy\-only and Harness\-only denote the RLVR\([Peng et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib17)\)and Self\-Harness\([Zhang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib18)\)baselines\.We instantiateSelf\-HPOwith Qwen3\.5\-35B\-A3B\([Qwen Team, 2026](https://arxiv.org/html/2610.22791#bib.bib21)\)as the base policy, and mini\-SWE\-agent\([Yang et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib15)\)as the seed harness\. Across three coding benchmarks,Self\-HPOconsistently outperforms policy\-only and harness\-only optimization, with the average resolve rate improving at every optimization step, as demonstrated in Figure[1](https://arxiv.org/html/2610.22791#S1.F1)\. On SWE\-Bench Verified\([Chowdhury et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib24)\), it improves the base resolve rate from 61\.0% to 69\.5%, with transferable gains on the out\-of\-distribution SWE\-Bench Multilingual\([Yang et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib25)\)and SWE\-Bench Pro\([Deng et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib26)\)\. Ablations further show that evolving the harness search set with hard tasks identified during policy learning yields better performance than reusing a fixed set\. We also extendSelf\-HPOto the external\-model setup where Opus 4\.8 is used as an external harness proposer, which further raises the resolve rate on SWE\-Bench Verified to 71\.6%\. The main contributions of this work are as follows:

- •We proposeSelf\-HPO, a recursive self\-improving framework for coding agents that alternates harness and policy optimization, where the agent uses its own policy as the harness proposer and adapts harness search to the policy’s evolving failure modes\.
- •We demonstrate thatSelf\-HPOconsistently outperforms policy\-only and harness\-only baselines across three coding benchmarks, improving the average resolve rate from 57\.8% to 64\.8%, with gains generalizing to out\-of\-distribution settings\.
- •Our study shows that both optimized components ofSelf\-HPOretain gains even when paired with their base counterparts, while their joint gain exceeds the sum of their individual gains\. We also find thatSelf\-HPOnot only improves reliability \(pass@1\) but also expands the capability frontier \(pass@kk\) of the agent\.

## 2Self\-HPO: Self\-Evolving via Harness\-Policy Optimization

![Refer to caption](https://arxiv.org/html/2610.22791v1/method_overview.png)Figure 2:Overview of Self\-HPO\.Top \(H\-step\):The frozen policy analyzes the agent’s failures on the search set and proposes new harness candidates\.Bottom \(P\-step\):With the updated harness, the policy evolves through reinforcement learning\. The updated policy becomes the next harness proposer, while tasks never resolved during this P\-step form the next search set\.### 2\.1Problem Setup

Given a software\-engineering taskxx, a coding agent combines a policyPP, parameterized by an LLM, with a harnessHH, which controls context, tools, feedback, and termination\. The resulting trajectoryτ∼πH,P\(⋅∣x\)\\tau\\sim\\pi\_\{H,P\}\(\\cdot\\mid x\)is scored by an executable verifiery⁡\(x,τ\)∈\{0,1\}y\(x,\\tau\)\\in\\\{0,1\\\}\. We measure a harness–policy pair by its expected resolve rate on the training data𝒟\\mathcal\{D\}:

J\(H,P\)=𝔼x∼𝒟𝔼τ∼πH,P\(⋅∣x\)\[y\(x,τ\)\]\.J\(H,P\)=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\}\\,\\mathbb\{E\}\_\{\\tau\\sim\\pi\_\{H,P\}\(\\cdot\\mid x\)\}\\big\[y\(x,\\tau\)\\big\]\.\(1\)
We study the problem of self\-improving a coding agent’s own harness and policy\. The harness changes which behaviors the current policy can reliably express, while a policy update changes the failures that the next harness should target\. As shown in Figure[2](https://arxiv.org/html/2610.22791#S2.F2), each H\-step holds the policy fixed while using it to analyze failures and propose harness revisions, and each P\-step then freezes this harness and updates the policy through reinforcement learning\. The updated policy becomes the next harness proposer, with hard tasks identified from the training process forming the next search set\.

We call this alternating framework Harness\-Policy Optimization \(HPO\) and implement it in two setups\. In the self\-improving setup \(Self\-HPO\), the evolving policy proposes its own harness candidates, so no separate proposer model is needed\. For reference, we also instantiate the same HPO loop with Opus 4\.8 as a fixed external proposer\.

Algorithm 1Self Harness\-Policy Optimization \(Self\-HPO\)1:Seed harness

H0H\_\{0\}, base policy

P0P\_\{0\}, training data

𝒟\\mathcal\{D\}, harness search budget

KK
2:

t←0t\\leftarrow 0, randomly initialize harness search set

𝒮t⊂𝒟\\mathcal\{S\}\_\{t\}\\subset\\mathcal\{D\}
3:whilenot convergeddo⊳\\trianglerightStopping criterion \(§[2](https://arxiv.org/html/2610.22791#S2)\)

4:/\* H\-step: search harness with the current policy as proposer \*/

5:Score

v0v\_\{0\}, trajectories

𝒯0←Eval⁡\(Ht,Pt,𝒮t\)\\mathcal\{T\}\_\{0\}\\leftarrow\\operatorname\{Eval\}\(H\_\{t\},P\_\{t\},\\mathcal\{S\}\_\{t\}\), experience

ℰ←\{\(Ht,𝒯0\)\}\\mathcal\{E\}\\leftarrow\\\{\(H\_\{t\},\\mathcal\{T\}\_\{0\}\)\\\},

vbest←v0v\_\{\\mathrm\{best\}\}\\leftarrow v\_\{0\},

Ht\+1←HtH\_\{t\+1\}\\leftarrow H\_\{t\}
6:for

k=1,…,Kk=1,\\ldots,Kdo

7:

Htk←ProposeHarness⁡\(Ht\+1,Pt,ℰ\)H\_\{t\}^\{k\}\\leftarrow\\operatorname\{ProposeHarness\}\(H\_\{t\+1\},P\_\{t\},\\mathcal\{E\}\)⊳\\trianglerightSelf\-proposer \(§[2\.2](https://arxiv.org/html/2610.22791#S2.SS2)\)

8:

vk,𝒯k←Eval⁡\(Htk,Pt,𝒮t\)v\_\{k\},\\mathcal\{T\}\_\{k\}\\leftarrow\\operatorname\{Eval\}\(H\_\{t\}^\{k\},P\_\{t\},\\mathcal\{S\}\_\{t\}\),

ℰ←ℰ∪\{\(Htk,𝒯k\)\}\\mathcal\{E\}\\leftarrow\\mathcal\{E\}\\cup\\\{\(H\_\{t\}^\{k\},\\mathcal\{T\}\_\{k\}\)\\\}
9:if

vk\>vbestv\_\{k\}\>v\_\{\\mathrm\{best\}\}then

10:

Ht\+1←HtkH\_\{t\+1\}\\leftarrow H\_\{t\}^\{k\},

vbest←vkv\_\{\\mathrm\{best\}\}\\leftarrow v\_\{k\}
11:/\* P\-step: train the policy under the updated harness \*/

12:

Pt\+1,𝒯RL←RL⁡\(Ht\+1,Pt,𝒟\)P\_\{t\+1\},\\mathcal\{T\}\_\{\\mathrm\{RL\}\}\\leftarrow\\operatorname\{RL\}\(H\_\{t\+1\},P\_\{t\},\\mathcal\{D\}\)
13:

𝒮t\+1←SelectHardTasks⁡\(𝒯RL\)\\mathcal\{S\}\_\{t\+1\}\\leftarrow\\operatorname\{SelectHardTasks\}\(\\mathcal\{T\}\_\{\\mathrm\{RL\}\}\)⊳\\trianglerightEvolving search set \(§[2\.3](https://arxiv.org/html/2610.22791#S2.SS3)\)

14:

t←t\+1t\\leftarrow t\+1
15:return

HtH\_\{t\},

PtP\_\{t\}

A formal description ofSelf\-HPOis presented in Algorithm[1](https://arxiv.org/html/2610.22791#alg1)\. Starting from a base policyP0P\_\{0\}, a seed harnessH0H\_\{0\}, and a random search set𝒮0⊂𝒟\\mathcal\{S\}\_\{0\}\\subset\\mathcal\{D\}, the method alternates two updates\. At roundtt\(starting fromt=0t=0\), the H\-step keepsPtP\_\{t\}fixed and uses it to proposeKKharness candidates\{Htk\}k=1K\\\{H\_\{t\}^\{k\}\\\}\_\{k=1\}^\{K\}, retaining the best\-scoring harness on𝒮t\\mathcal\{S\}\_\{t\}asHt\+1H\_\{t\+1\}\. The P\-step then freezesHt\+1H\_\{t\+1\}and updatesPtP\_\{t\}toPt\+1P\_\{t\+1\}on𝒟\\mathcal\{D\}using reinforcement learning\. The resulting training rollouts identify the next hard\-task set𝒮t\+1\\mathcal\{S\}\_\{t\+1\}and the loop stops when the hard\-task set stabilizes between rounds\.

### 2\.2H\-step: Failure\-Conditioned Harness Search

#### Self\-proposer for harness optimization\.

The H\-step uses the frozenPtP\_\{t\}as both task solver and harness proposer\. The current pair\(Ht,Pt\)\(H\_\{t\},P\_\{t\}\)first generates trajectories on𝒮t\\mathcal\{S\}\_\{t\}, and the proposer inspects the current failures and produces targeted revisions to the prompt or executable runtime logic \(see prompt in Appendix[D](https://arxiv.org/html/2610.22791#A4)\)\. Each of theKKsearch iterations revises the current best harness, starting fromHtH\_\{t\}, using the accumulated experience of all evaluated harnesses and their trajectories\. All candidates are evaluated on the same search set𝒮t\\mathcal\{S\}\_\{t\}\.

#### Evolving search set\.

The search set remains fixed within an H\-step but evolves between rounds\. For initialization, we sample𝒮0\\mathcal\{S\}\_\{0\}uniformly from𝒟\\mathcal\{D\}\. Subsequent search sets consist of hard tasks identified from the training trajectories generated during the preceding P\-step, where a task is considered hard if it is never resolved in any of its rollouts in that P\-step\. The next H\-step regenerates trajectories on those hard tasks with the updated harness–policy pair, producing fresh failure evidence for proposal\. Harness search therefore follows the failures exposed by the preceding policy update instead of repeatedly optimizing against the initial tasks\.

### 2\.3P\-step: Policy Optimization under the Optimized Harness

The P\-step improves the policy through reinforcement learning under the optimized harnessHt\+1H\_\{t\+1\}\. Starting fromPtP\_\{t\}, we collect on\-policy trajectories over𝒟\\mathcal\{D\}as the policy is updated\. An executable verifier assigns binary outcome rewards, and our default optimizer combines Group Relative Policy Optimization \(GRPO\)\([Shao et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib31)\)with Balanced Adaptive Rollout \(BAR\) from Orchard\([Peng et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib17)\)\. BAR adaptively draws additional trajectories to construct reward\-balanced groups, reducing uninformative zero\-variance groups under sparse outcome rewards, and GRPO then updates the policy using the resulting group\-relative advantages\. The tasks that were never resolved during this P\-step are identified as the hard\-task set𝒮t\+1\\mathcal\{S\}\_\{t\+1\}, which will be passed together with the updated policyPt\+1P\_\{t\+1\}to the next H\-step\.

## 3Experiments

### 3\.1Experimental Setup

#### Harness, model, and data\.

All experiments use Qwen3\.5\-35B\-A3B\([Qwen Team, 2026](https://arxiv.org/html/2610.22791#bib.bib21)\)as the base policy and mini\-SWE\-agent\([Yao et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib23);[Yang et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib15)\)as the seed harness\. Following Orchard\([Peng et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib17)\), we construct a training set of 2,484 tasks from SWE\-rebench\([Badertdinov et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib93)\), SWE\-rebench V2\([Badertdinov et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib20)\), and Scale\-SWE\([Zhao et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib94)\), which is used for both harness search and policy optimization\. We perform two rounds of alternating harness and policy optimization based on the stopping criterion, withK=6K=6harness candidates per H\-step and 50 RL updates per P\-step\. Appendix[A](https://arxiv.org/html/2610.22791#A1)describes the implementation details\.

#### Benchmarks and evaluation metrics\.

SWE\-Bench Verified \(SWE\-Verified\) contains 500 human\-validated GitHub issues and is our primary benchmark\([Jimenez et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib14);[Chowdhury et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib24)\)\. We further evaluate out\-of\-distribution performance on SWE\-Bench Multilingual \(SWE\-Multi; 300 tasks\)\([Yang et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib25)\)and the public V2 set of SWE\-Bench Pro \(SWE\-Pro; 642 tasks\)\([Deng et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib26)\), covering other programming languages and more diverse repository\-level tasks\. We report resolve rate, the percentage of tasks whose generated patches pass the benchmark verifier\. Main results are averaged over three evaluation runs \(avg@3\) with standard deviations\.

#### Baselines\.

We compare our method with policy\-only and harness\-only baselines under two setups, with the base agent as a reference\. In the*self\-improving setup*, RLVR\([Peng et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib17)\)updates the policy under the seed harness, whereas Self\-Harness\([Zhang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib18)\)optimizes the harness using the base policy as proposer without updating its weights\. Self\-HPO alternates both updates and uses the current policy to propose harness changes\. In the*external\-model setup*, supervised fine\-tuning \(SFT\) learns from resolved Opus 4\.8 trajectories on the same training tasks, while Meta\-Harness\([Lee et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib16)\)uses Opus 4\.8 to optimize the harness with the target policy fixed\. HPO combines external harness proposals with RL policy updates\.

Table 1:Main results on in\-distribution \(SWE\-Bench Verified\) and out\-of\-distribution evaluations \(SWE\-Bench Multilingual and SWE\-Bench Pro\)\.The optimization columns indicate whether each method updates the harness or policy\. We report the mean resolve rates with standard deviations over three runs\. Bold marks the best result within each setup\.MethodHarness OptPolicy OptSWE\-VerifiedSWE\-MultiSWE\-ProAvg\.Base✘✘61\.0±0\.261\.0\{\\scriptstyle\\,\\pm\\,0\.2\}54\.2±0\.354\.2\{\\scriptstyle\\,\\pm\\,0\.3\}58\.3±1\.258\.3\{\\scriptstyle\\,\\pm\\,1\.2\}57\.8±0\.457\.8\{\\scriptstyle\\,\\pm\\,0\.4\}Setting 1: External\-model setup \(Opus 4\.8\)SFT✘✓65\.0±1\.165\.0\{\\scriptstyle\\,\\pm\\,1\.1\}49\.6±0\.849\.6\{\\scriptstyle\\,\\pm\\,0\.8\}49\.1±0\.649\.1\{\\scriptstyle\\,\\pm\\,0\.6\}54\.6±0\.554\.6\{\\scriptstyle\\,\\pm\\,0\.5\}Meta\-Harness\([Lee et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib16)\)✓✘69\.1±1\.969\.1\{\\scriptstyle\\,\\pm\\,1\.9\}56\.5±0\.956\.5\{\\scriptstyle\\,\\pm\\,0\.9\}64\.8±1\.464\.8\{\\scriptstyle\\,\\pm\\,1\.4\}63\.5±0\.863\.5\{\\scriptstyle\\,\\pm\\,0\.8\}HPO \(ours\)✓✓71\.6±0\.9\\mathbf\{71\.6\}\{\\scriptstyle\\,\\bm\{\\pm\}\\,\\mathbf\{0\.9\}\}61\.4±0\.7\\mathbf\{61\.4\}\{\\scriptstyle\\,\\bm\{\\pm\}\\,\\mathbf\{0\.7\}\}66\.0±0\.5\\mathbf\{66\.0\}\{\\scriptstyle\\,\\bm\{\\pm\}\\,\\mathbf\{0\.5\}\}66\.3±0\.4\\mathbf\{66\.3\}\{\\scriptstyle\\,\\bm\{\\pm\}\\,\\mathbf\{0\.4\}\}Setting 2: Self\-improving setupRLVR\([Peng et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib17)\)✘✓67\.5±0\.867\.5\{\\scriptstyle\\,\\pm\\,0\.8\}56\.3±1\.056\.3\{\\scriptstyle\\,\\pm\\,1\.0\}61\.8±1\.361\.8\{\\scriptstyle\\,\\pm\\,1\.3\}61\.9±0\.661\.9\{\\scriptstyle\\,\\pm\\,0\.6\}Self\-Harness\([Zhang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib18)\)✓✘65\.2±1\.565\.2\{\\scriptstyle\\,\\pm\\,1\.5\}54\.3±0\.754\.3\{\\scriptstyle\\,\\pm\\,0\.7\}60\.7±0\.960\.7\{\\scriptstyle\\,\\pm\\,0\.9\}60\.1±0\.660\.1\{\\scriptstyle\\,\\pm\\,0\.6\}Self\-HPO \(ours\)✓✓69\.5±0\.9\\mathbf\{69\.5\}\{\\scriptstyle\\,\\bm\{\\pm\}\\,\\mathbf\{0\.9\}\}59\.7±0\.8\\mathbf\{59\.7\}\{\\scriptstyle\\,\\bm\{\\pm\}\\,\\mathbf\{0\.8\}\}65\.2±1\.4\\mathbf\{65\.2\}\{\\scriptstyle\\,\\bm\{\\pm\}\\,\\mathbf\{1\.4\}\}64\.8±0\.6\\mathbf\{64\.8\}\{\\scriptstyle\\,\\bm\{\\pm\}\\,\\mathbf\{0\.6\}\}

### 3\.2Main Results

#### Self\-HPOconsistently outperforms policy\-only and harness\-only optimization\.

As shown in Table[1](https://arxiv.org/html/2610.22791#S3.T1), jointly optimizing the harness and policy achieves the best performance on all three benchmarks within each setup\. Without an external proposer, Self\-HPO reaches69\.5%69\.5\\%on SWE\-Verified, outperforming RLVR and Self\-Harness by2\.02\.0and4\.34\.3points, respectively\. Its average score improves from57\.8%57\.8\\%for Base to64\.8%64\.8\\%, exceeding the stronger single\-component baseline, RLVR, by2\.92\.9points\. The same pattern holds with external supervision: HPO surpasses Meta\-Harness by2\.52\.5points on SWE\-Verified and2\.82\.8points on average\. These consistent gains support optimizing the harness and policy together\. Appendix[B](https://arxiv.org/html/2610.22791#A2.SS0.SSS0.Px5)further compares these results with frontier models on SWE\-Verified\.

#### The gains generalize to out\-of\-distribution benchmarks\.

Although harness search and policy optimization use only tasks from Python projects, the gains transfer to SWE\-Multi, which covers nine non\-Python languages, and SWE\-Pro, which spans a broader range of languages and engineering tasks\. Self\-HPO achieves59\.7%59\.7\\%and65\.2%65\.2\\%, respectively, improving over Base by5\.55\.5and6\.96\.9points and over RLVR by3\.43\.4points on both benchmarks\. The benefit is particularly clear on SWE\-Multi, where Self\-Harness improves over Base by only0\.10\.1point, compared with5\.55\.5points for Self\-HPO\. The same trend holds in the external\-model setup, where HPO reaches61\.4%61\.4\\%and66\.0%66\.0\\%, outperforming both single\-component baselines\. These results show that the benefits of joint harness–policy optimization extend beyond the Python training domain\.

#### Learning from Opus 4\.8 through harness optimization is more effective than SFT distillation\.

SFT and Meta\-Harness represent two different approaches for transferring knowledge from Opus 4\.8: SFT distills generated trajectories into the model weights, whereas Meta\-Harness uses external proposals to optimize the harness while keeping the model fixed\. Meta\-Harness outperforms SFT by4\.14\.1points on SWE\-Verified and8\.98\.9points on average\. The gap is largest out of domain: SFT falls below Base on both SWE\-Multi and SWE\-Pro, while Meta\-Harness improves on both\. HPO further improves on Meta\-Harness, indicating that policy optimization remains beneficial when coupled with an optimized harness\.

![Refer to caption](https://arxiv.org/html/2610.22791v1/optimization_trajectories.png)Figure 3:Optimization trajectories across harness and policy updates\.Resolve rates \(%\) on SWE\-Bench Verified, SWE\-Bench Multilingual, and SWE\-Bench Pro are shown on the common Base–H1H\_\{1\}–P1P\_\{1\}–H2H\_\{2\}–P2P\_\{2\}schedule\. Self\-HPO alternates H\- and P\-steps, whereas Self\-Harness and RLVR update only the harness and policy, respectively\. Markers with dashed outlines indicate steps at which the corresponding method performs no update\.
#### Alternating harness and policy optimization drives continued agent improvement\.

Figure[3](https://arxiv.org/html/2610.22791#S3.F3)traces the optimization trajectories of Self\-HPO and the two one\-sided baselines\. Self\-HPO and Self\-Harness begin with the same first H\-step, but their trajectories diverge once Self\-HPO updates its policy\. Across all three benchmarks, the second H\-step improves the updated Self\-HPO policy, and the following P\-step provides a further gain\. In contrast, another harness\-only update does not consistently improve performance: Self\-Harness drops from55\.0%55\.0\\%to54\.3%54\.3\\%on SWE\-Multi\. The final Self\-HPO pair surpasses both single\-component baselines on every benchmark, supporting repeated adaptation of the harness to the evolving policy\. Figure[6](https://arxiv.org/html/2610.22791#A2.F6)in the appendix provides a candidate\-level view of harness search on SWE\-Verified\.

### 3\.3Ablations

We examine the policy\-update strategy, search\-set construction, and harness proposer on SWE\-Verified\. Table[2](https://arxiv.org/html/2610.22791#S3.T2)uses HPO as the reference for ablating the shared H\-step–P\-step framework, and replacing its external proposer with the current policy gives Self\-HPO\.

Table 2:Ablation study\.Resolve rates reported on SWE\-Bench Verified\.#### RL yields stronger policy updates than SFT/RFT\.

We replace RL in the P\-step with either SFT on trajectories distilled from Opus 4\.8 or rejection\-sampling fine\-tuning \(RFT\) on the target policy’s own successful trajectories\. SFT reduces the final resolve rate from71\.6%71\.6\\%to67\.2%67\.2\\%, while RFT reaches70\.5%70\.5\\%, trailing RL by1\.11\.1points\. Even when fine\-tuning uses the policy’s own successful trajectories, it still falls short of RL, supporting RL as the default P\-step\.

#### Evolving the search set helps harness optimization keep pace with the policy\.

Reusing the initial search set lowers performance from71\.6%71\.6\\%to70\.1%70\.1\\%\. The evolving set instead directs the next H\-step toward hard tasks from the preceding P\-step rollouts\. The gain supports refreshing the search target as the agent improves: failures relevant to an earlier policy need not remain the most useful targets for its successor\.

#### Self\-proposal retains strong performance without an external proposer\.

Replacing Opus 4\.8 with the current target policy yields69\.5%69\.5\\%, a2\.12\.1\-point decrease from HPO\. Although external proposals provide an advantage, Self\-HPO still exceeds both self\-improving single\-component baselines in Table[1](https://arxiv.org/html/2610.22791#S3.T1)\. Therefore, an external model is helpful but not required for our method\.

### 3\.4Analysis

![Refer to caption](https://arxiv.org/html/2610.22791v1/passk_curve.png)\(a\)Pass@kk
![Refer to caption](https://arxiv.org/html/2610.22791v1/pass1_breakdown.png)\(b\)Pass@1 by task difficulty
\(c\)Zero\-success groups during RL
Figure 4:Pass@kkperformance on SWE\-Bench Verified and RL training signal\.\(a\) pass@kkcurve of Base andSelf\-HPO; \(b\) pass@1 by task difficulty, categorized by Base success ratepp: Hard \(p=0p=0\), Medium \(0<p≤0\.50<p\\leq 0\.5\), Easy \(0\.5<p<10\.5<p<1\), and Resolved \(p=1p=1\)\. \(c\) Fraction of zero\-success rollout groups for RLVR and Self\-HPO’s final P\-step\.Table 3:Cross\-evaluating Self\-HPO policies and harnesses on SWE\-Bench Verified\.Rows vary the harness and columns vary the policy\.#### Both optimized components retain benefits even when paired with their base counterparts\.

Table[3](https://arxiv.org/html/2610.22791#S3.T3)cross\-evaluates policies and harnesses\. ApplyingH2H\_\{2\}to the base policy improves resolve rate from61\.0%61\.0\\%to64\.5%64\.5\\%, while deployingP2P\_\{2\}under the seed harness reaches64\.4%64\.4\\%\. CombiningH2H\_\{2\}andP2P\_\{2\}reaches69\.5%69\.5\\%, an8\.58\.5\-point gain over Base that exceeds the sum of their individual gains \(3\.5\+3\.4=6\.93\.5\+3\.4=6\.9points\) by1\.61\.6points\. These results show that each component retains part of the improvement on its own, while combining them gives the largest gain\.

#### Self\-HPO improves reliability \(pass@1\) and expands the capability frontier \(pass@kk\)\.

To examine whether the gains persist beyond pass@1, Figures[4\(a\)](https://arxiv.org/html/2610.22791#S3.F4.sf1)and[4\(b\)](https://arxiv.org/html/2610.22791#S3.F4.sf2)compare Base and the final Self\-HPO pair using eight attempts per task on SWE\-Verified\. As shown in Figure[4\(a\)](https://arxiv.org/html/2610.22791#S3.F4.sf1), Self\-HPO remains above Base for everykkfrom 1 to 8, and pass@8 increases from80\.0%80\.0\\%to84\.8%84\.8\\%, a gain of4\.84\.8points\. Figure[4\(b\)](https://arxiv.org/html/2610.22791#S3.F4.sf2)shows pass@1 gains of17\.217\.2and12\.912\.9points on Medium and Easy tasks, where Base sometimes succeeds but remains inconsistent, and a6\.16\.1\-point gain on Hard tasks\. At the same time, Self\-HPO’s pass@1 decreases by2\.22\.2points on the Resolved group, showing that improvement is not uniform across tasks\. Together, the gains on Medium and Easy tasks demonstrate improved reliability, while the gains on Hard tasks and the higher pass@8 show the expanded capability frontier\.

#### Self\-HPOreduces zero\-success rollout groups during training\.

During the final Self\-HPO P\-step \(Figure[4\(c\)](https://arxiv.org/html/2610.22791#S3.F4.sf3)\), the fraction of zero\-success groups, in which no rollout succeeds, declines to6\.5%6\.5\\%over the final ten updates, whereas it remains mostly between11%11\\%and17%17\\%over the last 50 updates of RLVR\. Under binary outcome rewards, such groups give GRPO no within\-group reward contrast, so reducing them alleviates reward sparsity during policy optimization\.

`\\iow\_now:Ne˝\\iow\_now:Ne˝class ProgressTrackingAgent\(DefaultAgent\): \# seed agent\\iow\_now:Ne˝ \# default loop: query the model, then run its command\\iow\_now:Ne˝ def step\(self\):\\iow\_now:Ne˝ return self\.execute\_actions\(self\.query\(\)\)\\iow\_now:Ne˝\\iow\_now:Ne˝ \# stop the episode once the step budget is used up\\iow\_now:Ne˝ def query\(self\):\\iow\_now:Ne˝ if 0 < self\.config\.step\_limit <= self\.n\_calls:\\iow\_now:Ne˝ raise LimitsExceeded\(\{…, "submission": ""\}\)\\iow\_now:Ne˝ \|\\bad\{\(1\) budget runs out $\\rightarrow$ empty result\}\|\\iow\_now:Ne˝ …\\iow\_now:Ne˝\\iow\_now:Ne˝class OrchardEnvironment: \# sandbox\\iow\_now:Ne˝ \# after each command: did the model submit?\\iow\_now:Ne˝ def \_check\_finished\(self, output\):\\iow\_now:Ne˝ if lines\[0\] == "COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT":\\iow\_now:Ne˝ submission = ""\.join\(lines\[1:\]\)\\iow\_now:Ne˝ raise Submitted\(\{…, "submission": submission\}\)\\iow\_now:Ne˝ \|\\bad\{accepted as\-is, even if the patch is:\}\|\\iow\_now:Ne˝ \|\\bad\{\(2\) empty \(new file not staged\)\}\|\\iow\_now:Ne˝ \|\\bad\{\(3\) built on a guessed API / contract\}\|\\iow\_now:Ne˝ \|\\bad\{\(4\) malformed and does not apply\}\| Seed harness \\iow\_now:Ne˝\\iow\_now:Ne˝class AgentHarness\(ProgressTrackingAgent\): \# inherits seed\\iow\_now:Ne˝ \# same loop; overrides only \|\\cfn\{cStep\}\{step\}\|\(\), adds checks\\iow\_now:Ne˝ def step\(self\):\\iow\_now:Ne˝ \# budget valve \(fires once\): checked before every step\\iow\_now:Ne˝ if near\_step\_limit or near\_time\_limit:\\iow\_now:Ne˝ self\.add\_messages\(rescue\_text\)\\iow\_now:Ne˝ \|\\good\{\(1\) warn early; prompt a best\-effort submission\}\|\\iow\_now:Ne˝ try:\\iow\_now:Ne˝ return super\(\)\.step\(\) \# = \|\\cfn\{cExec\}\{execute\\string\_actions\}\|\(\|\\cfn\{cQuery\}\{query\}\|\(\)\)\\iow\_now:Ne˝ except Submitted as e: \# raised by \|\\cfn\{cCheck\}\{\\string\_check\\string\_finished\}\|\\iow\_now:Ne˝ \# empty\-patch guard \(fires once\)\\iow\_now:Ne˝ if \(is\_empty\(sub\) and not self\.\_empty\_guarded\\iow\_now:Ne˝ and has\_runway\(\)\):\\iow\_now:Ne˝ self\.add\_messages\(empty\_guard\_text\)\\iow\_now:Ne˝ return \[\]\\iow\_now:Ne˝ \|\\good\{\(2\) reject empty patch; ask to git add \-N new files\}\|\\iow\_now:Ne˝ \# contract grounding: first submit past the empty guard\\iow\_now:Ne˝ if not self\.\_grounded and has\_runway\(\):\\iow\_now:Ne˝ self\.add\_messages\(ground\_text\)\\iow\_now:Ne˝ return \[\]\\iow\_now:Ne˝ \|\\good\{\(3\) hold once; ask model to check APIs against repo tests\}\|\\iow\_now:Ne˝ \# final submit: patch fallback\\iow\_now:Ne˝ self\.\_apply\_fallback\(e\)\\iow\_now:Ne˝ raise\\iow\_now:Ne˝ \|\\good\{\(4\) apply check fails $\\rightarrow$ submit the sandbox’s git diff\}\| Optimized harness`Figure 5:Seed versus optimized harness \(HPO, second H\-step\)\.Left: the default agent loop, with the four failure modes it lets through \(✗\)\. Right: the optimized harness subclasses the seed agent and overrides onlystep\(\), and each added check \(✓\) addresses the matching failure mode\. Shared functions and classes use the same color on both sides\. See Figure[8](https://arxiv.org/html/2610.22791#A4.F8)for a detailed diff\.

### 3\.5Case Studies

#### Harness evolution adds targeted controls around the existing agent loop\.

Figure[5](https://arxiv.org/html/2610.22791#S3.F5)contrasts the seed harness with the harness obtained after HPO’s second H\-step\. The optimized harness retains the underlying model\-query and command\-execution loop, but intervenes before budget exhaustion and at submission\. First, it warns the model as the step or time limit approaches and prompts a best\-effort submission before the episode ends with an empty result\. Second, a one\-time empty\-patch guard asks the model to include newly created files viagit add \-N\. Third, it delays the first submission that passes the empty\-patch guard to prompt a check of API names and signatures against the repository and tests\. Finally, a fallback uses the sandbox’s Git diff when the submitted patch fails the apply check\. These changes target distinct ways in which useful repository edits can fail to become a valid submission\. They combine explicit runtime checks, such as detecting an empty patch, with behavioral guidance, such as requesting an interface check\. Harness optimization can therefore change execution and termination logic, in addition to prompts\. Appendix[D](https://arxiv.org/html/2610.22791#A4)gives a detailed diff against the seed harness together with the injected messages, and Appendix[E](https://arxiv.org/html/2610.22791#A5)walks through a task on which an evolved harness changes the outcome for the same policy\.

## 4Related Work

#### Harness optimization for LLM agents\.

Harnesses govern an agent’s interaction with tools and environments\([Yao et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib23);[Yang et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib15)\)\. Optimization spans prompts and programs\([Pryzant et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib91);[Khattab et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib50);[Fernando et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib51);[Yuksekgonul et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib90);[Agrawal et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib52)\), contexts\([Zhang et al\., 2026f](https://arxiv.org/html/2610.22791#bib.bib41);[Ye et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib56);[Sun et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib74)\), and workflows\([Hu et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib42);[Zhang et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib43);[Wang et al\., 2025a](https://arxiv.org/html/2610.22791#bib.bib72);[Gao et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib73)\)\. Agents reuse feedback through refinement and reflection\([Madaan et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib54);[Shinn et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib53)\), skills\([Wang et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib81)\), and memory\([Zhao et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib82);[Wang et al\., 2025c](https://arxiv.org/html/2610.22791#bib.bib83)\)\. Program evolution also improves solutions and agents\([Novikov et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib77);[Lange et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib76);[Zelikman et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib55);[Zhang et al\., 2026b](https://arxiv.org/html/2610.22791#bib.bib44);[Zhang et al\., 2026c](https://arxiv.org/html/2610.22791#bib.bib62)\)\. Recent systems revise harnesses from execution evidence\([Lee et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib16);[Lin et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib29);[Zhang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib18);[Park et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib30);[Wei et al\., 2026b](https://arxiv.org/html/2610.22791#bib.bib64);[Xu et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib4)\), shared experience\([Weng et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib71)\), demonstrations\([Che et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib69);[Jiang et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib13)\), or self\-preferences\([Pan et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib19)\)\. Other work studies continual adaptation\([Liu et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib39);[Xu et al\., 2026b](https://arxiv.org/html/2610.22791#bib.bib100)\), task selection\([Miyai et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib67)\), generalization through search controls and modular edits\([Zhang et al\., 2026d](https://arxiv.org/html/2610.22791#bib.bib68);[Xia et al\., 2026b](https://arxiv.org/html/2610.22791#bib.bib9);[Wu et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib10)\), and fixed\-budget evaluation\([Ursekar et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib70)\)\. These methods adapt the agent around a fixed task\-solving policy, whereas Self\-HPO also updates the policy that the harness serves\.

#### Policy learning and self\-improvement\.

Iterative training uses filtered reasoning\([Zelikman et al\., 2022](https://arxiv.org/html/2610.22791#bib.bib57);[Singh et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib58);[Yuan et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib27)\), refreshed preferences\([Chen et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib60);[Yuan et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib59);[Pang et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib66)\), self\-play curricula\([Zhao et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib61);[Huang et al\., 2026c](https://arxiv.org/html/2610.22791#bib.bib47);[Huang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib63)\), negative teachers\([Pei et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib48)\), or evolving environments\([Huang et al\., 2026b](https://arxiv.org/html/2610.22791#bib.bib49)\)\. Software engineering agents learn from executable environments\([Pan et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib86);[Jain et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib87);[Yang et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib25)\), software\-evolution supervision\([Wei et al\., 2025a](https://arxiv.org/html/2610.22791#bib.bib88)\), and self\-play code repair\([Wei et al\., 2025b](https://arxiv.org/html/2610.22791#bib.bib89);[Choi et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib80)\)\. Policy improvement also draws on verifiable\-reward RL\([Shao et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib31);[Guo et al\., 2025](https://arxiv.org/html/2610.22791#bib.bib32);[Peng et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib17)\), harness\-guided distillation\([Ye et al\., 2026b](https://arxiv.org/html/2610.22791#bib.bib38)\), and test\-time learning during solution search\([Wang et al\., 2025b](https://arxiv.org/html/2610.22791#bib.bib78);[Yuksekgonul et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib79)\)\. These methods train the policy under a fixed harness, whereas in Self\-HPO the harness co\-evolves with the policy being trained\.

#### Joint harness–policy evolution\.

Joint adaptation spans prompts and weights\([Soylu et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib2);[Tiwari et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib3);[Zhang et al\., 2026e](https://arxiv.org/html/2610.22791#bib.bib5);[Lu et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib6)\), executable harnesses and policies\([Hebbar et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib45);[Karten et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib46);[Chen et al\., 2026c](https://arxiv.org/html/2610.22791#bib.bib33);[Kim et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib40);[Luo et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib35);[Chen et al\., 2026b](https://arxiv.org/html/2610.22791#bib.bib36);[Chen et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib8)\), and policies with evolving skills or external state\([Xia et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib84);[Fu et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib85);[He et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib7);[Ning et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib65)\), including safety\-oriented co\-evolution\([Mao et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib12)\)\. MetaRSI composes data generation, harness editing, and model training within a self\-improvement loop\([Tan et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib1)\)\. Concurrent work traces interventions\([Fan and Huang, 2026](https://arxiv.org/html/2610.22791#bib.bib34)\), studies transfer and multi\-harness training\([Le et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib37);[Wei et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib11)\), and separates the ability to edit a harness from the ability to benefit from one\([Lin et al\., 2026b](https://arxiv.org/html/2610.22791#bib.bib75)\)\. Self\-HPO differs from these methods along two axes\. First, the policy trained in the P\-step becomes the harness proposer of the next H\-step, whereas WHALE\([Kim et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib40)\)keeps a frozen external proposer and Evo2\([Xu et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib4)\)trains a separate proposer for a frozen solver\. HASE\([Luo et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib35)\)also trains a self\-proposing policy, but places harness edits in the RL action space instead of alternating harness search with RL\. Second, Self\-HPO selects the next harness search set from tasks that remain unsolved during RL, whereas WHALE, Evo2, and HASE search on a fixed task set\. Appendix[C](https://arxiv.org/html/2610.22791#A3)gives a more detailed comparison with concurrent work\.

## 5Conclusion

We introduced Self\-HPO, a recursive self\-improving framework for coding agents that alternates harness and policy optimization\. The evolving policy proposes its own harness revisions, learns through RL under the selected harness, and supplies hard tasks for the next round of harness search\. Across three coding benchmarks, Self\-HPO consistently outperforms policy\-only and harness\-only baselines, improving the average resolve rate from57\.8%57\.8\\%to64\.8%64\.8\\%\. Analyses further show that harness and policy improvements are complementary and that Self\-HPO improves reliability \(pass@1\) and expands the capability frontier \(pass@kk\) over the base agent\.

## References

- Agasheet al\.\(2025\)S\. Agashe, K\. Wong, V\. Tu, J\. Yang, A\. Li, and X\. E\. WangAgent S2: a compositional generalist\-specialist framework for computer use agents\.arXiv preprint arXiv:2504\.00906\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p1.1)\.
- Agrawalet al\.\(2026\)L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. G\. Dimakis, I\. Stoica, D\. Klein, M\. Zaharia, and O\. KhattabGEPA: reflective prompt evolution can outperform reinforcement learning\.InInternational Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Badertdinovet al\.\(2025\)I\. Badertdinov, A\. Golubev, M\. Nekrashevich, A\. Shevtsov, S\. Karasik, A\. Andriushchenko, M\. Trofimova, D\. Litvintseva, and B\. YangelSWE\-rebench: an automated pipeline for task collection and decontaminated evaluation of software engineering agents\.arXiv preprint arXiv:2505\.20411\.Cited by:[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px1.p1.1)\.
- Badertdinovet al\.\(2026\)I\. Badertdinov, M\. Nekrashevich, A\. Shevtsov, and A\. GolubevSWE\-rebench V2: language\-agnostic SWE task collection at scale\.arXiv preprint arXiv:2602\.23866\.Cited by:[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px1.p1.1)\.
- Cheet al\.\(2026\)L\. Che, Y\. Yang, P\. Lin, X\. Cao, C\. Wang, X\. Wang, and J\. SuDemoEvolve: demonstration\-guided harness evolution under sparse feedback\.arXiv preprint arXiv:2605\.24539\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2021\)M\. Chenet al\.Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[Appendix B](https://arxiv.org/html/2610.22791#A2.SS0.SSS0.Px4.p1.1)\.
- Chenet al\.\(2026a\)M\. Chen, C\. Lv, G\. Zhang, H\. Chang, and S\. ZhouHarnessForge: joint harness and policy evolution for adaptive agent systems\.arXiv preprint arXiv:2606\.01779\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p2.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2026b\)T\. Chen, S\. Lu, K\. Zhao, W\. Meng, H\. Teng, T\. Li, C\. Li, X\. Liu, J\. Liang, Z\. Zhang, Y\. Xie, H\. Qu, K\. Shao, and J\. LuanHarnessX: a composable, adaptive, and evolvable agent harness foundry\.arXiv preprint arXiv:2606\.14249\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2026c\)Z\. Chen, T\. Xiao, H\. Zhu, Y\. Yuan, L\. Zhang, and J\. WangCo\-Harness: co\-evolving harnesses and model weights for LLM agents\.arXiv preprint arXiv:2607\.22688\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2024\)Z\. Chen, Y\. Deng, H\. Yuan, K\. Ji, and Q\. GuSelf\-play fine\-tuning converts weak language models to strong language models\.InForty\-first International Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Choiet al\.\(2026\)C\. Choi, Z\. Kaya, S\. Wu, T\. Ma, T\. Hashimoto, and L\. SchmidtAnchored self\-play for code repair\.arXiv preprint arXiv:2607\.03523\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Chowdhuryet al\.\(2024\)N\. Chowdhury, J\. Aung, C\. J\. Shern, O\. Jaffe, D\. Sherburn, G\. Starace, E\. Mays, R\. Dias, M\. Aljubeh, M\. Glaese, C\. E\. Jimenez, J\. Yang, L\. Ho, T\. Patwardhan, K\. Liu, and A\. MadryIntroducing SWE\-bench Verified\.External Links:[Link](https://openai.com/index/introducing-swe-bench-verified/)Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p4.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px2.p1.1)\.
- Denget al\.\(2025\)X\. Deng, J\. Da, E\. Pan, Y\. Y\. He, C\. Ide, K\. Garg, N\. Lauffer, A\. Park, N\. Pasari, C\. Rane, K\. Sampath, M\. Krishnan, S\. Kundurthy, S\. Hendryx, Z\. Wang, V\. Bharadwaj, J\. Holm, R\. Aluri, C\. B\. C\. Zhang, N\. Jacobson, B\. Liu, and B\. KenstlerSWE\-Bench Pro: can AI agents solve long\-horizon software engineering tasks?\.arXiv preprint arXiv:2509\.16941\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p4.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px2.p1.1)\.
- Fan and Huang \(2026\)T\. Fan and C\. HuangHELIX: model\-harness co\-evolution for recursive self\-improvement\.arXiv preprint arXiv:2608\.13951\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Fernandoet al\.\(2024\)C\. Fernando, D\. S\. Banarse, H\. Michalewski, S\. Osindero, and T\. RocktäschelPromptbreeder: self\-referential self\-improvement via prompt evolution\.InForty\-first International Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Fuet al\.\(2026\)Z\. Fu, Z\. Li, Q\. Ai, H\. Wu, M\. Wu, C\. Zhao, A\. Wang, G\. He, and C\. WangSelf\-play meets skill evolution: self\-evolving search agents that pose, solve, and remember\.arXiv preprint arXiv:2607\.29468\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Gaoet al\.\(2025\)H\. Gao, Y\. Liu, Y\. He, L\. Dou, C\. Du, Z\. Deng, B\. Hooi, M\. Lin, and T\. PangFlowReasoner: reinforcing query\-level meta\-agents\.arXiv preprint arXiv:2504\.15257\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Guoet al\.\(2025\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.DeepSeek\-R1 incentivizes reasoning in LLMs through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p2.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Heet al\.\(2024\)H\. He, W\. Yao, K\. Ma, W\. Yu, Y\. Dai, H\. Zhang, Z\. Lan, and D\. YuWebVoyager: building an end\-to\-end web agent with large multimodal models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 6864–6890\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p1.1)\.
- Heet al\.\(2026\)Z\. He, H\. Lin, B\. Han, W\. Zhu, H\. Fang, B\. Wang, X\. Zhu, R\. Li, and M\. ReimherrReSkill: reconciling skill creation with policy optimization in agentic RL\.arXiv preprint arXiv:2606\.01619\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Hebbaret al\.\(2026\)P\. Hebbar, Y\. Manawat, S\. Verboomen, A\. Ivanova, S\. Palanimalai, K\. Bhatia, and V\. BaskaranSIA: self improving AI with harness & weight updates\.arXiv preprint arXiv:2605\.27276\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Huet al\.\(2024\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.arXiv preprint arXiv:2408\.08435\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p2.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Huanget al\.\(2026a\)C\. Huang, H\. Liu, T\. Zheng, R\. Dai, L\. Huang, J\. Li, Z\. Li, Z\. Wei, Y\. Meng, and J\. HuangG\-Zero: self\-play for open\-ended generation from zero data\.arXiv preprint arXiv:2605\.09959\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Huanget al\.\(2026b\)C\. Huang, Z\. Wang, R\. Han, J\. Yan, Y\. Chen, Z\. CuiZhu, K\. Jiang, P\. Xia, H\. Yu, Y\. Zhuang, Y\. Ming, J\. Pan, B\. D\. Mishra, J\. Huang, B\. Gokturk, T\. Pfister, and C\. LeeEnvHarness: awakening static worlds for agent learning\.arXiv preprint arXiv:2608\.19880\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Huanget al\.\(2026c\)C\. Huang, W\. Yu, X\. Wang, H\. Zhang, Z\. Li, R\. Li, J\. Huang, H\. Mi, and D\. YuR\-Zero: self\-evolving reasoning LLM from zero data\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Jainet al\.\(2025\)N\. Jain, J\. Singh, M\. Shetty, L\. Zheng, K\. Sen, and I\. StoicaR2E\-Gym: procedural environments and hybrid verifiers for scaling open\-weights SWE agents\.arXiv preprint arXiv:2504\.07164\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Jianget al\.\(2026\)W\. Jiang, M\. Chu, Y\. Tian, Q\. Zhang, H\. Yang, R\. Yang, Y\. Liu, T\. Lv, and F\. LiHarnessEvolve: learning from reference trajectories for reliable agent self\-evolution\.arXiv preprint arXiv:2609\.00829\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. NarasimhanSWE\-bench: can language models resolve real\-world Github issues?\.InThe Twelfth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p1.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px2.p1.1)\.
- Kartenet al\.\(2026\)S\. Karten, J\. Zhang, T\. Upaa, R\. Feng, W\. Li, C\. Shi, C\. Jin, and K\. VodrahalliContinual Harness: online adaptation for self\-improving foundation agents\.arXiv preprint arXiv:2605\.09998\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Khattabet al\.\(2023\)O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Vardhamanan, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. PottsDSPy: compiling declarative language model calls into self\-improving pipelines\.arXiv preprint arXiv:2310\.03714\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Kimet al\.\(2026\)H\. Kim, Y\. Lee, G\. Lee, C\. Finn, and K\. LeeWHALE: a simple recipe for joint harness\-weight optimization\.arXiv preprint arXiv:2609\.00196\.Cited by:[Appendix C](https://arxiv.org/html/2610.22791#A3.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p2.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Langeet al\.\(2026\)R\. T\. Lange, Y\. Imajuku, and E\. CetinShinkaEvolve: towards open\-ended and sample\-efficient program evolution\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Leet al\.\(2026\)C\. Le, J\. Cheng, Q\. He, R\. Li, Y\. Li, and X\. ChenMulti\-harness reinforcement learning for coding agents: training gains and harness dependence\.arXiv preprint arXiv:2609\.04518\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Leeet al\.\(2026\)Y\. Lee, R\. Nair, Q\. Zhang, K\. Lee, O\. Khattab, and C\. FinnMeta\-Harness: end\-to\-end optimization of model harnesses\.arXiv preprint arXiv:2603\.28052\.Cited by:[Appendix C](https://arxiv.org/html/2610.22791#A3.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p2.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2610.22791#S3.T1.4.1.5.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Linet al\.\(2026a\)J\. Lin, S\. Liu, C\. Pan, L\. Lin, S\. Dou, Z\. Xi, X\. Huang, H\. Yan, Z\. Han, T\. Gui, and Y\. JiangAgentic harness engineering: observability\-driven automatic evolution of coding\-agent harnesses\.arXiv preprint arXiv:2604\.25850\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Linet al\.\(2026b\)M\. Lin, J\. Wu, Z\. Wang, Z\. Shi, Y\. Sang, B\. He, Z\. Liu, T\. Wei, Z\. Wu, Z\. Zhang, D\. Wang, X\. Zhang, B\. Dumoulin, C\. Xie, Y\. Zhou, S\. Wang, and H\. LuHarness updating is not harness benefit: disentangling evolution capabilities in self\-evolving LLM agents\.arXiv preprint arXiv:2605\.30621\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Liuet al\.\(2026\)Z\. Liu, Z\. Shi, Y\. Sang, B\. He, M\. Lin, T\. Wei, D\. Wang, B\. Dumoulin, W\. Jin, and H\. LuAdaptive Auto\-Harness: sustained self\-improvement for agentic system deployment on open\-ended task streams\.arXiv preprint arXiv:2606\.01770\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Luet al\.\(2026\)X\. Lu, K\. Zhang, J\. Yang, B\. Cao, Y\. Lu, H\. Lin, M\. He, X\. Han, and L\. SunP2\{\}^\{2\}O: joint policy and prompt optimization\.arXiv preprint arXiv:2603\.21877\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Luoet al\.\(2026\)H\. Luo, Y\. Huang, S\. Luo, F\. Liu, L\. Li, Z\. Hu, J\. Feng, and Q\. LiuHarness\-aware self\-evolving: co\-evolving model weights, harness, and task solutions\.arXiv preprint arXiv:2607\.03935\.Cited by:[Appendix C](https://arxiv.org/html/2610.22791#A3.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Madaanet al\.\(2023\)A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. ClarkSelf\-Refine: iterative refinement with self\-feedback\.arXiv preprint arXiv:2303\.17651\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Maoet al\.\(2026\)Q\. Mao, W\. Qu, D\. Guo, L\. Yuan, Q\. Liu, Y\. Li, G\. Chen, Y\. Fu, X\. Lin, X\. Hu, and D\. LiuSafeEvolve: harness\-policy co\-evolution from agent experience for safety alignment\.arXiv preprint arXiv:2609\.02786\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Miyaiet al\.\(2026\)A\. Miyai, K\. Aizawa, and T\. YamasakiTask\-CoEvolve: efficient harness optimization via adaptive validation task selection\.arXiv preprint arXiv:2608\.20169\.Cited by:[Appendix C](https://arxiv.org/html/2610.22791#A3.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Ninget al\.\(2026\)X\. Ning, D\. Fu, T\. Wei, Y\. Bei, X\. Yang, W\. Xu, Y\. Song, B\. Li, Z\. Li, H\. Zeng,et al\.EvoHarness\-RL: learning runtime harness coordination for self\-evolving agents\.arXiv preprint arXiv:2608\.05446\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Novikovet al\.\(2025\)A\. Novikov, N\. Vũ, M\. Eisenberger, E\. Dupont, P\. Huang, A\. Z\. Wagner, S\. Shirobokov, B\. Kozlovskii, F\. J\. R\. Ruiz, A\. Mehrabian, M\. P\. Kumar, A\. See, S\. Chaudhuri, G\. Holland, A\. Davies, S\. Nowozin, P\. Kohli, and M\. BalogAlphaEvolve: a coding agent for scientific and algorithmic discovery\.arXiv preprint arXiv:2506\.13131\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Panet al\.\(2025\)J\. Pan, X\. Wang, G\. Neubig, N\. Jaitly, H\. Ji, A\. Suhr, and Y\. ZhangTraining software engineering agents and verifiers with SWE\-Gym\.InForty\-second International Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Panet al\.\(2026\)W\. Pan, S\. Liu, C\. Lin, J\. Zeng, X\. Tang, X\. Zhou, Y\. Lu, and X\. JiaEvolving agents in the dark: retrospective harness optimization via self\-preference\.arXiv preprint arXiv:2606\.05922\.Note:Accepted to Findings of EMNLP 2026Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Panget al\.\(2024\)R\. Y\. Pang, W\. Yuan, K\. Cho, H\. He, S\. Sukhbaatar, and J\. WestonIterative reasoning preference optimization\.InAdvances in Neural Information Processing Systems,Vol\.37\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Parket al\.\(2026\)S\. Park, W\. Kim, R\. Tan, J\. Zhang, W\. Han, P\. Gao, C\. Park, Y\. Yao, R\. Fu, E\. Nallipogu, Q\. Lin, S\. Rajmohan, and D\. ZhangAutoSaddler: automatic harness optimization with durable updates from agent execution traces\.arXiv preprint arXiv:2608\.23041\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Peiet al\.\(2026\)R\. Pei, Z\. Wei, S\. Xu, X\. Zhu, W\. Chen, and Y\. MengNegative self\-distillation: learning to reason by avoiding flaws\.arXiv preprint arXiv:2609\.11699\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Penget al\.\(2026\)B\. Peng, W\. Yao, Q\. Wu, H\. Cheng, X\. Yu, R\. Yang, T\. Ge, A\. Sordoni, X\. Yuan, Y\. Shen, P\. He, T\. Zhang, Z\. Yu, and J\. GaoOrchard: an open\-source agentic modeling framework\.arXiv preprint arXiv:2605\.15040\.Cited by:[Appendix A](https://arxiv.org/html/2610.22791#A1.SS0.SSS0.Px4.p1.1),[Figure 1](https://arxiv.org/html/2610.22791#S1.F1),[§2\.3](https://arxiv.org/html/2610.22791#S2.SS3.p1.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2610.22791#S3.T1.4.1.8.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Pryzantet al\.\(2023\)R\. Pryzant, D\. Iter, J\. Li, Y\. Lee, C\. Zhu, and M\. ZengAutomatic prompt optimization with “gradient descent” and beam search\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 7957–7968\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.5\-35B\-A3B model card\.External Links:[Link](https://huggingface.co/Qwen/Qwen3.5-35B-A3B)Cited by:[Appendix A](https://arxiv.org/html/2610.22791#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p4.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px1.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu,et al\.DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[Appendix A](https://arxiv.org/html/2610.22791#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.22791#S2.SS3.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.arXiv preprint arXiv:2303\.11366\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Singhet al\.\(2024\)A\. Singh, J\. D\. Co\-Reyes, R\. Agarwal,et al\.Beyond human data: scaling self\-training for problem\-solving with language models\.Transactions on Machine Learning Research\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Soyluet al\.\(2024\)D\. Soylu, C\. Potts, and O\. KhattabFine\-tuning and prompt optimization: two great steps that work better together\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Sunet al\.\(2026\)Q\. Sun, S\. Nielsen, R\. Yokota, and Y\. TangEvolutionary context search for automated skill acquisition\.arXiv preprint arXiv:2602\.16113\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- SWE\-bench Team \(2026\)SWE\-bench TeamSWE\-bench leaderboards\.Note:Accessed: October 14, 2026External Links:[Link](https://www.swebench.com/)Cited by:[Appendix B](https://arxiv.org/html/2610.22791#A2.SS0.SSS0.Px5.p1.1),[Table 4](https://arxiv.org/html/2610.22791#A2.T4)\.
- Tanet al\.\(2026\)Z\. Tan, L\. Sun, Z\. Shi, Y\. Liu, J\. Wu, N\. Brooks, J\. Qian, X\. Shang, S\. Huang, Y\. Ding,et al\.MetaRSI / RSI2: a meta\-recursive self\-improving system for recursive self\-improving systems themselves\.arXiv preprint arXiv:2609\.06396\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Tiwariet al\.\(2026\)R\. Tiwari, K\. Sareen, L\. A\. Agrawal, J\. E\. Gonzalez, M\. Zaharia, K\. Keutzer, I\. S\. Dhillon, R\. Agarwal, and D\. KhatriLearning, fast and slow: towards LLMs that adapt continually\.arXiv preprint arXiv:2605\.12484\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Ursekaret al\.\(2026\)V\. Ursekar, A\. Shanker, Y\. Maurya, S\. Yasser, V\. S\. Kalmath, V\. Chatrath, and Y\. XueHarnessOpt\-Bench: evaluating LLMs at harness optimization\.arXiv preprint arXiv:2608\.06301\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2023\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2025a\)Y\. Wang, L\. Yang, G\. Li, M\. Wang, and B\. AragamScoreFlow: mastering LLM agent workflows via score\-based preference optimization\.arXiv preprint arXiv:2502\.04306\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2025b\)Y\. Wang, S\. Su, Z\. Zeng, E\. Xu, L\. Ren, X\. Yang, Z\. Huang, X\. He, L\. Ma, B\. Peng, H\. Cheng, P\. He, W\. Chen, S\. Wang, S\. S\. Du, and Y\. ShenThetaEvolve: test\-time learning on open problems\.arXiv preprint arXiv:2511\.23473\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025c\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.InForty\-second International Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Weiet al\.\(2026a\)H\. Wei, X\. Tu, Y\. Wang, Z\. Liu, R\. Xue, J\. Ren, X\. Zhang, D\. Zhao, and X\. FanHarnessBandit: joint learnability\-transferability scheduling for multi\-harness agentic reinforcement learning\.arXiv preprint arXiv:2609\.13739\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Weiet al\.\(2026b\)T\. Wei, Z\. Shi, M\. Lin, B\. He, Z\. Liu, Y\. Sang, Y\. Bei, X\. Ning, J\. Zou, T\. Li,et al\.Evo\-Harness: context\-to\-harness skill compilation for self\-evolving agents\.arXiv preprint arXiv:2608\.15071\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Weiet al\.\(2025a\)Y\. Wei, O\. Duchenne, J\. Copet, Q\. Carbonneaux, L\. Zhang, D\. Fried, G\. Synnaeve, R\. Singh, and S\. WangSWE\-RL: advancing LLM reasoning via reinforcement learning on open software evolution\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p2.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Weiet al\.\(2025b\)Y\. Wei, Z\. Sun, E\. McMilin, J\. Gehring, D\. Zhang, G\. Synnaeve, D\. Fried, L\. Zhang, and S\. WangToward training superintelligent software agents through self\-play SWE\-RL\.arXiv preprint arXiv:2512\.18552\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Weiet al\.\(2025c\)Z\. Wei, W\. Yao, Y\. Liu, W\. Zhang, Q\. Lu, L\. Qiu, C\. Yu, P\. Xu, C\. Zhang, B\. Yin, H\. Yun, and L\. LiWebAgent\-R1: training web agents via end\-to\-end multi\-turn reinforcement learning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 7909–7928\.Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p1.1)\.
- Wenget al\.\(2026\)Z\. Weng, A\. Antoniades, D\. Nathani, Z\. Zhang, X\. Pu, and X\. E\. WangGroup\-evolving agents: open\-ended self\-improvement via experience sharing\.arXiv preprint arXiv:2602\.04837\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Wuet al\.\(2026\)S\. Wu, J\. Ren, Y\. Li, H\. Li, C\. Yang, X\. Ma, Y\. Zhang, W\. Gu, J\. Yang, R\. Batista\-Navarro, C\. Zhang, X\. Liu, M\. Zhou, B\. Dai, and C\. LinModularRSI: modular and generalizable recursive harness self\-improvement\.arXiv preprint arXiv:2609\.14857\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Xiaet al\.\(2026a\)P\. Xia, J\. Chen, H\. Wang, J\. Liu, K\. Zeng, Y\. Wang, S\. Han, Y\. Zhou, X\. Zhao, H\. Chen, Z\. Zheng, C\. Xie, and H\. YaoSkillRL: evolving agents via recursive skill\-augmented reinforcement learning\.arXiv preprint arXiv:2602\.08234\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Xiaet al\.\(2026b\)P\. Xia, R\. Han, Z\. Wang, Y\. Chen, Y\. Zhuang, Y\. Lee, C\. Huang, H\. Yu, Z\. CuiZhu, Y\. Ming, H\. Yao, B\. Gokturk, T\. Pfister, and C\. LeeRRSI: regularized recursive self\-improvement of agent harnesses\.arXiv preprint arXiv:2609\.24972\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Xieet al\.\(2024\)T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, T\. J\. Hua, Z\. Cheng, D\. Shin, F\. Lei, Y\. Liu, Y\. Xu, S\. Zhou, S\. Savarese, C\. Xiong, V\. Zhong, and T\. YuOSWorld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p1.1)\.
- Xuet al\.\(2026a\)C\. Xu, S\. Zhang, W\. Zhou, J\. Lambert, B\. Chen, B\. Li, A\. Hard, and R\. MathewsEvo2\{\}^\{2\}: evolving the evolver through weight\-harness co\-evolution\.arXiv preprint arXiv:2610\.13025\.Cited by:[Appendix C](https://arxiv.org/html/2610.22791#A3.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Xuet al\.\(2026b\)R\. Xu, Y\. Chen, Z\. CuiZhu, B\. D\. Mishra, Y\. Ming, H\. Yu, R\. Han, C\. Lee, and T\. PfisterHarness2\{\}^\{2\}: recursive agent harnessing for an open world\.arXiv preprint arXiv:2610\.14224\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2024\)J\. Yang, C\. E\. Jimenez, A\. Wettig, K\. Lieret, S\. Yao, K\. R\. Narasimhan, and O\. PressSWE\-agent: agent\-computer interfaces enable automated software engineering\.InAdvances in Neural Information Processing Systems,Cited by:[Appendix A](https://arxiv.org/html/2610.22791#A1.SS0.SSS0.Px1.p1.1),[Appendix D](https://arxiv.org/html/2610.22791#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p4.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2025\)J\. Yang, K\. Lieret, C\. E\. Jimenez, A\. Wettig, K\. Khandpur, Y\. Zhang, B\. Hui, O\. Press, L\. Schmidt, and D\. YangSWE\-smith: scaling data for software engineering agents\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p4.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,Cited by:[Appendix D](https://arxiv.org/html/2610.22791#A4.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p1.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Yeet al\.\(2026a\)H\. Ye, X\. He, V\. Arak, H\. Dong, and G\. SongMeta context engineering via agentic skill evolution\.arXiv preprint arXiv:2601\.21557\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Yeet al\.\(2026b\)H\. Ye, Y\. Lu, H\. Dong, Z\. Su, and G\. SongHarness\-Zero: harness distillation via agent\-as\-harness\.arXiv preprint arXiv:2609\.24974\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Yuanet al\.\(2024\)W\. Yuan, R\. Y\. Pang, K\. Cho, X\. Li, S\. Sukhbaatar, J\. Xu, and J\. WestonSelf\-rewarding language models\.InInternational Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Yuanet al\.\(2023\)Z\. Yuan, H\. Yuan, C\. Li, G\. Dong, K\. Lu, C\. Tan, C\. Zhou, and J\. ZhouScaling relationship on learning mathematical reasoning with large language models\.arXiv preprint arXiv:2308\.01825\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Yuksekgonulet al\.\(2025\)M\. Yuksekgonul, F\. Bianchi, J\. Boen, S\. Liu, P\. Lu, Z\. Huang, C\. Guestrin, and J\. ZouOptimizing generative AI by backpropagating language model feedback\.Nature639,pp\. 609–616\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Yuksekgonulet al\.\(2026\)M\. Yuksekgonul, D\. Koceja, X\. Li, F\. Bianchi, J\. McCaleb, X\. Wang, J\. Kautz, Y\. Choi, J\. Zou, C\. Guestrin, and Y\. SunLearning to discover at test time\.arXiv preprint arXiv:2601\.16175\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Zelikmanet al\.\(2024\)E\. Zelikman, E\. Lorch, L\. Mackey, and A\. T\. KalaiSelf\-taught optimizer \(STOP\): recursively self\-improving code generation\.InConference on Language Modeling,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Zelikmanet al\.\(2022\)E\. Zelikman, Y\. Wu, J\. Mu, and N\. D\. GoodmanSTaR: bootstrapping reasoning with reasoning\.arXiv preprint arXiv:2203\.14465\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2026a\)H\. Zhang, S\. Zhang, K\. Li, C\. Zhang, Y\. Chen, Y\. Zhang, L\. Bai, and S\. HuSelf\-Harness: harnesses that improve themselves\.arXiv preprint arXiv:2606\.09498\.Cited by:[Appendix C](https://arxiv.org/html/2610.22791#A3.p1.1),[Figure 1](https://arxiv.org/html/2610.22791#S1.F1),[§1](https://arxiv.org/html/2610.22791#S1.p1.1),[§1](https://arxiv.org/html/2610.22791#S1.p2.1),[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2610.22791#S3.T1.4.1.9.1),[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026b\)J\. Zhang, S\. Hu, C\. Lu, R\. T\. Lange, and J\. CluneDarwin Gödel Machine: open\-ended evolution of self\-improving agents\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026c\)J\. Zhang, B\. Zhao, W\. Yang, J\. Foerster, J\. Clune, M\. Jiang, S\. Devlin, and T\. ShavrinaHyperagents\.arXiv preprint arXiv:2603\.19461\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2025\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang, B\. Zheng, B\. Liu, Y\. Luo, and C\. WuAFlow: automating agentic workflow generation\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026d\)L\. Zhang, R\. Zhou, D\. Song, Z\. Chen, Y\. Tian, J\. Yang, H\. Ma, C\. Li, G\. Feng, X\. Li, Y\. Jin, and Y\. XuHarnessCompass: guiding automatic harness evolution toward generalizable and effective agent harnesses\.arXiv preprint arXiv:2608\.01918\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026e\)L\. Zhang, R\. Chen, and B\. C\. StadieEvolutionary system prompt learning for reinforcement learning in LLMs\.arXiv preprint arXiv:2602\.14697\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2026f\)Q\. Zhang, C\. Hu, S\. Upasani, B\. Ma, F\. Hong, V\. Kamanuru, J\. Rainton, C\. Wu, M\. Ji, H\. Li, U\. Thakker, J\. Zou, and K\. OlukotunAgentic context engineering: evolving contexts for self\-improving language models\.InInternational Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: LLM agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2025\)A\. Zhao, Y\. Wu, Y\. Yue, T\. Wu, Q\. Xu, Y\. Yue, M\. Lin, S\. Wang, Q\. Wu, Z\. Zheng, and G\. HuangAbsolute zero: reinforced self\-play reasoning with zero data\.arXiv preprint arXiv:2505\.03335\.Cited by:[§4](https://arxiv.org/html/2610.22791#S4.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2026\)J\. Zhao, G\. Chen, F\. Meng, M\. Li, J\. Chen, H\. Xu, Y\. Sun, W\. X\. Zhao, R\. Song, Y\. Zhang, P\. Wang, C\. Chen, J\. Wen, and K\. JiaImmersion in the GitHub universe: scaling coding agents to mastery\.arXiv preprint arXiv:2602\.09892\.Cited by:[§3\.1](https://arxiv.org/html/2610.22791#S3.SS1.SSS0.Px1.p1.1)\.
- Zhenget al\.\(2024\)L\. Zheng, L\. Yin, Z\. Xie, C\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez, C\. Barrett, and Y\. ShengSGLang: efficient execution of structured language model programs\.InAdvances in Neural Information Processing Systems,Cited by:[Appendix A](https://arxiv.org/html/2610.22791#A1.SS0.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2024\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. NeubigWebArena: a realistic web environment for building autonomous agents\.InThe Twelfth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.22791#S1.p1.1)\.

## Appendix AImplementation Details

#### Agent implementation\.

We use Qwen3\.5\-35B\-A3B\([Qwen Team, 2026](https://arxiv.org/html/2610.22791#bib.bib21)\)as the backbone and SGLang\([Zheng et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib22)\)for model serving\. RL and SFT use the slime framework on one node with eight B200 GPUs\. The seed harness follows the mini\-SWE\-agent interface\([Yang et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib15)\), through which the model executes bash commands in a task\-specific repository environment\.

#### Optimization data\.

The initial search set𝒮0\\mathcal\{S\}\_\{0\}consists of 300 tasks randomly sampled from the training data𝒟\\mathcal\{D\}\. For subsequent H\-steps, the search set consists of hard tasks from𝒟\\mathcal\{D\}, identified using the verified trajectories collected during the preceding P\-step: a task is hard if it is never resolved in any of its rollouts in that P\-step\. These task IDs are reused to generate fresh failure evidence with the updated policy at the next H\-step\.

#### Harness optimization\.

Self\-HPO uses the current policy checkpoint as both task solver and harness proposer\. The proposer examines the experience of evaluated harnesses and their trajectories, including failed trajectories, and revises the current best harness\. HPO instead uses Opus 4\.8 to propose revisions, while retaining the same alternating H\-step–P\-step structure\. In both configurations, each H\-step evaluatesK=6K=6candidates, each run once per search task, and the selected harness is held fixed during the next policy update\. The Self\-Harness and Meta\-Harness baselines use the same search budget\.

#### Policy optimization\.

Each P\-step initializes from the policy produced by the preceding round and collects on\-policy trajectories under the updated harness\. The executable verifier supplies outcome rewards, and the policy is optimized with GRPO and Balanced Adaptive Rollout \(BAR\)\([Shao et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib31);[Peng et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib17)\)\. Let𝒜⁡\(τi\)\\mathcal\{A\}\(\\tau\_\{i\}\)denote the assistant\-generated token positions in trajectoryτi\\tau\_\{i\},A^i\\widehat\{A\}\_\{i\}its group\-relative advantage, andρi,u​\(θ\)\\rho\_\{i,u\}\(\\theta\)the ratio of updated\-policy to rollout\-policy token probabilities\. The clipped objective is

ℒP​\(θ\)=−𝔼i​\[1\|𝒜⁡\(τi\)\|​∑u∈𝒜⁡\(τi\)min⁡\(ρi,u​\(θ\)​A^i,clip⁡\(ρi,u​\(θ\),1−εlow,1\+εhigh\)​A^i\)\],\\mathcal\{L\}\_\{P\}\(\\theta\)=\-\\mathbb\{E\}\_\{i\}\\left\[\\frac\{1\}\{\|\\mathcal\{A\}\(\\tau\_\{i\}\)\|\}\\sum\_\{u\\in\\mathcal\{A\}\(\\tau\_\{i\}\)\}\\min\\left\(\\rho\_\{i,u\}\(\\theta\)\\widehat\{A\}\_\{i\},\\operatorname\{clip\}\(\\rho\_\{i,u\}\(\\theta\),1\-\\varepsilon\_\{\\mathrm\{low\}\},1\+\\varepsilon\_\{\\mathrm\{high\}\}\)\\widehat\{A\}\_\{i\}\\right\)\\right\],\(2\)whereεlow=0\.2\\varepsilon\_\{\\mathrm\{low\}\}=0\.2andεhigh=0\.28\\varepsilon\_\{\\mathrm\{high\}\}=0\.28are the clipping thresholds\. BAR generates eight rollouts per task and selects four to form a reward\-balanced group with a success fraction between 0\.25 and 0\.75\. Sampling continues until 32 groups pass filtering, yielding 128 training trajectories per batch\. Each batch is optimized for one epoch with a global batch size of 128, corresponding to one optimizer update\. We use Adam with a constant learning rate of10−610^\{\-6\}, weight decay 0\.01, an entropy coefficient of10−410^\{\-4\}, and no KL penalty\. Rollouts use temperature 0\.95 and top\-pp0\.95\. RLVR uses the same configuration under the seed harness, with 50 updates per P\-step\.

## Appendix BAdditional Evaluation Details

#### Evaluation configurations\.

All evaluations use a step limit of 250 and temperature and top\-ppof 0\.95\. Task attempts that fail because of sandbox or API errors are rerun, and resolve rates are computed over all tasks in each benchmark\.

#### SFT configuration\.

The SFT baseline in Table[1](https://arxiv.org/html/2610.22791#S3.T1)fine\-tunes the base policy on Opus 4\.8 trajectories collected on the RL training tasks, keeping only resolved trajectories\. It trains for two epochs with batch size 64 and a constant learning rate of5×10−65\\times 10^\{\-6\}after a 10% warmup\. In Table[2](https://arxiv.org/html/2610.22791#S3.T2), SFT instead replaces RL in the P\-step of HPO\.

#### Search\-frontier curves\.

![Refer to caption](https://arxiv.org/html/2610.22791v1/search_frontier.png)Figure 6:Best\-so\-far performance during harness evolution on SWE\-Bench Verified\.Circles show individual candidate scores\. Candidates 1–6 are shared, while 7–12 belong to the separate Self\-Harness and Self\-HPO branches\. Vertical increases and stars mark Self\-HPO’s two P\-steps\. Dashed horizontal lines show Base and final RLVR performance\.Figure[6](https://arxiv.org/html/2610.22791#A2.F6)provides a candidate\-level view of harness search, complementing the selected H\-step and P\-step endpoints in Figure[3](https://arxiv.org/html/2610.22791#S3.F3)\. After the shared first H\-step reaches64\.3%64\.3\\%, continued harness\-only search improves the best score to65\.2%65\.2\\%\. Self\-HPO instead updates the policy to67\.8%67\.8\\%before resuming search\. Its second H\-step then reaches68\.6%68\.6\\%, followed by69\.5%69\.5\\%after the second policy update\. The separation between the two branches shows the benefit of policy updates alongside continued harness search\.

#### Pass@kkcalculation\.

Figure[4](https://arxiv.org/html/2610.22791#S3.F4)compares Base and the final Self\-HPO pair with eight attempts per task\. Following[Chen and others \(2021\)](https://arxiv.org/html/2610.22791#bib.bib28), for a task withccsuccesses amongn=8n=8attempts, we compute

pass​@k=1−\(n−ck\)\(nk\),k=1,…,8,\\operatorname\{pass@\}k=1\-\\frac\{\\binom\{n\-c\}\{k\}\}\{\\binom\{n\}\{k\}\},\\qquad k=1,\\ldots,8,and average over tasks\. Pass@1 averages the success frequency, whereas pass@8 records whether any of the eight attempts succeeds\. The shaded regions show 95% bootstrap confidence intervals\.

#### Comparison with frontier models\.

Table[4](https://arxiv.org/html/2610.22791#A2.T4)compares our method with frontier models on the SWE\-Bench Verified leaderboard\([SWE\-bench Team, 2026](https://arxiv.org/html/2610.22791#bib.bib92)\)\.

Table 4:Comparison with frontier models on SWE\-Bench Verified\.Resolve rates \(%\) of frontier models on the SWE\-Bench Verified leaderboard\([SWE\-bench Team, 2026](https://arxiv.org/html/2610.22791#bib.bib92)\)and of Qwen3\.5\-35B\-A3B with methods from Table[1](https://arxiv.org/html/2610.22791#S3.T1)\. Bold marks Self\-HPO\. Parameter counts are those stated in the developers’ model cards \(“–” if not stated\)\.

## Appendix CConceptual Comparison with Related Methods

Table[5](https://arxiv.org/html/2610.22791#A3.T5)compares Self\-HPO with Meta\-Harness\([Lee et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib16)\), Self\-Harness\([Zhang et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib18)\), Task\-CoEvolve\([Miyai et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib67)\), Evo2\([Xu et al\., 2026a](https://arxiv.org/html/2610.22791#bib.bib4)\), WHALE\([Kim et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib40)\), and HASE\([Luo et al\., 2026](https://arxiv.org/html/2610.22791#bib.bib35)\)in terms of coverage of the coding domain, how the policy learns, who proposes harness changes, and how harness\-search tasks are selected\.

Table 5:Conceptual comparison ofSelf\-HPOand related methods\.Coding indicates whether the method is evaluated on coding tasks\. Policy update indicates whether the task\-solving policy is frozen or updated through training such as supervised fine\-tuning \(SFT\) or reinforcement learning \(RL\)\. Harness proposer is the model that modifies the harness, which can be either frozen or updated across rounds\. Harness search set denotes the task set used to collect diagnostic trajectories for harness optimization\. In Self\-HPO, the evolving policy itself proposes harness revisions, and hard tasks identified during training form the next harness search set, keeping harness optimization targeted to the policy’s evolving failure modes\.#### The trained solver becomes the next harness proposer\.

Self\-Harness revises its harness with a fixed policy, WHALE uses a frozen external proposer alongside policy learning, and Evo2trains a separate proposer while the task\-solving policy stays frozen\. Self\-HPO instead reuses the updated task\-solving policy as the proposer in the next round: the H\-step holds policy weights fixed, and the P\-step performs RL under the selected harness\. HASE also trains a self\-proposing policy, but places task solving and harness editing in a joint RL action space, with persistent edits reviewed at phase boundaries\.

#### Policy training also determines the next harness\-search tasks\.

The P\-step supplies two inputs to the next H\-step: the updated proposerPt\+1P\_\{t\+1\}and a search set𝒮t\+1\\mathcal\{S\}\_\{t\+1\}drawn from hard tasks encountered during RL\. Thus, policy learning changes both*who proposes*and*which tasks guide the search*\. Task\-CoEvolve also adapts task selection, but uses outcome variance across candidate harnesses within search rather than difficulty exposed during policy training\. Self\-HPO updates task IDs between H\-steps and keeps them fixed within each H\-step\.

## Appendix DHarness Details and Prompt Templates

#### Seed harness\.

The seed harnessH0H\_\{0\}is the mini\-SWE\-agent ReAct loop\([Yao et al\., 2023](https://arxiv.org/html/2610.22791#bib.bib23);[Yang et al\., 2024](https://arxiv.org/html/2610.22791#bib.bib15)\): the agent issues bash tool calls in a task\-specific sandbox and ends the episode by printing the patch after a fixed submission command\. Its task prompt contains the issue description followed by fixed instructions, including the exact command used to submit the patch\.

#### Harness search space\.

A candidate harness is a Python class that subclasses the seed agent, together with its prompt templates\. It may override the agent’s step, for example to inspect the agent’s state, return feedback before accepting a submission, replace the submitted patch, or intervene at any step\. All repository access goes through the sandbox\. The policy and sampling configuration \(temperature and top\-ppof 0\.95\) are fixed across candidates\.

#### Harness proposer\.

Figure[7](https://arxiv.org/html/2610.22791#A4.F7)shows the system prompt of the harness proposer\. The proposer runs as a coding agent with shell access to the parent harness, its trajectories on the search set, reference fixes for search\-set tasks, the outputs of earlier candidates, and notes that the proposer appends after each search round\.

Harness Proposer System PromptYou are the harness optimizer for a fixed policy \{P\_t\}\. The search set contains \{N\}hard training tasks that the policy never solved during the preceding P\-step\. Theparent harness is \{H\_best\}, the current best harness\. The starting harness of this searchsolves \{s\}% of the search set under the frozen sampling configuration\. Your goal is to design ONE new harness mechanism, of your own design,that is added on top of the parent harness and solves more search\-set tasks than theparent\.CONSTRAINTS\- Keep every method of the parent harness verbatim and add exactly one new mechanism\.\- The policy, sampling configuration \(temperature 0\.95, top\-p 0\.95\), step limit,maximum tokens, and verifier are frozen\.\- Never inject reference patches or task\-specific facts into the agent’s context; themechanism must compute its own signal at runtime\.EXPERIENCE \(read it before designing; treat it as evidence, not a mandate\)\- Candidates evaluated so far in this search: scores, code, and trajectories\.\- Notes left by earlier proposals in this search\.Diagnostics computed from the starting harness’s runs on the search set:\- Coverage gap: the source files required by the reference fix versus the files theagent touched\.\- Precision gap: failures in which every required file was edited but the fix wasincorrect\.\- Submission loss: failures in which a real source edit appears in the trajectory butthe final submitted patch is empty or malformed\.\- Reference\-fix archetypes \(multi\-file, signature change, new import, guard, errorhandling, \.\.\.\) versus what the agent produced\.Decide for yourself which failure is the largest and most reachable by a harness, orfind another one\.WORKFLOW0\. Review the experience: the earlier notes and every candidate evaluated so far\.For the most recent one, read the tasks it made worse and better than the startingharness \(trajectories and reference fixes\), and work out why\.1\. Read several failing trajectories and compare a few with the reference fix to formyour own hypothesis about the dominant failure cause\.2\. Design a mechanism with a detectable trigger in the agent’s own state and aconcrete action: for example, intercept a submission and return feedback so thatthe agent gets another turn to act on it, replace the submitted patch, or act atany step\.3\. Implement it in a copy of the parent harness\. All repository access must gothrough the sandbox environment\.4\. Write a one\-line purpose \(diagnosed cause \-\> mechanism\), append what you learnedthis round to the notes, and stop\. The search then checks that the candidate keepsevery parent method and fires on real sandbox tasks, and returns any failure to youto fix\.

Figure 7:Harness proposer system prompt\(condensed\)\. Values in braces change between search iterations and rounds\. The parent is the current best harness, and in the first round the search set is the random𝒮0\\mathcal\{S\}\_\{0\}and the parent starts from the seed harness\. Implementation\-specific instructions, such as file paths, the sandbox interface, and validation scripts, are omitted\.
#### Optimized harness\.

Figure[5](https://arxiv.org/html/2610.22791#S3.F5)shows a condensed view of the harness selected in HPO’s second H\-step\. Figure[8](https://arxiv.org/html/2610.22791#A4.F8)shows the same harness as a diff against the seed harness\. The optimized harness keeps the seed prompts unchanged and inherits the seed agent loop\. Its only change is an override ofstep\(\)that adds four checks\. Three of these checks add a message to the conversation, listed in full in Figure[9](https://arxiv.org/html/2610.22791#A4.F9)\.

`\\iow\_now:Ne˝\\iow\_now:Ne˝— seed harness H0: ProgressTrackingAgent\.step\(\) \(inherited agent loop\)\\iow\_now:Ne˝\+\+\+ optimized harness: AgentHarness\(ProgressTrackingAgent\)\.step\(\)\\iow\_now:Ne˝@@ class AgentHarness\(ProgressTrackingAgent\) @@\\iow\_now:Ne˝ def step\(self\):\\iow\_now:Ne˝\+ \# \(1\) budget valve: once, shortly before the step or time limit, if nothing was submitted\\iow\_now:Ne˝\+ if not self\.\_rescued and not self\.\_submitted\_once and \(near\_step\_limit\(\) or near\_time\_limit\(\)\):\\iow\_now:Ne˝\+ self\.\_rescued = True\\iow\_now:Ne˝\+ self\.add\_messages\(user\(RESCUE\_TEXT\)\) \# the step then continues as usual\\iow\_now:Ne˝\- return self\.execute\_actions\(self\.query\(\)\)\\iow\_now:Ne˝\+ try:\\iow\_now:Ne˝\+ return self\.execute\_actions\(self\.query\(\)\) \# unchanged seed step\\iow\_now:Ne˝\+ except Submitted as e: \# the policy issued the submit command\\iow\_now:Ne˝\+ self\.\_submitted\_once = True\\iow\_now:Ne˝\+ sub = e\.submission\\iow\_now:Ne˝\+ \# \(2\) empty\-patch guard: hold an empty submission once\\iow\_now:Ne˝\+ if is\_empty\(sub\) and not self\.\_empty\_guarded and has\_runway\(\):\\iow\_now:Ne˝\+ self\.\_empty\_guarded = True\\iow\_now:Ne˝\+ self\.add\_messages\(user\(EMPTY\_GUARD\_TEXT\)\)\\iow\_now:Ne˝\+ return \[\] \# not submitted; the policy continues\\iow\_now:Ne˝\+ \# \(3\) contract grounding: hold the first submission that passes the empty guard once\\iow\_now:Ne˝\+ if not self\.\_grounded and has\_runway\(\):\\iow\_now:Ne˝\+ self\.\_grounded = True\\iow\_now:Ne˝\+ self\.add\_messages\(user\(GROUND\_TEXT\)\)\\iow\_now:Ne˝\+ return \[\]\\iow\_now:Ne˝\+ \# \(4\) patch fallback on the final submission\\iow\_now:Ne˝\+ if not is\_empty\(sub\) and not applies\_cleanly\(sub\): \# ‘git apply –check‘ on the base commit\\iow\_now:Ne˝\+ auth = sandbox\("git add \-N <untracked files\> && git diff"\)\\iow\_now:Ne˝\+ if auth: e\.submission = auth \# submit the sandbox’s own diff instead\\iow\_now:Ne˝\+ raise \# accept the submission\\iow\_now:Ne˝ Optimized harness \(diff against the seed\)`

`Figure 8: Harness changes from HPO’s second H\-step\. Shown as a diff against the seed agent’s step\(\) \(\+ added, −\- removed\)\. Helper calls are condensed into short names\. The message texts are listed in Figure 9\.``\(1\) Budget valve: RESCUE\_TEXT \[BUDGET ALMOST EXHAUSTED \-\- STOP EXPLORING AND SUBMIT NOW\] You are nearly out of steps/time and have not submitted a patch\. An empty result scores zero, but a focused best\-effort edit can pass\. Right now: 1\. Decide the single most likely fix location from what you already know \-\- do NOT start new open\-ended exploration\. 2\. Make that concrete edit to the source file \(the smallest change that matches the PR’s literal request\)\. 3\. Immediately create and submit the patch, in SEPARATE commands: git diff \-\- <the source files you modified\> \> patch\.txt echo COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT && cat patch\.txt Do this in your next one or two commands\. \(2\) Empty\-patch guard: EMPTY\_GUARD\_TEXT \[YOUR SUBMISSION IS EMPTY \-\- it contains no source\-code changes, so it cannot fix anything\. Not finalized\.\] You have not edited any source file yet \(patch\.txt is empty or has no diff hunks\)\. Do NOT give up\. Re\-localize the bug: 1\. Re\-read the PR and grep for the exact symbol/message/flag it names\. 2\. Open the file where that symbol lives and make the concrete edit the PR requires\. 3\. IMPORTANT \-\- did your fix CREATE a NEW file \(a brand\-new module\)? A plain \`git diff\` does NOT show untracked new files, so your patch looks empty even though the file exists\. Stage new files first so they appear as additions: cd /testbed && git add \-N <the new file\(s\) you created\> 4\. Verify with \`cd /testbed && git \-\-no\-pager diff\` that a real change \(or a \`new file mode\` hunk\) now shows\. 5\. Then \`cd /testbed && git diff \-\- <files\> \> patch\.txt\` and submit with: echo COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT && cat patch\.txt \(3\) Contract grounding: GROUND\_TEXT \[GROUND YOUR DIFF IN THE REPO’S EXISTING TESTS \-\- one\-time check; your submission is NOT finalized yet\] The most common way this task is failed is submitting a patch that APPLIES cleanly but implements the WRONG contract \-\- a guessed function/option name, a signature the codebase never uses, a rename or new spelling that no caller expects, or an error/return shape different from what the code around it uses\. Re\-reading the PR will NOT catch this, because your reading of the PR is what produced the wrong contract\. The codebase itself is the authority\. The repo ships EXISTING tests that already exercise the code you touched and the API next to it \-\- they show the EXACT names, signatures, argument spellings, and error/return types the code is expected to have\. Use them as ground truth now\. 1\. Print your changes and identify the concrete symbols you added or changed: git \-\-no\-pager diff List every NEW or RENAMED symbol/option/message string in your diff\. 2\. For each such symbol, search the repo’s tests and existing call sites for how that symbol \-\- and the nearest EXISTING sibling API \-\- is actually used: grep \-rn \-\-include=’\*\.py’ \-e ’<symbol\>’ \-e ’<sibling\_api\>’ \. \| head \-40 Then OPEN and READ the specific test files that reference the area you changed \(e\.g\. tests/ or test\_\*\.py near your file\) and note the exact spelling, call signature, and expected result/exception the codebase uses\. 3\. Confront your diff against what the tests/call sites actually expect\. If you introduced a name, option spelling, signature, or error shape that NO existing test or caller uses \-\- while an existing convention does \-\- change your diff to MATCH the codebase’s convention\. Do not invent a new API; conform to the one already there\. Also confirm you did not MISS a behavior the tests imply is required\. 4\. If you changed anything, regenerate the patch: git diff \-\- <the source files you modified\> \> patch\.txt and re\-inspect patch\.txt so it contains ONLY your intended source changes\. 5\. When your diff conforms to the codebase’s real contract, re\-issue the EXACT submit command: echo COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT && cat patch\.txt This check happens only once \-\- after you re\-submit, it goes through\. Figure 9: Messages added by the optimized harness\. The messages that the harness in Figure 8 adds to the conversation, shown in full\. Each is added at most once per task and is followed by another policy turn\. The numbers match the checks in Figures 5 and 8\. Appendix E Case Study: Where the Harness Intervenes To illustrate how an evolved harness changes the outcome for a fixed policy, we compare two runs of the final Self\-HPO policy P2P\_\{2\} on the SWE\-Bench Verified task sphinx\-doc\_\_sphinx\-9658: one under the seed harness H0H\_\{0\} and one under the evolved harness H2H\_\{2\}\. Here H2H\_\{2\} is the harness from Self\-HPO’s second H\-step, which differs from the HPO harness in Appendix D\. We show the final steps of each run, labeling the task, the agent’s reasoning, its actions, the resulting observations, and any message injected by the harness\. “\[…\]\[\\dots\]” marks omitted content\. Purple notes summarize the key steps, so the example can be followed by reading them alone\. The two runs reach the same point: the policy has edited the source, the existing tests pass, and it prepares to finish with a summary of its changes\. Under H0H\_\{0\} \(Figure 10\), this summary becomes the final submission, so no code change is evaluated\. Under H2H\_\{2\} \(Figure 11\), the harness holds the first submission, reports that the submitted patch contains no files, and returns control to the policy, which writes the diff, checks the fix, and submits a valid patch\. The PR\-intent reminder in this example is a false positive, since its keywords come from unrelated parts of the issue text\. Case 1 \(Failure Case\): Policy P2P\_\{2\} with the seed harness H0H\_\{0\} Task <pr\_description\> Consider the following PR description: Inherited classes not correctly documented when mocked \#\#\# Describe the bug We’re experiencing an issue when documenting classes that inherit mocked classes\. \[\.\.\.\] The base class is listed as "Bases: \`torch\.nn\.\`" instead of "Bases: \`torch\.nn\.Module\`"\. \[\.\.\.\] </pr\_description\> \[task instructions of the seed harness\] \[131 earlier steps omitted\] Agent Good, I only modified the source files, not test files\. Let me now run a final comprehensive test to make sure the fix works correctly: Action cd /testbed && python \-m pytest tests/test\_util\_typing\.py tests/test\_ext\_autodoc\.py \\ tests/test\_ext\_autodoc\_automodule\.py tests/test\_ext\_inheritance\_diagram\.py \-x \-\-tb=short 2\>&1 \| tail \-20 Observation \[\.\.\.\] ================== 99 passed, 7 skipped, 7 warnings in 2\.00s =================== ▶\\blacktriangleright \(1\) The agent’s edits are in place and the existing tests pass\. Agent All tests pass\. The fix is complete\. Let me summarize what was done: \#\# Summary \[\.\.\.\] Action cd /testbed && echo COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT && cat << ’EOF’ \#\# Summary of Changes \#\#\# Issue When documenting classes that inherit from mocked classes, the "Bases" section incorrectly showed \`torch\.nn\.\` instead of \`torch\.nn\.Module\`\. \[\.\.\.\] All existing tests pass with these changes\. EOF ▶\\blacktriangleright \(2\) To finish, the agent prints a prose summary of its changes instead of the patch file\. Final submission \#\# Summary of Changes \#\#\# Issue \[\.\.\.\] All existing tests pass with these changes\. ▶\\blacktriangleright \(3\) The seed harness accepts this output as the submission\. No code change reaches the grader, so the task is unresolved\. Figure 10: Failure case: policy P2P\_\{2\} with the seed harness H0H\_\{0\}\. The run ends with a prose summary as the submission, so sphinx\-doc\_\_sphinx\-9658 is unresolved\. Case 2 \(Success Case\): Policy P2P\_\{2\} with the evolved harness H2H\_\{2\} Task \[\.\.\. same task as above \.\.\.\] \[114 earlier steps omitted\] Agent The fix looks correct\. Let me run a broader test to make sure I haven’t broken anything: Action cd /testbed && python3 \-m pytest tests/test\_ext\_autodoc\*\.py tests/test\_util\_typing\.py \\ tests/test\_ext\_inheritance\_diagram\.py \-x 2\>&1 \| tail \-20 Observation \[\.\.\.\] ================== 195 passed, 7 skipped, 7 warnings in 3\.53s ================== ▶\\blacktriangleright \(1\) Same situation as under H0H\_\{0\}: the edits are in place and the tests pass\. Agent All 195 tests pass \(7 skipped\)\. The fix is complete and working\. Let me write a summary of the fix: \#\# Summary \[\.\.\.\] Action cd /testbed && echo COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT && cat sphinx/util/typing\.py \| head \-130 \\ \| tail \-20 && echo "\-\-\-" && cat sphinx/util/typing\.py \| tail \-10 ▶\\blacktriangleright \(2\) The agent again tries to finish without submitting a patch\. Harness \[Full Files Auditor \- Coverage Check\] \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- You touched 1 files during exploration: \[’sphinx/util/typing\.py’\] Your submitted patch has 0 files: \(none\) FOUND GAP: 1 touched files NOT in your patch: Potential missing files: \- sphinx/util/typing\.py \[PR Intent Reminder\] \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- Your patch may not address the core PR intent\. Key terms from PR: \- sphinx\.ext\.viewcode \- sphinx\_autodoc\_typehints \- version \- sphinxcontrib\.apidoc \- sphinx\.ext\.napoleon Your submitted patch does not mention these key terms\. Consider: Does your edit actually address the issue described? ▶\\blacktriangleright \(3\) H2H\_\{2\} holds this submission once and reports that the submitted patch contains no files\. Agent The task mentioned a patch file was created\. Let me verify my changes are correct and submit them properly\. Action cd /testbed && git diff \> patch\.txt && cat patch\.txt Observation diff \-\-git a/sphinx/util/typing\.py b/sphinx/util/typing\.py index 87707d48f\.\.142503422 100644 \-\-\- a/sphinx/util/typing\.py \+\+\+ b/sphinx/util/typing\.py @@ \-117,6 \+117,12 @@ def restify\(cls: Optional\[Type\]\) \-\> str: return cls elif cls in INVALID\_BUILTIN\_CLASSES: return ’:py:class:\`%s\`’ % INVALID\_BUILTIN\_CLASSES\[cls\] \+ elif \_get\_ismock\(\)\(cls\): \+ \# This is a mock object\. Use its type’s attributes\. \+ if sys\.version\_info \>= \(3, 7\): \+ return \_restify\_py37\(type\(cls\)\) \+ else: \+ return \_restify\_py36\(type\(cls\)\) elif inspect\.isNewType\(cls\): \[\.\.\.\] ▶\\blacktriangleright \(4\) Prompted by the feedback, the agent writes its working\-tree changes into a patch\. Agent The patch shows my changes correctly\. Let me verify the fix works correctly and then submit it\. Action cd /testbed && python3 \-c " from sphinx\.ext\.autodoc\.mock import \_MockModule from sphinx\.util\.typing import restify mock\_mod = \_MockModule\(’torch\.nn’\) class DeepKernel\(mock\_mod\.Module\): pass result = restify\(DeepKernel\.\_\_orig\_bases\_\_\[0\]\) print\(’Result:’, result\) \[\.\.\.\]" Observation Result: :py:class:\`torch\.nn\.Module\` Expected: :py:class:\`torch\.nn\.Module\` Fix works: True ▶\\blacktriangleright \(5\) It checks the fix on the example from the issue\. Agent The fix is working correctly\. Now let me submit the final output\. Action echo COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT && cat /testbed/patch\.txt Final submission diff \-\-git a/sphinx/util/typing\.py b/sphinx/util/typing\.py \[\.\.\.\] ▶\\blacktriangleright \(6\) The final submission is a valid patch, which resolves the task\. Figure 11: Success case: policy P2P\_\{2\} with the evolved harness H2H\_\{2\}\. The harness holds the first submission, and the policy then submits a valid patch that resolves sphinx\-doc\_\_sphinx\-9658\.`