Great paper on self-evolving agent harnesses.

Self-evolving agent harnesses have two practical problems:

1. Search is slow, because every candidate harness needs repeated agent runs and code edits.

2. Fixes overfit, because each failure is patched as if it were a harness bug, even when the model caused it.

Ecdysis analyzes failures across a batch of tasks and repairs only patterns that recur.

Several diagnostic roles agree on a change specification before any code is modified.

Harness training runs up to 1.84x faster than existing harness evolution methods, and the resulting harnesses gain 18.56% in reasoning accuracy. They also transfer better across LLMs, use fewer tokens, and match full-data training with a quarter of the data.

Paper: https://arxiv.org/abs/2609.11677

Chat with Paper: https://academy.dair.ai/papers/ecdysis-efficient-and-effective-training-of-runtime-harnesses-for-llm-agents-2609.11677…

Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents

Source: https://arxiv.org/html/2609.11677 Ruiqing Yue††thanks:Yu Cui proposed the algorithm, and Ruiqing Yue validated its effectiveness through experiments.Affiliation:Chengdu Institute of Computer Applications, Chinese Academy of SciencesAffiliation:University of Chinese Academy of SciencesYu Cui11footnotemark:1Zhuoyu SunAffiliation:Beijing Institute of TechnologySicheng PanAffiliation:Beijing Institute of TechnologyXianhong XueAffiliation:Chengdu Institute of Computer Applications, Chinese Academy of SciencesAffiliation:University of Chinese Academy of SciencesTingyu LiAffiliation:Beijing Institute of TechnologyTing LiAffiliation:Beijing University of TechnologyWenzhuo ZhuAffiliation:Beijing University of TechnologyYi ChenAffiliation:Chengdu Institute of Computer Applications, Chinese Academy of SciencesAffiliation:University of Chinese Academy of SciencesYifei LiuAffiliation:Beijing Institute of TechnologyBaohan HuangAffiliation:Beijing Institute of TechnologyZhe CuiAffiliation:Chengdu Institute of Computer Applications, Chinese Academy of SciencesAffiliation:University of Chinese Academy of SciencesHaibin ZhangAffiliation:Yangtze Delta Region Institute of Tsinghua University, ZhejiangAffiliation:Jiaxing Key Laboratory of Artificial Intelligence and Cyber Resiliencehttps://github.com/cuiyu-ai/EcdysisProject Lead: Yu Cui <[email protected]>Cong ZuoAffiliation:Beijing Institute of Technology

Abstract

Self-evolving runtime harnesses can substantially improve the capabilities of large language model (LLM) agents and provide a promising paradigm for optimizing agent execution. Existing harness evolution methods typically rely on iterative search, repeatedly evaluating and revising candidate harnesses based on execution feedback from task instances. While this paradigm enables continuous harness optimization, it incurs substantial time overhead due to repeated agent executions and code modifications, and may overfit to observed tasks and specific failure patterns, resulting in degraded generalization to unseen tasks. We identify the lack of principled failure diagnosis as a key bottleneck in harness evolution: an observed failure can reflect either model-specific deficiencies or systematic harness deficiencies, and directly optimizing against individual failures can lead to unnecessary model-specific accommodation. We therefore proposeEcdysis, an efficient and effective framework that distinguishes model-specific accommodation from harness-level repair and biases adaptation toward systematic harness deficiencies by identifying recurring cross-task failure patterns.Ecdysisadopts a batch-level cross-instance failure aggregation paradigm to jointly analyze failure evidence from multiple task instances and further introduces Failure-Driven Collaborative Refinement (FDCR) to diagnose failure causes and iteratively refine harness modification specifications. By combining cross-instance failure analysis with multi-role diagnosis,Ecdysisenables more effective harness evolution with lower end-to-end training time. Experiments across multiple LLMs and benchmarks show thatEcdysisachieves up to a 1.84×\timesspeedup in harness training compared with existing harness evolution methods, while improving the reasoning accuracy of the resulting harnesses by 18.56%. Moreover, harnesses trained withEcdysisexhibit stronger generalization across LLMs and reduce inference-time token consumption.

1Introduction

Large language models (LLMs) have demonstrated increasingly strong capabilities in complex reasoning. However, the capabilities of the underlying model alone are often insufficient for practical LLM agents[1]. Recent studies have shown that runtime harnesses can substantially enhance agent capabilities through inference-time mechanisms such as task planning, tool interaction, and context management[2,3,4]. The widespread adoption of coding agents, such as Codex[5], further demonstrates that, even with the underlying LLM fixed, the runtime harness plays a crucial role in determining agent performance. Unlike manually designed harnesses, recent harness self-evolution methods[6,7,8,9]enable harnesses to automatically modify their own code based on execution feedback. By iteratively identifying failures and revising the harness, this paradigm provides a promising approach to automatically optimizing runtime harnesses for LLM agents.

Existing harness evolution methods typically rely on iterative search, repeatedly evaluating and revising candidate harnesses based on execution feedback collected from task instances. While this paradigm enables continuous optimization of runtime harnesses using task feedback, it faces substantial efficiency and generalization challenges. Each harness update typically requires additional agent execution, code modification, and result verification, resulting in substantial training-time overhead. Moreover, as evolution proceeds, the harness may become increasingly specialized to observed tasks and specific failure patterns, leading to undesirable deviations in harness evolution[10,11]and degraded generalization to unseen tasks[12]. These limitations suggest that effective harness evolution requires not only collecting execution feedback, but also reliably interpreting such feedback and translating it into appropriate harness modifications.

We identify reliable failure attribution as a key challenge in harness self-evolution. An observed failure may arise from the current model’s behavior or from a systematic deficiency in the harness. This distinction is critical because harness evolution can either performmodel-specific accommodation, adapting to idiosyncrasies of the current model, orharness-level repair, addressing systematic deficiencies in the interaction mechanism. Individual failures provide limited evidence for distinguishing these cases, and simply collecting more failures does not resolve the ambiguity. Without reliable attribution, evolution may overfit to observed failure patterns and degrade generalization to unseen tasks. We therefore seek to identify failure mechanisms that recur across task instances as stronger evidence for systematic harness repair.

This motivates the following research question:Can failure evidence across task instances distinguish model-specific accommodation from systematic harness repair and guide more generalizable harness evolution?To address this issue, we proposeEcdysis, an efficient framework for evolving runtime harnesses for LLM agents.Ecdysisadopts a batch-level cross-instance failure aggregation paradigm to jointly analyze failures from multiple task instances. Rather than treating failures as independent modification signals, it identifies recurring failure patterns across tasks and prioritizes those providing stronger evidence for systematic harness deficiencies.

To further improve the reliability of failure diagnosis,Ecdysisintroduces Failure-Driven Collaborative Refinement (FDCR)[13,14,15]. Different roles examine the collected failure evidence and iteratively refine potential failure causes and candidate harness modifications from complementary perspectives. A moderator then synthesizes these analyses into a structured harness modification specification, reducing ambiguity in failure interpretation and improving the reliability of the resulting modifications. By combining batch-level cross-instance failure analysis with multi-role diagnosis,Ecdysisextracts more informative signals from each training round, prioritizing systematic harness repair over unnecessary model-specific accommodation and thereby enabling more effective harness evolution while reducing end-to-end training time.

We systematically evaluateEcdysisacross multiple LLMs and benchmarks. Experimental results show thatEcdysisachieves up to a 1.84×\timesspeedup in harness training compared with existing harness evolution methods while improving the reasoning accuracy of the resulting harnesses by 18.56%. Further experiments demonstrate that harnesses trained withEcdysisexhibit stronger cross-LLM generalization, while also reducing inference-time token consumption. These results demonstrate that reliable failure diagnosis can improve both the efficiency and generalizability of runtime harness evolution. Our contributions are summarized as follows:

• •EcdysisTraining Framework. We proposeEcdysis, a runtime harness training framework that analyzes recurring failure patterns across task instances to distinguish systematic harness deficiencies from model-specific behavior, and introduces Failure-Driven Collaborative Refinement (FDCR) to generate structured harness modification specifications.
• •Efficient and Generalizable Harness Evolution. We systematically evaluateEcdysisacross multiple LLMs and benchmarks. Results demonstrate substantial improvements in harness training efficiency and agent reasoning performance, together with stronger cross-model generalization and reduced token consumption.
• •Insights for Training Data Curation. We propose a failure-aware training data selection strategy that considers interaction structures and failure mechanisms across task instances, prioritizing tasks that expose novel execution paths and harness deficiencies while reducing redundant failure signals. By aggregating failures across instances,Ecdysisidentifies shared deficiencies and enables more efficient harness evolution with fewer, more diagnostic training tasks. Our experiments show thatEcdysisachieves performance comparable to full-data training while using only one-quarter of the training data, substantially reducing the overhead of harness evolution.
• •An Empirical Analysis of Model Accommodation in Harness Evolution. Through extensive fine-grained manual analysis of the training process, we identify and quantify the tendency of failure-driven evolution to over-accommodate model-specific limitations using the model-accommodation ratio. Based on this analysis, we formulate a theoretical perspective in which excessive model accommodation can induce harness evolution drift and undermine cross-model generalization, providing an explanation for the improved generalization achieved byEcdysis.

2Related Work

Inference-time mechanisms such as tool interaction, deliberate planning, and feedback-driven adaptation have substantially enhanced LLM agents[3,16]. Building on these advances, automatic optimization has progressed from prompt optimization[17,18]to agentic workflows and self-modifying agent programs[19,20,21]. More recently, several works have focused specifically on evolving runtime harnesses, using execution feedback and iterative evaluation to propose and validate harness modifications[22,6,23,24,25,26]. Although harnesses are ultimately optimized to improve LLM task performance, harness evolution should not simply adapt to every observed failure, as failures may reflect model limitations rather than harness deficiencies, leading to model-specific accommodation.Ecdysistherefore treats recurrent failure patterns across independent task instances as stronger evidence of systematic harness deficiencies, guiding evolution toward reusable behavioral improvements rather than instance-specific adaptation.

3Ecdysis

3.1Preliminary Analysis

The runtime harness can substantially influence task performance. Across six Qwen3 model-domain combinations, removing all harness layers yields an average accuracy of 29.72%, whereas a fixed, human-configured harness achieves 50.28%. Serially modifying the harness with a coding agent after each failure achieves 43.33%, which remains below the 50.28% achieved by the fixed human configuration. These results indicate that the runtime harness is a meaningful optimization target, while also suggesting that effective automatic harness evolution remains challenging. A more fundamental challenge lies in failure attribution: an observed failure may arise from the current model’s behavior or from a systematic harness deficiency. The former leads tomodel-specific accommodation, whereas the latter calls forharness-level repair. Individual failures provide limited evidence for distinguishing these cases, while treating failures from a common harness issue as independent modification signals can also incur redundant modification and validation costs.

This distinction motivates a conceptual decomposition of harness adaptation. Consider the aggregate harness updateΔ​H\Delta Hproduced over an evolution round, composed of a set of independent modification decisions issued by the coding agent. Each decision either accommodates a limitation of the current task model or repairs a systematic harness deficiency, contributing toΔ​Hmodel\Delta H_{\mathrm{model}}andΔ​Hharness\Delta H_{\mathrm{harness}}, respectively. Lettingt∈[0,1]t\in[0,1]denote the proportion of these decisions that are model-specific accommodation, the aggregate update decomposes as

Δ​H=t​Δ​Hmodel+(1−t)​Δ​Hharness,t∈[0,1],\Delta H=t\,\Delta H_{\mathrm{model}}+(1-t)\,\Delta H_{\mathrm{harness}},\qquad t\in[0,1],so thatttis at once the fraction of model-accommodating decisions and the weight of the model-accommodation component in the overall update. A largerttmay resolve observed failures efficiently but can overfit to the current model and training tasks. We hypothesize that failure patterns recurring across distinct task instances provide stronger evidence for systematic harness deficiencies than isolated failures, and therefore can shift adaptation away from unnecessary model-specific accommodation. Cross-task recurrence is used as an inductive bias rather than as proof of causal attribution.

3.2Methodology

Algorithm 1EcdysisTraining for Harness Self-EvolutionInput:Fixed task model

θ\theta, runtime environment

ℰ\mathcal{E}, training task set

𝒟train\mathcal{D}_{\mathrm{train}}, initial harness

HbaseH_{\mathrm{base}}, failure threshold

λ\lambda, number of evolution rounds

RR, and number of refinement passes

KK Output:Frozen final harness

HFH_{F} 1

H0←HbaseH_{0}\leftarrow H_{\mathrm{base}};

2fori=1i=1toRRdo

3

𝒯i←Collect⁡(θ,Hi−1,ℰ,𝒟train)\mathcal{T}_{i}\leftarrow\operatorname{Collect}(\theta,H_{i-1},\mathcal{E},\mathcal{D}_{\mathrm{train}})# execution trajectories;

4

fλ(τ)←𝕀[S(τ)<λ]f_{\lambda}(\tau)\leftarrow\mathbb{I}\!\left[S(\tau)<\lambda\right];

5

ℬi←Aggregate⁡({τ∈𝒯i:fλ​(τ)=1})\mathcal{B}_{i}\leftarrow\operatorname{Aggregate}\!\left(\{\tau\in\mathcal{T}_{i}:f_{\lambda}(\tau)=1\}\right)# structured failure evidence;

6ifℬi=∅\mathcal{B}_{i}=\varnothingthen

7

Hi←Hi−1H_{i}\leftarrow H_{i-1};

8continue;

9

𝒢i←Group⁡(ℬi)\mathcal{G}_{i}\leftarrow\operatorname{Group}(\mathcal{B}_{i})# failure patterns;

10

𝒜←{Analyst,Critic,Engineer}\mathcal{A}\leftarrow\{\operatorname{Analyst},\operatorname{Critic},\operatorname{Engineer}\};

11

ℳi←∅\mathcal{M}_{i}\leftarrow\varnothing# shared role transcript;

12fork=1k=1toKKdo

13foreach*A∈𝒜A\in\mathcal{A}*do

14

mik,A←A⁡(𝒢i,ℳi)m_{i}^{k,A}\leftarrow A(\mathcal{G}_{i},\mathcal{M}_{i});

15

ℳi←ℳi∘mik,A\mathcal{M}_{i}\leftarrow\mathcal{M}_{i}\mathbin{\circ}m_{i}^{k,A};

16

qi←Moderator⁡(𝒢i,ℳi)q_{i}\leftarrow\operatorname{Moderator}(\mathcal{G}_{i},\mathcal{M}_{i})# structured modification specification;

17

Hic←Edit⁡(Hi−1,qi)H_{i}^{\mathrm{c}}\leftarrow\operatorname{Edit}(H_{i-1},q_{i})# coding agent;

18if*Jtrain​(Hic)>Jtrain​(Hi−1)J_{\mathrm{train}}(H_{i}^{\mathrm{c}})>J_{\mathrm{train}}(H_{i-1})*then

19

Hi←HicH_{i}\leftarrow H_{i}^{\mathrm{c}};

20else

21

Hi←Hi−1H_{i}\leftarrow H_{i-1};

22

HF←HRH_{F}\leftarrow H_{R};

23returnHFH_{F};

Given a task modelθ\theta, a runtime environmentℰ\mathcal{E}, a training task set𝒟train\mathcal{D}_{\mathrm{train}}, and an initial runtime harnessHbaseH_{\mathrm{base}},Ecdysisoptimizes the harness using execution trajectories collected from the training tasks. Throughout training, the task model parameters and runtime environment remain fixed. We initialize the harness asH0=HbaseH_{0}=H_{\mathrm{base}}and denote byHiH_{i}the validated harness retained after evolution roundii.

For each evolution roundi∈{1,2,…,R}i\in\{1,2,\ldots,R\},Ecdysisproceeds in three stages. First, it executes the training tasks with the current harnessHi−1H_{i-1}and aggregates the resulting failure evidence. Second, it transforms the aggregated evidence into a structured modification specification, which is provided to a coding agent to modifyHi−1H_{i-1}and produce a candidate harnessHicH_{i}^{\mathrm{c}}. Third, it validatesHicH_{i}^{\mathrm{c}}on the training set and retains it only if it improves the overall training score; otherwise, the previous harness is preserved. AfterRRrounds, training terminates andEcdysisfreezes the most recently validated harness, denoted byHF=HRH_{F}=H_{R}. The complete procedure is summarized in Algorithm1.

LetJtrain​(H)J_{\mathrm{train}}(H)denote the overall score of harnessHHon the training set under the fixed evaluation framework. In evolution roundii,Ecdysisaccepts the candidate harness if and only if

Jtrain​(Hic)>Jtrain​(Hi−1).J_{\mathrm{train}}\!\left(H_{i}^{\mathrm{c}}\right)>J_{\mathrm{train}}(H_{i-1}). Accordingly,

Hi={Hic,if​Jtrain​(Hic)>Jtrain​(Hi−1),Hi−1,otherwise.H_{i}=\begin{cases}H_{i}^{\mathrm{c}},&\text{if }J_{\mathrm{train}}\!\left(H_{i}^{\mathrm{c}}\right)>J_{\mathrm{train}}(H_{i-1}),\\ H_{i-1},&\text{otherwise}.\end{cases} This criterion constrains only the overall training-set score and does not require every individual training task to improve. We next describe the two core components ofEcdysis: Batch-Level Failure Aggregation and Failure-Driven Collaborative Refinement (FDCR).

3.3Batch-Level Failure Aggregation

At evolution roundii,Ecdysisexecutes the training tasks using the current harnessHi−1H_{i-1}and collects the resulting execution trajectories, denoted by𝒯i\mathcal{T}_{i}. A task specifies the target objective, whereas its execution trajectory records the concrete process leading to the observed outcome. The trajectory therefore provides execution-level evidence for failure analysis beyond the final task-level signal.

For each trajectoryτ∈𝒯i\tau\in\mathcal{T}_{i}, the fixed evaluation framework produces a task scoreS⁡(τ)S(\tau). We define the binary failure signal as

fλ(τ)=𝕀[S(τ)<λ],f_{\lambda}(\tau)=\mathbb{I}\!\left[S(\tau)<\lambda\right], whereλ\lambdais a predefined threshold andfλ​(τ)=1f_{\lambda}(\tau)=1indicates failure. Trajectories satisfying this condition are converted into structured records to form the failure evidence setℬi\mathcal{B}_{i}. Each record retains the task identifier, failure decision, termination reason, tool-call history, and necessary execution context. This procedure uses the fixed evaluation framework as the training signal and does not modify or replace its scoring mechanism.

The failure records are organized into failure groups to facilitate the analysis of recurring patterns.Ecdysisprioritizes groups that cover at least two distinct tasks, since repeated failures from a single task alone do not establish that the underlying failure mechanism generalizes across tasks. Groups containing only one task identifier are nevertheless retained as auxiliary evidence for subsequent analysis. Importantly, failure groups serve as diagnostic evidence rather than mandatory repair targets. They help determine whether an observed pattern indicates a harness deficiency and guide decisions regarding the modification level, trigger conditions, modification scope, and safety constraints. Thus, a candidate harness need not address every observed failure group.

Batch-level aggregation also reduces repeated coding-agent calls for candidate harness modification. Letni=|ℬi|n_{i}=|\mathcal{B}_{i}|denote the number of failure records collected in roundii. Processing each failure independently would requirenin_{i}coding-agent calls, whereasEcdysismakes a single coding-agent call for each nonempty round. Hence,

Nround=∑i=1R𝕀⁡(ni>0)≤∑i=1Rni=Nserial.N_{\mathrm{round}}=\sum_{i=1}^{R}\mathbb{I}(n_{i}>0)\leq\sum_{i=1}^{R}n_{i}=N_{\mathrm{serial}}. The inequality is strict whenever at least one evolution round contains multiple failure records.

3.4Failure-Driven Collaborative Refinement

For each evolution round containing failure evidence,Ecdysisapplies Failure-Driven Collaborative Refinement (FDCR) to transform the aggregated failure patterns into a structured harness modification specification. Inspired by collaborative criticism and iterative refinement in multi-agent systems[13,14], FDCR uses observed failure patterns as the driving evidence for role-based diagnosis and iterative refinement, rather than treating individual failures as independent modification requests.

FDCR separates failure analysis and modification planning from the actual harness implementation. Given the failure groups𝒢i\mathcal{G}_{i}, the Analyst, Critic, and Engineer iteratively refine modification proposals through a shared transcript. The Analyst identifies potential harness deficiencies and proposes minimal, targeted updates. The Critic evaluates these proposals against the observed failure evidence and examines potential risks, including overly broad triggers, unintended blocking of legitimate behavior, violations of the runtime contract, and regressions on previously successful tasks. The Engineer tracks agreements and unresolved disagreements and identifies issues requiring clarification in subsequent refinement. The roles are executed sequentially, with each role receiving the accumulated transcript so that subsequent analysis can build on preceding discussion. The refinement proceeds for a fixed number of rounds.

After the role-based refinement, the Moderator reads the failure evidence and the complete transcript and produces a structured modification specification. The Moderator serves as a conservative arbitration stage that consolidates the refined proposals, resolves remaining disagreements, and prioritizes modifications that address recurring failure patterns while avoiding unnecessary changes. The resulting specification describes the identified failure patterns, proposed harness changes, and relevant implementation guidance, but does not directly modify the harness. Instead, the coding agent uses the specification together with the relevant execution evidence and the source code of the current harness to modifyHi−1H_{i-1}and produce the candidate harnessHicH_{i}^{\mathrm{c}}.

FDCR and the coding agent therefore have distinct responsibilities: FDCR diagnoses cross-instance failure patterns and refines the corresponding modification specification, whereas the coding agent performs the actual harness modification. This separation keeps failure analysis structured and failure-driven while delegating implementation to the coding agent. By prioritizing failure patterns recurring across distinct task instances, FDCR biases the modification process toward systematic harness-level repairs and away from unnecessary model-specific accommodation.

3.5Evaluation

We evaluateEcdysisfrom three perspectives: inference performance, training efficiency, and cost.

• •Inference Performance. On the held-out split, we reportaverage task accuracy,Pass@3, andPass^3. Pass@3 denotes the proportion of tasks solved in at least one of the three trials, whereas Pass^3 denotes the proportion solved in all three trials.
• •Training Efficiency.Training Timemeasures the end-to-end wall-clock time from the initial training evaluation until the final harness is frozen. For held-out evaluation, we reportFinal Evaluation Tokens, defined as the total input and output tokens consumed by the task model.
• •Cost. We reportInput Cache Hit RateandAPI Cost. Input Cache Hit Rate is the ratio of cached input tokens to total input tokens. API Cost is reported separately for training task evaluation and harness modification.

4Experiments

4.1Experimental Setup

Models. Our evaluation follows prior work[27]and considers the compatibility between LLM reasoning capabilities and benchmark difficulty. We evaluate five task LLMs: Qwen3-8B, Qwen3-14B, Qwen3-32B[28], MiniMax-M2.7 (230B)111https://huggingface.co/MiniMaxAI/MiniMax-M2.7, and Llama-3.1-8B222https://huggingface.co/meta-llama/Llama-3.1-8B. For reproducibility, all LLMs are accessed through APIs. We use the same inference configuration for all task models in both training and held-out evaluation. For evaluation parameters, we refer to the baseline methods. The sampling temperature is set to 0.0. We use OpenCode333https://opencode.aiwith DeepSeek-V4-Pro as its underlying model as the coding agent. InEcdysis(w/ FDCR), the Analyst, Critic, Engineer, and Moderator also use DeepSeek-V4-Pro[29].

Datasets. To ensure comprehensive coverage of diverse task types, we use AgentBench[30]for relatively simple tasks andτ2\tau^{2}-Bench for more complex and challenging tasks. Following prior work, we use the Airline and Retail subsets ofτ2\tau^{2}-Bench, which we refer to asτ2\tau^{2}-Airline andτ2\tau^{2}-Retail, respectively. Both subsets require agents to perform tool interactions that continuously modify the environment state[31]. For each subset, we select 20 tasks from the training split and 20 tasks from the test split for all models and methods. For held-out evaluation, we evaluate each test task in three independent trials, resetting the environment before each trial. Thus, each combination of model, subset, and method contains 60 held-out trajectories.

Baselines and Ablation Study. For the runtime harness, we adoptLife-Harness, a mature and well-structured harness framework[27]. To enable fair and controlled comparisons, we construct five runtime harness configurations, covering a direct baseline, a human-optimized baseline, and three harness evolution strategies.

• •Direct: retains the base agent loop, message handling, and tool interface required byτ2\tau^{2}-Bench, while disabling the harness layers adopted from Life-Harness. This configuration serves as the minimal baseline without any additional harness mechanisms.
• •Human-Augmented Harness (Human-Aug.): uses a fixed harness without harness evolution. This harness is obtained through human-involved optimization in prior work and serves as a strong baseline representing the performance of a manually optimized runtime harness. Moreover, the human-optimized harness is used as the common base harness for harness post-training. That is, all three evolution-based methods described below are initialized from the same Human-Aug. harness, thereby keeping the initial harness configuration fixed across methods.
• •Self-Evolution (SE): performs instance-level serial updates. Each failure record independently invokes the coding agent, and the modifications produced for individual failure instances are applied sequentially and accumulated into a single round-level candidate. The accumulated candidate is then evaluated in the next complete training evaluation[27].
• •Ecdysis(w/o FDCR): performs mixed training by aggregating failure evidence from multiple training tasks and evolution rounds. For each evolution round with non-empty failure evidence, it constructs a single round-level modification plan from the aggregated evidence and invokes the coding agent once to implement the proposed changes.
• •Ecdysis(w/ FDCR): operates on the same mixed training input asEcdysis(w/o FDCR). The Analyst, Critic, and Engineer agents jointly analyze the aggregated failure evidence through two rounds of FDCR. The Moderator then synthesizes the results and formulates the final modification plan, which is subsequently implemented by the coding agent.

Across all three evolution-based methods, only the runtime harness is updated during training, while the model parameters remain fixed. Thus, the comparison isolates the effect of different harness optimization strategies while controlling for the initial harness configuration and the underlying model.

Evaluation Protocol. The three methods share the same training-time evaluation and candidate acceptance protocol, differing only in how they organize failure evidence and construct candidate modifications. We use Qwen3-8B as the task model and allow at most three candidate-generation rounds. Starting from the same initial harness, each method generates candidates from the resulting evidence, which are evaluated only in the subsequent complete evaluation and never on the trajectories used for their generation. A candidate is retained only if it strictly improves over the previously retained harness; otherwise, it is discarded. Accepted candidates are used for subsequent generation. After the final candidate is generated, a final evaluation over all training tasks is performed solely to determine whether it is accepted, with no further candidate generation. All evolution decisions are based exclusively on the training split. After evolution, the resulting harnesses are frozen and evaluated on held-out data under two independent protocols: first, Qwen3-8B is evaluated on all held-out tasks in each subset with three trials per task; second, we evaluate the full matrix of five task models, two subsets, and five methods, directly reusing the harnesses evolved with Qwen3-8B for the three evolution-based methods without further evolution.

5Results

5.1Overall Results

Across the ten combinations of five task models and twoτ2\tau^{2}-Bench subsets,Ecdysis(w/ FDCR) achieves the highest average accuracy (Table1). Its relative improvements over SE and Human-Aug. are 27.1% and 14.8%, respectively. It also improves Pass^3 by 55.2% relative to SE, showing more consistent success across repeated trials.

Table 1:Overall task performance and inference efficiency across five LLMs and two datasets. Inference efficiency is measured by token consumption (M) and execution time (s).Beyond task accuracy,Ecdysisalso improves both training and inference efficiency relative to SE. We report the training results in Section5.3and the inference results in Section6.2, and analyze the sources of these gains in Section7.

5.2Ablation Results

All three task-performance metrics improve from SE toEcdysis(w/o FDCR). The accuracy increases from 46.67% to 54.67%, an improvement of 8.00%. Meanwhile, Pass@3 and Pass^3 reach 66.50% and 42.00%, improving by 3.00% and 13.00%, respectively. Adding FDCR further increases the average task success rate to 59.33%, Pass@3 to 71.50%, and Pass^3 to 45.00%, corresponding to additional gains of approximately 4.67%, 5.00%, and 3.00% overEcdysis(w/o FDCR). These results show that round-level aggregation of failure evidence across task instances provides performance gains on its own, with FDCR further improving the aggregate results.

5.3Training Efficiency

The training-efficiency results show that round-level failure aggregation reduces repeated modification calls during harness evolution.Ecdysis(w/o FDCR) completes end-to-end training in 1,292.4 seconds onτ2\tau^{2}-Retail and 2,510.8 seconds onτ2\tau^{2}-Airline, corresponding to speedups of 1.42×\timesand 3.23×\timesover SE.Ecdysis(w/ FDCR) takes 1,405.9 and 4,403.0 seconds, corresponding to speedups of 1.30×\timesand 1.84×\times(Table6). API cost follows the same pattern. Onτ2\tau^{2}-Retail andτ2\tau^{2}-Airline, SE costs $8.484 and $6.382, whereasEcdysis(w/o FDCR) reduces these costs to $2.485 and $2.136, andEcdysis(w/ FDCR) costs $5.763 and $2.609, respectively (see Table4). Overall,Ecdysisdecouples efficient failure-driven evolution from costly collaborative refinement. Failure aggregation provides the primary efficiency gains, while FDCR serves as an accuracy-oriented refinement module that trades additional evolution-time cost for improved harness quality.

6Complete Evaluation Results

6.1Task Performance

Averaged over the five task models and the three datasets (τ2\tau^{2}-Airline,τ2\tau^{2}-Retail, and AgentBench), the average accuracy increases from 58.67% under SE to 69.56% withEcdysis(w/ FDCR), a relative gain of 18.56%. Results onτ2\tau^{2}-Airline andτ2\tau^{2}-Retail are detailed in Table2, and those on AgentBench are reported in Table7. The improvement is especially pronounced for Qwen3-8B on theτ2\tau^{2}-Airline subset, where accuracy rises from 35.00% to 60.00%, Pass@3 from 50.00% to 80.00%, and Pass^3 from 20.00% to 40.00%. Notably, this improvement extends to models not used during evolution. On the same subset, Qwen3-32B improves from 51.67% under SE to 68.33% withEcdysis. These results demonstrate that a harness evolved with Qwen3-8B transfers to other task models without further evolution.

Table 2:Held-out task performance across five LLMs and two datasets.

6.2Inference Efficiency

We report token consumption and mean per-trajectory runtime for the complete held-out matrix in Table3. Final evaluation tokens include only the input and output tokens consumed by the task model and user simulator during the formal held-out trajectories. Runtime is measured per trajectory rather than as the wall-clock duration of the concurrent evaluation.

Table 3:Inference cost results across LLMs, datasets, and methods. Inference efficiency is measured by token consumption (M) and execution time (s).ModelDatasetMethodEfficiencyTokens (M)Time (s)Qwen3-8Bτ2\tau^{2}-AirlineDirect9.544±\pm0.35267.55±\pm3.07Human-Aug.13.961±\pm0.666108.58±\pm15.42Self-Evolution12.321±\pm1.205175.23±\pm27.82Ecdysis(w/o FDCR)12.516±\pm0.428102.43±\pm1.59Ecdysis(w/ FDCR)12.070±\pm1.45898.87±\pm17.85τ2\tau^{2}-RetailDirect8.470±\pm0.416108.92±\pm15.41Human-Aug.9.536±\pm0.182113.88±\pm2.25Self-Evolution9.483±\pm0.495171.32±\pm18.96Ecdysis(w/o FDCR)8.205±\pm0.118180.13±\pm20.16Ecdysis(w/ FDCR)9.790±\pm0.326142.30±\pm20.28Qwen3-14Bτ2\tau^{2}-AirlineDirect13.685±\pm0.858103.27±\pm1.99Human-Aug.15.718±\pm1.539101.83±\pm19.15Self-Evolution24.308±\pm0.420197.61±\pm11.94Ecdysis(w/o FDCR)16.089±\pm1.954157.02±\pm31.85Ecdysis(w/ FDCR)16.417±\pm1.46257.16±\pm11.15τ2\tau^{2}-RetailDirect19.239±\pm2.460134.06±\pm20.80Human-Aug.20.099±\pm3.226158.45±\pm42.92Self-Evolution14.392±\pm3.268107.25±\pm31.44Ecdysis(w/o FDCR)18.424±\pm2.998128.51±\pm49.22Ecdysis(w/ FDCR)12.746±\pm1.28290.82±\pm18.23Qwen3-32Bτ2\tau^{2}-AirlineDirect6.469±\pm0.34975.51±\pm9.04Human-Aug.11.047±\pm0.287105.05±\pm8.83Self-Evolution13.206±\pm0.483117.99±\pm3.33Ecdysis(w/o FDCR)9.748±\pm0.57891.34±\pm9.55Ecdysis(w/ FDCR)11.670±\pm1.009100.36±\pm13.24τ2\tau^{2}-RetailDirect7.182±\pm0.06677.40±\pm4.93Human-Aug.12.297±\pm1.81388.55±\pm11.70Self-Evolution9.023±\pm0.35379.43±\pm14.13Ecdysis(w/o FDCR)7.895±\pm0.22073.45±\pm1.11Ecdysis(w/ FDCR)9.015±\pm0.299102.02±\pm5.28MiniMax-M2.7τ2\tau^{2}-AirlineDirect5.252±\pm0.171135.52±\pm12.59Human-Aug.6.554±\pm0.074154.62±\pm3.26Self-Evolution8.161±\pm0.327170.69±\pm15.05Ecdysis(w/o FDCR)6.127±\pm0.083139.25±\pm3.80Ecdysis(w/ FDCR)6.526±\pm0.218202.03±\pm9.94τ2\tau^{2}-RetailDirect5.173±\pm0.048115.32±\pm4.57Human-Aug.6.054±\pm0.077270.38±\pm40.65Self-Evolution5.728±\pm0.052164.34±\pm3.70Ecdysis(w/o FDCR)5.779±\pm0.119335.37±\pm43.90Ecdysis(w/ FDCR)5.804±\pm0.051201.45±\pm4.04Llama-3.1-8Bτ2\tau^{2}-AirlineDirect5.818±\pm0.36032.23±\pm5.01Human-Aug.9.640±\pm0.14451.68±\pm3.22Self-Evolution10.319±\pm0.29943.38±\pm1.74Ecdysis(w/o FDCR)10.321±\pm0.43451.97±\pm4.52Ecdysis(w/ FDCR)9.015±\pm0.178159.60±\pm15.38τ2\tau^{2}-RetailDirect6.181±\pm0.26629.31±\pm4.14Human-Aug.8.586±\pm0.45032.32±\pm5.33Self-Evolution8.724±\pm0.27633.53±\pm5.73Ecdysis(w/o FDCR)8.438±\pm0.25454.72±\pm1.40Ecdysis(w/ FDCR)8.517±\pm0.22832.30±\pm4.29

Averaged over the ten model and subset combinations, final evaluation tokens are 11.57M for SE, 10.35M forEcdysis(w/o FDCR), and 10.16M forEcdysis(w/ FDCR), corresponding to relative reductions of 10.48% and 12.19%. This indicates that the efficiency acquired during evolution persists on held-out trajectories without any test-time modification. Runtime follows the same trend, withEcdysis(w/ FDCR) reducing the mean from 126.08 to 118.69 seconds per trajectory. The largest gain is on Qwen3-14B overτ2\tau^{2}-Airline, where the mean runtime drops from 197.61 to 57.16 seconds, corresponding to a 3.46×\timesspeedup. Because runtime also reflects model service latency and trajectory length, we report the full means and standard deviations in Table3rather than only the aggregate.

7Analysis of Evolution Process

To further explain the training efficiency results in Section5.3, this section analyzes the actual execution of different harness evolution methods from four aspects: candidate generation, API cost, token usage, and end-to-end training time. All statistics are collected from the complete evolution runs described in Section4.1. The three methods use the same training evaluation, candidate validation, rollback, and early stopping protocols. We report the model calls and resource usage observed during actual execution. Therefore, these results reflect the cost of different candidate generation mechanisms along their actual execution paths rather than theoretical budgets normalized by a fixed number of evolution rounds.

7.1Candidate Generation

Different evolution methods exhibit different candidate generation costs. Compared with SE,Ecdysisrequires fewer coding-agent calls, mainly because the two methods use different update granularities. SE generates and accumulates modifications for individual failures, whileEcdysisaggregates the failure evidence within each evolution round and generates candidate modifications from the aggregated evidence. As the number of failures within a round increases, the candidate generation cost of SE grows accordingly, whileEcdysiscontinues to generate candidates at the round level. The runtime logs further reveal differences between the two methods during actual candidate generation. Forτ2\tau^{2}-Retail, approximately 67% of the coding-agent calls from SE completed normally, compared with approximately 97% inτ2\tau^{2}-Airline. In contrast, all candidate generation calls fromEcdysiscompleted normally in both domains. These results show that accumulating modifications for individual failures not only increases candidate generation cost but is also associated with a higher proportion of incomplete calls in some runs. By aggregating failure evidence,Ecdysisreduces repeated candidate generation and exhibits more stable execution behavior in our experiments.

7.2API Cost and Token Usage

Table 4:Comparison of API costs betweenEcdysisand baseline methods during harness training.We divide the API cost of the complete training process into training evaluation and evolution (Table4). Training evaluation covers model calls made during training task execution. Evolution covers the cost of candidate analysis and implementation after failure evidence is generated, including calls to the coding agent, FDCR when enabled, and other model calls during the evolution process. Each cost is calculated from the input tokens, output tokens, and API prices applicable at execution time, as recorded in the call logs. The training evaluation cost remains below $1 for all three methods, with most cost differences arising during evolution. SE incurs total costs of $8.484 and $6.382, respectively.Ecdysis(w/o FDCR) reduces these costs to $2.485 and $2.136, corresponding to reductions of 70.71% and 66.53%. The total costs ofEcdysis(w/ FDCR) are $5.763 and $2.609, representing reductions of 32.07% and 59.12% relative to SE. Overall, bothEcdysisconfigurations incur lower total API costs than SE. This result is consistent with the reduction in coding-agent calls discussed above and suggests that generating candidates from failure evidence aggregated over an entire round can effectively reduce the cost of repeated implementation. SE achieves cache hit rates of 89.35% and 89.47% onτ2\tau^{2}-Retail andτ2\tau^{2}-Airline, respectively (see Table5). In comparison, the cache hit rates ofEcdysis(w/o FDCR) increase to 96.01% and 95.29%, whileEcdysis(w/ FDCR) reaches 95.42% and 93.28%. Notably, onτ2\tau^{2}-Retail,Ecdysis(w/ FDCR) incurs substantially lower total API cost than SE despite using more input tokens. These results further demonstrate the cost efficiency ofEcdysis.

Table 5:Input-cache utilization during harness evolution.

7.3End-to-End Evolution Time

We report the end-to-end training time from the initial training evaluation to the point when the final harness passes validation and is frozen in Table6. Forτ2\tau^{2}-Retail,Ecdysis(w/o FDCR) andEcdysis(w/ FDCR) achieve end-to-end training speedups of 1.42×\timesand 1.30×\timesover SE, respectively. Forτ2\tau^{2}-Airline, the corresponding speedups reach 3.23×\timesand 1.84×\times. These results indicate that generating candidates from failure evidence aggregated over an entire round reduces redundant work during evolution and consistently shortens end-to-end training time across both domains.

Table 6:Training time (s) for the three harness self-evolution methods.

8Task Structure Analysis

Motivation. Recent work on on-policy distillation (OPD) shows that training efficiency depends not only on the amount of training data, but also on the information contained in individual training instances.Hou et al. [32]find that a small set of carefully selected hard examples can nearly match training on a much larger dataset, with the gains attributed primarily to the longer reasoning trajectories induced by challenging problems. This raises a related question for harness evolution:*which training instances provide the most informative signal for harness training?*Unlike model training, where longer reasoning trajectories can be particularly valuable, harness training requires sufficient task complexity and behavioral breadth to exercise diverse interaction structures and expose distinct failure mechanisms. We therefore view training tasks not merely as execution instances, but as diagnostic probes that reveal both the coverage of runtime behaviors and the deficiencies of the harness.

8.1Cross-Task Structural Regularities

Although Retail and Airline tasks differ in entities, tools, and user goals, they exhibit recurring interaction structures. Multi-step and multi-intent tasks require persistent goal tracking; conditional branches require adaptation to intermediate tool results; state constraints impose preconditions on subsequent actions; and cross-tool dependencies require consistent identification and propagation of entities, attributes, and relations. High-risk operations, intent changes, and interruptions further require appropriate confirmation, termination, or plan revision. These recurring structures give rise to common harness failure modes across tasks with different surface semantics. For example, stale information after state updates, ambiguous entity references, and downstream tool calls without required prerequisite information reflect deficiencies in state synchronization, entity resolution, and tool coordination, respectively. Similarly, failures involving state checks, entity propagation, operation preconditions, and tool ordering can recur across training and held-out tasks despite substantial differences in their surface forms.

This observation suggests that effective harness improvements should capture underlying runtime constraints rather than memorize task-specific failures. For example, the harness should re-query state after relevant updates, resolve ambiguous entities before side-effecting operations, and verify required dependency information before invoking downstream tools. Such abstractions allow improvements identified from one task to generalize to structurally related failures in unseen tasks.

8.2Insights for Training Data Curation

The cross-task regularities motivate a failure-aware approach to training data curation. Training tasks differ in the diagnostic evidence they provide: some expose novel interaction structures or previously uncovered harness deficiencies, whereas others produce redundant manifestations of known failure mechanisms. Consequently, selecting training data based solely on task quantity or surface diversity can incur substantial execution cost without providing commensurate information for harness evolution. We therefore propose a failure-aware training data selection strategy that jointly considers interaction structures and failure mechanisms across task instances. Specifically, we prioritize tasks that expose novel execution paths or previously under-covered harness deficiencies, while reducing tasks that provide redundant failure signals. This yields a more informative training set with broader coverage of runtime behaviors and fewer redundant executions. Data curation is closely coupled with failure diagnosis. Failures observed on individual tasks may represent different surface manifestations of the same underlying harness deficiency. By aggregating failure evidence across task instances,Ecdysisidentifies shared deficiencies and generates more generalizable harness modifications, rather than repeatedly addressing isolated failures. Overall, effective harness training should prioritize informative tasks over data quantity. By selecting tasks with novel execution paths and failure mechanisms,Ecdysisenables more efficient harness evolution with fewer, more diagnostic training tasks.

To further validate the potential of training data curation, we conduct an additional experiment using five training failures, corresponding to one-quarter of the original training set. We trainEcdysiswith this reduced training set and compare the resulting performance against that of full-data training. The results are summarized in Table13. For a fair comparison and ablation study, we also evaluate the training results obtained by randomly selecting five training samples. Overall, reduced-data training achieves performance comparable to full-data training while substantially reducing training cost during harness evolution. In contrast, the random selection strategy yielded inferior performance.

8.3Beyond Local Failure-Driven Evolution

To better understand how failure-driven evolution modifies the runtime harness, we conducted a fine-grained manual analysis of the training-time data and quantifiedttintroduced in Section3.1. Here,ttdenotes the proportion of coding-agent-issued modification decisions that accommodate limitations of the task model, among all independent modification decisions. We find that local failure-driven evolution can overfit to task-specific outcomes by promoting a particular training-task answer into a general runtime constraint. In some cases, the evolution agent removes a legitimate action option to avoid a specific model error, replacing a conditional decision that should be made by the model with a global harness-level prohibition. While such modifications may improve the triggering evaluation case, they also shrink the valid action space of the harness and may impair its generalization across models.

Quantitatively, we observet=60.0%t=60.0\%for Self-Evolution (SE), compared witht=45.5%t=45.5\%forEcdysis, corresponding to a reduction of 14.5 percentage points. This result suggests that local failure-driven evolution is more prone to adapting the harness to model-specific deficiencies, whereas the cross-task failure analysis inEcdysisprovides broader evidence before committing to a harness-level modification. Conceptually, the objective ofEcdysiscan be viewed as attenuating the model-accommodation component fromtttot′=β​tt^{\prime}=\beta t, whereβ∈(0,1)\beta\in(0,1), rather than treating every observed failure as direct evidence for modifying the harness. By reducing such model-specific over-adaptation,Ecdysiscan preserve a more general valid action space and reduce the dependence of the resulting harness on the model used during evolution, thereby improving its cross-model generalization.

9Conclusion

In this paper, we presentEcdysis, an efficient framework for self-evolving runtime harnesses for LLM agents. Our central insight is that execution failures are not uniformly actionable: they may reflect model-specific limitations rather than harness deficiencies. Treating individual failures as direct modification signals can therefore over-accommodate the evolving model and compromise harness generalization.Ecdysisaddresses this challenge by shifting failure-driven evolution from individual failures to recurring cross-task failure patterns. By aggregating failure evidence across instances and applying FDCR, it provides stronger evidence for systematic harness deficiencies before committing to harness-level modifications. Across multiple LLMs and reasoning benchmarks,Ecdysisimproves harness training efficiency and agent reasoning performance while achieving stronger cross-model generalization and lower token consumption. Fine-grained analysis further reveals that reducing model-specific accommodation is closely associated with more generalizable harness evolution. Overall, our findings suggest that effective harness evolution requires not only correcting failures, but also determining which failures constitute reliable evidence for changes that generalize beyond the model and tasks used during evolution.

Ethical Considerations

In this paper, AI assistants are used to polish the writing. We also use AI agents to assist with programming. All AI-assisted outputs are reviewed and verified by the authors.

References

• [1]Z. Chen, R. Tang, G. Deng, F. Wu, J. Wu, Z. Jiang, V. Prasanna, A. Cohan, and X. Wang(2025)LocAgent: graph-guided LLM agents for code localization.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.),Vienna, Austria,pp. 8697–8727.External Links:Link,Document,ISBN 979-8-89176-251-0Cited by:§1.
• [2]C. Jiang, J. Zhong, Y. Fu, K. Tian, J. Yang, K. Zhao, Y. Wang, T. Luo, W. Wang, Y. Zuo,et al.(2026)Self-improving agents in the era of experience: a survey of self-to meta-evolution.Cited by:§1.
• [3]S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao(2023)ReAct: synergizing reasoning and acting in language models.InThe Eleventh International Conference on Learning Representations,External Links:LinkCited by:§1,§2.
• [4]G. Wan, M. Ling, X. Ren, R. Han, S. Li, and Z. Zhang(2026)COMPASS: enhancing agent long-horizon reasoning with evolving context.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),M. Liakata, V. P. Moreira, J. Zhang, and D. Jurgens (Eds.),San Diego, California, United States,pp. 3360–3380.External Links:Link,Document,ISBN 979-8-89176-390-6Cited by:§1.
• [5]R. Zhang, W. Dai, H. V. Pham, G. Uddin, J. Yang, and S. Wang(2026)Engineering pitfalls in ai coding tools: an empirical study of bugs in claude code, codex, and gemini cli.InProceedings of the 34th ACM International Conference on the Foundations of Software Engineering,pp. 393–403.External Links:ISBN 9798400726361,LinkCited by:§1.
• [6]H. Lee, J. Xu, J. Seely, D. Lee, M. Zaharia, and Y. Tang(2026)Recursive harness self-improvement.External Links:2607.15524,LinkCited by:§1,§2.
• [7]M. Chen, J. Wang, Z. Liu, Y. Wang, H. Zheng, and Q. Wang(2026)From failed trajectories to reliable llm agents: diagnosing and repairing harness flaws.arXiv preprint arXiv:2606.06324.Cited by:§1.
• [8]W. Wang, P. Piękos, L. Nanbo, F. Laakom, Y. Chen, M. Ostaszewski, M. Zhuge, and J. Schmidhuber(2026)Huxley-g\”odel machine: human-level coding agent development by an approximation of the optimal self-improving machine.InThe Fourteenth International Conference on Learning Representations,External Links:LinkCited by:§1.
• [9]S. Shao, K. Zhang, Q. Li, S. Wang, H. Wang, W. Jiao, Y. Lu, Y. Guo, W. Liu, and W. Zhang(2026)Harness-r1: learning to edit executable runtime harnesses from agent failure trajectories.arXiv preprint arXiv:2608.02276.Cited by:§1.
• [10]S. Shao, Q. Ren, D. Liu, C. Qian, B. Wei, D. Guo, Y. JingYi, X. Song, L. Zhang, W. Zhang, and J. Shao(2026)Your agent may misevolve: emergent risks in self-evolving LLM agents.InThe Fourteenth International Conference on Learning Representations,External Links:LinkCited by:§1.
• [11]W. Jiang, M. Chu, Y. Tian, Q. Zhang, H. Yang, R. Yang, Y. Liu, T. Lv, and F. Li(2026)HarnessEvolve: learning from reference trajectories for reliable agent self-evolution.arXiv preprint arXiv:2609.00829.Cited by:§1.
• [12]Y. Wang, H. Zhu, Z. Hu, Y. Yuan, Z. Chen, S. Senthil, H. Hajishirzi, Y. Tsvetkov, P. Dasigi, and T. Xiao(2026)Rethinking the evaluation of harness evolution for agents.InCOLM 2026 The 2nd Workshop on Lifelong Agents: Learning, Aligning, and Evolving,External Links:LinkCited by:§1.
• [13]P. Yu, G. Chen, and J. Wang(2025)Table-critic: a multi-agent framework for collaborative criticism and refinement in table reasoning.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.),Vienna, Austria,pp. 17432–17451.External Links:Link,Document,ISBN 979-8-89176-251-0Cited by:§1,§3.4.
• [14]J. Chen, A. Prasad, S. Saha, E. Stengel-Eskin, and M. Bansal(2025)MAgICoRe: multi-agent, iterative, coarse-to-fine refinement for reasoning.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.),Suzhou, China,pp. 32663–32686.External Links:Link,Document,ISBN 979-8-89176-332-6Cited by:§1,§3.4.
• [15]J. Zhang, Z. Tian, X. Feng, X. Chen, and C. Chen(2025)Enhancing recommendation explanations through user-centric refinement.InFindings of the Association for Computational Linguistics: EMNLP 2025,C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.),Suzhou, China,pp. 8177–8191.External Links:Link,Document,ISBN 979-8-89176-335-7Cited by:§1.
• [16]N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao(2023)Reflexion: language agents with verbal reinforcement learning.InAdvances in Neural Information Processing Systems,A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.),Vol.36,pp. 8634–8652.External Links:Document,LinkCited by:§2.
• [17]O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. V. A, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts(2024)DSPy: compiling declarative language model calls into state-of-the-art pipelines.InThe Twelfth International Conference on Learning Representations,External Links:LinkCited by:§2.
• [18]C. Yang, X. Wang, Y. Lu, H. Liu, Q. V. Le, D. Zhou, and X. Chen(2024)Large language models as optimizers.InInternational Conference on Learning Representations,B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.),Vol.2024,pp. 12028–12068.External Links:LinkCited by:§2.
• [19]S. Hu, C. Lu, and J. Clune(2025)Automated design of agentic systems.InInternational Conference on Learning Representations,Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.),Vol.2025,pp. 21344–21377.External Links:LinkCited by:§2.
• [20]J. Zhang, J. Xiang, Z. Yu, F. Teng, X. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, B. Zheng, B. Liu, Y. Luo, and C. Wu(2025)AFlow: automating agentic workflow generation.InInternational Conference on Learning Representations,Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.),Vol.2025,pp. 34040–34077.External Links:LinkCited by:§2.
• [21]J. Zhang, S. Hu, C. Lu, R. T. Lange, and J. Clune(2026)Darwin gödel machine: open-ended evolution of self-improving agents.InThe Fourteenth International Conference on Learning Representations,External Links:LinkCited by:§2.
• [22]H. Zhang, S. Zhang, K. Li, C. Zhang, Y. Chen, Y. Zhang, L. Bai, and S. Hu(2026)Self-harness: harnesses that improve themselves.External Links:2606.09498,LinkCited by:§2.
• [23]X. Yu, B. Peng, R. Xu, H. Zou, Q. Wu, H. Cheng, W. Yao, N. Singh, Z. Yu, and J. Gao(2026)OpenForgeRL: train harness-native agents in any environment.External Links:2607.21557,LinkCited by:§2.
• [24]R. Wang, Y. Shi, Z. Li, Z. Li, Y. Yu, J. Yang, K. Panaganti, H. Mi, D. Zhou, and Leoweiliang(2026)Harness handbook: making evolving agent harnesses readable,navigable, and editable.External Links:2607.13285,LinkCited by:§2.
• [25]Y. Huang, W. Wang, H. Bao, Y. Ma, X. Luo, Y. Nian, H. Zhuang, Z. Liu, Y. Zhao, and X. Zhang(2026)MemoHarness: agent harnesses that learn from experience.External Links:2607.14159,LinkCited by:§2.
• [26]J. Xu, Y. Zhang, A. Chen, W. Li, J. Liang, and D. Yang(2026)Verify smarter, evolve further: efficient harness evolution through behavior-aware verification.arXiv preprint arXiv:2608.27311.Cited by:§2.
• [27]T. Xu, H. Wen, and M. Li(2026)Adapting the interface, not the model: runtime harness adaptation for deterministic llm agents.arXiv preprint arXiv:2605.22166.Cited by:3rd item,§4.1,§4.1.
• [28]A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu(2025)Qwen3 technical report.External Links:2505.09388,LinkCited by:§4.1.
• [29]A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Ling, C. Lu, C. Zhao, C. Deng, C. Hou, C. Xu, C. Shao, C. Ruan, C. Sun, D. Dai, D. Guo, D. Yang, D. Chen, D. Li, D. Ji, E. Li, F. Wei, F. Lin, F. Yuan, F. Xia, F. Dai, G. Hao, G. Chen, G. Cao, G. Meng, G. Li, H. Yu, H. Zhang, H. Xu, H. Li, H. Liang, H. Zhang, H. Luo, H. Wei, H. Yuan, H. Zhang, H. Luo, H. Chen, H. Ji, H. Zhang, H. Ding, H. Tang, H. Cao, H. Gao, H. Qu, H. Zeng, J. Yang, J. Zhu, J. Luo, J. Song, J. Yu, J. Huang, J. Cai, J. Liang, J. Zhou, J. Ye, J. Li, J. Xu, J. Hu, J. Yang, J. Chen, J. Yan, J. Chen, J. Zhou, J. Xiang, J. Yuan, J. Cheng, J. Zhou, J. Zhu, J. Yu, J. Sun, J. Ran, J. Jiang, J. Qiu, J. Li, J. Zheng, J. Song, K. Dong, K. Gao, K. Guan, K. Zhou, K. Huang, K. Yu, L. Wang, L. Zhang, L. Wang, L. Xia, L. Zhang, L. Zhao, L. Guo, L. Luo, L. Ma, L. Zhu, L. Wang, L. Cai, L. Zhang, L. Chen, M. Di, M. Xu, M. Mei, M. Wang, M. Zhang, M. Zhang, M. Tang, M. Li, M. Zhou, M. Han, N. Wang, P. Huang, P. Wang, P. Cong, P. Wang, P. Zhang, Q. Wang, Q. Zhu, Q. Li, Q. Chen, Q. Du, Q. Jiang, R. Tian, R. Xu, R. Lu, R. Xu, R. Ge, R. Zhang, R. Pan, R. Wang, R. Chen, R. Yin, R. Xu, R. Shen, R. Zhang, R. Chen, S. Liu, S. Lu, S. Sun, S. Zhou, S. Chen, S. Cai, S. Nie, S. Wu, S. Chen, S. Hu, S. Liu, S. Hu, S. Ma, S. Wang, S. Yu, S. Zhou, S. Pan, S. Yu, S. Zhou, T. Ni, T. Yun, T. Jin, T. Pei, T. Ye, T. Lin, T. Ji, T. Cui, T. Yue, T. Yu, T. Wang, W. Zhang, W. Xiao, W. Zeng, W. An, W. Zhao, W. Liu, W. Liang, W. Pang, W. Luo, W. Yao, W. Gao, W. Yang, W. Huang, W. Hou, W. Zhang, W. Ma, X. Gao, X. He, X. Wang, X. Wang, X. Bi, X. Liu, X. Wang, X. Chen, X. Zhang, X. Nie, X. Sun, X. Wang, X. Cheng, X. Liu, X. Xie, X. Liu, X. Liu, X. Yu, X. Li, X. Yang, X. Zhang, X. Chen, X. Wang, X. Su, X. Chen, X. Lin, X. Fu, Y. Yan, Y. Wang, Y. Ma, Y. Luo, Y. Zhang, Y. Xu, Y. Ma, Y. Huang, Y. Li, Y. Li, Y. Xu, Y. Zhao, Y. Sun, Y. Wang, Y. Qian, Y. Shao, Y. Yu, Y. Zhang, Y. Ding, Y. Shi, Y. Wu, Y. Xiong, Y. Ma, Y. He, Y. Tang, Y. Zhou, Y. Luo, Y. Zhong, Y. Piao, Y. Wang, Y. Zhang, Y. Chen, Y. Tan, Y. Wei, Y. Ma, Y. Liu, Y. Yang, Y. Guo, Y. Wu, Y. Wu, Y. Li, Y. Cheng, Y. Ou, Y. Xu, Y. Li, Y. Wang, Y. Yang, Y. Xu, Y. Wu, Y. Meng, Y. Zou, Y. Zha, Y. Xiong, Y. Chen, Y. Lin, Y. Cao, Y. Wang, Y. Zhang, Y. Yan, Y. Lin, Y. Gu, Y. Luo, Y. You, Y. Liu, Y. Zhou, Y. Zhou, Y. Huang, Z. Wu, Z. Wang, Z. Zhao, Z. Ren, Z. Zhang, Z. Sha, Z. Fu, Z. Ju, Z. Xu, Z. Xie, Z. Zhang, Z. Gao, Z. Hao, Z. Gou, Z. Ma, Z. Yan, Z. Shao, Z. Huang, Z. Chen, Z. Wu, Z. Ren, Z. Wu, Z. Li, Z. Zhang, Z. Xu, Z. Wang, Z. Qu, Z. Gu, Z. Zhu, Z. Li, Z. Zhang, Z. Xie, Z. Gao, Z. Wan, Z. Pan, and Z. Yao(2026)DeepSeek-v4: towards highly efficient million-token context intelligence.External Links:2606.19348,LinkCited by:§4.1.
• [30]X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang, S. Zhang, X. Deng, A. Zeng, Z. Du, C. Zhang, S. Shen, T. Zhang, Y. Su, H. Sun, M. Huang, Y. Dong, and J. Tang(2024)AgentBench: evaluating LLMs as agents.InThe Twelfth International Conference on Learning Representations,External Links:LinkCited by:§4.1.
• [31]V. Barres, H. Dong, S. Ray, X. Si, and K. R. Narasimhan(2026)τ2\tau^{2}-Bench: evaluating conversational agents in a dual-control environment.InForty-third International Conference on Machine Learning,External Links:LinkCited by:§4.1.
• [32]Z. Hou, J. Zhang, X. Cai, and K. You(2026)What matters in on-policy distillation? a perspective on data efficiency and data selection.External Links:2609.05198,LinkCited by:§8.

Table 7:Accuracy results across five models on AgentBench.Table 8:Inference cost results across five LLMs and methods on AgentBench.Table 9:Input-cache utilization during harness evolution on AgentBench.Table 10:Immediate post-evolution evaluation of the evolved harnesses on AgentBench with Qwen3-8B.Table 11:Harness self-evolution API costs on AgentBench.Table 12:Training time (s) for the three harness self-evolution methods on AgentBench.Table 13:Data-efficient harness evolution on theτ2\tau^{2}-Retail evaluated with Qwen3-32B. Harnesses are evolved using Qwen3-8B. The Full setting uses the complete training set, whereas Reduced and Random each use five training failures, corresponding to one-quarter of the full training set.