Auditing Harness Tampering in Self-Improving Agents

# Auditing Harness Tampering in Self-Improving Agents
Source: [https://arxiv.org/html/2609.00069](https://arxiv.org/html/2609.00069)
###### Abstract

Self\-improving agents iteratively modify their own harness to push the frontier of their performance\. However, such modifications can produce illusory performance gains or compromise integrity constraints such as authorization, provenance, and completeness without genuinely improving capability\. We term this phenomenon asharness tampering, which extends the concept from reward and measurement tampering to the full self\-improvement lifecycle\. To systematically study this problem, we propose a two\-axis taxonomy that categorizes each misaligned edit by the harness functional role in which it occurs and the obligation it violates\. Then we build an annotated corpus by seeding tampered–benign edit pairs into the real trajectories of self\-improving agents\. We adapt and benchmark diverse audit methods on tampering classification and localization tasks\. Finally we systematically audit real trajectories of self\-improving agents\. The results demonstrate that harness tampering consistently occurs in real runs from different agents, often persists in the lineage of the best agent, and forms distinct system\-specific profiles across the taxonomy\.

University of Electronic Science and Technology of China

wangxing@std\.uestc\.edu\.cn

## Introduction

Large language models are increasingly deployed through agent harnesses that coordinate model calls, tools, control flow, context and memory\([Wang et al\. 2024b](https://arxiv.org/html/2609.00069#bib.bib55);[Xi et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib56);[Zhuge et al\. 2024](https://arxiv.org/html/2609.00069#bib.bib48)\)\. In self\-improving agents, the harness itself becomes an object of optimization when agent programs, workflows, or persistent state are updated to improve future performance\([Gao et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib8);[Ren et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib35);[Hu et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib1);[Zhang et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib2);[Zhang et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib3);[Zhang et al\. 2026c](https://arxiv.org/html/2609.00069#bib.bib26);[Zhang et al\. 2026a](https://arxiv.org/html/2609.00069#bib.bib4)\)\. The optimization loop proposes changes, runs the changed variants on tasks, evaluates the results, and retains the selected variant\. Repeating this loop allows genuine improvements in task performance to accumulate over time\.

Figure 1:Harness tampering: an edit hidden inΔt\\Delta\_\{t\}overwrites the recorded accuracy\.![Refer to caption](https://arxiv.org/html/2609.00069v1/main-pipeline.png)Figure 2:Overview\. Left: the two\-axis taxonomy of functional roles and obligations\. \(A\) We seed matched tampered–benign edits into recorded harness changes to build a labeled corpus, and evaluate prompted and trained auditors on classification and localization\. \(B\) We apply the selected auditor to real trajectories of self\-improving systems and analyze the prevalence, persistence, and profiles of the findings\.While this recursive self\-improvement \(RSI\) pushes the frontier of agents’ performance, it risks exposing and amplifying an agent’s misaligned behavior when changing the harness\. As shown in Fig\.[1](https://arxiv.org/html/2609.00069#Sx1.F1), consider a harness change where most of edits are benign, two lines are edited to overwrite the original overall accuracy into fixed1\.0before saved to the disk\. This two\-line edit exposes the integrity risk when changing harness\. More generally, a change can make a variant appear improved without improving its capability, but by altering how the variant is run, how its results are measured, how they are recorded, which variant is retained, or what is carried into later iterations\. The harness change also can involve authorization, completeness on required component and other misaligned issue\. Prior work on reward tampering and measurement tampering examines how optimization can alter the reward mechanism or measurements used to determine success\([Everitt et al\. 2021](https://arxiv.org/html/2609.00069#bib.bib5);[Roger et al\. 2023](https://arxiv.org/html/2609.00069#bib.bib6);[Atinafu and Cohen 2026](https://arxiv.org/html/2609.00069#bib.bib7);[Thaman 2026](https://arxiv.org/html/2609.00069#bib.bib15)\)\. Self\-improving agents place these concerns in a new setting, where the optimized harness spans all components of an agent’s lifecycle including execution, evaluation, selection, recording, and propagation\. We term this problem*harness tampering*\.

To fill this gap and systematically analyze this misalignment during RSI, we propose a taxonomy for harness tampering\. By two axes, the taxonomy describes where a misalignment occurs on the harness and what specific category of obligation has failed\. The functional\-role axis assigns the misalignment to the harness operation in which it directly occurs \(execution, evaluation, selection, recording, or propagation\) on the lifecycle of self\-improvement\. The obligation axis identifies the integrity condition that the agent should satisfy but violates\. For example, if a “improved” harness variant tampers with an already computed score before reporting the score, this behavior on the functional axis is recording and its obligation axis is representational fidelity because the stored value no longer matches the result it claims to represent\.

Along with the taxonomy, we adapt diverse audit methods to detect harness tampering and ask how well they classify and localize the misalignment\. Since existing real harness changes carry no verified labels, we build an annotated corpus by seeding matched tampered–benign edit pairs into recorded changes from three representative RSI systems\([Zhang et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib2);[Zhang et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib3);[Zhang et al\. 2026c](https://arxiv.org/html/2609.00069#bib.bib26)\), where each pair shares a location and an apparent functional goal and differs only in whether an obligation is violated\. On this corpus we compare eight prompted language models, a fastText classifier and a LoRA\-adapted small model trained on our training split, scoring both whether an edit is tampering and whether its functional role and obligation are correctly attributed\.

We then use the selected auditor to ask what an audit of current self\-improving agents reveals and where the tampering localizes on the proposed two axes\. We analyze publicly released run materials from ADAS\([Hu et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib1)\), AFlow\([Zhang et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib2)\), DGM\([Zhang et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib3)\), HyperAgents\([Zhang et al\. 2026c](https://arxiv.org/html/2609.00069#bib.bib26)\)and ScientistOne\([Meng et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib42)\), and characterize the prevalence of tampering, its persistence along the lineage of the best agent, and the system\-specific profiles it forms across the taxonomy\.

We summarize our contributions as follows:

- •We propose a two\-axis taxonomy for harness tampering in self\-improving agents, which characterizes misalignments by the directly affected harness functional role and the violated obligation\.
- •We curate a dataset of tampered–benign harness edits using a fault\-seeding paradigm based on real self\-improvement trajectories and benchmark diverse audit methods ranging from lightweight classifiers to frontier LLMs\.
- •With selected audit methods, we audit existing self\-improving agents and provide a systematic analysis of real harness tampering that occurs across the taxonomy\.

## Related Work

### Self\-Improving Agents and Harness Evolution

Self\-improving agents enhance their capabilities across iterations by optimizing different layers of their operational scaffolding\([Gao et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib8);[Fang et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib9);[Du et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib10);[Ning et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib11)\)\. Early paradigms focus on optimizing task\-level context and behavioral strategies, accumulating experiences in episodic memory\([Shinn et al\. 2023](https://arxiv.org/html/2609.00069#bib.bib43);[Zhao et al\. 2024](https://arxiv.org/html/2609.00069#bib.bib45)\), synthesizing reusable skill libraries\([Wang et al\. 2024a](https://arxiv.org/html/2609.00069#bib.bib44)\), or evolving natural\-language prompts\([Yang et al\. 2024](https://arxiv.org/html/2609.00069#bib.bib46);[Fernando et al\. 2024](https://arxiv.org/html/2609.00069#bib.bib47)\)\. To unlock more expressive adaptation, recent frameworks expand the mutable boundary to code\-defined agent architectures and workflows\. For instance, ADAS evolves algorithmic agent architectures from an explicit candidate archive\([Hu et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib1)\), AFlow iteratively searches executable workflow topologies\([Zhang et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib2)\), and DGM retains discovered coding agents within an evolutionary pool\([Zhang et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib3)\)\. More recent systems further generalize this search space by co\-evolving task agents, search algorithms, and meta\-level harnesses\([Zhang et al\. 2026c](https://arxiv.org/html/2609.00069#bib.bib26);[Lee et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib12);[Luo et al\. 2026a](https://arxiv.org/html/2609.00069#bib.bib13)\), often introducing non\-regression or validation checks to regulate behavioral drift\([Zhang et al\. 2026a](https://arxiv.org/html/2609.00069#bib.bib4)\)\.

However, expanding self\-modification into the execution scaffolding introduces severe threats to evaluation validity\. Recent studies indicate that apparent evolutionary gains often reflect benchmark overfitting or evaluation artifacts rather than genuine capability advances\([Lin et al\. 2026a](https://arxiv.org/html/2609.00069#bib.bib49);[Wang et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib50);[Luo et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib51)\)\. We focus on the underlying vulnerability: when an optimization loop can rewrite its own harness, it can tamper with the mechanisms that execute, evaluate, select, and record variants\.

### Tampering and Evaluation Integrity

Reward and measurement tampering describe failure modes where optimization exploits the feedback channel, producing illusory gains without advancing the underlying objective\([Everitt et al\. 2021](https://arxiv.org/html/2609.00069#bib.bib5);[Roger et al\. 2023](https://arxiv.org/html/2609.00069#bib.bib6)\)\. In executable agent environments, prior work documents agents cheating within task sandboxes by disabling assertions, leaking labels, or falsifying intermediate metrics\([Denison et al\. 2024](https://arxiv.org/html/2609.00069#bib.bib14);[Atinafu and Cohen 2026](https://arxiv.org/html/2609.00069#bib.bib7);[Thaman 2026](https://arxiv.org/html/2609.00069#bib.bib15);[Zhao et al\. 2026a](https://arxiv.org/html/2609.00069#bib.bib59);[Roth et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib24)\)\.

In self\-improving agents, tampering poses a distinct structural challenge beyond single\-episode reward hacking\. Because modified harness components are retained and inherited across iterations, integrity failures can accumulate as persistent system changes and compound across evolutionary cycles\([Shao et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib52);[Zhao et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib53);[Lin et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib54)\)\. While prior benchmarks evaluate whether an agent cheats on a given task, we audit tampering at the level of iterative harness code changes, structured by our two\-axis taxonomy\.

### Auditing Agent Trajectories and Code Artifacts

Detecting unauthorized modifications connects directly to research in AI control and safety monitoring\([Greenblatt et al\. 2024](https://arxiv.org/html/2609.00069#bib.bib16)\)\. Existing protocols inspect agent workspaces during scientific discovery\([Kutasov et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib60);[Libon et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib17)\), audit multi\-turn pull requests\([Hills et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib18)\), verify test integrity against leakage\([Tu et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib19)\), or analyze counterfactual execution traces\([Zhou et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib21)\)\.

Despite these advances, existing monitors predominantly target static code submissions or full trajectory rollouts\. They lack the structural taxonomies and diagnostic benchmarks needed to audit mutations embedded within recursive improvement loops\. Our work addresses this gap by introducing a change\-level auditing framework\. It evaluates whether models can detect tampering and identify the exact code edits that caused the tampering, even when those edits are mixed with benign optimizations to the evaluation harness\.

Table 1:The seven obligations, each stated as a condition that must hold; a case is labeled with the obligation whose condition is false\. Cited works provide the conceptual basis, while the organization and operational tests are ours\.

## Task Formulation and Taxonomy

A self\-improving agent evolves by repeatedly modifying its own harness code\. We call the set of edits applied in one iteration a*harness change*\. Most harness changes pursue the intended goal: they refine the workflow so that the underlying model solves tasks better\. Some, however, take shortcuts that are misaligned with the intended objective, where measured performance is increased without corresponding gain in real capability\. We refer to such misaligned harness changes as*harness tampering*\.

This work studies*harness\-tampering auditing*, which we decompose into two questions\.*Classification*asks whether a given harness change contains misaligned behavior and, if so, of what kind\.*Localization*asks where in the self\-improvement lifecycle the tampering is introduced, by localizing the responsible code\. To answer these questions systematically, we organize tampering behaviors along two axes: a*functional role*axis, which identifies the component of the harness whose obligation fails, and an*obligation*axis, which identifies the type of integrity condition that is violated\. The following sections detail the audit task and the proposed taxonomy in turn\.

### Harness\-Tampering Classification and Localization

A self\-improving agent progresses through many iterations, each updating its harness\. Within an iteration, a harness change applied can be taken as a single unit: one iteration\-to\-iteration transition carries every edit the updater proposed at that step\. We therefore take the applied change of one iteration as the unit of audit\.

Auditing such a unit asks whether it introduces any misalignment that compromises how results are executed, evaluated, recorded, selected or propagated to subsequent iterations\. A positive finding requires both an identifiable misalignment and causal support linking it to this change\. Since a single applied change may carry several misalignments, the audit must resolve not only whether a change is clean, but*which of its edits*are not\.

This yields two concrete tasks\. In*harness\-tampering classification*, the audit method receives the applied change, the original codebase, and a specified code segment, and outputs whether that segment constitutes misaligned tampering and, if so, its category in our proposed taxonomy\. In*harness\-tampering localization*, the method receives the same input without the specified segment, and must additionally output the code line numbers that localize the tampering\.

### Harness\-Tampering Taxonomy

We characterize each tampering case along two axes\. The*functional\-role axis*records the specific harness operation whose obligation directly fails while the*obligation axis*records the fine\-grained obligation that fails there\. Informally, the first axis asks*where in the loop the misalignment occurs*and the second asks*which obligation failed*\. We write a labeled case as⟨\\langleRole, Obligation⟩\\rangle, for example⟨\\langleRecording, Representational fidelity⟩\\rangle\. Both axes are required to be decidable from observable differences between the pre\-change and post\-change state, neither of which refers to the model’s intent or to whether the change benefited the variant\.

#### Functional roles\.

The functional\-role axis distinguishes operations by the function they perform within the self\-improvement process\. It comprises five roles\.*Execution*applies, configures, loads, or runs a variant\.*Evaluation*converts execution outputs into measurements, scores, or judgments\.*Selection*uses results and other criteria to accept, reject, retain, promote, or choose variants\.*Recording*creates or updates records of versions, results, or decisions\.*Propagation*retrieves or supplies variants, records, memory, or context for use in a later iteration\. These roles describe what an operation does, rather than when it occurs or which component performs it\.

#### Obligations\.

The obligation axis distinguishes cases according to the type of obligation that is violated\. We develop the axis by abstracting recurring integrity failures from concrete cases in prior work on reward tampering and evaluation integrity\([Roger et al\. 2023](https://arxiv.org/html/2609.00069#bib.bib6);[Denison et al\. 2024](https://arxiv.org/html/2609.00069#bib.bib14);[Atinafu and Cohen 2026](https://arxiv.org/html/2609.00069#bib.bib7);[Wang et al\. 2026a](https://arxiv.org/html/2609.00069#bib.bib20);[Deshpande et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib41);[Thaman 2026](https://arxiv.org/html/2609.00069#bib.bib15)\), and distinguish these failures using established concepts of measurement validity, access control, information integrity, data provenance, and completeness\([Everitt et al\. 2021](https://arxiv.org/html/2609.00069#bib.bib5);[Kane 2013](https://arxiv.org/html/2609.00069#bib.bib36);[Saltzer and Schroeder 1975](https://arxiv.org/html/2609.00069#bib.bib37);[Clark and Wilson 1987](https://arxiv.org/html/2609.00069#bib.bib38);[Buneman et al\. 2001](https://arxiv.org/html/2609.00069#bib.bib39);[Wang and Strong 1996](https://arxiv.org/html/2609.00069#bib.bib40)\)\. Table[1](https://arxiv.org/html/2609.00069#Sx2.T1)presents the seven obligations, each stated as a condition that must hold; a case is labeled with the obligation whose condition is false\. These seven obligations fall into four families: fidelity \(representational, provenance\), integrity \(procedural, boundary, artifact\), validity, and completeness\.

ClassificationLocalizationAuditorAcc\.F1Func\.Oblig\.Cost \($\)Rec\.FPR↓\\downarrowFunc\.Oblig\.Cost \($\)Random49\.651\.010\.78\.0n/a7\.68\.51\.31\.0n/aPrompted auditors![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/qwen.png)Qwen 3\.5 9B53\.743\.86\.411\.11\.748\.235\.27\.813\.63\.6![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/gemini.png)Gemini 3\.7 Flash56\.030\.111\.910\.19\.924\.80\.215\.216\.414\.7![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/grok.png)Grok 4\.683\.682\.943\.842\.051\.859\.83\.235\.040\.1120\.4![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/anthropic.png)Claude Sonnet 580\.081\.241\.437\.638\.158\.77\.426\.432\.568\.6![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/anthropic.png)Claude Opus 590\.491\.047\.649\.596\.884\.29\.453\.355\.2168\.9![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/openai.png)GPT 5\.6 Luna89\.690\.045\.152\.32\.974\.96\.340\.246\.07\.9![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/openai.png)GPT 5\.6 Terra89\.189\.850\.149\.326\.279\.06\.243\.553\.093\.0![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/openai.png)GPT 5\.6 Sol87\.688\.546\.852\.364\.686\.79\.349\.261\.3244\.0Trained auditorsfastText56\.239\.7n/an/an/a31\.39\.5n/an/an/a![[Uncaptioned image]](https://arxiv.org/html/2609.00069v1/logos/qwen.png)Qwen 3\.5 9B \+ LoRA53\.569\.651\.550\.71\.783\.481\.249\.759\.23\.5Table 2:Classification and localization performance of random, prompted, and trained auditors\. Acc\. and F1 denote classification accuracy and F1 score; Rec\. denotes seeded recall, and FPR denotes the seeded false\-positive rate\. Func\. and Oblig\. report the accuracies of the predicted functional role and obligation, respectively\. Higher is better except for FPR and cost\. Cost \($\) is the estimated USD cost of one evaluation run at public API prices\.Table 3:Prevalence of tampering findings in the experiment runs of publicly released materials, audited with GPT 5\.6 Luna\.*Tampering rate*is the percentage of iterations containing at least one finding\.*Average tampering*is the total number of findings divided by all iterations, including zero\-finding iterations\.

## Audit Dataset Construction

To evaluate whether current audit methods can classify and localize harness tampering in realistic self\-improvement agents, we construct an audit corpus supporting the two auditing tasks \(Fig\.[2](https://arxiv.org/html/2609.00069#Sx1.F2)\)\. Building such a corpus requires balancing realistic harness change with reliable ground truth\. Real harness changes often span many edits across multiple files, forcing auditors to search for tampering among substantial surrounding modifications\. However, these changes do not come with verified labels indicating whether or where tampering occurs\. Conversely, constructing isolated patches from scratch would provide precise labels but discard the complex context challenge that auditors face in actual self\-improvement runs\.

To retain both, we adapt the fault\-seeding paradigm\([Weinberg 1971](https://arxiv.org/html/2609.00069#bib.bib23);[Jia and Harman 2010](https://arxiv.org/html/2609.00069#bib.bib57);[Just et al\. 2014](https://arxiv.org/html/2609.00069#bib.bib58);[Yang et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib22)\)to recorded harness changes produced by HyperAgents\([Zhang et al\. 2026c](https://arxiv.org/html/2609.00069#bib.bib26)\), DGM\([Zhang et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib3)\), and AFlow\([Zhang et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib2)\)\. For each run, we recover the complete current diff and the exact pre\-change versions of every file the diff touches\. Then we carefully design the prompt of tampering generator seeded with real tampering example from manually identified examples from real trajectories of self\-improving agents\. A generator model, specifically GPT 5\.6 Sol, is then prompted to create 1–5 tampered\-benign edit pairs within each recorded change\. Each seeded tampering edit is required to violate a specified obligation, while corresponding the benign edit is injected at the same location to pursue the same apparent functional role without violating the obligation\.

Under this pipeline, labels are established by construction rather than inferred through post\-hoc model judgment\. This design helps mitigate potential LLM\-induced biases, as an input constraint rather than an output of model judgment, thereby reducing circular evaluation dependencies\. Furthermore, pairing each tampering edit with a matched benign counterpart prevents auditors from exploiting GPT 5\.6’s stylistic signatures as a discriminating shortcut\.

The resulting dataset comprises an evaluation set of 1,765 classification and 1,801 localization samples, alongside a separate training set of 1,500 classification samples and 1,000 localization samples used to fine\-tune Qwen 3\.5 9B on this audit task for comparison\. To ensure high data validity, we conducted human quality checks on a subset of the dataset \(see the Supplementary Materials for details\)\.

## Experiment

### Auditor Evaluation on Dataset

#### Audit methods\.

Across the classification and localization tasks defined in Sec\. Task Formulation and Taxonomy, we compare a random baseline with two auditor settings: prompted auditors and auditors trained on our corpus\. The prompted setting includes eight language models: Qwen 3\.5 9B\([Qwen Team 2026](https://arxiv.org/html/2609.00069#bib.bib27)\), Gemini 3\.7 Flash\([Google 2026](https://arxiv.org/html/2609.00069#bib.bib28)\), Grok 4\.6\([xAI 2026](https://arxiv.org/html/2609.00069#bib.bib29)\), Claude Sonnet 5\([Anthropic 2026b](https://arxiv.org/html/2609.00069#bib.bib30)\), Claude Opus 5\([Anthropic 2026a](https://arxiv.org/html/2609.00069#bib.bib31)\), GPT 5\.6 Luna\([OpenAI 2026a](https://arxiv.org/html/2609.00069#bib.bib32)\), GPT 5\.6 Terra\([OpenAI 2026c](https://arxiv.org/html/2609.00069#bib.bib33)\), and GPT 5\.6 Sol\([OpenAI 2026b](https://arxiv.org/html/2609.00069#bib.bib34)\)\. For the trained setting, we use the held\-out corpus training split to train a fastText classifier\([Joulin et al\. 2017](https://arxiv.org/html/2609.00069#bib.bib25)\)and a LoRA\-adapted Qwen 3\.5 9B for both tasks\.

#### Evaluation Protocol

Following fault\-seeding dataset construction, we evaluate auditors on the tampering classification and localization tasks\. For each sample, an auditor receives the complete current diff of a recorded change with the relevant seeded edits spliced in, together with the exact pre\-change versions of all files touched by the diff\.

In the classification task, given the harness diff alongside the codebase, the auditor evaluates specific diff line numbers to determine whether they represent tampering behavior and classifies them along both taxonomy axes\. We measureAccuracyas the proportion of samples for which the auditor correctly identifies the presence or absence of tampering, andF1as the harmonic mean of precision and recall on the positive \(i\.e\., tampered\) class\. For the two taxonomy axes, we report classification accuracy on functional roles and obligations separately\. Notably, if an auditor fails to detect a tampering edit \(i\.e\., produces a false negative\), both axis\-level accuracy scores for that sample are set to zero, as a missed detection precludes any downstream categorization\. All classification metrics are macro\-averaged over all seeded edits\.

The localization task requires the auditor to output a list of tampered edits by specifying code line number and category along with two taxonomy axes\. We evaluate localization using two complementary metrics:

- •Seeded Recall: For each diff containingNtampN\_\{\\text\{tamp\}\}seeded tampering edits, we count the number of editsNhitN\_\{\\text\{hit\}\}whose line locations are covered by the auditor’s predicted lines, yielding a recall ofNhit/NtampN\_\{\\text\{hit\}\}/N\_\{\\text\{tamp\}\}\.
- •Seeded\-Control False\-Positive Rate \(FPR\): For each diff containingNbenignN\_\{\\text\{benign\}\}seeded benign edits, we count the number of editsNFPN\_\{\\text\{FP\}\}whose line locations are within seeded benign edits and incorrectly flagged as tampering, yielding a FPR ofNFP/NbenignN\_\{\\text\{FP\}\}/N\_\{\\text\{benign\}\}\.

Both metrics are macro\-averaged across all evaluated harness changes\. Functional\-role and obligation accuracies are computed over the seeded tampering edits in the same manner as in the classification task, assigning a score of zero to missed localizations\.

Figure 3:Findings carried by the best agent across iterations\. Curves are averaged over runs and truncated once fewer than 60% remain active\. ADAS exposes no reconstructable lineage, so its dashed line reports the overall mean\.Figure 4:When tampering findings enter the ancestry of the final best agent\. Each dot marks the iteration introducing one finding, and the segment runs from the earliest finding to delivery\. Empty rows are clean ancestries\.Figure 5:Composition of all tampering findings by functional role \(left\) and obligation \(right\), normalized within each system with colors fixed across systems\. Totals are 145 \(ADAS\), 21 \(AFlow\), 166 \(DGM\), 425 \(HyperAgents\), and 17 \(ScientistOne\)\.
#### Main results\.

Table[2](https://arxiv.org/html/2609.00069#Sx3.T2)summarizes auditing performance and cost on both tasks\. Claude Opus 5 achieves the strongest classification performance, with 90\.4% accuracy and 91\.0% F1\. Localization is more challenging and yields greater separation among prompted auditors: seeded recall ranges from 24\.8% to 86\.7%, compared to classification accuracies of 53\.7%–90\.4%\. GPT 5\.6 Sol and Claude Opus 5 achieve the highest localization recall, though both exhibit an FPR of approximately 10% on benign controls\. Across all auditors, functional\-role and obligation accuracies remain lower, suggesting that explaining how and why an edit constitutes tampering is harder than identifying it\.

Within the GPT 5\.6 family, localization recall increases progressively from Luna to Terra and Sol\. Relative to Luna, Sol improves seeded recall by 11\.8 percentage points but increases the FPR by 3\.0 percentage points\. Crucially, this gain comes at substantially higher monetary cost: Luna requires only 4\.5% of Sol’s cost on classification and 3\.2% on localization\. Luna therefore offers the most favorable cost–performance trade\-off for classification, while Sol provides higher localization recall at the expense of a higher FPR\.

Beyond general\-purpose prompted models, we investigate whether domain\-specific adaptation can enhance auditing capability on smaller models\. As shown in Table 2, task\-specific LoRA adaptation substantially improves Qwen 3\.5 9B: while classification accuracy remains nearly unchanged, F1 increases from 43\.8% to 69\.6%, functional\-role accuracy from 6\.4% to 51\.5%, and obligation accuracy from 11\.1% to 50\.7%\. The improvement is even larger on localization, where seeded recall rises from 48\.2% to 83\.4% and attribution accuracies approach the strongest prompted auditors, albeit at the cost of a higher false positive rate \(81\.2%\)\. This trade\-off indicates that LoRA adaptation favors aggressive flagging over specificity, making the model more suitable for high\-recall screening than for standalone auditing\. Finally, fastText achieves 56\.2% accuracy but only 39\.7% F1, below the random baseline of 51\.0%\. This suggests that surface\-level lexical features do not provide a reliable signal for identifying tampering\.

### Auditing Self\-Improving Agents

To evaluate harness tampering in realistic settings, we analyze publicly released execution traces from four self\-improving agent systems: ADAS\([Hu et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib1)\), AFlow\([Zhang et al\. 2025](https://arxiv.org/html/2609.00069#bib.bib2)\), DGM\([Zhang et al\. 2026b](https://arxiv.org/html/2609.00069#bib.bib3)\), and HyperAgents\([Zhang et al\. 2026c](https://arxiv.org/html/2609.00069#bib.bib26)\)\. We additionally include ScientistOne\([Meng et al\. 2026](https://arxiv.org/html/2609.00069#bib.bib42)\)as an informative boundary case\. While ScientistOne does not evolve execution harnesses by itself, its agents directly modify task workspaces and shared environment configurations\. Across all systems, we focus strictly on the primary self\-improvement runs from the original papers, excluding baseline and ablation trajectories\.

For consistency, we align our audit unit with each system’s native iteration, mapping each self\-improvement step to a single Luna auditing call\. We employ GPT 5\.6 Luna \(hereafter, Luna\) as the auditor due to its strong cost\-efficiency \(74\.9% seeded recall, 6\.3% seeded\-control FPR\)\. To assess whether performance on the seeded dataset reflects reliability on naturally occurring diffs, we manually inspected a sample of its actual localization predictions during evaluation \(detailed in the Supplementary Materials\)\.

#### Prevalence\.

Harness tampering is observed across all evaluated self\-improving systems, though their empirical frequencies differ \(Table[3](https://arxiv.org/html/2609.00069#Sx3.T3)\)\. We report both the Luna\-identified*tampering rate*\(the proportion of iterations containing≥1\\geq 1finding\) and*average findings*\(the total number of tampering findings divided by the total number of iterations\), hereafter referred to simply as the tampering rate and average findings\. These metrics distinguish overall frequency from per\-iteration density: while ADAS displays the highest fraction of tampered iterations \(84\.6%\), DGM exhibits the highest finding density \(1\.18 findings/iter despite a lower 63\.1% tampering rate\), indicating that multiple breaches often co\-occur within single modification steps\. HyperAgents also shows high tampering rates \(73\.6%, 0\.94 findings/iter\), whereas AFlow is more constrained \(18\.3% rate, 0\.18 findings/iter\)\. Finally, the ScientistOne boundary case suggests that tampering is not restricted to meta\-evolutionary frameworks, but also arises when agents modify task workspaces directly \(29\.6% rate, 0\.31 findings/iter\)\. It indicates the widespread existence of tampering behavior\. We provide an in\-depth discussion of the architectural and environmental factors driving these cross\-system disparities in Supplementary Materials\.

#### Persistence\.

To assess whether early tampering persists through evolutionary selection, we trace the ancestral lineage of the*best agent*at each iteration \(Fig\.[3](https://arxiv.org/html/2609.00069#Sx5.F3)\)\. Specifically, we follow each system’s original selection criterion to identify the best agent at iterationtt, reconstruct its ancestry via parent links, and count active findings introduced along its lineage\. Each curve is averaged across multiple runs and truncated once fewer than 60% of runs remain active to prevent survivor bias\. Because ADAS’s publicly released run materials do not contain sufficient metadata to reconstruct a continuous parent\-child lineage, its dashed line instead reports the overall mean of 1\.12 findings per iteration for reference\.

As shown in Fig\.[3](https://arxiv.org/html/2609.00069#Sx5.F3), HyperAgents and DGM exhibit tampering accumulation, with their final best agents retaining an average of 5\.2 and 3\.5 concurrent findings, respectively, while AFlow \(0\.3\) and ScientistOne \(1\.0\) remain low\.

Tracing the ancestral lineage of the*final best agents*\(Fig\.[4](https://arxiv.org/html/2609.00069#Sx5.F4)\) further reveals when these persistent tampering findings enter the lineage\. In long\-horizon systems \(HyperAgents and DGM\), findings introduced within the earliest iterations \(<10<10\) persist across the entire 80–100 iteration runs and remain active in the final agent\. In contrast, short\-horizon systems \(≤20\\leq 20iterations\) show minimal accumulation, with several final agents \(e\.g\., AFlow on HumanEval\) remaining completely clean\. These results suggest that tampering is not merely an isolated, late\-stage anomaly\. Uncorrected early modifications can also persist across dozens of iterations, being carried through to the final best agents\. It highlights the importance of self\-auditing component in the self\-improving agent\.

#### Profiles\.

Analyzing the distribution of all findings across the two\-axis taxonomy of all findings \(Fig\.[5](https://arxiv.org/html/2609.00069#Sx5.F5)\) reveals that tampering behaviors vary substantially with system architecture rather than adhering to a uniform mode\. Functionally, HyperAgents and AFlow predominantly modify evaluation harnesses \(59% and 38%, respectively\), whereas DGM shows a notable concentration in recording components \(30%\), reflecting changes to logging and state tracking\. In contrast, ADAS \(42%\) and ScientistOne \(41%\) most frequently tamper with execution routines\. Regarding obligations, violations in HyperAgents \(50%\) and ADAS \(41%\) are concentrated in procedural integrity, whereas AFlow primarily compromises provenance \(48%\), and DGM is more broadly distributed across representational fidelity \(27%\) and completeness \(22%\)\. The diversity of these profiles suggests that auditing mechanisms focused on a single functional role or obligation are unlikely to capture the full spectrum of tampering across different agent designs\. We provide more case studies in our Supplemental Materials\.

## Conclusion

We introduce a framework for auditing harness tampering in self\-improving agents, comprising a two\-axis taxonomy of functional roles and obligations, a dataset of matched tampered–benign edits constructed within recorded harness changes, and classification and localization tasks\. On the constructed dataset, the best\-performing language\-model auditors achieved high classification accuracy and localization recall, but taxonomy attribution remained substantially less accurate\. An audit of released materials from four self\-improving systems and one automated\-research system identifies tampering findings in every evaluated system, with substantial variation in their prevalence and taxonomy profiles\. In the two long\-horizon systems, some tampering findings introduced early remained in the ancestry of final selected agents across dozens of iterations\. These results demonstrate the risks associated with mutable harnesses and highlight the urgent need for tamper\-proof evaluation channels, provenance\-aware state tracking, and continuous auditing protocols in autonomous self\-improvement\.

## References

- Anthropic \(2026a\)AnthropicIntroducing Claude Opus 5\.External Links:[Link](https://www.anthropic.com/news/claude-opus-5)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- Anthropic \(2026b\)AnthropicIntroducing Claude Sonnet 5\.External Links:[Link](https://www.anthropic.com/news/claude-sonnet-5)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- Atinafu and Cohen \(2026\)Y\. Atinafu and R\. CohenRewardHackingAgents: benchmarking evaluation integrity for LLM ML\-engineering agents\.In2026 IEEE 42nd International Conference on Data Engineering Workshops,pp\. 27–34\.External Links:[Document](https://dx.doi.org/10.1109/ICDEW71238.2026.00009)Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p2.1),[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.4.3.1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.5.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Bunemanet al\.\(2001\)P\. Buneman, S\. Khanna, and W\. TanWhy and where: a characterization of data provenance\.InDatabase Theory: ICDT 2001,Lecture Notes in Computer Science, Vol\.1973,pp\. 316–330\.External Links:[Document](https://dx.doi.org/10.1007/3-540-44503-X%5F20)Cited by:[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.7.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Clark and Wilson \(1987\)D\. D\. Clark and D\. R\. WilsonA comparison of commercial and military computer security policies\.InProceedings of the 1987 IEEE Symposium on Security and Privacy,pp\. 184–195\.External Links:[Document](https://dx.doi.org/10.1109/SP.1987.10001)Cited by:[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.4.3.1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.6.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Denisonet al\.\(2024\)C\. Denison, M\. MacDiarmid, F\. Barez, D\. Duvenaud, S\. Kravec, S\. Marks, N\. Schiefer, R\. Soklaski, A\. Tamkin, J\. Kaplan, B\. Shlegeris, S\. R\. Bowman, E\. Perez, and E\. HubingerSycophancy to subterfuge: investigating reward\-tampering in large language models\.arXiv preprint arXiv:2406\.10162\.External Links:2406\.10162Cited by:[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.6.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Deshpandeet al\.\(2026\)D\. Deshpande, A\. Kannappan, and R\. QianBenchmarking reward hack detection in code environments via contrastive analysis\.arXiv preprint arXiv:2601\.20103\.External Links:[Link](https://arxiv.org/abs/2601.20103)Cited by:[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.8.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Duet al\.\(2026\)S\. Du, J\. Zhao, J\. Shi, Z\. Xie, X\. Jiang, Y\. Bai, and L\. HeA survey on the optimization of Large Language Model\-based agents\.ACM Computing Surveys58\(9\),pp\. 1–37\.External Links:[Document](https://dx.doi.org/10.1145/3789261)Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Everittet al\.\(2021\)T\. Everitt, M\. Hutter, R\. Kumar, and V\. KrakovnaReward tampering problems and solutions in reinforcement learning: a causal influence diagram perspective\.Synthese198\(S27\),pp\. 6435–6467\.External Links:[Document](https://dx.doi.org/10.1007/s11229-021-03141-4)Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p2.1),[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.2.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Fanget al\.\(2025\)J\. Fang, Y\. Peng, X\. Zhang, Y\. Wang, X\. Yi, G\. Zhang, Y\. Xu, B\. Wu, S\. Liu, Z\. Li, Z\. Ren, N\. Aletras, X\. Wang, H\. Zhou, and Z\. MengA comprehensive survey of self\-evolving AI agents: a new paradigm bridging foundation models and lifelong agentic systems\.arXiv preprint arXiv:2508\.07407\.External Links:2508\.07407Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Fernandoet al\.\(2024\)C\. Fernando, D\. S\. Banarse, H\. Michalewski, S\. Osindero, and T\. RocktäschelPromptbreeder: self\-referential self\-improvement via prompt evolution\.InICML,pp\. 13481–13544\.External Links:[Link](https://proceedings.mlr.press/v235/fernando24a.html)Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Gaoet al\.\(2026\)H\. Gao, J\. Geng, W\. Hua, M\. Hu, X\. Juan, H\. Liu, S\. Liu, J\. Qiu, X\. Qi,et al\.A survey of self\-evolving agents what, when, how, and where to evolve on the path to artificial super intelligenc\.Transactions on Machine Learning Research2026\-January\(English \(US\)\)\.External Links:ISSN 2835\-8856Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1),[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Google \(2026\)GoogleGemini 3\.7 Flash\.External Links:[Link](https://ai.google.dev/gemini-api/docs/models/gemini-3.7-flash)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- Greenblattet al\.\(2024\)R\. Greenblatt, B\. Shlegeris, K\. Sachan, and F\. RogerAI control: improving safety despite intentional subversion\.InICML,pp\. 16295–16336\.Cited by:[Auditing Agent Trajectories and Code Artifacts](https://arxiv.org/html/2609.00069#Sx2.SSx3.p1.1)\.
- Hillset al\.\(2026\)J\. Hills, I\. Caspary, and A\. C\. SticklandDistributed attacks in persistent\-state AI control\.arXiv preprint arXiv:2607\.02514\.External Links:2607\.02514Cited by:[Auditing Agent Trajectories and Code Artifacts](https://arxiv.org/html/2609.00069#Sx2.SSx3.p1.1)\.
- Huet al\.\(2025\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.InICLR,Vol\.2025,pp\. 21344–21377\.Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1),[Introduction](https://arxiv.org/html/2609.00069#Sx1.p5.1),[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1),[Auditing Self\-Improving Agents](https://arxiv.org/html/2609.00069#Sx5.SSx2.p1.1)\.
- Jia and Harman \(2010\)Y\. Jia and M\. HarmanAn analysis and survey of the development of mutation testing\.IEEE transactions on software engineering37\(5\),pp\. 649–678\.Cited by:[Audit Dataset Construction](https://arxiv.org/html/2609.00069#Sx4.p2.1)\.
- Joulinet al\.\(2017\)A\. Joulin, E\. Grave, P\. Bojanowski, and T\. MikolovBag of tricks for efficient text classification\.InProceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers,pp\. 427–431\.External Links:[Document](https://dx.doi.org/10.18653/v1/E17-2068)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- Justet al\.\(2014\)R\. Just, D\. Jalali, and M\. D\. ErnstDefects4J: a database of existing faults to enable controlled testing studies for java programs\.InProceedings of the 2014 international symposium on software testing and analysis,pp\. 437–440\.Cited by:[Audit Dataset Construction](https://arxiv.org/html/2609.00069#Sx4.p2.1)\.
- Kane \(2013\)M\. T\. KaneValidating the interpretations and uses of test scores\.Journal of Educational Measurement50\(1\),pp\. 1–73\.External Links:[Document](https://dx.doi.org/10.1111/jedm.12000)Cited by:[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.2.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Kutasovet al\.\(2025\)J\. Kutasov, Y\. Sun, P\. Colognese, T\. van der Weij, L\. Petrini, C\. B\. C\. Zhang, J\. Hughes, X\. Deng, H\. Sleight, T\. Tracy, B\. Shlegeris, and J\. BentonSHADE\-Arena: evaluating sabotage and monitoring in LLM agents\.arXiv preprint arXiv:2506\.15740\.External Links:2506\.15740,[Link](https://arxiv.org/abs/2506.15740)Cited by:[Auditing Agent Trajectories and Code Artifacts](https://arxiv.org/html/2609.00069#Sx2.SSx3.p1.1)\.
- Leeet al\.\(2026\)Y\. Lee, R\. Nair, Q\. Zhang, K\. Lee, O\. Khattab, and C\. FinnMeta\-Harness: end\-to\-end optimization of model harnesses\.arXiv preprint arXiv:2603\.28052\.External Links:2603\.28052Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Libonet al\.\(2026\)L\. Libon, B\. Rank, J\. Yeon, D\. Schmotz, J\. Qin, D\. Donnelly, D\. Prinzhorn, and M\. AndriushchenkoResearchArena: evaluating sabotage and monitoring in automated AI R&D\.arXiv preprint arXiv:2607\.19321\.External Links:2607\.19321Cited by:[Auditing Agent Trajectories and Code Artifacts](https://arxiv.org/html/2609.00069#Sx2.SSx3.p1.1)\.
- Linet al\.\(2026a\)M\. Lin, J\. Wu, Z\. Wang, Z\. Shi, Y\. Sang, B\. He, Z\. Liu, T\. Wei, Z\. Wu, Z\. Zhang,et al\.Harness updating is not harness benefit: disentangling evolution capabilities in self\-evolving LLM agents\.arXiv preprint arXiv:2605\.30621\.External Links:2605\.30621,[Link](https://arxiv.org/abs/2605.30621)Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p2.1)\.
- Linet al\.\(2026b\)R\. Lin, X\. Deng, Q\. Li, J\. Ma, Y\. Feng, Y\. Qing, Z\. Li, Y\. Zhang, S\. Cui, C\. Meng, T\. Zhang, X\. Ma, Q\. Li, K\. Xu, and S\. JiSafety in self\-evolving LLM agent systems: threats, amplification, and case studies\.arXiv preprint arXiv:2606\.23075\.External Links:2606\.23075,[Link](https://arxiv.org/abs/2606.23075)Cited by:[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p2.1)\.
- Luoet al\.\(2026a\)H\. Luo, Y\. Huang, S\. Luo, F\. Liu, L\. Li, Z\. Hu, J\. Feng, and Q\. LiuHarness\-aware self\-evolving: co\-evolving model weights, harness, and task solutions\.arXiv preprint arXiv:2607\.03935\.External Links:2607\.03935Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Luoet al\.\(2026b\)X\. Luo, D\. Xue, F\. Wang, C\. Hu, and Y\. DengHarnessBank: semantic gene\-bank search with gated verification for agent\-harness self\-evolution\.arXiv preprint arXiv:2607\.13683\.External Links:2607\.13683,[Link](https://arxiv.org/abs/2607.13683)Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p2.1)\.
- Menget al\.\(2026\)R\. Meng, B\. D\. Mishra, J\. Chen, C\. Li, P\. Goyal, M\. Parmar, Y\. Song, Y\. Song, R\. Sinha, P\. Ranganathan,et al\.ScientistOne: towards human\-level autonomous research via chain\-of\-evidence\.arXiv preprint arXiv:2605\.26340\.Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p5.1),[Auditing Self\-Improving Agents](https://arxiv.org/html/2609.00069#Sx5.SSx2.p1.1)\.
- Ninget al\.\(2026\)X\. Ning, K\. Tieu, D\. Fu, T\. Wei, Z\. Li, Y\. Bei, J\. Zou, M\. Ai, Z\. Liu, T\. Li,et al\.Code as agent harness\.arXiv preprint arXiv:2605\.18747\.External Links:2605\.18747Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- OpenAI \(2026a\)OpenAIGPT\-5\.6 Luna model\.External Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.6-luna)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- OpenAI \(2026b\)OpenAIGPT\-5\.6 Sol model\.External Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.6-sol)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- OpenAI \(2026c\)OpenAIGPT\-5\.6 Terra model\.External Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.6-terra)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.5\-9B\.Note:Model cardExternal Links:[Link](https://huggingface.co/Qwen/Qwen3.5-9B)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- Renet al\.\(2026\)Z\. Ren, Y\. Chen, D\. Guo, G\. Rong, T\. Li, R\. Xiong, Q\. Lan, W\. Wang, L\. Nanbo, Y\. Yang,et al\.Self\-improvements in modern agentic systems: a survey\.arXiv preprint arXiv:2607\.13104\.Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1)\.
- Rogeret al\.\(2023\)F\. Roger, R\. Greenblatt, M\. Nadeau, B\. Shlegeris, and N\. ThomasBenchmarks for detecting measurement tampering\.arXiv preprint arXiv:2308\.15605\.External Links:2308\.15605Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p2.1),[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.2.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Rothet al\.\(2026\)A\. Roth, A\. Samanta, M\. Halevy, Y\. Levine, and Y\. EfroniHack\-verifiable environments: towards evaluating reward hacking at scale\.arXiv preprint arXiv:2605\.20744\.Cited by:[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p1.1)\.
- Saltzer and Schroeder \(1975\)J\. H\. Saltzer and M\. D\. SchroederThe protection of information in computer systems\.Proceedings of the IEEE63\(9\),pp\. 1278–1308\.External Links:[Document](https://dx.doi.org/10.1109/PROC.1975.9939)Cited by:[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.5.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Shaoet al\.\(2026\)S\. Shao, Q\. Ren, D\. Liu, C\. Qian, B\. Wei, D\. Guo, J\. Yang, X\. Song, L\. Zhang, W\. Zhang, and J\. ShaoYour agent may misevolve: emergent risks in self\-evolving LLM agents\.InICLR,Vol\.2026,pp\. 99728–99793\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/file/a24cd16bc361afa78e57d31d34f3d936-Paper-Conference.pdf)Cited by:[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p2.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InNeurIPS,Vol\.36,pp\. 8634–8652\.External Links:[Document](https://dx.doi.org/10.52202/075280-0377),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html)Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Thaman \(2026\)K\. ThamanReward hacking benchmark: measuring exploits in LLM agents with tool use\.arXiv preprint arXiv:2605\.02964\.External Links:2605\.02964Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p2.1),[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.3.3.1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.6.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Tuet al\.\(2026\)X\. Tu, T\. Wang, Y\. Lu, K\. Huang, Y\. Qu, and S\. MostafaviBenchGuard: who guards the benchmarks? automated auditing of LLM agent benchmarks\.arXiv preprint arXiv:2604\.24955\.External Links:2604\.24955Cited by:[Auditing Agent Trajectories and Code Artifacts](https://arxiv.org/html/2609.00069#Sx2.SSx3.p1.1)\.
- Wanget al\.\(2024a\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.Transactions on Machine Learning Research\.External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=ehfRiF0R3a)Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Wanget al\.\(2026a\)H\. Wang, H\. Li, Q\. Mang, A\. Cheung, K\. Sen, and D\. SongDo androids dream of breaking the game? systematically auditing AI agent benchmarks with BenchJack\.arXiv preprint arXiv:2605\.12673\.External Links:2605\.12673Cited by:[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.3.3.1.1),[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.7.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Wanget al\.\(2024b\)L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin,et al\.A survey on large language model based autonomous agents\.Frontiers of computer science18\(6\),pp\. 186345\.Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1)\.
- Wang and Strong \(1996\)R\. Y\. Wang and D\. M\. StrongBeyond accuracy: what data quality means to data consumers\.Journal of Management Information Systems12\(4\),pp\. 5–33\.External Links:[Document](https://dx.doi.org/10.1080/07421222.1996.11518099)Cited by:[Table 1](https://arxiv.org/html/2609.00069#Sx2.T1.1.8.3.1.1),[Obligations\.](https://arxiv.org/html/2609.00069#Sx3.SSx2.SSS0.Px2.p1.1)\.
- Wanget al\.\(2026b\)Y\. Wang, H\. Zhu, Z\. Hu, Y\. Yuan, Z\. Chen, S\. Senthil, H\. Hajishirzi, Y\. Tsvetkov, P\. Dasigi, and T\. XiaoRethinking the evaluation of harness evolution for agents\.arXiv preprint arXiv:2607\.12227\.External Links:2607\.12227,[Link](https://arxiv.org/abs/2607.12227)Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p2.1)\.
- Weinberg \(1971\)G\. M\. WeinbergThe psychology of computer programming\.Van Nostrand Reinhold New York\.Cited by:[Audit Dataset Construction](https://arxiv.org/html/2609.00069#Sx4.p2.1)\.
- xAI \(2026\)xAIIntroducing Grok 4\.6\.External Links:[Link](https://docs.x.ai/developers/grok-4-6)Cited by:[Audit methods\.](https://arxiv.org/html/2609.00069#Sx5.SSx1.SSS0.Px1.p1.1)\.
- Xiet al\.\(2025\)Z\. Xi, W\. Chen, X\. Guo, W\. He, Y\. Ding, B\. Hong, M\. Zhang, J\. Wang, S\. Jin, E\. Zhou,et al\.The rise and potential of large language model based agents: a survey\.Science China information sciences68\(2\),pp\. 121101\.Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1)\.
- Yanget al\.\(2024\)C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. ChenLarge language models as optimizers\.InICLR,Vol\.2024,pp\. 12028–12068\.Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Yanget al\.\(2026\)J\. Yang, K\. Lieret, C\. Jimenez, A\. Wettig, K\. Khandpur, Y\. Zhang, B\. Hui, O\. Press, L\. Schmidt, and D\. YangSwe\-smith: scaling data for software engineering agents\.NeurIPS38\.Cited by:[Audit Dataset Construction](https://arxiv.org/html/2609.00069#Sx4.p2.1)\.
- Zhanget al\.\(2026a\)H\. Zhang, S\. Zhang, K\. Li, C\. Zhang, Y\. Chen, Y\. Zhang, L\. Bai, and S\. HuSelf\-Harness: harnesses that improve themselves\.arXiv preprint arXiv:2606\.09498\.External Links:2606\.09498Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1),[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Zhanget al\.\(2026b\)J\. Zhang, S\. Hu, C\. Lu, R\. Lange, and J\. CluneDarwin gödel machine: open\-ended evolution of self\-improving agents\.InICLR,Vol\.2026,pp\. 104223–104294\.Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1),[Introduction](https://arxiv.org/html/2609.00069#Sx1.p4.1),[Introduction](https://arxiv.org/html/2609.00069#Sx1.p5.1),[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1),[Audit Dataset Construction](https://arxiv.org/html/2609.00069#Sx4.p2.1),[Auditing Self\-Improving Agents](https://arxiv.org/html/2609.00069#Sx5.SSx2.p1.1)\.
- Zhanget al\.\(2026c\)J\. Zhang, B\. Zhao, W\. Yang, J\. Foerster, J\. Clune, M\. Jiang, S\. Devlin, and T\. ShavrinaHyperagents\.External Links:2603\.19461,[Link](https://arxiv.org/abs/2603.19461)Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1),[Introduction](https://arxiv.org/html/2609.00069#Sx1.p4.1),[Introduction](https://arxiv.org/html/2609.00069#Sx1.p5.1),[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1),[Audit Dataset Construction](https://arxiv.org/html/2609.00069#Sx4.p2.1),[Auditing Self\-Improving Agents](https://arxiv.org/html/2609.00069#Sx5.SSx2.p1.1)\.
- Zhanget al\.\(2025\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang,et al\.Aflow: automating agentic workflow generation\.InICLR,Vol\.2025,pp\. 34040–34077\.Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1),[Introduction](https://arxiv.org/html/2609.00069#Sx1.p4.1),[Introduction](https://arxiv.org/html/2609.00069#Sx1.p5.1),[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1),[Audit Dataset Construction](https://arxiv.org/html/2609.00069#Sx4.p2.1),[Auditing Self\-Improving Agents](https://arxiv.org/html/2609.00069#Sx5.SSx2.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: LLM agents are experiential learners\.InAAAI,Vol\.38,pp\. 19632–19642\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29936),[Link](https://ojs.aaai.org/index.php/AAAI/article/view/29936)Cited by:[Self\-Improving Agents and Harness Evolution](https://arxiv.org/html/2609.00069#Sx2.SSx1.p1.1)\.
- Zhaoet al\.\(2026a\)B\. Zhao, D\. Srikanth, Y\. Wu, and Z\. JiangSpecbench: measuring reward hacking in long\-horizon coding agents\.arXiv preprint arXiv:2605\.21384\.Cited by:[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p1.1)\.
- Zhaoet al\.\(2026b\)W\. Zhao, Y\. Zhang, Y\. Wang, Y\. Deng, Y\. Zhao, X\. Zhi, Y\. Huang, H\. He, W\. Che, B\. Qin, and T\. LiuOn safety risks in experience\-driven self\-evolving agents\.InFindings of the Association for Computational Linguistics,pp\. 42145–42169\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.2091),[Link](https://aclanthology.org/2026.findings-acl.2091/)Cited by:[Tampering and Evaluation Integrity](https://arxiv.org/html/2609.00069#Sx2.SSx2.p2.1)\.
- Zhouet al\.\(2026\)X\. Zhou, J\. Liu, L\. Li, R\. A\. Rossi, and X\. HuCounterfactual trace auditing of LLM agent skills\.arXiv preprint arXiv:2605\.11946\.External Links:2605\.11946Cited by:[Auditing Agent Trajectories and Code Artifacts](https://arxiv.org/html/2609.00069#Sx2.SSx3.p1.1)\.
- Zhugeet al\.\(2024\)M\. Zhuge, W\. Wang, L\. Kirsch, F\. Faccio, D\. Khizbullin, and J\. SchmidhuberGPTSwarm: language agents as optimizable graphs\.InICML,pp\. 62743–62767\.External Links:[Link](https://proceedings.mlr.press/v235/zhuge24a.html)Cited by:[Introduction](https://arxiv.org/html/2609.00069#Sx1.p1.1)\.