分析和缓解编程智能体中的成本低效行为

# Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents
Source: [https://arxiv.org/html/2609.30725](https://arxiv.org/html/2609.30725)
Nan Jiang†Shanchao LiangAnik DeyYi WuLin TanAffiliation:Purdue UniversityAffiliation:\{hu954, liang422, dey58, wu1827, lintan\}@purdue\.edu,†nan\.nathan\.jiang@gmail\.com

###### Abstract

Although effective, coding agents often incur substantial monetary costs\. Their recurring*cost\-inefficient behaviors*remain underexplored\. We conduct the first study of behavioral cost inefficiencies in coding agents, analyzing 1,200 trajectories from Claude Code and Mini\-SWE\-Agent across four configurations on SWE\-bench Verified\. We identify three cost\-inefficient behaviors:*subsumed retrieval*,*similar script generation*, and*test re\-execution*\. We then evaluate three mitigation strategies: structure\-aware retrieval, agent\-synthesized skills, and developer\-designed skills, over 10k trajectories on held\-out SWE\-bench Verified and Pro tasks\. Our main findings are: \(1\) The three behaviors affect 79\.00%–98\.00% of coding tasks and account for up to 22\.75% of task cost\. \(2\) Structure\-aware retrieval can introduce retrieval overhead and alter agent delegation, causing inconsistent improvements in retrieval efficiency and cost increases of up to 28\.14%\. \(3\) Agent\-synthesized skills tend to produce low\-level, trace\-specific guidance, limiting their effectiveness and generality\. \(4\) In contrast, developer\-designed skills provide high\-level, trace\-agnostic guidance, reducing cost by up to 41\.73%, roughly twice the maximum gain from agent\-synthesized skills\.

## 1Introduction

Coding agents have become increasingly capable, with systems such as Claude Code, Codex, and Cursor combining LLMs with tools and reusable skills\([Anthropic, 2025b](https://arxiv.org/html/2609.30725#bib.bib5);[OpenAI, 2025](https://arxiv.org/html/2609.30725#bib.bib6);[Anysphere, 2026](https://arxiv.org/html/2609.30725#bib.bib7)\)\. Yet, this capability comes at a steep cost\. Token spending can reach millions of dollars annually\([Bai et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib16)\), and Uber reportedly exhausted its annual AI coding budget within four months\([Angelo, 2026](https://arxiv.org/html/2609.30725#bib.bib3)\)\. The pressure also affects model providers: subscribers can consume tens of thousands of dollars of model usage on a monthly plan, leaving OpenAI losing money on subscriptions and Anthropic rate\-limiting Claude Code\([TechCrunch, 2025](https://arxiv.org/html/2609.30725#bib.bib61);[Zeff, 2025](https://arxiv.org/html/2609.30725#bib.bib62)\)\. In addition, higher spending does not reliably improve performance\([Kapoor et al\., 2024](https://arxiv.org/html/2609.30725#bib.bib33)\)\. These trends make the cost efficiency of coding agents a growing concern for both individual developers and enterprises\.

Despite the effectiveness of existing cost\-reduction approaches\([Wang et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib26);[Su et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib31);[Liu et al\., 2026a](https://arxiv.org/html/2609.30725#bib.bib12)\), recurring cost\-inefficient behaviors during agent execution remain largely undetected, accumulating substantial end\-to\-end cost\. Figure[1](https://arxiv.org/html/2609.30725#S1.F1)illustrates this problem using Claude Code on SWE\-bench Verified taskdjango\-13158\([Chowdhury et al\., 2024](https://arxiv.org/html/2609.30725#bib.bib1)\), which requires fixingQuerySet\.none\(\)to return an empty result on combined querysets\. Claude Code delegates code exploration to a subagent, retrieves additional context, applies a patch, and validates it with generated and repository tests\. Although the task passes, three cost\-inefficient behaviors account for 17\.25% of its cost111Behavior cost is computed from the input, output, and cache tokens consumed by each flagged action using the corresponding model token prices \(Appendix[A\.1\.2](https://arxiv.org/html/2609.30725#A1.SS1.SSS2)\)\.\. First, the main agent retrieves 20 lines fromquery\.pyand 85 lines fromcompiler\.pyalready covered by subagent retrievals\. Thissubsumed retrievalaccounts for 6\.96% of task cost\. Second, it generates four similar inline test scripts sharing 21 lines of logic with minor variations\. Thissimilar script generationconsumes another 7\.10%\. Third, it incorrectly executestest\_qs\_combinatorseight times in the same way without updating the patch, repeatedly hitting the same module error\. Thistest re\-executioncosts 3\.19%\. These observations motivate our first research question \(RQ1\):What cost\-inefficient behavior patterns occur in coding agents?

![Refer to caption](https://arxiv.org/html/2609.30725v1/Motivation-Example-Django-13158.png)Figure 1:Cost\-inefficient behaviors of Claude Code on SWE\-bench Verified taskdjango\-13158\.Among the detected behaviors, subsumed retrieval is the most prevalent\. A promising remedy is structure\-aware retrieval\([Ouyang et al\., 2025b](https://arxiv.org/html/2609.30725#bib.bib51);[Liu et al\., 2025b](https://arxiv.org/html/2609.30725#bib.bib52);[Chen et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib53)\), which enables direct access to complete code entities and explicit dependencies, reducing broad\-to\-narrow re\-reads that cause subsumed retrieval\. Although prior work reports improved localization accuracy and efficiency, whether these benefits generalize across agent architectures and translate into end\-to\-end cost savings remains unclear\. We therefore askRQ2:How does structure\-aware retrieval affect cost\-inefficient behaviors and the cost efficiency of coding agents?

While structure\-aware retrieval primarily targets retrieval\-related inefficiencies, agent skills\([Anthropic, 2025a](https://arxiv.org/html/2609.30725#bib.bib4)\)offer a lightweight intervention for addressing broader cost\-inefficient behaviors\. Such skills provide guidance that agents can apply adaptively throughout execution\. We therefore study two complementary approaches: agent\-synthesized skills and developer\-designed skills\. We askRQ3:How doagent\-synthesized skills\(SynSkills\) anddeveloper\-designed skills\(DevSkills\) affect cost\-inefficient behaviors and the cost efficiency of coding agents?

To answer these RQs, we study cost\-inefficient behaviors and their mitigation across four configurations: Claude Code \(CC\) with Sonnet 4\.6 \(S46\)\([Anthropic, 2026b](https://arxiv.org/html/2609.30725#bib.bib8)\), and Mini\-SWE\-Agent \(MSA\)\([Yang et al\., 2024](https://arxiv.org/html/2609.30725#bib.bib2)\)withS46, MiniMax\-M3 \(MM3\)\([MiniMax, 2026](https://arxiv.org/html/2609.30725#bib.bib9)\), or Qwen\-3\.5 Plus \(Q35\+\)\([Qwen Team, 2026](https://arxiv.org/html/2609.30725#bib.bib10)\)\. ForRQ1, we analyze 1,200 trajectories on 300 SWE\-bench Verified tasks\. ForRQ2–RQ3, we evaluate CodeGraph\([Mchenry, 2026](https://arxiv.org/html/2609.30725#bib.bib11)\)as a representative structure\-aware retrieval tool, alongside SynSkills and DevSkills, on 200 held\-out Verified and 100 SWE\-bench Pro tasks\([Deng et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib50)\), with repeated runs to account for execution stochasticity\.

Our paper makes the following contributions:

- •We conduct a systematic study of behavioral cost inefficiencies in coding agents, analyzing 1,200 trajectories from four agent configurations on SWE\-bench Verified\. Our findings include: - –We identify three recurring cost\-inefficient behaviors:subsumed retrieval \(SubRetrv\), retrieving code fully covered by prior retrievals;similar script generation \(SimScrpt\), regenerating highly similar scripts with only minor changes, andtest re\-execution \(ReTest\), rerunning identical tests without an updated patch\. Together, they cover 79\.00%–98\.00% of tasks and account for 6\.86%–22\.75% of task monetary cost\. - –SubRetrvis the most prevalent behavior, affecting up to 92\.33% of tasks and 11\.41% of cost\. SubRetrv in Claude Code mainly stems from re\-reading subagent\-retrieved code; in Mini\-SWE\-Agent, it arises from zoom\-in localization, low\-feedback editing, and long debugging loops\. - –SimScrptaffects up to 68\.00% of tasks and occurs 5\.91–9\.98×\\timesmore often in Mini\-SWE\-Agent than in Claude Code\. Built\-in guidance in Claude Code largely confines SimScrpt to ephemeral scripts, while Mini\-SWE\-Agent also regenerates similar files for testing and editing\. - –ReTestaffects 49\.67%–83\.00% of tasks and contributes up to 5\.39% of task cost\. It primarily arises from repository\-specific test knowledge gaps, test\-signal recovery, and progress stalls\.
- •We address a key gap in the evaluation of structure\-aware retrieval by examining whether it consistently improves retrieval efficiency and whether such gains translate into end\-to\-end cost savings\. - –Structure\-aware retrieval is not inherently cost\-efficient\.Depending on agent architecture and model, its verbose feedback and altered agent delegation can introduce retrieval overhead, yielding inconsistent SubRetrv reductions and end\-to\-end cost increases of up to 28\.14%\.
- •We design and evaluate a lightweight agent\-driven framework that synthesizes behavioral optimization skills from diagnosed cost\-inefficient behaviors\. - –Agent\-synthesized skills tend to produce low\-level, trace\-specific guidance, limiting their effectiveness and generality\.They yield no robust cost increase and reduce cost by at most 22\.32%, roughly half the maximum achieved by developer\-designed skills\.
- •We develop a compact set of behavior\-targeted optimization skills\.Our high\-level, trace\-agnostic developer\-designed skills achieve the largest cost savings, reducing task cost by 7\.88%–41\.73%, with maximum reductions of 38\.57%–88\.91% across the three behaviors\. These results highlight the value of human insights in agent behavioral optimization\.

## 2Study Setup

Research Questions\. ForRQ1, we analyze 1,200 trajectories across four agent configurations and 300 tasks\. We normalize trajectories and label actions using an author\-defined taxonomy with rule\-based and LLM\-assisted labeling \(Appendix[A\.1\.1](https://arxiv.org/html/2609.30725#A1.SS1.SSS1)\)\. We then define and manually calibrate detectors to identify three cost\-inefficient behaviors and examine their underlying mechanisms\. ForRQ2–RQ3, we evaluate three approaches for mitigating the cost\-inefficient behaviors, comparing each with its baseline on Pass@1, task monetary cost \(cost\), cost\-of\-pass \(CoP\)\([Erol et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib54)\), and behavior changes\. InRQ2, we equip agents with CodeGraph for structure\-aware retrieval\.CCaccesses this through an MCP tool, whileMSAuses command\-line wrappers\. For SynSkills inRQ3, we build a lightweight analysis agent using each configuration’s backbone model to distill corrective rules from its RQ1 trajectories and apply Trace2Skill\([Ni et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib34)\)to consolidate them into a configuration\-specific skill set\. For DevSkills, we manually derive a shared skill set from the RQ1 findings under two principles: \(1\) generalize across configurations and \(2\) mitigate cost\-inefficient behaviors without sacrificing task performance\. Following Trace2Skill, all skill sets are preloaded into the agent system prompt for evaluation \(Appendix[A\.4](https://arxiv.org/html/2609.30725#A1.SS4)\)\.

Agent Configuration\.To disentangle the effects of agent framework and backbone model on cost\-inefficient behaviors, we study two agent frameworks and vary the model within one of them\.CCis a proprietary, production\-grade framework with structuredRead/Edittools and subagent delegation, whereasMSAis an open\-source, single\-agent framework that interacts with repositories through shell commands\. We evaluate four agent configurations:CCwithS46as the main model and Haiku 4\.5 \(H45\) for subagents, andMSAwithS46,MM3, orQ35\+\. ComparingCCwithMSAS46captures framework\-associated differences under the same main model; comparing the threeMSAconfigurations reveals model\-associated differences under a fixed framework\.

Benchmarks & Task Split\.To separate behavior discovery from mitigation evaluation, we split the 500*SWE\-bench Verified*tasks within each repository by creation date: the earliest 300 tasks form theRQ1analysis set and the remaining 200 form the held\-out set*Verified\-200*forRQ2–RQ3, while preserving repository distribution\. For cross\-benchmark evaluation, we sample 100 tasks from*SWE\-bench Pro*\(*Pro\-100*\), covering all 11 repositories and favoring costlier tasks \(Appendix[A\.1\.3](https://arxiv.org/html/2609.30725#A1.SS1.SSS3)\)\. Together,*Verified\-200*and*Pro\-100*form the 300\-task mitigation evaluation set forRQ2–RQ3\.

Controlling Execution Randomness\.To account for run\-to\-run noise inRQ2–RQ3, we run eight baselines \(four configurations×\\timestwo benchmarks\) three times and estimate noise floors using the standard deviation of Pass@1 and coefficients of variation of cost and CoP\. For unreplicated approaches, we transfer the matching baseline’s noise floor, following the standard equal\-variance assumption that our mitigations do not alter the sampling and infrastructure nondeterminism driving run\-to\-run noise\([Ouyang et al\., 2025a](https://arxiv.org/html/2609.30725#bib.bib60)\)\. We validate this assumption on nine replicated approach cells spanning approaches, agent frameworks, and benchmarks; eight show no distinguishable noise amplification, while we use the larger observed noise for the exception \(Appendix[A\.1\.4](https://arxiv.org/html/2609.30725#A1.SS1.SSS4)\)\. We classify an effect as*robust*at one\-sidedp<0\.05p<0\.05, imposing a stricter threshold on single\-run effects\. Because behavior\-level metrics are noisier, we report their changes only as mechanistic evidence\.

## 3RQ1: Analyzing Cost\-Inefficient Behaviors in Coding Agents

Table 1:Prevalence, frequency, and cost contribution of cost\-inefficient behaviors across agent configurations\.Tasks: percentage of tasks exhibiting the behavior;Freq\.: average number of behaviors per task;Cost: average share of per\-task monetary cost attributable to the behavior\.CCMSAS46MSAMM3MSAQ35\+BehaviorsTasksFreq\.CostTasksFreq\.CostTasksFreq\.CostTasksFreq\.CostSubRetrv64\.33%2\.155\.01%87\.33%3\.458\.42%92\.33%5\.147\.88%89\.33%6\.3711\.41%SimScrpt20\.67%0\.431\.02%51\.33%2\.549\.57%68\.00%4\.298\.85%57\.67%3\.207\.91%ReTest49\.67%2\.090\.83%69\.00%2\.513\.17%83\.00%5\.295\.39%66\.00%2\.103\.43%Total79\.00%4\.676\.86%97\.33%8\.5021\.16%98\.00%14\.7222\.12%96\.67%11\.6822\.75%

We identifythreecost\-inefficient behaviors acrossinformation retrieval, script management, and solution validationin coding agent execution\. \(1\)Subsumed Retrieval:retrieving code fully covered by previous retrievals\. \(2\)Similar Script Generation:regenerating highly similar scripts with minor changes\. \(3\)Test Re\-Execution:re\-executing the identical tests without updated patches\.

Figure 2:Temporal distribution of the cost\-inefficient behaviors\. Bold curves aggregate configurations; thin curves show individual ones\.Table[1](https://arxiv.org/html/2609.30725#S3.T1)reveals that the three behaviors are both prevalent and costly, collectively affecting at least 79\.00% of tasks and accounting for up to 22\.75% of task cost, andCCconsistently exhibits the lowest overall level of inefficiency \(Appendix[A\.2\.1](https://arxiv.org/html/2609.30725#A1.SS2.SSS1)\)\. Figure[2](https://arxiv.org/html/2609.30725#S3.F2)further shows their temporal patterns: SubRetrv concentrates in the first half of the trajectory, coinciding with the retrieval\-heavy stage; ReTest shifts toward the middle\-to\-late stages, where the trajectory becomes more test\-heavy for patch validation; and SimScrpt is distributed more broadly rather than confined to a specific phase\. We next formalize each behavior and analyze its detection, severity, and underlying mechanisms\.

### 3\.1Subsumed Retrieval

Detection and Severity\.For each retrievalbbreturning at least five non\-empty lines of content, we scan preceding retrieval actions backward and identify the first actionaawhose returned context fully covers that ofbb\. We treat\(a,b\)\(a,b\)as a SubRetrv pair and flagbbas one SubRetrv action\.*Table[1](https://arxiv.org/html/2609.30725#S3.T1)shows that SubRetrv is the most prevalent cost\-inefficient behavior, affecting 64\.33–92\.33% of tasks, occurring 2\.15–6\.37 times per task, and accounting for 5\.01–11\.41% of task cost\.*

Table 2:SubRetrv scenarios\. Pairs are assigned by top\-to\-bottom priority\.SubRetrv Pair ShareOccurrence ScenarioCCMSAS46MSAMM3MSAQ35\+Cross\-Agent SubRetrv50\.15%———Patch\-Adjacent SubRetrv8\.67%26\.47%17\.96%19\.56%Within\-Sequence SubRetrv26\.47%30\.05%16\.54%46\.91%Long\-Distance SubRetrv11\.30%26\.57%52\.40%24\.69%Other3\.41%16\.91%13\.10%8\.84%Underlying Mechanisms\.To study when and why SubRetrv occurs and how its mechanisms differ betweenCCandMSA, we classify each SubRetrv pair into different scenarios in Table[2](https://arxiv.org/html/2609.30725#S3.T2)\. The results reveal substantial differences across agent architectures\.Cross\-Agent SubRetrv, where a main\-agent retrieval is subsumed by a previous subagent retrieval, is unique toCCand accounts for 50\.15% of its SubRetrv\.CCoffloads exploration to subagents, which return summaries rather than the retrieved code\. So the main agent may re\-retrieve the same region when needed later, as illustrated in Figure[1](https://arxiv.org/html/2609.30725#S1.F1)\.

Patch\-Adjacent SubRetrvoccurs when the SubRetrv immediately precedes or follows a patch\. It is more common inMSA\(17\.96–26\.47%\) than inCC\(8\.67%\) due to different editing interfaces\.MSA’s shell\-based editing, such assed \-ior generated scripts, may provide little feedback or fail silently, prompting agents to re\-read code for localization or post\-edit inspection\. In contrast,CC’s retrieval tools provide line numbers, while its editing tools report detailed edit outcomes, including the updated code, reducing such scenarios\.

Within\-Sequence SubRetrvoccurs when both retrievals in a SubRetrv pair fall within the same uninterrupted retrieval sequence\. It typically follows a locate\-then\-zoom pattern, where the agent first reads a broad region and then retrieves a covered subregion for precise localization\. This is common inMSA, where commands such ascatomit line numbers and often trigger follow\-upgrep\-norsed\-ncalls;CC’sReadalready returns line\-numbered code, reducing this need\.

Long\-Distance SubRetrvoccurs when both retrievals in a SubRetrv pair are separated by at least 10 steps\. Manual inspection shows that it typically reflects longer debugging loops, where earlier context becomes less salient and is re\-retrieved\. It is most prominent inMSAMM3\(52\.40%\), which is consistent with its longest average trajectory length \(77\.99 steps vs\. 29\.99–58\.06 for others\)\.

Finding 1\.Subsumed Retrievalis the most prevalent behavior, affecting 64\.33–92\.33% of tasks and contributing 5\.01–11\.41% of task cost\. Its causes are architecture\-dependent: cross\-agent SubRetrv dominates inCC, whileMSAmainly suffers from low\-feedback editing, zoom\-in localization, and long debugging loops\.

### 3\.2Similar Script Generation

Behavior Detection\.We define SimScrpt as regenerating near\-duplicate scripts with minor differences rather than editing existing scripts, wasting output tokens and forgoing artifact reuse\. We extract*ephemeral scripts*\(e\.g\.,python\-cand heredocs\) and*file\-based scripts*\(\.pyfiles\) from agent actions, remove blank and comment\-only lines, and retain scripts with at least five lines\. For each generation actionbb, we identify the nearest preceding actionaawhose script has line\-level Jaccard similarity of at least0\.600\.60222We select0\.600\.60by manually inspecting 20 SimScrpt pairs per configuration\. Lower thresholds often match generic scaffolding, while0\.600\.60retains highly similar task\-specific logic\.\. We treat\(a,b\)\(a,b\)as a SimScrpt pair and countbbas one occurrence\.

Prevalence and Severity\.Table[1](https://arxiv.org/html/2609.30725#S3.T1)shows that SimiScript affects up to 68\.00% of tasks and contributes up to 9\.57% of task cost\. Interestingly, SimScrpt is substantially more severe inMSA, affecting 51\.33–68\.00% of tasks and contributing 7\.91–9\.57% of task cost, versus 20\.67% and 1\.02% inCC\. It also occurs 5\.91–9\.98×\\timesmore frequently inMSA\(2\.54–4\.29 vs\. 0\.43\)\. Figure[5](https://arxiv.org/html/2609.30725#S3.F5)further reveals a pronounced high\-frequency tail inMSA: 42–66 tasks per configuration exhibit at least six SimScrpt occurrences, compared with only two inCC\.

Figure 3:Distribution of SimScrpt occurrences per task\.Figure 4:Distribution of SimScrpt pair step distances\.Figure 5:Functions of ephemeral and file\-based SimScrpt scripts\.
Underlying Mechanisms\.*When and why SimScrpt occurs\.*SimScrpt arises during issue reproduction, code editing, and patch validation, explaining its broad distribution in Figure[2](https://arxiv.org/html/2609.30725#S3.F2)\. Figure[5](https://arxiv.org/html/2609.30725#S3.F5)further reveals two patterns: 39\.89–48\.84% of occurrences immediately follow a similar script, typically reflecting minor revisions with full\-script regeneration, while 21\.71–34\.60% occur after at least six intervening steps\. The underlying causes differ by script form\. Ephemeral scripts leave no reusable artifact, requiring agents to regenerate similar code when needed again\. File\-based scripts persist on disk, but become less salient to the agent as its trajectory grows\. Reusing them requires re\-reading files for precise editing, while shell\-based edits such assed\-iare error\-prone, as discussed in[Patch\-Adjacent SubRetrv](https://arxiv.org/html/2609.30725#S3.SS1)\. Regenerating a near\-duplicate script can therefore become the easier immediate choice\.

*Ephemeral vs\. file\-based SimScrpt scripts\.*To understand their differences, we classify SimScrpt scripts by their functionalities using abstract syntax tree \(AST\) analysis \(Figure[5](https://arxiv.org/html/2609.30725#S3.F5)\)\. A*patch script*modifies source files; an*assertion test*performs explicit pass/fail validation; and an*inspection probe*queries or prints runtime information, such as object values, attributes, or program state\. Scripts are assigned by priority: patch, test, then probe\. Ephemeral SimScrpt scripts overwhelmingly consist of inspection probes \(81\.03–93\.45%\)\. Agents commonly usepython\-cand heredocs as disposable probes: they generate a script, inspect certain output, and discard it, then regenerate a similar probe when the information is needed again\. File\-based SimScrpt scripts are more functionally diverse: inspection probes decrease, while assertion tests and, inMSA, patch scripts account for larger shares\. Because these files persist and encode reusable validation or editing logic, regenerating them represents a clearer missed opportunity for artifact reuse\.

*Why SimScrpt script forms differ across architectures\.*Figures[5](https://arxiv.org/html/2609.30725#S3.F5)and[5](https://arxiv.org/html/2609.30725#S3.F5)show thatCCprimarily exhibits ephemeral SimScrpt, whereasMSAexhibits both script forms\. This difference is consistent withCC’s built\-in instruction: “NEVER create files unless they’re absolutely necessary for achieving your goal\.” This discourages persistent script creation and shifts toward ephemeral generation, whereasMSAhas no such constraint\. Patch scripts are rare inCCbecause its built\-inEditandWritetools directly support modification action, whereasMSA’s poor feedback of shell\-based editing more often triggers agents to generate their own editing scripts\.

Finding 2\.Similar Script Generationaffects up to 68\.00% of tasks and contributes up to 9\.57% of task cost, occurring 5\.91–9\.98×\\timesmore frequently inMSAthan inCC\.CC’s built\-in guidance largely confines SimScrpt to ephemeral scripts, whereasMSAalso repeatedly regenerates file\-based testing and editing scripts\. These patterns suggest that an ideal agent should persist reusable logic and use inline scripts for simple one\-off probes\.

### 3\.3Test Re\-Execution

Detection and Severity\.We partition trajectories into inter\-patch windows because a new patch may justify rerunning tests, whereas repeated executions without a patch update validate the same code state\. Within each window, we group executions by the executed tests, retain the final execution as potentially decision\-relevant, and flag preceding ones as ReTest\.*Table[1](https://arxiv.org/html/2609.30725#S3.T1)shows that ReTest affects 49\.67–83\.00% of tasks and contributes up to 5\.39% of task cost*\.MSAMM3has the highest prevalence and frequency, averaging 5\.29 occurrences per task, over twice that of any other configuration\.

Underlying Mechanisms\.We identify three common causes of ReTest\. \(1\)*Repository\-specific test knowledge gaps*arise when agents are unfamiliar with the proper test runner or configuration, causing repeated failures\. \(2\)*Test\-signal recovery*occurs when test output is truncated, ambiguous, or poorly captured by the agent, prompting reruns to recover debugging information\. \(3\)*Progress stalls*arise when agents enter long reasoning loops without updating the patch, repeatedly reconsidering the same diagnosis and rerunning tests despite sufficient prior signals\. Such behavior reflects stalled decision\-making without advancing the solution\.

Finding 3\.Test Re\-Executionaffects 49\.67–83\.00% of tasks and accounts for up to 5\.39% of task cost, withMSAMM3exhibiting the highest prevalence and frequency\. ReTest commonly arises from repository\-specific test knowledge gaps, test\-signal recovery, and progress stalls\.

More detection details, analysis, and examples for RQ1 are provided in Appendix[A\.2](https://arxiv.org/html/2609.30725#A1.SS2)\.

Table 3:Effects of the three approaches on Pass@1, average cost, CoP, and the occurrence of identified behaviors\. Subscripts show changes from baseline\. For Pass@1, cost, and CoP,green/redindicate robust improvements/regressions, whilegrayindicates changes within noise; for behaviors, green/red indicate fewer/more instances\.Verified\-200Pro\-100ApproachMetricCCMSAS46MSAMM3MSAQ35\+CCMSAS46MSAMM3MSAQ35\+CodeGraphPass@1 \(pp\)74\.17\-0\.8372\.50\+0\.0075\.50\+0\.3372\.50\+5\.3355\.17\+1\.9251\.72\+1\.9257\.47\-1\.1539\.08\-3\.45Cost \($\)0\.596\+8\.30%0\.619\-3\.03%0\.591\+28\.14%0\.112\-3\.29%1\.061\+12\.19%1\.132\-0\.25%0\.985\+8\.39%0\.150\+12\.99%CoP0\.804\+10\.22%0\.854\-3\.04%0\.782\+27\.52%0\.155\-10\.63%1\.923\+8\.69%2\.188\-4\.32%1\.715\+10\.20%0\.388\+23\.20%SubRetrv0\.16\-84\.18%2\.87\-25\.33%6\.36\+18\.01%4\.65\-24\.25%0\.38\-75\.72%2\.90\+3\.33%6\.34\+34\.74%4\.51\+13\.01%SimScrpt0\.26\-25\.85%2\.90\-17\.01%5\.90\+28\.49%2\.68\-39\.61%0\.10\-0\.21%1\.64\+63\.26%1\.99\-1\.33%0\.79\-53\.46%ReTest1\.75\+13\.85%2\.05\-8\.07%5\.55\+7\.34%2\.21\+0\.30%0\.56\-0\.89%2\.13\+11\.27%4\.49\+25\.45%0\.78\+6\.90%SynSkillsPass@1 \(pp\)74\.50\-0\.5072\.00\-0\.5075\.50\+0\.3369\.67\+2\.5055\.00\+0\.6752\.00\+0\.0062\.00\+2\.0046\.00\+1\.00Cost \($\)0\.541\-1\.38%0\.496\-22\.32%0\.462\+0\.15%0\.115\-0\.70%0\.826\-8\.86%1\.006\-6\.13%0\.766\-11\.90%0\.132\-2\.22%CoP0\.726\-0\.12%0\.689\-21\.79%0\.611\-0\.33%0\.166\-4\.40%1\.503\-9\.48%1\.936\-6\.40%1\.235\-15\.09%0\.286\-4\.51%SubRetrv0\.56\-44\.62%3\.40\-11\.38%4\.17\-22\.56%5\.83\-5\.05%0\.98\-34\.31%2\.67\+2\.50%3\.76\-14\.29%3\.86\+3\.21%SimScrpt0\.20\-41\.85%1\.92\-45\.10%3\.98\-13\.43%3\.66\-17\.63%0\.08\-11\.28%1\.60\+79\.32%1\.19\-35\.44%1\.41\-5\.87%ReTest1\.75\+13\.20%1\.46\-34\.53%4\.67\-9\.86%2\.03\-7\.58%0\.62\+25\.44%1\.22\-26\.88%3\.19\+0\.84%0\.78\+22\.80%DevSkillsPass@1 \(pp\)73\.50\-1\.5073\.00\+0\.5073\.00\-2\.1769\.83\+2\.6752\.00\-2\.3352\.00\+0\.0062\.00\+2\.0046\.00\+1\.00Cost \($\)0\.473\-13\.94%0\.372\-41\.73%0\.387\-16\.03%0\.103\-11\.34%0\.894\-1\.51%0\.733\-33\.00%0\.801\-7\.88%0\.140\+3\.57%CoP0\.643\-11\.83%0\.510\-42\.13%0\.530\-13\.61%0\.148\-14\.83%1\.718\+3\.58%1\.409\-33\.19%1\.291\-11\.21%0\.305\+1\.28%SubRetrv0\.35\-64\.89%1\.86\-51\.52%4\.05\-24\.82%5\.08\-17\.25%0\.60\-59\.78%1\.47\-43\.57%4\.02\-8\.36%4\.23\+12\.92%SimScrpt0\.15\-54\.93%1\.18\-66\.17%4\.02\-12\.45%4\.42\-0\.49%0\.01\-88\.91%0\.55\-38\.36%1\.85\+0\.36%1\.43\-4\.02%ReTest1\.47\-4\.75%1\.37\-38\.57%3\.62\-30\.05%2\.04\-7\.13%0\.72\+45\.68%1\.28\-23\.29%3\.00\-5\.16%0\.66\+3\.71%

## 4Mitigating Cost\-Inefficient Behaviors in Coding Agents

This section evaluates the effectiveness of the three mitigation approaches in optimizing cost efficiency and the identified cost\-inefficient behaviors \(Table[3](https://arxiv.org/html/2609.30725#S3.T3)\)\. Figure[6](https://arxiv.org/html/2609.30725#S4.F6)further relates changes in behavior\-attributed cost to task cost\. We first compare the three approaches across all settings, then examine each in depth\.

### 4\.1Cross\-Approach Comparison

We compare all three approaches across agent configurations and benchmarks\. Three patterns emerge\.First, mitigation effectiveness generally increases from CodeGraph to agent\-synthesized skills to developer\-designed skills\.CodeGraph reduces SubRetrv inconsistently and even increases cost robustly in four of eight settings\. SynSkills achieve moderate savings, whereas DevSkills produce the broadest reductions in both task cost and diagnosed behaviors\.Second, mitigation benefits depend on agent architecture and benchmark\.Improvements are generally larger underMSA, whileCC’s system prompt and tool abstractions already suppress several inefficient behaviors, leaving less room for improvement\. All approaches also perform better overall on Verified\-200 than Pro\-100 in identified behaviors\. For skill\-based approaches, this partly reflects a generalization challenge, as the skills are derived from Verified trajectories and applied to Pro\.Third, task cost changes are amplified relative to the changes in behavior\-attributed cost\.In Figure[6](https://arxiv.org/html/2609.30725#S4.F6), the fitted slopes are 2\.83 on Verified\-200 and 1\.33 on Pro\-100\. Two effects can amplify the savings\. \(1\) Preventing an inefficient action shortens the trajectory, reducing repeated cache\-reading of earlier context in later LLM calls\. \(2\) It can also prevent additional unflagged actions triggered by certain cost\-inefficient behaviors\. For example, eliminating a ReTest can also avoid subsequent reasoning that interprets its output or diagnoses the same failure\.

Figure 6:Changes in average task cost versus behavior\-attributed cost, both relative to baseline task cost\. The shaded quadrant indicates joint improvement, and the lines show the fitted linear trends\.
### 4\.2RQ2: Effectiveness of Structure\-aware Retrieval

Table[3](https://arxiv.org/html/2609.30725#S3.T3)shows thatCodeGraph neither consistently reduces SubRetrv nor improves task cost efficiency\.333CodeGraph cannot run on 13 Pro\-100 tasks \(Appendix[A\.3\.1](https://arxiv.org/html/2609.30725#A1.SS3.SSS1)\)\. Its Pro\-100 results are therefore measured and compared on the remaining 87 tasks\.UnderCC, SubRetrv drops by over 75% on both benchmarks, yet cost rises by 8\.30% on Verified\-200 and robustly by 12\.19% on Pro\-100\. UnderMSA, SubRetrv increases in four of six settings\. Overall, CodeGraph yields no robust cost reduction but four robust increases of 8\.39–28\.14%, and Pass@1 remains largely unchanged except forMSAQ35\+on Verified\-200 \(\+5\.33 pp\)\. Thus, substantial SubRetrv reduction does not guarantee lower end\-to\-end cost\.

Table 4:Effects of CodeGraph on average task token usage, average output tokens per CodeGraph and other retrieval call, and LLM/tool calls\. Subscripts show percentage changes from the baselines\.Verified\-200Pro\-100MetricCCMSAS46MSAMM3MSAQ35\+CCMSAS46MSAMM3MSAQ35\+Task tokens \(M\)0\.91\-1\.70%1\.09\-0\.37%4\.13\+29\.29%1\.63\-1\.15%1\.70\-0\.36%2\.20\-0\.32%7\.28\+9\.11%2\.39\+13\.39%Output tokens / call777\+140\.96%233\+5\.98%251\+5\.94%292\-3\.39%955\+55\.74%354\+9\.81%302\-5\.64%370\-5\.56%CodeGraph5,1998761,2051,1225,2731,1111,0071,111Other retrieval313258277350643395789541LLM calls19\.91\-19\.72%40\.79\-7\.22%95\.39\+13\.60%60\.15\-2\.94%29\.18\-23\.21%56\.17\-5\.27%143\.91\+8\.20%71\.05\+5\.68%Main agent \(S46\)19\.91\-0\.42%–––29\.18\+8\.17%–––Subagent \(H45\)0\.00\-100\.00%–––0\.00\-100\.00%–––Tool calls19\.43\-26\.61%42\.98\-8\.44%94\.66\+12\.97%59\.57\-3\.87%32\.74\-31\.47%66\.94\-5\.63%143\.37\+8\.07%71\.52\+6\.39%CodeGraph2\.235\.076\.756\.313\.608\.136\.335\.65Other retrieval7\.16\-53\.66%14\.20\-34\.13%35\.38\-2\.92%22\.30\-23\.22%15\.68\-53\.68%35\.34\-23\.87%75\.01\+3\.16%39\.34\-3\.44%

Table[4](https://arxiv.org/html/2609.30725#S4.T4)explains why\.ForCC, verbose CodeGraph feedback and reduced use of cheaper subagents offset the savings from fewer SubRetrv actions\. CodeGraph cuts LLM and tool calls by 19\.72–23\.21% and 26\.61–31\.47%, respectively, but each query returns 8\.2–16\.6×\\timesmore tokens than other retrieval actions\. Total token usage therefore remains nearly unchanged\. Meanwhile,H45subagent calls drop from 4\.81 and 11\.03 per task to zero on both benchmarks, whileS46main\-agent calls change only slightly \(−\-0\.42% and \+8\.17%\)\. Thus, roughly the same token volume shifts toS46, whose token price is 3×\\timesthat ofH45, increasingCC’s cost by 8\.30% on Verified\-200 and robustly by 12\.19% on Pro\-100\.

ForMSA, cost rises because agents use CodeGraph additively, stacking CodeGraph queries and their verbose feedback on top of ordinary retrieval rather than replacing it\.ForMSAMM3on both benchmarks andMSAQ35\+on Pro\-100, ordinary retrieval remains nearly unchanged while agents add 5\.65–6\.75 CodeGraph calls per task; these extra calls and their verbose outputs raise total token usage by 9\.11–29\.29% and cost robustly by 8\.39–28\.14%\. In the remaining settings, agents do replace ordinary retrieval with CodeGraph queries: ordinary retrieval drops by over 20%, but the verbose CodeGraph feedback keeps total token usage nearly unchanged, leaving cost savings indistinguishable from the baseline\.

Finding 4\.Structure\-aware retrieval, as exemplified by CodeGraph, is not inherently cost\-efficient\. Verbose feedback and tool integration can add retrieval overhead and alter agent delegation, leading to inconsistent SubRetrv reduction and higher cost\. It should therefore be evaluated with explicit attention to feedback volume, tool integration, and model orchestration\.

More details on CodeGraph integration and usage are provided in Appendix[A\.3](https://arxiv.org/html/2609.30725#A1.SS3)\.

### 4\.3RQ3: Agent\-Synthesized vs\. Developer\-Designed Skills

Unlike CodeGraph, which changes how agents access code and targets only retrieval inefficiencies, skills can address broader cost\-inefficient behaviors\. SynSkills are configuration\-specific sets of 23–41 operational rules\. They target the three cost\-inefficient behaviors through concrete, scenario\-specific instructions, such as handling failed string substitutions, shell quoting issues, and permission errors\. In contrast, DevSkills are seven general behavioral principles shared by all configurations\. They require agents to state a concrete hypothesis before retrieval, reuse available context, and justify necessary re\-reads \(SubRetrv\); to persist and revise scripts instead of creating similar variants \(SimScrpt\); and to understand the test harness, capture test output, and rerun tests only when code changes or new evidence warrant it \(ReTest\)\.

DevSkills are substantially more effective and consistent than SynSkills\. SynSkills robustly reduce average cost in three of eight settings by 8\.86–22\.32%, with no robust cost increase or Pass@1 degradation \(Table[3](https://arxiv.org/html/2609.30725#S3.T3)\)\. In comparison, DevSkills robustly reduce cost in six settings by 7\.88–41\.73%, nearly twice the maximum saving from SynSkills, with only one small Pass@1 drop \(−\-1\.50 pp forCCon Verified\-200\)\. The behavior\-level results show the same pattern\. For example, DevSkills reduce all three behaviors inMSAS46on both benchmarks, including SubRetrv by 51\.52% and 43\.57%, compared with 11\.38% and a 2\.50% increase under SynSkills\.

One possible mechanism is that DevSkills encode higher\-level, trace\-agnostic guidance, whereas SynSkills retain low\-level, trace\-specific instructions\.For example, to mitigate SimScrpt, theMSAS46SynSkills specify when to use ephemeral scripts or overwrite scripts, and how to handle non\-ASCII shell content\. Such rules depend on specific tasks and repositories, limiting generalization\. In contrast, DevSkills cover the same cases with one principle: persist and reuse artifacts rather than regenerate similar variants\. The principle applies to any task, which helps explain the stronger and more consistent effectiveness of DevSkills and the value of human abstraction in behavioral optimization\.

Finding 5\.Agent\-synthesized skills tend to produce low\-level, trace\-specific guidance, limiting their effectiveness and generality\. In contrast, developer\-designed skills provide high\-level, trace\-agnostic guidance and achieve broader cost savings, robustly reducing cost by 7\.88–41\.73% in six of eight settings\.

More analysis and specific skill content for RQ3 are in Appendix[A\.4](https://arxiv.org/html/2609.30725#A1.SS4)\.

## 5Related Work

##### Agent Trajectory Analysis\.

Observability platforms support execution tracing and cost monitoring\([LangChain, 2026](https://arxiv.org/html/2609.30725#bib.bib13);[Langfuse, 2026](https://arxiv.org/html/2609.30725#bib.bib14);[AgentOps, 2026](https://arxiv.org/html/2609.30725#bib.bib15)\)\. Trajectory analyses further examine how agents approach tasks, where execution goes wrong, and how costs accumulate\([Ceka et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib18);[Bouzenia and Pradel, 2025](https://arxiv.org/html/2609.30725#bib.bib19);[Majgaonkar et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib20);[Zhang et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib21);[Kim et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib22);[Bai et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib16);[Salim et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib17)\), identifying issues such as repeated actions and loss of coherence\([Liu et al\., 2026a](https://arxiv.org/html/2609.30725#bib.bib12);[Kim et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib48)\)\. However, how recurring inefficient behaviors contribute to agent costs, why they occur, and whether they can be mitigated remain underexplored\. To address this gap, we define and detect three cost\-inefficient behaviors and analyze their prevalence, monetary impact, and underlying mechanisms across coding agents\.

##### Agent Cost Reduction\.

Existing approaches improve agent cost efficiency through four strategies\. Context compression and interaction reduction decrease the token usage of each step and the number of interactions\([Gao and Peng, 2025](https://arxiv.org/html/2609.30725#bib.bib23);[Pan et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib24);[Nayab et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib25);[Wang et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib26);[Luk et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib56);[Xiao et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib27);[Li et al\., 2026c](https://arxiv.org/html/2609.30725#bib.bib28);[Jia et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib47)\)\. Agent orchestration allocates work across suitable models and tools\([Shekhar et al\., 2024](https://arxiv.org/html/2609.30725#bib.bib29);[Wang et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib30);[Su et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib31);[Liu et al\., 2025a](https://arxiv.org/html/2609.30725#bib.bib32);[Ong et al\., 2024](https://arxiv.org/html/2609.30725#bib.bib49)\)\. Runtime supervision intervenes when agents make errors, enter loops, or deviate from expected workflows\([Lin et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib55);[Liu et al\., 2026a](https://arxiv.org/html/2609.30725#bib.bib12)\)\. However, their effects on specific cost\-inefficient behaviors remain underexplored\. We complement them by using diagnosed behaviors to guide optimizations and evaluating their effects on these behaviors, cost, and task performance\. A fourth strategy improves agent toolsets\. Structure\-aware retrieval, for example, uses code graphs to guide repository navigation\([Ouyang et al\., 2025b](https://arxiv.org/html/2609.30725#bib.bib51);[Liu et al\., 2025b](https://arxiv.org/html/2609.30725#bib.bib52)\)\. Despite reported gains in localization accuracy and efficiency\([Chen et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib53)\), whether these gains consistently translate into end\-to\-end cost savings remains unclear\. We therefore examine its effects on retrieval efficiency and task cost across agent configurations\.

##### Agent Skills\.

Agent skills\([Anthropic, 2025a](https://arxiv.org/html/2609.30725#bib.bib4)\)provide reusable instructions and optional resources to guide task executions\. Recent systems build skills from execution experience\([Ni et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib34);[Alzubi et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib35);[Ma et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib36);[Yang et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib41);[Xia et al\., 2026b](https://arxiv.org/html/2609.30725#bib.bib44);[Liu et al\., 2026b](https://arxiv.org/html/2609.30725#bib.bib38)\), incorporating reinforcement learning\([Xia et al\., 2026a](https://arxiv.org/html/2609.30725#bib.bib37);[Li et al\., 2026b](https://arxiv.org/html/2609.30725#bib.bib45)\), across web, multimodal, and generalist\-agent settings\([Zheng et al\., 2025](https://arxiv.org/html/2609.30725#bib.bib40);[Jiang et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib42);[Zhou et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib43)\)\. Prior work evaluates skill effectiveness\([Li et al\., 2026a](https://arxiv.org/html/2609.30725#bib.bib46)\)and optimizes skills for both task success and inference cost\([Gong et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib39)\)\. We derive skills from diagnosed cost\-inefficient behaviors and compare our agent\-synthesized and developer\-designed skills in terms of agent cost efficiency\.

## 6Threats to Validity

Agent and Task Coverage\.Budget limits our study to four agent configurations, SWE\-bench Verified, and 100 SWE\-bench Pro tasks\. Behaviors and intervention effects may differ elsewhere\.Behavior Identification\.Parsing, labeling, and detection mechanisms may introduce detection errors despite explicit implementations and manual calibration\.Cost and Outcome Measurement\.We measure LLM inference monetary cost, excluding execution time and hardware costs\. Pass@1 may miss regressions beyond test satisfaction\.Execution Randomness\.We triplicate baselines and nine intervention settings; budget limits others to single runs\. Robustness estimates rely on baseline\-derived variability, which may not fully establish statistical significance\.Structure\-aware Retrieval\.CodeGraph’s popularity and robust structural retrieval implementations motivate its selection, but this may limit generalization to other tools\.Skill Design and Scope\.Findings depend on our agent\-synthesis pipeline and developer\-skill design\. Script\-based or other skill formats may differ from our prompt\-based interventions\.

## 7Conclusions & Future Work

We analyze Claude Code and Mini\-SWE\-Agent trajectories on SWE\-bench Verified, identifying three prevalent cost\-inefficient behaviors: subsumed retrieval, similar script generation, and test re\-execution\. Evaluating three mitigation approaches on Verified and Pro shows that structure\-aware retrieval can increase cost due to its verbose feedback and altered agent delegation\. Agent\-synthesized skills offer low\-level, trace\-specific guidance and moderate savings, while developer\-designed skills provide high\-level, trace\-agnostic guidance and achieve the strongest and most consistent savings\. Future work will extend this analysis to longer\-horizon, interactive tasks and more diverse agents, examining inefficiencies beyond monetary cost\. One could also use behavior diagnoses to guide optimizations and generate more efficient trajectories for agent training\.

## 8AI Use Statement

##### AI within our experimental methods\.

Analyzing and mitigating the cost\-inefficient behaviors of the coding agents is the object of our study, and their benchmark execution trajectories are used as our experimental data\. We combine rule\-based parsing with LLM\-based action labeling under an author\-defined taxonomy \(Section[2](https://arxiv.org/html/2609.30725#S2)\)\. The authors design the labeling rules and thresholds, manually calibrate them, and examine their application to agent trajectories\. In RQ3 \(Section[4\.3](https://arxiv.org/html/2609.30725#S4.SS3)\), an LLM\-based pipeline generates agent\-synthesized skills as a mitigating approach for evaluation\.

##### AI assistance in research and writing\.

We use generative AI assistants to help debug and refactor author\-written code, refine manuscript structure and wording, and improve table and figure layouts\. The authors independently design the research questions, metrics, detection rules, analysis, findings, and conclusions\. We review the AI\-assisted work and take responsibility for the final manuscript, code, analysis, and results\.

## References

- AgentOps \(2026\)AgentOpsAgentOps documentation\.Note:[https://docs\.agentops\.ai/v2/introduction](https://docs.agentops.ai/v2/introduction)Accessed: 2026\-08Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Alzubiet al\.\(2026\)S\. Alzubi, N\. Provenzano, J\. Bingham, W\. Chen, and T\. VuEvoSkill: automated skill discovery for multi\-agent systems\.External Links:2603\.02766,[Link](https://arxiv.org/abs/2603.02766)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Angelo \(2026\)J\. AngeloUber burned through its entire 2026 AI budget in four months\. now its COO is questioning whether it’s worth it\.Note:[https://fortune\.com/2026/05/26/uber\-coo\-ai\-spending\-tokens\-claude\-code/](https://fortune.com/2026/05/26/uber-coo-ai-spending-tokens-claude-code/)Fortune\. Accessed: 2026\-08Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p1.1)\.
- Anthropic \(2025a\)AnthropicAgent skills\.Note:[https://platform\.claude\.com/docs/en/agents\-and\-tools/agent\-skills/overview](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview)Accessed: 2026\-08Cited by:[§A\.4](https://arxiv.org/html/2609.30725#A1.SS4.p2.1),[§1](https://arxiv.org/html/2609.30725#S1.p4.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Anthropic \(2025b\)AnthropicClaude Code: anthropic’s agentic coding tool\.Note:[https://claude\.com/product/claude\-code](https://claude.com/product/claude-code)Accessed: 2026\-08Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p1.1)\.
- Anthropic \(2026a\)AnthropicClaude Code v2\.1\.169\.Note:[https://www\.npmjs\.com/package/@anthropic\-ai/claude\-code/v/2\.1\.169](https://www.npmjs.com/package/@anthropic-ai/claude-code/v/2.1.169)npm package; the system prompt and tool descriptions are shipped with the package\. Accessed: 2026\-09Cited by:[§A\.1](https://arxiv.org/html/2609.30725#A1.SS1.p2.1),[§A\.2\.1](https://arxiv.org/html/2609.30725#A1.SS2.SSS1.p1.1)\.
- Anthropic \(2026b\)AnthropicIntroducing Claude Sonnet 4\.6\.Note:[https://www\.anthropic\.com/news/claude\-sonnet\-4\-6](https://www.anthropic.com/news/claude-sonnet-4-6)Accessed: 2026\-08Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p5.1)\.
- Anysphere \(2026\)AnysphereCursor: the AI code editor\.Note:[https://cursor\.com](https://cursor.com/)Accessed: 2026\-08Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p1.1)\.
- Baiet al\.\(2026\)L\. Bai, Z\. Huang, X\. Wang, J\. Sun, R\. Mihalcea, E\. Brynjolfsson, A\. Pentland, and J\. PeiHow do AI agents spend your money? analyzing and predicting token consumption in agentic coding tasks\.External Links:2604\.22750,[Link](https://arxiv.org/abs/2604.22750)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p1.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Bouzenia and Pradel \(2025\)I\. Bouzenia and M\. PradelUnderstanding software engineering agents: a study of thought\-action\-result trajectories\.In2025 IEEE/ACM 40th International Conference on Automated Software Engineering \(ASE\),pp\. 2846–2857\.External Links:[Document](https://dx.doi.org/10.1109/ASE63991.2025.00234),[Link](https://doi.org/10.1109/ASE63991.2025.00234)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Cekaet al\.\(2025\)I\. Ceka, H\. Mitchell, S\. Pujar, L\. Buratti, S\. Ramji, J\. Yang, G\. Kaiser, and B\. RayUnderstanding automated program repair agents through the lens of traceability: an empirical study\.Note:Accepted at ISSTA 2026External Links:2506\.08311,[Link](https://arxiv.org/abs/2506.08311)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2025\)Z\. Chen, R\. Tang, G\. Deng, F\. Wu, J\. Wu, Z\. Jiang, V\. Prasanna, A\. Cohan, and X\. WangLocAgent: graph\-guided LLM agents for code localization\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8697–8727\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.426),[Link](https://aclanthology.org/2025.acl-long.426/)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p3.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Chowdhuryet al\.\(2024\)N\. Chowdhury, J\. Aung, C\. J\. Shern, O\. Jaffe, D\. Sherburn, G\. Starace, E\. Mays, R\. Dias, M\. Aljubeh, and M\. GlaeseIntroducing SWE\-bench verified\.Note:[https://openai\.com/index/introducing\-swe\-bench\-verified/](https://openai.com/index/introducing-swe-bench-verified/)OpenAI Blog\. Accessed: 2026\-08Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p2.1)\.
- Denget al\.\(2025\)X\. Deng, J\. Da, E\. Pan, Y\. Y\. He, C\. Ide, K\. Garg, N\. Lauffer, A\. Park, N\. Pasari, C\. Rane, K\. Sampath, M\. Krishnan, S\. Kundurthy, S\. Hendryx, Z\. Wang, V\. Bharadwaj, J\. Holm, R\. Aluri, C\. B\. C\. Zhang, N\. Jacobson, B\. Liu, and B\. KenstlerSWE\-bench pro: can AI agents solve long\-horizon software engineering tasks?\.External Links:2509\.16941,[Link](https://arxiv.org/abs/2509.16941)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p5.1)\.
- Erolet al\.\(2026\)M\. H\. Erol, B\. El, M\. Suzgun, M\. Yuksekgonul, and J\. Y\. ZouCost\-of\-pass: an economic framework for evaluating language models\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 34103–34127\.Cited by:[§2](https://arxiv.org/html/2609.30725#S2.p1.1)\.
- Gao and Peng \(2025\)P\. Gao and C\. PengMore with less: an empirical study of turn\-control strategies for efficient coding agents\.External Links:2510\.16786,[Link](https://arxiv.org/abs/2510.16786)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Georgeset al\.\(2007\)A\. Georges, D\. Buytaert, and L\. EeckhoutStatistically rigorous Java performance evaluation\.InProceedings of the 22nd Annual ACM SIGPLAN Conference on Object\-Oriented Programming, Systems, Languages, and Applications \(OOPSLA\),pp\. 57–76\.External Links:[Document](https://dx.doi.org/10.1145/1297027.1297033)Cited by:[§A\.1\.4](https://arxiv.org/html/2609.30725#A1.SS1.SSS4.p2.1)\.
- Gonget al\.\(2026\)J\. Gong, R\. Gu, Z\. Fei, Y\. Cao, L\. Twist, A\. Geiger, S\. Han, D\. Sobania, F\. Sarro, and J\. M\. ZhangSkillMOO: multi\-objective optimization of agent skills for software engineering\.InProceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering \(ASE 2026\), New Ideas and Emerging Results \(NIER\) Track,External Links:[Document](https://dx.doi.org/10.1145/3832783.3834550),[Link](https://doi.org/10.1145/3832783.3834550)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Goodman \(1992\)S\. N\. GoodmanA comment on replication, P\-values and evidence\.Statistics in Medicine11\(7\),pp\. 875–879\.External Links:[Document](https://dx.doi.org/10.1002/sim.4780110705),[Link](https://doi.org/10.1002/sim.4780110705)Cited by:[§A\.1\.4](https://arxiv.org/html/2609.30725#A1.SS1.SSS4.p4.1)\.
- Jiaet al\.\(2026\)H\. Jia, E\. T\. Barr, and S\. MechtaevCompressing code context for LLM\-based issue resolution\.External Links:2603\.28119,[Link](https://arxiv.org/abs/2603.28119)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Jianget al\.\(2026\)G\. Jiang, Z\. Su, X\. Qu, and Y\. R\. FungXSkill: continual learning from experience and skills in multimodal agents\.Note:Accepted at ICML 2026External Links:2603\.12056,[Link](https://arxiv.org/abs/2603.12056)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Kalibera and Jones \(2013\)T\. Kalibera and R\. JonesRigorous benchmarking in reasonable time\.InProceedings of the 2013 International Symposium on Memory Management \(ISMM\),pp\. 63–74\.External Links:[Document](https://dx.doi.org/10.1145/2464157.2464160)Cited by:[§A\.1\.4](https://arxiv.org/html/2609.30725#A1.SS1.SSS4.p2.1)\.
- Kapooret al\.\(2024\)S\. Kapoor, B\. Stroebl, Z\. S\. Siegel, N\. Nadgir, and A\. NarayananAI agents that matter\.External Links:2407\.01502,[Link](https://arxiv.org/abs/2407.01502)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p1.1)\.
- Kimet al\.\(2026\)M\. Kim, D\. Wang, S\. Cui, F\. Farmahinifarahani, T\. Y\. Zhuo, S\. Garg, B\. Ray, R\. Mukherjee, and V\. KumarCoherence collapse: diagnosing why code agents fail after reaching the right code\.External Links:2603\.24631,[Link](https://arxiv.org/abs/2603.24631)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Kimet al\.\(2025\)W\. Kim, S\. Park, Y\. In, S\. Kim, D\. Lee, and C\. ParkBeyond the final answer: evaluating the reasoning trajectories of tool\-augmented agents\.External Links:2510\.02837,[Link](https://arxiv.org/abs/2510.02837)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- LangChain \(2026\)LangChainLangSmith observability\.Note:[https://docs\.langchain\.com/langsmith/observability](https://docs.langchain.com/langsmith/observability)Accessed: 2026\-08Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Langfuse \(2026\)LangfuseLangfuse observability overview\.Note:[https://langfuse\.com/docs/observability/overview](https://langfuse.com/docs/observability/overview)Accessed: 2026\-08Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2026a\)X\. Li, W\. Chen, Y\. Liu, S\. Zheng, X\. Chen, Y\. He, Y\. Li, B\. You, H\. Shen, J\. Sun, S\. Wang, B\. Li, Q\. Zeng, D\. Wang, X\. Zhao, Y\. Wang, R\. B\. Chaim, Z\. Di, Y\. Gao, J\. He, Y\. He, L\. Jing, L\. Kong, X\. Lan, J\. Li, S\. Li, Y\. Li, Y\. Lin, X\. Liu, X\. Liu, H\. Lyu, Z\. Ma, B\. Wang, R\. Wang, T\. Wang, W\. Ye, Y\. Zhang, H\. Xing, Y\. Xue, S\. Dillmann, and H\. LeeSkillsBench: benchmarking how well agent skills work across diverse tasks\.External Links:2602\.12670,[Link](https://arxiv.org/abs/2602.12670)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2026b\)Y\. Li, R\. Miao, Z\. Qi, and T\. LanARISE: agent reasoning with intrinsic skill evolution in hierarchical reinforcement learning\.External Links:2603\.16060,[Link](https://arxiv.org/abs/2603.16060)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2026c\)Z\. Li, Q\. Dong, J\. Ma, D\. Zhang, K\. Jia, and Z\. SuiSelfBudgeter: adaptive token allocation for efficient LLM reasoning\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 21135–21156\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1063),[Link](https://aclanthology.org/2026.findings-acl.1063/)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Linet al\.\(2026\)F\. Lin, S\. Chen, R\. Fang, H\. Wang, and T\. LinStop wasting your tokens: towards efficient runtime multi\-agent systems\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=pzFhtpkabh)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2026a\)S\. Liu, Y\. Chen, R\. Krishna, S\. Sinha, J\. Ganhotra, and R\. JabbarvandProcess\-centric analysis of agentic software systems\.Proceedings of the ACM on Programming Languages10\(OOPSLA1\)\.Note:Article 163, 28 pagesExternal Links:[Document](https://dx.doi.org/10.1145/3798271),[Link](https://doi.org/10.1145/3798271)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p2.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2025a\)T\. Liu, Z\. Wang, J\. Miao, I\. Hsu, J\. Yan, J\. Chen, R\. Han, F\. Xu, Y\. Chen, K\. Jiang, S\. Daruki, Y\. Liang, W\. Y\. Wang, T\. Pfister, and C\. LeeBudget\-aware tool\-use enables effective agent scaling\.External Links:2511\.17006,[Link](https://arxiv.org/abs/2511.17006)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2025b\)X\. Liu, B\. Lan, Z\. Hu, Y\. Liu, Z\. Zhang, F\. Wang, M\. Q\. Shieh, and W\. ZhouCodexGraph: bridging large language models and code repositories via code graph databases\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 142–160\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.7),[Link](https://aclanthology.org/2025.naacl-long.7/)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p3.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2026b\)X\. Liu, X\. Luo, L\. Li, G\. Huang, J\. Liu, and H\. QiaoSkillForge: forging domain\-specific, self\-evolving agent skills in cloud technical support\.InProceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval,External Links:[Document](https://dx.doi.org/10.1145/3805712.3808466),[Link](https://doi.org/10.1145/3805712.3808466)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Luket al\.\(2026\)C\. Luk, M\. M\. Najafi, Z\. Jia, W\. Yang, X\. Li, J\. Zhu, Y\. Ren, L\. Chen, and G\. CongContextSniper: AntTrail’s token\-efficient code memory for repository\-level program repair\.External Links:2607\.01916,[Link](https://arxiv.org/abs/2607.01916)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Maet al\.\(2026\)Z\. Ma, S\. Yang, Y\. Ji, X\. Wang, Y\. Wang, Y\. Hu, T\. Huang, and X\. ChuSkillClaw: let skills evolve collectively with agentic evolver\.External Links:2604\.08377,[Link](https://arxiv.org/abs/2604.08377)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Majgaonkaret al\.\(2025\)O\. Majgaonkar, Z\. Fei, X\. Li, F\. Sarro, and H\. YeUnderstanding code agent behaviour: an empirical study of success and failure trajectories\.External Links:2511\.00197,[Link](https://arxiv.org/abs/2511.00197)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Mchenry \(2026\)C\. MchenryCodeGraph: semantic code intelligence for AI coding agents\.Note:[https://github\.com/colbymchenry/codegraph](https://github.com/colbymchenry/codegraph)Accessed: 2026\-08Cited by:[§A\.3\.1](https://arxiv.org/html/2609.30725#A1.SS3.SSS1.p2.1),[§1](https://arxiv.org/html/2609.30725#S1.p5.1)\.
- Miller \(2024\)E\. MillerAdding error bars to evals: a statistical approach to language model evaluations\.arXiv preprint arXiv:2411\.00640\.External Links:[Link](https://arxiv.org/abs/2411.00640)Cited by:[§A\.1\.4](https://arxiv.org/html/2609.30725#A1.SS1.SSS4.p2.1)\.
- MiniMax \(2026\)MiniMaxMiniMax M3: frontier coding, 1M context, native multimodality—all in one model\.Note:[https://www\.minimax\.io/blog/minimax\-m3](https://www.minimax.io/blog/minimax-m3)Accessed: 2026\-08Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p5.1)\.
- Nayabet al\.\(2026\)S\. Nayab, G\. Rossolini, M\. Simoni, A\. Saracino, G\. Buttazzo, N\. Manes, and F\. GiacomelliConcise thoughts: impact of output length on LLM reasoning and cost\.Information Sciences757,pp\. 123939\.External Links:[Document](https://dx.doi.org/10.1016/j.ins.2026.123939),[Link](https://doi.org/10.1016/j.ins.2026.123939)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Niet al\.\(2026\)J\. Ni, Y\. Liu, X\. Liu, Y\. Sun, M\. Zhou, P\. Cheng, D\. Wang, E\. Zhao, X\. Jiang, and G\. JiangTrace2Skill: distill trajectory\-local lessons into transferable agent skills\.External Links:2603\.25158,[Link](https://arxiv.org/abs/2603.25158)Cited by:[§A\.4\.1](https://arxiv.org/html/2609.30725#A1.SS4.SSS1.p4.1),[§A\.4](https://arxiv.org/html/2609.30725#A1.SS4.p2.1),[§2](https://arxiv.org/html/2609.30725#S2.p1.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Onget al\.\(2024\)I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. W\. Kadous, and I\. StoicaRouteLLM: learning to route LLMs with preference data\.External Links:2406\.18665,[Link](https://arxiv.org/abs/2406.18665)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- OpenAI \(2025\)OpenAIIntroducing codex\.Note:[https://openai\.com/index/introducing\-codex/](https://openai.com/index/introducing-codex/)Accessed: 2026\-08Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p1.1)\.
- Ouyanget al\.\(2025a\)S\. Ouyang, J\. M\. Zhang, M\. Harman, and M\. WangAn empirical study of the non\-determinism of ChatGPT in code generation\.ACM Transactions on Software Engineering and Methodology34\(2\),pp\. 1–28\.Note:Article 42External Links:[Document](https://dx.doi.org/10.1145/3697010),[Link](https://doi.org/10.1145/3697010)Cited by:[§2](https://arxiv.org/html/2609.30725#S2.p4.1)\.
- Ouyanget al\.\(2025b\)S\. Ouyang, W\. Yu, K\. Ma, Z\. Xiao, Z\. Zhang, M\. Jia, J\. Han, H\. Zhang, and D\. YuRepoGraph: enhancing AI software engineering with repository\-level code graph\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/4a4a3c197deac042461c677219efd36c-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p3.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Panet al\.\(2025\)D\. Pan, Z\. Sun, C\. Zhang, D\. Lo, and X\. DuThe hidden cost of readability: how code formatting silently consumes your LLM budget\.External Links:2508\.13666,[Link](https://arxiv.org/abs/2508.13666)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Piebald AI \(2026\)Piebald AIClaude code system prompts\.Note:[https://github\.com/Piebald\-AI/claude\-code\-system\-prompts/tree/06bfbc6](https://github.com/Piebald-AI/claude-code-system-prompts/tree/06bfbc6)Prompts extracted from Claude Code v2\.1\.169\. Accessed: 2026\-09Cited by:[§A\.2\.1](https://arxiv.org/html/2609.30725#A1.SS2.SSS1.p1.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.5: towards native multimodal agents\.Note:[https://qwen\.ai/blog?id=qwen3\.5](https://qwen.ai/blog?id=qwen3.5)Accessed: 2026\-08Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p5.1)\.
- Salimet al\.\(2026\)M\. Salim, J\. Latendresse, S\. Khatoonabadi, and E\. ShihabTokenomics: quantifying where tokens are used in agentic software engineering\.External Links:2601\.14470,[Link](https://arxiv.org/abs/2601.14470)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Shao and Chow \(2002\)J\. Shao and S\. ChowReproducibility probability in clinical trials\.Statistics in Medicine21\(12\),pp\. 1727–1742\.External Links:[Document](https://dx.doi.org/10.1002/sim.1177),[Link](https://doi.org/10.1002/sim.1177)Cited by:[§A\.1\.4](https://arxiv.org/html/2609.30725#A1.SS1.SSS4.p4.1)\.
- Shekharet al\.\(2024\)S\. Shekhar, T\. Dubey, K\. Mukherjee, A\. Saxena, A\. Tyagi, and N\. KotlaTowards optimizing the costs of LLM usage\.External Links:2402\.01742,[Link](https://arxiv.org/abs/2402.01742)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Suet al\.\(2025\)H\. Su, S\. Diao, X\. Lu, M\. Liu, J\. Xu, X\. Dong, Y\. Fu, P\. Belcak, H\. Ye, H\. Yin, Y\. Dong, E\. Bakhturina, T\. Yu, Y\. Choi, J\. Kautz, and P\. MolchanovToolOrchestra: elevating intelligence via efficient model and tool orchestration\.External Links:2511\.21689,[Link](https://arxiv.org/abs/2511.21689)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p2.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- TechCrunch \(2025\)TechCrunchOpenAI is losing money on its pricey ChatGPT Pro plan, CEO Sam Altman says\.Note:[https://techcrunch\.com/2025/01/05/openai\-is\-losing\-money\-on\-its\-pricey\-chatgpt\-pro\-plan\-ceo\-sam\-altman\-says/](https://techcrunch.com/2025/01/05/openai-is-losing-money-on-its-pricey-chatgpt-pro-plan-ceo-sam-altman-says/)Accessed: 2026\-09Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p1.1)\.
- Wanget al\.\(2025\)F\. Wang, H\. Liu, Z\. Dai, J\. Zeng, Z\. Zhang, Z\. Wu, C\. Luo, Z\. Li, X\. Tang, Q\. He, and S\. WangAgentTTS: large language model agent for test\-time compute\-optimal scaling strategy in complex tasks\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Document](https://dx.doi.org/10.52202/085713-3288),[Link](https://papers.neurips.cc/paper_files/paper/2025/hash/8d9bbba8cac9cabb54e85ee7f21441c8-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2026\)Y\. Wang, Y\. Shi, M\. Yang, R\. Zhang, S\. He, H\. Lian, Y\. Chen, S\. Ye, K\. Cai, and X\. GuSWE\-Pruner: self\-adaptive context pruning for coding agents\.External Links:2601\.16746,[Link](https://arxiv.org/abs/2601.16746)Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p2.1),[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Xiaet al\.\(2026a\)P\. Xia, J\. Chen, H\. Wang, J\. Liu, K\. Zeng, Y\. Wang, S\. Han, Y\. Zhou, X\. Zhao, H\. Chen, Z\. Zheng, C\. Xie, and H\. YaoSkillRL: evolving agents via recursive skill\-augmented reinforcement learning\.External Links:2602\.08234,[Link](https://arxiv.org/abs/2602.08234)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Xiaet al\.\(2026b\)P\. Xia, J\. Chen, X\. Yang, H\. Tu, J\. Liu, K\. Xiong, S\. Han, S\. Qiu, H\. Ji, Y\. Zhou, Z\. Zheng, C\. Xie, and H\. YaoMetaClaw: just talk – an agent that meta\-learns and evolves in the wild\.External Links:2603\.17187,[Link](https://arxiv.org/abs/2603.17187)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Xiaoet al\.\(2026\)Y\. Xiao, P\. Gao, C\. Peng, and Y\. XiongReducing cost of LLM agents with trajectory reduction\.Proceedings of the ACM on Software Engineering3\(FSE\),pp\. 1241–1263\.External Links:[Document](https://dx.doi.org/10.1145/3797084),[Link](https://doi.org/10.1145/3797084)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2024\)J\. Yang, C\. E\. Jimenez, A\. Wettig, K\. Lieret, S\. Yao, K\. Narasimhan, and O\. PressSWE\-agent: agent\-computer interfaces enable automated software engineering\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Document](https://dx.doi.org/10.52202/079017-1601),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/5a7c947568c1b1328ccc5230172e1e7c-Abstract-Conference.html)Cited by:[§A\.1\.3](https://arxiv.org/html/2609.30725#A1.SS1.SSS3.p2.1),[§1](https://arxiv.org/html/2609.30725#S1.p5.1)\.
- Yanget al\.\(2026\)Y\. Yang, J\. Li, Q\. Pan, B\. Zhan, Y\. Cai, L\. Du, J\. Zhou, K\. Chen, Q\. Chen, X\. Li, B\. Zhang, and L\. HeAutoSkill: experience\-driven lifelong learning via skill self\-evolution\.External Links:2603\.01145,[Link](https://arxiv.org/abs/2603.01145)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§A\.4\.1](https://arxiv.org/html/2609.30725#A1.SS4.SSS1.p2.1)\.
- Zeff \(2025\)M\. ZeffAnthropic unveils new rate limits to curb Claude Code power users\.Note:[https://techcrunch\.com/2025/07/28/anthropic\-unveils\-new\-rate\-limits\-to\-curb\-claude\-code\-power\-users/](https://techcrunch.com/2025/07/28/anthropic-unveils-new-rate-limits-to-curb-claude-code-power-users/)TechCrunch\. Accessed: 2026\-09Cited by:[§1](https://arxiv.org/html/2609.30725#S1.p1.1)\.
- Zhanget al\.\(2025\)G\. Zhang, J\. Wang, J\. Chen, W\. Zhou, K\. Wang, and S\. YanAgenTracer: who is inducing failure in the LLM agentic systems?\.External Links:2509\.03312,[Link](https://arxiv.org/abs/2509.03312)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px1.p1.1)\.
- Zhenget al\.\(2025\)B\. Zheng, M\. Y\. Fatemi, X\. Jin, Z\. Z\. Wang, A\. Gandhi, Y\. Song, Y\. Gu, J\. Srinivasa, G\. Liu, G\. Neubig, and Y\. SuSkillWeaver: web agents can self\-improve by discovering and honing skills\.External Links:2504\.07079,[Link](https://arxiv.org/abs/2504.07079)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.
- Zhouet al\.\(2026\)H\. Zhou, S\. Guo, A\. Liu, Z\. Yu, Z\. Gong, B\. Zhao, Z\. Chen, M\. Zhang, Y\. Chen, J\. Li, R\. Yang, Q\. Liu, X\. Yu, J\. Zhou, N\. Wang, C\. Sun, and J\. WangMemento\-Skills: let agents design agents\.External Links:2603\.18743,[Link](https://arxiv.org/abs/2603.18743)Cited by:[§5](https://arxiv.org/html/2609.30725#S5.SS0.SSS0.Px3.p1.1)\.

## Appendix AAppendix

This appendix provides supplementary details for our study\. First, we describe additional study setup details, including the versions and configurations of Claude Code and Mini\-SWE\-Agent, action\-labeling procedures, benchmark task split, and how we handle the agent execution randomness\. Second, we provide additional results and analysis for RQ1, including the performance of the four agent configurations on the 300\-task RQ1 analysis set, extra detection details, analysis, and examples of the three cost\-inefficient behaviors\. Third, we provide CodeGraph integration details and further results analysis for RQ2\. Finally, we detail the implementation of agent\-synthesized skills and provide the full content of both agent\-synthesized and developer\-designed skills in RQ3\.

### A\.1Additional Details for Study Setup

In this section, we will provide the specific version and configuration details of the agents included in our study, and then we will introduce the agent action labeling process, benchmark task split, and the details of handling agent execution randomness\.

We use the same agent versions and configurations across all RQs\.CCuses CLI v2\.1\.169\([Anthropic, 2026a](https://arxiv.org/html/2609.30725#bib.bib63)\)with a $3\.00 per\-task cost limit for SWE\-Bench Verified tasks and a $5\.00 per\-task cost limit for SWE\-Bench Pro tasks; the backbone model temperature is not exposed through the CLI\.MSAuses v2\.3\.0 with temperature 0\.0, a 250\-step limit, and the same per\-task cost limit as CC\. We next elaborate on the action\-labeling procedure, benchmark split details, and our treatment of execution randomness described in Section[2](https://arxiv.org/html/2609.30725#S2)\.

Table 5:Action\-label taxonomy\.CategoryLabelDescriptionRetrievalRetrieve\\\_StructureSearches or explores the repository structure or current location\.Retrieve\\\_CodeReads or searches source code files, except test files or documents\.Retrieve\\\_TestsReads or searches test files specifically\.Retrieve\\\_DocsReads documentation \(non\-code references such as README\)\.ModificationGenerate\\\_PatchModifies a file inside the working tree \(e\.g\.,/testbed\) to fix the issue\.Generate\\\_TestsWrites tests or reproduction scripts \(e\.g\.,python\-c\)\.Generate\\\_ToolsCreates custom tools for the agent’s own use\.Create\\\_Files\\\_DirectoriesCreates new files or directories \(e\.g\.,touch,mkdir\)\.Remove\\\_ContentRemoves code or files the agent previously created\.VerificationExecute\\\_Existing\\\_TestsRuns existing repository tests\.Execute\\\_Generated\\\_TestsExecutes agent\-generated code\.Review\\\_PatchReviews the applied patch \(e\.g\.,gitdiff\)\.GitExecute\\\_GitRuns git commands other thangitdiff\.OtherFinish\\\_TaskSubmits the final solution\.Env\\\_InspectionChecks or installs packages; inspects environment variables or resources\.Pure\\\_ReasoningThe step contains only reasoning, without calling any tool\.Delegate\\\_SubagentDispatches a subagent via theAgenttool \(CConly\)\.OtherDoes not fit any label above\.#### A\.1\.1Action Labeling

Why label actions\.The action labels serve four purposes\. First, the two agent frameworks expose actions in different surface forms:CCissues named tool calls whileMSAissues raw shell commands\. We therefore assign every step one or two semantic action labels before any detection, which projects both frameworks onto a shared vocabulary and makes their trajectories comparable\. Second, they restrict each detector to its relevant step population: SubRetrv compares returned context only across retrieval steps, SimScrpt mines scripts only from script\-generation or script\-execution steps, and ReTest groups only test\-execution steps\. Third, they provide the structural boundaries the detections rely on\. For example, a step labeledGenerate\\\_Patchcloses the current ReTest group, so re\-running a test after an updated patch is never flagged\. Finally, they support the aggregate analysis, such as the retrieval\-action accounting in Table[2](https://arxiv.org/html/2609.30725#S3.T2)\(Patch\-Adjacent SubRetrv and Within\-Sequence SubRetrv\)\. The same labeling pipeline is applied uniformly to every run in RQ1–RQ3\.

Label taxonomy\.We use 18 labels in five categories, summarized in Table[5](https://arxiv.org/html/2609.30725#A1.T5)\. ActionDelegate\\\_Subagentarises only inCC’s tool set\. A tool call may carry up to two labels\. For example, an inlinepython\-creproduction script both authors and executes agent\-generated code \(Generate\\\_Tests&Execute\\\_Generated\\\_Tests\)\.

Rule\-based labeling\.Steps are first labeled by deterministic rules, with one rule family per harness\. ForMSA, whose actions are raw shell commands, each command is split into atomic sub\-commands \(\\&\\&,;, and pipes\) after unwrapping prefixes such astimeout; navigation\-only fragments \(cd,export\) are dropped\. Read\-only verbs \(cat,head,sed\-n,grep,ls,find, …\) map to*Retrieval*, with test\-path, documentation\-path, and source\-extension patterns selecting the sub\-label\. Write channels \(heredocs,sed\-i,tee, output redirection\) map to*Modification*\. Modifying an existing file inside the working directory yieldsGenerate\\\_Patch; a test\-looking file yieldsGenerate\\\_Tests; ephemeral script generations \(python\-c,python<<EOF\) are labeled asGenerate\\\_Tests\+\+Execute\\\_Generated\\\_Tests, unless they call an existing test API or open a working\-tree file for writing\. Test runners map toExecute\\\_Existing\\\_TestsorExecute\\\_Generated\\\_Testsdepending on their target;gitdiffmaps toReview\\\_Patch; othergitcommands map toExecute\\\_Git; the submission marker \(MSA only\) maps toFinish\\\_Task; and steps without any command map toPure\\\_Reasoning\. ForCC, whose actions are named tool calls, tools map directly \(Read/Grepto*Retrieval*by path,GlobtoRetrieve\\\_Structure,Edit/Writeto*Modification*by path,AgenttoDelegate\\\_Subagent\), andBashcalls are routed through the same command classifier asMSA\.

LLM fallback\.A step is forwarded to an LLM judge \(GPT\-5\.1\) whenever any of its atomic commands matches no rule\. The judge receives the step’s thought and commands together with the taxonomy definitions and returns at most two labels with a brief justification\. Among the RQ1 trajectories, the fallback handles 0\.94% ofCCsteps, 31\.17% ofMSAS46steps, 1\.62% ofMSAMM3steps, and 1\.18% ofMSAQ35\+steps, accounting for 7\.26% of the total steps of four agent configurations on 300 Verified tasks\. By submitting all LLM\-labeling requests through the batch mode, the labeling cost is less than 0\.2 cents per trajectory\.

Resulting distribution\.Figure[7](https://arxiv.org/html/2609.30725#A1.F7)shows the label distribution over the RQ1 trajectories among four agent configurations\. The pattern is consistent across configurations: reading source code \(Retrieve\\\_Code, 25\.65–33\.12%\) together with generating and executing scripts \(Generate\\\_Tests\+\+Execute\\\_Generated\\\_Tests, 23\.66–34\.07%\) dominate every configuration, followed by existing\-test execution \(12\.15–13\.57%\)\. Architecture differences appear in the tail: onlyCCdelegates work to subagents \(Delegate\\\_Subagent, 1\.83% of its actions\), whileMSAconfigurations review patches throughgitdiffmore often \(3\.57–5\.04% vs\. 0\.32%\)\.

Figure 7:Distribution of action labels over the RQ1 trajectories\.Table 6:Task selection by repository on SWE\-bench Verified and SWE\-bench Pro\.SWE\-bench Verified\(500 tasks\)SWE\-bench Pro\(731 public tasks\)RepositoryAnalysisEval\.RepositoryPoolEval\.astropy/astropy139ansible/ansible9612django/django13992element\-hq/element\-web568matplotlib/matplotlib2014flipt\-io/flipt8510mwaskom/seaborn11future\-architect/vuls629pallets/flask10gravitational/teleport7610psf/requests53internetarchive/openlibrary9111pydata/xarray139navidrome/navidrome578pylint\-dev/pylint64NodeBB/NodeBB448pytest\-dev/pytest127protonmail/webclients659scikit\-learn/scikit\-learn1913qutebrowser/qutebrowser7910sphinx\-doc/sphinx2618tutao/tutanota205sympy/sympy4530Total300200Total731100
#### A\.1\.2Behavior Cost Attribution

For each action flagged as cost\-inefficient, we use theo200k\_basetokenizer fromtiktokento count the tokens in its reasoning, commands, and tool output\. We then attribute costs using the corresponding model’s token prices\. Generated content is charged at the output\-token price, while both generated content and tool output incur input\-token or cache\-token charges when carried into subsequent LLM requests\. We sum these charges to obtain the action’s attributed cost\.

#### A\.1\.3Benchmarks & Task Split

Table[6](https://arxiv.org/html/2609.30725#A1.T6)lists the tasks per repository used for analysis inRQ1and for evaluation inRQ2–RQ3\. For*SWE\-bench Verified*, we order the tasks of each repository by creation date and assign the earliest tasks, about 60% per repository, to the analysis set and the remaining ones to*Verified\-200*\.pallets/flaskhas a single task, which falls into the analysis set\.

For*SWE\-bench Pro*, we draw a stratified random sample of 100 tasks from the 731 public tasks with a fixed seed\. The*Pool*column gives the public tasks available in each repository\. Each repository receives a quota that grows with the square root of its pool size, with a minimum of four tasks, so that all 11 repositories are represented, and repositories with more tasks do not crowd out smaller ones\. Within each repository, tasks are ranked by their cost in the public SWE\-bench Pro leaderboard run of SWE\-agent\([Yang et al\., 2024](https://arxiv.org/html/2609.30725#bib.bib2)\)with Claude Sonnet 4\.5 \(250\-turn limit, no cost cap\)444When we selected the tasks on June 26, 2026, this was the best\-performing leaderboard run whose released trajectories include cost information \(43\.70% resolved\)\. The share of resolved tasks in*Pro\-100*matches that of this run\.and split into three equal\-sized terciles\. The low, mid, and high terciles are sampled at 20%, 30%, and 50%, since costlier tasks are expected to expose more cost\-inefficient behaviors\.

Table 7:Noise floor of the eight baselines\. The floorssis the standard deviation of Pass@1 \(pp\) and the coefficient of variation of cost and CoP \(%\)\. The last two columns give the smallest single\-run\|Δ\|\|\\Delta\|classified as robust \(1\.90​s1\.90\\,s\)\. Per\-run cost is averaged over the tasks completed in all three runs\.Pass@1CostCoPMin\. robust\|Δ\|\|\\Delta\|,R=1R\{=\}1BenchmarkConfigRun 1 / 2 / 3 \(%\)spasss\_\{\\mathrm\{pass\}\}\(pp\)Run 1 / 2 / 3 \($\)scosts\_\{\\mathrm\{cost\}\}\(%\)sCoPs\_\{\\mathrm\{CoP\}\}\(%\)CostPass@1 \(pp\)Verified\-200CC74\.00 / 75\.50 / 75\.500\.870\.598 / 0\.525 / 0\.5158\.359\.5615\.87%1\.65Verified\-200MSAS4672\.00 / 71\.50 / 74\.001\.320\.674 / 0\.605 / 0\.6365\.405\.4910\.25%2\.51Verified\-200MSAMM378\.00 / 71\.50 / 76\.003\.330\.463 / 0\.442 / 0\.4773\.802\.857\.21%6\.33Verified\-200MSAQ35\+65\.00 / 70\.50 / 66\.002\.930\.123 / 0\.110 / 0\.1155\.709\.6810\.82%5\.57Pro\-100CC55\.00 / 56\.00 / 52\.002\.080\.917 / 0\.922 / 0\.8812\.441\.944\.64%3\.95Pro\-100MSAS4655\.00 / 49\.00 / 52\.003\.001\.037 / 1\.067 / 0\.9834\.148\.437\.87%5\.70Pro\-100MSAMM360\.00 / 64\.00 / 56\.004\.000\.859 / 0\.852 / 0\.8962\.709\.325\.13%7\.60Pro\-100MSAQ35\+47\.00 / 41\.00 / 47\.003\.460\.126 / 0\.126 / 0\.1322\.837\.225\.38%6\.58
#### A\.1\.4Controlling Agent Execution Randomness

This section describes how the run\-to\-run noise floors inRQ2–RQ3are measured, how the effect of single\-run approach cells is judged, and how the underlying equal\-variance assumption was validated\. All numbers are produced by the same pipeline that generates Table[3](https://arxiv.org/html/2609.30725#S3.T3)\.

Design Overview\.Re\-running every \(benchmark, configuration, approach, and baseline\) cell three times \(2×\\times4×\\times\(3\+1\)\(3\+1\)×\\times3=96\\ =\\ 96runs in total\) would cost about $7,500 at the per\-run costs we observed, so we proceed in three steps with selective re\-running 8 baseline settings and 9 approach cells\. First, every*baseline*setting \(two benchmarks×\\timesfour agent configurations\) is executed three times to estimate the run\-to\-run noise floors in Table[7](https://arxiv.org/html/2609.30725#A1.T7), following standard practice in performance benchmarking\([Georges et al\., 2007](https://arxiv.org/html/2609.30725#bib.bib65);[Kalibera and Jones, 2013](https://arxiv.org/html/2609.30725#bib.bib66)\)and LLM evaluations\([Miller, 2024](https://arxiv.org/html/2609.30725#bib.bib67)\)\. Second, 15 approach cells are executed once, and their effects are judged against the noise floors transferred from their corresponding baseline,*under the equal\-variance assumption that our approaches do not materially alter the sampling and infrastructure nondeterminism driving run\-to\-run noise*\. We further verify this assumption directly on the nine replicated cells in Table[8](https://arxiv.org/html/2609.30725#A1.T8), selected to cover both benchmarks, all three approaches, and both agent frameworks at low replication cost\. Here are the nine approach cells we select to replicate: six cells run all three approaches on both benchmarks withMSAQ35\+, the cheapest configuration to replicate; one cell runs developer\-designed skills on Verified\-200 withMSAMM3, the noisiest configuration based on its baseline noise floor; the remaining two cells, whose single\-run effects are borderline, run developer\-designed skills and CodeGraph on Verified\-200 withCC\. In total, theRQ2–RQ3evaluation comprises over 10k trajectories in SWE\-Bench Verified and Pro\. For RQ1, each agent configuration is only executed once on 300 Verified tasks\. The remainder of this section details these steps in turn: the measured noise floors, the decision rule that classifies an observed effect as robust, the validation of the equal\-variance assumption together with a stress test showing that the verdicts hold even if it were violated, and the reason behavior metrics carry no robustness verdicts\.

Table 8:Approach cells executed three times to verify the equal\-variance assumption\. Each metric lists the approach and baseline noise \(sapprs\_\{\\mathrm\{appr\}\}/sbases\_\{\\mathrm\{base\}\}\) and their ratio\. The remaining 15 approach cells are single runs\.Pass@1 \(pp\)Cost \($\)CoPBenchmarkConfigApproachPer runsapprs\_\{\\mathrm\{appr\}\}/sbases\_\{\\mathrm\{base\}\}RatioPer runsapprs\_\{\\mathrm\{appr\}\}/sbases\_\{\\mathrm\{base\}\}RatioRatioVerified\-200CCDevSkills73\.00 / 73\.50 / 74\.000\.50 / 0\.870\.580\.496 / 0\.482 / 0\.4307\.40 / 8\.350\.890\.81Verified\-200CCCodeGraph73\.50 / 73\.50 / 75\.501\.15 / 0\.871\.330\.584 / 0\.608 / 0\.5812\.52 / 8\.350\.300\.38Verified\-200MSAMM3DevSkills72\.00 / 72\.50 / 74\.501\.32 / 3\.330\.400\.371 / 0\.381 / 0\.4105\.23 / 3\.801\.381\.19Verified\-200MSAQ35\+DevSkills67\.50 / 72\.00 / 70\.002\.25 / 2\.930\.770\.107 / 0\.099 / 0\.1033\.83 / 5\.700\.670\.73Verified\-200MSAQ35\+SynSkills70\.00 / 66\.00 / 73\.003\.51 / 2\.931\.200\.119 / 0\.112 / 0\.1153\.01 / 5\.700\.530\.48Verified\-200MSAQ35\+CodeGraph73\.00 / 72\.00 / 72\.500\.50 / 2\.930\.170\.109 / 0\.115 / 0\.1133\.01 / 5\.700\.530\.38Pro\-100MSAQ35\+DevSkills48\.00 / 42\.00 / 48\.003\.46 / 3\.461\.000\.133 / 0\.130 / 0\.1341\.67 / 2\.830\.590\.86Pro\-100MSAQ35\+SynSkills49\.00 / 45\.00 / 44\.002\.65 / 3\.460\.760\.127 / 0\.124 / 0\.1251\.18 / 2\.830\.420\.66Pro\-100MSAQ35\+CodeGraph41\.38 / 35\.63 / 40\.233\.04 / 4\.140\.730\.142 / 0\.162 / 0\.1388\.85 / 3\.432\.581\.58

Noise floors\.Table[7](https://arxiv.org/html/2609.30725#A1.T7)lists the three baseline runs of eight settings\. The noise floorssis the standard deviation \(SD\) of Pass@1 \(in pp\), and the coefficients of variation \(CV\) of cost \(in %\) and CoP \(in %\)\. Pass@1 floors range from 0\.87 pp \(CCon Verified\-200\) to 4\.00 pp \(MSAMM3on Pro\-100\)\. The stability order within each benchmark isCC\>\>MSAS46\>\>MSAQ35\+\>\>MSAMM3\. Cost floors range from 2\.44% to 8\.35%\. CoP floors are generally wider than cost floors \(up to 3\.45×\\times,MSAMM3on Pro\-100\) because CoP inherits the variance of the Pass@1 in its denominator\. For CodeGraph on Pro\-100, which runs on 87 tasks \(Appendix[A\.3\.1](https://arxiv.org/html/2609.30725#A1.SS3.SSS1)\), all noise floors are re\-estimated on the same 87 tasks: 3\.51, 3\.51, 3\.45, and 4\.14 pp for Pass@1, 2\.59%, 3\.99%, 2\.67%, and 3\.43% for cost, and 5\.91%, 9\.20%, 8\.28%, and 10\.81% for CoP, forCC,MSAS46,MSAMM3, andMSAQ35\+, respectively\.

Robust decision rule\.Cost and CoP changes \(Δ\\Delta\) are computed on the tasks completed in every run of both the approach and its baseline\. For an approach cell withRRruns compared to its three\-run baseline, the standard error ofΔ\\DeltaisσΔ=s​1/R\+1/3\\sigma\_\{\\Delta\}=s\\sqrt\{1/R\+1/3\}\. We classifyΔ\\Deltaas robust when\|Δ\|/σΔ≥1\.645\|\\Delta\|/\\sigma\_\{\\Delta\}\\geq 1\.645, i\.e\., when a replication would reproduce its sign with probability at least 95% under a normal model\([Goodman, 1992](https://arxiv.org/html/2609.30725#bib.bib58);[Shao and Chow, 2002](https://arxiv.org/html/2609.30725#bib.bib57)\)\.For a single\-run cell this reduces to\|Δ\|≥1\.90​s\|\\Delta\|\\geq 1\.90\\,s, which is a stricter bar than a replicated cell’s\. The resulting thresholds for each setting are given in the last two columns of Table[7](https://arxiv.org/html/2609.30725#A1.T7)\.

Validating the equal\-variance assumption\.Table[8](https://arxiv.org/html/2609.30725#A1.T8)reports nine approach cells executed three times\. All other 15 approach cells are single runs\. Across the nine replicated cells, the approach\-side Pass@1 SD is 0\.17–1\.33×\\timesthe baseline’s, and the cost SD is 0\.30–1\.38×\\timesthe baseline’s in eight of the nine cells\. However, these ratios are themselves imprecise: for three normal runs, the sample variance satisfies2​s2/σ2∼χ222s^\{2\}/\\sigma^\{2\}\\sim\\chi^\{2\}\_\{2\}, so an estimatedsscarries a relative standard error of1−π/4/\(π/2\)≈52%\\sqrt\{1\-\\pi/4\}\\,/\\,\(\\sqrt\{\\pi\}/2\)\\approx 52\\%, and ratios moderately above one are expected even under equal noise\. Approach runs are thus not systematically noisier than baseline runs, which is what the transfer of baseline noise floors requires\. The single exception is CodeGraph withMSAQ35\+on Pro\-100, whose cost noise is 2\.58×\\timesits baseline’s \(8\.85% vs\. 3\.43%\)\. For this cell we use its own observed noise floor \(8\.85% for cost and 17\.05% for CoP\)\. Its cost and CoP increases remain robust\. The same amplification is not observed for CodeGraph withCCon Verified\-200 \(cost ratio 0\.38\), so we do not widen CodeGraph floors elsewhere\.

Moreover, the verdicts in Table[3](https://arxiv.org/html/2609.30725#S3.T3)carry a safety margin beyond this validation: even if the equal\-variance assumption were violated at the largest magnitude we observed, none of these verdicts would change for two reasons\. First, every single\-run effect classified as robust remains robust even after its transferred noise floor is inflated by1\.5×1\.5\\times, whereas the largest amplification observed among the replicated cells outside the exception is only1\.38×1\.38\\times\( the cost of DevSkill withMSAMM3on Verified\-200, in Table[8](https://arxiv.org/html/2609.30725#A1.T8)\)\. Second, though each robust effect retains a residual probability of reversing its sign in a replication, summing these probabilities over the 13 robust cost effects yields only 0\.07 expected sign reversals if the entire study were rerun\.

Behavior metrics\.The per\-task behavior frequencies reported in Table[3](https://arxiv.org/html/2609.30725#S3.T3)are considerably noisier than cost\. Across the eight baselines, the CV of a behavior’s per\-task frequency over the three runs averages 15\.68% and reaches 44\.47% \(SimScrpt ofMSAQ35\+on Pro\-100\), whereas cost floors range from 2\.44% to 8\.35%; it exceeds the cost floor of the same setting in 20 of the 24 baseline\-behavior combinations\. We therefore report behavior changes in Table[3](https://arxiv.org/html/2609.30725#S3.T3)as mechanistic evidence and do not attach robustness verdicts to them\.

### A\.2Additional Results for RQ1

In this section, we provide additional results and analysis for RQ1\. We first report Pass@1, task cost, and CoP for the four agent configurations on the 300 RQ1 tasks\. We further provide additional statistics, analysis, and illustrative examples for the three identified cost\-inefficient behaviors\.

Table 9:Pass@1, cost, and CoP of the four agent configurations inRQ1\.ConfigPass@1Cost \($\)CoPCC76\.67%0\.5200\.679MSAS4674\.33%0\.5540\.745MSAMM372\.67%0\.3650\.502MSAQ35\+65\.67%0\.1020\.156Task performance and cost\.Table[9](https://arxiv.org/html/2609.30725#A1.T9)reports Pass@1, average task cost, and CoP of the four agent configurations on the RQ1 analysis set\.CCachieves the highest Pass@1 \(76\.67%\) at a per\-task cost slightly belowMSAS46\($0\.52 vs\. $0\.55\)\.MSAMM3trades 1\.67 pp of Pass@1 againstMSAS46for a 34\.04% lower task cost, andMSAQ35\+is the cheapest configuration \($0\.10 per task\) but also the least accurate \(65\.67%\)\.

#### A\.2\.1Why CC Exhibits Fewer Cost\-Inefficient Behaviors than MSA

A large part ofCC’s advantage lies in its built\-in system prompt and tool descriptions\([Anthropic, 2026a](https://arxiv.org/html/2609.30725#bib.bib63);[Piebald AI, 2026](https://arxiv.org/html/2609.30725#bib.bib64)\)\. They already contain instructions that mitigate inefficient behaviors before any external optimization approach is applied, as listed below\.

SubRetrv:

- •Bashtool description: “IMPORTANT: Avoid using this tool to run commands, unless …a dedicated tool cannot accomplish your task\. Instead, use the appropriate dedicated tool…”, with an explicit mapping: file search viaGlobinstead offind, content search viaGrepinstead ofgrep, file reading viaReadinstead ofcat\.
- •Agenttool \(used for delegating subagents\) description: “Launch up to N agents IN PARALLEL \(single message, multiple tool calls\) to efficiently explore the codebase\. Use 1 agent when the task is isolated to known files…” However, this is double\-edged: it keeps the main context compact, but the summaries omit the exact code, so the main agent sometimes re\-reads the same region later\. This causes the Cross\-Agent SubRetrv that accounts for 50\.15% ofCC’s SubRetrv \(Table[2](https://arxiv.org/html/2609.30725#S3.T2)\)\.

However, no built\-in instructions contain context like “do not re\-read what you already retrieved”, which is one of the reasons whyCCstill has SubRetrv, and other mitigation approaches can still optimize such behavior\.

SimScrpt:

- •System prompt: “NEVER create files unless they’re absolutely necessary for achieving your goal\. ALWAYS prefer editing an existing file to creating a new one\.” It suppresses scratch file creation at the source, making file\-based SimScrpt rare onCC\.CCcreates 0\.21 draft files per task versus 2\.72 to 7\.07 forMSA, and file\-based SimScrpt appears in only twoCCtasks\.
- •Edittool description: “ALWAYS prefer editing existing files\. NEVER write new files unless explicitly required\.” The rich feedback of theEdittool and such descriptions make CC barely generate scripts for code editing, which is a major category inMSA’s file\-based SimScrpt\.

Since creating an issue reproduction script can still help solve the task,CCachieves this with inlinepython\-cone\-shots\. This explains why its SimScrpt is almost entirely ephemeral rather than file\-based\.

ReTest:we find no built\-in instruction that discourages re\-running tests without updated patches or asks the agent to reuse previous test output\. Consistently, ReTest is the one behavior whereCCshows no clear advantage: 2\.09 executions per task, the same magnitude asMSAS46\(2\.51\) andMSAQ35\+\(2\.10\)\.

Other instructions improve cost efficiency but are not tied to the specific cost\-inefficient behaviors we identified\.“You have the capability to call multiple tools in a single response\. When multiple independent pieces of information are requested and all commands are likely to succeed, run multiple tool calls in parallel for optimal performance\.” helps reduce round trips; “Your responses should be short and concise\.” reduces output tokens\.

#### A\.2\.2Subsumed Retrieval

Our detection mechanisms for SubRetrv do not consider code clones beyond whitespace differences, which may underestimate SubRetrv; we leave clone\-aware detection to future work\.

Table 10:Total retrieved context and the lines re\-read by SubRetrv actions\.MetricCCMSAS46MSAMM3MSAQ35\+Avg\. total lines read1,0325739121,051Avg\. lines re\-read by SubRetrv11496137190Overlapping rate11\.05%16\.69%15\.00%18\.10%Table[10](https://arxiv.org/html/2609.30725#A1.T10)quantifies the retrieved context behind SubRetrv: across configurations, agents re\-read 11\.05–18\.10% of all retrieved code lines through SubRetrv actions\. Notably, retrieval volume alone does not determine SubRetrv severity\.MSAS46reads the fewest lines per task \(573\) yet re\-reads a larger share \(16\.69%\) thanCC, which reads almost twice as many \(1,032\) with the lowest share \(11\.05%\)\. How the architecture manages the retrieved context therefore matters more than how much it retrieves\. Below, we illustrate SubRetrv occurrence scenarios with concrete examples\.

![Refer to caption](https://arxiv.org/html/2609.30725v1/patch-adj-sr.png)Figure 8:Patch\-Adjacent SubRetrv ofMSAQ35\+on SWE\-bench Verified taskdjango\-13158\.Patch\-Adjacent SubRetrv\. As illustrated in Figure[8](https://arxiv.org/html/2609.30725#A1.F8), ondjango\-13158,MSAQ35\+attempts to modifyset\\\_emptyat step 40 usingsed\-i, but the edit silently fails\. The agent then re\-reads the target region at step 41, detects that the function remains unchanged, and subsequently generates a Python script to apply the patch successfully\. However, the step\-41 retrieval is fully covered by an earlier retrieval at step 29, making it a SubRetrv instance\. This example illustrates how limited and error\-prone editing support can induce Patch\-Adjacent SubRetrv: when an editing tool is difficult to use reliably or provides insufficient feedback about the resulting code state, the agent may re\-read previously retrieved code to verify or recover from the edit\.

![Refer to caption](https://arxiv.org/html/2609.30725v1/within-sequence-sr.png)Figure 9:Within\-Sequence SubRetrv ofMSAQ35\+on SWE\-bench Verified taskastropy\-8707\.Within\-Sequence SubRetrv\. As illustrated in Figure[9](https://arxiv.org/html/2609.30725#A1.F9), onastropy\-8707,MSAQ35\+reads the entirefits/header\.py\(2,243 lines\) withcatat step 2\. The agent then zooms back into the same file for specific functions and precise localization during the following retrieval sequence: steps 3, 10, and 11 re\-read 74, 79, and 47 lines respectively, each fully contained in the step\-2 read\.

![Refer to caption](https://arxiv.org/html/2609.30725v1/long-distance-sr.png)Figure 10:Long\-Distance SubRetrv ofMSAMM3on SWE\-bench Verified taskmatplotlib\-24627\.Long\-Distance SubRetrv\. As illustrated in Figure[10](https://arxiv.org/html/2609.30725#A1.F10), onmatplotlib\-24627,MSAMM3reads lines 150–200 ofartist\.pyat step 22 and re\-retrieves lines 184–196 of the same file at step 194, after a 172\-step gap\. The re\-read is triggered by a long debugging loop and a progress stall: at steps 184–193 the agent runs the same pickle test six times while its reasoning repeatedly reverses itself \(“Wait, let me check…”, “Actually, let me re\-read the test…”, “Hmm, the test…”\)\. To re\-anchor its analysis, the agent re\-reads the previously retrieved region, along with regions oftest\\\_pickle\.pyand\\\_version\.pythat it had also retrieved before\.

#### A\.2\.3Similar Script Generation

Classifying script functionalities\.We classify each SimScrpt script by its function using AST analysis\. A script is a*patch script*if it opens a file for writing and transforms its content, throughre\.sub,str\.replace, or a read\-modify\-write sequence\. A script is classified as an*assertion test*if it containsassertstatements, importspytestorunittest, or callsself\.assert\*methods\. A script is an*inspection probe*if it only queries or prints runtime information, such asprint,repr, anddircalls, without producing a pass or fail verdict\. Scripts matching several signals are assigned by priority: patch, then test, then probe\. Scripts that cannot be parsed by AST \(0\.61–5\.54%\) are classified using pattern matching\.

Ephemeral SimScrpt\. The motivation example in Figure[1](https://arxiv.org/html/2609.30725#S1.F1)already contains this behavior\. To reproduce theunion\(\)\.none\(\)issue ondjango\-13158,CCemits four inlinepython\-cscripts of 23 to 29 lines\. Every script re\-emits the same logic:1configuring Django settings,2initializing the test environment and database,3creating test data,4building the querysets, and5verifying the\.none\(\)behavior that the issue asks to fix\. Yet the four scripts differ by only a few lines each: after a missing\-table failure in earlier probes, the first script updates the database initialization withmigrate\-\-run\-syncdb; the second updates the test\-environment setup; the third expands the combined\-query test scenarios, replacingcount\(\)withlist\(\)after Django rejectscount\(\)on combined querysets; and the fourth clears the query ordering withorder\\\_by\(’’\)after a default\-ordering conflict in the union query\. Because each script vanishes after execution, the agent regenerates the identical script logic on every iteration\.

![Refer to caption](https://arxiv.org/html/2609.30725v1/file-based-simscrpt.png)Figure 11:File\-based SimScrpt ofMSAMM3on SWE\-bench Verified taskmatplotlib\-24627\.File\-based SimScrpt\. As illustrated in Figure[11](https://arxiv.org/html/2609.30725#A1.F11), onmatplotlib\-24627,MSAMM3writes/tmp/patch\.pyat step 55 to modifyaxes/\\\_base\.py, then creates/tmp/patch2\.pyat step 93 and/tmp/patch3\.pyat step 113 as its fix evolves\. The three scripts share the same scaffold, which reads the file, applies a string replacement, and writes the file back; only the strings for replacement change\. Although the script already exists on disk, the agent versions its fix by writing a near\-duplicate sibling file each time instead of editing the existing one\.

Ideally, to prevent the File\-based SimScrpt in Figure[11](https://arxiv.org/html/2609.30725#A1.F11), the agent would persist a reusablepatch\.pythat accepts the target file and the old and new strings as parameters\. The script should provide explicit feedback on whether the replacement succeeded and report specific errors, such as missing or ambiguous matches\. The agent could then reuse the same script for subsequent patches, avoiding repeated script generation and its associated cost\.

#### A\.2\.4Test Re\-Execution

Identifying re\-executions\.The executed tests \(test ID\) are extracted from the commands labeled asExecute\_Existing\_TestsandExecute\_Generated\_Testsfrom each inter\-patch window\. For example, the following commandspytesttests/foo/test\\\_bar\.py,python\-mdjangotestfoo\.test\\\_bar, andpythontests/runtests\.pyfoo\.test\\\_barmap to the same test ID:test\\\_bar\. Flags and output filters such as\-vortaildo not affect the extracted tests, and the test IDs of the agent\-generated test scripts are recorded by their file path\. Within each group with the same test ID, we conservatively retain the final execution as potentially decision\-relevant and flag all preceding executions as ReTest\.

Figure[13](https://arxiv.org/html/2609.30725#A1.F13)shows that ReTest frequency is long\-tailed\. Depending on the configuration, 23 to 107 affected tasks contain at least six ReTest actions\. The tail is most pronounced forMSAMM3, where 107 of the 249 affected tasks contain at least six ReTest actions and 50 contain at least ten, compared with 39 and 8 of 149 forCC\. Figure[13](https://arxiv.org/html/2609.30725#A1.F13)shows that most ReTest actions immediately follow the previous execution: 48\.01–72\.84% occur at a step distance of one\.CCis the most concentrated, with 72\.84% at distance one and only 1\.76% at ten or more, whereasMSAMM3has the heaviest long\-distance tail at 18\.78%\. Immediate ReTest is consistent with test knowledge gaps and test\-signal recovery, where the agent retries with a different runner or output filter, while long\-distance ReTest is consistent with progress stalls\. Below, we illustrate the three causes with concrete examples\.

Figure 12:Distribution of ReTest actions per task\.Figure 13:Distribution of step distances between ReTest actions within a ReTest group\.Repository\-specific test knowledge gaps\. Agents may lack knowledge of a repository’s expected test runner, working directory, or configuration, leading to repeated failed validations\. For example, in Figure[1](https://arxiv.org/html/2609.30725#S1.F1),CCmakes eight failed attempts to runtest\_qs\_combinators: it repeatedly usespython\-mdjangotest, which cannot resolve thequeriesmodule\.CCthen inspects Django’s test structure and discovers its repository\-specific test runner\. It then invokestests/runtests\.pywith the correct test label, successfully passing all 31 tests\.

Test\-signal recovery\. A test execution may fail to provide usable signals because its output is truncated, ambiguous, or not effectively captured by the agent, prompting repeated execution to recover the missing information\. Such reruns can be locally reasonable because understanding the failure signal is critical for deciding the next debugging step, but they reveal inefficiency in how test results are delivered and retained\. For example, onsympy\-13878\(Figure[14](https://arxiv.org/html/2609.30725#A1.F14)\),MSAMM3runstest\_gammaeight times between steps 131–139 while applying different test feedback filters and keyword searches to diagnose two reported exceptions\. The failures stem from a Python 3\.9 compatibility issue that promotes aDeprecationWarningto an exception, butMSAMM3fails to capture an explicit exception marker from the filtered output such asError,Traceback, orException\. As a result, the agent repeatedly greps and truncates the test output without isolating the cause\. It eventually usesgitstashand reruns the test, successfully identifying the environment\-related issue\.

![Refer to caption](https://arxiv.org/html/2609.30725v1/test-signal-recovery-example.png)Figure 14:Test\-signal recovery ofMSAMM3on SWE\-bench Verified tasksympy\-13878\.Progress stalls\. In some cases, prior executions already provide clear test signals, yet the agent continues running the same tests without modifying the patch\. In these cases, the agent often exhibits decision\-making stalls, repeating the same reasoning and validation without advancing the solution\. For example, ondjango\-11141,MSAS46applies a patch at step 66, with onlytest\_load\_empty\_dirfailing because it encodes the legacy behavior that the issue intends to remove\. The agent recognizes that it should either update this outdated test, which is the correct solution, or redesign its implementation to preserve the old behavior\. However,MSAS46hesitates over whether updating the test functions satisfies the instruction requirements\. It consequently reruns themigrationstest suite 22 times andtest\_makemigrations\_no\_init18 times without another patch until the step budget is exhausted and the task fails\.

### A\.3Additional Details for RQ2

This section provides the additional details for Section[4\.2](https://arxiv.org/html/2609.30725#S4.SS2)\. Appendix[A\.3\.1](https://arxiv.org/html/2609.30725#A1.SS3.SSS1)describes why we choose CodeGraph, and how CodeGraph is integrated into the two agent frameworks\. Appendix[A\.3\.2](https://arxiv.org/html/2609.30725#A1.SS3.SSS2)reports how agents actually use CodeGraph \(Table[12](https://arxiv.org/html/2609.30725#A1.T12)\)\.

#### A\.3\.1CodeGraph Integration

Why CodeGraph\.We choose CodeGraph as a representative tool of structure\-aware retrieval for two reasons\. First, it is one of the most widely adopted tools of this kind: when we ran our experiments in July 2026, the open\-source repository had 63\.8K stars on GitHub, and its npm package was downloaded 359K times in that month\. Second, it robustly supports integration into both agent frameworks: it ships an MCP server forCCand a CLI forMSA, and it covers 22 languages in the version we use, including Python, JavaScript, TypeScript, and Go, the languages of the SWE\-bench Verified and Pro tasks\.

We use CodeGraph v0\.9\.9\([Mchenry, 2026](https://arxiv.org/html/2609.30725#bib.bib11)\), an open\-source code\-intelligence tool that parses a repository into an AST\-based index of symbols and their relations \(definitions, calls, inheritance, and test coverage\)\. The index is built once per task before the agent starts and is refreshed automatically when the agent edits files\. CodeGraph is read\-only; it never modifies files or runs code\.CCaccesses it through an MCP server that exposes seven functionalities, andMSAaccesses it through two types of shell commands:cgsrc\(read a symbol’s source code by name\) andcg\(locate symbols, trace call relationships, estimate change impact, list files\)\. Both harnesses receive equivalent CodeGraph instructions in the system and task prompts\. The instructions designate CodeGraph as the primary way to locate and read symbols and to trace call relationships, instead of usinggreporfindfor symbol lookup\. Ordinary shell commands and tools remain allowed for non\-symbol text retrieval such as configuration files, documentation, and error\-message strings, and for editing, execution, and other purposes\.

Task coverage on Pro\-100\.CodeGraph cannot run on 13 of the Pro\-100 tasks\. For 11 of them, the task image is built on Alpine Linux, which uses the musl C library, whereas CodeGraph ships a Node\.js binary compiled against glibc, so the binary cannot be executed in these containers\. CodeGraph also fails on the remaining two tasks, one fromansibleand one fromteleport\. We have not identified the specific cause\. We therefore evaluate CodeGraph on the other 87 tasks and compare it with the baseline restricted to the same 87 tasks\. On this subset, the baseline Pass@1 is 53\.26%, 49\.81%, 58\.62%, and 42\.53% forCC,MSAS46,MSAMM3, andMSAQ35\+, respectively\.

Table 11:CodeGraph interfaces ofCC\(MCP tools\) andMSA\(shell commands\), using functionjson\\\_scriptindjango/utils/html\.pyas an example\.FunctionalityCCtool callMSAcommandRead sourcecodegraph\\\_explore\(query="json\\\_script"\)cgsrc"json\\\_script"Locate symbolcodegraph\\\_search\(query="json\\\_script"\)cgquery"json\\\_script"Read callerscodegraph\\\_callers\(symbol="json\\\_script"\)cgcallers"json\\\_script"Read calleescodegraph\\\_callees\(symbol="json\\\_script"\)cgcallees"json\\\_script"Analyze change impactcodegraph\\\_impact\(symbol="json\\\_script"\)cgimpact"json\\\_script"cgaffecteddjango/utils/html\.pyShow file structurecodegraph\\\_files\(path="django/utils"\)cgfiles\-\-filterdjango/utilsCheck index statuscodegraph\\\_status\(\)cgstatus

Table[11](https://arxiv.org/html/2609.30725#A1.T11)maps the two interfaces to the underlying functionalities, using the target functionjson\\\_scriptfrom the taskdjango\-15103as the example\. To read its source code,CCcallscodegraph\\\_explore\(query="json\\\_script"\)andMSArunscgsrc"json\\\_script"; both return the source code of the matching symbols with file paths and line ranges, andcodegraph\\\_exploreadditionally appends the callers, relationships, and dependencies of every match\. To locate a symbol without retrieving the exact context,codegraph\\\_searchandcgqueryreturn only its type, location, and signature\. For call relationships,codegraph\\\_callers/cgcallersandcodegraph\\\_callees/cgcalleesreturn the names and locations of the function’s callers and callees\. To analyze what code is affected by changing a symbol,codegraph\\\_impactandcgimpactreturn the symbols that depend on the target symbol, andcgaffectedlists the test files that cover the given source file\. To show file structure,codegraph\\\_filesandcgfilesreturn the indexed file tree under a directory together with the programming language and the number of symbols in each file\. To check index status,codegraph\\\_statusandcgstatusreport the numbers of indexed files, symbols, relations, and whether it is up to date with the working tree\.

#### A\.3\.2CodeGraph Usage

Table[12](https://arxiv.org/html/2609.30725#A1.T12)reports how agents use CodeGraph on Verified\-200 and Pro\-100\. Agents adopt the tool in 74\.00–97\.50% of tasks\. Reading source accounts for 54\.16–98\.88% of CodeGraph queries and locating symbols for most of the rest \(0\.45–36\.48%\), which together make up 99\.33% and 89\.46% ofCC’s queries on Verified\-200 and Pro\-100 and 80\.84–95\.30% ofMSA’s\. Relationship queries \(callers, callees, and change impact\) receive 0\.56–1\.60% ofCC’s queries and 3\.24–7\.15% ofMSA’s\. File\-structure queries are rare on Verified\-200 \(0\.11–4\.35%\) but more common on Pro\-100 \(7\.32–14\.79%\), consistent with the larger, multi\-language repositories of SWE\-bench Pro\. Index\-status queries are almost never issued\.

Table 12:CodeGraph usage inRQ2, including the percentage of tasks in which the agent issued at least one CodeGraph query, and the share of CodeGraph queries by functionality\.Verified\-200Pro\-100CCMSAS46MSAMM3MSAQ35\+CCMSAS46MSAMM3MSAQ35\+CodeGraph task coverage \(%\)91\.5095\.0097\.5093\.1777\.0085\.0077\.0074\.00Share of CodeGraph queries \(%\)Read source98\.8873\.9769\.0358\.8281\.4763\.8064\.8254\.16Locate symbol0\.4520\.6518\.0736\.487\.9917\.0419\.4631\.51Read callers0\.563\.624\.283\.301\.603\.245\.366\.29Read callees0\.000\.000\.000\.000\.000\.000\.180\.00Analyze change impact0\.000\.492\.580\.450\.000\.001\.610\.47Show file structure0\.111\.274\.350\.958\.9514\.797\.327\.44Check index status0\.000\.001\.700\.000\.001\.131\.250\.14

### A\.4Additional Details for RQ3

In this section, we provide additional details on the agent\-synthesized and developer\-designed skills, including how skills are delivered to the agents, the design of the trajectory analysis agent, the skill synthesis pipeline, and the specific skill contents\.

Skill delivery\.Agent skills\([Anthropic, 2025a](https://arxiv.org/html/2609.30725#bib.bib4)\)are usually loaded on demand: the agent initially sees only the skill descriptions and reads the fullSKILL\.mdonce it selects a certain skill, which keeps the context small when many skills are available\. Our setting involves no skill\-selection decision, because each configuration has a single skill set whose rules apply to every coding task\. Following Trace2Skill\([Ni et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib34)\), we therefore preload the entire skill set into the system prompt\. The preloading is complete rather than partial because our skill sets contain only behavioral rules, with no bundled scripts or resources that would otherwise be discovered on demand\. Keeping the skill set always present also yields a controlled comparison, since the rules are available on every task rather than conditionally loaded\.

#### A\.4\.1Agent\-Synthesized Skills

Trajectory analysis agent design\.For the agent\-synthesized skills inRQ3, we build a lightweight analysis agent that distills corrective rules from each configuration’s RQ1 trajectories using its own backbone model\. For each trajectory, the agent receives the task outcome, monetary cost, and step count, together with the RQ1 diagnoses, including the flagged cost\-inefficient behavior types, their frequencies, and the flagged step indices\.

Table 13:Toolset of the trajectory\-analysis agent\.ToolReturnsget\_skeletonStep indices, action labels, and truncated actions for a trajectory overview\.get\_step\(\[ids\]\)Full thoughts, commands, and command outputs for selected steps\.get\_gt\_patchThe ground\-truth patch of the task\.get\_agent\_submissionThe agent’s final submitted patch\.Table 14:Cost of synthesizing skills\.Agent,One\-time, andMergeare the total costs of the agent analysis, the single\-call analysis, and the hierarchical consolidation;Total: the total cost of synthesizing skills;Task cost: the cost of solving all 300 RQ1 tasks;Ratio: Total / Task cost\.ConfigAgent \($\)One\-time \($\)Merge \($\)Total \($\)Task cost \($\)RatioCC20\.331\.780\.6522\.76156\.0014\.59%MSAS4634\.482\.031\.0837\.59166\.0622\.64%MSAMM33\.721\.650\.495\.86109\.505\.35%MSAQ35\+2\.921\.230\.124\.2730\.6013\.95%

Table 15:Effects of three approaches on Pass@1, cost, and CoP\. SynSkills is reported with and without synthesis cost \(Table[15](https://arxiv.org/html/2609.30725#A1.T15)\) amortized over Verified\-200 and Pro\-100 \(\+ synthesis\)\.green/redindicate robust improvements/regressions, whilegrayindicates changes within the run\-to\-run noise floor; darker shades indicate larger changes\.Verified\-200Pro\-100ApproachMetricCCMSAS46MSAMM3MSAQ35\+CCMSAS46MSAMM3MSAQ35\+CodeGraphPass@1 \(pp\)74\.17\-0\.8372\.50\+0\.0075\.50\+0\.3372\.50\+5\.3355\.17\+1\.9251\.72\+1\.9257\.47\-1\.1539\.08\-3\.45Cost \($\)0\.596\+8\.30%0\.619\-3\.03%0\.591\+28\.14%0\.112\-3\.29%1\.061\+12\.19%1\.132\-0\.25%0\.985\+8\.39%0\.150\+12\.99%CoP0\.804\+10\.22%0\.854\-3\.04%0\.782\+27\.52%0\.155\-10\.63%1\.923\+8\.69%2\.188\-4\.32%1\.715\+10\.20%0\.388\+23\.20%SynSkillsPass@1 \(pp\)74\.50\-0\.5072\.00\-0\.5075\.50\+0\.3369\.67\+2\.5055\.00\+0\.6752\.00\+0\.0062\.00\+2\.0046\.00\+1\.00Cost \($\)0\.541\-1\.38%0\.496\-22\.32%0\.462\+0\.15%0\.115\-0\.70%0\.826\-8\.86%1\.006\-6\.13%0\.766\-11\.90%0\.132\-2\.22%CoP0\.726\-0\.12%0\.689\-21\.79%0\.611\-0\.33%0\.166\-4\.40%1\.503\-9\.48%1\.936\-6\.40%1\.235\-15\.09%0\.286\-4\.51%SynSkills\+ synthesisPass@1 \(pp\)74\.50\-0\.5072\.00\-0\.5075\.50\+0\.3369\.67\+2\.5055\.00\+0\.6752\.00\+0\.0062\.00\+2\.0046\.00\+1\.00Cost \($\)0\.617\+12\.51%0\.621\-2\.70%0\.481\+4\.39%0\.130\+11\.55%0\.902\-0\.49%1\.132\+5\.78%0\.785\-9\.65%0\.146\+8\.48%CoP0\.828\+13\.88%0\.863\-2\.04%0\.637\+3\.88%0\.186\+7\.38%1\.641\-1\.18%2\.176\+5\.52%1\.266\-12\.93%0\.317\+6\.01%DevSkillsPass@1 \(pp\)73\.50\-1\.5073\.00\+0\.5073\.00\-2\.1769\.83\+2\.6752\.00\-2\.3352\.00\+0\.0062\.00\+2\.0046\.00\+1\.00Cost \($\)0\.473\-13\.94%0\.372\-41\.73%0\.387\-16\.03%0\.103\-11\.34%0\.894\-1\.51%0\.733\-33\.00%0\.801\-7\.88%0\.140\+3\.57%CoP0\.643\-11\.83%0\.510\-42\.13%0\.530\-13\.61%0\.148\-14\.83%1\.718\+3\.58%1\.409\-33\.19%1\.291\-11\.21%0\.305\+1\.28%

Equipped with four read\-only tools \(Table[13](https://arxiv.org/html/2609.30725#A1.T13)\), the agent verifies flagged actions against the actual step content in the trajectory and finally returns a list of skill candidates, each pairing a confirmed cost\-inefficient anti\-pattern with a corrective rule\. The agent runs as a ReAct loop\([Yao et al\., 2023](https://arxiv.org/html/2609.30725#bib.bib59)\)at temperature00, with a limit of8,1928\{,\}192tokens per call and5050steps per trajectory\.

Skill synthesis pipeline\.Running this agent on every trajectory is costly\. In preliminary experiments onMSAS46, the agent tends to read each flagged step and its surrounding context in detail\. The number of steps the analysis agent consumes is close to that of the original task\-solving trajectory\. As a result, the per\-trajectory analysis cost is close to the cost of solving the task itself\.

To control this cost, we pair the agent with a cheaper single LLM call\. We rank each configuration’s trajectories by the cost share of their diagnosed cost\-inefficient behaviors, analyze the top 25% with the agent, and process the remaining affected trajectories with a single LLM call, one trajectory at a time\. Beyond the inputs given to the agent, the single\-call mode additionally receives a compact representation of the trajectory, in which each step is reduced to its action label together with its thought, action, and output truncated to their first and last lines\. Finally, we apply the Trace2Skill\([Ni et al\., 2026](https://arxiv.org/html/2609.30725#bib.bib34)\)hierarchical consolidation to merge the per\-trajectory rules into one skill set per configuration\.

Table[15](https://arxiv.org/html/2609.30725#A1.T15)reports the synthesis cost of each configuration against the cost of solving the corresponding 300 tasks in RQ1\. Table[15](https://arxiv.org/html/2609.30725#A1.T15)amortizes this cost over the 300 evaluation tasks to the per\-task cost of SynSkills, and compares the effects of the three mitigating approaches\. Once amortized, the cost advantage of SynSkills largely disappears: the only robust cost reduction that remains isMSAMM3on Pro\-100 \(\-9\.65%\)\.MSAQ35\+turns into robust increases on both benchmarks \(\+11\.55% and \+8\.48%\), and no setting keeps a robust CoP improvement\. Two large savings in Table[3](https://arxiv.org/html/2609.30725#S3.T3),MSAS46on Verified\-200 \(\-22\.32%\) andCCon Pro\-100 \(\-8\.86%\), shrink to\-2\.70%and\-0\.49%because these two configurations also have the most expensive synthesis cost \($37\.59 and $22\.76\)\. Because DevSkills incur no synthesis cost, they reduce cost more than the amortized SynSkills in seven of the eight settings, the exception beingMSAMM3on Pro\-100 \(\-7\.88%versus\-9\.65%\)\. However, the skill\-synthesis surcharge scales inversely with the number of applied tasks\. It would fall below 2\.00% of the baseline cost if applied over 3,000 tasks\. So the amortized picture depends on the deployment scale rather than on the skills themselves\.

Agent\-Synthesized Skill Content\.

Claude Code:

\#\#Validation&Testing

\-Runeachtestcommand\*\*oncepercodestate\*\*\.Neverre\-issueanidenticalcommandwithoutaninterveningcodeorenvironmentchange\.

\-Beforerunninganytestsuite,spendonesteplocatingthecanonicaltestrunner\(check‘runtests\.py‘,‘pytest\.ini‘,‘tox\.ini‘,‘Makefile‘,‘README‘,orCIconfig\)andthecorrectworkingdirectory\.Usetherightinvocationonthefirsttry\.

\-Whenacommandfails,readthefullerror,diagnosetherootcauseonce,makeaconcretecorrectivechange,thenretryexactlyonce\.Neverre\-issueanunchangedfailingcommand\.

\-Afterabroadersuitepasses,skipre\-runninganysubsetitalreadycovers\.

\-Capturefulltestoutputinasinglerun\(e\.g\.,verboseflagsor‘tee‘\),thenapplymultiplegrep/analysisstepstothatcapturedresultratherthanre\-executingforeachfilter\.

\-Whenatestfailureisconfirmedaspre\-existing,out\-of\-scope,orenvironmental,donotre\-runoverlappingsubsetstoconfirmthesameoutcome\.Movedirectlytoadifferentapproach\.

\-Onceapermissionbarrierisconfirmed\(e\.g\.,fileownedbyroot,nosudo\),stopprobingit\.Immediatelypivottoknownalternativewritepaths\.

\#\#Reading&Navigation

\-Readeachsourcefileorregion\*\*onlyoncepersession\*\*\.BeforeissuinganyReadorGrepaction,checkwhetherthetargetcontentwasalreadyretrievedandthefileisunchanged—reusecachedcontent\.

\-Consolidateallrequiredcontextintoasingleread,orusetargetedgreptofindneededlinenumbersfirst,thenreadonce\.Onlyre\-readwhenthefilehasbeenexternallymodifiedorapreviouslyunreadsectionisneeded\.

\-Alwaysread\(orconfirmyouhavealreadyread\)afilebeforeattemptingtoeditit\.Agrep/snippetresultaloneisinsufficient—readtherelevantsectionbeforeissuinganedit\.

\-Whenmultiplegitcommandsreturnnomeaningfuloutput,stopusinggit\.Switchimmediatelytoreadingsourcefilesandrunningreproductionscripts\.

\#\#Editing

\-Beforemakinganycodeedit,fullytraceallrelevantdatatypes,branches,andcallsites,thenwritethecompletecorrectreplacementinoneedit\.Avoidincrementalmicro\-editsthateachleavethecodebroken\.

\-Beforewritinganypatch,reasonthrough\(a\)theexactcallsitethatcanreceiveinvalidinput,\(b\)theminimalinvariantthatguardsit,and\(c\)whetherfixingthecallervs\.calleeissafer\.Committooneapproachandimplementitfully\.

\-Editsourcefilesin\-placeusingtheagent’sbuilt\-inEdit/Writetool,‘sed‘,or‘patch‘ratherthancreatingintermediatecopiesornumberedvariants\.

\-Whenusingstr\_replaceorpatch,ensurethetargetstringexactlymatchesexistingcontent\.Afterapplyinganypatch,immediatelyviewthemodifiedregiontoconfirmcorrectnessbeforerunningtests\.

\-Whenalibraryclassorsymbolisalreadyconfirmedbypriorcodebaseanalysis,skipaliveimport\-checkcommand\.Trustprioranalysisandproceeddirectlytoediting\.

\#\#Script&ArtifactHygiene

\-Prefertheproject’sexistingtestsuiteoverad\-hocverificationscripts\.Onlycreateacustomreproductionscriptifnoexistingtestcoversthespecificscenario\.

\-Beforewritinganyscript,searchtheexistingcodebasefirst\(grepforsimilarmethods,checkexistingtests,inspectthetargetfunction’ssignature\)\.Existingcodeusuallyrevealsthecorrectapproachdirectly\.

\-Designa\*\*singlecomprehensive\*\*diagnosticorreproductionscriptupfrontcoveringallkeyscenarios\(basiccase,edgecases,regression\)\.Verifyit:\(a\)importsexist;\(b\)exercisesthecodeunderinvestigation;\(c\)outputdistinguishes"bugpresent"from"bugfixed"\.Overwriteinplaceratherthancreatingnumberedvariants\.

\-Writemulti\-lineverificationscriptstoanamedtemporaryfile\(e\.g\.,‘/tmp/repro\.py‘\)andexecutethatfileratherthanusinginline‘python\-c‘commands\.Neverplaceitinadirectoryon‘sys\.path‘ornameitafterastandardlibrarymodule\.Deleteitafteruse\.

\-Whenascriptfailsforanenvironmentalreason\(missingmodule,wrongpath,missingsetup\),fixtheissueinthesamescriptratherthancreatinganewvariantfile\.

\-Useproperassertions\(e\.g\.,‘assertresult==expected‘\)inverificationscriptsratherthanprintingandvisuallyinspectingoutput\.

\-Donotrunscriptstoconfirmfactsalreadyevidentfromreadingsourcecode,ortore\-confirmbehavioralreadyvalidatedbyapassingtestsuite\.

\-Donotwriteintermediateanalysisornotestotemporaryfilesjusttoreadthembackimmediately\.Keepsuchinformationinreasoningcontext\.

\-Whenafile\-readordirectory\-listingreturns"Nosuchfileordirectory",verifythedirectoryexistsfirst;ifitdoesnot,abandonthatpathentirelyratherthanretryingwithdifferentfilenames\.

Mini\-SWE\-Agent \(Sonnet 4\.6\):

\#\#RepositoryNavigation&Orientation

\-Beforeinvokinganytestrunner,inspecttherepositorystructureonce\(README,CIconfig,Makefile,tox\.ini,pytest\.ini,runtests\.py\)todeterminethecanonicalinvocationandworkingdirectory\.

\-Whenacommandfails,diagnosetherootcausebeforeretrying:distinguishenvironment/configerrorsfromactualtestfailures\.

\-Whensourcecodeinspectionmakesbehaviourclear,skipwritingexploratoryscriptstoconfirmit;readmodulesourcedirectlywith‘grep‘/‘sed‘insteadoflaunchinganinterpreter\.

\-Onceadecisivecheckconfirmstherequiredstate,stopinvestigatingandreportthefinding;donotre\-examinethesameevidencefromdifferentangles\.

\#\#Editing&Patching

\-Forsmall,localizededitsusetargetedin\-placetools\(‘sed\-i‘withlinenumbers,‘patch‘,‘awk‘,minimalheredoc,orprecise‘str\_replace‘\)ratherthanread\-entire\-file/write\-entire\-filecycles\.

\-Beforeanysubstitution,verifytheexacttargetstring\(e\.g\.,‘cat\-A‘or‘print\(repr\(\.\.\.\)\)‘\)toavoidsilentno\-ops;iftworeplacementattemptsbothno\-op,switchtoafundamentallydifferentmethod\(line\-number\-based‘sed‘/‘patch‘\)\.

\-Afterapplyingapatch,verifywithasingletargetedread\(‘grep\-n‘,‘sed\-n’START,ENDp’‘,or‘gitdiff‘\)—notafullfilere\-read\.

\-Reservefull\-filerewritesforchangesscatteredthroughoutthefile\.

\-Beforemakinganyspeculativechange,verifywithtargeted‘grep‘/‘read‘thatthechangeisactuallyneededandthatthepatternexists\.

\-Donotrevertacodechangeunlessaconcretetestorexecutionresultshowsitiswrong;neverrevertspeculatively\.

\-Ifafileisaccidentallyemptiedorcorrupted,immediatelyrestoreitfromversioncontrol\(‘gitcheckout\-\-<file\>‘\)\.

\-Avoidsplittingfilesintomultipletemporarypartstoinsertasmallchange;useasingleatomicedittopreventquotingpitfallsandpartial\-writecorruption\.

\#\#Script&ArtifactHygiene

\-Writeonepersistentrepro/verificationscriptatthestartofdebugging;consolidateallexploratory,diagnostic,andverificationlogicintoit;edititinplaceratherthancreatingnumberedvariants\.

\-Beforecreatinganynewscript,checkwhetherthedesiredchangeisalreadypresent\(‘grep‘/‘sed‘\);ifso,skipthescriptentirely\.

\-Forshorttasks,prefer‘python3\-c‘one\-linersorheredocinvocationsovercreatingnamedscriptfiles\.

\-Wheniteratingonascriptorpatch,overwritethesamefile—nevercreatenumberedvariants\.

\-Executeascriptimmediatelyafterwritingit;donotoverwriteitmultipletimeswithoutrunningit\.

\-Whenareproductionscriptfailstotriggerthebug,diagnosetherootcause\(re\-readtheissue,inspectthecodepath\)beforerewriting—writeonecomprehensivescriptcoveringallcases\.

\-Usedirectshellcommands\(‘cat‘,‘sed‘,‘grep‘,‘gitdiff‘\)toinspectcontentordisplaydiffsratherthanwritingatemporaryPythonscript\.

\-Whenatestfixtureorconfigurationfails,fixtheexistingfixtureinplaceratherthancreatingaparallelone;reuseexistinghelpers\.

\-Fornon\-ASCIIcontentinshell,useheredocsyntaxoratempscriptfilewith‘PYTHONIOENCODING=utf\-8‘ratherthaninline‘\-c‘commands\.

\#\#Validation&Testing

\-Runtherelevanttestsuiteexactlyoncepercode\-changestate;onlyre\-runafteraninterveningcodechange\.

\-Capturesufficientdiagnosticinformationinasinglewell\-formedinvocation\(pipeto‘tee‘,use‘grep\-E‘forsummarylines,oruse‘\-v‘\)sopass/failstatusanddetailsarevisibleatonce\.

\-Runthefix\-verificationscriptoncebeforethefix\(toconfirmthebug\)andonceafter\(toconfirmresolution\);donotre\-rununchangedtestsforreassurance\.

\-Beforerunningaverificationscript,ensurethefixiscompleteandhandlesallknownedgecases;enumerateedgecasesfirst,thenimplementandtestonce\.

\-Whenalargetestsuiteconsistentlytimesout,stopretryingwithminorflagvariations;runtargetedsubsetsrelevanttothechangeinstead\.

\-Whenverifyingpre\-existingfailures,asingle‘gitstash→\\rightarrowrun→\\rightarrowgitstashpop‘cycleissufficient;recordthebaselineonceandcompareagainstit\.

\-Beforerunningareproductionscript,mentallytracetherelevantlogicend\-to\-endandfixallvisibleerrorsbeforethefirstexecutionratherthaniterativelypatchingandre\-running\.

\-Beforere\-readingafileorregion,checkwhetherithasbeenmodifiedsincethelastread;ifunchangedandstillincontext,skipthere\-readentirely\.

Mini\-SWE\-Agent \(MiniMax\-M3\):

\#\#Validation&testexecution

\-Planeverytest,search,orverificationcommandupfront:pickverbosity,outputformat,andfilterflagsonce,thenrunitonceandcapturefulloutput\(redirect,tee,oragenerouswindow\)\.

\-Composeinvocationwithfilteringinasinglerun\(‘\.\.\.\|grep\.\.\.‘,‘\.\.\.\|tail\-n\.\.\.‘,orsummaryflags\)ratherthanfull\-suiterunsfollowedbyre\-runswithdifferentpost\-processing\.

\-Batchmultiplerelatedtargetsintooneinvocation;forlongruns,redirectfulloutputtoafileonceandapplyallsubsequentfilterstothesavedlog\.

\-Trustpassingresults:skippost\-fixsanityprobes\(importchecks,identitychecks,unrelatedmodulereloads\)thatdon’texercisethechangedbehavior\.

\-Don’tre\-invokethesamesuiteundercosmeticflagchanges,differentfilters,or"justtoconfirmagain\."Re\-executeonlywhenunderlyingcodeorinputshavemateriallychanged\.

\-Scoperegressionverificationtomodulesthatdirectlyexercisethechangedpathplusasmallsetofclearlyrelatedones;deferbroadercoveragetotheproject’sownCI\.

\#\#Scripts&reproductionartifacts

\-Maintainonecanonicalreproduction/verificationscriptperdebuggingconcern;evolveitinplace—appendcases,parametrizeviaflags/argv,refineassertions—insteadofspawningnumberedvariants\.

\-Persistreusablereproductionlogicinasinglesmallfileonfirstuse,thenre\-executethatfileratherthanfightinginline‘python\-c‘invocationswithre\-importsandre\-quoting\.

\-Consolidateexploratoryprobes,verificationchecks,andrelatedcasesintoonecomprehensivescriptoronebatchedinvocationfromthestart;reserveinlinecommandsforgenuinelydisposablesingle\-statementchecks\.

\-Ifaheredoc\-basedscriptkeepsfailingonquoting,abandonitforasimplerreproductionratherthandebugginginvisiblecharacters\.

\-Designreproscriptstobeself\-containedandleverageexistingprojectinfrastructure\(in\-memorysetup,inlinemodeldefinitions,temporarytestmethods\)insteadoffragilestandalonescriptsthatdependonseparately\-wiredappmodulesorsys\.pathhacks\.

\-Cleanuptempfilesattheendofthetask\.

\#\#Filereading&searchcaching

\-Treatcontentalreadyreadordiffedascachedfortherestofthesession:neverre\-cat,re\-sed,orre\-grepthesameunchangedregion\.

\-Relyon‘gitdiff‘/‘gitshow‘astheauthoritativesourceforpost\-editverification;preferonetargeteddifforonenarrowreadoverre\-cattingthepatchedregion\.

\-Onlyre\-fetchafteraconfirmededit,anexternalchange,truncation,oragenuinelynewlinerange\.

\-Onceasearchenumeratesandcategorizesitsresults,recordtheconclusionandmoveon;ifagrepyieldsnothingrelevant,pivottoreadingthecurrentcodeorrunningthefailingtestinsteadofrepeatingwithminorkeywordvariations\.

\#\#Editingapproach

\-Forlocalizededits\(aflag,import,block,ormethod\),applyasurgicalin\-placepatch—narrowanchor\-basedreplace,‘sed\-i‘,oradirectedittool—ratherthanreconstructingthewholemoduleviaheredocorfull\-fileoverwrite\.

\-Planthechangebycheckingsurroundingcontextsotheeditlandsinonepasswithoutfollow\-uprewrites;batchmultiplecoordinatededitsintooneatomicpatchanddry\-runagainsttheoldcontent\.

\-Reservefull\-filerewritesforgenuinelynewfilesortransformationsthatgenuinelyspanthefile\.

\-Onasmallfollow\-uperror,surgicallyfixthedifferinglineinplaceinsteadofrestartingtheedit\.

\-Whenthefixapproachisstilluncertain,prototypeinascratchbufferbeforecommittingtotrackedsourcesospeculativeeditsdon’ttriggercostlyround\-trips\.

\#\#Failurerecovery&toolingstrategy

\-Afterasmallnumberoffailedattemptsatthesameedit,command,orscript,stopiteratinganddiagnosetherootcause\(wrongpath,missingmodule,environmentissue,escapingproblem\)viatargetedlookupbeforeretrying\.

\-Staticallyanalyzethecodepathorreadthefullerrortracetoidentifyallrootcausesupfrontandfixtheminonerevision;avoidtherun→\\rightarrowsee\-one\-error→\\rightarrowfix\-one\-error→\\rightarrowrun\-againcycle\.

\-Ifaneditreportsnochangeorfailsidentically,committoadifferentstrategyonthefirstfailureratherthanre\-issuingthesamecommand\.

\-Switchtooling\(cleanheredoc,environmentflagslike‘PYTHONIOENCODING‘\)ratherthanre\-quotingthesameproblematicinvocation\.

\-Completeeacheditandscratchscriptinasingledirectpass;don’toscillatebetweenwrite→\\rightarrowrun→\\rightarrowrevert→\\rightarrowwrite\-newloopsoriterativelyoverwriteascratchfiletofixtypos—adjustthescript’scontentitself\.

\#\#Explorationdiscipline

\-Beforebroadsearches\(filesystem\-widegrep/find,githistoryarchaeology,APIlookups\),verifythehypothesiswithcheaplocalsignals:targeted‘gitblame‘,file\-scoped‘gitlog‘,project\-conventioncheck,orasinglefailingassertion\.

\-Caphistoryspelunkingatafewfocusedqueriesandboundinitialexplorationtowhat’sneededtopicktheedit;deferdeeparchitecturalinvestigation\(MRO,classhierarchies,dundermethods,modulemaps\)untilafterafirstattemptifthesimplerfixdoesnotsuffice\.

\-Capbroadsweeps\(wide‘gitlog\-\-grep‘,date\-rangeorkeywordvariations\)tooneortwoattemptsbeforeswitchingstrategy;stopsearchingoncethediffhasmadethefixclear\.

\-Onceafixisimplemented,verifiedagainstarepro,andpassingtherelevanttests,leaveitinplace—don’tspendstepsgreppingforticketnumbersorupstreamcommits,anddon’trevertaworkingchangetore\-deriveitfromscratch\.

\#\#Scope&projectconventions

\-Definethescopeofexploration,edits,andoptionalartifacts\(changelog,docs,supplementaryfiles\)beforestarting;avoidapplying\-then\-revertingspeculativechanges\.

\-Capsetup\-onlyoperationsatoneattempteach\(singlestash/popbaseline,singlesleepsizedforworstcase\)andreusethecapturedsignalforsubsequentdecisions\.

\-Beforeauthoringreproornewtests,surveyexistingprojectpatterns:testlayout,bootstrapconventions,runnerconfig,andanypriorreproducerforthesamecodepath\.

\-Readexistingtestscoveringthebehaviorbeingchanged\(includinginvalid\-input/negativecases\)andinspectrelevantsourcetoconfirmattributenames,APIshape,andrequiredimports—soiterationfocusesonbugbehaviorratherthan‘AttributeError‘/missing\-registrationloops\.

\-Whenathird\-partypackageseemsrequired,firstcheckwhetheritisalreadyinstalledorwhetheraminimalin\-repostubcanexercisethesamecodepath\.

\-Whenataskexplicitlyforbidsmodifyingtests,directalleffortatthesource\-codefixplusminimalverificationthattheexistingsuitestillpasses;don’tinvestcyclescomputingcorrectedassertionvaluesorapplying\-then\-revertingtestedits\.

\#\#Outputhandling

\-Whenoutputsuchasadifforcommandresultmustbebothinspectedandpersisted\(e\.g\.,apatchdeliverable\),doitinasinglecommandusing‘tee‘or‘\>file&&catfile‘,thenreferencethesavedfileatsubmission\.

\-Don’twriteacommand’soutputtoafilejusttoreaditbackintocontext—referencethein\-contextdisplaydirectly;onlypersistartifactsactuallyneededforfinalsubmission\.

\#\#API&mockinghygiene

\-WhenapublicAPIexposesacallable\-basedextensionparameter\(hook,keep\_attrs\-style\),preferpassingthecallableovermanuallyreorderingpositionalargumentsorpost\-processingtheresult—thecallableformisthedocumentedextensionpointandavoidscascadingsideeffectsoninferredoutputmetadata\.

\-Beforewritingareproductionthatmocksorstubsaclass,readthetargetmethodonceandenumerateeveryattributeanddunderitreferences;onetargetedreadeliminatesmulti\-attemptthrashingcausedbymissingprotocolmethodsdiscoveredafterthefact\.

\#\#Versioncontrolhygiene

\-Useselectiveversion\-controloperationsthattargetonlythefilesintendedtochange;avoidbroadpatternsthatmaysilentlydiscardoroverwriteunrelatedpriorworkonothermodifiedfiles\.

Mini\-SWE\-Agent \(Qwen\-3\.5 Plus\):

\#\#Context&Navigation

\-Cacheretrievedfilecontent,searchresults,andverifiedpatterns;referencethiscontextinsteadofre\-readingunlessthefilehasbeenmodifiedoraspecificmissingsectionisrequired\.

\-Navigatedirectlytotargetlinesorusecomprehensivesearchqueriesimmediately,avoidingpreliminaryreadsofunrelatedsectionsorfragmentedsearches\.

\-Consolidateverificationintoasinglereadoperationafterapplyingmultiplesequentialeditstothesameregion,ratherthanreadingaftereachindividualchange\.

\-Usetargetedsearchcommandswithspecificpatternsinsteadofbroadlylistingfilesorinvestigatingfeaturesnotstrictlyrequiredbythetask\.

\#\#Editing&ArtifactHygiene

\-Modifyexistingfilesin\-placeusingtargeted,surgicaledits\(e\.g\.,line\-specificreplacements\)insteadofrewritingentirefilesorrecreatingthemfromscratch\.

\-Consolidateallrelatedtestscenarios,investigations,andfixesintoasinglefile,avoidingthecreationofmultiplenumberedvariants,separatescripts,orsequentialpatchartifacts\.

\-Applycodemodificationsdirectlyusingin\-placeeditingtoolsinsteadofcreatingtemporarywrapperscripts,intermediatepatchfiles,ornumberedvariants\.

\-Preferdirectprogrammaticeditingoverchainedshellcommandsforcomplexchanges;ifacommandfails,analyzetheerrororfallbacktomanuallogicimmediatelyratherthanretryingbrokensyntax\.

\-Plancompletedirectorylayoutsandspecifyexplicittextencodingwhencreatingscriptstogenerateallrequiredfilesinasinglepass\.

\-Verifyfilesaretrackedbyversioncontrolandrelevanttothedeliverablebeforemodifying,skippingbuildartifacts,cachedirectories,oruntrackedfiles\.

\-Avoidunnecessarygitoperationslikestash/popcycleswhentheworkingstateiscorrect,andreviewthefinaldiffexactlyonceafterallchangesarecomplete\.

\#\#Validation&Testing

\-Executevalidationcommandsonlyoncepercodestateusingminimal,focusedtestcasesthatdirectlyvalidatespecifichypotheses,capturingfulloutputtoavoidre\-execution\.

\-Consolidatemultipletestscenarios,edgecases,verificationvariants,anddiagnosticdetailsintoasinglecomprehensivecommandorscriptexecution\.

\-Whenvalidationfails,diagnosetherootcauseandextractallnecessaryinformationfromthatsingleruntofixerrorsimmediately,ratherthanattemptingmultiplesimilarinvocations\.

\-Establishabaselineforpre\-existingfailuresandverifyenvironmentdependenciesearlybyrunningthefulltestsuiteoncebeforeapplyingfixes\.

\-Skipre\-runningvalidation,revertingchanges,orrepeatingtestsiftheunderlyingcodehasnotchanged,thestateisconfirmedviainspection,orstaticverificationsuffices\.

\-Onceanissueisreproducedortherootcauseidentified,proceeddirectlytofiximplementationwithoutadditionalreproductionattemptsorconsecutiveverificationsteps\.

\-Optimizesearchandverificationbycombiningrelatedpatternsintosingleoperations,scopingchecksstrictlytomodifiedfiles,andtrustingtargetededitssucceededunlessanerrorisreported\.

\-Specifyexplicitencodingsoruseautomatichandlingtoolsfromthefirstattemptwhenprocessingsourcecodetopreventencoding\-relatedretries\.

\#\#DiagnosticEfficiency

\-Prioritizestaticinspectionofsourcecode,diffs,andenvironmentfactorstoanswerquestionsbeforeexecutingad\-hocdiagnosticcommandsorcreatingdebugartifacts\.

\-Consolidatemultipleexploratorychecksanddiagnosticcommandsintosinglecomprehensivescriptsorbatchedinvocations,removingtemporaryartifactsimmediatelyafteruse\.

\-Explicitlydocumentkeyfindingsfromfailediterationstopreventrepeatingunproductivepatterns\.

\-Verifytheexistenceandaccessibilityofallrequiredoutputartifactsandensurethetestenvironmentisfullyinitializedbeforeinitiatingfinalsubmission\.

#### A\.4\.2Full Text of the Developer\-Designed Skills

The developer\-designed skills are listed below\.

1\.Hypothesis\-Before\-Read

Beforeeveryfileread,stateinoneconcisesentence:Whatyouexpecttofindandwhy\.Ifyoucannotstateaspecificexpectation,reasonfromexistingcontextfirst\.Donotreadblindly\.

2\.ReuseContext

Beforereadingcontent\(‘grep/cat/sed/\.\.\.‘\),checkwhetheritisalreadyavailableinthecurrentcontext\.Ifyes,reuseitinsteadoffetchingitagain\.Re\-readonlywhennecessary,andbrieflystatewhy\(e\.g\.,"confirmingexactstringbeforeedit","contentistoofarbackincontexttorelyon"\)\.Avoidissuingmultipleoverlappingreadstoretrievethesamecontextviadifferenttools\.

3\.TargetedPatchOnly

Whenmodifyingsourcecode:

\-\*\*DONOTuse\*\*:‘cat\>file<<’EOF’‘or‘open\(f,’w’\)\.write\("""\.\.\."""\)‘tooverwriteanentireexistingfile\.

\-\*\*Use\*\*:Targetedediting,suchas‘sed\-i‘orcreatingyourownpythonscript\(‘content\.replace\(old\_substr,new\_substr\)‘,triple\-quotedformulti\-line\)\.

Writetheminimalchange\.Identifytheexactoldstringandnewstringbeforepatching\.

4\.UnderstandtheTestHarnessBeforeBlindlyRunningTests

Identifytherepositorytestmechanismbeforerunningtests/afterrunningtestswithwrongcommands:

\-Checkexistingdocumentation/CIconfigs/testscriptsforthecorrectrunner,environmentvariables,andtestsyntax\.

\-Treatimport,configuration,dependency,andtest\-loaderfailuresasharnesserrors,notpatchfailures\.

Skipthisifyoualreadyknowtheworkingcommand\.

5\.CaptureTestOutput;Filter,Don’tRerun

Whenrunningatest,redirectfulloutputtoafile,suchas:

‘‘‘

<test\_command\>\>test\_out\.txt2\>&1

‘‘‘

Extracttestsignalpreciselyfromapriorruninsteadofre\-runningthetest\.Onlyrerunifthesourcecodehaschangedsincethelastrun\.

6\.PersistArtifacts;VersionOnlyWithIntent

Whencreatingreproduction,test,orpatchscripts,savereusableormulti\-steplogicasaworkspacefile;useinlinecommandsonlyforsimple,one\-offprobes\.Donotcreatenumberedvariantssuchas‘repro2\.py‘or‘fix\_v3\.py‘;revisetheoriginalartifactunlessseparateversionsarenecessary\.Statethereasonbeforecreating\.

7\.BreakLoops

Whenrepeatedreads,tests,oreditsproducenonewevidence,\*\*Stop\.\*\*Youmaybestuck\.Donotrepeatthesameactionandstatewhatremainsuncertain,formaconcretehypothesis,thentakeonemateriallydifferentactiontomoveon\.

Ifyouarestuckonsub\-problems\(e\.g\.,verificationissueslikewrongtestenvironmentsetup\),submityourbestcurrentsolution\.Acorrectpatchyoucannotfullyverifyisbetterthanabudgetexhaustedwithoutsubmission\.