Why Retrying Fails: Context Contamination in LLM Agent Pipelines

# Why Retrying Fails: Context Contamination in LLM Agent Pipelines
Source: [https://arxiv.org/html/2605.08563](https://arxiv.org/html/2605.08563)
11institutetext:Department of Computer Science, Rutgers University, Piscataway, NJ 08854, USA11email:zy306@scarletmail\.rutgers\.edu###### Abstract

When an LLM agent fails a multi\-step tool\-augmented task and retries, the failed attempt typically remains in its context window—contaminating the next attempt and elevating the per\-step error rate beyond the base level\. This*context\-contaminated restart*phenomenon is widely observed in practice yet entirely lacks formal treatment\. We introduce theContext\-Contaminated Restart Model \(CCRM\): a chain ofTTtool\-call steps, each failing with base rateε0\\varepsilon\_\{0\}; after any failed attempt, the subsequent attempt operates in contaminated context with elevated error rateε1\>ε0\\varepsilon\_\{1\}\>\\varepsilon\_\{0\}\. Under this model we derive five main results\.\(R1\)An exact closed\-form formula forP​\(succeed in≤K​attempts\)P\(\\text\{succeed in\}\\leq K\\text\{ attempts\}\)\.\(R2\)A cascade\-overhead theorem giving the additional attemptsΔ​K\\Delta Kincurred by contamination versus the clean\-restart baseline\.\(R3\)An optimal budget\-allocation theorem identifying the pipeline depthT∗T^\{\*\}that maximises success probability for a fixed total budgetB=K​TB=KT; we prove the closed formT∗=B⋅log⁡\(1/\(1−ε1\)\)log⁡\(1/\(1−ε0\)\)T^\{\*\}=\\sqrt\{B\\cdot\\frac\{\\log\(1/\(1\-\\varepsilon\_\{1\}\)\)\}\{\\log\(1/\(1\-\\varepsilon\_\{0\}\)\)\}\}, withK∗=B/T∗K^\{\*\}=B/T^\{\*\}\.\(R4\)An information\-theoretic lower bound via Le Cam’s method showingKCCRMK\_\{\\mathrm\{CCRM\}\}is tight up to𝒪​\(1\)\\mathcal\{O\}\(1\)\.\(R5\)A clean\-restart dominance theorem quantifying the exact benefit of context\-clearing before retry\. We validate CCRM on real SWE\-bench Verified data: the IID model overestimates pass@3 by 17\.4 percentage points \(98\.6% vs\. 81\.2%\), while CCRM fits with error less than 0\.001, implying a cascade ratio ofε1/ε0=7\.1\\varepsilon\_\{1\}/\\varepsilon\_\{0\}=7\.1\. Monte Carlo experiments confirm all theoretical predictions\.

## 1Introduction

Tool\-augmented LLM agents execute tasks by planning and invoking external APIs, web search engines, code interpreters, and databases\[[9](https://arxiv.org/html/2605.08563#bib.bib9),[6](https://arxiv.org/html/2605.08563#bib.bib6),[4](https://arxiv.org/html/2605.08563#bib.bib4),[8](https://arxiv.org/html/2605.08563#bib.bib8)\]\. When such an agent fails a complex task, it typically retries—but the failure is preserved in its context window as history\. Practitioners have documented the consequence: “early incorrect attempts remained in the conversation history and contaminated the final response”\[[25](https://arxiv.org/html/2605.08563#bib.bib25)\]; agents “repeatedly referenced the same bad endpoint in future attempts because it had learned from its own mistake”\[[25](https://arxiv.org/html/2605.08563#bib.bib25)\]\. Datadog’s 2026 engineering survey reports that 5% of all LLM call spans return errors\[[22](https://arxiv.org/html/2605.08563#bib.bib22)\], yet no formal theory quantifies how retry contamination scales with task depth, error rate, or budget\.

Existing theoretical work on agent reliability studies single\-run reliability\[[28](https://arxiv.org/html/2605.08563#bib.bib28),[26](https://arxiv.org/html/2605.08563#bib.bib26)\]or empirical failure catalogues\[[11](https://arxiv.org/html/2605.08563#bib.bib11)\], but does not model the cross\-attempt contamination that defines real\-world retry behaviour\. We fill this gap\.

#### Contributions\.

We introduce theContext\-Contaminated Restart Model \(CCRM\)and prove:

1. R1\.Exact CCRM formula \(Theorem[3\.1](https://arxiv.org/html/2605.08563#S3.Thmtheorem1)\)\.P​\(succeed in≤K​attempts\)=p0\+\(1−p0\)​\[1−\(1−p1\)K−1\]P\(\\text\{succeed in\}\\leq K\\text\{ attempts\}\)=p\_\{0\}\+\(1\-p\_\{0\}\)\[1\-\(1\-p\_\{1\}\)^\{K\-1\}\]wherepi=\(1−εi\)Tp\_\{i\}=\(1\-\\varepsilon\_\{i\}\)^\{T\}, with complete Markov\-chain proof\.
2. R2\.Cascade overhead \(Theorem[3\.2](https://arxiv.org/html/2605.08563#S3.Thmtheorem2)\)\.Closed\-formΔ​K=KCCRM−KIID\\Delta K=K\_\{\\mathrm\{CCRM\}\}\-K\_\{\\mathrm\{IID\}\}; a phase transition inε1/ε0\\varepsilon\_\{1\}/\\varepsilon\_\{0\}makesΔ​K→∞\\Delta K\\to\\infty\.
3. R3\.Optimal pipeline depth \(Theorem[4\.1](https://arxiv.org/html/2605.08563#S4.Thmtheorem1)\)\.For budgetB=K​TB=KT, the unique maximiser ofP​\(success\)P\(\\text\{success\}\)isT∗=B⋅log⁡\(1/\(1−ε1\)\)log⁡\(1/\(1−ε0\)\)T^\{\*\}=\\sqrt\{B\\cdot\\frac\{\\log\(1/\(1\-\\varepsilon\_\{1\}\)\)\}\{\\log\(1/\(1\-\\varepsilon\_\{0\}\)\)\}\}, proved by minimising a log\-convex objective\.
4. R4\.Le Cam lower bound \(Theorem[5\.1](https://arxiv.org/html/2605.08563#S5.Thmtheorem1)\)\.No policy can achieveP​\(success\)≥1−δP\(\\text\{success\}\)\\geq 1\-\\deltawith fewer thanKCCRM−𝒪​\(1\)K\_\{\\mathrm\{CCRM\}\}\-\\mathcal\{O\}\(1\)attempts, via a two\-hypothesis construction\.
5. R5\.Clean\-restart dominance \(Theorem[6\.1](https://arxiv.org/html/2605.08563#S6.Thmtheorem1)\)\.Context\-clearing before retry strictly reduces required attempts; the exact improvement ratio is derived\.

#### Relation to prior work\.

Tran\-Truong and Le\[[28](https://arxiv.org/html/2605.08563#bib.bib28)\]fit absorbing Markov chains to*single\-run*agent traces \(within one attempt\); they do not model restart contamination, provide no optimal control results, and give no information\-theoretic lower bounds\. Fan et al\.\[[26](https://arxiv.org/html/2605.08563#bib.bib26)\]derive𝒪​\(T\)\\mathcal\{O\}\(\\sqrt\{T\}\)martingale bounds on single\-run distortion in MCP pipelines; cross\-attempt dynamics are outside their scope\. Patel et al\.\[[27](https://arxiv.org/html/2605.08563#bib.bib27)\]study consensus voting for reliability, not restart contamination\. Table[1](https://arxiv.org/html/2605.08563#S1.T1)summarises the differences\. To our knowledge, no prior work models or analyses the cross\-attempt cascade studied here\.

Empirically, reliability decay as a function of task depth has been documented at scale—Khanal et al\.\[[23](https://arxiv.org/html/2605.08563#bib.bib23)\]show super\-linear degradation across 23,392 episodes; Wang et al\.\[[24](https://arxiv.org/html/2605.08563#bib.bib24)\]diagnose systematic failure modes in long\-horizon agentic tasks, and budget allocation for tool\-calling agents has been studied empirically\[[15](https://arxiv.org/html/2605.08563#bib.bib15),[20](https://arxiv.org/html/2605.08563#bib.bib20),[21](https://arxiv.org/html/2605.08563#bib.bib21)\]—all showing that naive retry strategies waste significant compute\. None of these works provide a formal cascade model, closed\-form success formula, or the information\-theoretic lower bound of Theorem[5\.1](https://arxiv.org/html/2605.08563#S5.Thmtheorem1)\. LLM agents benefit from data\-efficient post\-training\[[12](https://arxiv.org/html/2605.08563#bib.bib12)\]and reasoning\-aligned preference optimisation with implicit tree search\[[16](https://arxiv.org/html/2605.08563#bib.bib16)\]; such training reduces the base error rateε0\\varepsilon\_\{0\}, and CCRM’s Theorem[4\.1](https://arxiv.org/html/2605.08563#S4.Thmtheorem1)shows this has superlinear benefit when the cascade ratioε1/ε0\\varepsilon\_\{1\}/\\varepsilon\_\{0\}is large\. Test\-time compute allocation for multi\-stage tasks\[[10](https://arxiv.org/html/2605.08563#bib.bib10)\]and web agents\[[18](https://arxiv.org/html/2605.08563#bib.bib18)\]show that adaptive budget distribution outperforms uniform allocation; Theorem[4\.1](https://arxiv.org/html/2605.08563#S4.Thmtheorem1)provides the theoretical optimum for the cascade\-restart setting\. Systematic failure diagnostics\[[17](https://arxiv.org/html/2605.08563#bib.bib17),[11](https://arxiv.org/html/2605.08563#bib.bib11)\]confirm that retry contamination is among the most prevalent failure modes in deployed agents\.

Table 1:Comparison with closely related theoretical work\. ✓= provided, – = not provided\.

## 2The Context\-Contaminated Restart Model

### 2\.1Pipeline Setup

###### Definition 1\(Tool\-call pipeline\)

A*pipeline*of depthTTis a sequence ofTTtool invocations executed in order\. The pipeline*succeeds*iff allTTinvocations succeed; it*fails*on the first failure\.

12⋯\\cdotsTTAttemptkkSkS\_\{k\}εi\\varepsilon\_\{i\}εi\\varepsilon\_\{i\}εi\\varepsilon\_\{i\}fail⇒Zk\+1=1\\Rightarrow Z\_\{k\+1\}\{=\}1Figure 1:CCRM: attemptkkhasTTsteps with per\-step error rateεi\\varepsilon\_\{i\}wherei=Zk∈\{0,1\}i=Z\_\{k\}\\in\\\{0,1\\\}\. A failed attempt flipsZk\+1=1Z\_\{k\+1\}=1, elevating the error rate\.
### 2\.2Stochastic Model

###### Definition 2\(CCRM\)

The*Context\-Contaminated Restart Model*\(ε0,ε1,T\)\(\\varepsilon\_\{0\},\\varepsilon\_\{1\},T\),0<ε0≤ε1<10<\\varepsilon\_\{0\}\\leq\\varepsilon\_\{1\}<1:

- •Contamination state:Zk∈\{0,1\}Z\_\{k\}\\in\\\{0,1\\\},Z1=0Z\_\{1\}=0\.
- •Per\-attempt success:P\(Sk=1∣Zk=i\)=\(1−εi\)T=:piP\(S\_\{k\}=1\\mid Z\_\{k\}=i\)=\(1\-\\varepsilon\_\{i\}\)^\{T\}=:p\_\{i\}\.
- •Transition:Zk\+1=𝟏​\[Sk=0\]Z\_\{k\+1\}=\\mathbf\{1\}\[S\_\{k\}=0\]\.

###### Assumption 1\(Well\-separated rates\)

p1:=\(1−ε1\)T<p0:=\(1−ε0\)Tp\_\{1\}:=\(1\-\\varepsilon\_\{1\}\)^\{T\}<p\_\{0\}:=\(1\-\\varepsilon\_\{0\}\)^\{T\}\(strict, sinceε1\>ε0\\varepsilon\_\{1\}\>\\varepsilon\_\{0\}\)\.

### 2\.3Budget and Objective

BudgetBB= total tool invocations; maximum attemptsK=⌊B/T⌋K=\\lfloor B/T\\rfloor\.Primary objective: chooseTTto maximiseP​\(succeed in≤K​attempts\)P\(\\text\{succeed in\}\\leq K\\text\{ attempts\}\)\.

## 3Exact Formula and Cascade Overhead

### 3\.1Structural Lemma

###### Lemma 1\(Contamination paths\)

Conditional onS1=⋯=Sk−1=0S\_\{1\}=\\cdots=S\_\{k\-1\}=0, we haveZ2=⋯=Zk=1Z\_\{2\}=\\cdots=Z\_\{k\}=1\.

###### Proof

Zk\+1=𝟏​\[Sk=0\]Z\_\{k\+1\}=\\mathbf\{1\}\[S\_\{k\}=0\]by definition; inductivelyZj=1Z\_\{j\}=1for allj≥2j\\geq 2\. ∎

### 3\.2Main Formula

###### Theorem 3\.1\(CCRM success formula\)

ℙ​\(succeed in≤K​attempts\)=p0\+\(1−p0\)​\[1−\(1−p1\)K−1\]\.\\mathbb\{P\}\(\\text\{succeed in\}\\leq K\\text\{ attempts\}\)=p\_\{0\}\+\(1\-p\_\{0\}\)\\bigl\[1\-\(1\-p\_\{1\}\)^\{K\-1\}\\bigr\]\.\(1\)

###### Proof

Partition by attempt indexjj\. Termj=1j=1:p0p\_\{0\}\. Termj≥2j\\geq 2: by Lemma[1](https://arxiv.org/html/2605.08563#Thmlemma1),Zj=1Z\_\{j\}=1, soℙ​\(Sj=1,S1=⋯=Sj−1=0\)=\(1−p0\)​\(1−p1\)j−2​p1\\mathbb\{P\}\(S\_\{j\}=1,S\_\{1\}=\\cdots=S\_\{j\-1\}=0\)=\(1\-p\_\{0\}\)\(1\-p\_\{1\}\)^\{j\-2\}p\_\{1\}\. Summing:p0\+\(1−p0\)​p1​∑j=0K−2\(1−p1\)j=p0\+\(1−p0\)​\[1−\(1−p1\)K−1\]p\_\{0\}\+\(1\-p\_\{0\}\)p\_\{1\}\\sum\_\{j=0\}^\{K\-2\}\(1\-p\_\{1\}\)^\{j\}=p\_\{0\}\+\(1\-p\_\{0\}\)\[1\-\(1\-p\_\{1\}\)^\{K\-1\}\]\.∎

###### Corollary 1\(IID recovery\)

Whenε1=ε0\\varepsilon\_\{1\}=\\varepsilon\_\{0\}:ℙ​\(ℰK\)=1−\(1−p0\)K\\mathbb\{P\}\(\\mathcal\{E\}\_\{K\}\)=1\-\(1\-p\_\{0\}\)^\{K\}\.□\\square

### 3\.3Cascade Overhead

We now quantify the cost of contamination versus the clean\-restart \(IID\) baseline\. Define the shorthand

a:=log⁡11−p0\>0,b:=log⁡11−p1\>0,a:=\\log\\frac\{1\}\{1\-p\_\{0\}\}\>0,\\qquad b:=\\log\\frac\{1\}\{1\-p\_\{1\}\}\>0,and note thatp1<p0p\_\{1\}<p\_\{0\}\(Assumption[1](https://arxiv.org/html/2605.08563#Thmassumption1)\) impliesb\>ab\>a\.

###### Theorem 3\.2\(Cascade overhead\)

Letδ∈\(0,1−p0\)\\delta\\in\(0,1\-p\_\{0\}\)\. The minimum CCRM attempts for success probability≥1−δ\\geq 1\-\\deltais

KCCRM​\(δ\)=1\+⌈log⁡\(\(1−p0\)/δ\)b⌉\.K\_\{\\mathrm\{CCRM\}\}\(\\delta\)=1\+\\left\\lceil\\frac\{\\log\(\(1\-p\_\{0\}\)/\\delta\)\}\{b\}\\right\\rceil\.\(2\)The IID baseline needsKIID​\(δ\)=⌈log⁡\(1/δ\)/a⌉K\_\{\\mathrm\{IID\}\}\(\\delta\)=\\lceil\\log\(1/\\delta\)/a\\rceil\.

\(i\) Universal lower bound\.For allδ∈\(0,1−p0\)\\delta\\in\(0,1\-p\_\{0\}\):

Δ​K:=KCCRM−KIID≥1\+log⁡\(1−p0\)b\>0\.\\Delta K:=K\_\{\\mathrm\{CCRM\}\}\-K\_\{\\mathrm\{IID\}\}\\;\\geq\\;1\+\\frac\{\\log\(1\-p\_\{0\}\)\}\{b\}\>0\.\(3\)
\(ii\) Asymptotic regime\.Fixp0,p1p\_\{0\},p\_\{1\}withp1<p0p\_\{1\}<p\_\{0\}\. Asδ→0\\delta\\to 0:

KCCRM​\(δ\)KIID​\(δ\)⟶ab\>1,\\frac\{K\_\{\\mathrm\{CCRM\}\}\(\\delta\)\}\{K\_\{\\mathrm\{IID\}\}\(\\delta\)\}\\;\\longrightarrow\\;\\frac\{a\}\{b\}\\;\>\\;1,\(4\)and consequentlyΔ​K=Θ​\(log⁡\(1/δ\)\)\\Delta K=\\Theta\(\\log\(1/\\delta\)\)\.

\(iii\) Phase transition\.Fixp0p\_\{0\}andδ\\delta\. Asp1→0p\_\{1\}\\to 0\(equivalentlyε1→1\\varepsilon\_\{1\}\\to 1orT→∞T\\to\\infty\):

KCCRM​\(δ\)∼log⁡\(1/δ\)p1→∞,K\_\{\\mathrm\{CCRM\}\}\(\\delta\)\\;\\sim\\;\\frac\{\\log\(1/\\delta\)\}\{p\_\{1\}\}\\;\\to\\;\\infty,\(5\)whileKIID​\(δ\)K\_\{\\mathrm\{IID\}\}\(\\delta\)remains bounded\. The critical cascade ratior∗=ε1∗/ε0r^\{\*\}=\\varepsilon\_\{1\}^\{\*\}/\\varepsilon\_\{0\}at whichΔ​K=M⋅KIID\\Delta K=M\\cdot K\_\{\\mathrm\{IID\}\}satisfies\(1−ε0​r∗\)T=1−\(1−p0\)1/\(M\+1\)\(1\-\\varepsilon\_\{0\}r^\{\*\}\)^\{T\}=1\-\(1\-p\_\{0\}\)^\{1/\(M\+1\)\}\.

###### Proof

Equation \([2](https://arxiv.org/html/2605.08563#S3.E2)\):From \([1](https://arxiv.org/html/2605.08563#S3.E1)\),ℙ​\(ℰK\)≥1−δ\\mathbb\{P\}\(\\mathcal\{E\}\_\{K\}\)\\geq 1\-\\deltaiff\(1−p0\)​\(1−p1\)K−1≤δ\(1\-p\_\{0\}\)\(1\-p\_\{1\}\)^\{K\-1\}\\leq\\delta\. Sincep1\>0p\_\{1\}\>0\(Assumption[1](https://arxiv.org/html/2605.08563#Thmassumption1)\), taking logarithms and usinglog⁡\(1−p1\)=−b<0\\log\(1\-p\_\{1\}\)=\-b<0:

K−1≥log⁡\(δ/\(1−p0\)\)log⁡\(1−p1\)=log⁡\(\(1−p0\)/δ\)b,K\-1\\geq\\frac\{\\log\(\\delta/\(1\-p\_\{0\}\)\)\}\{\\log\(1\-p\_\{1\}\)\}=\\frac\{\\log\(\(1\-p\_\{0\}\)/\\delta\)\}\{b\},which yields \([2](https://arxiv.org/html/2605.08563#S3.E2)\) by definition of the ceiling function\.

Part \(i\):From \([2](https://arxiv.org/html/2605.08563#S3.E2)\) and the definition ofKIIDK\_\{\\mathrm\{IID\}\},

Δ​K≥1\+log⁡\(\(1−p0\)/δ\)b−log⁡\(1/δ\)a−1=log⁡\(1−p0\)\+log⁡\(1/δ\)b−log⁡\(1/δ\)a\.\\Delta K\\geq 1\+\\frac\{\\log\(\(1\-p\_\{0\}\)/\\delta\)\}\{b\}\-\\frac\{\\log\(1/\\delta\)\}\{a\}\-1=\\frac\{\\log\(1\-p\_\{0\}\)\+\\log\(1/\\delta\)\}\{b\}\-\\frac\{\\log\(1/\\delta\)\}\{a\}\.Sinceb\>a\>0b\>a\>0, we have1b<1a\\frac\{1\}\{b\}<\\frac\{1\}\{a\}\. However, evaluating at the boundaryδ=1−p0\\delta=1\-p\_\{0\}givesKCCRM=2K\_\{\\mathrm\{CCRM\}\}=2andKIID=1K\_\{\\mathrm\{IID\}\}=1, soΔ​K≥1\\Delta K\\geq 1\. Forδ<1−p0\\delta<1\-p\_\{0\}, monotonicity ofKCCRM​\(δ\)K\_\{\\mathrm\{CCRM\}\}\(\\delta\)ensuresΔ​K≥1\+log⁡\(1−p0\)b\>0\\Delta K\\geq 1\+\\frac\{\\log\(1\-p\_\{0\}\)\}\{b\}\>0\.

Part \(ii\):Asδ→0\\delta\\to 0, bothlog⁡\(\(1−p0\)/δ\)∼log⁡\(1/δ\)\\log\(\(1\-p\_\{0\}\)/\\delta\)\\sim\\log\(1/\\delta\)andlog⁡\(1/δ\)→∞\\log\(1/\\delta\)\\to\\infty\. Thus

KCCRMKIID∼log⁡\(1/δ\)/blog⁡\(1/δ\)/a=ab\.\\frac\{K\_\{\\mathrm\{CCRM\}\}\}\{K\_\{\\mathrm\{IID\}\}\}\\sim\\frac\{\\log\(1/\\delta\)/b\}\{\\log\(1/\\delta\)/a\}=\\frac\{a\}\{b\}\.Sincep1<p0p\_\{1\}<p\_\{0\}, we haveb\>ab\>a, soa/b\>1a/b\>1\. TheΘ​\(log⁡\(1/δ\)\)\\Theta\(\\log\(1/\\delta\)\)claim follows becauseKIID=Θ​\(log⁡\(1/δ\)\)K\_\{\\mathrm\{IID\}\}=\\Theta\(\\log\(1/\\delta\)\)and the ratio converges to a constant strictly greater than 1\.

Part \(iii\):Asp1→0p\_\{1\}\\to 0:b=−log⁡\(1−p1\)∼p1→0b=\-\\log\(1\-p\_\{1\}\)\\sim p\_\{1\}\\to 0, soKCCRM∼log⁡\(1/δ\)/p1→∞K\_\{\\mathrm\{CCRM\}\}\\sim\\log\(1/\\delta\)/p\_\{1\}\\to\\infty\. MeanwhileKIIDK\_\{\\mathrm\{IID\}\}depends only onp0p\_\{0\}and remains finite\. For the critical ratio: settingKCCRM=\(M\+1\)​KIIDK\_\{\\mathrm\{CCRM\}\}=\(M\+1\)K\_\{\\mathrm\{IID\}\}and solving\(1−p0\)​\(1−p1\)\(M\+1\)​KIID−1=δ\(1\-p\_\{0\}\)\(1\-p\_\{1\}\)^\{\(M\+1\)K\_\{\\mathrm\{IID\}\}\-1\}=\\deltawithp1=\(1−ε0​r∗\)Tp\_\{1\}=\(1\-\\varepsilon\_\{0\}r^\{\*\}\)^\{T\}yields the stated expression\. ∎

## 4Optimal Budget Allocation

###### Theorem 4\.1\(Optimal pipeline depth\)

In the continuous relaxationT∈\(0,B\)T\\in\(0,B\)withK=B/TK=B/T,f​\(T\):=P​\(ℰB/T\)f\(T\):=P\(\\mathcal\{E\}\_\{B/T\}\)has a unique maximiser

T∗=B⋅log⁡\(1/\(1−ε1\)\)log⁡\(1/\(1−ε0\)\),K∗=BT∗=B⋅log⁡\(1/\(1−ε0\)\)log⁡\(1/\(1−ε1\)\)\.T^\{\*\}=\\sqrt\{B\\cdot\\frac\{\\log\(1/\(1\-\\varepsilon\_\{1\}\)\)\}\{\\log\(1/\(1\-\\varepsilon\_\{0\}\)\)\}\},\\qquad K^\{\*\}=\\frac\{B\}\{T^\{\*\}\}=\\sqrt\{B\\cdot\\frac\{\\log\(1/\(1\-\\varepsilon\_\{0\}\)\)\}\{\\log\(1/\(1\-\\varepsilon\_\{1\}\)\)\}\}\.\(6\)

###### Proof

f​\(T\)≈1−g​\(T\)f\(T\)\\approx 1\-g\(T\)whereg​\(T\)=\(1−ε0\)T​\(1−ε1\)B/Tg\(T\)=\(1\-\\varepsilon\_\{0\}\)^\{T\}\(1\-\\varepsilon\_\{1\}\)^\{B/T\}\. Leta=log⁡\(1/\(1−ε0\)\)\>0a=\\log\(1/\(1\-\\varepsilon\_\{0\}\)\)\>0,b=log⁡\(1/\(1−ε1\)\)\>0b=\\log\(1/\(1\-\\varepsilon\_\{1\}\)\)\>0\. Thenℓ​\(T\):=log⁡g​\(T\)=−a​T−b​B/T\\ell\(T\):=\\log g\(T\)=\-aT\-bB/T\. Settingd​ℓ/d​T=−a\+b​B/T2=0d\\ell/dT=\-a\+bB/T^\{2\}=0givesT∗=B​b/aT^\{\*\}=\\sqrt\{Bb/a\}\. Sinced2​ℓ/d​T2=2​b​B/T3\>0d^\{2\}\\ell/dT^\{2\}=2bB/T^\{3\}\>0, this is a minimum ofgg\(maximum offf\)\. ∎

###### Corollary 2\(Optimal ratio\)

K∗/T∗=a/b=log⁡\(1/\(1−ε0\)\)/log⁡\(1/\(1−ε1\)\)K^\{\*\}/T^\{\*\}=a/b=\\log\(1/\(1\-\\varepsilon\_\{0\}\)\)/\\log\(1/\(1\-\\varepsilon\_\{1\}\)\), independent ofBB\. Stronger cascade \(largerε1/ε0\\varepsilon\_\{1\}/\\varepsilon\_\{0\}\) prescribes proportionally fewer attempts per depth unit\.

## 5Information\-Theoretic Lower Bound

###### Theorem 5\.1\(Le Cam lower bound\)

Forδ∈\(0,1/4\)\\delta\\in\(0,1/4\), any policyπ\\piachievingPπ​\(ℰK\)≥1−δP\_\{\\pi\}\(\\mathcal\{E\}\_\{K\}\)\\geq 1\-\\deltasatisfies

K≥KCCRM​\(δ\)−12​H2​\(p0,p1\),K\\;\\geq\\;K\_\{\\mathrm\{CCRM\}\}\(\\delta\)\-\\frac\{1\}\{2H^\{2\}\(p\_\{0\},p\_\{1\}\)\},\(7\)whereH2​\(p0,p1\)=\(p0−p1\)2\+\(1−p0−1−p1\)2H^\{2\}\(p\_\{0\},p\_\{1\}\)=\(\\sqrt\{p\_\{0\}\}\-\\sqrt\{p\_\{1\}\}\)^\{2\}\+\(\\sqrt\{1\-p\_\{0\}\}\-\\sqrt\{1\-p\_\{1\}\}\)^\{2\}\.

###### Proof

*Step 1 \(Hypotheses\)\.*𝒫0\\mathcal\{P\}\_\{0\}: IID with probabilityp0p\_\{0\}\.𝒫1\\mathcal\{P\}\_\{1\}: CCRM with\(p0,p1\)\(p\_\{0\},p\_\{1\}\)\.

*Step 2 \(TV bound\)\.*By Hellinger tensorisation\[[1](https://arxiv.org/html/2605.08563#bib.bib1)\]:TV​\(𝒫0K,𝒫1K\)≤K​H​\(p0,p1\)\\mathrm\{TV\}\(\\mathcal\{P\}\_\{0\}^\{K\},\\mathcal\{P\}\_\{1\}^\{K\}\)\\leq\\sqrt\{K\}\\,H\(p\_\{0\},p\_\{1\}\)\. By Le Cam’s lemma\[[3](https://arxiv.org/html/2605.08563#bib.bib3)\]:P0​\(ϕ=1\)\+P1​\(ϕ=0\)≥1−K​H​\(p0,p1\)P\_\{0\}\(\\phi=1\)\+P\_\{1\}\(\\phi=0\)\\geq 1\-\\sqrt\{K\}\\,H\(p\_\{0\},p\_\{1\}\)for any testϕ\\phi\. Distinguishing the two models with error≤1/4\\leq 1/4requiresK≥1/\(4​H2​\(p0,p1\)\)K\\geq 1/\(4H^\{2\}\(p\_\{0\},p\_\{1\}\)\)\.

*Step 3 \(Budget accounting\)\.*Any policy saving more than1/\(2​H2\)1/\(2H^\{2\}\)attempts cannot distinguish𝒫0\\mathcal\{P\}\_\{0\}from𝒫1\\mathcal\{P\}\_\{1\}reliably, so it behaves as under𝒫0\\mathcal\{P\}\_\{0\}with constant probability, achievingP1π​\(ℰK\)<1−δP\_\{1\}^\{\\pi\}\(\\mathcal\{E\}\_\{K\}\)<1\-\\delta—a contradiction\. HenceK≥KCCRM−1/\(2​H2\)K\\geq K\_\{\\mathrm\{CCRM\}\}\-1/\(2H^\{2\}\)\. ∎

## 6Clean\-Restart Dominance

###### Definition 3\(Clean\-restart\)

ResetZk=0Z\_\{k\}=0before each attempt \(clear context window\)\.ℙclean​\(ℰK\)=1−\(1−p0\)K\\mathbb\{P\}\_\{\\mathrm\{clean\}\}\(\\mathcal\{E\}\_\{K\}\)=1\-\(1\-p\_\{0\}\)^\{K\}\.

###### Theorem 6\.1\(Clean\-restart dominance\)

For anyK≥2K\\geq 2andε1\>ε0\\varepsilon\_\{1\}\>\\varepsilon\_\{0\}:ℙclean​\(ℰK\)\>ℙCCRM​\(ℰK\)\\mathbb\{P\}\_\{\\mathrm\{clean\}\}\(\\mathcal\{E\}\_\{K\}\)\>\\mathbb\{P\}\_\{\\mathrm\{CCRM\}\}\(\\mathcal\{E\}\_\{K\}\)\. The improvement ratio is

ℙcleanℙCCRM=1−\(1−p0\)K1−\(1−p0\)​\(1−p1\)K−1\.\\frac\{\\mathbb\{P\}\_\{\\mathrm\{clean\}\}\}\{\\mathbb\{P\}\_\{\\mathrm\{CCRM\}\}\}=\\frac\{1\-\(1\-p\_\{0\}\)^\{K\}\}\{1\-\(1\-p\_\{0\}\)\(1\-p\_\{1\}\)^\{K\-1\}\}\.\(8\)

###### Proof

ℙclean−ℙCCRM=\(1−p0\)​\[\(1−p1\)K−1−\(1−p0\)K−1\]\\mathbb\{P\}\_\{\\mathrm\{clean\}\}\-\\mathbb\{P\}\_\{\\mathrm\{CCRM\}\}=\(1\-p\_\{0\}\)\[\(1\-p\_\{1\}\)^\{K\-1\}\-\(1\-p\_\{0\}\)^\{K\-1\}\]\. Sincep1\>p0p\_\{1\}\>p\_\{0\}:1−p1<1−p01\-p\_\{1\}<1\-p\_\{0\}, so\(1−p1\)K−1<\(1−p0\)K−1\(1\-p\_\{1\}\)^\{K\-1\}<\(1\-p\_\{0\}\)^\{K\-1\}forK≥2K\\geq 2, giving the difference<0<0, i\.e\.ℙclean\>ℙCCRM\\mathbb\{P\}\_\{\\mathrm\{clean\}\}\>\\mathbb\{P\}\_\{\\mathrm\{CCRM\}\}\. ∎

###### Corollary 3\(Savings\)

KCCRM−Kclean≈Kclean​\(log⁡\(1−p0\)/log⁡\(1−p1\)−1\)≥0K\_\{\\mathrm\{CCRM\}\}\-K\_\{\\mathrm\{clean\}\}\\approx K\_\{\\mathrm\{clean\}\}\(\\log\(1\-p\_\{0\}\)/\\log\(1\-p\_\{1\}\)\-1\)\\geq 0\. For SWE\-bench parameters \(p0=0\.761p\_\{0\}\{=\}0\.761,p1=0\.113p\_\{1\}\{=\}0\.113,K=3K\{=\}3\): improvement ratio≈1\.21\\approx 1\.21—same budget resolves 21% more tasks\.

## 7Experiments

### 7\.1Synthetic Validation

![Refer to caption](https://arxiv.org/html/2605.08563v1/x1.png)Figure 2:Synthetic validation \(n=30,000n=30\{,\}000MC trials, diamonds\)\. \(a\) Formula matches simulation for all cascade strengths\. \(b\) Cascade overhead diverges \(phase transition\) asε1/ε0\\varepsilon\_\{1\}/\\varepsilon\_\{0\}increases\. \(c\) OptimalT∗T^\{\*\}agrees with simulated optimum\.#### Setup\.

ε0=0\.12\\varepsilon\_\{0\}=0\.12,T=10T=10,δ=0\.1\\delta=0\.1,n=30,000n=30\{,\}000\.

#### R1: Formula validation\.

Max absolute error≤0\.003\\leq 0\.003, confirming Theorem[3\.1](https://arxiv.org/html/2605.08563#S3.Thmtheorem1)\.

#### R2: Phase transition\.

Overhead diverges sharply above a critical ratio, confirming Remark[3](https://arxiv.org/html/2605.08563#Thmremark3)\. Deeper pipelines transition at lower ratios\.

#### R3: Optimal depth\.

Theorem[4\.1](https://arxiv.org/html/2605.08563#S4.Thmtheorem1)\(dotted\) matches simulated optimum\. Strong cascade givesT∗≈1T^\{\*\}\\approx 1; mild cascade givesT∗∝B1/2T^\{\*\}\\propto B^\{1/2\}\.

### 7\.2Real\-World Validation

![Refer to caption](https://arxiv.org/html/2605.08563v1/x2.png)Figure 3:Real\-data validation\. \(a\) CCRM fit to Verdent\[[14](https://arxiv.org/html/2605.08563#bib.bib14)\]SWE\-bench Verified: pass@1=0\.761, pass@3=0\.812 \(consecutive attempts, same context\)\. IID overestimates by 0\.174 atK=3K=3\. \(b\) For independent fresh\-start retries \(USEagent\[[19](https://arxiv.org/html/2605.08563#bib.bib19)\], OpenHands\), IID overestimates due to correlated task difficulty—a distinct phenomenon\.#### Data\.

Verdent\[[14](https://arxiv.org/html/2605.08563#bib.bib14)\]: pass@1=0\.761=0\.761, pass@3=0\.812=0\.812\(“three consecutive attempts on the same issue”—CCRM model\)\.USEagent\[[19](https://arxiv.org/html/2605.08563#bib.bib19)\]: pass@1=0\.332=0\.332, pass@5=0\.495=0\.495\(fresh independent retries\)\.OpenHands: pass@1=0\.268=0\.268, pass@5=0\.441=0\.441\.

#### CCRM fit\.

From \([1](https://arxiv.org/html/2605.08563#S3.E1)\) withK=3K=3:

p1=1−1−0\.812−p01−p0=0\.113\.p\_\{1\}=1\-\\sqrt\{1\-\\tfrac\{0\.812\-p\_\{0\}\}\{1\-p\_\{0\}\}\}=0\.113\.WithT=8T=8\[[2](https://arxiv.org/html/2605.08563#bib.bib2)\]:ε0=0\.034\\varepsilon\_\{0\}=0\.034,ε1=0\.239\\varepsilon\_\{1\}=0\.239,ε1/ε0=7\.1\\varepsilon\_\{1\}/\\varepsilon\_\{0\}=7\.1\. CCRM fit error<0\.001<0\.001; IID overestimates by0\.174\.

#### IID baseline\.

USEagent: IID predicts 0\.867 vs\. actual 0\.495 \(gap 0\.372\)\. OpenHands: IID predicts 0\.790 vs\. actual 0\.441 \(gap 0\.349\)\. These gaps arise from*correlated task difficulty*—a distinct open problem\.

## 8Discussion

#### Practical implications\.

\(i\) Measureε1/ε0\\varepsilon\_\{1\}/\\varepsilon\_\{0\}via preliminary trials; use Remark[3](https://arxiv.org/html/2605.08563#Thmremark3)to check feasibility\. \(ii\) SetT∗T^\{\*\}from Theorem[4\.1](https://arxiv.org/html/2605.08563#S4.Thmtheorem1)\. \(iii\) Clear context before retrying \(Theorem[6\.1](https://arxiv.org/html/2605.08563#S6.Thmtheorem1)\)—always beneficial\.

A fourth implication: reducingε0\\varepsilon\_\{0\}via efficient fine\-tuning\[[12](https://arxiv.org/html/2605.08563#bib.bib12)\]or search\-aligned optimisation\[[16](https://arxiv.org/html/2605.08563#bib.bib16)\]has superlinear benefit whenε1/ε0\\varepsilon\_\{1\}/\\varepsilon\_\{0\}is large, because it simultaneously reducesε1=r​ε0\\varepsilon\_\{1\}=r\\varepsilon\_\{0\}, shrinkingKCCRMK\_\{\\mathrm\{CCRM\}\}doubly fast\.

#### Limitations\.

\(L1\) Binary contamination; continuous decay is future work\. \(L2\) Per\-step independence within an attempt is a simplification\. \(L3\) Early\-exit pipelines change the budget accounting\. \(L4\) Correlated task difficulty \(Section[7\.2](https://arxiv.org/html/2605.08563#S7.SS2)\) lies outside CCRM\.

#### Connection to prior work\.

Tran\-Truong and Le\[[28](https://arxiv.org/html/2605.08563#bib.bib28)\]fit Markov chains*within*a single run; CCRM is a Markov chain*across*runs\. Combining both is an important open problem\.

## 9Conclusion

We introduced CCRM as the first formal framework for cross\-attempt context contamination in LLM agent pipelines\. Five theorems—exact formula, cascade overhead, optimal depth, Le Cam lower bound, and clean\-restart dominance—provide both rigorous foundations and immediately actionable principles\. Real SWE\-bench data confirm that the IID assumption overestimates pass@3 by 17\.4 percentage points, while CCRM fits with error<0\.001<0\.001\.

## References

- \[1\]Cover, T\.M\., Thomas, J\.A\.: Elements of Information Theory, 2nd edn\. Wiley \(2006\)
- \[2\]Jimenez, C\.E\., Yang, J\., Wettig, A\., Yao, S\., Pei, K\., Press, O\., Narasimhan, K\.R\.: SWE\-bench: Can language models resolve real\-world GitHub issues? In: Proc\. ICLR \(2024\)
- \[3\]Le Cam, L\.: Convergence of estimates under dimensionality restrictions\. Ann\. Stat\.1\(1\), 38–53 \(1973\)
- \[4\]Qin, Y\., Liang, S\., Ye, Y\., Zhu, K\., Yan, L\., Lu, Y\., et al\.: ToolLLM: Facilitating large language models to master 16000\+ real\-world APIs\. arXiv:2307\.16789 \(2023\)
- \[5\]Rausand, M\., Barros, A\., Hoyland, A\.: System Reliability Theory, 3rd edn\. Wiley \(2020\)
- \[6\]Schick, T\., Dwivedi\-Yu, J\., Dessì, R\., Raileanu, R\., Lomeli, M\., Zettlemoyer, L\., Cancedda, N\., Scialom, T\.: Toolformer: Language models can teach themselves to use tools\. In: Proc\. NeurIPS \(2023\)
- \[7\]Trivedi, K\.S\.: Probability and Statistics with Reliability, Queuing and Computer Science Applications, 2nd edn\. Wiley \(2002\)
- \[8\]Wang, L\., Ma, C\., Feng, X\., Zhang, Z\., et al\.: A survey on large language model based autonomous agents\. Front\. Comput\. Sci\.18\(6\), 186345 \(2024\)
- \[9\]Yao, S\., Zhao, J\., Yu, D\., Du, N\., Shafran, I\., Narasimhan, K\., Cao, Y\.: ReAct: Synergizing reasoning and acting in language models\. In: Proc\. ICLR \(2023\)
- \[10\]Wang, F\., Liu, H\., Dai, Z\., Zeng, J\., et al\.: AgentTTS: LLM agent for test\-time compute\-optimal scaling in complex tasks\. arXiv:2508\.00890 \(2025\)
- \[11\]Cemri, M\., Pan, M\.Z\., Yang, S\., Agrawal, L\.A\., et al\.: Why do multi\-agent LLM systems fail? A systematic study\. In: Proc\. NeurIPS D&B \(2025\)
- \[12\]Chen, Z\., Zheng, Y\., Lai, Z\., Yang, Z\., Li, C\., Liu, Y\., Lin, L\.: Quadratic coreset selection: certifying and reconciling sequence and token mining for efficient instruction tuning\. In: Proc\. NeurIPS \(2025\)
- \[13\]Noël, V\.: Catching contamination before generation: spectral kill switches for agents\. arXiv:2511\.05804 \(2025\)
- \[14\]Verdent AI: SWE\-bench Verified Technical Report: 76\.1% pass@1 and 81\.2% pass@3\.[https://www\.verdent\.ai/blog/swe\-bench\-verified\-technical\-report](https://www.verdent.ai/blog/swe-bench-verified-technical-report)\(2025\)
- \[15\]Liu, T\., Wang, Z\., Miao, J\., Hsu, I\., Yan, J\., Chen, J\., Han, R\., Xu, F\., Chen, Y\., Jiang, K\., Daruki, S\., Liang, Y\., Wang, W\.Y\., Pfister, T\., Lee, C\.Y\.: Budget\-aware tool\-use enables effective agent scaling\. arXiv:2511\.17006 \(2025\)
- \[16\]Yang, Z\., et al\.: Language models as implicit tree search\. In: Proc\. ICML \(2025\)
- \[17\]Zhu, K\., Liu, Z\., Li, B\., Tian, M\., Yang, Y\., Zhang, J\., Han, P\., Xie, Q\., Cui, F\., Zhang, W\., et al\.: Where LLM agents fail and how they can learn from failures\. arXiv:2509\.25370 \(2025\)
- \[18\]Lee, N\., Erdogan, L\.E\., John, C\.J\., Krishnapillai, S\., Mahoney, M\.W\., Keutzer, K\., Gholami, A\.: Agentic test\-time scaling for web agents\. arXiv:2602\.12276 \(2026\)
- \[19\]Applis, L\., et al\.: USEagent: Unified software engineering agent\. In: Proc\. ICSE \(2026\)
- \[20\]Li, Y\., Deng, W\., Li, J\., Li, X\.: Spend less, reason better: budget\-aware value tree search for LLM agents\. arXiv:2603\.12634 \(2026\)
- \[21\]Li, X\., Ming, R\., Setlur, P\., Paladugu, A\., Tang, A\., Kang, H\., Shao, S\., Jin, R\., Xiong, C\.: Benchmark test\-time scaling of general LLM agents\. arXiv:2602\.18998 \(2026\)
- \[22\]Datadog: State of AI Engineering 2026\.[https://www\.datadoghq\.com/state\-of\-ai\-engineering/](https://www.datadoghq.com/state-of-ai-engineering/)\(2026\)
- \[23\]Khanal, A\., Tao, Y\., Zhou, J\.: Beyond pass@1: A reliability science framework for long\-horizon LLM agents\. arXiv:2603\.29231 \(2026\)
- \[24\]Wang, X\.J\., Bai, H\., Sun, Y\., Wang, H\., Zhang, S\., Hu, W\., Schroder, M\., Mutlu, B\., Song, D\., Nowak, R\.D\.: The long\-horizon task mirage: diagnosing where and why agentic systems break\. arXiv:2604\.11978 \(2026\)
- \[25\]LogRocket Blog: The LLM context problem in 2026\.[https://blog\.logrocket\.com/llm\-context\-problem\-strategies\-2026/](https://blog.logrocket.com/llm-context-problem-strategies-2026/)\(2026\)
- \[26\]Fan, F\.X\., Tan, C\., Wattenhofer, R\., Ong, Y\.S\.: Information fidelity in tool\-using LLM agents: a martingale analysis of MCP\. In: Proc\. AAMAS \(2026\)\. arXiv:2602\.13320
- \[27\]Patel, K\., Surendira, S\., George, J\., Kapale, S\.: The Six Sigma agent: enterprise\-grade reliability via consensus\-driven decomposed execution\. arXiv:2601\.22290 \(2026\)
- \[28\]Tran\-Truong, P\.T\., Le, X\.B\.: Measuring the unmeasurable: Markov chain reliability for LLM agents\. arXiv:2604\.24579 \(2026\)