Context Poisoning as Extreme-Value Attention Interference in Long-Context Language Models

# Context Poisoning as Extreme-Value Attention Interference in Long-Context Language Models
Source: [https://arxiv.org/html/2609.22101](https://arxiv.org/html/2609.22101)
###### Abstract

Large language models can process increasingly long prompts, yet their ability to locate and use decisive evidence may degrade as irrelevant or confusable context is added\. We formulate this phenomenon, which we call*context poisoning*, as extreme\-value interference in attention: the decisive\-evidence score is upper\-bounded, while the maximum score among effective distractors grows with their number\. Under a softmax retrieval abstraction, we derive a finite\-sample upper bound showing that maintaining a fixed accuracy target above base rate requires the evidence margin to scale asΩ⁡\(log⁡N\)\\Omega\(\\sqrt\{\\log N\}\), whereNNdenotes the effective distractor count rather than necessarily the raw context length\. The analysis connects long\-context degradation to score aliasing, positional aliasing, and softmax dilution\. Controlled experiments show that retrieval accuracy decreases as total context grows in the presence of embedded hard negatives, that the same\-format condition produces the largest observed accuracy drop among the tested distractor constructions at fixed context length, and that retrieval gating can improve evidence use while its net benefit depends on preserving evidence recall\. These results motivate evidence bottlenecks, alias\-resistant representations, retrieve\-then\-reason architectures, verifier\-mediated memory, and contrastive anti\-poison training\.

## Introduction

Increasing a language model’s context window is not the same as increasing its usable memory\. Transformer attention gives every token a path to every other token\([30](https://arxiv.org/html/2609.22101#bib.bib28)\), and large autoregressive models can exploit prompts for in\-context adaptation\([4](https://arxiv.org/html/2609.22101#bib.bib4)\)\. However, long input does not guarantee robust evidence use\. Empirical studies report strong position effects and degradation when relevant information is buried in the middle of a long context\([21](https://arxiv.org/html/2609.22101#bib.bib19)\), sensitivity to irrelevant information\([28](https://arxiv.org/html/2609.22101#bib.bib26)\), and benchmark\-level gaps between nominal and effective context length\([2](https://arxiv.org/html/2609.22101#bib.bib2);[1](https://arxiv.org/html/2609.22101#bib.bib1);[14](https://arxiv.org/html/2609.22101#bib.bib13);[10](https://arxiv.org/html/2609.22101#bib.bib10)\)\. This paper uses the term*context poisoning*for a non\-training\-data phenomenon: the input itself becomes contaminated by distractors, aliases, and positional artifacts as the window grows\.

The practical symptom is familiar\. A prompt contains a small set of key facts, then many thousands of irrelevant or weakly related tokens\. A model that would answer correctly from the short prompt may answer incorrectly from the long prompt, quote a distractor, or ignore the key fact\. Existing long\-context systems attack the computational side through recurrence, compression, sparse attention, or fast exact attention\([7](https://arxiv.org/html/2609.22101#bib.bib7);[24](https://arxiv.org/html/2609.22101#bib.bib22);[3](https://arxiv.org/html/2609.22101#bib.bib3);[33](https://arxiv.org/html/2609.22101#bib.bib30);[6](https://arxiv.org/html/2609.22101#bib.bib6);[18](https://arxiv.org/html/2609.22101#bib.bib16);[8](https://arxiv.org/html/2609.22101#bib.bib8)\)\. Those methods make long contexts feasible, but feasibility alone does not imply robust extraction\. Positional methods such as ALiBi, RoPE, interpolation, and YaRN improve extrapolation behavior\([23](https://arxiv.org/html/2609.22101#bib.bib21);[29](https://arxiv.org/html/2609.22101#bib.bib27);[5](https://arxiv.org/html/2609.22101#bib.bib5);[22](https://arxiv.org/html/2609.22101#bib.bib20)\), but long\-context failures can persist even within claimed windows\.

#### Contributions\.

First, we give formal definitions for context poisoning, score aliasing, and effective distractor count\. Second, we prove a theorem: in a simple but diagnostic attention model, the probability that a distractor captures enough attention increases withNN, and maintaining accuracy requires a signal margin that grows likelog⁡N\\sqrt\{\\log N\}\. Third, we derive design principles that turn the bound into system requirements: reduceNN, increase the evidence margin, verify evidence before decoding, and regularize aliasing\. Fourth, we report two controlled empirical studies in the main paper: a hard\-negative retrieval decay curve on a production model \([Empirical study: retrieval decay with embedded hard negatives](https://arxiv.org/html/2609.22101#Sx8)\) and a fixed\-length distractor\-confusability ablation across three models \([Empirical study: distractor confusability at fixed context length](https://arxiv.org/html/2609.22101#Sx9)\)\. The appendix provides a retrieve\-then\-reason gate experiment at context lengths up to512512K tokens \([Appendix FEmpirical Test of the Gate\-Mitigation Prediction](https://arxiv.org/html/2609.22101#A6)\), together with the full proofs, additional implementation details, complete experimental protocols and supplemental tables, attention\-distribution diagnostics, a confusable\-decoy dose\-response analysis, and a lost\-in\-the\-middle reanalysis\. Across these components, the paper connects a finite\-sample failure bound to observable changes in distractor count, distractor composition, and retrieval recall\. We do not claim that every LLM failure is explained by a single attention head or Gaussian scores\. The theorem is a stress test: if a long\-context architecture cannot control the extreme distractor term in this abstraction, it should not be expected to remain robust merely by scaling the raw context window\.

The remainder of the paper is organized as follows\. We first formalize context poisoning and the effective distractor count, establish the extreme\-value attention bound, examine the mechanisms predicted by the theory, and derive architectural alternatives motivated by the bound\.[Experimental setup and metrics](https://arxiv.org/html/2609.22101#Sx7)then introduces the evaluation framework, followed by two empirical studies in the main paper:[Empirical study: retrieval decay with embedded hard negatives](https://arxiv.org/html/2609.22101#Sx8)and[Empirical study: distractor confusability at fixed context length](https://arxiv.org/html/2609.22101#Sx9)\. The appendix contains the retrieve\-then\-reason gate experiment \([Appendix FEmpirical Test of the Gate\-Mitigation Prediction](https://arxiv.org/html/2609.22101#A6)\), the full theoretical proofs and additional implementation details, the complete distractor\-confusability protocol and supplemental results \([Appendix CDistractor confusability ablation: full protocol and results](https://arxiv.org/html/2609.22101#A3)\), the attention\-distribution diagnostics and confusable\-decoy dose\-response analysis \([Appendix DAttention\-distribution diagnostics and dose\-response: from assumption to mechanism](https://arxiv.org/html/2609.22101#A4)\), and the lost\-in\-the\-middle reanalysis \([Appendix ELost\-in\-the\-middle reanalysis](https://arxiv.org/html/2609.22101#A5)\)\.

## Related work

#### Efficient long\-context architectures\.

Long\-context language modeling builds on transformers\([30](https://arxiv.org/html/2609.22101#bib.bib28)\), large\-scale pretraining\([9](https://arxiv.org/html/2609.22101#bib.bib9);[25](https://arxiv.org/html/2609.22101#bib.bib23)\), and in\-context learning\([4](https://arxiv.org/html/2609.22101#bib.bib4)\)\. Transformer\-XL and Compressive Transformers introduced recurrence and memory compression\([7](https://arxiv.org/html/2609.22101#bib.bib7);[24](https://arxiv.org/html/2609.22101#bib.bib22)\)\. Longformer, Big Bird, Performer, and linear attention reduce long\-sequence cost\([3](https://arxiv.org/html/2609.22101#bib.bib3);[33](https://arxiv.org/html/2609.22101#bib.bib30);[6](https://arxiv.org/html/2609.22101#bib.bib6);[18](https://arxiv.org/html/2609.22101#bib.bib16)\), while FlashAttention improves exact\-attention efficiency\([8](https://arxiv.org/html/2609.22101#bib.bib8)\)\. ALiBi, RoPE, positional interpolation, and YaRN address positional extrapolation and context\-window extension\([23](https://arxiv.org/html/2609.22101#bib.bib21);[29](https://arxiv.org/html/2609.22101#bib.bib27);[5](https://arxiv.org/html/2609.22101#bib.bib5);[22](https://arxiv.org/html/2609.22101#bib.bib20)\)\. These methods make long contexts feasible, but feasibility does not ensure that decisive evidence remains distinguishable from many competitive candidates\.

#### Selection, compression, and memory\.

Retrieval\-augmented generation selects a smaller evidence set before decoding\([20](https://arxiv.org/html/2609.22101#bib.bib18);[13](https://arxiv.org/html/2609.22101#bib.bib12);[16](https://arxiv.org/html/2609.22101#bib.bib14)\), and prompt compression removes low\-utility tokens\([17](https://arxiv.org/html/2609.22101#bib.bib15)\)\. Attention\-sink methods instead preserve selected states to stabilize streaming generation\([31](https://arxiv.org/html/2609.22101#bib.bib29)\)\. These approaches expose a shared tradeoff: reducing the candidate set can lower interference, but an imperfect selector can discard required evidence\. Proposition[1](https://arxiv.org/html/2609.22101#Thmproposition1)expresses this tradeoff as replacingNNbyKKeffective distractors while paying a recall penaltyη\\eta\.

#### Long\-context evaluation and failure analysis\.

Lost\-in\-the\-middle effects show that central evidence can be underused\([21](https://arxiv.org/html/2609.22101#bib.bib19)\), and irrelevant information can distract reasoning even when the evidence remains present\([28](https://arxiv.org/html/2609.22101#bib.bib26)\)\. Positional interventions can partly recover evidence use\([34](https://arxiv.org/html/2609.22101#bib.bib31)\)\. LongBench, L\-Eval, ZeroSCROLLS, and RULER broaden evaluation across tasks and context lengths\([2](https://arxiv.org/html/2609.22101#bib.bib2);[1](https://arxiv.org/html/2609.22101#bib.bib1);[27](https://arxiv.org/html/2609.22101#bib.bib25);[14](https://arxiv.org/html/2609.22101#bib.bib13)\)\. Recent controlled work further shows degradation despite controlled or perfect retrieval\([10](https://arxiv.org/html/2609.22101#bib.bib10)\), reinforcing the distinction between nominal capacity, evidence retrieval, and evidence use\.

#### Relation to our contribution\.

Our contribution is to formalize one failure mode as extreme\-value competition among distractor scores\. We distinguish raw length from effective distractor count, derive the margin growth required for reliable evidence use, and connect mitigations to terms in the bound\. The experiments test complementary implications by varying hard\-negative count, distractor composition at fixed length, and the recall–interference tradeoff of a retrieval gate\.

## Formalizing context poisoning

Let a queryqqrequest an answery⋆y^\{\\star\}from a contextCN=\(x0,…,xN\)C\_\{N\}=\(x\_\{0\},\\ldots,x\_\{N\}\)\. The index set of relevant evidence isR⊆\{0,…,N\}R\\subseteq\\\{0,\\ldots,N\\\}, andD=\{0,…,N\}∖RD=\\\{0,\\ldots,N\\\}\\setminus Rare distractors\. A modelfθf\_\{\\theta\}induces an answer distributionpθ​\(y∣q,CN\)p\_\{\\theta\}\(y\\mid q,C\_\{N\}\)and a score mapsθ​\(q,xi,i\)s\_\{\\theta\}\(q,x\_\{i\},i\)used, explicitly or implicitly, to select evidence\. The score includes token semantics and position: for a transformer head one may write

si=⟨WQ​hq,WK​\(hi\+pi\)⟩d,s\_\{i\}=\\frac\{\\langle W\_\{Q\}h\_\{q\},W\_\{K\}\(h\_\{i\}\+p\_\{i\}\)\\rangle\}\{\\sqrt\{d\}\},\(1\)wherehih\_\{i\}is a contextual token representation andpip\_\{i\}denotes a positional code or bias\.

###### Definition 1\(Context poisoning curve\)\.

Fix a task distribution𝒟\\mathcal\{D\}, a short reference lengthN0N\_\{0\}, and an augmentation operatorTNT\_\{N\}that appendsN−N0N\-N\_\{0\}irrelevant or nuisance tokens while preserving the answer\. The accuracy at lengthNNis

A\(N\)=ℙ\(q,CN0,y⋆\)∼𝒟\[arg​maxypθ\(y∣q,TN\(CN0\)\)=y⋆\]\.A\(N\)=\\mathbb\{P\}\_\{\(q,C\_\{N\_\{0\}\},y^\{\\star\}\)\\sim\\mathcal\{D\}\}\\left\[\\operatorname\*\{arg\\,max\}\_\{y\}p\_\{\\theta\}\(y\\mid q,T\_\{N\}\(C\_\{N\_\{0\}\}\)\)=y^\{\\star\}\\right\]\.\(2\)The poisoning curve isP⁡\(N\)=A⁡\(N0\)−A⁡\(N\)P\(N\)=A\(N\_\{0\}\)\-A\(N\)\. Context poisoning occurs on\[N0,N1\]\[N\_\{0\},N\_\{1\}\]whenP⁡\(N\)P\(N\)is positive and non\-negligible for someN≤N1N\\leq N\_\{1\}, even thoughTNT\_\{N\}does not change the ground\-truth answer\.

###### Definition 2\(Score aliasing and effective distractor count\)\.

For a queryqq, relevant itemr∈Rr\\in R, and distractori∈Di\\in D, the pair\(r,i\)\(r,i\)isε\\varepsilon\-score\-aliased if\|si−sr\|≤ε\|s\_\{i\}\-s\_\{r\}\|\\leq\\varepsilon\. It isε\\varepsilon\-position\-aliased if the positional contribution obeys

\|⟨WQ​hq,WK​pi⟩−⟨WQ​hq,WK​pr⟩\|≤ε\.\\left\|\\langle W\_\{Q\}h\_\{q\},W\_\{K\}p\_\{i\}\\rangle\-\\langle W\_\{Q\}h\_\{q\},W\_\{K\}p\_\{r\}\\rangle\\right\|\\leq\\varepsilon\.\(3\)The effective distractor countNeff​\(q,C\)N\_\{\\rm eff\}\(q,C\)is the number of distractors whose scores have non\-negligible upper\-tail probability near a relevant score\. In the theorem below,Neff=NN\_\{\\rm eff\}=Nby construction; in a system with retrieval or gating,NeffN\_\{\\rm eff\}is the number of tokens or chunks surviving the gate\.

Figure[1](https://arxiv.org/html/2609.22101#Sx3.F1)summarizes the issue\. Long contexts produce many opportunities for an irrelevant token to become close to, or exceed, the relevant score\. This can happen semantically, positionally, or through attention\-normalization effects\. The key mathematical feature is not the average distractor score; it is the maximum distractor score\.

A\. Score aliasing in a long inputpositiond1d\_\{1\}d2d\_\{2\}d3d\_\{3\}x⋆x\_\{\\star\}d4d\_\{4\}d5d\_\{5\}djd\_\{j\}dNd\_\{N\}x⋆x\_\{\\star\}djd\_\{j\}qqs⋆s\_\{\\star\}sj≈s⋆s\_\{j\}\\approx s\_\{\\star\}white box = decisive evidence; hatched box = aliased distractorB\. Extreme\-value interferenceattention logits⋆s\_\{\\star\}MNM\_\{N\}required margintypical distractorsMN=maxi⁡ZiM\_\{N\}=\\max\_\{i\}Z\_\{i\}Poisoning event:MNM\_\{N\}is large enough thatα⋆=exp⁡\(s⋆/τ\)/∑iexp⁡\(si/τ\)\\alpha\_\{\\star\}=\\exp\(s\_\{\\star\}/\\tau\)/\\sum\_\{i\}\\exp\(s\_\{i\}/\\tau\)falls below an evidence threshold\.

Figure 1:Score and positional aliasing in a long context\. A relevant tokenx⋆x\_\{\\star\}competes not with an average distractor but with the maximum of many distractors\. AsNeffN\_\{\\rm eff\}grows, the right tail creates aliased distractors whose logits can dominate the softmax denominator\.
## A theorem for attention\-mediated poisoning

The theorem studies a single retrieval bottleneck\. This abstraction is intentionally small: many LLM computations are multi\-layer and multi\-head, yet final answer quality often depends on whether decisive evidence receives enough usable representation mass\. The result states that a fixed signal margin cannot protect against an unbounded number of effective distractors\.

###### Definition 3\(ρ\\rho\-faithful evidence use\)\.

Consider a model with attention massα⋆,N\\alpha\_\{\\star,N\}assigned to the decisive evidence\. For a thresholdρ∈\(0,1\)\\rho\\in\(0,1\)and base accuracya0∈\[0,1\]a\_\{0\}\\in\[0,1\], the decoder isρ\\rho\-faithful if

ℙ⁡\(y^=y⋆∣α⋆,N<ρ\)≤a0\.\\mathbb\{P\}\(\\widehat\{y\}=y^\{\\star\}\\mid\\alpha\_\{\\star,N\}<\\rho\)\\leq a\_\{0\}\.\(4\)Herea0a\_\{0\}may be the chance accuracy of a multiple\-choice task, the accuracy of a prior\-only model, or the accuracy of an abstaining verifier\.

###### Theorem 1\(Extreme\-value attention poisoning\)\.

Fix temperatureτ\>0\\tau\>0, thresholdρ∈\(0,1\)\\rho\\in\(0,1\), base accuracya0∈\[0,1\]a\_\{0\}\\in\[0,1\], and noise scaleσ\>0\\sigma\>0\. For each context lengthNN, let the context contain one decisive item with attention logitS⋆≤γS\_\{\\star\}\\leq\\gammaalmost surely andNNeffective distractors with independent logits

Z1,…,ZN​∼iid​𝒩​\(0,σ2\)\.Z\_\{1\},\\ldots,Z\_\{N\}\\overset\{\\rm iid\}\{\\sim\}\\mathcal\{N\}\(0,\\sigma^\{2\}\)\.\(5\)Let

α⋆,N=exp⁡\(S⋆/τ\)exp⁡\(S⋆/τ\)\+∑i=1Nexp⁡\(Zi/τ\)\\alpha\_\{\\star,N\}=\\frac\{\\exp\(S\_\{\\star\}/\\tau\)\}\{\\exp\(S\_\{\\star\}/\\tau\)\+\\sum\_\{i=1\}^\{N\}\\exp\(Z\_\{i\}/\\tau\)\}\(6\)be the relevant attention mass, and assume the decoder isρ\\rho\-faithful\. Then the expected accuracyAN=ℙ⁡\(y^=y⋆\)A\_\{N\}=\\mathbb\{P\}\(\\widehat\{y\}=y^\{\\star\}\)satisfies

AN≤a0\+\(1−a0\)​\[Φ⁡\(γ\+τ​log⁡1−ρρσ\)\]N,A\_\{N\}\\leq a\_\{0\}\+\(1\-a\_\{0\}\)\\left\[\\Phi\\left\(\\tfrac\{\\gamma\+\\tau\\log\\tfrac\{1\-\\rho\}\{\\rho\}\}\{\\sigma\}\\right\)\\right\]^\{N\},\(7\)whereΦ\\Phiis the standard normal cdf\. Consequently, ifγ,τ,ρ,σ\\gamma,\\tau,\\rho,\\sigmaare fixed anda0a\_\{0\}is fixed, thenlim supN→∞AN≤a0\\limsup\_\{N\\rightarrow\\infty\}A\_\{N\}\\leq a\_\{0\}\. Conversely, for the upper bound to permitAN≥1−εA\_\{N\}\\geq 1\-\\varepsilonwithε<1−a0\\varepsilon<1\-a\_\{0\}, it is necessary that

γ\\displaystyle\\gamma≥σ​Φ−1​\(\(1−ε−a01−a0\)1/N\)−τ​log⁡1−ρρ\\displaystyle\\geq\\sigma\\Phi^\{\-1\}\\left\(\\left\(\\frac\{1\-\\varepsilon\-a\_\{0\}\}\{1\-a\_\{0\}\}\\right\)^\{1/N\}\\right\)\-\\tau\\log\\frac\{1\-\\rho\}\{\\rho\}\(8\)=σ​2​log⁡N\+O⁡\(1\)\\displaystyle=\\sigma\\sqrt\{2\\log N\}\+O\(1\)for fixedε,a0,ρ,τ,σ\\varepsilon,a\_\{0\},\\rho,\\tau,\\sigmaasN→∞N\\to\\infty\.

#### Interpretation\.

Equation \([7](https://arxiv.org/html/2609.22101#Sx4.E7)\) is a formal poisoning curve\. It says that the model’s usable memory decays as the chance of at least one high\-scoring distractor increases\. The theorem also explains why “just attend over more tokens” can fail: unless architecture or training increases the relevant margin withNN, the attention mass on the relevant evidence eventually becomes too small\. The dependence is onNeffN\_\{\\rm eff\}, not necessarily raw token length\. A perfect gate that reduces the candidate set fromNNtoKKchanges the bound by replacingNNwithKK\.

###### Corollary 1\(Random relevant logit\)\.

SupposeS⋆S\_\{\\star\}is independent ofZ1,…,ZNZ\_\{1\},\\ldots,Z\_\{N\}andℙ⁡\(S⋆\>γ\)≤δ\\mathbb\{P\}\(S\_\{\\star\}\>\\gamma\)\\leq\\delta\. Then

AN≤δ\+a0\+\(1−a0\)​\[Φ⁡\(γ\+τ​log⁡1−ρρσ\)\]N\.A\_\{N\}\\leq\\delta\+a\_\{0\}\+\(1\-a\_\{0\}\)\\left\[\\Phi\\left\(\\frac\{\\gamma\+\\tau\\log\\frac\{1\-\\rho\}\{\\rho\}\}\{\\sigma\}\\right\)\\right\]^\{N\}\.\(9\)Thus occasional high relevant logits help only through the tail probabilityδ\\delta; robust long\-context use requires systematic margin growth or a smaller effective distractor set\.

## Mechanisms predicted by the theorem

#### Softmax dilution\.

Even when no single distractor wins, the denominator∑iexp⁡\(Zi/τ\)\\sum\_\{i\}\\exp\(Z\_\{i\}/\\tau\)can grow\. The theorem uses a maximum event because it yields a clean lower bound on failure; denominator growth gives a second, cumulative failure route\. Smaller temperatureτ\\tausharpens attention, but it also increases sensitivity to logit noise and can over\-commit to an aliased distractor\.

#### Position aliasing\.

Long\-context extrapolation requires distinguishing positions never or rarely seen during training\. Methods such as ALiBi and RoPE encode useful inductive biases\([23](https://arxiv.org/html/2609.22101#bib.bib21);[29](https://arxiv.org/html/2609.22101#bib.bib27)\), while interpolation and scaling methods extend the range of positional features\([5](https://arxiv.org/html/2609.22101#bib.bib5);[22](https://arxiv.org/html/2609.22101#bib.bib20)\)\. The definition above highlights a failure case: if two far\-apart positions induce similar query\-key contributions, a distractor can become indistinguishable from evidence for a given head\. “Found in the middle” style interventions can be interpreted as changing positional logits to reduce the number of near\-ties around relevant evidence\([34](https://arxiv.org/html/2609.22101#bib.bib31)\)\.

#### Retrieval versus use\.

The result is compatible with work showing that retrieval can be imperfect\([21](https://arxiv.org/html/2609.22101#bib.bib19);[14](https://arxiv.org/html/2609.22101#bib.bib13)\), but it also covers the stronger case in which retrieval succeeds while problem solving still degrades\([10](https://arxiv.org/html/2609.22101#bib.bib10)\)\. In the theorem, the evidence item is present, but its score advantage is bounded; failure arises as the growing candidate set increases the maximum competing score\. In a multi\-step reasoning problem, similar interference can occur at multiple evidence selection stages\.

## Architectural alternatives

The bound suggests a design principle: long\-context systems should not merely increase the raw token window; they should controlNeffN\_\{\\rm eff\}, amplify the evidence margin, and verify the evidence before answer generation\. Figure[2](https://arxiv.org/html/2609.22101#Sx6.F2)shows a candidate architecture\.

long contextCNC\_\{N\}segmenterB1,…,BmB\_\{1\},\\ldots,B\_\{m\}evidence gateGK​\(q,CN\)G\_\{K\}\(q,C\_\{N\}\)KKevidencecellsverifier andcompressorshort\-contextdecoderalias\-resistantposition codecontrastiveanti\-poison losscitation andconsistency audity^\\widehat\{y\}O⁡\(N\)O\(N\)candidatesK≪NK\\ll NretainedcontrolNeffN\_\{\\rm eff\}and increase margin

Figure 2:Poison\-resilient long\-context architecture\. A gate converts a rawO⁡\(N\)O\(N\)context intoKKevidence cells, alias\-resistant positional coding reduces near\-ties, contrastive losses push distractors below evidence, and a verifier forces the decoder to reason over recited, auditable evidence rather than the full noisy context\.### Evidence bottleneck and retrieve\-then\-reason

A model\-agnostic mitigation is to split long\-context processing into two calls or two internal phases: retrieve and recite the evidence, then solve from the shorter evidence prompt\. This resembles retrieval\-augmented generation\([20](https://arxiv.org/html/2609.22101#bib.bib18);[13](https://arxiv.org/html/2609.22101#bib.bib12);[16](https://arxiv.org/html/2609.22101#bib.bib14)\), but the retrieval source may be the user’s own context rather than an external corpus\. The theorem predicts that, conditional on retaining the decisive evidence, the extreme\-value term changes fromΦ​\(t\)N\\Phi\(t\)^\{N\}toΦ​\(t\)K\\Phi\(t\)^\{K\}\. Evidence omissions introduce a separate penalty governed by the gate’s miss probabilityη\\eta\. The intervention is therefore useful only when reducing the candidate set does not substantially reduce evidence recall\.

###### Proposition 1\(Gate substitution\)\.

Assume the theorem’s conditions after a gateGKG\_\{K\}returns a set of at mostKKdistractors and includes the decisive evidence with probability at least1−η1\-\\eta\. Then

AN≤η\+a0\+\(1−a0\)​\[Φ⁡\(γ\+τ​log⁡1−ρρσ\)\]K\.A\_\{N\}\\leq\\eta\+a\_\{0\}\+\(1\-a\_\{0\}\)\\left\[\\Phi\\left\(\\frac\{\\gamma\+\\tau\\log\\frac\{1\-\\rho\}\{\\rho\}\}\{\\sigma\}\\right\)\\right\]^\{K\}\.\(10\)The gate improves the bound when the recall penaltyη\\etais smaller than the reduction in extreme\-value risk from replacingNNbyKK\.

### Alias\-resistant positional and memory design

Position encodings should be evaluated by their worst\-case confusability, not only by perplexity\. One can regularize heads so that for sampled query statesqqand relevant positionsrr,

ℒalias=𝔼\[log∑i∈Dexp\(si−sr\+m⁡\(i,r\)τa\)\],\\mathcal\{L\}\_\{\\rm alias\}=\\mathbb\{E\}\\left\[\\log\\sum\_\{i\\in D\}\\exp\\left\(\\frac\{s\_\{i\}\-s\_\{r\}\+m\(i,r\)\}\{\\tau\_\{a\}\}\\right\)\\right\],\(11\)wherem⁡\(i,r\)m\(i,r\)is a distance\-aware margin\. This loss penalizes the log\-sum\-exp of distractor gaps and directly targets the denominator and maximum terms in the theorem\. For streaming applications, preserving a small set of attention sinks can stabilize decoding\([31](https://arxiv.org/html/2609.22101#bib.bib29)\); however, sinks should be separated from semantic evidence cells so that sink tokens do not become spurious evidence\.

### Verifier\-mediated memory

A verifier can transform long\-context use from implicit attention to explicit evidence checking\. Given candidate cellsE1,…,EKE\_\{1\},\\ldots,E\_\{K\}, the verifier produces a structured state

z=\(claims,support spans,conflicts,abstain flag\)\.z=\(\\text\{claims\},\\text\{support spans\},\\text\{conflicts\},\\text\{abstain flag\}\)\.\(12\)The decoder receiveszzand a compact evidence prompt\. This creates two audit points: whether the key evidence was selected, and whether the answer follows from selected evidence\. It also enables selective abstention when the evidence margin is below a calibrated threshold\.

### Training objective

For synthetic and real long\-context tasks, construct triples\(q,x⋆,D,y⋆\)\(q,x\_\{\\star\},D,y^\{\\star\}\)with hard negatives\. Lets⋆s\_\{\\star\}be the score for the decisive evidence andsis\_\{i\}for distractors\. A direct anti\-poison objective is

ℒ=\\displaystyle\\mathcal\{L\}=\{\}ℒans−λ​log⁡exp⁡\(s⋆/τ\)exp⁡\(s⋆/τ\)\+∑i∈Dexp⁡\(si/τ\)\\displaystyle\\mathcal\{L\}\_\{\\rm ans\}\-\\lambda\\log\\frac\{\\exp\(s\_\{\\star\}/\\tau\)\}\{\\exp\(s\_\{\\star\}/\\tau\)\+\\sum\_\{i\\in D\}\\exp\(s\_\{i\}/\\tau\)\}\(13\)\+μ∑i∈Dmax\{0,si−s⋆\+mi\},\\displaystyle\+\\mu\\sum\_\{i\\in D\}\\max\\\{0,s\_\{i\}\-s\_\{\\star\}\+m\_\{i\}\\\},wheremim\_\{i\}increases for semantically close or positionally confusable distractors\. The second term controls attention mass; the third controls the maximum distractor gap\. Prompt\-compression methods such as LongLLMLingua are complementary when they preserve decisive evidence and remove low\-utility tokens\([17](https://arxiv.org/html/2609.22101#bib.bib15)\)\. Table[1](https://arxiv.org/html/2609.22101#Sx6.T1)summarizes these mitigation mechanisms and their corresponding effects on the theoretical bound\.

Table 1:Mitigation mechanisms and their mathematical target\.

## Experimental setup and metrics

The theory recommends measuring the whole poisoning curve, not just the maximum accepted length\. A benchmark instance should preserve the short\-context answer while independently varying raw lengthNN, effective distractor countNeffN\_\{\\rm eff\}, relevant position, number of evidence items, semantic similarity of distractors, and positional regime\. Existing suites such as LongBench, L\-Eval, ZeroSCROLLS, and RULER provide useful starting points\([2](https://arxiv.org/html/2609.22101#bib.bib2);[1](https://arxiv.org/html/2609.22101#bib.bib1);[27](https://arxiv.org/html/2609.22101#bib.bib25);[14](https://arxiv.org/html/2609.22101#bib.bib13)\)\.

We report four quantities where available: \(i\) short\-context accuracyA⁡\(N0\)A\(N\_\{0\}\), \(ii\) the poisoning curveP⁡\(N\)=A⁡\(N0\)−A⁡\(N\)P\(N\)=A\(N\_\{0\}\)\-A\(N\), \(iii\) evidence recall, and \(iv\) conditional answer accuracy after successful retrieval\. The last quantity separates retrieval failure from evidence\-use failure\([10](https://arxiv.org/html/2609.22101#bib.bib10)\)\. Paired conditions reuse the same questions, and failure outputs are inspected when possible to distinguish supplied\-decoy confusion from hallucination or formatting error\.

Table 2:Overview of the controlled studies\. The first two studies are reported in the main paper; the gate intervention is reported in the appendix\. The designs vary complementary axes rather than repeating the same manipulation\.Table[2](https://arxiv.org/html/2609.22101#Sx7.T2)divides the hypothesis space across candidate count, distractor composition, and mitigation\. We summarize controlled study properties rather than model architecture because the hosted systems do not expose comparable internal specifications\. No single study validates the full theory; together they test distinct, non\-redundant implications\.

## Empirical study: retrieval decay with embedded hard negatives

[Experimental setup and metrics](https://arxiv.org/html/2609.22101#Sx7)introduces a minimal synthetic test of Theorem[1](https://arxiv.org/html/2609.22101#Thmtheorem1)—key\-value retrieval with a variable number of hard negatives\. We carry out that test here, checking whether the theorem’s central prediction, that accuracy decays toward a chance floor as the number of distractors grows, holds for a modern production model\.

#### Setup\.

We embed a single target fact in a passage ofNNsurrounding sentences and ask GPT\-4o to retrieve it\. The filler is drawn from a public\-domain novel, so it reads as ordinary prose rather than an artificial list\. Critically, the target fact is not the only sentence of its kind: several*decoy*sentences, each stating a different value in the same natural phrasing as the true fact, are woven into the passage\. This makes the task a genuine test of retrieval under hard negatives rather than trivial pattern matching—the model must identify*which*value is being asked about from narrative context, not merely notice that a value is present\. The target’s position is randomized on each trial, andNNis varied across\{100,300,500,1000\}\\\{100,300,500,1000\\\}with1010independent trials per condition\.

#### Prediction\.

Theorem[1](https://arxiv.org/html/2609.22101#Thmtheorem1)predicts that retrieval accuracy decays toward a chance floora0a\_\{0\}asNNgrows, following

A⁡\(N\)≈a0\+\(1−a0\)​qN,q=Φ⁡\(b/σ\)∈\(0,1\),A\(N\)\\approx a\_\{0\}\+\(1\-a\_\{0\}\)\\,q^\{N\},\\qquad q=\\Phi\(b/\\sigma\)\\in\(0,1\),\(14\)unless the target’s evidence margin grows withNN\. We fit Eq\. \([14](https://arxiv.org/html/2609.22101#Sx8.E14)\) to the observed accuracy by nonlinear least squares, and we inspected every incorrect response individually to confirm that failures reflect genuine confusion among competing candidates \(the model’s answer was always one of the decoy values actually present in that trial’s passage\) rather than hallucination or task misunderstanding\.

Table 3:Retrieval accuracy vs\. number of distractorsNN, GPT\-4o, 10 trials per condition\.
#### Results\.

Table[3](https://arxiv.org/html/2609.22101#Sx8.T3)shows accuracy holding at100%100\\%atN=100N=100before dropping to a stable80%80\\%byN=300N=300and remaining there throughN=1000N=1000\. Fitting Eq\. \([14](https://arxiv.org/html/2609.22101#Sx8.E14)\) givesa^0=0\.777\\hat\{a\}\_\{0\}=0\.777,q^=0\.9962\\hat\{q\}=0\.9962, withR2=0\.74R^\{2\}=0\.74\.

#### Interpretation\.

These results provide initial behavioral evidence consistent with the theorem’s qualitative prediction: retrieval accuracy decreases as exposure to hard\-negative distractors grows\. Every observed error selected a decoy value present in the prompt, ruling out hallucination, refusal, or task misunderstanding as the immediate failure type\. Because the experiment does not observe internal attention scores, it does not establish attention interference as the unique underlying mechanism\. The fitted exponential form is also exploratory rather than conclusive \(R2=0\.74R^\{2\}=0\.74\): accuracy drops betweenN=100N=100andN=300N=300and then plateaus throughN=1000N=1000, suggesting a threshold\-like transition over the tested range\. A denser sweep around this transition and more trials per condition are needed to estimate the curve’s shape reliably\.

This study varies candidate exposure but cannot separate raw count from candidate composition\. The next study holds record count and approximate token length fixed while changing the form of competition\.

## Empirical study: distractor confusability at fixed context length

We instantiate the evaluation protocol with a controlled study that holds raw context length fixed while varying observable properties of distractor confusability\. This design tests a behavioral implication of score aliasing \(Definition[2](https://arxiv.org/html/2609.22101#Thmdefinition2)\): retrieval performance may depend not only on the number of tokens or records in the context, but also on how strongly the distractors compete with the gold evidence\. Because the experiment does not directly observe attention scores, it should be interpreted as a behavioral test of this implication rather than a direct measurement of the effective distractor countNeffN\_\{\\rm eff\}\.

#### Setup\.

Each instance is a key–value retrieval query overN=300N=300records \(approximately20,00020\{,\}000–21,00021\{,\}000tokens after length matching\) with exactly one gold record; the model must return the answer and cite the evidence record ID\. We evaluate three models from different model families and deployment settings—Claude Sonnet 4, GPT\-4\.1, and Gemma 3 \(12B\)—at temperature zero\. HoldingNNfixed, we compare four distractor constructions: unrelated filler, same\-domain/different\-attribute records, same\-format/different\-entity records, and near\-alias records whose entities share a surname with the gold entity\. These conditions separately manipulate topical overlap, structural overlap, and entity\-name overlap; they should not be interpreted as points on a single monotonic similarity scale\. Records are padded to a fixed length so the manipulation is not confounded with token count, and evidence position is balanced across beginning, middle, and end\. The full protocol, prompts, and per\-model tables are provided in the appendix \([Appendix CDistractor confusability ablation: full protocol and results](https://arxiv.org/html/2609.22101#A3)\)\.

#### Results\.

Table[4](https://arxiv.org/html/2609.22101#Sx9.T4)and Figure[3](https://arxiv.org/html/2609.22101#Sx9.F3)summarize the outcome\. Accuracy is near\-ceiling for unrelated and same\-domain distractors for all three models, then decreases in the same\-format condition: Claude falls to0\.8890\.889\(an11\.111\.1percentage\-point paired drop against baseline,95%95\\%CI\[\+2\.8,\+22\.2\]\[\+2\.8,\+22\.2\]—the only statistically significant paired contrast in the study\), GPT\-4\.1 falls to0\.9670\.967, and Gemma falls to approximately0\.850\.85\. The character\-trigram Jaccard measure serves specifically as an entity\-name lexical\-overlap check\. It remains below0\.010\.01for the first three conditions and rises sharply to0\.3170\.317only in the near\-alias condition \(Figure[3](https://arxiv.org/html/2609.22101#Sx9.F3), right\)\. Thus, the decrease in the same\-format condition occurs despite low entity\-name overlap and cannot be explained by name similarity alone\.

Table 4:Answer accuracy by model and distractor construction at fixed context length \(N=300N=300records,∼\\sim20K tokens, errors excluded\)\. The rightmost column reports entity\-name lexical overlap, measured by character\-trigram Jaccard similarity to the gold entity\. Gemma values are approximate due to a high endpoint error rate\. Full results with evidence recall, valid counts, and paired contrasts are provided in the appendix \([Appendix CDistractor confusability ablation: full protocol and results](https://arxiv.org/html/2609.22101#A3)\)\.![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/Exp2Figures.png)Figure 3:Distractor\-confusability ablation at fixed context length \(N=300N=300\)\.Left:answer accuracy and evidence recall across four distractor constructions; all models show their largest decrease in the same\-format condition, while the two frontier models recover toward baseline in the near\-alias condition\.Right:entity\-name lexical\-overlap check—the character\-trigram Jaccard measure is near zero for the first three conditions and rises only in the near\-alias condition\. The accuracy decrease in the same\-format condition therefore cannot be attributed to high entity\-name overlap\.
#### Two findings\.

First,*structural overlap is more damaging in these data than entity\-name overlap*\. The largest observed degradation for every model occurs in the same\-format condition, where all300300records share the gold record’s template but refer to different entities\. This pattern is behaviorally consistent with increased score aliasing, because a shared structure may create a larger set of competitive candidates\. However, the experiment does not directly observe attention scores or determine which distractors fall within anε\\varepsilon\-aliasing band\. Second, we observe a*near\-alias paradox*: despite substantially greater entity\-name similarity, accuracy in the near\-alias condition recovers toward baseline for the two frontier models \(Claude1\.0001\.000, GPT\-4\.10\.9830\.983\)\. One possible explanation is that the first name remains a discriminative cue, whereas structurally identical records provide fewer template\-level features for distinguishing the gold record\. This explanation is a hypothesis rather than a directly tested mechanism\. Gemma shows the largest drops overall, but its endpoint rejected96%96\\%of these∼\\sim20K\-token calls, so its estimates are underpowered and should be interpreted as suggestive\.

#### Interpretation and cross\-study synthesis\.

The study supports the narrower conclusion that distractor composition matters even when record count and approximate token length are held constant\. The same\-format condition produces the largest observed accuracy decrease, whereas the condition with the greatest entity\-name overlap does not\. This pattern is consistent with the distinction between raw context size and effective distractor count, although the experiment does not directly measureNeffN\_\{\\rm eff\}or the underlying attention\-score geometry\.

Under the proposed interpretation, structurally identical records may increase the number of candidates that compete strongly with the gold record, while the near\-alias condition may preserve discriminative entity\-level cues\. The retrieval\-decay study complements this result by varying the number of exposed hard negatives rather than their construction\. Together, the two studies indicate that neither raw token count nor lexical similarity alone explains long\-context failure; candidate exposure and structural confusability both matter\. This synthesis motivates the appendix gate experiment, which tests whether reducing the competitive candidate set improves evidence use once retrieval recall is measured separately\. The findings also motivate defenses such as record\-format diversification, explicit provenance metadata, and retrieval mechanisms that penalize suspiciously repetitive templates\.

## Limitations and conclusion

The theorem isolates one long\-context failure mechanism rather than modeling an entire LLM\. It assumes independent Gaussian distractor logits, a single decisive item, and a threshold model of evidence use\. Real systems contain correlated representations, multiple heads, residual pathways, memorized priors, tools, and instruction hierarchies\. The result should therefore be interpreted as a mechanistic diagnostic, not as a calibrated predictor for a specific model\. The appendix attention analyses test these assumptions directly and show that, although the Gaussian i\.i\.d\. abstraction is not exact, the predicted extreme\-value scale remains a useful reference\.

The empirical evidence is also deliberately controlled\. The retrieval\-decay study uses only1010trials per condition, while the fixed\-length ablation measures behavioral proxies for confusability rather than attention\-score margins directly\. Unequal endpoint reliability also makes the Gemma estimates suggestive\. Both main\-paper studies emphasize retrieval\-style tasks, so code repositories, conversations, and temporally conflicting documents may exhibit additional failure modes\.

Within these limits, the paper establishes a consistent result across theory and experiment\. A fixed evidence margin becomes increasingly fragile as the effective distractor set grows; retrieval performance depends on both candidate exposure and distractor structure; and reducing the competitive set helps only when required evidence is preserved\. Future work should therefore cross context length with distractor construction, connect internal score margins to end\-to\-end errors, and compare fixed with adaptive retrieval budgets\. Gate studies should report final accuracy, all\-support recall, and conditional accuracy together, because any one metric can conceal failure in another stage\.

The central conclusion is that a larger context window increases both available information and the opportunities for distractors to compete with decisive evidence\. The relevant quantity is the effective number and strength of confusable tokens or chunks, not raw window length alone\. Robust long\-context systems should therefore combine evidence selection, alias control, recall\-preserving retrieval, and explicit verification\. A context window is capacity; usable memory is capacity plus reliable addressing\.

## References

- Anet al\.\(2024\)C\. An, S\. Gong, M\. Zhong, X\. Zhao, M\. Li, J\. Zhang, L\. Kong, and X\. QiuL\-eval: instituting standardized evaluation for long context language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p1.1),[Long\-context evaluation and failure analysis\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px3.p1.1),[Experimental setup and metrics](https://arxiv.org/html/2609.22101#Sx7.p1.1)\.
- Baiet al\.\(2024\)Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou, Y\. Dong, J\. Tang, and J\. LiLongBench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p1.1),[Long\-context evaluation and failure analysis\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px3.p1.1),[Experimental setup and metrics](https://arxiv.org/html/2609.22101#Sx7.p1.1)\.
- Beltagyet al\.\(2020\)I\. Beltagy, M\. E\. Peters, and A\. CohanLongformer: the long\-document transformer\.External Links:2004\.05150Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Brownet al\.\(2020\)T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 1877–1901\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p1.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2023\)S\. Chen, S\. Wong, L\. Chen, and Y\. TianExtending context window of large language models via positional interpolation\.External Links:2306\.15595Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1),[Position aliasing\.](https://arxiv.org/html/2609.22101#Sx5.SS0.SSS0.Px2.p1.1)\.
- Choromanskiet al\.\(2021\)K\. Choromanski, V\. Likhosherstov, D\. Dohan, X\. Song, A\. Gane, T\. Sarlos, P\. Hawkins, J\. Davis, A\. Mohiuddin, L\. Kaiser, D\. Belanger, L\. Colwell, and A\. WellerRethinking attention with performers\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Daiet al\.\(2019\)Z\. Dai, Z\. Yang, Y\. Yang, J\. Carbonell, Q\. V\. Le, and R\. SalakhutdinovTransformer\-XL: attentive language models beyond a fixed\-length context\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 2978–2988\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Daoet al\.\(2022\)T\. Dao, D\. Y\. Fu, S\. Ermon, A\. Rudra, and C\. ReFlashAttention: fast and memory\-efficient exact attention with IO\-awareness\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 16344–16359\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Devlinet al\.\(2019\)J\. Devlin, M\. Chang, K\. Lee, and K\. ToutanovaBERT: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of NAACL\-HLT,pp\. 4171–4186\.Cited by:[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Duet al\.\(2025\)Y\. Du, M\. Tian, S\. Ronanki, S\. Rongali, S\. Bodapati, A\. Galstyan, A\. Wells, R\. Schwartz, E\. A\. Huerta, and H\. PengContext length alone hurts LLM performance despite perfect retrieval\.InFindings of the Association for Computational Linguistics: EMNLP,pp\. 23281–23298\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p1.1),[Long\-context evaluation and failure analysis\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px3.p1.1),[Retrieval versus use\.](https://arxiv.org/html/2609.22101#Sx5.SS0.SSS0.Px3.p1.1),[Experimental setup and metrics](https://arxiv.org/html/2609.22101#Sx7.p2.1)\.
- Gemma Team \(2025\)Gemma TeamGemma 3 technical report\.arXiv preprint arXiv:2503\.19786\.Cited by:[Appendix D](https://arxiv.org/html/2609.22101#A4.p2.1),[Appendix E](https://arxiv.org/html/2609.22101#A5.p2.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[Appendix D](https://arxiv.org/html/2609.22101#A4.p7.1)\.
- Guuet al\.\(2020\)K\. Guu, K\. Lee, Z\. Tung, P\. Pasupat, and M\. ChangREALM: retrieval\-augmented language model pre\-training\.InInternational Conference on Machine Learning,pp\. 3929–3938\.Cited by:[Selection, compression, and memory\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px2.p1.1),[Evidence bottleneck and retrieve\-then\-reason](https://arxiv.org/html/2609.22101#Sx6.SSx1.p1.1)\.
- Hsiehet al\.\(2024\)C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, and B\. GinsburgRULER: what’s the real context size of your long\-context language models?\.InConference on Language Modeling \(COLM\),External Links:2404\.06654Cited by:[Appendix F](https://arxiv.org/html/2609.22101#A6.SSx1.SSS0.Px1.p1.1),[Introduction](https://arxiv.org/html/2609.22101#Sx1.p1.1),[Long\-context evaluation and failure analysis\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px3.p1.1),[Retrieval versus use\.](https://arxiv.org/html/2609.22101#Sx5.SS0.SSS0.Px3.p1.1),[Experimental setup and metrics](https://arxiv.org/html/2609.22101#Sx7.p1.1)\.
- Izacardet al\.\(2021\)G\. Izacard, M\. Caron, L\. Hosseini, S\. Riedel, P\. Bojanowski, A\. Joulin, and E\. GraveUnsupervised dense information retrieval with contrastive learning\.Note:arXiv:2112\.09118External Links:[Document](https://dx.doi.org/10.48550/arXiv.2112.09118)Cited by:[Appendix E](https://arxiv.org/html/2609.22101#A5.p2.1)\.
- Izacard and Grave \(2021\)G\. Izacard and E\. GraveLeveraging passage retrieval with generative models for open domain question answering\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics,pp\. 874–880\.Cited by:[Selection, compression, and memory\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px2.p1.1),[Evidence bottleneck and retrieve\-then\-reason](https://arxiv.org/html/2609.22101#Sx6.SSx1.p1.1)\.
- Jianget al\.\(2023\)H\. Jiang, Q\. Wu, X\. Luo, D\. Li, C\. Lin, Y\. Yang, and L\. QiuLongLLMlingua: accelerating and enhancing LLMs in long context scenarios via prompt compression\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Cited by:[Selection, compression, and memory\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px2.p1.1),[Training objective](https://arxiv.org/html/2609.22101#Sx6.SSx4.p1.2)\.
- Katharopouloset al\.\(2020\)A\. Katharopoulos, A\. Vyas, N\. Pappas, and F\. FleuretTransformers are RNNs: fast autoregressive transformers with linear attention\.InInternational Conference on Machine Learning,pp\. 5156–5165\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Kwiatkowskiet al\.\(2019\)T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee, K\. Toutanova, L\. Jones, M\. Kelcey, M\. Chang, A\. M\. Dai, J\. Uszkoreit, Q\. Le, and S\. PetrovNatural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 453–466\.Cited by:[Appendix E](https://arxiv.org/html/2609.22101#A5.p2.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Kuttler, M\. Lewis, W\. Yih, T\. Rocktaschel, S\. Riedel, and D\. KielaRetrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.Cited by:[Selection, compression, and memory\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px2.p1.1),[Evidence bottleneck and retrieve\-then\-reason](https://arxiv.org/html/2609.22101#Sx6.SSx1.p1.1)\.
- Liuet al\.\(2024\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[Appendix D](https://arxiv.org/html/2609.22101#A4.p3.1),[Appendix E](https://arxiv.org/html/2609.22101#A5.p1.1),[Appendix E](https://arxiv.org/html/2609.22101#A5.p2.1),[Appendix E](https://arxiv.org/html/2609.22101#A5.p4.1),[Introduction](https://arxiv.org/html/2609.22101#Sx1.p1.1),[Long\-context evaluation and failure analysis\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px3.p1.1),[Retrieval versus use\.](https://arxiv.org/html/2609.22101#Sx5.SS0.SSS0.Px3.p1.1)\.
- Penget al\.\(2023\)B\. Peng, J\. Quesnelle, H\. Fan, and E\. ShippoleYaRN: efficient context window extension of large language models\.External Links:2309\.00071Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1),[Position aliasing\.](https://arxiv.org/html/2609.22101#Sx5.SS0.SSS0.Px2.p1.1)\.
- Presset al\.\(2022\)O\. Press, N\. A\. Smith, and M\. LewisTrain short, test long: attention with linear biases enables input length extrapolation\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1),[Position aliasing\.](https://arxiv.org/html/2609.22101#Sx5.SS0.SSS0.Px2.p1.1)\.
- Raeet al\.\(2020\)J\. W\. Rae, A\. Potapenko, S\. M\. Jayakumar, and T\. P\. LillicrapCompressive transformers for long\-range sequence modelling\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Raffelet al\.\(2020\)C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. LiuExploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of Machine Learning Research21\(140\),pp\. 1–67\.Cited by:[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Robertson and Zaragoza \(2009\)S\. Robertson and H\. ZaragozaThe probabilistic relevance framework: BM25 and beyond\.Foundations and Trends in Information Retrieval3\(4\),pp\. 333–389\.Cited by:[Appendix B](https://arxiv.org/html/2609.22101#A2.p1.1),[Appendix F](https://arxiv.org/html/2609.22101#A6.SSx1.SSS0.Px2.p1.1)\.
- Shahamet al\.\(2023\)U\. Shaham, M\. Ivgi, A\. Efrat, J\. Berant, and O\. LevyZeroSCROLLS: a zero\-shot benchmark for long text understanding\.InFindings of the Association for Computational Linguistics: EMNLP,External Links:2305\.14196Cited by:[Long\-context evaluation and failure analysis\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px3.p1.1),[Experimental setup and metrics](https://arxiv.org/html/2609.22101#Sx7.p1.1)\.
- Shiet al\.\(2023\)F\. Shi, X\. Chen, K\. Misra, N\. Scales, D\. Dohan, E\. H\. Chi, N\. Scharli, and D\. ZhouLarge language models can be easily distracted by irrelevant context\.InInternational Conference on Machine Learning,pp\. 31210–31227\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p1.1),[Long\-context evaluation and failure analysis\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px3.p1.1)\.
- Suet al\.\(2024\)J\. Su, Y\. Lu, S\. Pan, A\. Murtadha, B\. Wen, and Y\. LiuRoFormer: enhanced transformer with rotary position embedding\.Neurocomputing568,pp\. 127063\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1),[Position aliasing\.](https://arxiv.org/html/2609.22101#Sx5.SS0.SSS0.Px2.p1.1)\.
- Vaswaniet al\.\(2017\)A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. PolosukhinAttention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p1.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Xiaoet al\.\(2024\)G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. LewisEfficient streaming language models with attention sinks\.InInternational Conference on Learning Representations,Cited by:[Selection, compression, and memory\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px2.p1.1),[Alias\-resistant positional and memory design](https://arxiv.org/html/2609.22101#Sx6.SSx2.p1.2)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2369–2380\.Cited by:[Appendix F](https://arxiv.org/html/2609.22101#A6.SSx1.SSS0.Px1.p1.1)\.
- Zaheeret al\.\(2020\)M\. Zaheer, G\. Guruganesh, A\. Dubey, J\. Ainslie, C\. Alberti, S\. Ontanon, P\. Pham, A\. Ravula, Q\. Wang, L\. Yang, and A\. AhmedBig bird: transformers for longer sequences\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 17283–17297\.Cited by:[Introduction](https://arxiv.org/html/2609.22101#Sx1.p2.1),[Efficient long\-context architectures\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2024\)Z\. Zhang, R\. Chen, S\. Liu, Z\. Yao, O\. Ruwase, B\. Chen, X\. Wu, and Z\. WangFound in the middle: how language models use long contexts better via plug\-and\-play positional encoding\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:2403\.04797Cited by:[Long\-context evaluation and failure analysis\.](https://arxiv.org/html/2609.22101#Sx2.SS0.SSS0.Px3.p1.1),[Position aliasing\.](https://arxiv.org/html/2609.22101#Sx5.SS0.SSS0.Px2.p1.1)\.

## Appendix AProofs

###### Proof of Theorem[1](https://arxiv.org/html/2609.22101#Thmtheorem1)\.

Define

b=γ\+τ​log⁡1−ρρ,MN=max1≤i≤N⁡Zi\.b=\\gamma\+\\tau\\log\\frac\{1\-\\rho\}\{\\rho\},\\qquad M\_\{N\}=\\max\_\{1\\leq i\\leq N\}Z\_\{i\}\.\(15\)IfMN≥bM\_\{N\}\\geq b, then there exists a distractorjjsuch that

Zj−S⋆≥Zj−γ≥τ​log⁡1−ρρ\.Z\_\{j\}\-S\_\{\\star\}\\geq Z\_\{j\}\-\\gamma\\geq\\tau\\log\\frac\{1\-\\rho\}\{\\rho\}\.\(16\)Using only this distractor in the softmax denominator gives

α⋆,N\\displaystyle\\alpha\_\{\\star,N\}=exp⁡\(S⋆/τ\)exp⁡\(S⋆/τ\)\+∑i=1Nexp⁡\(Zi/τ\)\\displaystyle=\\frac\{\\exp\(S\_\{\\star\}/\\tau\)\}\{\\exp\(S\_\{\\star\}/\\tau\)\+\\sum\_\{i=1\}^\{N\}\\exp\(Z\_\{i\}/\\tau\)\}\(17\)≤exp⁡\(S⋆/τ\)exp⁡\(S⋆/τ\)\+exp⁡\(Zj/τ\)=11\+exp⁡\(\(Zj−S⋆\)/τ\)\\displaystyle\\leq\\frac\{\\exp\(S\_\{\\star\}/\\tau\)\}\{\\exp\(S\_\{\\star\}/\\tau\)\+\\exp\(Z\_\{j\}/\\tau\)\}=\\frac\{1\}\{1\+\\exp\(\(Z\_\{j\}\-S\_\{\\star\}\)/\\tau\)\}\(18\)≤11\+\(1−ρ\)/ρ=ρ\.\\displaystyle\\leq\\frac\{1\}\{1\+\(1\-\\rho\)/\\rho\}=\\rho\.\(19\)Therefore, ifMN\>bM\_\{N\}\>b, thenα⋆,N<ρ\\alpha\_\{\\star,N\}<\\rho, and hence

\{α⋆,N≥ρ\}⊆\{MN≤b\}\.\\\{\\alpha\_\{\\star,N\}\\geq\\rho\\\}\\subseteq\\\{M\_\{N\}\\leq b\\\}\.Since theZiZ\_\{i\}are i\.i\.d\. Gaussian,MNM\_\{N\}has a continuous distribution, so

ℙ⁡\(MN≤b\)=∏i=1Nℙ⁡\(Zi≤b\)=\[Φ⁡\(bσ\)\]N\.\\mathbb\{P\}\(M\_\{N\}\\leq b\)=\\prod\_\{i=1\}^\{N\}\\mathbb\{P\}\(Z\_\{i\}\\leq b\)=\\left\[\\Phi\\left\(\\frac\{b\}\{\\sigma\}\\right\)\\right\]^\{N\}\.\(20\)Thus

ℙ⁡\(α⋆,N≥ρ\)≤ℙ⁡\(MN≤b\)=\[Φ⁡\(bσ\)\]N\.\\mathbb\{P\}\(\\alpha\_\{\\star,N\}\\geq\\rho\)\\leq\\mathbb\{P\}\(M\_\{N\}\\leq b\)=\\left\[\\Phi\\left\(\\frac\{b\}\{\\sigma\}\\right\)\\right\]^\{N\}\.\(21\)Byρ\\rho\-faithfulness,

AN\\displaystyle A\_\{N\}=ℙ⁡\(y^=y⋆,α⋆,N≥ρ\)\+ℙ⁡\(y^=y⋆,α⋆,N<ρ\)\\displaystyle=\\mathbb\{P\}\(\\widehat\{y\}=y^\{\\star\},\\alpha\_\{\\star,N\}\\geq\\rho\)\+\\mathbb\{P\}\(\\widehat\{y\}=y^\{\\star\},\\alpha\_\{\\star,N\}<\\rho\)\(22\)≤ℙ⁡\(α⋆,N≥ρ\)\+a0​ℙ​\(α⋆,N<ρ\)\\displaystyle\\leq\\mathbb\{P\}\(\\alpha\_\{\\star,N\}\\geq\\rho\)\+a\_\{0\}\\mathbb\{P\}\(\\alpha\_\{\\star,N\}<\\rho\)\(23\)=a0\+\(1−a0\)​ℙ​\(α⋆,N≥ρ\)\\displaystyle=a\_\{0\}\+\(1\-a\_\{0\}\)\\mathbb\{P\}\(\\alpha\_\{\\star,N\}\\geq\\rho\)\(24\)≤a0\+\(1−a0\)​\[Φ⁡\(γ\+τ​log⁡1−ρρσ\)\]N\.\\displaystyle\\leq a\_\{0\}\+\(1\-a\_\{0\}\)\\left\[\\Phi\\left\(\\frac\{\\gamma\+\\tau\\log\\frac\{1\-\\rho\}\{\\rho\}\}\{\\sigma\}\\right\)\\right\]^\{N\}\.\(25\)This proves the finite\-sample accuracy bound stated in Theorem[1](https://arxiv.org/html/2609.22101#Thmtheorem1)\. Ifbbis fixed, thenΦ⁡\(b/σ\)<1\\Phi\(b/\\sigma\)<1unlessb=\+∞b=\+\\infty, so the exponential term tends to zero andlim supN→∞AN≤a0\\limsup\_\{N\\to\\infty\}A\_\{N\}\\leq a\_\{0\}\.

For the bound to permit accuracy at least1−ε1\-\\varepsilon, it is necessary that

a0\+\(1−a0\)​Φ​\(b/σ\)N≥1−ε\.a\_\{0\}\+\(1\-a\_\{0\}\)\\Phi\(b/\\sigma\)^\{N\}\\geq 1\-\\varepsilon\.\(26\)Rearranging gives

Φ⁡\(b/σ\)≥\(1−ε−a01−a0\)1/N\.\\Phi\(b/\\sigma\)\\geq\\left\(\\frac\{1\-\\varepsilon\-a\_\{0\}\}\{1\-a\_\{0\}\}\\right\)^\{1/N\}\.\(27\)BecauseΦ−1\\Phi^\{\-1\}is increasing,

b≥σ​Φ−1​\(\(1−ε−a01−a0\)1/N\)\.b\\geq\\sigma\\Phi^\{\-1\}\\left\(\\left\(\\frac\{1\-\\varepsilon\-a\_\{0\}\}\{1\-a\_\{0\}\}\\right\)^\{1/N\}\\right\)\.\(28\)Substituting

b=γ\+τ​log⁡1−ρρb=\\gamma\+\\tau\\log\\frac\{1\-\\rho\}\{\\rho\}yields the finite\-sample margin requirement stated in Theorem[1](https://arxiv.org/html/2609.22101#Thmtheorem1)\.

To obtain its asymptotic form, define

r=1−ε−a01−a0∈\(0,1\)andκ=−log⁡r\>0\.r=\\frac\{1\-\\varepsilon\-a\_\{0\}\}\{1\-a\_\{0\}\}\\in\(0,1\)\\qquad\\text\{and\}\\qquad\\kappa=\-\\log r\>0\.Then

r1/N=exp⁡\(−κN\)=1−κN\+O⁡\(N−2\)\.r^\{1/N\}=\\exp\\left\(\-\\frac\{\\kappa\}\{N\}\\right\)=1\-\\frac\{\\kappa\}\{N\}\+O\(N^\{\-2\}\)\.\(29\)Using the standard Gaussian upper\-quantile asymptotic,

Φ−1​\(1−κN\+O⁡\(N−2\)\)=2​log⁡N\+O⁡\(1\)\.\\Phi^\{\-1\}\\left\(1\-\\frac\{\\kappa\}\{N\}\+O\(N^\{\-2\}\)\\right\)=\\sqrt\{2\\log N\}\+O\(1\)\.\(30\)Therefore,

γ≥σ​2​log⁡N\+O⁡\(1\),\\gamma\\geq\\sigma\\sqrt\{2\\log N\}\+O\(1\),\(31\)which proves the stated asymptotic margin requirement\. ∎

###### Proof of the corollary\.

LetE=\{S⋆≤γ\}E=\\\{S\_\{\\star\}\\leq\\gamma\\\}\. OnEE, the theorem’s proof applies conditionally\. OnEcE^\{c\}, bound accuracy by one\. Hence

AN\\displaystyle A\_\{N\}≤ℙ⁡\(Ec\)\\displaystyle\\leq\\mathbb\{P\}\(E^\{c\}\)\+ℙ⁡\(E\)​\(a0\+\(1−a0\)​\[Φ⁡\(γ\+τ​log⁡1−ρρσ\)\]N\)\\displaystyle\\quad\+\\mathbb\{P\}\(E\)\\left\(a\_\{0\}\+\(1\-a\_\{0\}\)\\left\[\\Phi\\left\(\\tfrac\{\\gamma\+\\tau\\log\\tfrac\{1\-\\rho\}\{\\rho\}\}\{\\sigma\}\\right\)\\right\]^\{N\}\\right\)\(32\)≤δ\+a0\+\(1−a0\)​\[Φ⁡\(γ\+τ​log⁡1−ρρσ\)\]N\.\\displaystyle\\leq\\delta\+a\_\{0\}\+\(1\-a\_\{0\}\)\\left\[\\Phi\\left\(\\tfrac\{\\gamma\+\\tau\\log\\tfrac\{1\-\\rho\}\{\\rho\}\}\{\\sigma\}\\right\)\\right\]^\{N\}\.\(33\)∎

## Appendix BAdditional implementation details

A practical implementation of the retrieve\-then\-reason architecture shown in Figure[2](https://arxiv.org/html/2609.22101#Sx6.F2)of the main paper can be built without changing a base LLM: chunk the context, retrieve top\-KKchunks with a lexical and dense hybrid scorer\([26](https://arxiv.org/html/2609.22101#bib.bib24)\), ask the model to quote relevant spans, and start a new short\-context call that contains only the quoted spans and the question\. A model\-internal implementation can train a gate and verifier jointly with the decoder\. In both cases, the evaluation should report gate recall separately from final answer accuracy\.

## Appendix CDistractor confusability ablation: full protocol and results

This section provides the complete protocol and supplemental tables for the distractor\-confusability study reported in the main paper \([Empirical study: distractor confusability at fixed context length](https://arxiv.org/html/2609.22101#Sx9)\)\.

### Task and distractor constructions

Each instance poses a factual question about a fictional character drawn from well\-known literary universes \(Harry Potter, The Lord of the Rings, The Chronicles of Narnia, A Song of Ice and Fire\); the entities are used only to construct newly written, factual\-style records, and no copyrighted text is reproduced\. The model receives one gold record and299299distractor records, for a total ofNrecords=300N\_\{\\mathrm\{records\}\}=300records, and must return a JSON object containing the extracted answer and the cited evidence record ID\. Illustrated for the question “What is Stannis Baratheon’s castle?” \(gold answer: Dragonstone; gold record ID: gold\-0000\), the four distractor conditions are:

- •Unrelated filler \(baseline\)\.Distractors are catalogue notes about unrelated objects \(e\.g\. “a dragon egg”\), with no lexical or semantic overlap with the gold entity or attribute\. The gold record is the only topically relevant record\.
- •Same domain, different attribute\.Distractors are facts from the same fictional universe but about different attributes and entity types \(lineages, battles, geography\), creating topical but non\-responsive overlap\.
- •Same format, different entity\.Every distractor uses the gold record’s exact template \(“Authoritative record: X’s castle is Y”\) for a clearly different character, so the model must discriminate purely on entity name among300300structurally identical records\.
- •Near\-alias entity\.Distractors use the same template and name entities that share the gold entity’s surname \(e\.g\. Robert, Renly, and Joffrey Baratheon versus Stannis Baratheon\), each giving a factually correct but wrong\-for\-the\-question answer, so a model that matches on surname alone extracts the wrong answer\.

### Controls

The context containsNrecords=300N\_\{\\mathrm\{records\}\}=300records in every condition\. Each distractor record is padded to a fixed word count with neutral filler tokens so the similarity manipulation is not confounded with text length\. Evidence position is balanced across beginning, middle, and end, and each base instance appears exactly once per similarity condition to enable within\-instance paired comparisons\. A model\-independent lexical\-similarity proxy \(character\-trigram Jaccard between each distractor entity name and the gold entity name\) is computed per condition as a manipulation check\. API errors \(timeouts, rate limits, context\-length rejections\) are excluded from accuracy calculations; only successful responses are scored\.

### Models and scale

Claude Sonnet 4 and GPT\-4\.1 were accessed through an OAuth2 gateway; Gemma 3 \(12B\) through a pay\-per\-token foundation\-model API\. Claude and GPT used6060base instances×4\\times\\,4levels=240=240conditions each; Gemma used an expanded design of460460base instances \(1,8401\{,\}840conditions\) to accumulate valid calls despite a high endpoint error rate\. Each call processes roughly20,00020\{,\}000tokens of context\. Answer accuracy uses loose string normalization \(lowercased, articles stripped, whitespace collapsed\); evidence recall requires the correct gold record ID; paired accuracy drops are estimated by paired bootstrap \(5,0005\{,\}000iterations, seed123123\)\.

### Construct validity and length matching

The entity\-name lexical\-overlap proxy remains small in the unrelated, same\-domain, and same\-format conditions \(0\.0040\.004–0\.0100\.010\) and rises to0\.3170\.317in the near\-alias condition\. This verifies that the near\-alias construction specifically increases entity\-name overlap, whereas the same\-format construction does not\. Median approximate token counts are similar across conditions, so the observed accuracy differences are not explained by large changes in context length \(Table[5](https://arxiv.org/html/2609.22101#A3.T5)\)\.

Table 5:Construct\-validity and length\-matching check: mean lexical\-similarity proxy \(character\-trigram Jaccard between distractor and gold entity names\) and median approximate token count by distractor condition, atNrecords=300N\_\{\\mathrm\{records\}\}=300\.
### API reliability

At∼\\sim20K tokens, endpoint reliability varied sharply across providers \(Table[6](https://arxiv.org/html/2609.22101#A3.T6)\)\. GPT\-4\.1 completed all calls; Claude’s error rate scaled with token count \(approximately11\.7%11\.7\\%at∼\\sim9K,35\.4%35\.4\\%at∼\\sim20K,59\.6%59\.6\\%at∼\\sim35K in companion runs\); Gemma’s endpoint rejected most∼\\sim20K\-token inputs at a context\-length limit\. All accuracy analyses use valid \(non\-error\) calls only\.

Table 6:Endpoint reliability at∼\\sim20K tokens\. Accuracy analyses are computed on valid calls only\.
### Full accuracy, evidence recall, and paired contrasts

Table[7](https://arxiv.org/html/2609.22101#A3.T7)reports answer accuracy and evidence recall for every model and distractor condition with valid counts; Table[8](https://arxiv.org/html/2609.22101#A3.T8)reports paired accuracy drops relative to the unrelated\-filler baseline\. Claude’s\+0\.111\+0\.111same\-format drop is the only paired contrast whose reported95%95\\%confidence interval excludes zero\. GPT\-4\.1 has a smaller point\-estimate drop in the same direction \(\+0\.033\+0\.033\), but its confidence interval includes zero\. Notably, GPT\-4\.1’s evidence recall remains at1\.0001\.000across all conditions: when it errs, it cites the correct record but extracts an incorrect answer, indicating an evidence\-use rather than a retrieval failure\.

Table 7:Answer accuracy and evidence recall by model and distractor condition \(Nrecords=300N\_\{\\mathrm\{records\}\}=300,∼\\sim20K tokens, errors excluded\)\. Gemma counts and values are approximate due to the96\.1%96\.1\\%error rate and should be interpreted with caution\.Table 8:Paired accuracy drop relative to the unrelated\-filler baseline \(positive==harder; paired bootstrap,5,0005\{,\}000iterations, errors excluded\)\. Gemma is omitted: sparse matched pairs \(n=4n=4–55\) preclude reliable paired testing\.Across the four condition\-level means, the descriptive Spearman correlations between the lexical\-overlap proxy and mean accuracy areρ=−0\.258\\rho=\-0\.258for Claude,ρ=−0\.738\\rho=\-0\.738for GPT\-4\.1, andρ=−0\.738\\rho=\-0\.738for Gemma\. Because these coefficients are based on only four condition\-level observations and include tied accuracy values, they are descriptive summaries rather than inferential evidence\.

### Limitations and adversarial\-RAG implications

Several caveats qualify these results\. Claude’s35\.4%35\.4\\%error rate raises a survivorship concern—valid calls may under\-represent the hardest instances—so the effect should be replicated on a higher\-throughput endpoint\. Gemma is underpowered \(72/1,84072/1\{,\}840valid\); its aggregate pattern is suggestive but not conclusive, and a dedicated run at shorter context \(N≈64N\\approx 64–128128\) would provide more reliable estimates\. The near\-alias manipulation tops out at Jaccard0\.3170\.317, which frontier models handle well; exact\-paraphrase or same\-entity/different\-date distractors could reveal stronger effects\. Because the entities are drawn from popular fiction, pretraining memorization may make the task easier than novel\-entity retrieval in production RAG, so the surprising same\-format\-over\-near\-alias ordering warrants replication with non\-fictional entities\. Finally, the present analysis evaluates the four distractor conditions atN=300N=300and therefore does not estimate an interaction between context length and distractor construction\. The observed same\-format drop suggests a potential vulnerability: injecting structurally similar records containing plausible alternative answers may degrade accuracy even when the gold evidence remains present\. In this experiment, the same\-format condition produced a larger point\-estimate drop than the near\-alias condition, although that difference was not tested directly\. The results motivate defenses that diversify record formats, attach provenance metadata, and de\-rank suspiciously repetitive templates\. The present study does not estimate how this effect changes with context length\.

## Appendix DAttention\-distribution diagnostics and dose\-response: from assumption to mechanism

Theorem[1](https://arxiv.org/html/2609.22101#Thmtheorem1)’s bound rests on a specific structural assumption: that distractor attention logits are independent, identically distributed Gaussian noise\. This section tests that assumption directly on an open\-weight model and then asks two follow\-up questions that an internal diagnostic cannot answer on its own: does the assumption violation correspond to behavioral failure, and, if raw context length alone does not explain failure, what does? These findings complement, rather than replace, the empirical studies reported in the main paper \([Empirical study: retrieval decay with embedded hard negatives](https://arxiv.org/html/2609.22101#Sx8)and[Empirical study: distractor confusability at fixed context length](https://arxiv.org/html/2609.22101#Sx9)\)\.

We use Gemma\-3\-4B\-IT\([11](https://arxiv.org/html/2609.22101#bib.bib11)\)withoutput\_attentionsenabled, run locally via Hugging Facetransformers\. Each trial builds a needle\-in\-haystack prompt: one evidence sentence stating a random secret code, embedded amongNNdistractor sentences, followed by a question asking for the code\. For each of the3434layers and88attention heads \(272272layer\-head pairs\), we sum the final answer\-generating token’s post\-softmax attention weights over the tokens belonging to each distractor sentence, producing a sentence\-level attention\-mass value\. We analyze the logarithm of this quantity after robust median/MAD standardization\. Because sentence aggregation occurs before the logarithm, this statistic is a sentence\-level log\-attention score rather than a recovered raw attention logit\. We sweepN∈\{20,50,100\}N\\in\\\{20,50,100\\\}with 10 trials each for the attention analysis \(limited by the quadratic memory cost of storing full attention weights\), and separately test purely behavioral accuracy \(no attention extraction, hence much cheaper\) at largerNNand under varying distractor content\.

![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_sink.png)Figure 4:Mean attention by sentence position only, independent of content, pooled across trials/layers/heads\.Figure[4](https://arxiv.org/html/2609.22101#A4.F4)diagnoses a positional confound at everyNNtested: attention depends measurably on position alone\. AtN=20N=20andN=50N=50this reproduces the U\-shaped “lost in the middle” effect reported by[21](https://arxiv.org/html/2609.22101#bib.bib19); atN=100N=100it instead shows a sharp step, consistent with Gemma\-3’s architecture interleaving sliding\-window and global\-attention layers\.

Table 9:Attention distribution tests byNN\(34 layers×\\times8 heads, 10 trials perNN\)\.The Gaussianity tests reject normality at everyNN, and the KS tests reject an exact Gumbel fit at everyNNfor Gemma\-3 \(Table[9](https://arxiv.org/html/2609.22101#A4.T9)\)\. The Ljung–Box test does not reject independence atN=20N=20orN=50N=50\(p=0\.958p=0\.958andp=0\.320p=0\.320, respectively\), but rejects it strongly atN=100N=100\(p<10−99p<10^\{\-99\}\)\. Thus, the Gaussian i\.i\.d\. abstraction is not quantitatively exact, although the nature of the departure changes with context length\. Heavy tails may increase extreme\-score risk, whereas dependence can either amplify or suppress it\. We therefore treat Theorem[1](https://arxiv.org/html/2609.22101#Thmtheorem1)as a diagnostic baseline rather than a calibrated quantitative bound for these models\.

![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_extreme_value.png)Figure 5:Q–Q plots of the empirical maximum standardized distractor scores against the chosen asymptotic Gumbel approximation, byNN\.Figure[5](https://arxiv.org/html/2609.22101#A4.F5)compares the empirical maximum standardized distractor scores with an asymptotic Gumbel approximation\. Under an exact i\.i\.d\. standard\-Gaussian model, the finite\-NNmaximum has CDFFN​\(x\)=Φ​\(x\)NF\_\{N\}\(x\)=\\Phi\(x\)^\{N\}; a Gumbel distribution arises only asymptotically after appropriate centering and scaling\. Rejection of the selected Gumbel approximation atN∈\{20,50,100\}N\\in\\\{20,50,100\\\}therefore does not by itself constitute a direct finite\-sample test of the Gaussian i\.i\.d\. hypothesis\.

![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_margin.png)Figure 6:Empirical mean of the standardized maximum distractor score vs\. the theorem’s own asymptotic prediction,2​log⁡N\\sqrt\{2\\log N\}\.Despite this, the theorem’s own margin\-growth prediction \(Figure[6](https://arxiv.org/html/2609.22101#A4.F6)\) tracks the data reasonably closely at two of three points—2\.31 vs\. 2\.45 atN=20N=20and 3\.02 vs\. 3\.04 atN=100N=100—showing the abstraction captures the right order of magnitude even where its distributional assumptions do not hold exactly\.

A natural question is whether the findings above reflect a general property of transformer attention or an artifact specific to Gemma\-3’s architecture, which interleaves sliding\-window \(local\) attention layers with global\-attention layers\. We repeated the identical experiment on Llama\-3\-8B\-Instruct\([12](https://arxiv.org/html/2609.22101#bib.bib32)\), which uses full causal \(global\) attention in every layer, with no sliding\-window mechanism\. Table[10](https://arxiv.org/html/2609.22101#A4.T10)summarizes the two architectures\.

Table 10:Architectural comparison of the two models tested\.![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_sink_llama3.png)Figure 7:Attention\-sink diagnostic on Llama\-3\-8B\-Instruct \(see Figure[4](https://arxiv.org/html/2609.22101#A4.F4)for Gemma\-3\)\.Figure[7](https://arxiv.org/html/2609.22101#A4.F7)shows a smooth positional profile at all threeNN, with no sharp discontinuity atN=100N=100\. The absence of the step in Llama\-3 is consistent with an architecture\-specific explanation for the Gemma\-3 pattern\. However, this two\-model comparison does not isolate sliding\-window/global layer mixing as the cause, because the models also differ in size, training, tokenization, and other architectural details\.

Table 11:Attention distribution tests on Llama\-3\-8B\-Instruct, byNN\(32 layers×\\times32 heads, 10 trials perNN\)\.The cross\-architecture comparison reveals model\-specific departures from the Gaussian i\.i\.d\. abstraction \(Tables[9](https://arxiv.org/html/2609.22101#A4.T9)and[11](https://arxiv.org/html/2609.22101#A4.T11)\)\. Gemma\-3’s excess kurtosis increases withNN\(2\.70→9\.122\.70\\to 9\.12\), whereas Llama\-3’s decreases \(2\.32→0\.892\.32\\to 0\.89\)\. AtN=100N=100, the Ljung–Box test rejects independence for both models\. Because the analyses pool different numbers of layer\-head pairs, however, the rawQQstatistics are not directly comparable as measures of violation severity\. For Gemma\-3, the reported KSpp\-values reject an exact Gumbel fit\. Table[11](https://arxiv.org/html/2609.22101#A4.T11)reports KS statistics for Llama\-3 but not the correspondingpp\-values, so we do not make a formal rejection claim for Llama\-3 without those values\.

Despite these departures, the empirical mean maximum for Llama\-3 is93%93\\%,94%94\\%, and97%97\\%of the2​log⁡N\\sqrt\{2\\log N\}prediction atN=20N=20,5050, and100100, respectively \(Figure[8](https://arxiv.org/html/2609.22101#A4.F8)\)\. Thus, the asymptotic scale remains a useful order\-of\-magnitude reference, even though the exact distributional assumptions are not satisfied\.

![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_margin_llama3.png)Figure 8:Margin growth on Llama\-3\-8B\-Instruct vs\. the theorem’s asymptotic prediction \(cf\. Figure[6](https://arxiv.org/html/2609.22101#A4.F6)for Gemma\)\.Does this internal violation translate into behavioral failure? For each trial we additionally generated the model’s real answer and checked whether it contained the true code\. Retrieval accuracy was100%100\\%at everyNN\(30/30 trials total; Figure[9](https://arxiv.org/html/2609.22101#A4.F9)\), while excess kurtosis rose monotonically over the same range\. For this task—simple single\-fact retrieval against generic filler—the internal assumption violation is not sufficient on its own to produce behavioral failure\.

![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_retrieval_vs_attention.png)Figure 9:Retrieval accuracy \(left\) vs\. excess kurtosis of the attention distribution \(right\), byNN\.If neither the assumption violation norNNalone predicts failure, does raw context length matter at all? Extending purely behavioral testing \(no attention extraction, hence tractable at much largerNN\) with the same generic\-filler distractors toN=100,200,400N=100,200,400\(up to∼\\sim5,400 tokens\), accuracy remained100%100\\%throughout\. Increasing the number of generic\-filler distractors from100100to400400did not reduce observed accuracy for this model on this task\. This result applies only to the tested context range and distractor construction\.

We therefore varied*what*the distractors said rather than how many there were, holdingN∈\{20,50,100\}N\\in\\\{20,50,100\\\}fixed\. Low\-similarity distractors \(generic, topically unrelated filler\) and medium\-similarity distractors \(the same numeric\-code format, distinguished by a different label such as “backup code”\) both left accuracy at100%100\\%at everyNNtested\. High\-similarity distractors—identical phrasing to the evidence sentence, differing only in the number—collapsed accuracy to near zero at everyNNtested instead \(0\.00, 0\.07, and 0\.00 atN=20N=20,5050, and100100respectively\)\. This approximatelyNN\-invariant pattern over the three tested values supports the paper’s distinction \(Definition[2](https://arxiv.org/html/2609.22101#Thmdefinition2)\) between raw distractor countNNand effective distractor countNeffN\_\{\\mathrm\{eff\}\}: over the tested range, confusability is more predictive of failure than raw distractor count\. Because this three\-condition low/medium/high comparison cannot determine how much confusability is required, we then fixedN=50N=50total distractors and sweptkk, the number of high\-similarity decoys mixed into an otherwise generic\-filler haystack, from00to5050\(Table[12](https://arxiv.org/html/2609.22101#A4.T12), Figure[10](https://arxiv.org/html/2609.22101#A4.F10)\)\.

Table 12:Retrieval accuracy versus the numberkkof confusable decoys, withNdist=50N\_\{\\mathrm\{dist\}\}=50total distractors fixed\.Each point is based on1515trials, so one additional correct response changes the reported accuracy by1/15≈0\.0671/15\\approx 0\.067\. Accordingly, the adjacent changes fromk=2k=2\(0\.330\.33\) tok=3k=3\(0\.400\.40\), and fromk=5k=5\(0\.070\.07\) tok=10k=10\(0\.130\.13\), each correspond to only one additional correct response\. These local reversals should therefore not be interpreted as evidence of a non\-monotonic underlying effect\. The robust feature is the overall steep initial decline followed by a low plateau\.

Accuracy drops sharply within the first two confusable decoys \(k=0k=0:1\.001\.00,k=1k=1:0\.670\.67,k=2k=2:0\.330\.33\)\. A single confusable decoy among fifty distractors reduces observed accuracy by approximately one third, and two reduce it by approximately two thirds\. Fromk=5k=5throughk=35k=35, observed accuracy remains in the7%7\\%–13%13\\%range across a sevenfold increase inkk\. Atk=50k=50, when every distractor is confusable, observed accuracy is0/150/15\. Thus, most of the observed degradation occurs after introducing a small number of confusable decoys, while additional decoys produce comparatively little further change\.

![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_dose_response.png)Figure 10:Retrieval accuracy versus the numberkkof confusable decoys, withNdist=50N\_\{\\mathrm\{dist\}\}=50total distractors fixed\.In qualitative inspection, some failures under high similarity or largerkkincluded hedging behaviors that were not observed atk=0k=0, such as listing multiple candidate codes or generating an unrequested confidence statement or rationale\. Because these behaviors were not coded systematically, we treat them as anecdotal indications of uncertainty rather than as a measured result\.

Taken together, these results provide a coherent but task\-specific picture\. The Gaussian i\.i\.d\. assumption is not exact, and the form of its violation differs across the two architectures\. Nevertheless, the theorem’s margin\-growth prediction remains close in order of magnitude to the observed maxima\. In this retrieval task, behavioral failure tracks the number of textually confusable decoys more closely than raw context length over the tested range\. Only a small number of such decoys is needed to produce most of the observed degradation\. These findings support interpretingNNin the bound as an effective distractor countNeffN\_\{\\rm eff\}, while not establishing that raw context length is irrelevant for other tasks, models, or longer\-context regimes\.

## Appendix ELost\-in\-the\-middle reanalysis

Definition[1](https://arxiv.org/html/2609.22101#Thmdefinition1)defines the poisoning curveP⁡\(N\)=A⁡\(N0\)−A⁡\(N\)P\(N\)=A\(N\_\{0\}\)\-A\(N\): the drop in accuracy as context lengthNNgrows\. This has so far been supported partly by citing prior work\([21](https://arxiv.org/html/2609.22101#bib.bib19)\); here we re\-run a version of their setup directly on our own model to report a concrete, reproducible number\.

We use the publicly released data from[21](https://arxiv.org/html/2609.22101#bib.bib19), which pair real questions from the Natural Questions dataset\([19](https://arxiv.org/html/2609.22101#bib.bib17)\)with Wikipedia passages retrieved using Contriever\-MSMARCO\([15](https://arxiv.org/html/2609.22101#bib.bib33)\)\. Each example has one gold passage containing the true answer and several distractor passages\. We sampled 40 fixed questions, reused across every condition so results are paired rather than confounded by question difficulty, and evaluated them on Gemma\-3\-4B\-IT\([11](https://arxiv.org/html/2609.22101#bib.bib11)\)\(Hugging Facetransformers, greedy decoding\)\. A model’s answer is scored correct if it contains the known gold answer string\.

Two conditions were tested\. In the position condition,Ndocs=20N\_\{\\mathrm\{docs\}\}=20documents are fixed and the gold passage is moved to zero\-indexed position00,44,99,1414, or1919\. In the scaling condition, the gold passage is held at zero\-indexed position00, which had the highest observed accuracy in the first condition, whileNdocs∈\{10,20,30\}N\_\{\\mathrm\{docs\}\}\\in\\\{10,20,30\\\}is varied\. The second condition therefore measures the effect of document count while holding the gold passage’s position fixed\.

![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_position_curve.png)Figure 11:Accuracy vs\. position of the gold document,N=20N=20, 40 real NQ questions\.Accuracy is highest when the gold passage is first \(52\.5%52\.5\\%\) and ranges from32\.5%32\.5\\%to40\.0%40\.0\\%at the other tested positions, a decrease of12\.512\.5–20\.020\.0percentage points \(Figure[11](https://arxiv.org/html/2609.22101#A5.F11)\)\. This provides descriptive evidence of a position effect in this sample and is consistent with[21](https://arxiv.org/html/2609.22101#bib.bib19)\.

Table 13:Accuracy and poisoning curveP⁡\(N\)=A⁡\(10\)−A⁡\(N\)P\(N\)=A\(10\)\-A\(N\), gold passage fixed at position 0\.![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/fig_poisoning_curve.png)Figure 12:Accuracy andP⁡\(N\)P\(N\)vs\. context length, gold passage fixed at position 0\.With the gold passage fixed at position00, which had the highest observed accuracy,P⁡\(N\)P\(N\)remains close to zero acrossN∈\{10,20,30\}N\\in\\\{10,20,30\\\}\(Table[13](https://arxiv.org/html/2609.22101#A5.T13), Figure[12](https://arxiv.org/html/2609.22101#A5.F12)\)\. We therefore observe no monotonic decrease in accuracy with document count over this range\. Together with the position experiment, these results indicate that position has a larger observed effect than raw document count for this model and within the tested range\. They do not establish that position remains the dominant factor at larger context sizes or for other models\. The result is compatible with Theorem[1](https://arxiv.org/html/2609.22101#Thmtheorem1)’s emphasis on effective distractor exposure and Definition[2](https://arxiv.org/html/2609.22101#Thmdefinition2), but this experiment does not measureNeffN\_\{\\rm eff\}directly\.

Two limitations are worth noting\. First, the study uses only4040questions, so the estimates have substantial uncertainty\. Second, the scaling condition tests only up toN=30N=30documents\. WhetherP⁡\(N\)P\(N\)remains near zero at larger values ofNN, or whether position and length effects interact at greater scale, remains unresolved\.

## Appendix FEmpirical Test of the Gate\-Mitigation Prediction

Proposition[1](https://arxiv.org/html/2609.22101#Thmproposition1)predicts that replacing a raw candidate set ofNNcontext chunks with a bounded retrieved set ofK≪NK\\\!\\ll\\\!Nchunks trades context\-length interference against a retrieval\-error penaltyη^=1−Recall@​K\\hat\{\\eta\}=1\-\\text\{Recall@\}K\. We test this prediction directly, following the reporting protocol in[Experimental setup and metrics](https://arxiv.org/html/2609.22101#Sx7): we measure answer accuracy and evidence recall separately, and we isolate the model’s use of retrieved evidence from the quality of retrieval itself\. We emphasize the scope of the experiment: it tests the qualitative tradeoff described by the proposition, but it does not validate the theorem’s Gaussian\-logit assumptions or itslog⁡N\\sqrt\{\\log N\}asymptotic rate\.

### Design

#### Benchmark\.

We use RULER’sqa\_2task\([14](https://arxiv.org/html/2609.22101#bib.bib13)\), which is built on HotpotQA\([32](https://arxiv.org/html/2609.22101#bib.bib34)\)\. Each instance is a multi\-hop question whose two gold supporting paragraphs are embedded among distractor paragraphs presented as numberedDocumentnn:blocks\. We construct instances directly from the HotpotQA distractor development set to preserve HotpotQA’s gold supporting\-fact titles, which allows us to measure evidence recall rather than relying on an answer\-string proxy\.

Starting from each question’s native two gold and eight distractor paragraphs, we add distractor paragraphs drawn from a global pool until a target token length is reached\. We evaluate seven lengths,\{4,8,16,32\}\\\{4,8,16,32\\\}K and\{128,256,512\}\\\{128,256,512\\\}K tokens, using the same100100questions at every length so that comparisons are paired across both conditions and lengths\. Lengths are measured with theo200k\_basetokenizer\. The median raw candidate countNNgrows from2929chunks at44K to3,7113\{,\}711chunks at512512K \(Table[14](https://arxiv.org/html/2609.22101#A6.T14)\)\.

#### Conditions\.

Each instance is answered under two conditions that differ only in the documents supplied\. Therawcondition passes allNNdocuments in source order\. Thegatedcondition scores each document against the question using BM25\([26](https://arxiv.org/html/2609.22101#bib.bib24)\), retains the top\-KKdocuments, and restores them to their original source order before answering\. Restoring source order ensures that the intervention changes context length without also changing document ordering\.

Both conditions use the same prompt template, system instruction, and decoding configuration: temperature00, top\-pp11, a maximum of6464generated tokens, and one generation call\. We deliberately omit a quote\-the\-evidence\-first step from the gated condition because such a step would confound retrieval gating with an additional inference procedure\.

#### Gate andKKselection\.

EachDocumentnn:block is treated as one retrieval chunk\. Queries and documents use identical lowercasing and tokenization for BM25 scoring\. Because the benchmark is multi\-hop, the gate succeeds only when it retrieves all annotated supporting documents:

Hiti\(K\)=Hiti\(K\)=\{Gi⊆Ri\(K\)\}\.\\mathrm\{Hit\}\_\{i\}\(K\)=\\mathrm\{Hit\}\_\{i\}\(K\)=\\mathbf\{1\}\\\!\\left\\\{G\_\{i\}\\subseteq R\_\{i\}\(K\)\\right\\\}\.\(34\)The estimated retrieval penalty is

η^=1−1M​∑i=1MHiti​\(K\)\.\\hat\{\\eta\}=1\-\\frac\{1\}\{M\}\\sum\_\{i=1\}^\{M\}\\mathrm\{Hit\}\_\{i\}\(K\)\.\(35\)
On a disjoint100100\-question development split, we sweepK∈\{2,4,8,16\}K\\in\\\{2,4,8,16\\\}and select the smallest value achieving all\-support Recall@K≥0\.90K\\geq 0\.90at every length through3232K\. This procedure selectsK=16K=16\. Retrieval and selection ofKKare performed offline and require no test\-set model calls\.

#### Model\.

All answers are generated by GPT\-4\.1 \(gpt\-4\.1\_2025\-04\-14\) at temperature00\. We verified that the serving endpoint accepts inputs of at least520520K tokens, and no raw prompt was truncated\. The study consists of1,4001\{,\}400generations, forming700700paired raw–gated comparisons \(100×7100\\times 7pairs\), with zero endpoint errors\.

### Metrics

We report normalized exact match \(EM\) and token F1 using the official HotpotQA scorer\. The primary effect is the paired difference

Δ​EM​\(N,K\)=Agate−Araw,\\Delta\\mathrm\{EM\}\(N,K\)=A\_\{\\mathrm\{gate\}\}\-A\_\{\\mathrm\{raw\}\},\(36\)with95%95\\%confidence intervals computed from10,00010\{,\}000paired bootstrap samples and paired binary outcomes evaluated using an exact McNemar test\.

To separate evidence use from retrieval quality, we also report conditional accuracy on the retrieval\-hit subset and its difference

Δuse=Agate\|hit−Araw\|hit\.\\Delta\_\{\\mathrm\{use\}\}=A\_\{\\mathrm\{gate\}\\mid\\mathrm\{hit\}\}\-A\_\{\\mathrm\{raw\}\\mid\\mathrm\{hit\}\}\.\(37\)On this subset, the gate is known to have retained all annotated supporting evidence\. A positiveΔuse\\Delta\_\{\\mathrm\{use\}\}is therefore consistent with a benefit from reducing competing context rather than merely from successful retrieval\.

### Results

Table 14:GPT\-4\.1 on RULERqa\_2/HotpotQA: raw long context versus a BM25 top\-KKretrieval gate withK=16K=16, frozen on a disjoint development split\. Each length uses the same100100questions at temperature00\. The retrieval penalty isη^=1−Recall@​K\\hat\{\\eta\}=1\-\\text\{Recall@\}K\.Δ​EM\\Delta\\mathrm\{EM\}is the paired gated\-minus\-raw difference with a10,00010\{,\}000\-sample paired\-bootstrap95%95\\%confidence interval\.Δuse\\Delta\_\{\\mathrm\{use\}\}is the gated\-minus\-raw difference restricted to instances for which all annotated supporting evidence was retrieved\. McNemarppis the exact two\-sided test\.![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/main_results.png)Figure 13:GPT\-4\.1 withK=16K=16\.\(a\)Raw accuracy is approximately stable through3232K and then decreases with context length, while gated accuracy varies less\.\(b\)A fixedK=16K=16meets the0\.900\.90all\-support recall target through3232K, after which recall decreases as the candidate set grows\.\(c\)On instances for which the gate retrieves all annotated supporting evidence, raw accuracy decreases with context length while gated accuracy remains comparatively stable, providing evidence of improved evidence use after competing context is removed\.Table[14](https://arxiv.org/html/2609.22101#A6.T14)and Figure[13](https://arxiv.org/html/2609.22101#A6.F13)support four findings\.

#### \(1\) No measurable poisoning through3232K\.

Up to3232K, raw EM remains approximately stable \(→0\.650\.66\\\!\\to\\\!0\.65\), and the gate provides no statistically detectable net benefit\. The observedΔ​EM\\Delta\\mathrm\{EM\}values remain within\[−0\.05,\+0\.05\]\[\-0\.05,\+0\.05\], all reported confidence intervals include zero, and McNemarp≥0\.18p\\geq 0\.18\.

#### \(2\) Degradation appears beyond3232K\.

From3232K to512512K, raw EM decreases from0\.650\.65to0\.570\.57and raw F1 decreases from0\.780\.78to0\.700\.70\. No comparable downward trend is observed among the tested lengths from44K through3232K\. The gated condition exhibits a smaller overall decrease across the longest tested contexts\.

#### \(3\) The gate improves evidence use on retrieval\-hit instances\.

Figure[13](https://arxiv.org/html/2609.22101#A6.F13)\(c\) restricts evaluation to questions for which the gate retrieves all annotated supporting evidence\. On this subset, raw accuracy decreases from0\.690\.69at1616K to0\.5950\.595at512512K, whereas gated accuracy remains near0\.710\.71\. The correspondingΔuse\\Delta\_\{\\mathrm\{use\}\}values are\+0\.070\+0\.070,\+0\.061\+0\.061, and\+0\.114\+0\.114at128128K,256256K, and512512K, respectively\. Because both conditions contain all annotated gold evidence on this subset, the pattern is consistent with a benefit from removing competing distractor context, rather than merely from successful retrieval\.

#### \(4\) A fixedKKincurs a growing retrieval penalty\.

WithKKfixed at1616, all\-support recall decreases from0\.940\.94to0\.790\.79as the raw candidate count grows to approximately3,7003\{,\}700chunks\. Correspondingly,η^\\hat\{\\eta\}increases from0\.060\.06to0\.210\.21\. The evidence\-use benefit is therefore offset by the14%14\\%–21%21\\%of instances for which the bounded gate omits required evidence, and the netΔ​EM\\Delta\\mathrm\{EM\}remains near zero at the longest contexts\. This pattern matches the qualitative tradeoff in Proposition[1](https://arxiv.org/html/2609.22101#Thmproposition1): reducing the candidate set fromNNtoKKcan improve evidence use, but that benefit is offset when the retrieval penaltyη^\\hat\{\\eta\}grows with the candidate set\.

### ScalingKKto Preserve Evidence Recall

To separate the evidence\-use benefit from the retrieval penalty, we re\-run only the gated condition at128128K,256256K, and512512K with a length\-specific value ofKKselected on the development split to restore all\-support Recall@K≥0\.90K\\geq 0\.90\. The resulting values areK=64K=64,128128, and192192, respectively\. The raw predictions are unchanged and reused from the primary experiment\. Even atK=192K=192, the gated prompt remains approximately2525K tokens, compared with approximately504504K tokens in the raw condition\.

Table 15:ScalingKKto restore recall at extreme lengths\. Only the gated arm is re\-run; raw predictions are cached and unchanged\. Each length\-specific value ofKKis selected on the development split to achieve Recall@K≥0\.90K\\geq 0\.90\.![Refer to caption](https://arxiv.org/html/2609.22101v1/Figures/largerK_comparison.png)Figure 14:Comparison between fixedK=16K=16and a retrieval budget scaled to maintain all\-support Recall@KKof at least0\.900\.90at extreme context lengths\.Left:gated accuracy remains comparatively stable under the scaled retrieval budget while raw accuracy decreases\.Middle:the paired gated\-minus\-raw difference becomes more positive after the retrieval penalty is reduced\.Right:the scaled values ofKKmaintain recall above the target where fixedK=16K=16does not\.As shown in Table[15](https://arxiv.org/html/2609.22101#A6.T15)and Figure[14](https://arxiv.org/html/2609.22101#A6.F14), restoring recall makes the net gate advantage more positive at the longest tested contexts\. At512512K, increasingKKfrom1616to192192raises gated EM from0\.590\.59to0\.640\.64, while raw EM is0\.570\.57\. The correspondingΔ​EM\\Delta\\mathrm\{EM\}increases from\+0\.02\+0\.02to\+0\.070\+0\.070, with a95%95\\%confidence interval of\[\+0\.000,\+0\.140\]\[\+0\.000,\+0\.140\]and McNemarp=0\.092p=0\.092\. At256256K,Δ​EM\\Delta\\mathrm\{EM\}increases from−0\.01\-0\.01to\+0\.050\+0\.050\.

At128128K, fixedK=16K=16already retains most annotated support \(recall0\.860\.86\), and increasingKKslightly reducesΔ​EM\\Delta\\mathrm\{EM\}from\+0\.05\+0\.05to\+0\.03\+0\.03, consistent with reintroducing additional distractors\. In the logistic model

correct∼condition×log2⁡N,\\mathrm\{correct\}\\sim\\mathrm\{condition\}\\times\\log\_\{2\}N,\(38\)the estimated gate\-by\-length interaction increases from\+0\.035\+0\.035with fixedK=16K=16to\+0\.084\+0\.084under the scaled retrieval budget\. These point estimates suggest that the retrieval budget may need to grow with the candidate set to maintain evidence recall, rather than being held fixed independently ofNN\.

### Discussion and Limitations

The experiment supports the gate\-mitigation prediction within the tested regime\. For GPT\-4\.1, no measurable degradation is observed through3232K\. At longer contexts, raw accuracy decreases, while the retrieval\-hit analysis indicates that the gate can improve the model’s use of evidence that remains available\. The net accuracy benefit becomes more positive whenKKis increased to keep the retrieval penaltyη^\\hat\{\\eta\}small\. Together, these results are consistent with theN→KN\\\!\\to\\\!Kversusη^\\hat\{\\eta\}tradeoff described by Proposition[1](https://arxiv.org/html/2609.22101#Thmproposition1)\.

Several limitations bound these claims\. First, withM=100M=100questions, the strongest net effect at512512K remains borderline: the confidence interval has a lower bound of zero and McNemarp≈0\.09p\\approx 0\.09\. A larger evaluation set would improve precision; under standard square\-root scaling, doubling the sample size would reduce standard errors by approximately a factor of1/21/\\sqrt\{2\}, not by one half\.

Second, we study a single hosted model\. Other models may enter the degradation regime at different context lengths, so replication across model families and deployment settings is needed to assess the generality of finding \(2\)\.

Third, contexts are constructed and measured using a tokenizer proxy rather than the proprietary serving tokenizer\. The reported lengths are therefore internally consistent but may not exactly equal server\-side token counts\. In addition, temperature\-00hosted decoding is not necessarily perfectly deterministic\.

Finally, as stated at the outset, these results test the proposition’s qualitative tradeoff rather than the theorem’s distributional assumptions or asymptotic rate\.