Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits

# Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits
Source: [https://arxiv.org/html/2609.17983](https://arxiv.org/html/2609.17983)
###### Abstract

KV\-cache reuse can reduce inference cost in retrieval\-augmented generation and agentic systems, but cached contexts may become stale when retrieved knowledge, working memory, or user state is edited\. Under causal self\-attention, even a local edit can affect downstream KV states\. A full re\-prefill reliably restores consistency but is costly, whereas refreshing only the edited span can leave downstream dependencies stale\. We formulate in\-place repair as budgeted recomputation and compare training\-free position\-selection policies on a factual RAG benchmark with matched direct and derived edits\. Across three model families, all policies repair direct cases, but derived cases clearly separate them\. At the primary budget, a contiguous edit\-local window recovers at least 0\.94 of the post\-edit answer margin and substantially outperforms attention\-based, KV\-deviation, and structural selectors\. Mechanistic analysis shows that position sets effective under clean\-state transplantation can fail under actual recomputation because scattered positions inherit surrounding staleness\. The edit\-local advantage also depends on adjacency and largely disappears when the answer\-bearing text moves downstream\. Because answer\-relevant edits almost always corrupt model behavior, failure severity is difficult to predict, and repair is 13–21× faster than full re\-prefill, our results support unconditional edit\-local repair when the dependent text remains adjacent to the edit\.

1University of South Florida, Electrical and Computer Engineering

2DEVCOM Army Research Laboratory

\{mmao, xlin2\}@usf\.edu, wyatt\.t\.mackey\.civ@army\.mil

## Introduction

Figure 1:Problem and probe design\.\(a\) An in\-place edit to a cached document leaves the edited span’s KV entries encoding old text and all downstream entries stale\. \(b\) Each benchmark item pairs the same question with two context structures: a*direct*probe, where the edit rewrites the answer text, and a*derived*probe, where the answer sits in an unedited lookup entry downstream of the edited alias\. Without repair the cache returns the pre\-edit answer\.RAG pipelines and LLM\-based agents increasingly reuse KV states for retrieved documents or memory blocks across requests, avoiding repeated prefill\([Yao et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib1);[Bergman et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib2);[Ye et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib4);[Pan et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib3)\)\. This optimization assumes cached source content does not change\. In deployment, that assumption breaks when a factual record is corrected\([Ouyang et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib5);[Cohen et al\. 2024](https://arxiv.org/html/2609.17983#bib.bib14)\), a policy is revised, or an agent’s working memory is updated\([Packer et al\. 2023](https://arxiv.org/html/2609.17983#bib.bib17)\)\. The source text then carries the update, while retained KV tensors encode the older version\. A request can therefore be answered from a representation that no longer matches the knowledge base\. Existing document\-level reuse methods optimize reuse and composition, but do not synchronize cached representations after a source edit\.

Repairing this mismatch is not confined to the edited span\. Continuing to serve from a stale cache can reduce response accuracy\([Ouyang et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib5)\), and causal self\-attention leaves every downstream KV state dependent on the old content\. A full re\-prefill restores consistency but re\-encodes an entire long context after an edit of only a few tokens\. Recent work has begun to repair the cache in place after such small edits, but each proposed method carries a limitation\. KVEraser replaces a target span with learned steering states, yet requires per\-model training and targets deletion rather than factual replacement\([Li et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib6)\)\. MTN recomputes downstreamnotesranked by their causal effect, using an oracle signal in controlled agent tasks\([Li 2026](https://arxiv.org/html/2609.17983#bib.bib7)\)\. Neither answers the matched\-budget question for factual RAG under actual recomputation\. To build a low\-budget, training\-free repair policy, we therefore first ask which downstream positions matter most once the edited span itself has been refreshed\.

Our study pairs factual edits with contexts of roughly50005000retrieved tokens\. Each question either asks for the edited fact directly or requires a two\-hop derivation through unchanged downstream text\. Starting from the stale cache, every policy refreshes the known edit span and selectsKKdownstream positions under the same recomputation budget\. We compare an edit\-local window, structural delimiters\([Li 2026](https://arxiv.org/html/2609.17983#bib.bib7)\), stale\-query attention\([Wang et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib8)\), CacheBlend\-based KV deviation\([Yao et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib1)\), and random selection\. A transplant\-derived causal ranking is included only as a non\-deployable diagnostic\. Evaluation covers development and held\-out cohorts across Llama, Qwen, and Mistral, plus a10991099\-item variant that moves the answer\-carrying text downstream\. We use pre\-specified criteria and report held\-out margin recovery and flip rate\.

Three results define the baseline\. AtK=32K\{=\}32, every policy saturates on direct questions, while derived answers separate them\. The edit\-local window recovers0\.940\.94–1\.011\.01of the answer margin and flips0\.950\.95–1\.001\.00of held\-out items, beating every alternative by0\.460\.46–1\.001\.00margin recovery \(Holm\-correctedp≤3×10−9p\\leq 3\\times 10^\{\-9\}\)\. This advantage depends on adjacency\. Moving the answer\-carrying text250250tokens downstream reduces edit\-local recovery to0\.010\.01–0\.090\.09, with stale\-query attention helping on only one model family\. Transplant recoverability also fails to predict repair under recomputation\. The causal ranking recovers0\.920\.92–0\.970\.97of the margin when clean states are transplanted, but the same positions recover only0\.010\.01–0\.210\.21when recomputed atK=8K\{=\}8\. Recomputation is worse on every held\-out item\. A transplanted state imports information from a correct cache, whereas a recomputed state reads stale surrounding states\. A contiguous edit\-local window succeeds by rebuilding that dependency chain in order\. Finally, answer\-relevant edits almost always break the stale cache \(base rate≥0\.988\\geq 0\.988\), and cheap features poorly predict failure severity \(best out\-of\-foldρ=0\.17\\rho=0\.17\)\. Because repair runs1313–21×21\\timesfaster than re\-prefill, applying edit\-local repair unconditionally is the practical policy within this setting\. It is also the training\-free baseline that future selection methods must beat\.

In summary, this paper makes the following key contributions:

- •We formulate stale\-cache repair as budgeted recomputation and introduce paired direct, derived, and distance\-controlled factual edits\.
- •We compare five training\-free selectors at a matched budget across three model families, establishingEditLocalas a strong adjacent\-block baseline\.
- •We separate localization under clean\-state transplant from repair under recomputation\.
- •We show that answer\-relevant edits warrant unconditional repair within this benchmark because failure is frequent, severity is hard to predict, and repair is inexpensive\.

## Related Works

#### Document\-level KV reuse\.

RAGCache stores intermediate states of retrieved knowledge, while TurboRAG precomputes per\-chunk KV caches for reuse across queries\([Jin et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib9);[Lu et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib10)\)\. Both reduce repeated prefill by reusing stored states of retrieved text\. Neither addresses how those states should be updated after the source text changes\. HoH shows that outdated retrieved evidence can reduce RAG accuracy even when current evidence is available\([Ouyang et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib5)\), but it studies stale information at the text level rather than repair of an already cached representation\. We study the systems problem that follows a source edit: how much of one stale document cache must be recomputed?

#### Selective recomputation\.

Selective recomputation addresses a different cache mismatch\. CacheBlend restores missing cross\-attention by recomputing positions with large KV deviations, while ProphetKV prioritizes positions using query relevance\([Yao et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib1);[Wang et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib8)\)\. EPIC recomputes a small, fixed set of initial chunk tokens, Cache\-Craft repairs reusable chunk caches through limited recomputation, and KVShare selects high\-deviation states during prefill and decoding\([Hu et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib11);[Agarwal et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib12);[Yang et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib13)\)\. Across these methods, the source chunks are unchanged and the mismatch comes from reusing their caches in a new context\. Our experiments transfer the deviation\- and query\-based selection signals to a cache whose source text has changed and compare them at the same downstream\-position budget\.

#### KV\-cache editing\.

KV\-cache editing is closest to our setting\. KVEraser replaces a target span with learned steering states to erase its influence\([Li et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib6)\)\.[Li](https://arxiv.org/html/2609.17983#bib.bib7)\([2026](https://arxiv.org/html/2609.17983#bib.bib7)\)shows that changing a field can leave old conclusions in downstream states and ranks those states by causal effect\. Leyline provides serving primitives for removing or replacing cached spans, including positional correction for length\-changing edits\([Ma et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib15)\)\. We isolate length\-preserving factual replacement and ask which training\-free selector works when its chosen positions are actually recomputed at a matched budget\. This separates our setting from learned erasure, cache splicing, and position sets ranked through oracle\-state transplantation\.

## Cache Repair as Budgeted Recomputation

#### Question Setting\.

A serving system prefills a contextCCand stores its key–value states for reuse\. After an edit producesC′C^\{\\prime\}, the stored cache𝒦⁡\(C\)\\mathcal\{K\}\(C\)is stale; a full prefill𝒦⁡\(C′\)\\mathcal\{K\}\(C^\{\\prime\}\)is the*oracle*repair target, not a competing method\. In\-place repair instead refreshes selected entries of𝒦⁡\(C\)\\mathcal\{K\}\(C\)toward this target\.

We isolate one contiguous, length\-preserving edit:\|C\|=\|C′\|=n\|C\|=\|C^\{\\prime\}\|=nandci=ci′c\_\{i\}=c^\{\\prime\}\_\{i\}outside the half\-open spanS=\[sstart,send\)S=\[s\_\{\\mathrm\{start\}\},s\_\{\\mathrm\{end\}\}\)\. Positions and rotary phases therefore remain fixed; length\-changing edits, which also require positional correction, are outside our scope\([Ma et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib15)\)\. Each benchmark item pairs a*direct*condition, whose answer lies inSS, with a*derived*condition, whose answer lies in unchanged downstream text but depends on the edited fact \(Figure[1](https://arxiv.org/html/2609.17983#Sx1.F1)b\)\. The pair otherwise shares its retrieved documents, insertion point, subject, value pair, and query, and we report the conditions separately\.

#### The repair interface\.

Every policy uses the same operator and differs only in itsKKselected positions\. LetD=\[send,n−1\)D=\[s\_\{\\mathrm\{end\}\},n\-1\)be the downstream candidate pool and letAπ⊆DA\_\{\\pi\}\\subseteq D,\|Aπ\|=K\|A\_\{\\pi\}\|=K, be the positions chosen by policyπ\\pi\. The repaired set isPπ=S∪AπP\_\{\\pi\}=S\\cup A\_\{\\pi\}, and

𝒦~=ℛ⁡\(𝒦⁡\(C\),C′,Pπ\)\.\\widetilde\{\\mathcal\{K\}\}=\\mathcal\{R\}\\big\(\\mathcal\{K\}\(C\),C^\{\\prime\},P\_\{\\pi\}\\big\)\.The operator recomputesPπP\_\{\\pi\}fromC′C^\{\\prime\}layer by layer and leaves all other states unchanged, using the same selective\-recomputation primitive as prior systems\([Yao et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib1)\)\. Its full\-position endpoint is

ℛ⁡\(𝒦⁡\(C\),C′,\{0,…,n−1\}\)=𝒦⁡\(C′\)\.\\mathcal\{R\}\\big\(\\mathcal\{K\}\(C\),\\,C^\{\\prime\},\\,\\\{0,\\dots,n\-1\\\}\\big\)\\;=\\;\\mathcal\{K\}\(C^\{\\prime\}\)\.\(1\)
The known edit span is always recomputed and shared across policies, while the fresh query is never cached and is recomputed during scoring\. Thus,KKcounts only additional downstream positions\. We exclude upstream positions because causal attention prevents them from depending on the edit, and verify this invariant numerically\. The question is which deployable rule choosesAπA\_\{\\pi\}most effectively\.

#### Cost accounting\.

We separate three costs: common work for the edit span and fresh query, the matched repair budget ofK⋅LK\\cdot Ltoken\-layers, and selector overhead\. The common term cancels in policy comparisons, while overhead is reported outsideKK\. Text\- and position\-only rules have no selector forward;Attentionadds a stale\-cache scoring pass\([Wang et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib8)\), andCacheBlendadds roughly one prefill layer over the prefix\([Yao et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib1)\)\. We use wall\-clock latency as the headline systems measure and token\-layers as a device\-independent secondary measure; Section[Results](https://arxiv.org/html/2609.17983#Sx5)reports both\. The full construction, interface invariants, and cost decomposition appear in the supplementary appendix\.

#### What counts as repaired\.

Answer accuracy alone cannot distinguish caches that emit the same string with different underlying preferences\. Each item therefore defines a pre\-edit answeraolda\_\{\\mathrm\{old\}\}and a post\-edit answeranewa\_\{\\mathrm\{new\}\}\. For cache statexx, letmxm\_\{x\}be the difference between their length\-averaged, teacher\-forced log probabilities\. Our primary metric is

MR=mrepaired−mstalemoracle−mstale,\\mathrm\{MR\}\\;=\\;\\frac\{m\_\{\\mathrm\{repaired\}\}\-m\_\{\\mathrm\{stale\}\}\}\{m\_\{\\mathrm\{oracle\}\}\-m\_\{\\mathrm\{stale\}\}\},\(2\)where00denotes no change from the stale cache and11reaches the oracle\. MR is unclipped, and zero\-gap items are excluded\.

We pair this internal measure with visible behavior\. Letyby\_\{b\}be the greedy continuation of at most1616tokens for retained itembb\. The flip rate is

Flip=1\|ℬ\|∑b∈ℬ𝟏\[anew∈yb∧aold∉yb\]\.\\mathrm\{Flip\}\\;=\\;\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{b\\in\\mathcal\{B\}\}\\mathbf\{1\}\\big\[\\,a\_\{\\mathrm\{new\}\}\\in y\_\{b\}\\;\\wedge\\;a\_\{\\mathrm\{old\}\}\\notin y\_\{b\}\\,\\big\]\.\(3\)MR captures partial recovery before the decision boundary, whereas flip records whether generation exposes only the new answer\. We report both, with KL recovery as a secondary distributional measure\. The appendix gives the complete scoring and edge\-case rules\.

#### Selection policies\.

Table[1](https://arxiv.org/html/2609.17983#Sx3.T1)compares four deployable signals—edit proximity, structure, stale\-query attention\([Wang et al\. 2026](https://arxiv.org/html/2609.17983#bib.bib8)\), and KV deviation\([Yao et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib1)\)—with a matched random control\. These policies may use the stale cache, edited context, edit location, and query, but never the oracle cache or answers\. Grey rows use unavailable oracle or carrier information and are diagnostics, not deployable methods or upper bounds on recomputation\.

Table 1:Selection policies, frozen before held\-out evaluation\. OnlyKKvaries; grey rows are diagnostics\.The primary budget isK=32K\{=\}32; curves overK∈\{8,16,32,64\}K\\in\\\{8,16,32,64\\\}are descriptive\. The structuralEditLocal@∂@\\partialvariant in Section[How Far the Window Should Extend](https://arxiv.org/html/2609.17983#Sx5.SSx2)is exploratory because it was inferred from those curves\. Full policy definitions and provenance appear in the appendix\.

#### Transplant is not recomputation\.

A selected position set can be used in two ways\. Transplant copies its KV states from the oracle cache into the stale cache, while recomputation rebuilds them within the stale cache\. Only recomputation is deployable because transplant requires the oracle cache\. We use transplant only as a localization diagnostic\.

The two operators provide different information\. A transplanted state was computed with an otherwise correct cache and can therefore import information from outside the selected set\. A recomputed state must instead read the still\-stale cache around it\. Positions that appear sufficient under transplant may therefore fail under recomputation\. We apply both operators to identical position sets to measure this gap\.

Figure 2:Contiguity, not localization, decides repair\.\(a\) Selections for a representative held\-out Llama item atK=32K\{=\}32\. Orange marks the edit span, repaired outside the budget, and grey marks states left stale\. OnlyEditLocalforms an unbroken path from the edit through the answer\. \(b\) Margin recovery under transplant and recomputation on identical sets from1515development items\. Lines connect the two operators\. Solid lines denote Llama and dashed lines Qwen\. GreyCausalOracleresults are mechanism diagnostics without tests\.

## Experimental Setup

#### Paired edit benchmark\.

Each*item*contains roughly50005000tokens of shuffled HotpotQAfullwikivalidation paragraphs\([Yang et al\. 2018](https://arxiv.org/html/2609.17983#bib.bib16)\), used only as realistic retrieval background; HotpotQA questions and answers are not used\. We insert one synthetic record at a document boundary about30%30\\%into the context\.*Background*denotes the filler paragraphs, whereas*item*denotes the full construction and is the unit of every paired comparison\. The direct block states the queried field and rewrites its value\. The derived block instead rewrites an alias that selects between two downstream lookup rows; both rows are present before the edit and remain unchanged\. Thus, only the location of the answer relative to the edit differs between the conditions\. All synthetic subjects, aliases, and values are absent from the background\.

Build\-time filters require the old and edited prefixes to have equal length under every tokenizer, differ in one span of at most six tokens, and place that span between25%25\\%and40%40\\%of the context\. Direct and derived pairs must match within3%3\\%in relative edit position\. At evaluation, we retain only items for which the oracle answers correctly and a stale\-to\-oracle margin gap exists\. These model\-dependent filters define retention\. The appendix gives the full construction and filter provenance\.

#### Distance\-controlled variant\.

The main derived block makes the edit and answer\-carrying text adjacent\. To vary this geometry, we split its alias and lookup records into two documents, leave the alias in place, and move the lookup record downstream\. We defined=scarrier−send∈\{0,250,1500\}d=s\_\{\\mathrm\{carrier\}\}\-s\_\{\\mathrm\{end\}\}\\in\\\{0,250,1500\\\}\. The direct condition appears only atd=0d\{=\}0\. Atd=0d\{=\}0, the derived construction is byte\-identical to the main benchmark\. The distance axis extends rather than replaces the main setting; all policies, budgets, and metrics remain fixed\.

#### Models and decoding\.

We use Llama\-3\.1\-8B\-Instruct, Qwen3\-8B, and Mistral\-7B\-Instruct\-v0\.3 in bfloat16 with SDPA attention, decoding greedily with no chain of thought\. Qwen3 receives an empty think block so that it answers directly\. Because the three tokenizers map the same text to different token counts, a fixedKKdoes not cover identical text spans across families\. Before evaluation, a frozen harness gate compares the repaired cached path with a full forward at the answer position under a bfloat16 tolerance fixed after the initial smoke test\.

#### Pre\-specified held\-out evaluation\.

Development uses100100items\. The policy roster, metrics, budgets, and tests are pre\-specified and applied consistently across all three model families\. Held\-out items use different seeds and HotpotQA indices disjoint from development\. A candidate\-pool expansion rule is fixed in advance, preventing retention shortfalls from being repaired by loosening filters\. Retention is7575direct and3434derived for Llama,7575in each condition for Qwen, and6060direct and5858derived for Mistral\. Llama’s lower derived count reflects failure to solve some two\-hop items even from a clean prefill\.

All comparisons are paired by item\. We report95%95\\%percentile intervals from10,00010\{,\}000bootstrap resamples\. A policy mean is resampled directly; a contrast is formed per item before resampling so that pairing is preserved\. Paired Wilcoxon tests accompany the contrasts\. AtK=32K\{=\}32, all1515comparisons among six policies form one Holm\-corrected family within each model and condition;CarrierWindowappears only in the distance variant\. Curves overKKare descriptive and are not tested\. Win–tie–loss uses a primary MR tie band of0\.050\.05, fixed before held\-out evaluation from the spread of semantically equivalent arms, with0\.010\.01and0\.100\.10as appendix sensitivities\. Direct and derived conditions are never pooled\. The appendix records the cohort and statistical details\.

## Results

Across all three model families, every policy saturates in the direct condition, which serves as a sanity check\. The derived condition separates the methods:EditLocalnearly restores the full answer margin when the dependent text is adjacent to the edit, but its advantage largely disappears once that text moves downstream\. Transplant\-ranked positions also lose most of their value under actual recomputation\. Together with near\-universal stale\-cache failure and the low cost of repair, these results support unconditional edit\-local repair within the adjacent\-block setting\.

Table 2:Held\-out results for the derived condition atK=32K\{=\}32\. MR is Eq\.[2](https://arxiv.org/html/2609.17983#Sx3.E2), reported as the mean±\\pmthe half\-width of a95%95\\%percentile interval from10,00010\{,\}000bootstrap resamples, which is symmetric to within0\.0030\.003; Flip is Eq\.[3](https://arxiv.org/html/2609.17983#Sx3.E3)\. Every paired contrast againstEditLocalis Holm\-significant within its model’s1515\-pair family \(p≤3×10−9p\\leq 3\\times 10^\{\-9\}\)\. The grey row is a transplant\-ranked diagnostic, not a method or an upper bound under recomputation\. Direct rows are omitted because all policies achieve0\.980\.98–1\.001\.00\.### The Edit\-Local Window Wins

At the frozenK=32K\{=\}32,EditLocalrecovers0\.9930\.993,1\.0071\.007, and0\.9370\.937of the margin on Llama, Qwen, and Mistral and flips9595–100%100\\%of held\-out items \(Table[2](https://arxiv.org/html/2609.17983#Sx5.T2)\)\. The best rival,Attentionon Llama, reaches0\.5330\.533\. No deployable rival clears0\.080\.08on Qwen or Mistral\. The greyCausalOraclereaches only0\.100\.10–0\.410\.41\. On Llama, rivals can move probability toward the new answer, but none flips more than one of3434items\. MR thus captures partial internal repair, while Flip records whether generation exposes the new value\.

Figure[2](https://arxiv.org/html/2609.17983#Sx3.F2)a makes the geometric difference visible\.CausalOracleincludes the answer token but recovers only0\.350\.35, andCacheBlendstops inside the row holding the old value\. OnlyEditLocalrebuilds an unbroken path from the edit through the dependent text\. It beats the oracle on all3434,7575, and5858held\-out items, and its weakest record against any rival is3232wins,22ties, and no losses\. KL recovery follows MR at0\.9990\.999,1\.0001\.000, and0\.9430\.943\.

Qwen’s mean MR above one reflects slight overshoot beyond the oracle margin, which Eq\.[2](https://arxiv.org/html/2609.17983#Sx3.E2)leaves visible\. Mistral reaches1\.0021\.002atK=64K\{=\}64, consistent with its tokenizer requiring more than3232tokens to cover the same text\.

Figure 3:Margin recovery against the budgetKKon the held\-out set, derived condition\. Budgets double along a logarithmic axis, and bands are95%95\\%bootstrap intervals\. The direct condition is omitted because every policy lies between0\.980\.98and1\.001\.00fromK=8K\{=\}8on\. The curves are descriptive, and the specification reserves testing forK=32K\{=\}32\.EditLocalis threshold\-shaped and saturates once the window covers the remainder of the injected block\. The grey dashed line is the transplant\-ranked oracle, a diagnostic rather than a method\.
### How Far the Window Should Extend

Direct results stay near one at every budget and are omitted from Figure[3](https://arxiv.org/html/2609.17983#Sx5.F3)\. In the derived condition,EditLocalrises from0\.110\.11to1\.001\.00on Llama, from0\.010\.01to1\.011\.01on Qwen, and from0\.010\.01to1\.001\.00on Mistral asKKincreases from88to6464\. The transition occurs atK=32K\{=\}32\. AtK=64K\{=\}64,Attentionreaches0\.940\.94on Llama but only0\.180\.18and0\.300\.30on Qwen and Mistral, and every other rule remains at or below0\.450\.45\. More budget alone does not produce the same gain\.

The threshold matches the block geometry, an exploratory observation made after unblinding the curves\. From the edit to the block boundary, the dependent text spans a median of2424Llama tokens \(maximum2626\) and2828Qwen tokens \(maximum3030\)\. A window of1616tokens is incomplete, while3232covers the path\. This motivatesEditLocal@∂@\\partial, which recomputes to the next document boundary with no selector forward\. It coincides withK=32K\{=\}32on Llama and Qwen, while Mistral needs a slightly larger budget\. This structural rule is an interpretation of the curves, not a separately tested arm, and still assumes that dependent text begins where the edit ends\.

Table 3:Mean MR atK=32K\{=\}32as the lookup table movesddtokens downstream\. Sample sizes are191/118191/118\(Llama\),372/359/354372/359/354\(Qwen\), and343/216/221343/216/221\(Mistral\)\. Bootstrap intervals are omitted\. Every half\-width is at most0\.060\.06\. The post\-hoc Llamad=1500d\{=\}1500cell is excluded\. Grey rows are diagnostics, andCarrierWindowreceives the true carrier location\.
### Moving the Answer Away from the Edit

The byte\-identicald=0d\{=\}0tier reproduces the main results, ruling out the two\-document builder as the source of the distance effect\. For example, QwenEditLocalscores1\.0021\.002rather than1\.0071\.007, andCacheBlendscores0\.0750\.075rather than0\.0760\.076\. Tests are Holm\-corrected within each four\-pair cell family\.

Adjacency is load\-bearing\. Atd≥250d\\geq 250,EditLocalfalls to0\.010\.01–0\.090\.09MR and0\.000\.00–0\.040\.04Flip\. It mostly tiesRandomon Qwen and Mistral, while Llama’s remaining advantage is only\+0\.07\+0\.07\.EditLocal@∂@\\partialalso stops before the carrier\.Attentionretains useful signal only on Llama, reaching0\.390\.39atd=250d\{=\}250and0\.840\.84atK=64K\{=\}64\. It stays at or below0\.110\.11on Qwen and Mistral\. The main win is therefore an adjacency effect, not a general advantage of the selector\.

Search is only part of the problem\. Even with the true location,CarrierWindowrecovers just0\.430\.43–0\.740\.74on Llama and Mistral, compared with0\.890\.89–0\.950\.95on Qwen\. The probe still moves the dependency as intended:6161–71%71\\%of the oracle’s positive effect mass lies in the carrier block, and at most3%3\\%remains in the edited block\. Recovery from a correctly placed window is thus model\-dependent\.

Llama retains only118118solvable items atd=250d\{=\}250, about one third of the pool, so its distance cells cover only solvable examples\. Distance also changes absolute position and recency, which partly confounds the attention result\.

### Transplant Recoverability Does Not Imply Recompute Repairability

The grey oracle ranks positions by causal effect under transplant, but the same sets behave differently under recomputation \(Figure[2](https://arxiv.org/html/2609.17983#Sx3.F2)b\)\. On1515development items, transplant recovers0\.930\.93–0\.990\.99of the margin, while recomputation reaches only0\.030\.03–0\.410\.41\. The gap is specific to scattered sets:EditLocalandRandomdiffer by less than0\.020\.02between operators, and all direct cells differ by less than0\.0030\.003\.

Held\-out results repeat the gap on all three families\. AtK=8K\{=\}8, transplant recovers0\.9180\.918,0\.9520\.952, and0\.9690\.969on Llama, Qwen, and Mistral, compared with0\.2090\.209,0\.0110\.011, and0\.0080\.008under recomputation\. Recomputation is worse on every held\-out item, and all167167recorded position sets match when replayed\. Even atK=32K\{=\}32, recomputation reaches only0\.100\.10–0\.410\.41\.

A transplanted state comes from a fully correct cache and can import information from outside the chosen set\. A recomputed state reads the stale cache around it, so scattered positions inherit staleness\. A contiguous window instead rebuilds the forward chain in order\. Transplant recoverability therefore shows where clean information can act, not what sparse recomputation can reconstruct\. Development diagnostics support this account:Structuralomits content tokens carrying most positive effect mass, while stale\-query attention ranks them too deep to fit the budget\.

### Repair Should Be Unconditional

We tested cheap repair triggers on Llama and Qwen without retention filters \(n=175n\{=\}175per condition\)\. After an answer\-relevant edit, the stale cache fails on at least98\.8%98\.8\\%of items per condition, reaches100%100\\%in both direct conditions, and never falls below97\.3%97\.3\\%in any split\. This result is limited to edits that touch the answer chain, but within that scope there is little for a gate to separate\.

Severity is no easier to predict\. Ridge and random\-forest models using text, position, edit, and embedding features reach a best out\-of\-fold Spearmanρ=0\.17\\rho=0\.17\(R2=0\.05R^\{2\}=0\.05\) under grouped five\-fold cross\-validation\. Edit type distinguishes direct from derived cases, but the remaining features do not support a useful severity gate\. Near\-certain failure, weak predictability, and low repair cost favor unconditional repair\.

Table 4:Wall\-clock cost of repair \(span plusK=32K\{=\}32downstream positions, all layers\) against a full re\-prefill of the edited prefix\. Median of2020timed runs after33warmup runs, on one RTX 5090, bfloat16, batch11\. Both sides exclude the fresh\-query forward, which is identical for every method\. Repair includes a defensive cache copy \(<2<2ms\), and in\-place repair is marginally faster\.
### What Repair Costs

At roughly5,0005\{,\}000tokens, repair takes3131–3535ms versus458458–502502ms for re\-prefill, about15×15\\timesfaster on all three models \(Table[4](https://arxiv.org/html/2609.17983#Sx5.T4)\)\. The ratio reaches about20×20\\timesat88K\. RaisingKKfrom88to6464adds only11–44ms, making a structural boundary such asEditLocal@∂@\\partialinexpensive\. The token\-layer proxy gives117×117\\times–234×234\\timesbut overstates the measured gain\. Selector overhead also favors local rules, which require no scoring forward, unlikeAttentionandCacheBlend\.

#### Sanity checks\.

No frozen guard fired on the held\-out batch\. Stale and oracle outputs reproduced byte\-equal generations with margins within10−310^\{\-3\}, and all oracle rankings replayed without mismatches\. The endpoint identity in Eq\.[1](https://arxiv.org/html/2609.17983#Sx3.E1)passes on Llama\. Qwen passes in margin \(0\.9970\.997–1\.0131\.013\) but exceeds the frozen value\-tensor L2 tolerance\. A null\-content control places the discrepancy within bfloat16 numerical noise\. We retain the failed label\. On Mistral, three of8080bfloat16 smoke states changed argmax, and all three matched exactly in float32\.

## Discussion

#### What to deploy\.

For a direct question, refreshing the edited span completes the repair\. For an answer derived through adjacent text, recompute through the end of the block\. This rule reads only text and position, and its forward is1313–21×21\\timescheaper than re\-prefill\. A structural stopping point such asEditLocal@∂@\\partialavoids tuningKK, and repair should be unconditional because failure is nearly certain while severity is hard to predict\. This recommendation applies when the answer dependency remains in the edited block\. Beyond that boundary, a system needs a different repair operator, not only a larger fixed budget\.

#### Coverage, not importance\.

Chunk\-composition methods repair states computed from correct inputs but missing cross\-chunk attention, so large KV deviations can be useful signals\([Yao et al\. 2025](https://arxiv.org/html/2609.17983#bib.bib1)\)\. An edit inside a cached document creates a different defect: a broken dependency chain\. Scattered high\-scoring positions are recomputed from stale inputs, while a contiguous window rebuilds the path in order\. Selection quality is therefore a property of the set, not each position in isolation\. Deviation and attention scores can help when surrounding states are sound\. When they are stale, repair must cover the path that produces the answer\.

#### Localization evidence overstates what repair can do\.

Cache\-editing work can rank positions by patching clean states into a corrupted run\([Li 2026](https://arxiv.org/html/2609.17983#bib.bib7)\), but that ranking need not transfer to recomputation\. Transplant imports states produced in a correct context, including information from outside the chosen set\. Recomputation must rebuild those states from the stale cache\. The operator gap shows that a rule intended for recomputation must be tested under recomputation\. Transplant locates where clean information can influence the output, but it is not an upper bound on deployable repair\.

#### Where repair still fails\.

Once dependent text moves250250tokens away,EditLocalcollapses and only stale\-query attention on Llama provides a useful deployable alternative\. Search is not the whole problem:CarrierWindowknows the carrier location yet remains far below full recovery on Llama and Mistral\. A second pass or a window spanning the edit\-to\-carrier path may restore the missing inputs, but neither is tested here\. Qwen’s stronger result also shows that the boundary is model\-dependent\.

#### Limitations\.

Edits are single, contiguous, and length\-preserving, and the synthetic blocks make every edit answer\-relevant\. The failure rate therefore does not extend to arbitrary edits, and the two\-value metrics do not measure long free\-form answers\. Llama retains3434of7575held\-out derived items and about one third atd=250d\{=\}250, so those cells cover only solvable examples, and distance also changes recency\. The transplant panel uses1515development items without Mistral, although the operator gap is also measured on held\-out data\.EditLocal@∂@\\partialwas inferred from unblinded curves rather than tested as a registered arm\. The models are dense and near88B, and timing uses one GPU at batch11while excluding the identical query forward\. Multi\-edit and length\-changing updates, larger or sparse models, and production serving remain open\.

## Conclusion

We framed in\-place repair of a stale KV cache as budgeted recomputation and compared training\-free selection policies at a matched budget on paired factual edits across three model families\. Recomputing a contiguous window from the edit to the end of its block restores post\-edit behavior almost completely wherever the dependent text is adjacent, beats every signal\-based rule by a wide margin, and runs1313–21×21\\timesfaster than a re\-prefill\. The result carries a mechanism and a boundary\. Repair works by rebuilding a forward dependency chain, so position sets that look sufficient under transplant fail under recomputation, and the same window fails once the chain grows long\. Edit\-local recomputation is the baseline that any future policy for this problem has to beat\.

## References

- Agarwalet al\.\(2025\)S\. Agarwal, S\. Sundaresan, S\. Mitra, D\. Mahapatra, A\. Gupta, R\. Sharma, N\. J\. Kapu, T\. Yu, and S\. SainiCache\-craft: managing chunk\-caches for efficient retrieval\-augmented generation\.Proceedings of the ACM on Management of Data3\(3\),pp\. 1–28\.Cited by:[Selective recomputation\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px2.p1.1)\.
- Bergmanet al\.\(2025\)S\. Bergman, A\. Kermarrec, D\. Petrescu, R\. Pires, M\. Randl, M\. De Vos, and J\. ZhangLeveraging approximate caching for faster retrieval\-augmented generation\.InProceedings of the 26th International Middleware Conference,pp\. 340–353\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p1.1)\.
- Cohenet al\.\(2024\)R\. Cohen, E\. Biran, O\. Yoran, A\. Globerson, and M\. GevaEvaluating the ripple effects of knowledge editing in language models\.Transactions of the Association for Computational Linguistics12\.External Links:2307\.12976Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p1.1)\.
- Huet al\.\(2025\)J\. Hu, W\. Huang, W\. Wang, H\. Wang, T\. Hu, Z\. Qin, H\. Feng, X\. Chen, Y\. Shan, and T\. XieEPIC: efficient position\-independent caching for serving large language models\.InInternational Conference on Machine Learning,pp\. 24391–24402\.Cited by:[Selective recomputation\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px2.p1.1)\.
- Jinet al\.\(2025\)C\. Jin, Z\. Zhang, X\. Jiang, F\. Liu, S\. Liu, X\. Liu, and X\. JinRagcache: efficient knowledge caching for retrieval\-augmented generation\.ACM Transactions on Computer Systems44\(1\),pp\. 1–27\.Cited by:[Document\-level KV reuse\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px1.p1.1)\.
- Li \(2026\)B\. LiModels take notes at prefill: kv cache can be editable and composable\.arXiv preprint arXiv:2606\.17107\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p2.1),[Introduction](https://arxiv.org/html/2609.17983#Sx1.p3.1),[KV\-cache editing\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px3.p1.1),[Localization evidence overstates what repair can do\.](https://arxiv.org/html/2609.17983#Sx6.SSx6.SSS0.Px3.p1.1)\.
- Liet al\.\(2026\)M\. Li, S\. Liu, D\. Fu, H\. Wang, Y\. Xia, H\. Li, H\. Yan, and P\. LiKVEraser: learning to steer kv cache for efficient localized context erasing\.arXiv preprint arXiv:2606\.17034\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p2.1),[KV\-cache editing\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px3.p1.1)\.
- Luet al\.\(2025\)S\. Lu, H\. Wang, Y\. Rong, Z\. Chen, and Y\. TangTurborag: accelerating retrieval\-augmented generation with precomputed kv caches for chunked text\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 6599–6612\.Cited by:[Document\-level KV reuse\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px1.p1.1)\.
- Maet al\.\(2026\)B\. Ma, J\. Eitzinger, and H\. KoestlerLeyline: KV cache directives for agentic inference\.External Links:2606\.01065Cited by:[KV\-cache editing\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px3.p1.1),[Question Setting\.](https://arxiv.org/html/2609.17983#Sx3.SS0.SSS0.Px1.p2.1)\.
- Ouyanget al\.\(2025\)J\. Ouyang, T\. Pan, M\. Cheng, R\. Yan, Y\. Luo, J\. Lin, and Q\. LiuHoh: a dynamic benchmark for evaluating the impact of outdated information on retrieval\-augmented generation\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 6036–6063\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p1.1),[Introduction](https://arxiv.org/html/2609.17983#Sx1.p2.1),[Document\-level KV reuse\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px1.p1.1)\.
- Packeret al\.\(2023\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: towards LLMs as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p1.1)\.
- Panet al\.\(2026\)Z\. Pan, A\. D\. PATEL, Y\. Shen, Z\. Hu, Y\. Guan, W\. Li, L\. Qin, Y\. Wang, and Y\. DingKVFlow: efficient prefix caching for accelerating llm\-based multi\-agent workflows\.Advances in Neural Information Processing Systems38,pp\. 126246–126265\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p1.1)\.
- Wanget al\.\(2026\)S\. Wang, J\. Chen, Y\. Pan, H\. Huang, Y\. Hao, X\. Zou, W\. Xia, W\. Zhang, H\. Wang, J\. Li,et al\.ProphetKV: user\-query\-driven selective recomputation for efficient kv cache reuse in retrieval\-augmented generation\.arXiv preprint arXiv:2602\.02579\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p3.1),[Selective recomputation\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px2.p1.1),[Cost accounting\.](https://arxiv.org/html/2609.17983#Sx3.SS0.SSS0.Px3.p1.1),[Selection policies\.](https://arxiv.org/html/2609.17983#Sx3.SS0.SSS0.Px5.p1.1)\.
- Yanget al\.\(2025\)H\. Yang, R\. Zhang, M\. Huang, W\. Wang, Y\. Tang, Y\. Li, Y\. Liu, and D\. ZhangKvshare: an llm service system with efficient and effective multi\-tenant kv cache reuse\.arXiv preprint arXiv:2503\.16525\.Cited by:[Selective recomputation\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2369–2380\.Cited by:[Paired edit benchmark\.](https://arxiv.org/html/2609.17983#Sx4.SS0.SSS0.Px1.p1.1)\.
- Yaoet al\.\(2025\)J\. Yao, H\. Li, Y\. Liu, S\. Ray, Y\. Cheng, Q\. Zhang, K\. Du, S\. Lu, and J\. JiangCacheblend: fast large language model serving for rag with cached knowledge fusion\.InProceedings of the twentieth European conference on computer systems,pp\. 94–109\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p1.1),[Introduction](https://arxiv.org/html/2609.17983#Sx1.p3.1),[Selective recomputation\.](https://arxiv.org/html/2609.17983#Sx2.SS0.SSS0.Px2.p1.1),[The repair interface\.](https://arxiv.org/html/2609.17983#Sx3.SS0.SSS0.Px2.p1.2),[Cost accounting\.](https://arxiv.org/html/2609.17983#Sx3.SS0.SSS0.Px3.p1.1),[Selection policies\.](https://arxiv.org/html/2609.17983#Sx3.SS0.SSS0.Px5.p1.1),[Coverage, not importance\.](https://arxiv.org/html/2609.17983#Sx6.SSx6.SSS0.Px2.p1.1)\.
- Yeet al\.\(2026\)H\. Ye, Z\. Gao, M\. Ma, Q\. Wang, Y\. Fu, M\. Chung, Y\. Lin, Z\. Liu, J\. Zhang, D\. Zhuo,et al\.Kvcomm: online cross\-context kv\-cache communication for efficient llm\-based multi\-agent systems\.Advances in Neural Information Processing Systems38,pp\. 17882–17928\.Cited by:[Introduction](https://arxiv.org/html/2609.17983#Sx1.p1.1)\.