Models Take Notes at Prefill: KV Cache Can Be Editable and Composable

# KV Cache Can Be Editable and Composable
Source: [https://arxiv.org/html/2606.17107](https://arxiv.org/html/2606.17107)
## Models Take Notes at Prefill: KV Cache Can Be Editable and Composable

###### Abstract

Prefix caching reuses prefill only across an exactly shared prefix, so one changed field invalidates the entire downstream cache\. Yet overwriting the field’s own key/value vectors and reusing the rest leaves the model acting on the*old*value\. The reason, established causally across four model families: at prefill the model has already written the*field\-conditioned conclusion*onto downstream*notes*; the field’s own key/value drives under1%1\\%of the decision\. Read as a notebook of memoized conclusions, two capabilities follow\.\(1\) It is editable\.A salient*erratum*amends the notes; and with*chain\-of\-thought*, editing the field alone recovers the decision \(1\.001\.00at88B,∼1%\{\\sim\}1\\%compute\), while without CoT it is ignored\.\(2\) It is composable\.The notes are position\-portable, so a precompiled skill can be RoPE\-repositioned and spliced into any context, indistinguishable from full recompute \(logit cosine0\.900\.90–0\.9990\.999, twelve models\) atO​\(L\)O\(L\)rather thanO​\(L2\)O\(L^\{2\}\)time\-to\-first\-token\. A unified edit\+compose agent stays decision\-identical to recompute at up to14\.9×14\.9\\timeslower latency\. The approach applies to any per\-token attention KV cache, validated across scale, quantization, Mixture\-of\-Experts, and multimodal caches, and extends to several attention variants through small adapters\. Because the erratum is append\-only, it composes with production prefix caching: in an online vLLM benchmark it keeps the prefix cache\-aligned \(98\.5%98\.5\\%hit\-rate\), cuttingp​90p90time\-to\-first\-token by5353–398×398\\times\.

## 1 Introduction

Modern LLM agents re\-read long, mostly\-static instructions on every turn—a system policy, a tool specification, retrieved documents\. Key/value \(KV\) caching makes this affordable by reusing the prefill across turns, but only across an*exact*shared prefix\. The moment one token changes inside the reused region—a timestamp, a user id, an order’s status—the keys and values of*every*later token are invalidated, because each attended to the token that changed\. The de\-facto workaround—*hoisting*all mutable content to the end so the static prefix stays cache\-aligned—pushes an inference\-layer constraint into the application layer: fields referenced in several places, nested sub\-agent prompts, and dynamically assembled contexts cannot all be cleanly hoisted, and the application must enumerate every mutable field in advance\.

This paper starts from a concrete puzzle\. The region of the cache*before*a field is, by construction, independent of the field’s value—we measure a key/value deviation of exactly0\.00\.0when the field changes\. One might therefore hope to*surgically*refresh only the field’s own keys and values, leave the rest of the cache stale, and pay almost nothing\. We find this fails completely: the model’s decision reverts to the*old*field value, as if the edit never happened \([Figure˜1](https://arxiv.org/html/2606.17107#S1.F1)b\)\.

### The discovery\.

The reason, which we establish causally, is that transformers do not defer their reasoning to decode time\. At*prefill*the model already computes the*field\-conditioned conclusion*and writes it onto downstream tokens—disproportionately onto aggregator/delimiter tokens that later positions attend through\. The decision then reads these*notes*, not the field itself: across models the field’s own KV causally drives less than1%1\\%of the decision, while the downstream notes drive essentially all of it\. The KV cache is best understood not as a frozen byproduct of prefill but as a*notebook of memoized conclusions*\([Figure˜1](https://arxiv.org/html/2606.17107#S1.F1)a\)\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x1.png)Figure 1:Models take notes at prefill\.\(a\) At prefill the model memoizes the field\-conditioned conclusion onto downstream aggregator tokens \(orange\); at decode the decision reads those notes \(blue\)\. \(b\) Consequently, surgically editing the field’s own KV is ignored \(without a reasoning chain\), but the decision is recovered cheaply: recompute the*affected*downstream suffix, or—cheaper and robust— append a salient*erratum*\(P\(new decision\), no\-CoT model; with chain\-of\-thought the field\-only refresh alone also works,[Figure˜2](https://arxiv.org/html/2606.17107#S1.F2)c\)\. \(c\) Because the notes are position\-portable, a precompiled skill can be pasted into a new context inO​\(L\)O\(L\)rather thanO​\(L2\)O\(L^\{2\}\)time\.
### Two capabilities from one mechanism\.

Once the cache is read as a notebook of memoized conclusions, we can manipulate those conclusions directly—editing or reusing them in place rather than recomputing them—in two ways\. \(1\) If the conclusion is already written downstream, then*editing*a field means amending the notes, not recomputing them\. The cheap and robust fix is a one\-line salient*erratum*that overrides the stale notes; one can instead recompute the affected downstream notes \(reliably for the full affected suffix, cheaply\-but\-unreliably for a top\-KK,field\+selective@KK\)\. An even cheaper in\-place refresh of the field alone is gated by*chain\-of\-thought*: with a reasoning chain the model re\-reads the field and the edit suffices, without it the edit is ignored\. \(2\) If the notes are localized and position\-portable, then a reusable skill can be*composed*into a new context by repositioning and splicing its cached notes—no recompute\. A*unifying*experiment, editing a field*inside*a transplanted skill, shows the two operations act on the same notes\.[Figure˜2](https://arxiv.org/html/2606.17107#S1.F2)previews both capabilities\. More broadly, we view editing and composition as first instances of a*programmable*KV cache—a structured memory that systems, and eventually models trained to expose it, can read, write, and rearrange rather than only extend linearly\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x2.png)Figure 2:Editing and composing, previewed\(Qwen3\-8B unless noted; detail in[Sections˜3](https://arxiv.org/html/2606.17107#S3),[4](https://arxiv.org/html/2606.17107#S4)and[6](https://arxiv.org/html/2606.17107#S6)\)\. \(a\)*KV editing landscape*: naive edits \(stale, field\-only without CoT, CacheBlend\) fail, while the append\-onlyerratum/field\+erratumreach full\-reprefill correctness cheaply and robustly \(hoisting also works but needs prompt surgery\)\. \(b\)*Recompute the affected notes*: recovery climbs as more of the post\-field*affected suffix*is recomputed—reliable in the limit, but the cheap top\-KKversion \(field\+selective@KK\) is unreliable \([Figure˜5](https://arxiv.org/html/2606.17107#S4.F5)\)\. \(c\)*Chain\-of\-thought, not model size, gates the cheapest edit*: with CoT the near\-free field\-only refresh alone recovers the decision \(1\.001\.00\); without CoT it is ignored \(0\.000\.00\); the erratum fixes both\. \(d\)*Composing\+\+editing is lossless*: the unified agent stays decision\-identical to full recompute across models\.
### Structure of the paper\.

- •A mechanism\([Section˜3](https://arxiv.org/html/2606.17107#S3)\)—*attention\-mediated memoized inference*—established by four causal probes \(locality patching, suffix concentration, linear probing, circuit knockout\) and four further controls \(content/conclusion dissociation, layer\-timing, specificity, note\-injection;[Appendix˜C](https://arxiv.org/html/2606.17107#A3)\), replicated across*four model families*\(Qwen3, Llama\-3\.1, Gemma\-2, Mistral\), resolved to a*component\-level circuit*—named read/write heads, a causal conclusion direction, an SAE feature, attention\-vs\-MLP, and causal scrubbing \([Appendix˜D](https://arxiv.org/html/2606.17107#A4)\)—and connected to the delimiter\-token aggregation seen in interpretability\.
- •An editing capability\([Section˜4](https://arxiv.org/html/2606.17107#S4)\): naive KV editing fails; the erratum /field\+erratumfix matches the hoist\-to\-end oracle without prompt surgery; an analysis of when the∼1%\{\\sim\}1\\%\-compute in\-place edit suffices \(it requires reasoning and is strongly model\-dependent\); and a head\-to\-head with*weight*editing \(ROME, LoRA\) showing it is the wrong tool for mutable per\-request state \(global contamination, collateral,3030–50×50\\timesslower\)\.
- •A composing capability\([Section˜5](https://arxiv.org/html/2606.17107#S5)\): position\-portable transplant of precompiled skills,O​\(L\)O\(L\)vs\.O​\(L2\)O\(L^\{2\}\)TTFT \(13\.9×13\.9\\timesat3232k\), with a seam\-repair knob—building on prior caching work \([Section˜2](https://arxiv.org/html/2606.17107#S2)\), our addition being the mechanism that explains it and a correctness lens\.
- •The unification\([Section˜6](https://arxiv.org/html/2606.17107#S6)\): this experiment and a unified edit\+compose agent over thirteen models\.
- •An application: user memory\([Section˜7](https://arxiv.org/html/2606.17107#S7)\): the large, mutable user\-memory document is both composed \(precompiled, repositioned, spliced\) and edited \(in\-place / erratum\) as one set of notes—decision\-faithful to full recompute at2\.32\.3–4\.3×4\.3\\timeslower time\-to\-first\-token, validated to7070B and on real long\-conversation memory \(LoCoMo, transplant≡\\equivfull recompute in QA accuracy\)—with a pre\-registered, statistically\-controlled evaluation\.
- •Applicability to multimodal and new attention mechanisms\([Section˜8](https://arxiv.org/html/2606.17107#S8)\): the mechanism carries from small models through MoE and low\-bit quantization to multimodal image caches; lightweight adapters bring it to MLA and interleaved M\-RoPE; a mask\-based remedy handles sliding\-window attention; and we chart where it breaks down against the 2026 sparse/compressed\-attention frontier\.
- •Systems payoff\([Section˜9](https://arxiv.org/html/2606.17107#S9)\): a real agentic environment and a comprehensive online vLLM serving benchmark \(V1 engine, continuous batching, Poisson load\)—98\.5%98\.5\\%vs\.1%1\\%prefix\-cache hit\-rate,5353–398×398\\timeslowerp​90p90TTFT, and throughput gains that grow with load to14\.5×14\.5\\times\.

## 2 Related work

### Where computation is stored, and how to edit it\.

A line of interpretability work localizes stored*knowledge*in transformer weights and edits it there: ROME and MEMIT\[[22](https://arxiv.org/html/2606.17107#bib.bib1),[23](https://arxiv.org/html/2606.17107#bib.bib2)\]locate and rewrite factual associations in MLP weights, while circuit analyses such as the indirect\-object\-identification study\[[32](https://arxiv.org/html/2606.17107#bib.bib4)\]trace how specific computations are carried by attention heads\. Weight editing targets*durable, global*facts; we compare against a faithful ROME and a LoRA fine\-tune empirically \([Table˜1](https://arxiv.org/html/2606.17107#S4.T1)\) and find them ill\-suited to*mutable per\-request*state—a global edit contaminates concurrent requests and damages unrelated decisions—which is exactly the niche the editable KV cache fills\. Closest in spirit,Lindsey and others \[[17](https://arxiv.org/html/2606.17107#bib.bib5)\]find models commit*plans*to specific tokens \(e\.g\. a planned rhyme stored on a line\-break token\) during the forward pass\. We study a complementary object: not weights and not decode\-time computation, but the*KV cache*—the activations an inference system already stores and an editor can directly manipulate—and show it holds*memoized conclusions*concentrated on aggregator/delimiter tokens\. To our knowledge this is the first causal account of why in\-place KV field editing fails and what to do instead\.

### KV reuse and composable caching\.

Reusing precomputed KV beyond an exact prefix is an active systems topic, and our*composing*capability builds directly on it; we claim none of the caching machinery as novel\. Prompt Cache\[[7](https://arxiv.org/html/2606.17107#bib.bib6)\]precomputes reusable prompt modules with position placeholders and splices them\. CacheBlend\[[35](https://arxiv.org/html/2606.17107#bib.bib7)\]reuses non\-prefix chunk KV and selectively recomputes∼15%\{\\sim\}15\\%of tokens to restore cross\-attention\. EPIC\[[11](https://arxiv.org/html/2606.17107#bib.bib8)\]introduces position\-independent caching withAttnLink, recomputing only a few chunk\-boundary tokens \(exploiting attention sinks\) for near\-linear recompute\. CacheSlide\[[18](https://arxiv.org/html/2606.17107#bib.bib9)\]reuses KV in a position\-*aware*way via relative\-position\-dependent caching, and MPIC\[[40](https://arxiv.org/html/2606.17107#bib.bib10)\]extends position\-independent caching to the multimodal setting by recomputing image\-boundary tokens; KVLink\[[34](https://arxiv.org/html/2606.17107#bib.bib11)\]is a further reuse system\. Mapped onto our terms, our RoPE\-repositioning is CacheSlide’s relative\-position reuse, our seam\-repair is the boundary recompute of CacheBlend/EPIC/MPIC, and our image\-KV transplant is MPIC’s idea \(we*re\-rotate*M\-RoPE rather than recompute\)\. Our contributions over this line are orthogonal: \(i\) the*mechanism*that explains*why*boundary recompute is what is needed; \(ii\) a*decision\-governance*evaluation—does a transplanted skill still*govern the tool decision*, rather than only preserve perplexity or throughput; \(iii\) the*editing*axis and the edit\+compose unification; and \(iv\) adapters that extend the operations to new attention representations \(MLA, interleaved M\-RoPE, sliding\-window\)\. We note honestly that this editable/composable\-cache direction grew directly out of the position\-independent and position\-aware caching of EPIC and CacheSlide\[[11](https://arxiv.org/html/2606.17107#bib.bib8),[18](https://arxiv.org/html/2606.17107#bib.bib9)\], and out of discussions with Junhao Hu \(first author of EPIC; see Acknowledgements\)\.

### Prefix caching, KV compression, and reuse systems\.

Production prefix caching \(vLLM Automatic Prefix Caching, SGLang RadixAttention\) reuses exact prefixes\. A large literature instead*compresses*or*evicts*the cache: StreamingLLM keeps attention sinks\[[33](https://arxiv.org/html/2606.17107#bib.bib12)\]; H2O\[[39](https://arxiv.org/html/2606.17107#bib.bib13)\], Scissorhands\[[20](https://arxiv.org/html/2606.17107#bib.bib16)\], and SnapKV\[[16](https://arxiv.org/html/2606.17107#bib.bib14)\]evict low\-importance tokens; Quest\[[29](https://arxiv.org/html/2606.17107#bib.bib15)\]keeps all tokens but attends sparsely\. Serving systems stream or share cached KV across requests—CacheGen\[[19](https://arxiv.org/html/2606.17107#bib.bib17)\]for fast loading and RAGCache\[[14](https://arxiv.org/html/2606.17107#bib.bib18)\]for retrieval reuse\. These change*which*tokens are present, and compose with our edit/transplant operations only over retained tokens; we treat them, the latent/decoupled\-RoPE representation of Multi\-head Latent Attention\[[2](https://arxiv.org/html/2606.17107#bib.bib29),[3](https://arxiv.org/html/2606.17107#bib.bib30)\], and the 2026 sparse/compressed\-attention designs as scope boundaries in[Section˜8](https://arxiv.org/html/2606.17107#S8)\. Our repositioning relies on rotary position embeddings\[[28](https://arxiv.org/html/2606.17107#bib.bib27)\]and their extensions\[[25](https://arxiv.org/html/2606.17107#bib.bib28)\]\.

### Activation\-level interventions\.

Beyond weight editing, a body of work intervenes on*activations*: steering and inference\-time interventions\[[15](https://arxiv.org/html/2606.17107#bib.bib19)\], task and function vectors\[[12](https://arxiv.org/html/2606.17107#bib.bib20),[30](https://arxiv.org/html/2606.17107#bib.bib21)\], and the causal\-mediation/activation\-patching methodology we adapt\[[31](https://arxiv.org/html/2606.17107#bib.bib22)\]\. These edit residual\-stream directions to change behavior; we instead read and write the*KV cache*itself—the per\-token activations a serving system already persists—which is what makes the intervention both interpretable and deployable\.

### Agents and tool use\.

Our evaluation targets tool\-using agents in the style of ReAct\[[37](https://arxiv.org/html/2606.17107#bib.bib25)\]and Toolformer\[[27](https://arxiv.org/html/2606.17107#bib.bib26)\], and we measure end\-to\-end task success on theτ2\\tau^\{2\}\-bench tool\-agent\-user benchmark\[[1](https://arxiv.org/html/2606.17107#bib.bib24)\], the dual\-control successor ofτ\\tau\-bench\[[36](https://arxiv.org/html/2606.17107#bib.bib23)\], where state changes mid\-trajectory make cache staleness a first\-class correctness problem\.

## 3 The discovery: memoized inference in the KV cache

![Refer to caption](https://arxiv.org/html/2606.17107v1/x3.png)Figure 3:Four causal probes for memoized inference\.\(a\) Refreshing the field’s own KV recovers≈0\\approx 0of the decision; recomputing the downstream recovers it fully\. \(b\) Recovery is suffix\-concentrated, accruing only as many post\-field tokens are patched\. \(c\) The decision reads almost entirely from downstream notes, not the field token\. \(d\) Once the value is present, override wording is redundant and “re\-evaluate” phrasing hurts—the note is a committed conclusion\.### Where the conclusion is written and read\.

We first state the account that the probes in this section establish\. A transformer prefills the prompt left\-to\-right, each position attending to all earlier ones\. At a small number of positions*after*the mutable field—*aggregator*tokens such as punctuation, newlines, and section breaks, which later tokens route attention through—the model does more than cache the raw token: it computes the field\-conditioned answer \(e\.g\. “status isshipped, so the action is*deny*”\) and writes that conclusion into those positions’ cached key/value vectors at mid\-to\-late layers\. These cached vectors are the*notes*, and they lie*downstream*of the field\. At decode time the decision token does not re\-derive the answer from the field; it attends back to the note positions, and a small set of late\-layer attention heads read the stored conclusion into the output logit\. Refreshing the field’s*own*KV therefore changes little, since the conclusion was written elsewhere and the field itself accounts for under1%1\\%of the decision\. The four probes below establish each link of this chain causally:*where*the notes lie, that they encode a*conclusion*rather than a copy of the field, and that the decision*reads*them\.

### Setup\.

We study a minimal but agent\-realistic decision: a context contains a policy rule and a*mutable field*whose value determines the correct action \(e\.g\. “cancel an order only if its status ispending”; withstatus=shippedthe correct action flips from*cancel*to*deny*\)\. After prefilling the full context we compare four cache states at the decision token:*stale*\(old field, old downstream\),*field\-only*\(refresh the field’s KV, leave the downstream stale\),*full\-downstream*\(recompute everything after the field\), and*oracle*\(a clean prefill of the new value\)\. We report*decision recovery*: the fraction of the oracle’s flip that a cache state reproduces \(0= behaves like stale,11= behaves like oracle\)\. Four independent probes converge on one account \([Figure˜3](https://arxiv.org/html/2606.17107#S3.F3)\);[Appendix˜B](https://arxiv.org/html/2606.17107#A2)gives a complete worked example—the verbatim prompt, the erratum, and recorded model responses\.

### \(1\) Locality: the field’s own KV barely matters\.

Refreshing only the field’s KV recovers essentially none of the decision—field\-only recovery is−0\.028\-0\.028on Llama\-3\.1\-8B and near zero across models—whereas recomputing the downstream recovers it fully \(1\.01\.0\) \([Figure˜3](https://arxiv.org/html/2606.17107#S3.F3)a\)\. The field is read*indirectly*: its causal contribution to the decision is under1%1\\%\.

### \(2\) Suffix concentration: the effect lives downstream and late\.

Sweeping how many downstream tokens we patch \(ranked by causal effect\) shows recovery accrues slowly and saturates only as we include many tokens after the field, with the causal mass concentrated in mid/late layers \([Figure˜3](https://arxiv.org/html/2606.17107#S3.F3)b\)\. The information the decision needs is not in the field but distributed over the tokens that followed it at prefill \([Figure˜3](https://arxiv.org/html/2606.17107#S3.F3)c\)\.

### \(3\) Linear decodability\.

The field\-conditioned conclusion is linearly decodable from those downstream tokens’ residual stream at prefill time—i\.e\. the model has already*computed and written down*the answer, not merely copied the field\.

### \(4\) Knockout and dose\-response\.

Ablating the high\-effect downstream tokens flips the decision back to stale, and the effect grows with the number/position of memoizing tokens, ruling out a diffuse explanation: specific aggregator/delimiter positions carry the conclusion\. This mirrors the delimiter\-token aggregation reported byLindsey and others \[[17](https://arxiv.org/html/2606.17107#bib.bib5)\]for forward planning; here the stored quantity is a backward\-looking, field\-conditioned*conclusion*\.

### What the note contains\.

If the note were a verbatim copy of the field, restating the value would suffice and emphatic wording would be inert\. Instead, a wording ablation \([Figure˜3](https://arxiv.org/html/2606.17107#S3.F3)d\) shows that once the corrected value is present, the override phrasing is*redundant*\(bare value, tagged update, and explicit override all reach≈1\.0\\approx 1\.0P\(safe\)\), while aggressive “disregard your earlier conclusion and re\-evaluate” phrasing actively*hurts*\(0\.810\.81\)\. The note behaves like a committed conclusion that a late, salient correction can overwrite—but that confrontational instructions can destabilize\. We name the phenomenonattention\-mediated memoized inference\. The remainder of this section stress\-tests the account—further controls, more architecture families, and a component\-level circuit—before the rest of the paper builds on it\.

### Separating the stored conclusion from the field content\.

A linear probe can*decode*the conclusion from the downstream tokens, but decodability alone does not show the decision*uses*it—the same tokens also encode the field’s raw content, and a probe finds both\. Four causal controls \(three models each: Qwen3\-8B/4B and Llama\-3\.1\-8B;[Appendix˜C](https://arxiv.org/html/2606.17107#A3)\) close that gap; the causal patch, not the probe, is the instrument that separates the two\.

\(i\) It is the conclusion, not the field content\.We hold the field value byte\-identical and flip a single rule token \(a polarity*trigger*\) so the*conclusion*inverts while the field*content*stays constant\. Transplanting the downstream notes then carries the entire flipped conclusion \(recovery0\.9980\.998–1\.0091\.009\), whereas patching the changed rule token carries none \(−0\.007\-0\.007to\+0\.007\+0\.007\)\. Since the content never changed, the notes cannot merely be re\-encoding it\.

\(ii\) The note is written before it is read\.Within the single prefill pass the conclusion becomes decodable on the downstream aggregator about twelve layers*earlier*\(relative depth0\.310\.31–0\.390\.39\) than the point at which the decision token commits to its answer \(depth0\.730\.73–0\.770\.77\): the note already exists at prefill, ahead of the read\.

\(iii\) A few specific tokens carry it\.Transplanting the eight*highest\-effect*downstream positions recovers0\.740\.74–0\.790\.79of the decision, while eight*random*downstream positions recover≤0\.035\{\\leq\}0\.035: the conclusion lives in a few specific aggregator tokens, not a diffuse code\.

\(iv\) The decision follows the note, even when the note is wrong\.Injecting a*false*note \(the opposite conclusion’s downstream KV\) into an otherwise\-consistent cache makes the decision follow the written note*against its own live field*\(recovery≈1\.0\\approx 1\.0\); a handful of note tokens suffice\. The account also holds off the synthetic template: field\-only recovery stays≈0\{\\approx\}0for multi\-hop reasoning and free\-form conversational phrasing, and is bounded only for near\-verbatim attribute lookup, where the field is itself partly a copy \([Appendix˜C](https://arxiv.org/html/2606.17107#A3)\)\.

### Not a model\-family artifact: four families\.

To rule out the aggregator\-token account being a Qwen3/Llama tokenizer artifact, we replicate five probes \(the locality probe and the four deep controls\) on two further architecture families,Gemma\-2\-9BandMistral\-7B\(a tokenizer\-robust readout; for Gemma\-2 we keep its attention/logit soft\-capping intact\)\. Every result holds: field\-only recovery0\.0050\.005/0\.1370\.137vs\. full\-downstream1\.01\.0; the conclusion/content dissociation \(trigger\-only≈0\\approx 0with the field held identical vs\. notes≈1\.0\\approx 1\.0\); top\-88vs\. random\-88specificity \(0\.950\.95vs\.0\.480\.48\); false\-note injection \(0\.980\.98–1\.01\.0, follow\-rate1\.01\.0\); and write\-before\-read timing \(write depth0\.190\.19–0\.260\.26vs\. decision commit0\.470\.47–0\.480\.48\)\. The mechanism is consistent across*four*families \(Qwen3, Llama\-3\.1, Gemma\-2, Mistral;[Appendix˜C](https://arxiv.org/html/2606.17107#A3)\)\.

### A component\-level circuit\.

The probes above show*where*the conclusion is stored; five further interventions \([Appendix˜D](https://arxiv.org/html/2606.17107#A4); replicated across Llama\-3\.1, Qwen3, Gemma\-2, and Mistral\) show*which components*write and read it\. The pattern is*distributed write, concentrated read*\.

Write \(many components, mid layers\)\.Mid\-layer*attention*—not the MLPs—does most of the writing onto the aggregator tokens \(≈0\.6\\approx 0\.6of the write on Llama\)\. The conclusion is stored*redundantly*: it is easy to*decode*\(a trained sparse\-autoencoder feature separates the two conclusions at AUC1\.01\.0\), yet its causal content is spread across∼10\{\\sim\}10–3030features along a shared low\-rank direction \(that direction carries≈25×\\approx 25\\timesthe decision\-effect of a random one\)\.

Read \(few components, late layers\)\.A small, nameable set of late*read heads*—those attending from the decision token back to the aggregators—funnels the stored conclusion into the output logit \(1212such heads recover0\.780\.78of the decision; the same number of random heads recover≈0\\approx 0\)\.

Control\.Causal scrubbing confirms the note alone governs the decision: resampling it to the opposite conclusion flips the decision, while resampling everything else leaves it unchanged\. As at the token level, the single feature that best*decodes*the conclusion is not by itself*causally*sufficient—decodability and causation come apart, now among individual features\. With the mechanism established, the rest of the paper turns it into capabilities: the cache is editable \([Section˜4](https://arxiv.org/html/2606.17107#S4)\) and composable \([Section˜5](https://arxiv.org/html/2606.17107#S5)\)\.

## 4 Consequence I: the cache is editable

The mechanism tells us how to repair the cache after a field changes, not only why a naive edit fails\. Because the decision reads a conclusion stored in the downstream notes \([Section˜3](https://arxiv.org/html/2606.17107#S3)\),*editing*means amending those notes rather than recomputing the prefix; this section establishes when a naive edit fails, the two ways to repair the notes, and which to prefer\.[Figure˜4](https://arxiv.org/html/2606.17107#S4.F4)contrasts the two cache operations this section and the next make precise: editing appends a salient correction; composing repositions and splices precompiled KV\.

### Naive editing fails—by design\.

The mechanism predicts the in\-place edit will fail, and it does: refreshing the field’s KV while reusing the stale downstream leaves the decision at the old value \([Figure˜1](https://arxiv.org/html/2606.17107#S1.F1)b\)\. The prefix before the field is genuinely reusable \(deviation0\.00\.0\); the problem is that the conclusion was already memoized after it\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x4.png)Figure 4:Two cache operations\.\(a\)*Editable*, two ways to amend the stale notes:field\+selectiverecomputes the field plus the top\-KKhighest\-effect downstream notes \(O​\(K\)O\(K\); recomputed cells outlined in red, the rest reused stale\), while theerratumappends one salient correction \(O​\(1\)O\(1\), the whole prefix is reused\)\. \(b\)*Composable*: precompute a skill’s KV in isolation, RoPE\-reposition it to the target positions, and splice it in \(O​\(L\)O\(L\), no reprefill\)\.
### Two ways to fix it, neither a full reprefill\.

If the stale notes carry an old conclusion, there are two interventions\.*\(i\) Recompute the affected notes*after the field\. The safe version recomputes the*whole*suffix after the field—exactly the*full\-downstream*state from[Section˜3](https://arxiv.org/html/2606.17107#S3)\(recovery1\.01\.0\)—but pays the post\-field prefill\. The cheaper version,field\+selective@KK, recomputes only the field plus theKKdownstream tokens that carry most of the decision—the high\-effect aggregator positions identified in[Section˜3](https://arxiv.org/html/2606.17107#S3), ranked by their causal effect on the decision—and reuses every other cached token unchanged\. It is cheaper but*unreliable*: the smallestKKthat restores the decision is strongly model\-dependent \(below\)\.*\(ii\) Append an erratum*: one salient line late in the context—“\[STATE UPDATE\] field→\\rightarrownew; overrides any earlier value and conclusion”—so the decision token attends to a fresh, authoritative note \(verbatim template in[Appendix˜B](https://arxiv.org/html/2606.17107#A2)\)\. The erratum is the cheap, robust default: appended after the field \(field\+erratum\) it matches the strong*hoist\-to\-end*oracle*without rewriting the prompt*\(P\(correct\)1\.001\.00on the gated\-task frontier,[Figure˜2](https://arxiv.org/html/2606.17107#S1.F2)a;[Table˜6](https://arxiv.org/html/2606.17107#A5.T6)\), and being append\-only it composes with prefix caching \([Section˜9](https://arxiv.org/html/2606.17107#S9)\)\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x5.png)Figure 5:Editing is model\-dependent\(the editing landscape and the chain\-of\-thought split are previewed in[Figure˜2](https://arxiv.org/html/2606.17107#S1.F2)\)\. \(a\)*Given*chain\-of\-thought \(which gates whether the edit works at all\), the residual stickiness is model\-dependent: the near\-free field\-only recovery varies non\-monotonically across model sizes\. \(b\)field\+selective@KK: the minimal recomputeK⋆K^\{\\star\}to reach full quality is likewise model\-dependent\.
### Chain\-of\-thought, not model size, gates the cheapest edit\.

A third, near\-free option—refresh the field’s own KV alone \(∼1%\{\\sim\}1\\%compute\) and nothing else—works only when some later computation actually re\-reads the field\. A reasoning*chain*does exactly that, so with chain\-of\-thought the field\-only edit alone recovers the decision \(1\.001\.00on Qwen3\-8B\); without the chain the*identical*edit on the*same*model is ignored \(0\.000\.00\) and the decision commits to the stale note \([Figure˜2](https://arxiv.org/html/2606.17107#S1.F2)c\)\. The divider is therefore the CoT*mode*, not raw scale—reasoning\-native models default to CoT, instruction\-tuned ones to direct answers—so without CoT one of the two real fixes above is required\. The dependence is two\-level, and it is worth stating precisely: CoT gates*whether*the field\-only edit can work at all \(the binary0\.000\.00/1\.001\.00split above is set by mode, not size\), while*conditional on*CoT the residual*stickiness*—how completely the cheap edit recovers the decision—is model\-dependent \([Figure˜5](https://arxiv.org/html/2606.17107#S4.F5)a; and E3 of[Section˜7](https://arxiv.org/html/2606.17107#S7), where it climbs with scale within the Qwen3 family\)\. These two dependencies—CoT\-gating and per\-model stickiness—are the whole story for the cheap edit, and we keep the rest brief here; the per\-model stickiness, thefield\+selective@KKsweep \(K⋆≈4K^\{\\star\}\\approx 4at88B but\>64\>64at44B, varying non\-monotonically with scale;[Figure˜5](https://arxiv.org/html/2606.17107#S4.F5)a,b\), and the layer\-wise account are detailed in[Appendices˜E](https://arxiv.org/html/2606.17107#A5)and[C](https://arxiv.org/html/2606.17107#A3)\. We reportfield\+selective@KKhonestly as a genuine but*unreliable*surgical tool, not a default\.

### Which edit to use: no single dominant method\.

A controlled comparison \(full table in[Appendix˜E](https://arxiv.org/html/2606.17107#A5)\) shows no single dominant method\. Hoist\-to\-end is cheapest but demands prompt surgery and pre\-identification of every field;field\+erratummatches it with no surgery at a one\-line append;in\_placeis near\-free but only with chain\-of\-thought; a KV\-deviation\-ranked selective recompute \(CacheBlend\-style\[[35](https://arxiv.org/html/2606.17107#bib.bib7)\]\) underperforms here because it chases changed keys rather than the tokens that*memoized the conclusion*\. The practical recommendation isfield\+erratumas the robust default, within\_placeas a free fast\-path under chain\-of\-thought\.

### Why not edit the weights?

A natural objection: to act on a changed field, why not edit the model’s*weights*\(ROME/MEMIT\[[22](https://arxiv.org/html/2606.17107#bib.bib1),[23](https://arxiv.org/html/2606.17107#bib.bib2)\]\) or fine\-tune, rather than the cache? We compare on the paper’s gated task \(statuspending→\\rightarrowshipped, so*cancel*→\\rightarrow*deny*\) against a faithful rank\-one ROME—validated on the canonical factual edit first \(“the Eiffel Tower is in”*Paris*→\\rightarrow*Rome*, locality intact\), so the baseline is not crippled—and a LoRA fine\-tune\[[10](https://arxiv.org/html/2606.17107#bib.bib3)\]\([Table˜1](https://arxiv.org/html/2606.17107#S4.T1); methodology in[Appendix˜I](https://arxiv.org/html/2606.17107#A9)\)\. All offield\+erratum, ROME, and LoRA*succeed*at flipping the target decision\. But a weight edit is*global*: the same model instance can no longer holdstatus=shippedfor one request andpendingfor another, so*all*concurrent orders that are genuinely still pending are wrongly flipped \(cross\-request contamination1\.01\.0\), and half of an unrelated decision battery drifts \(0\.50\.5\)—the ROME/fine\-tune specificity tax—at33–66s per edit \(plus a one\-time covariance pass for ROME\)\. The append\-only erratum lives in a*per\-sequence*cache: zero cross\-request contamination, zero collateral,114114ms, and it composes with prefix caching \([Section˜9](https://arxiv.org/html/2606.17107#S9)\)\. Weight editing targets*durable, global facts*; mutable per\-request, per\-turn state is the wrong job for it—which is precisely the niche the editable KV cache fills\.

Table 1:KV editing vs\. weight editingfor mutable per\-request state \(Llama\-3\.1\-8B\)\. Weight edits succeed at the target yet are global \(contaminate concurrent requests\), damage unrelated decisions, and are3030–50×50\\timesslower per edit\.

## 5 Consequence II: the cache is composable

![Refer to caption](https://arxiv.org/html/2606.17107v1/x6.png)Figure 6:Composing the cache\.\(a\) Pasting a precompiled skill isO​\(L\)O\(L\)vs\. full reprefill’sO​\(L2\)O\(L^\{2\}\); TTFT speedup reaches13\.9×13\.9\\timesat3232k tokens\. \(b\) The transplanted skill matches full recompute in next\-token logits \(cosine0\.900\.90–0\.9990\.999\) across the model family\.### From mechanism to prediction\.

If a region’s notes are localized and re\-derivable from context the decision can still see, then the notes should be*position\-portable*: we can compute a reusable chunk’s KV once, in isolation, move it to a new absolute position, and splice it in\. Concretely we precompile a long*skill*\(a policy or tool specification\), and because the attention library caches post\-RoPE keys, we re\-rotate the chunk’s keys from their source positions to the target positions \(values are position\-free\) before concatenating \([Figure˜4](https://arxiv.org/html/2606.17107#S4.F4)b\)\. This is the position\-aware reuse ofLiuet al\.\[[18](https://arxiv.org/html/2606.17107#bib.bib9)\]; our point is that the*mechanism predicts the splice should match a full reprefill*—the same next\-token logits and decisions\.

### Transplant is behaviorally indistinguishable from full recompute\.

It does\. The spliced skill matches a full reprefill in next\-token logits with cosine0\.900\.90–0\.9990\.999across the full model family \(twelve models; full roster in[Appendix˜A](https://arxiv.org/html/2606.17107#A1)\)—Qwen3\-1\.7B through 32B \(including FP8 and the 30B\-A3B Mixture\-of\-Experts\), Gemma\-2/3, Mistral\-7B, Llama\-3\.1\-8B and 70B, and DeepSeek\-R1\-Distill\-Llama\-8B \([Figure˜6](https://arxiv.org/html/2606.17107#S5.F6)b\)—and on the models competent at the task it preserves*correct*skill\-following across88diverse domains and33families \(24/2424/24, cosine0\.980\.98–0\.9990\.999\), including16/1616/16under chain\-of\-thought\.

### Context\-robustness and the seam\.

A chunk precompiled in isolation matches one that attended to the real preceding context, because the decision re\-derives from context it still sees\. The one residual error is a*seam*at the chunk’s start—the first tokens that, in a full prefill, would have attended to the now\-missing prefix\. Recomputing a few boundary tokens \(*seam\-repair*\) closes it; this is exactly the boundary recompute of CacheBlend/EPIC/MPIC, and[Section˜3](https://arxiv.org/html/2606.17107#S3)explains why it is the boundary, specifically, that needs repair\.

### Linear\-time TTFT\.

Full reprefill of a length\-LLskill isO​\(L2\)O\(L^\{2\}\); transplant isO​\(L\)O\(L\)\(a re\-rotation pass plus the suffix\)\. Time\-to\-first\-token speedups grow with skill length:3×3\\timesat22k tokens,9\.8×9\.8\\timesat88k, and13\.9×13\.9\\timesat3232k on an88B model \([Figure˜6](https://arxiv.org/html/2606.17107#S5.F6)a\)\. A library of skills composes \(decisions preserved forN=1N=1–44concurrent skills\)\.

### Generality: content type, insertion position, and agentic tool\-calling\.

Transplantation is not specific to rule\-like skills\. It preserves decisions for*facts/RAG*passages as well as rules; for chunks inserted in the system area*and*mid\-trajectory as tool results; and—measured with actual function calls rather than a proxy—it preserves*agentic tool\-calling*\(N=108N\{=\}108with bootstrap CIs: function\-call accuracy1\.001\.00on Mistral\-7B, Llama\-3\.1\-8B/70B, and Qwen3\-8B;0\.97​\[0\.94,1\.0\]0\.97\\,\[0\.94,1\.0\]on Qwen3\-32B\-FP8 and 30B\-A3B, whose transplanted\-vs\-full tool\-call*agreement*is1\.001\.00\)\. The one consistent exception is sliding\-window attention \(Gemma\), which we diagnose and fix in[Section˜8](https://arxiv.org/html/2606.17107#S8)\. Full per\-domain scorecards are in[Appendix˜F](https://arxiv.org/html/2606.17107#A6)\.

## 6 Combining edit and compose

![Refer to caption](https://arxiv.org/html/2606.17107v1/x7.png)Figure 7:Edit and compose can be combined\.\(a\) Editing inside a transplant: a field edited*inside*a transplanted skill reproduces the editing mechanism, and the*composed*cache matches the*recomputed*cache for every method \(points on the diagonal\)\. \(b\) A unified edit\+compose agent over thirteen models \(1010domains×\\times1010instances,300300decisions;120120on the two Gemma models\): unified\-vs\-full agreement \(bars\) and cumulative\-TTFT speedup \(markers\)\.### The keystone: editing inside a transplant\.

If editing and composing are truly two operations on the same object, then editing a field that lives*inside a transplanted skill*should behave exactly as editing a field in a normally\-prefilled context\. We test this directly: transplant a skill whose body contains a mutable field, then apply each editing method to that field and measure recovery, comparing the*composed*cache \(skill spliced in\) against a fully*recomputed*cache\. The editing mechanism reproduces verbatim \([Figure˜7](https://arxiv.org/html/2606.17107#S6.F7)a\): the in\-place edit is weak \(≈0\.05\\approx 0\.05\), selective recompute recovers \(sel​@​32≈0\.80\\mathrm\{sel\}@32\\approx 0\.80\), the erratum is strongest, and crucially*composed≈\\approxrecomputed*for every method \(points lie on the diagonal\) across Gemma\-2\-9B and Llama\-3\.1\-8B\. The notes a transplant pastes in are the same notes an edit amends—one notebook\.

### A unified edit\+compose agent\.

We embody both operations in a single live agent loop: a long policy is*composed*once and never re\-prefilled; as the world changes across turns the mutable state is*edited*by appended errata; and each turn reuses the longest cached prefix and prefills only the delta\. Against a reprefill\-every\-turn baseline, over1010agent domains×\\times1010instances \(300300decisions\) per model \(4040instances,120120decisions, on the two Gemma models\) and across thirteen models \(the twelve transplant models of[Figure˜6](https://arxiv.org/html/2606.17107#S5.F6)plus Qwen3\-0\.6B;[Appendix˜A](https://arxiv.org/html/2606.17107#A1)\), the unified path is decision\-identical to full recompute with agreement0\.810\.81–1\.001\.00\(e\.g\. Llama\-3\.1\-8B0\.9630\.963, Mistral\-7B0\.9830\.983, Llama\-3\.1\-70B1\.001\.00\) at cumulative\-TTFT speedups up to14\.9×14\.9\\times\([Figure˜7](https://arxiv.org/html/2606.17107#S6.F7)b\); the speedup scales with policy length×\\timesturns\. Editing and composing therefore run in one loop, decision\-identical to full recompute and at lower cumulative TTFT, across the model family\. The leave\-stale\+\+erratum cache also does*not*compound error over long trajectories: across a2828\-turn stress test where the gating field toggles every turn, the decision logits stay faithful to a full reprefill \(cosine0\.99\+0\.99\{\+\}, flat with trajectory length\) with no systematic drift \([Appendix˜G](https://arxiv.org/html/2606.17107#A7)\)\.

## 7 Application: editable and composable user memory

![Refer to caption](https://arxiv.org/html/2606.17107v1/x8.png)Figure 8:E1 — placement and the pre\-digestion cost\.Decision accuracy vs\. integration depth \(n\_facts\) for memory read*early*\(solid\) vs\.*late*\(dashed\), across all evaluated models, under \(a\) direct answering and \(b\) chain\-of\-thought\. Direct answering is at chance for the reasoning\-native Qwen3 models \(so CoT is the operative regime\); under CoT, late placement forgoes prefill\-time pre\-digestion and carries a small accuracy cost vs\. early that the text quantifies on Qwen3\-4B \(GEE\-logistic, growing with memory length\)\. Late placement is still preferred because it enablesO​\(L\)O\(L\)editing/transplant \(efficiency quantified in E5\)\.A concrete, high\-value instance of this editable, composable notebook is*user memory*: the large, dynamically summarized profile of facts an assistant re\-reads every turn\. Memory is big \(thousands–tens of thousands of tokens\), reused across turns, and*mutated mid\-session*by tool calls—so where it lives in the prompt is a dilemma the mechanism of[Section˜3](https://arxiv.org/html/2606.17107#S3)explains\. Placed at the*front*\(\[sys\]\[MEM\]\[traj\]\), the trajectory memoizes memory\-conditioned conclusions downstream, so a memory change forces a costly downstream reprefill \(and an in\-place edit is ignored\)\. Placed at the*end*\(\[sys\]\[traj\]\[MEM\]\), memory’s KV depends on the whole trajectory and must be re\-attended every turn\. We resolve this by treating memory as a*skill that is also edited*: precompile it once in isolation, place it late \(\[sys\]\[traj\]\[MEM\]\[query\]so the decode reads it directly\), RoPE\-reposition it each turn, repair one boundary token, and edit it in place when it changes\.

### Placement and the pre\-digestion cost \(E1;[Figure˜8](https://arxiv.org/html/2606.17107#S7.F8)\)\.

Pre\-digestion is real, and we quantify it\. Under full recompute and chain\-of\-thought, reading memory*late*carries a small but*statistically significant*accuracy cost vs\.*early*\(Qwen3\-4B,192192personas per cell: GEE\-logisticβlate=−0\.44\\beta\_\{\\text\{late\}\}=\-0\.44,p=0\.018p=0\.018\), and the cost*grows with memory length*: the early−\-late gap is≈0\{\\approx\}0at22k tokens but\+0\.09\{\+\}0\.09at1616k and\+0\.16\{\+\}0\.16at3232k\. The mechanism explains it—late placement forgoes prefill\-time pre\-digestion and relies on the decode/CoT to integrate raw memory\. This is a*tradeoff, not free*: late placement is still preferred because it enablesO​\(L\)O\(L\)editing/transplant at2\.32\.3–4\.3×4\.3\\timeslower TTFT and the transplant is faithful at a fixed placement \(both shown below\), so the net is small accuracy for large efficiency—with early placement preferable when memory is very long and accuracy\-critical\. \(Direct one\-shot decisions are at chance for the reasoning\-native Qwen3 family*to 32B*, breaking only at Llama\-3\.1\-70B, direct0\.810\.81; CoT is the operative regime across scale—see[Appendix˜H](https://arxiv.org/html/2606.17107#A8)\.\)

![Refer to caption](https://arxiv.org/html/2606.17107v1/x9.png)Figure 9:E2 — memory transplant is faithful, to 70B\.\(a\) Decision agreement with full recompute under late placement; a single seam\-repair token closes the start\-of\-chunk gap across ten models\. \(b\) The re\-rotation is necessary—the naive no\-rotate control collapses, while rotated\+\+1 seam token matches full recompute\.
### Memory transplant is faithful, to 70B \(E2;[Figure˜9](https://arxiv.org/html/2606.17107#S7.F9)\)\.

Across ten models from0\.60\.6B to7070B, a precompiled\+repositioned memory chunk reproduces the full\-recompute decision logits with cosine0\.940\.94–0\.99960\.9996; a*single*seam\-repair token closes the start\-of\-chunk boundary gap \(Llama\-3\.1\-8B late: cos0\.94→0\.9940\.94\\rightarrow 0\.994\)\. The cleanest decision\-governance test is Llama\-3\.1\-70B, whose decisions*genuinely vary*\(so agreement is non\-trivial, unlike the constant\-answer regime of smaller models\): the late, seam\-repaired transplant reproduces the full\-recompute decision0\.930\.93of the time at logit cosine0\.9970\.997, and*late placement beats early*\(0\.930\.93vs\.0\.830\.83\) exactly as the mechanism predicts \(the decode reads memory directly rather than through pre\-digested notes across the transplant boundary\)\. The no\-rotation control collapses \(late decision agreement0\.180\.18vs\.0\.780\.78rotated\), confirming the re\-rotation is necessary\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x10.png)Figure 10:E3 — memory is editable mid\-session\.Reusing stale memory recovers a toggled fact essentially never; every real edit recovers it\. Consistent with[Section˜4](https://arxiv.org/html/2606.17107#S4), the near\-free in\-place edit suffices under chain\-of\-thought and*strengthens with scale*, with the append\-only erratum as the robust fallback\.
### Memory is editable mid\-session \(E3;[Figure˜10](https://arxiv.org/html/2606.17107#S7.F10)\)\.

When a stored fact is toggled, reusing the*stale*memory recovers the flipped decision essentially never \(≤0\.03\\leq 0\.03\)\. Every real edit recovers it, and—consistent with[Section˜4](https://arxiv.org/html/2606.17107#S4)—under chain\-of\-thought the*near\-free in\-place edit*\(one token recomputed\) suffices, and*strengthens with scale*: correctness0\.92→0\.99→1\.000\.92\\rightarrow 0\.99\\rightarrow 1\.00across Qwen3\-1\.7B/4B/14B \(and0\.980\.98at 32B\)\. On models where the chain does not re\-read the field, the append\-onlyerratumis the robust fallback \(McNemarerratum\>\>in\_placeon Llama\-3\.1\-8B,p=0\.031p=0\.031\)\. This is the editing axis that concurrent KV\-cache memory systems lack: MemArt\[[38](https://arxiv.org/html/2606.17107#bib.bib31)\]and EPIC\[[11](https://arxiv.org/html/2606.17107#bib.bib8)\]retrieve and splice*static*memory blocks position\-independently but have no in\-place memory*update*; our additions are the editing operation, the decision\-governance lens, and the mechanism \([Section˜3](https://arxiv.org/html/2606.17107#S3)\) that explains why boundary recompute suffices\.

### Editing inside transplanted memory reproduces the mechanism\.

Editing a field*inside a transplanted memory chunk*in direct mode \(Llama\-3\.1\-70B, whose decisions vary so recovery is measurable\) reproduces[Section˜3](https://arxiv.org/html/2606.17107#S3)’s memoization verbatim: refreshing the field’s KV alone recovers only0\.550\.55of the flipped decision \(the conclusion was memoized*downstream*, not in the field\), and recovery climbs monotonically as more downstream tokens are recomputed \(0\.55→0\.840\.55\\rightarrow 0\.84atK=16→0\.94K\{=\}16\\rightarrow 0\.94full\), exactly as for a normally\-prefilled context\. Under chain\-of\-thought this stickiness*dissolves*—a selective@KKsweep is flat at≈0\.98\{\\approx\}0\.98for allKK\(the chain re\-reads the field, soK⋆≈1K^\{\\star\}\{\\approx\}1\)\. Edit and compose thus act on the same notes, and the direct/CoT dissociation matches the editing law of[Section˜4](https://arxiv.org/html/2606.17107#S4)\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x11.png)Figure 11:LoCoMo — external validity on real conversations\.Over all1,5401\{,\}540answerable questions per model, transplanting the multi\-session dialogue memory is statistically equivalent to full recompute in QA accuracy \(TOST\) on the Qwen3 models and within2\.72\.7points on Llama\-3\.1\-8B\.
### Real memory: LoCoMo external validity \([Figure˜11](https://arxiv.org/html/2606.17107#S7.F11)\)\.

The synthetic gated decisions isolate the mechanism; to test real memory we run the long\-conversation QA benchmark LoCoMo\[[21](https://arxiv.org/html/2606.17107#bib.bib32)\]\(MemArt’s setting\): the multi\-session dialogue \(median∼19\.7\{\\sim\}19\.7k tokens\) is the memory, precompiled and spliced before each question\. Over*all*1,5401\{,\}540answerable questions per model, transplant is*statistically equivalent*to full recompute in QA accuracy on Qwen3\-4B, 14B, and 32B \(TOST, margin0\.030\.03:\|Δ\|≤0\.015\|\\Delta\|\\leq 0\.015\) and within a small−2\.7\-2\.7points on Llama\-3\.1\-8B, with answer\-token logit cosine0\.9910\.991–0\.9980\.998throughout\. The*compose*axis thus holds on real conversational memory, not only synthetic decisions\.

### Granularity is a free knob \(E4;[Figure˜24](https://arxiv.org/html/2606.17107#A7.F24)\)\.

Splitting memory intoSSindependently\-precompiled blocks makes a localized editS×S\\timescheaper \(recompute one block\) and is*decision\-lossless*up toS=16S\{=\}16\(agreement flat at1\.001\.00on Qwen3\-4B\), because the independent facts are integrated at read time, not within memory\. Logit cosine degrades withSS\(0\.998→0\.9530\.998\\rightarrow 0\.953\) but*identically whether the gating facts are contiguous in one block or split across blocks*—splitting*independent*relevant facts across independently\-precompiled blocks does not specifically hurt\. Genuinely cross\-*referential*facts, however, do: in a controlled test \([Appendix˜G](https://arxiv.org/html/2606.17107#A7)\) where a decision needs a two\-hop chain \(aDEFINITIONline names which setting gates, whose value lives elsewhere\), splitting the linked pair across a block boundary drops decision agreement with full recompute to0\.460\.46vs\.0\.760\.76when the pair stays in one block—a0\.300\.30penalty \(Llama\-3\.1\-8B,n=80n\{=\}80, full\-recompute accuracy0\.850\.85; McNemarp<10−6p\{<\}10^\{\-6\}\), because blockBB’s isolated precompute never attended to its referentAA\. The same split costs an independent fact pair only0\.040\.04\. Practical guidance: keep cross\-referential facts in one block \([Figure˜24](https://arxiv.org/html/2606.17107#A7.F24)\)\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x12.png)Figure 12:End\-to\-end agent\.\(a\) Per\-decision median TTFT for the proposed compose\+\+edit agent vs\. oracle, front, and end\-placement baselines\. \(b\) Cumulative TTFT speedup of2\.32\.3–4\.3×4\.3\\timesover reprefill\-every\-turn, growing with model size, at faithful next\-token decisions\.
### End\-to\-end agent \([Figure˜12](https://arxiv.org/html/2606.17107#S7.F12)\)\.

We implement a live agent that composes memory once, re\-rotates it each turn, and edits it on tool\-driven changes\. Over1212\-turn sessions \(1616–120120sessions per model\) with≈\\approx2k\-token memories, against a reprefill\-every\-turn\-at\-the\-end baseline it cuts cumulative time\-to\-first\-token by2\.32\.3–4\.3×4\.3\\times\(growing with model size, to 32B\), and against front\-placement reprefill\-on\-change by1\.41\.4–3\.3×3\.3\\times, while reproducing the full\-reprefill next\-token logits at a token\-matched oracle \(cosine0\.970\.97–0\.990\.99\)\. One honest caveat: greedy chains\-of\-thought are sensitive to sub\-percent logit differences, so the exact reasoning*chain*is not always reproduced \(chain agreement0\.310\.31–0\.780\.78\) even though the next\-token decision is faithful and CoT*accuracy*is comparable; the clean decision\-governance equivalence is the short\-context editing result above \([Section˜4](https://arxiv.org/html/2606.17107#S4)\-style, agreement0\.890\.89–0\.950\.95\)\. User memory is thus a second instance of one set of notes that is both editable and composable\.

## 8 Applicability to multimodal and new attention mechanisms

![Refer to caption](https://arxiv.org/html/2606.17107v1/x13.png)Figure 13:Applicability to multimodal and new attention mechanisms\.\(a\) The operations work on any per\-token attention KV representation; we map each attention variant as free / adapter / config fix / partial / open / out\-of\-scope\. \(b\) Image\-KV transplant is near\-lossless across vision\-language models—images are position\-portable too\.### Scale, quantization, MoE\.

The transplant is faithful from0\.60\.6B to3232B, on FP8 checkpoints, on a 30B\-A3B Mixture\-of\-Experts, and on a44\-bit7070B model \(feasibility8/88/8, logit cosine0\.9860\.986\); the unified agent of[Section˜6](https://arxiv.org/html/2606.17107#S6)likewise spans all thirteen\.

### Multimodal: images take notes too\.

In an agent trajectory an image costs a full prefill—the vision tower*plus*prefilling its\>1\{\>\}1k soft\-tokens through the LM\. Because image notes are also position\-portable, we cache an image’s LM\-side KV once and splice it, re\-running only text\. Across120120diverse VQA tasks per model \(perception/reasoning/agentic\), the spliced image is near\-lossless versus full re\-encode—agreement0\.9580\.958–1\.01\.0on Qwen2\.5\-VL\-3B/7B/32B and Qwen3\-VL\-8B \([Figure˜13](https://arxiv.org/html/2606.17107#S8.F13)b\)—and reusing a cached image is2\.42\.4–8\.4×8\.4\\timesfaster TTFT\. Moving an image to a different trajectory position requires re\-rotating only the temporal axis of M\-RoPE; we handle both the*sectioned*\(Qwen2\.5\-VL\) and*interleaved*\(Qwen3\-VL\) layouts, with position\-shifted transplant lossless \(agreement0\.990\.99,Δ=161\\Delta\{=\}161positions\)\. This subsumes MPIC’s multimodal reuse\[[40](https://arxiv.org/html/2606.17107#bib.bib10)\], replacing its boundary\-recompute with exact re\-rotation\.

### The operations work on any per\-token attention KV representation\.

They depend on the cache*representation*, not the attention kernel, so we map exactly where they hold \([Figure˜13](https://arxiv.org/html/2606.17107#S8.F13)a\):

- •Free—throughput optimizations that keep per\-token KV: FlashAttention, paged attention/vLLM, and GQA/MQA \(already used by every model we ran\)\.
- •Adapter \(implemented, validated\)—representation changes \(diagrammed in[Figure˜22](https://arxiv.org/html/2606.17107#A6.F22)\)\.*MLA*\(DeepSeek\-V2/Coder\-V2\) caches a position\-free latent plus a small decoupled\-RoPE sub\-vector; our decoupledk\_pereposition re\-rotates only that sub\-vector, giving logit cosine0\.980\.98and composed\-vs\-full agreement1\.001\.00on DeepSeek\-Coder\-V2\-Lite\.*Interleaved M\-RoPE*\(above\) is the other adapter\.
- •Config fix—sliding\-window \(Gemma\): the default cache truncates sliding layers to the window, breaking uniform splices beyond it; keeping the*full*per\-token KV and letting the attention*mask*enforce the window restores correctness \(the previously\-failing unified agent now runs at agreement0\.930\.93–0\.940\.94;[Figure˜22](https://arxiv.org/html/2606.17107#A6.F22)c\)\.
- •Partial—hybrids with full\-attention layers \(Falcon\-H1\): the attention KV is transplantable but the per\-layer Mamba scan\-state is recurrent, not per\-token, so a correct transplant must re\-scan the Mamba path and saves only the attention fraction\.
- •Open frontier—sequence\-dimension KV compression \(DeepSeek\-V4’s CSA/HCA\[[5](https://arxiv.org/html/2606.17107#bib.bib40)\]\) merges tokens into sub\-token\-count entries, so edit/splice become block\-granular; DeepSeek\-V3\.2’s sparse attention \(DSA\[[4](https://arxiv.org/html/2606.17107#bib.bib39)\]\) is MLA plus top\-kkselection and inherits our MLA adapter\.
- •Out of scope—no per\-token attention KV: pure\-recurrent \(RWKV\[[24](https://arxiv.org/html/2606.17107#bib.bib34)\]\), pure\-SSM \(Mamba\[[9](https://arxiv.org/html/2606.17107#bib.bib33)\]\), and diffusion LMs\. The prompt\-level erratum still applies there, but it is not a KV operation\.

## 9 Systems payoff

![Refer to caption](https://arxiv.org/html/2606.17107v1/x14.png)Figure 14:Systems payoff\.\(a\) Online vLLM serving \(V1 engine, CUDA graphs, continuous batching, APC, Poisson arrivals\): the append\-only erratum keeps the prefix cache\-aligned \(98\.5%98\.5\\%vs\.1%1\\%APC hit\-rate\), so its throughput advantage grows with offered load—up to14\.5×14\.5\\timesat saturation—whilep​90p90TTFT is5353–398×398\\timeslower\. \(b\) Reusing a cached image \(skipping the vision tower and image\-token prefill\) accelerates time\-to\-first\-token, more so for larger images\.### A real agentic environment\.

On theτ2\\tau^\{2\}\-bench retail environment\[[1](https://arxiv.org/html/2606.17107#bib.bib24)\]—single tool\-decisions and a multi\-turn autonomous\-agent loop scored by the environment’s own tool enforcement—an agent that reuses a stale cache after a state change fails the task—it acts on the outdated field value—whilefield\+erratumpreserves task success at a fraction of the recompute cost\. On the real∼1\.4\{\\sim\}1\.4k\-token retail policy, transplant reproduces the clean decision \(composed==full on Llama\-3\.1 and Mistral\); the one hard case, a long buried field that must*flip*a conclusion, requires the robustfield\+erratumedit rather than transplant\-plus\-bare\- erratum—the same long\-context lesson the editing axis predicts, now in a real environment\.

### A comprehensive online serving benchmark\.

Because the erratum is append\-only, it composes with standard automatic prefix caching \(APC\): the static prefix stays cache\-aligned and only the short erratum \(and the decode\) is new work, whereas writing the new value*into*the prefix changes a cached block’s content hash and invalidates every downstream block\. We test this on vLLM’s V1 engine as a real online server—AsyncLLMEnginewith CUDA graphs, continuous batching, APC, and*Poisson*request arrivals at controlled offered load \(not an offline batch\)—over a shared∼8\{\\sim\}8k\-token agent policy with one mutable field, measuring TTFT percentiles, throughput, and the engine’s own APC hit\-rate \([Figure˜14](https://arxiv.org/html/2606.17107#S9.F14)a; full configuration in[Appendix˜I](https://arxiv.org/html/2606.17107#A9)\)\. The append\-only edit keeps the prefix a cache hit \(98\.5%98\.5\\%vs\.1\.0%1\.0\\%APC hit\-rate\); the in\-prefix baseline is prefill\-bound and*saturates at≈1\.5\{\\approx\}1\.5req/s*, so itsp​90p90time\-to\-first\-token collapses under load \(2222–5555s\) while the erratum stays8686ms–11s \(5353–398×398\\timeslower TTFT\)\. The throughput advantage*grows with offered load*—1\.6×1\.6\\timesat22req/s up to14\.5×\\mathbf\{14\.5\\times\}at saturation—exactly as predicted for a compute\-bound vs\. cache\-bound regime\. The image\-KV reuse of[Section˜8](https://arxiv.org/html/2606.17107#S8)contributes a complementary serving win—2\.42\.4–8\.4×8\.4\\timesfaster first\-token as image size grows \([Figure˜14](https://arxiv.org/html/2606.17107#S9.F14)b\)—by skipping the vision tower and image\-token prefill entirely\.

## 10 Limitations

Our operations require a per\-token attention KV cache, so pure\-recurrent, pure\-SSM, and diffusion models are out of scope, and hybrid attention\+SSM models are only partially served \(the recurrent state is not transplantable;[Section˜8](https://arxiv.org/html/2606.17107#S8)\)\. The MLA and sliding\-window adapters are validated but carry residual edge cases: a single transplanted*chunk that itself exceeds the sliding window*drops to logit cosine≈0\.89\\approx 0\.89\(real skills are far smaller\), and our small MLA checkpoints ship legacy\-cache custom modeling that we shim\. Sequence\-dimension KV compression \(DeepSeek\-V4\-class\[[5](https://arxiv.org/html/2606.17107#bib.bib40)\]\) and cross\-attention image caches are open: the unit of edit/transplant there becomes a compressed block rather than a token, which we have analyzed but not implemented\. Thefield\+selective@KKsurgical edit is unreliable \(model\- and domain\-dependent stickiness\) and we present it as such, not as a default\. Finally, while several studies use synthetic policies for controlled measurement, we mitigate this with the realτ2\\tau^\{2\}\-bench retail policy and real images; broader real\-workload evaluation remains future work\. More broadly, our operations exploit a mechanism that arises*for free*in today’s models; the direction this work points to is a KV cache*programmable*by design—models trained to expose composable, editable notes—which we leave to future work\.

## 11 Conclusion

A surgical edit to a field’s KV is ignored not because the cache is fragile but because the model has already done the work: at prefill it computes the field\-conditioned*conclusion*and writes it onto downstream aggregator tokens, so the decision only reads those notes back\. This reframes what a KV cache is—not a write\-once byproduct of prefill, but a structured record of intermediate conclusions that we can read, amend, and rearrange\. Two capabilities follow from the one mechanism:*editing*the notes \(a late, salient erratum in place of recomputation\) and*composing*them \(repositioning and splicing a precompiled skill inO​\(L\)O\(L\)time\)\.

The question this opens is larger than either operation\. If prefill routinely deposits reusable conclusions into the cache, then much of what a model has worked out mid\-context is already written down somewhere we can inspect and overwrite\. Editing and composition are two uses of that record, and they work today even though no model was trained for them\. The larger opportunity is to make this explicit: models trained to be*aware*that their KV can be composed and edited—exposing notes that are cleanly addressable, splice\-able, and revisable by design—would turn the cache from a linear, append\-only log into a*programmable*memory the system reads, writes, and rearranges\. Once the KV cache is programmable rather than merely linear, context engineering changes shape significantly:*skills, memory, and state become first\-class, reusable cache objects*rather than text re\-prefilled on every turn\. This paper is a first step toward the vision of programmable KV cache—a notebook the model keeps for itself, and one it can learn to read and edit\.

## Acknowledgements

We thank Junhao Hu, first author of EPIC \(Efficient Position\-Independent Caching\)\[[11](https://arxiv.org/html/2606.17107#bib.bib8)\]and a co\-author of CacheSlide\[[18](https://arxiv.org/html/2606.17107#bib.bib9)\], for discussions on editable KV cache that inspired this research project\. We thank BSQL Networking for hosting the RTX PRO 6000 GPU\. AI tools including Pine Copilot, Claude Code with Claude Fable 5 and Opus 4\.8 were used during this research\.

## References

- \[1\]V\. Barres, H\. Dong, S\. Ray, X\. Si, and K\. Narasimhan\(2025\)τ2\\tau^\{2\}\-Bench: evaluating conversational agents in a dual\-control environment\.arXiv preprint arXiv:2506\.07982\.Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px5.p1.2),[§9](https://arxiv.org/html/2606.17107#S9.SS0.SSS0.Px1.p1.3)\.
- \[2\]DeepSeek\-AI\(2024\)DeepSeek\-V2: a strong, economical, and efficient mixture\-of\-experts language model\.arXiv preprint arXiv:2405\.04434\.Cited by:[Appendix A](https://arxiv.org/html/2606.17107#A1.p1.1),[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[3\]DeepSeek\-AI\(2024\)DeepSeek\-V3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[4\]DeepSeek\-AI\(2025\)DeepSeek\-V3\.2: pushing the frontier of open large language models\.arXiv preprint arXiv:2512\.02556\.Cited by:[5th item](https://arxiv.org/html/2606.17107#S8.I1.i5.p1.1)\.
- \[5\]DeepSeek\-AI\(2026\)DeepSeek\-V4 technical report\.Note:[https://huggingface\.co/deepseek\-ai/DeepSeek\-V4\-Pro/blob/main/DeepSeek\_V4\.pdf](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf)Cited by:[§10](https://arxiv.org/html/2606.17107#S10.p1.3),[5th item](https://arxiv.org/html/2606.17107#S8.I1.i5.p1.1)\.
- \[6\]Gemma Team\(2024\)Gemma 2: improving open language models at a practical size\.arXiv preprint arXiv:2408\.00118\.Cited by:[Appendix A](https://arxiv.org/html/2606.17107#A1.p1.1)\.
- \[7\]I\. Gim, G\. Chen, S\. Lee, N\. Sarda, A\. Khandelwal, and L\. Zhong\(2024\)Prompt cache: modular attention reuse for low\-latency inference\.InProceedings of Machine Learning and Systems \(MLSys\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px2.p1.1)\.
- \[8\]A\. Grattafioriet al\.\(2024\)The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[Appendix A](https://arxiv.org/html/2606.17107#A1.p1.1)\.
- \[9\]A\. Gu and T\. Dao\(2024\)Mamba: linear\-time sequence modeling with selective state spaces\.InConference on Language Modeling \(COLM\),Cited by:[6th item](https://arxiv.org/html/2606.17107#S8.I1.i6.p1.1)\.
- \[10\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§4](https://arxiv.org/html/2606.17107#S4.SS0.SSS0.Px5.p1.8)\.
- \[11\]J\. Hu, W\. Huang, W\. Wang, H\. Wang, T\. Hu, Q\. Zhang, H\. Feng, X\. Chen, Y\. Shan, and T\. Xie\(2025\)EPIC: efficient position\-independent caching for serving large language models\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px2.p1.1),[§7](https://arxiv.org/html/2606.17107#S7.SS0.SSS0.Px3.p1.5),[Acknowledgements](https://arxiv.org/html/2606.17107#Sx1.p1.1)\.
- \[12\]G\. Ilharco, M\. T\. Ribeiro, M\. Wortsman, L\. Schmidt,et al\.\(2023\)Editing models with task arithmetic\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px4.p1.1)\.
- \[13\]A\. Q\. Jianget al\.\(2023\)Mistral 7B\.arXiv preprint arXiv:2310\.06825\.Cited by:[Appendix A](https://arxiv.org/html/2606.17107#A1.p1.1)\.
- \[14\]C\. Jin, Z\. Zhang, X\. Jiang, F\. Liu,et al\.\(2024\)RAGCache: efficient knowledge caching for retrieval\-augmented generation\.arXiv preprint arXiv:2404\.12457\.Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[15\]K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg\(2023\)Inference\-time intervention: eliciting truthful answers from a language model\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px4.p1.1)\.
- \[16\]Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh,et al\.\(2024\)SnapKV: LLM knows what you are looking for before generation\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[17\]J\. Lindseyet al\.\(2025\)On the biology of a large language model\.Transformer Circuits Thread, Anthropic\.Cited by:[Appendix D](https://arxiv.org/html/2606.17107#A4.SS0.SSS0.Px6.p1.2),[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2606.17107#S3.SS0.SSS0.Px6.p1.1)\.
- \[18\]Y\. Liu, Y\. Gu, L\. Zhang, C\. Wu, G\. Xue, J\. Li, M\. Guo, J\. Hu, and J\. Meng\(2026\)CacheSlide: unlocking cross position\-aware KV cache reuse for accelerating LLM serving\.InProceedings of the 24th USENIX Conference on File and Storage Technologies \(FAST\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2606.17107#S5.SS0.SSS0.Px1.p1.1),[Acknowledgements](https://arxiv.org/html/2606.17107#Sx1.p1.1)\.
- \[19\]Y\. Liu, H\. Li, Y\. Cheng, S\. Ray,et al\.\(2024\)CacheGen: KV cache compression and streaming for fast large language model serving\.InProceedings of ACM SIGCOMM,Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[20\]Z\. Liu, A\. Desai, F\. Liao, W\. Wang,et al\.\(2023\)Scissorhands: exploiting the persistence of importance hypothesis for LLM KV cache compression at test time\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[21\]A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang\(2024\)Evaluating very long\-term conversational memory of LLM agents\.InProceedings of the Association for Computational Linguistics \(ACL\),Cited by:[§7](https://arxiv.org/html/2606.17107#S7.SS0.SSS0.Px5.p1.7)\.
- \[22\]K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov\(2022\)Locating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2606.17107#S4.SS0.SSS0.Px5.p1.8)\.
- \[23\]K\. Meng, A\. S\. Sharma, A\. Andonian, Y\. Belinkov, and D\. Bau\(2023\)Mass\-editing memory in a transformer\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2606.17107#S4.SS0.SSS0.Px5.p1.8)\.
- \[24\]B\. Peng, E\. Alcaide, Q\. Anthony, A\. Albalak,et al\.\(2023\)RWKV: reinventing RNNs for the transformer era\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Cited by:[6th item](https://arxiv.org/html/2606.17107#S8.I1.i6.p1.1)\.
- \[25\]B\. Peng, J\. Quesnelle, H\. Fan, and E\. Shippole\(2024\)YaRN: efficient context window extension of large language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[26\]Qwen Team\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[Appendix A](https://arxiv.org/html/2606.17107#A1.p1.1)\.
- \[27\]T\. Schick, J\. Dwivedi\-Yu, R\. Dessì,et al\.\(2023\)Toolformer: language models can teach themselves to use tools\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px5.p1.2)\.
- \[28\]J\. Su, M\. Ahmed, Y\. Lu, S\. Pan, W\. Bo, and Y\. Liu\(2024\)RoFormer: enhanced transformer with rotary position embedding\.Neurocomputing568\.Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[29\]J\. Tang, Y\. Zhao, K\. Zhu, G\. Xiao, B\. Kasikci, and S\. Han\(2024\)Quest: query\-aware sparsity for efficient long\-context LLM inference\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[30\]E\. Todd, M\. Li, A\. S\. Sharma, A\. Mueller, B\. C\. Wallace, and D\. Bau\(2024\)Function vectors in large language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px4.p1.1)\.
- \[31\]J\. Vig, S\. Gehrmann, Y\. Belinkov,et al\.\(2020\)Investigating gender bias in language models using causal mediation analysis\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px4.p1.1)\.
- \[32\]K\. Wang, A\. Variengien, A\. Conmy, B\. Shlegeris, and J\. Steinhardt\(2023\)Interpretability in the wild: a circuit for indirect object identification in GPT\-2 small\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[Appendix D](https://arxiv.org/html/2606.17107#A4.SS0.SSS0.Px6.p1.2),[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px1.p1.1)\.
- \[33\]G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. Lewis\(2024\)Efficient streaming language models with attention sinks\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[34\]J\. Yanget al\.\(2025\)KVLink: accelerating large language models via efficient KV cache reuse\.arXiv preprint arXiv:2502\.16002\.Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px2.p1.1)\.
- \[35\]J\. Yao, H\. Li, Y\. Liu, S\. Ray, Y\. Cheng, Q\. Zhang, K\. Du, S\. Lu, and J\. Jiang\(2025\)CacheBlend: fast large language model serving for RAG with cached knowledge fusion\.InProceedings of the European Conference on Computer Systems \(EuroSys\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2606.17107#S4.SS0.SSS0.Px4.p1.1)\.
- \[36\]S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhan\(2024\)τ\\tau\-Bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.arXiv preprint arXiv:2406\.12045\.Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px5.p1.2)\.
- \[37\]S\. Yao, J\. Zhao, D\. Yu, N\. Du,et al\.\(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px5.p1.2)\.
- \[38\]Y\. Zeng, P\. Zuo, M\. Lyu, X\. Yang, H\. Wu, Y\. Xu, and Z\. Yu\(2025\)KVCache\-centric memory for LLM agents\.Note:Submitted to ICLR 2026; OpenReview, 18 September 2025Cited by:[§7](https://arxiv.org/html/2606.17107#S7.SS0.SSS0.Px3.p1.5)\.
- \[39\]Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng,et al\.\(2023\)H2O: heavy\-hitter oracle for efficient generative inference of large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px3.p1.1)\.
- \[40\]S\. Zhao, J\. Hu, R\. Huang, J\. Zheng, and G\. Chen\(2025\)MPIC: position\-independent multimodal context caching system for efficient MLLM serving\.arXiv preprint arXiv:2502\.01960\.Cited by:[§2](https://arxiv.org/html/2606.17107#S2.SS0.SSS0.Px2.p1.1),[§8](https://arxiv.org/html/2606.17107#S8.SS0.SSS0.Px2.p1.8)\.

## Appendix AModels evaluated

[Table˜2](https://arxiv.org/html/2606.17107#A1.T2)lists the models used across the paper, spanning the families Qwen3\[[26](https://arxiv.org/html/2606.17107#bib.bib38)\], Llama\-3\.1\[[8](https://arxiv.org/html/2606.17107#bib.bib35)\], Gemma\-2/3\[[6](https://arxiv.org/html/2606.17107#bib.bib36)\], Mistral\-7B\[[13](https://arxiv.org/html/2606.17107#bib.bib37)\], and DeepSeek\-V2/Coder\-V2\[[2](https://arxiv.org/html/2606.17107#bib.bib29)\]\. All runs are on a single RTX PRO 6000 \(Blackwell, 96 GB\); FP8 and 4\-bit checkpoints are the official quantized releases\.

Table 2:Model zoo\. “role” indicates the experiments a model appears in\.
## Appendix BWorked examples: prompts, errata, and responses

### The gated decision \(the paper’s running example\)\.

The mechanism and editing experiments use prompts of the following shape \(scenarioaccount\_role, the verbatim template from the released harness; the thirty neutral filler rules are abridged and long lines re\-wrapped for display\)\. The prompt is wrapped in each model’s chat template and the decision is read at the final position:

```
You are a customer-support agent for an online retailer.

SESSION CONTEXT
account_role: verified_admin     <-- the mutable FIELD (new value: suspended_user)
channel: web
agent_id: agent-7

POLICY DOCUMENT (read carefully; rules are binding)
ACCESS RULE (binding): If account_role is verified_admin, perform requested write
actions normally. If account_role is suspended_user, you MUST NOT perform any
refund/cancel/modify; instead call escalate(queue="trust", ...) and nothing else.
R1. Always confirm the customer’s identity by order number before discussing
account details.
[... R2-R30: twenty-nine more neutral filler rules ...]

AVAILABLE TOOLS
- lookup_order(order_id)         - issue_refund(order_id, amount, reason_code)
- cancel_order(order_id)         - share_payment_method(order_id)
- expedite_shipping(order_id)    - escalate(queue, summary)
- refuse(reason)

CONVERSATION SO FAR
user: Please refund 40.00 to my order A4471 for the shipping delay.
assistant: Let me check the account and policy before acting.

TASK
Decide the single next tool call. Respond with exactly one line:
tool_call: <name>(<args>)
Next action:
```

The four cache states of[Section˜3](https://arxiv.org/html/2606.17107#S3)are built from this prompt:*stale*prefills the old value,*oracle*the new,*field\-only*overwrites the KV of theaccount\_rolevalue span \(two tokens of a∼958\{\\sim\}958\-token templated prompt,0\.2%0\.2\\%recompute\), and*full\-downstream*recomputes everything after it\. In the position\-resolved causal patch \([Figure˜3](https://arxiv.org/html/2606.17107#S3.F3)b,c\), the highest\-effect downstream positions for a representative instance sit not on the field span but on aggregator/delimiter tokens—the end\-of\-rule punctuation \(around “\.\.\.nothing else\.”\) and the TASK\-block instruction tokens \(around “tool\_call: <”\) that the decision attends through\.

### The erratum, verbatim\.

The editing fix appends one salient line ahead of the decision prompt \(thefield\+erratumarm of[Section˜4](https://arxiv.org/html/2606.17107#S4)\):

```
[STATE UPDATE] account_role has just changed to suspended_user; this overrides
any earlier value. Apply the current value.
```

The wording ablation of[Figure˜3](https://arxiv.org/html/2606.17107#S3.F3)d varies this line from a bare restatement of the value to the aggressive “disregard your earlier conclusion and re\-evaluate”, which underperforms\.

### Recorded responses \(Qwen3\-8B, reasoning mode; one released instance\)\.

From the recordthinking\_qwen3\_8b\_think\.json\(inresults/\), abridged to the answer head:

```
oracle (clean prefill of suspended_user), 249 thinking tokens:
  tool_call: escalate(queue="trust", summary="Request to refund $40.00...")
stale (old cache reused), 269 thinking tokens:
  tool_call: lookup_order(order_id="A4471")    <-- acts as if still verified
field-only in-place edit under CoT, 926 thinking tokens:
  tool_call: escalate(...)                     <-- chain re-reads the field
```

The in\-place edit recovers the oracle decision under reasoning, but the recomputation has moved from prefill into the chain \(926926vs\.249249thinking tokens,∼3\.7×\{\\sim\}3\.7\\times\); without reasoning, the same edit is simply ignored \([Figure˜3](https://arxiv.org/html/2606.17107#S3.F3)a\)\.

### The dissociation pair \([Appendix˜C](https://arxiv.org/html/2606.17107#A3)\)\.

The conclusion/content control holds the field byte\-identical and flips one trigger token inside the rule:

```
ACCESS RULE (binding): If account_role is {trigger}, you MUST NOT perform any
refund, cancel, or modify action and must instead call escalate(queue="trust",
summary=...). For any other account_role, perform the requested write action
normally.
```

With the field fixed atverified\_admin, settingtrigger=verified\_adminmakes escalation correct, whiletrigger=suspended\_user\(field unchanged\) makes the write action correct: the two prompts differ in exactly one token, the field is constant, and only the*conclusion*flips\.

### A transplanted skill \([Section˜5](https://arxiv.org/html/2606.17107#S5)\)\.

A representative precompiled skill \(the2020neutral guidelines abridged\):

```
# SKILL: REFUND_POLICY
You handle refund requests. Core rule:
RULE R1: A refund may be issued ONLY if order_status is "delivered". For any
other status (pending, shipped, cancelled, returned) you MUST refuse the refund
and escalate to a human.
- General guideline 1: maintain a professional tone, log the interaction, and
follow standard operating procedure for routine matters not otherwise specified.
[... general guidelines 2-20 ...]
End of REFUND_POLICY skill.
```

The skill is prefilled once in isolation \(positions0\.\.L−10\.\.L\{\-\}1\), its keys re\-rotated to the target offset, and spliced after the system prompt; the task suffix

```
Order #7731 has order_status = "pending". The customer requests a refund. Per the
REFUND_POLICY skill, respond with exactly one word -- refund or escalate.
Decision:
```

then reads the spliced notes, and the decision \(escalate\) matches a full reprefill \([Figure˜6](https://arxiv.org/html/2606.17107#S5.F6)\)\.

## Appendix CDeep mechanism controls: dissociation, timing, specificity, writability

These four controls \(and an off\-template generalization\) tighten[Section˜3](https://arxiv.org/html/2606.17107#S3)from*decodability*to*causation, timing, and write\-access*\. Each runs on three models \(Qwen3\-8B, Qwen3\-4B, Llama\-3\.1\-8B\); scriptsesys/mechd\_\*\.py, recordsresults/mechd\_\*\.

### \(i\) Dissociation \([Table˜4](https://arxiv.org/html/2606.17107#A3.T4), left\)\.

A polarity\-parameterized rule names a single*trigger*value that selects the safe action, so flipping the trigger inverts the conclusion while the field value is byte\-identical across the pair \(the two prompts differ in exactly one token, inside the rule\)\. Transplanting the post\-trigger*notes*from the opposite\-conclusion cache carries essentially the entire flipped conclusion, whereas patching the differing rule token carries none—with the field held constant, the decision is reading a memoized conclusion, not field content\. A logit\-probe finds both the conclusion and the field identity linearly decodable from the same downstream delimiter, so decodability cannot itself adjudicate; the causal transplant is required\.

### \(ii\) Timing\.

Usingoutput\_hidden\_stateson the prefill, the conclusion becomes linearly decodable \(group\-CV logistic probe, conclusion⟂\\perpfield by the2×22\{\\times\}2design\) on the downstream aggregator at layer\-depth0\.39/0\.39/0\.310\.39/0\.39/0\.31\(Qwen3\-8B/4B, Llama\-3\.1\-8B\), while the decision token’s logit\-lens margin reaches its final sign only at depth0\.75/0\.77/0\.730\.75/0\.77/0\.73—the note is written∼12\{\\sim\}12layers before it is read, within one prefill\.

### \(iii\) Specificity \([Table˜4](https://arxiv.org/html/2606.17107#A3.T4), right\)\.

Ranking downstream positions by individual transplant effect, the top\-88recover0\.740\.74–0\.790\.79of the decision;88random downstream positions recover≤0\.035\{\\leq\}0\.035\. The conclusion sits on a few specific aggregator/delimiter tokens, not diffusely\.

### \(iv\) Writability\.

Overwriting an otherwise\-consistent cache’s downstream notes with the*opposite*conclusion’s notes \(the field token and prefix left intact and still implying the original answer\) drives the decision to the injected conclusion: continuous recovery0\.99/0\.98/1\.020\.99/0\.98/1\.02, with the top\-88note positions already flipping the belief in most instances\. Editing \([Section˜4](https://arxiv.org/html/2606.17107#S4)\) is the benign use of this same write\-access\.

### Off\-template generalization\.

Re\-running field\-only vs\. full\-downstream recovery on \(a\) a 2\-hop lookup, \(b\) free\-form conversational phrasing, and \(c\) attribute lookup: field\-only recovery is≈0\{\\approx\}0for \(a\) and \(b\) across models \(multi\-hop−0\.012\-0\.012to\+0\.006\+0\.006; natural−0\.012\-0\.012to\+0\.054\+0\.054\) with full\-downstream1\.001\.00, confirming the mechanism is not a template artifact; for near\-verbatim attribute lookup \(c\) the field is partly a copy and carries0\.250\.25–0\.630\.63, bounding the claim to*derived*conclusions\.

### Cross\-family replication \(Gemma\-2, Mistral\)\.

To rule out a Qwen3/Llama tokenizer artifact we re\-ran five probes \(the locality probe and the four controls above\) onGemma\-2\-9BandMistral\-7Bwith a tokenizer\-robust readout \(space\-prefixed single\-tokencancel/deny; Gemma\-2’s soft\-capping kept intact, as the attention\-knockout hook used only by the original circuit\-knockout probe otherwise corrupts it\), on the gated cancel/deny task \(n=18n\{=\}18primary,1818dissociation each;[Table˜3](https://arxiv.org/html/2606.17107#A3.T3)\)\. All five replicate: field\-only≈0\\approx 0vs\. full\-downstream1\.01\.0; dissociation trigger\-only≈0\\approx 0vs\. notes≈1\.0\\approx 1\.0; top\-8≫8\\ggrandom\-88; injection≈1\.0\\approx 1\.0; and write\-before\-read timing\. Two honest notes: Mistral’s field\-only recovery \(0\.1370\.137\) is slightly above zero \(still far below full\-downstream1\.01\.0\), and random\-88specificity runs higher on this task \(0\.480\.48–0\.530\.53vs\.≈0\.02\{\\approx\}0\.02on the Qwen/Llama tool\-call task\) though top\-88\(≈0\.95\\approx 0\.95\) still dominates\.

Table 3:Cross\-family replication of the five deep\-mechanism probes \(Gemma\-2\-9B, Mistral\-7B\)\. Recovery toward the target conclusion; bootstrap means\.Table 4:Deep mechanism controls \(recovery toward the target conclusion; three models\)\. Left: dissociation—patching the differing rule token vs\. the downstream notes, field held identical\. Right: specificity—top\-kkvs\. random\-kkdownstream positions, matched count\.

## Appendix DA component\-level circuit for memoized inference

[Section˜3](https://arxiv.org/html/2606.17107#S3)localizes the memoized conclusion \(which tokens, which layers, a linear probe\)\. This appendix pushes from*localization*to a*component\-level*account—which attention heads, which direction, attention vs\. MLP—with five interventions on the polarity 2×\\times2 task, where the field is held*byte\-identical*across the conclusion flip \(only one rule\-trigger token differs\), so every “conclusion” signal we attribute cannot be field content\. Primary model Llama\-3\.1\-8B, replicated across*four families*—Qwen3\-8B, Gemma\-2\-9B, Mistral\-7B \([Table˜5](https://arxiv.org/html/2606.17107#A4.T5)\)—all numbers are causal\-intervention recoveries with bootstrap CIs overn=12n\{=\}12instances \([Figure˜15](https://arxiv.org/html/2606.17107#A4.F15); scriptsesys/circ\_\*\.py, recordsresults/circ\_\*\)\.

### \(1\) Named read and write heads \([Figure˜15](https://arxiv.org/html/2606.17107#A4.F15)a,f\)\.

We rank heads by direct attribution and confirm causally by patching a single head’s output \(clean↔\\leftrightarrowcorrupt\) and re\-reading the decision\.*Read heads*—at the decision token—form a concentrated, nameable set: patching the topkkjointly recovers0\.19/0\.45/0\.59/0\.70/0\.780\.19/0\.45/0\.59/0\.70/0\.78of the decision atk=1/3/5/8/12k\{=\}1/3/5/8/12on Llama\-3\.1\-8B, and the top\-1212reach0\.77/0\.72/0\.820\.77/0\.72/0\.82on Qwen3\-8B/Gemma\-2\-9B/Mistral\-7B \(random\-head control≈0\\approx 0on all four\)\. The strongest read heads sit in late layers and attend decision→\\toaggregator \(Llama26\.3at0\.460\.46, Qwen26\.25at0\.430\.43, Gemma26\.9at0\.470\.47, Mistral20\.21at0\.310\.31\)\.*Write heads*—at the aggregator, at prefill—are more distributed: single\-aggregator patching of the top heads saturates at0\.28/0\.08/0\.24/0\.110\.28/0\.08/0\.24/0\.11\(Llama/Qwen/Gemma/Mistral\), because the write is spread over*many*aggregator tokens \(the suffix\-concentration of[Section˜3](https://arxiv.org/html/2606.17107#S3)\), so patching one position under\-counts it\. The decision\-side read is thus a tight bottleneck; the write is diffuse\.

### \(2\) A causal conclusion direction \([Figure˜15](https://arxiv.org/html/2606.17107#A4.F15)b\)\.

A single difference\-of\-means directiond^\\hat\{d\}on the aggregator residual \(fit*leave\-one\-scenario\-out*, field\-balanced\) transfers the conclusion: injecting thed^\\hat\{d\}\-component of the clean−\-corrupt residual into a corrupt run recovers0\.220\.22–0\.230\.23of the decision at Llama L12–14—≈39%\\approx 39\\%of the full single\-site residual patch and∼25×\{\\sim\}25\{\\times\}a random 1\-D direction \(0\.010\.01\); the difference\-of\-means direction beats a logistic\-probe direction \(0\.020\.02\), a known DAS phenomenon\. So the conclusion has a real shared linear component, but it is*redundantly*coded \(projectingd^\\hat\{d\}out of a clean run drops the decision only∼0\.17\{\\sim\}0\.17\)\. The direction is causal but similarly redundant on Gemma\-2\-9B \(along0\.100\.10vs\. random0\.000\.00, of full0\.610\.61\) and Mistral\-7B \(along0\.070\.07vs\. random0\.000\.00, of full0\.360\.36\)\. On Qwen3\-8B the conclusion is committed so strongly \(logit gap≈22\{\\approx\}22vs\.≈4\{\\approx\}4\) that single\-site directional recovery is near zero in relative terms—the read\-side circuit, which acts at the decision bottleneck, is the robust cross\-family result\.

### \(3\) A sparse SAE feature: decode≠\\neqcause \([Figure˜15](https://arxiv.org/html/2606.17107#A4.F15)e\)\.

We train a TopK SAE \(1616k dict,k=32k\{=\}32, FVU≈0\.001\{\\approx\}0\.001\) on layer\-1414residuals over diverse prompts\. The conclusion is*sparsely decodable*: two features reach AUC=1\.00=1\.00at separating SAFE from UNSAFE on the aggregator \(field\-controlled\), with∼5\{\\sim\}5features above0\.950\.95\. Yet it is*causally distributed*: clamping the single best feature to its SAFE level in a UNSAFE run recovers≈0\\approx 0, while clamping the top3/10/303/10/30jointly recovers0\.30/0\.50/0\.540\.30/0\.50/0\.54\(random\-feature control≈0\\approx 0; necessity0\.300\.30–0\.330\.33\)\. A feature can read the conclusion out perfectly without being causally sufficient alone—the feature\-level echo of the decodability\-vs\-causation dissociation in[Section˜3](https://arxiv.org/html/2606.17107#S3)\.

### \(4\) Causal scrubbing of write→\\tonote→\\toread \([Figure˜15](https://arxiv.org/html/2606.17107#A4.F15)d\)\.

We test the hypothesis “the decision is a function of the conclusion carried by the aggregator note” by resampling activations from inputs that agree with that labelling \(position\-aligned KV transplants from same\-scenario donors\)\.*Faithfulness*: resampling the entire downstream from a*same\-conclusion*different\-input donor preserves the decision \(drift0\.020\.02–0\.040\.04of the gap; decision\-logit cosine0\.9990\.999–1\.0001\.000\) on both models\.*Necessity*: resampling only the*note*from an*opposite\-conclusion*donor flips the decision0\.79/0\.68/0\.82/0\.710\.79/0\.68/0\.82/0\.71\(Llama/Qwen/Gemma/Mistral\), whereas resampling the rest of the downstream moves it only0\.19/0\.32/0\.18/0\.300\.19/0\.32/0\.18/0\.30\. The note governs; everything else is interchangeable\.

### \(5\) Attention writes the note \([Figure˜15](https://arxiv.org/html/2606.17107#A4.F15)c\)\.

Decomposing the SAFE−\-UNSAFE aggregator residual into per\-layer attention\- and MLP\-block contributions \(exact: the embedding is identical across the pair\), attention contributes the majority of the write at the causal conclusion layer—0\.62/0\.600\.62/0\.60at Llama L12/L14, and a striking0\.820\.82–0\.920\.92on Gemma\-2\-9B—with Qwen3\-8B \(0\.440\.44–0\.560\.56\) and Mistral\-7B \(0\.420\.42–0\.660\.66\) closer to parity\. Attention is the primary or co\-primary writer that routes the field/rule information onto the aggregator in every family, with the MLPs contributing the remainder\.

### Summary\.

The picture is a*distributed write, concentrated read*: at mid layers, attention routes the field\-conditioned conclusion onto aggregator tokens \(Exp\. 5\), in a redundant code that is sparsely decodable but causally spread over∼10\{\\sim\}10–3030features / a shared low\-rank direction \(Exp\. 2–3\); a small, nameable set of late read heads then funnels it into the decision logit \(Exp\. 1\), and causal scrubbing confirms the note alone governs the decision \(Exp\. 4\)\. This connects the “models take notes” account to the head\-level circuits ofWanget al\.\[[32](https://arxiv.org/html/2606.17107#bib.bib4)\]and to delimiter\-token aggregation\[[17](https://arxiv.org/html/2606.17107#bib.bib5)\], while remaining a statement about the*KV cache*an inference system already stores\.

Table 5:The circuit replicates across four families\.Read = decision recovery from jointly patching the top\-1212named read heads \(control in parens\)\. Write = top\-kkwrite\-head recovery at the single top aggregator \(distributed, so a floor\)\. Scrub: faithfulness drift under same\-conclusion resampling \(want≈0\{\\approx\}0\) and note\-vs\-rest interchange recovery\. Attn = attention share of the write at the causal layer\. Dir = 1\-D conclusion\-direction recovery \(alongd^\\hat\{d\}vs\. random\)\.![Refer to caption](https://arxiv.org/html/2606.17107v1/x15.png)Figure 15:A component\-level circuit for memoized inference\(deep dives Llama\-3\.1\-8B; head, write, and scrubbing panels show all four families\)\. \(a\) Cumulative decision recovery from jointly patching the top\-kknamed*read*heads \(concentrated,∼0\.78\{\\sim\}0\.78\) vs\.*write*heads \(distributed\)\. \(b\) A leave\-scenario\-out difference\-of\-means*conclusion direction*causally transfers the decision far above a random 1\-D direction\. \(c\) At the causal conclusion layer,*attention*writes the majority of the note\. \(d\) Causal scrubbing: same\-conclusion resampling is faithful \(drift≈0\{\\approx\}0\); swapping the*note*to the opposite conclusion flips the decision while swapping the rest does not\. \(e\) An SAE feature*decodes*the conclusion perfectly \(AUC=1\.0=1\.0\) yet is not causally sufficient alone—the cause is spread over∼10\{\\sim\}10–3030features\. \(f\) Read heads with the largest causal effect are those that attend decision→\\toaggregator\.

## Appendix EEditing: the baseline frontier and the K\-sweep

[Figure˜17](https://arxiv.org/html/2606.17107#A5.F17)plots the cost/correctness frontier behind[Section˜4](https://arxiv.org/html/2606.17107#S4): there is no single dominant method\.field\+erratumand the bareerratumreach full correctness at∼5\{\\sim\}5–13%13\\%recompute with no prompt surgery; hoist\-to\-end matches them but rewrites the prompt; the in\-place edit and a KV\-deviation\-ranked CacheBlend\-style recompute are cheap but incorrect on these gated decisions\. Numeric values are in[Table˜6](https://arxiv.org/html/2606.17107#A5.T6)\.[Figure˜17](https://arxiv.org/html/2606.17107#A5.F17)gives the fullfield\+selective@KKsweep across seven models, making the model\-dependence of the minimal recompute explicit: smallKKsuffices for the Qwen3 family but not for several others—the tool is effective but unreliable\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x16.png)Figure 16:Cost/correctness frontier\.
![Refer to caption](https://arxiv.org/html/2606.17107v1/x17.png)Figure 17:field\+selective@KKacross models\.

Table 6:Baseline frontier \(Qwen3\-8B, 8 gated tasks\)\. “surgery” = requires rewriting the prompt\.[Figure˜18](https://arxiv.org/html/2606.17107#A5.F18)shows that the editing fix is an*attention\-architecture*method: the erratum recovers the decision under reasoning on attention \(GQA\) and sliding\-window backbones, partially on a hybrid attention\+SSM model, and weakly on a pure SSM whose recurrent state has no per\-token look\-back\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x18.png)Figure 18:Erratum recovery under reasoning across attention, sliding\-window, hybrid, and pure\-SSM backbones\.
## Appendix FComposing: per\-domain scorecards, multimodal, and the MLA adapter

[Figure˜20](https://arxiv.org/html/2606.17107#A6.F20)is the composable scorecard: decision agreement between a transplanted skill and full recompute, by model and content type \(facts in two insertion points, and agentic tool\-calling\)\. Standard attention models are at or near1\.01\.0; the sliding\-window Gemma models are the consistent exception \(addressed in[Section˜8](https://arxiv.org/html/2606.17107#S8)\)\.[Figure˜21](https://arxiv.org/html/2606.17107#A6.F21)breaks the multimodal result down by task category, and[Figure˜20](https://arxiv.org/html/2606.17107#A6.F20)reports the MLA decoupled\-k\_peadapter fidelity and the transplant TTFT speedup across models\.[Figure˜22](https://arxiv.org/html/2606.17107#A6.F22)diagrams the three attention\-variant adapters of[Section˜8](https://arxiv.org/html/2606.17107#S8): what each representation caches, and exactly which slice of it the reposition touches\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x19.png)Figure 19:Composable agreement by model×\\timescontent type\.
![Refer to caption](https://arxiv.org/html/2606.17107v1/x20.png)Figure 20:MLA adapter fidelity \(a\) and transplant TTFT speedup across models \(b\)\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x21.png)Figure 21:Image\-KV transplant by task category \(perception / reasoning / agentic\), full re\-encode vs\. transplant, across four vision\-language models\. The transplant tracks full accuracy category\-by\-category\.![Refer to caption](https://arxiv.org/html/2606.17107v1/x22.png)Figure 22:The attention\-variant adapters\.\(a\) MLA caches a position\-free latentctc\_\{t\}plus a small decoupled\-RoPE sub\-vectork\_pe; repositioning re\-rotates onlyk\_peand copies the latent as\-is\. \(b\) M\-RoPE factors the rotary channels into temporal/height/width axes—sectioned \(Qwen2\.5\-VL\) or interleaved \(Qwen3\-VL\); moving an image within a trajectory re\-rotates only the temporal channels, since the spatial axes are intrinsic to the image\. \(c\) Sliding\-window layers default to a window\-truncated cache, which breaks any splice pastWW; keeping the full per\-token KV and letting the attention mask enforce the window restores uniform edit/splice semantics\.
## Appendix GLong\-horizon robustness and the cross\-referential memory test

This appendix collects two stress\-tests for the leave\-stale\+\+erratum scheme\. The first asks whether per\-edit errors*compound*over a long agent trajectory \(they do not\); the second asks whether splitting*cross\-referential*facts across independently\-precompiled blocks breaks a two\-hop chain—the case[Section˜7](https://arxiv.org/html/2606.17107#S7)’s E4 left open \(it does, and the mechanism explains why\)\.

### No compounding error over a long trajectory \([Figure˜23](https://arxiv.org/html/2606.17107#A7.F23)\)\.

A standing risk for any leave\-stale scheme is that small per\-edit errors*compound*over a long agent trajectory\. We test it directly\. A single gated field \(a clearance level\) toggles between a granting and a denying value every turn over a2828\-turn trajectory; we maintain ONE evolving KV cache—reuse the static prefix forever, apply each state change as an appended erratum, never recompute the downstream—and compare its per\-turn decision to a*full reprefill of the byte\-identical token sequence*\(errata included\)\. The only difference between the two is whether downstream KV was recomputed after each change, so per\-turn agreement isolates exactly the cost of leaving KV stale as a function of trajectory length\. Across three families \(Qwen3\-8B, Llama\-3\.1\-8B, Mistral\-7B\) the next\-token*decision logits stay faithful*—cosine0\.9870\.987–0\.9990\.999with a flat first\-third→\\tolast\-third profile \(e\.g\.0\.992→0\.9960\.992\\to 0\.996on Llama\)—and the agreement\-vs\-turn slope is within±0\.01\\pm 0\.01/turn \(no systematic decline\)\. Discrete decision agreement is high in aggregate \(0\.790\.79–0\.990\.99\) but noisier than the cosine, because these gated decisions sit near the action boundary \(oracle accuracy0\.520\.52–0\.820\.82\), where a sub\-percent logit difference can flip a discrete choice—the same boundary sensitivity noted for greedy CoT in[Section˜7](https://arxiv.org/html/2606.17107#S7)\. The leave\-stale\+\+erratum cache does not degrade with trajectory length: there is no compounding drift, only boundary noise\.

### Cross\-referential facts \(the[Section˜7](https://arxiv.org/html/2606.17107#S7)E4 test\)\.

E4 found splitting*independent*relevant facts across independently\-precompiled blocks decision\-lossless\. We test the case it left open—a genuinely*cross\-referential*chain\. A memory contains aDEFINITIONlineAAnaming which setting governs the request, whose valueBBlives elsewhere; the decision needs the two\-hop resolutionA→BA\\\!\\to\\\!B\. We layAAandBBso they straddle a block boundary and compare a transplant that*splits*them \(boundary betweenAAandBB\) against one that keeps them*colocated*\(boundary moved past both\), holding everything else fixed\. On Llama\-3\.1\-8B—which performs this two\-hop decision at full\-recompute accuracy0\.850\.85\(n=80n\{=\}80, balanced\)—splitting drops agreement with full recompute to0\.460\.46versus0\.760\.76colocated, a0\.300\.30penalty \(2525vs\.11discordant instances, McNemarp=8×10−7p\{=\}8\\times 10^\{\-7\}\); an*independent*fact pair split the same way costs only0\.040\.04\(Mistral\-7B\)\. The mechanism explains it: blockBBprecompiled in isolation never attended to its referentAA, so the chain is not memoized and the splice cannot restore it\. \(Qwen3\-8B is at chance on the direct two\-hop decision—consistent with[Section˜7](https://arxiv.org/html/2606.17107#S7)’s observation that direct gated decisions are at chance for the reasoning\-native family≤\\leq32B—so it is uninformative here\.\) The guidance is simple: keep cross\-referential facts within one precompiled block; independent facts may be split freely\.

![Refer to caption](https://arxiv.org/html/2606.17107v1/x23.png)Figure 23:No compounding error over a long trajectory\.A gated field toggles every turn for2828turns; one evolving leave\-stale\+\+erratum cache vs\. full reprefill of the identical text\. \(a\) Per\-turn decision agreement stays high with boundary noise \(no downward trend\)\. \(b\) The decision\-logit cosine stays flat at0\.99\+0\.99\{\+\}—no drift with trajectory length\.![Refer to caption](https://arxiv.org/html/2606.17107v1/x24.png)Figure 24:E4 — granularity is a free knob\([Section˜7](https://arxiv.org/html/2606.17107#S7)\)\. Splitting memory intoSSindependently\-precompiled blocks makes a localized editS×S\\timescheaper and stays decision\-lossless toS=16S\{=\}16; only genuinely cross\-referential facts must be kept in one block \(cross\-referential test above\)\.

## Appendix HThe user\-memory agent

The agent of[Section˜7](https://arxiv.org/html/2606.17107#S7)keeps the layout\[system\]\[trajectory\]\[MEMORY\]\[query\]\. The system prompt is prefilled once; the trajectory grows by cached deltas; the user\-memory chunk is precompiled once in isolation and RoPE\-repositioned to float just before the query each turn \(anO​\(Lmem\)O\(L\_\{\\text\{mem\}\}\)re\-rotation, no re\-prefill\) with one boundary token repaired; a memory change is applied either by recompiling the isolated chunk \(O​\(Lmem\)O\(L\_\{\\text\{mem\}\}\), once\) or by appending a salient erratum into the cached trajectory stream \(append\-only, composes with prefix caching\)\. Confirmatory protocol: gated decisions whose governing facts live in a Markdown memory, balanced labels, chain\-of\-thought as the competent regime \(direct one\-shot memory\-gated decisions are at chance for≤\\leq8B models\), with cluster\-bootstrap CIs \(10410^\{4\}, persona\-level\), TOST equivalence \(δ=0\.03\\delta\{=\}0\.03; cosine≥0\.98\\geq 0\.98\), GEE\-logistic \(cluster\-robust\), McNemar, and BH\-FDR\. Faithfulness is read against a token\-matched full\-reprefill oracle\. Pre\-registered hypotheses, margins, and the inclusion gate \(oracle accuracy≥0\.80\\geq 0\.80\) are released with the code\.

## Appendix IOnline serving and weight\-editing methodology

### Online vLLM serving \([Figure˜14](https://arxiv.org/html/2606.17107#S9.F14)a\)\.

vLLM V1AsyncLLMEngine, CUDA graphs enabled \(*not*enforce\_eager\), continuous batching, automatic prefix caching on,bfloat16, GPU memory utilization0\.850\.85\. Workload: a shared∼8,066\{\\sim\}8\{,\}066\-token policy \(realτ2\\tau^\{2\}\-bench retail policy plus neutral padding\) with one mutable field;9696requests per arm,6464output tokens each\. Requests arrive as a*Poisson*process at offered rates\{2,4,8,16\}\\\{2,4,8,16\\\}req/s and unthrottled \(saturation\)\. We timestamp first\-token and completion per request in the client loop \(TTFT, TPOT, end\-to\-end\) and read each arm’s APC hit\-rate from the engine’s Prometheus counters \(vllm:gpu\_prefix\_cache\_hits,…\_queries\)\. The in\-prefix baseline writes the new field value early \(invalidating downstream APC blocks\); the erratum keeps the old prefix and appends the update\. Headline: throughput speedup grows1\.6×→14\.5×1\.6\\times\{\\to\}14\.5\\timesas load rises to saturation;p​90p90TTFT5353–398×398\\timeslower; APC hit\-rate98\.5%98\.5\\%vs\.1\.0%1\.0\\%\.

### Weight editing \([Table˜1](https://arxiv.org/html/2606.17107#S4.T1)\)\.

ROME is implemented from scratch \(uncentered key covarianceC=𝔼​\[k​k⊤\]C\{=\}\\mathbb\{E\}\[kk^\{\\top\}\]at a mid MLP layer over a text sample, an optimized valuev⋆v^\{\\star\}, and the closed\-form rank\-one update ofdown\_proj\) and*validated on the canonical factual edit*\(“the Eiffel Tower is in”*Paris*→\\to*Rome*, locality preserved\) before use, so the baseline is faithful\. LoRA fine\-tunes \(r=8r\{=\}8,q,v,downprojections\) on the stale\-context decision until it flips\. All methods are evaluated on the same gated decision; cross\-request contamination is measured over88held\-out orders that are genuinely still pending \(correct==cancel\), and collateral over a1010\-item battery of unrelated single\-token gated decisions\.