In-Model Context Compression with Agentic Soft Tokens for Long-Horizon LLM Agents

# In-Model Context Compression with Agentic Soft Tokens for Long-Horizon LLM Agents
Source: [https://arxiv.org/html/2610.41291](https://arxiv.org/html/2610.41291)
Kai FanRunyu ZhangBinghuan WangYuxiang ZhangXue JiangYihong Dong

![[Uncaptioned image]](https://arxiv.org/html/2610.41291v1/assets/soft-token-icon.png)In\-Model Context Compression with Agentic Soft Tokens for Long\-Horizon LLM AgentsXuning He1Kai Fan2Runyu Zhang1Binghuan Wang3Yuxiang Zhang4 Xue Jiang5Yihong Dong1,🖂1Shanghai Jiao Tong University2Nanyang Technological University3Nankai University4Tongji University5Peking UniversityABSTRACTLong\-horizon LLM agents play an important role in solving complex and sustained tasks, with ever\-growing interaction contexts\. However, these accumulated contexts will raise substantial computational cost and obscure key decision\-relevant evidence\. Existing methods typically train external compressors or context\-management agents to rewrite context as shorter hard\-token sequences, but such external selection may not align with the LLM’s evolving representation and is necessarily lossy\. In this paper, we introduceAgentic Soft\-Token Compression \(ASTC\), a training\-free compression method using soft tokens within LLM agent for long\-horizon tasks\. Specifically, ASTC uses the model’s native recurrent dynamics to determine the already absorbed information, leveraging the commonly\-used hybrid Gated Delta Network \(GDN\)–attention\. It then carries this information forward as agentic soft tokens while retaining exact values as hard tokens\. The resulting compact sequence reduces later\-layer computation and yields reusable KV caches, extending the savings across the agent’s trajectory\. Extensive experiments on long\-horizon tasks show that ASTC matches or exceeds the uncompressed agent’s accuracy with up to 50\.68% fewer end\-to\-end Attention Pairs, while trained compressors require 15\.4×\\times–103\.7×\\timesmore Attention Pairs per successful task\. These findings highlight agentic soft tokens as a model\-native way to carry absorbed context forward as the task representation evolves\.GitHub Repo:[https://github\.com/RISE\-X\-Lab/Agentic\-Soft\-Token](https://github.com/RISE-X-Lab/Agentic-Soft-Token)Correspondence:[dongyh@sjtu\.edu\.cn](mailto:[email protected])

00footnotetext:🖂: Corresponding author\.## 1 Introduction

Large language models \(LLMs\) are increasingly deployed as agents that execute multi\-step workflows rather than produce isolated responses\([Hsu et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib27);[Liu et al\., 2026a](https://arxiv.org/html/2610.41291#bib.bib28);[Dong et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib31)\)\. Each step appends system instructions, skills, earlier decisions, tool interactions, code, and environmental feedback to the next model call\. The resulting context connects decisions across time, so losing an early constraint or observation can trigger an inconsistent action whose effects propagate through the remaining trajectory\. Preserving the full context, however, increases prefill and attention computation, enlarges the key–value \(KV\) cache, and forces relevant evidence to compete with stale or redundant information\. Long\-horizon agents therefore face the central challenge of controlling this growing cost without disrupting the cross\-step information flow required for coherent decisions\.

Reducing context under this constraint is not an ordinary summarization problem\. Agent context mixes semantic content with exact values\. Explanations, code, and repeated observations can become redundant once the acting LLM has integrated their information, whereas identifiers, API arguments, and numerical values may determine later actions and must remain exact\. Information that has little effect on the current action may also become decisive several steps later\. Such delayed relevance makes one\-shot importance estimates unreliable\. Compressibility therefore cannot be inferred from length, repetition, or semantic similarity alone\. It depends on what the acting LLM has already absorbed, what the current decision requires, and what a future step may need to recover\. Because compression recurs as the trajectory grows, each operation must save more computation than it consumes\. Effective context compression must therefore compact information already absorbed by the acting LLM, preserve exact or potentially useful evidence as the task evolves, and deliver savings across subsequent decoding and agent steps\.

Existing context\-compression methods typically rely on training, using either hard\-token or soft\-token compression\. Hard\-token methods prune or rewrite text, risking irreversible loss of exact values\([Jiang et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib7);[Pan et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib16);[Xu et al\., 2024a](https://arxiv.org/html/2610.41291#bib.bib19);[Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12);[Liu et al\., 2026b](https://arxiv.org/html/2610.41291#bib.bib14)\)\. Soft\-token methods preserve continuous representations but require separately trained encoders or compressors\([Mu et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib15);[Chevalier et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib3);[Ge et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib2);[Li et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib13)\)\. More fundamentally, separately trained compressors create a*model\-taste gap*\. The information they learn to preserve may not match what the acting LLM has already absorbed or will need later\. This motivates an in\-model approach that uses the acting LLM’s recurrent state to determine compressibility and carries absorbed information forward as agentic soft tokens\.

In this paper, we introduceAgentic Soft\-Token Compression \(ASTC\), a training\-free approach that uses the acting LLM’s own computation to decide what its growing context must retain\. The early layers have read the full context and the later layers would otherwise continue to process every original token\. Figureplaces this boundary within a continuing agent workflow and shows how the choice of compression stage changes task accuracy and workload\. In hybrid Gated Delta Network \(GDN\)–attention models\([Yang et al\., 2025](https://arxiv.org/html/2610.41291#bib.bib18)\), ASTC examines native recurrent updates and checks whether the retained state can recover a token’s contribution\. It keeps exact values explicit as hard tokens and carries eligible content forward as agentic soft tokens\. The shorter sequence reduces attention work for the current prediction and creates smaller KV caches that remain reusable as the agent continues\. Each compression decision is therefore informed by what the model has already absorbed, and its benefit extends across the trajectory\.

Across diverse long\-horizon tasks, ASTC maintains task quality close to or above the uncompressed model while reducing end\-to\-end Attention Pairs by up to 50\.68%\. Preserving exact values avoids the sharp accuracy loss seen when protection is removed, while the compact cache continues to save attention work over later decisions with only small changes in action distributions\. The same training\-free criterion extends beyond tool observations, matching uncompressed accuracy with 24\.52% fewer Decode KV interactions under full\-history compression and reducing workload in additional long\-context and code\-repair tasks\. ASTC combines the highest task accuracy with the lowest Attention Pairs per successful task, a 15\.4×\\times–103\.7×\\timesadvantage over trained compressors\.

Our contributions are summarized as follows:

- •We identify the core challenge of long\-horizon compression as reducing context cost without losing future\-relevant evidence or diverging from the acting LLM’s task representation\.
- •We introduce ASTC, a training\-free method that uses native recurrent dynamics to retain exact values as hard tokens and carry absorbed content forward as soft tokens with reusable compact KV caches across subsequent agent steps\.
- •We bound trained\-compressor objective mismatch, establish conditions under which ASTC has lower next\-token distortion than equal\-length hard\-token representations, and quantify attention savings from cache reuse\.
- •Across long\-horizon tasks, ASTC achieves comparable or better accuracy with up to 50\.68% fewer end\-to\-end Attention Pairs and a 15\.4×\\times–103\.7×\\timesadvantage in end\-to\-end Attention Pairs per successful task over trained compressors\.

## 2 Motivation

### 2\.1 In\-Model Soft Tokens

Context compression determines both how many tokens remain and which evidence reaches the acting LLM\. Existing methods make this decision before the model processes the compressed context, either pruning tokens or rewriting earlier interactions as shorter text\([Jiang et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib7);[Pan et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib16);[Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12);[Liu et al\., 2026b](https://arxiv.org/html/2610.41291#bib.bib14)\)\. This ordering creates a selection mismatch because the value of a context span is not intrinsic to its text\. It depends on how the acting LLM represents the current task, what it has already absorbed, and which action it is about to take; the same observation can be redundant at one step and decisive at another\. An external compressor must approximate this model\-dependent boundary from text alone\. In a long\-horizon trajectory, a mistaken deletion can propagate because later decisions are made from the retained context\.Hard\-token output adds a second bottleneck because retained information must be serialized as a shorter sequence of vocabulary tokens, so exact structure or features distributed across positions must be reconstructed as short text\. Even a correctly selected span can therefore lose task\-relevant information when forced into a short textual rewrite\. Moving compression inside the model lets the acting LLM judge compressibility where task information is represented, while soft tokens retain information in continuous contextual representations\. In\-model selection and soft\-token representation are therefore complementary, letting the acting LLM judge what matters and preserve that information in its own representation space, thereby avoiding both external selection mismatch and discrete textual reconstruction\.

### 2\.2 Tool Observations

Figure 1:Composition of accumulated context in real\-world agent trajectories\.We analyze agent trajectories from real\-world user interactions across systems such as Fable 5, GPT\-5\.6 Sol, Kimi K3, and GLM\-5\.2\([Anthropic, 2026](https://arxiv.org/html/2610.41291#bib.bib1);[OpenAI, 2026](https://arxiv.org/html/2610.41291#bib.bib4);[Moonshot AI, 2026](https://arxiv.org/html/2610.41291#bib.bib20);[Z\.AI, 2026](https://arxiv.org/html/2610.41291#bib.bib21)\)\. As Figure[1](https://arxiv.org/html/2610.41291#S2.F1)shows, a consistent pattern emerges across these trajectories\. Tool observations occupy the largest share of the accumulated context and continue to grow as the agent interacts with its environment\. In our experiments, tool observations are the primary compression target\.

Tool observations are difficult to compress because they mix compressible natural\-language and code spans with exact values, such as paths, identifiers, and checksums, that later actions may consume verbatim\. Our preliminary experiments show that the former tolerates aggressive compression, whereas changing a single character in the latter can invalidate a later action\. This heterogeneity makes tool observations a demanding but informative testbed\. Effective agent compression must therefore compact semantic information distributed across many tokens without altering the exact values on which future actions depend\.

### 2\.3 Model\-Native Selection

Our initial approach followed existing work by training a separate in\-model compressor to produce soft tokens\. We inserted an attention–MLP compressor after either a single backbone layer or multiple layers, thereby compressing the hidden sequence once or repeatedly, and trained it withOn\-policy distillation \(OPD\)\. Comparing these variants with ACON and LCLM reveals a consistent pattern: training\-based compression incurs substantial preparation cost yet delivers weak task performance, while repeated intervention across more layers offers no remedy \(Figure\)\([Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12);[Li et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib13)\)\. This failure reflects a difficult optimization problem\. An auxiliary compressor must infer what the acting LLM has already absorbed, retain information that later steps may require, and emit features compatible with downstream layers, all through an objective separate from the LLM’s native prediction process\.

Hybrid GDN–attention models already compute a signal closely aligned with this decision during their normal forward pass\. A GDN recurrent state acts as an online associative memory\. Its gated delta update can be written as\([Yang et al\., 2025](https://arxiv.org/html/2610.41291#bib.bib18)\)

𝐒t=αt​𝐒t−1\+βt​𝐫t​𝐤t⊤,𝐫t=𝐯t−αt​𝐒t−1​𝐤t,\\mathbf\{S\}\_\{t\}=\\alpha\_\{t\}\\mathbf\{S\}\_\{t\-1\}\+\\beta\_\{t\}\\mathbf\{r\}\_\{t\}\\mathbf\{k\}\_\{t\}^\{\\top\},\\qquad\\mathbf\{r\}\_\{t\}=\\mathbf\{v\}\_\{t\}\-\\alpha\_\{t\}\\mathbf\{S\}\_\{t\-1\}\\mathbf\{k\}\_\{t\},\(1\)whereαt\\alpha\_\{t\}controls how much previous memory survives,βt\\beta\_\{t\}controls the write strength, and𝐫t\\mathbf\{r\}\_\{t\}is the part of the incoming value that cannot be recovered from the retained state\. Information already represented by the state produces a small correction, whereas novel or conflicting information changes the memory more strongly\. The update therefore measures information relative to what the acting LLM currently remembers, not importance inferred from surface form alone\. Moreover, the gate and delta update are optimized jointly with the LLM’s predictions, so this distinction shares the representation space and task state used by downstream layers\. Reusing GDN dynamics avoids teaching an auxiliary compressor to approximate the acting LLM’s own notion of novelty\. Training\-free selection is therefore not merely a cheaper alternative; it follows from recognizing that the acting LLM has already performed the information judgment that compression requires\.

## 3 Related Work

Discrete context compression shortens the sequence before it reaches the acting LLM\. Selective Context\([Li et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib32)\)prunes redundant spans; LLMLingua\([Jiang et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib7)\)and LongLLMLingua\([Jiang et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib8)\)rank tokens for prompt pruning; LLMLingua\-2\([Pan et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib16)\)learns selection from distilled labels; and RECOMP\([Xu et al\., 2024a](https://arxiv.org/html/2610.41291#bib.bib19)\)trains compressors for retrieved evidence\. For extended agent workflows, ACON\([Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12)\)optimizes textual compression guidelines from task failures, while Context as a Tool \(CAT\)\([Liu et al\., 2026b](https://arxiv.org/html/2610.41291#bib.bib14)\)trains SWE\-Compressor to invoke context management during execution\. MEM1\([Zhou et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib10)\)instead trains a compact memory jointly with agent reasoning\. While these lines reduce accumulated context, text\-level selection can discard exact evidence or diverge from the acting LLM’s evolving task representation, and learned memory policies require additional training\. Meta Context Engineering\([Ye et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib26)\)co\-evolves agentic skills and context artifacts to optimize task\-specific context construction\.

Soft\-token methods retain information in continuous states\. Gist Tokens\([Mu et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib15)\)learns compressed prompt activations, AutoCompressors\([Chevalier et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib3)\)produce recursive summary vectors, and ICAE\([Ge et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib2)\)trains compact memory slots\. Sentinel Tokens\([Ren et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib33)\)and Activation Beacon\([Zhang et al\., 2025](https://arxiv.org/html/2610.41291#bib.bib34)\)compress intermediate activations for later processing\. DAST\([Chen et al\., 2025](https://arxiv.org/html/2610.41291#bib.bib35)\)allocates soft tokens according to contextual information density, while Latent Context Language Models \(LCLMs\)\([Li et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib13)\)train dedicated encoder–decoder models for large\-scale compression\. For tool\-using LMs,[Xu et al\. \(2024b\)](https://arxiv.org/html/2610.41291#bib.bib36)combine compressed representations with raw tokens for exact tool interfaces\. These studies show that continuous memory can support incremental reuse and coexist with exact\-token retention\. Their compression behavior is nevertheless learned through model adaptation or auxiliary components, and their primary settings differ from a growing history of agent actions and observations\.

KV\-cache eviction offers another route: H2O\([Zhang et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib29)\)retains recent and heavy\-hitter tokens, while SnapKV\([Li et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib30)\)selects important positions per attention head\. Archer\([He et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib25)\)adapts prompt\-K/V reuse to evolving response states in rollback\-capable diffusion language models\.

ASTC brings the continuous memory studied in soft\-token compression to the growing histories addressed by agent context\-management work\. By operating within the acting LLM without a separately trained compressor, it keeps compression tied to the model’s evolving task representation\. To the best of our knowledge, ASTC is among the first in\-model soft\-token methods designed for context compression in long\-horizon LLM agents\.

![Refer to caption](https://arxiv.org/html/2610.41291v1/figure_04_astc_method.png)Figure 2:Overview of ASTC for in\-model context compression and KV cache reuse\.
## 4 Method

### 4\.1 Agentic Soft\-Token Compression

At stepaa, early layers process every token in the uncached suffixΔ​X\(a\)\\Delta X^\{\(a\)\}, using cache𝒦<\(a−1\)\\mathcal\{K\}\_\{<\}^\{\(a\-1\)\}for prior context\. ASTC then compresses the new hidden states in tool\-observation spans𝒪\\mathcal\{O\}subject to protection maskPP:

Hℓ⋆new\\displaystyle H\_\{\\ell^\{\\star\}\}^\{\\mathrm\{new\}\}=F0:ℓ⋆\(ΔX\(a\);𝒦<\(a−1\)\)∈ℝm×d,\\displaystyle=F\_\{0:\\ell^\{\\star\}\}\(\\Delta X^\{\(a\)\};\\mathcal\{K\}\_\{<\}^\{\(a\-1\)\}\)\\in\\mathbb\{R\}^\{m\\times d\},\(2\)H~ℓ⋆new\\displaystyle\\widetilde\{H\}\_\{\\ell^\{\\star\}\}^\{\\mathrm\{new\}\}=𝒞\(Hℓ⋆new;𝒪,P\)∈ℝm~×d,m~≤m\.\\displaystyle=\\mathcal\{C\}\(H\_\{\\ell^\{\\star\}\}^\{\\mathrm\{new\}\};\\mathcal\{O\},P\)\\in\\mathbb\{R\}^\{\\widetilde\{m\}\\times d\},\\quad\\widetilde\{m\}\\leq m\.
PPkeeps recognized exact values and the action boundary as hard tokens, although it may miss some exact fields\. For each GDN headhh, let𝐒i−1,h\\mathbf\{S\}\_\{i\-1,h\}be its state before positionii, with key𝐤i,h\\mathbf\{k\}\_\{i,h\}, value𝐯i,h\\mathbf\{v\}\_\{i,h\}, and decayαi,h=exp⁡\(gi,h\)\\alpha\_\{i,h\}=\\exp\(g\_\{i,h\}\)\. Let𝐯i\\mathbf\{v\}\_\{i\}and𝜹i\\bm\{\\delta\}\_\{i\}concatenate the head\-wise vectors\. The first test measures innovation relative to the recurrent state:

𝜹i,h=𝐯i,h−αi,h​ℛ​\(𝐒i−1,h,𝐤i,h\),χi=RMS⁡\(𝜹i\)RMS⁡\(𝐯i\)\+ϵ,\\bm\{\\delta\}\_\{i,h\}=\\mathbf\{v\}\_\{i,h\}\-\\alpha\_\{i,h\}\\,\\mathcal\{R\}\(\\mathbf\{S\}\_\{i\-1,h\},\\mathbf\{k\}\_\{i,h\}\),\\qquad\\chi\_\{i\}=\\frac\{\\operatorname\{RMS\}\(\\bm\{\\delta\}\_\{i\}\)\}\{\\operatorname\{RMS\}\(\\mathbf\{v\}\_\{i\}\)\+\\epsilon\},\(3\)whereℛ⁡\(𝐒,𝐤\)=𝐒𝐤\\mathcal\{R\}\(\\mathbf\{S\},\\mathbf\{k\}\)=\\mathbf\{S\}\\mathbf\{k\}\. A smallχi\\chi\_\{i\}means the incoming value was largely predicted by the preceding state\. Prediction at positioniidoes not ensure that later updates retain the value, so ASTC also reads the observation\-end state𝐒o,h\\mathbf\{S\}\_\{o,h\}with a fixed, untrained spectral basis𝐐h∈ℝdk,h×M\\mathbf\{Q\}\_\{h\}\\in\\mathbb\{R\}^\{d\_\{k,h\}\\times M\}, using the pooling slot budgetMM:

𝐯^i,h=𝐒o,h​𝐐h​\(𝐐h⊤​𝐐h\+λ​𝐈\)−1​𝐐h⊤​𝐤i,h,ψi=RMS⁡\(𝐯i−𝐯^i\)RMS⁡\(𝐯i\)\+ϵ\.\\widehat\{\\mathbf\{v\}\}\_\{i,h\}=\\mathbf\{S\}\_\{o,h\}\\mathbf\{Q\}\_\{h\}\(\\mathbf\{Q\}\_\{h\}^\{\\top\}\\mathbf\{Q\}\_\{h\}\+\\lambda\\mathbf\{I\}\)^\{\-1\}\\mathbf\{Q\}\_\{h\}^\{\\top\}\\mathbf\{k\}\_\{i,h\},\\qquad\\psi\_\{i\}=\\frac\{\\operatorname\{RMS\}\(\\mathbf\{v\}\_\{i\}\-\\widehat\{\\mathbf\{v\}\}\_\{i\}\)\}\{\\operatorname\{RMS\}\(\\mathbf\{v\}\_\{i\}\)\+\\epsilon\}\.\(4\)Here𝐯^i\\widehat\{\\mathbf\{v\}\}\_\{i\}concatenates the per\-head reads\. ASTC selects an unprotected position only when it is both locally predictable \(χi<τχ\\chi\_\{i\}<\\tau\_\{\\chi\}\) and still readable at the endpoint \(ψi<τψ\\psi\_\{i\}<\\tau\_\{\\psi\}\)\.

The selected positions still need a compact representation\. ASTC partitions them into at mostMMcausal groups, trims outliers around a member of each group, and mean\-pools the retained states of groupGGat the boundary:

𝐳G=cast⁡\(1\|G\|​∑i∈G𝐡iFP32\)\.\\mathbf\{z\}\_\{G\}=\\operatorname\{cast\}\\\!\\left\(\\frac\{1\}\{\|G\|\}\\sum\_\{i\\in G\}\\mathbf\{h\}\_\{i\}^\{\\mathrm\{FP32\}\}\\right\)\.\(5\)The replacement stays in the acting model’s hidden space rather than being rewritten as text\. Only multi\-position groups with maximum post\-cast relative hidden\-state error at mostτh\\tau\_\{h\}become*agentic soft tokens*\. Protected, rejected, and unmerged positions remain hard tokens\. Each soft token takes its group’s latest logical position, and a stable sort restores causal order\.

Compression leaves the pre\-boundary caches at their original\-token lengths, while downstream attention caches the shorter mixed sequence\. This cache is reused during decoding and later agent steps, so an accepted merge saves work beyond the prefill in which it occurs\. Algorithm[1](https://arxiv.org/html/2610.41291#alg1)specifies the per\-observation commit gate; bounded pooling is detailed in Appendix[I\.1](https://arxiv.org/html/2610.41291#A9.SS1)\.

Algorithm 1ASTC at one prefill step with a persistent hybrid cache0:Uncached suffix

Δ​X\(a\)\\Delta X^\{\(a\)\}, existing layer\-specific hybrid cache

𝒦\(a−1\)\\mathcal\{K\}^\{\(a\-1\)\}, closed observation spans

𝒪\\mathcal\{O\}in this suffix, protection mask

PP, frozen

FF, boundary

ℓ⋆\\ell^\{\\star\}, reader

𝐐\\mathbf\{Q\}, thresholds

τχ,τψ,τh\\tau\_\{\\chi\},\\tau\_\{\\psi\},\\tau\_\{h\}, group cap

MM, budget

BB, remaining\-layer count

LremL\_\{\\mathrm\{rem\}\}, fraction

ρ\\rho
0:Next\-token logits; updated hybrid cache

𝒦\(a\)\\mathcal\{K\}^\{\(a\)\}
1:foreach observation

o∈𝒪o\\in\\mathcal\{O\}do

2:

ℰo←\{i∈o:Pi=0​andiis an original position\}\\mathcal\{E\}\_\{o\}\\leftarrow\\\{i\\in o:P\_\{i\}=0\\ \\text\{and $i$ is an original position\}\\\}
3:

ro←max⁡\(⌈B/Lrem⌉,⌈ρ​\|ℰo\|⌉\)r\_\{o\}\\leftarrow\\max\(\\lceil B/L\_\{\\mathrm\{rem\}\}\\rceil,\\lceil\\rho\|\\mathcal\{E\}\_\{o\}\|\\rceil\)
4:Bypass

ooif it already contains a soft token or

\|ℰo\|<ro\+1\|\\mathcal\{E\}\_\{o\}\|<r\_\{o\}\+1
5:endfor

6:Run

F0:ℓ⋆F\_\{0:\\ell^\{\\star\}\}on

Δ​X\(a\)\\Delta X^\{\(a\)\}with its existing cache; update early\-layer caches and obtain

Hℓ⋆newH\_\{\\ell^\{\\star\}\}^\{\\mathrm\{new\}\}
7:During this run, capture native GDN tensors and read endpoint states only for non\-bypassed spans

8:foreach non\-bypassed observation

o∈𝒪o\\in\\mathcal\{O\}do

9:

𝒞o←\{i∈ℰo:χi<τχ\}\\mathcal\{C\}\_\{o\}\\leftarrow\\\{i\\in\\mathcal\{E\}\_\{o\}:\\chi\_\{i\}<\\tau\_\{\\chi\}\\\}
10:Continue to the next observation if

\|𝒞o\|<ro\+1\|\\mathcal\{C\}\_\{o\}\|<r\_\{o\}\+1
11:

𝒞o←\{i∈𝒞o:ψi<τψ\}\\mathcal\{C\}\_\{o\}\\leftarrow\\\{i\\in\\mathcal\{C\}\_\{o\}:\\psi\_\{i\}<\\tau\_\{\\psi\}\\\}
12:Continue to the next observation if

\|𝒞o\|<ro\+1\|\\mathcal\{C\}\_\{o\}\|<r\_\{o\}\+1
13:

\(ℳo,𝒵o\)←BoundedPool​\(Hℓ⋆new​\[𝒞o\],M,τh\)\(\\mathcal\{M\}\_\{o\},\\mathcal\{Z\}\_\{o\}\)\\leftarrow\\textsc\{BoundedPool\}\(H\_\{\\ell^\{\\star\}\}^\{\\mathrm\{new\}\}\[\\mathcal\{C\}\_\{o\}\],M,\\tau\_\{h\}\)
14:if

\|ℳo\|−\|𝒵o\|≥ro\|\\mathcal\{M\}\_\{o\}\|\-\|\\mathcal\{Z\}\_\{o\}\|\\geq r\_\{o\}then

15:Replace

ℳo\\mathcal\{M\}\_\{o\}by

𝒵o\\mathcal\{Z\}\_\{o\}at each group’s greatest logical position

16:endif

17:endfor

18:Stably sort the mixed new suffix by logical position

19:Run

Fℓ⋆\+1:LF\_\{\\ell^\{\\star\}\+1:L\}on that suffix with its existing cache; update downstream attention KV and GDN recurrent and convolutional states

20:Apply final norm and LM head to the last protected original position

21:returnnext\-token logits and

𝒦\(a\)\\mathcal\{K\}^\{\(a\)\}

HereBBis the minimum downstream token–layer reduction,LremL\_\{\\mathrm\{rem\}\}counts layers after the boundary, andρ\\rhois the minimum reduction as a fraction of eligible positions in each observation\.

### 4\.2 Theoretical Analysis

We examine what native selection can certify, how a compact cache saves work over time, and how merging can shift downstream attention\. The distributional claims below concern a fixed next\-token prefix; they do not imply identical agent trajectories\.

#### 4\.2\.1 Model\-Native Selection and Continuous Representation

Fix a history, a next\-token prefix, and a boundaryℓ⋆\\ell^\{\\star\}\. LetHHbe the full boundary sequence andZZa length\-kkreplacement at fixed logical positions\. We measure the acting model’s next\-token distortion by

Dθ\(H,Z\)=DKL\(pθ\>ℓ⋆\(⋅∣H\)∥pθ\>ℓ⋆\(⋅∣Z\)\)\.D\_\{\\theta\}\(H,Z\)=D\_\{\\mathrm\{KL\}\}\\\!\\left\(p\_\{\\theta\}^\{\>\\ell^\{\\star\}\}\(\\cdot\\mid H\)\\,\\\|\\,p\_\{\\theta\}^\{\>\\ell^\{\\star\}\}\(\\cdot\\mid Z\)\\right\)\.\(6\)A separately trained compressor minimizes a surrogate objectiveJtrJ\_\{\\mathrm\{tr\}\}, which need not matchDθD\_\{\\theta\}\. On the same nonempty admissible set𝒜k\\mathcal\{A\}\_\{k\}, if both minima are attained,supZ∈𝒜k\|Dθ​\(H,Z\)−Jtr​\(H,Z\)\|≤ε\\sup\_\{Z\\in\\mathcal\{A\}\_\{k\}\}\|D\_\{\\theta\}\(H,Z\)\-J\_\{\\mathrm\{tr\}\}\(H,Z\)\|\\leq\\varepsilon, and the returnedZtrZ\_\{\\mathrm\{tr\}\}is withinξ\\xiof the surrogate minimum, then

0≤Dθ​\(H,Ztr\)−minZ∈𝒜k⁡Dθ​\(H,Z\)≤2​ε\+ξ\.0\\leq D\_\{\\theta\}\(H,Z\_\{\\mathrm\{tr\}\}\)\-\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}\}D\_\{\\theta\}\(H,Z\)\\leq 2\\varepsilon\+\\xi\.\(7\)This decomposition exposes a possible source of error, but does not establish a nonzero gap or rank methods with different feasible sets\.

ASTC instead reads the acting model’s own GDN signals\. The update in Equation[1](https://arxiv.org/html/2610.41291#S2.E1)yields a per\-head write bound proportional toχi\\chi\_\{i\}, whileψi\\psi\_\{i\}measures what remains readable at the observation endpoint\. These tests describe state coverage, but replacing positions can also shift downstream attention and recurrent states\. For a fixed next\-token prefix, suppose those effects admit aligned local perturbationseje\_\{j\}and the remaining logit map isLjL\_\{j\}\-Lipschitz at each site\. Write𝐥\\mathbf\{l\}and𝐥~\\widetilde\{\\mathbf\{l\}\}for their logits, andppandp~\\widetilde\{p\}for their next\-token distributions\. WithE=∑jLj​ejE=\\sum\_\{j\}L\_\{j\}e\_\{j\}, we obtain

∥𝐥~−𝐥∥∞≤E,DKL\(p∥p~\)≤2E\.\\\|\\widetilde\{\\mathbf\{l\}\}\-\\mathbf\{l\}\\\|\_\{\\infty\}\\leq E,\\qquad D\_\{\\mathrm\{KL\}\}\(p\\\|\\widetilde\{p\}\)\\leq 2E\.\(8\)Hence the next\-token distribution changes little whenEEis small, and the greedy token is unchanged if its original logit margin exceeds2​E2E\. The selector tests alone do not bound every term inEE\.

Stability addresses whether a replacement is safe; the representation space determines what it can retain at a fixed budget\. At equal physical lengthkkand fixed logical positions in an unchanged context, hard\-token reachable boundary states form a finite set𝒜khard\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}inside the continuous space𝒜ksoft=ℝk×d\\mathcal\{A\}\_\{k\}^\{\\mathrm\{soft\}\}=\\mathbb\{R\}^\{k\\times d\}\. With full\-context distributionpH=pθ\>ℓ⋆\(⋅∣H\)p\_\{H\}=p\_\{\\theta\}^\{\>\\ell^\{\\star\}\}\(\\cdot\\mid H\), letδk\(H\)=minZ∈𝒜khard∥pH−pθ\>ℓ⋆\(⋅∣Z\)∥1\\delta\_\{k\}\(H\)=\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}\}\\\|p\_\{H\}\-p\_\{\\theta\}^\{\>\\ell^\{\\star\}\}\(\\cdot\\mid Z\)\\\|\_\{1\}\. Set inclusion and Pinsker’s inequality give

infZ∈𝒜ksoftDθ​\(H,Z\)\\displaystyle\\inf\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{soft\}\}\}D\_\{\\theta\}\(H,Z\)≤minZ∈𝒜khard⁡Dθ​\(H,Z\),\\displaystyle\\leq\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}\}D\_\{\\theta\}\(H,Z\),\(9\)Dθ​\(H,ZASTC\)\\displaystyle D\_\{\\theta\}\(H,Z\_\{\\mathrm\{ASTC\}\}\)<minZ∈𝒜khardDθ\(H,Z\)ifE<δk\(H\)2/4\.\\displaystyle<\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}\}D\_\{\\theta\}\(H,Z\)\\quad\\text\{if \}E<\\delta\_\{k\}\(H\)^\{2\}/4\.The first inequality describes representational capacity\. The second gives a sufficient condition for ASTC’s particular soft\-token output to outperform every length\-kkhard\-token representation\. Its premise is not guaranteed by the selector thresholds; proofs appear in Appendix[A](https://arxiv.org/html/2610.41291#A1)\.

#### 4\.2\.2 One\-Pass and Amortized Efficiency

The boundary also determines when savings arise\. Early layers still read the new block, but later attention sees fewer positions in the same pass; the avoided work must exceed selection and pooling cost for a net gain\. Once the shorter KV cache persists, subsequent queries revisit fewer keys\.

###### Theorem 4\.1\(Amortized attention\-pair reduction\)\.

Suppose the affected downstream layers containA\>A\_\{\>\}full causal attention layers\. Before one commit, the current prefill has physical lengthnphysn\_\{\\mathrm\{phys\}\}, including an unchanged cached prefix, and allrrremoved positions belong to the newly processed block\. Under the same continuation ofNqN\_\{q\}later queries reaching these layers, assume the compared caches retain theirrr\-position difference\. Then the total reduction in query–key token pairs is

Δ​Q=A\>​r​\(2​nphys−r\+12\+Nq\)\.\\Delta Q=A\_\{\>\}r\\left\(\\frac\{2n\_\{\\mathrm\{phys\}\}\-r\+1\}\{2\}\+N\_\{q\}\\right\)\.\(10\)

The first term counts the current prefill, whileNqN\_\{q\}counts reuse of the same compact cache\. Savings therefore accumulate without another compression decision\. The proof in Appendix[A\.6](https://arxiv.org/html/2610.41291#A1.SS6)concerns attention pairs rather than wall\-clock speedup\.

#### 4\.2\.3 Downstream Attention Reweighting

Compression can also change which evidence downstream attention emphasizes\. A redundant group may attract attention through multiplicity alone\. The following idealized lemma isolates that effect\.

###### Lemma 4\.2\(Attention mass under idealized merging\)\.

For the same downstream query in both runs, after the last logical position of a merged groupGG\(\|G\|=g\>1\|G\|=g\>1\), assume its members and representative share scoressand value𝐯G\\mathbf\{v\}\_\{G\}\. Let retained positions have unchanged scores and values, with unnormalized massZR\>0Z\_\{R\}\>0and attention\-weighted value𝐯¯R\\overline\{\\mathbf\{v\}\}\_\{R\}\. Their total attention masses before and after replacement,ARA\_\{R\}andA~R\\widetilde\{A\}\_\{R\}, satisfy

A~R−AR=ZR​\(g−1\)​es\(ZR\+es\)​\(ZR\+g​es\)\>0,𝐮⊤​\(𝐨~−𝐨\)=\(A~R−AR\)​𝐮⊤​\(𝐯¯R−𝐯G\)\\widetilde\{A\}\_\{R\}\-A\_\{R\}=\\frac\{Z\_\{R\}\(g\-1\)e^\{s\}\}\{\(Z\_\{R\}\+e^\{s\}\)\(Z\_\{R\}\+ge^\{s\}\)\}\>0,\\qquad\\mathbf\{u\}^\{\\top\}\(\\widetilde\{\\mathbf\{o\}\}\-\\mathbf\{o\}\)=\(\\widetilde\{A\}\_\{R\}\-A\_\{R\}\)\\mathbf\{u\}^\{\\top\}\(\\overline\{\\mathbf\{v\}\}\_\{R\}\-\\mathbf\{v\}\_\{G\}\)\(11\)for any linear readout direction𝐮\\mathbf\{u\}\.

The group retains one representative but loses the extra attention weight of itsg−1g\-1copies\. If the retained value has a larger projection along𝐮\\mathbf\{u\}than𝐯G\\mathbf\{v\}\_\{G\}, the attention output shifts toward it\. This describes an idealized local effect, not a guarantee of improved task accuracy\. Appendix[A\.7](https://arxiv.org/html/2610.41291#A1.SS7)proves the lemma and bounds the unequal\-score and unequal\-value cases separately\.

## 5 Experiments

We evaluate ASTC with a frozen Qwen3\.5\-9B acting model on AppWorld\([Trivedi et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib17)\), OfficeBench\([Wang et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib11)\), and 8\-Objective QA\([Zhou et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib10);[Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12)\)\. We compare with uncompressed Base, prompted compression, LLMLingua\([Jiang et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib7)\), LLMLingua\-2\([Pan et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib16)\), ACON UT and UTCO\([Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12)\), and LCLM\([Li et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib13)\)\. In Table[1](https://arxiv.org/html/2610.41291#S5.T1), ASTC compresses only eligible tool observations\. We measure task quality and end\-to\-end Decode KV, Attention Pairs, and Generated Tokens\. Decode KV counts decoding query–cache interactions; Attention Pairs counts causal query–key pairs across prefill, decoding, and auxiliary calls\. Supplementary analyses and detailed setups are in Appendices[C](https://arxiv.org/html/2610.41291#A3)–[I](https://arxiv.org/html/2610.41291#A9)\.

### 5\.1 Main Results

Table 1:Accuracy and end\-to\-end inference workload on three long\-horizon agentic benchmarks\.*Note\.*Workload includes auxiliary compressor calls, including LLMLingua, and is relative to Base \(100%\)\.Boldandunderlinedmark the best and second\-best values by arrow\.

Table[1](https://arxiv.org/html/2610.41291#S5.T1)shows that ASTC offers a favorable balance of task quality and inference workload under the distinct interaction demands of the three benchmarks\. On AppWorld, it is the only evaluated method that improves accuracy over Base while reducing all three workload measures; competing compressors obtain isolated savings but sacrifice accuracy or increase work elsewhere\. On OfficeBench, ASTC remains close to Base in accuracy while substantially reducing Decode KV and Attention Pairs; Prompting achieves higher accuracy but uses more Decode KV and generated tokens than ASTC\. On 8\-Objective QA, ASTC achieves the best answer quality while reducing generated tokens\. LCLM has the lowest measured workload but answers none of the tasks correctly; ACON UTCO reduces Decode KV and Attention Pairs further than ASTC but has lower answer quality\. Taken together, these results place ASTC’s value at the trajectory level, where retained task quality and cumulative inference workload must be judged together\.

### 5\.2 Model\-Native Selector Controls

We ask whether ASTC’s GDN signals improve selection beyond what protection and sequence shortening provide\. We test this by retaining the acting model, protection rules, and legal group construction while changing the selection rule\. Hidden\-only uses hidden\-state reconstruction error to test whether local fidelity suffices; Random assigns fixed random scores to test whether protected grouping alone explains the gains\. Each policy runs the full AppWorld in closed loop, allowing compression choices to shape later actions, tool observations, and workload\. Figure[3](https://arxiv.org/html/2610.41291#S5.F3)compares task success with end\-to\-end Attention Pairs and shows ASTC’s strongest joint outcome, consistent with the value of model\-native selection\. Because trajectories and compression events differ, the comparison evaluates complete policies; paired bootstrap uncertainty makes the success advantage suggestive rather than conclusive\. The comparison suggests that the acting model’s accumulated state supplies information about compressibility beyond local reconstruction \(Appendix[G](https://arxiv.org/html/2610.41291#A7)\)\.

Figure 3:Selector controls on AppWorld\.
### 5\.3 Persistence After One Commit

Compression helps a long\-horizon agent only if its shorter KV cache remains useful when later steps revisit it, without substantially changing the agent’s decisions\. To isolate this mechanism, we branch from archived Base trajectories, allow at most one ASTC commit in one branch, and replay identical subsequent action tokens and tool results in both\. This fixed continuation separates the effect of persistent cache compression from changes caused by diverging actions or observations\. With no further commits allowed, savings at later horizons reflect reuse of the initial cache update rather than repeated compression\. At successive action horizons during the shared continuation, we measure KL divergence between conditional action\-token distributions and cumulative Attention Pairs saved since the commit\. Figure[4](https://arxiv.org/html/2610.41291#S5.F4)shows that the distributions remain close while savings recur at later actions, supporting the proposed amortization across the trajectory\. A separate autonomous one\-commit run probes whether this local stability carries through to task outcomes; its success is somewhat lower than archived Base and full ASTC, and differing archive versions make the comparison descriptive\. Appendix[H](https://arxiv.org/html/2610.41291#A8)reports all replayed actions and a fixed task cohort, showing how savings accumulate as the compact cache is reused\.

Figure 4:One\-commit cache reuse and task outcomes\. Middle: cumulative Attention Pairs saved since the commit; the observed task set varies across horizons\.
### 5\.4 Extended Context Compression

We extend ASTC beyond tool observations to assistant history and eligible full\-context regions\. Figure[5](https://arxiv.org/html/2610.41291#S5.F5)compares these scopes with the Tool setting from Table[1](https://arxiv.org/html/2610.41291#S5.T1)\.

Figure 5:Compression scopes on AppWorld\. Bars: workload reduction; diamonds: accuracy; dashed: Base \(32\.14%\)\. Workloads use each protocol’s matched Base\. Tool is from Table[1](https://arxiv.org/html/2610.41291#S5.T1); Assistant and Context are from Table[5](https://arxiv.org/html/2610.41291#A5.T5)\(Appendix[E](https://arxiv.org/html/2610.41291#A5)\)\.Assistant achieves 31\.55% accuracy while reducing Decode KV by 11\.16%, Attention Pairs by 3\.54%, and generated tokens by 7\.76%\.

Context achieves 30\.95% accuracy while reducing Decode KV, Attention Pairs, and generated tokens by 20\.50%, 19\.68%, and 9\.19%, respectively\.

Most Context compression occurs in tool returns\. The numerical results and further analysis are in Appendix[E](https://arxiv.org/html/2610.41291#A5), Table[5](https://arxiv.org/html/2610.41291#A5.T5)\.

### 5\.5 Cross\-Model Transfer

Table 2:Cross\-model transfer on AppWorld\.*Note\.*Qwen models: 168 tasks; Kimi: 68\. Workloads include failed tasks and are normalized to each model’s Base \(100%\)\.

Table[2](https://arxiv.org/html/2610.41291#S5.T2)compares ASTC with the uncompressed Base across three backbones\. On Qwen3\.5\-9B, accuracy increases from 32\.14% to 33\.33%, while Decode KV and Attention Pairs decrease by 18\.45% and 19\.30%, respectively\. On Qwen3\.8\-27B, accuracy decreases from 78\.57% to 77\.98%, with 8\.30% less Decode KV and 35\.57% fewer Attention Pairs, although generated tokens increase by 3\.82%\. On Kimi\-Linear, accuracy increases from 1\.47% to 4\.41%, while Decode KV and Attention Pairs decrease by 15\.99% and 11\.24%, respectively\. Protocols and selection limitations are in Appendix[F](https://arxiv.org/html/2610.41291#A6)\.

## 6 Conclusion

Long\-horizon agents must retain evidence for future decisions as context grows, while separately trained compressors may drift from the acting LLM’s evolving task representation\. ASTC uses native recurrent dynamics after early layers have read the full sequence, carrying absorbed content as agentic soft tokens and keeping exact values as hard tokens\. The shorter sequence cuts later\-layer computation and creates a compact KV cache whose savings persist across subsequent decisions\. Without compressor training, ASTC keeps accuracy close to or above the uncompressed agent with up to 50\.68% fewer end\-to\-end Attention Pairs; in the controlled comparison, it attains the highest accuracy and lowest Attention Pairs per successful task\. Ablations and fixed\-continuation analysis link these gains to exact\-value protection, model\-native selection, and persistent cache reuse\. These results show how the acting LLM can compact absorbed context into reusable state without losing exact evidence for later decisions\.

## References

- Anthropic \(2026\)AnthropicClaude Fable 5 and Claude Mythos 5\.External Links:[Link](https://www.anthropic.com/news/claude-fable-5-mythos-5)Cited by:[§2\.2](https://arxiv.org/html/2610.41291#S2.SS2.p1.1)\.
- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye,et al\.Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.External Links:[Link](https://arxiv.org/abs/2108.07732)Cited by:[Appendix D](https://arxiv.org/html/2610.41291#A4.p1.1),[§I\.3](https://arxiv.org/html/2610.41291#A9.SS3.p4.1)\.
- Baiet al\.\(2024\)Y\. Bai, S\. Tu, J\. Zhang, H\. Peng, X\. Wang, X\. Lv, S\. Cao, J\. Xu, L\. Hou, Y\. Dong, J\. Tang, and J\. LiLongBench v2: towards deeper understanding and reasoning on realistic long\-context multitasks\.arXiv preprint arXiv:2412\.15204\.External Links:[Link](https://arxiv.org/abs/2412.15204)Cited by:[Appendix D](https://arxiv.org/html/2610.41291#A4.p1.1),[§I\.3](https://arxiv.org/html/2610.41291#A9.SS3.p2.1)\.
- Chenet al\.\(2025\)S\. Chen, Y\. Li, Z\. Xu, Y\. Zeng, S\. Wu, X\. Hu, Z\. Shan, X\. Su, J\. Tang, Y\. Li, and H\. ZhengDAST: context\-aware compression in LLMs via dynamic allocation of soft tokens\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 20544–20552\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.1055),[Link](https://aclanthology.org/2025.findings-acl.1055/)Cited by:[§3](https://arxiv.org/html/2610.41291#S3.p2.1)\.
- Chevalieret al\.\(2023\)A\. Chevalier, A\. Wettig, A\. Ajith, and D\. ChenAdapting language models to compress contexts\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 3829–3846\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.232),[Link](https://aclanthology.org/2023.emnlp-main.232)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§3](https://arxiv.org/html/2610.41291#S3.p2.1)\.
- Donget al\.\(2026\)Y\. Dong, J\. Qian, H\. Zhang, P\. Wang, B\. Li, Z\. Jin, Y\. Li, G\. Li, X\. Yang, and X\. JiangAgentic Reverse Engineering: from I/O to code with discovery agent\.arXiv preprint arXiv:2605\.15334\.External Links:[Link](https://arxiv.org/abs/2605.15334)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p1.1)\.
- Geet al\.\(2024\)T\. Ge, J\. Hu, L\. Wang, X\. Wang, S\. Chen, and F\. WeiIn\-context autoencoder for context compression in a large language model\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=uREj4ZuGJE)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§3](https://arxiv.org/html/2610.41291#S3.p2.1)\.
- Heet al\.\(2026\)X\. He, Q\. Shi, Z\. Sheng, Y\. Tao, H\. Liu, G\. Li, X\. Jiang, and Y\. DongArcher: adaptive reuse of cached hidden states for efficient rollback in diffusion language models\.arXiv preprint arXiv:2608\.08086\.External Links:[Link](https://arxiv.org/abs/2608.08086)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p7.1),[§I\.3](https://arxiv.org/html/2610.41291#A9.SS3.p3.1),[§3](https://arxiv.org/html/2610.41291#S3.p3.1)\.
- Hsiehet al\.\(2024\)C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. GinsburgRULER: what’s the real context size of your long\-context language models?\.arXiv preprint arXiv:2404\.06654\.External Links:[Link](https://arxiv.org/abs/2404.06654)Cited by:[Appendix D](https://arxiv.org/html/2610.41291#A4.p1.1)\.
- Hsuet al\.\(2026\)C\. Hsu, K\. Gong, Y\. Wu, X\. Chen, M\. Liu, J\. Li, H\. Li, Z\. Liu, N\. Tang, J\. Chi, Z\. Fan, X\. He, X\. Yang, X\. Jiang, and Y\. DongOpenCollab: a multi\-agent coding framework with programmable collaboration and controllable runtime\.arXiv preprint arXiv:2609\.38345\.External Links:[Link](https://arxiv.org/abs/2609.38345)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p1.1)\.
- Jianget al\.\(2023\)H\. Jiang, Q\. Wu, C\. Lin, Y\. Yang, and L\. QiuLLMLingua: compressing prompts for accelerated inference of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 13358–13376\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.825),[Link](https://aclanthology.org/2023.emnlp-main.825)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p5.1),[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.41291#S2.SS1.p1.1),[§3](https://arxiv.org/html/2610.41291#S3.p1.1),[§5](https://arxiv.org/html/2610.41291#S5.p1.1)\.
- Jianget al\.\(2024\)H\. Jiang, Q\. Wu, X\. Luo, D\. Li, C\. Lin, Y\. Yang, and L\. QiuLongLLMLingua: accelerating and enhancing LLMs in long context scenarios via prompt compression\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1658–1677\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.91),[Link](https://aclanthology.org/2024.acl-long.91)Cited by:[§3](https://arxiv.org/html/2610.41291#S3.p1.1)\.
- Kanget al\.\(2026\)M\. Kang, W\. Chen, D\. Han, H\. A\. Inan, L\. Wutschitz, Y\. Chen, R\. Sim, and S\. RajmohanACON: optimizing context compression for long\-horizon LLM agents\.InProceedings of the 43rd International Conference on Machine Learning,External Links:[Link](https://arxiv.org/abs/2510.00615)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p1.1),[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p5.1),[§I\.2](https://arxiv.org/html/2610.41291#A9.SS2.p1.1),[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.41291#S2.SS1.p1.1),[§2\.3](https://arxiv.org/html/2610.41291#S2.SS3.p1.1),[§3](https://arxiv.org/html/2610.41291#S3.p1.1),[§5](https://arxiv.org/html/2610.41291#S5.p1.1)\.
- Kwiatkowskiet al\.\(2019\)T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee, K\. Toutanova, L\. Jones, M\. Kelcey, M\. Chang, A\. M\. Dai, J\. Uszkoreit, Q\. Le, and S\. PetrovNatural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 452–466\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00276),[Link](https://aclanthology.org/Q19-1026/)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p1.1)\.
- Liet al\.\(2026\)A\. Li, S\. McLeish, H\. Chen, N\. Kalra, Z\. Chen, A\. Gazizov, V\. A\. S\. K\. Morisetty, B\. Kailkhura, H\. Menon, Z\. Liu, B\. R\. Bartoldson, T\. Goldstein, S\. Lotfi, M\. Goldblum, and P\. IzmailovEnd\-to\-end context compression at scale\.arXiv preprint arXiv:2606\.09659\.External Links:[Link](https://arxiv.org/abs/2606.09659)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p5.1),[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§2\.3](https://arxiv.org/html/2610.41291#S2.SS3.p1.1),[§3](https://arxiv.org/html/2610.41291#S3.p2.1),[§5](https://arxiv.org/html/2610.41291#S5.p1.1)\.
- Liet al\.\(2023\)Y\. Li, B\. Dong, F\. Guerin, and C\. LinCompressing context to enhance inference efficiency of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 6342–6353\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.391),[Link](https://aclanthology.org/2023.emnlp-main.391/)Cited by:[§3](https://arxiv.org/html/2610.41291#S3.p1.1)\.
- Liet al\.\(2024\)Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. ChenSnapKV: LLM knows what you are looking for before generation\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/28ab418242603e0f7323e54185d19bde-Abstract-Conference.html)Cited by:[§3](https://arxiv.org/html/2610.41291#S3.p3.1)\.
- Liuet al\.\(2026a\)M\. Liu, W\. Wang, H\. Nan, Y\. Wu, P\. Li, S\. Huang, X\. He, X\. Jiang, and Y\. DongSPROUT: scaling agentic reinforcement learning for long\-horizon software engineering\.arXiv preprint arXiv:2610\.14405\.External Links:[Link](https://arxiv.org/abs/2610.14405)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p1.1)\.
- Liuet al\.\(2026b\)S\. Liu, B\. Jiang, J\. Yang, Y\. Li, J\. Guo, X\. Liu, and B\. DaiContext as a tool: context management for long\-horizon SWE\-agents\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 20604–20617\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1032),[Link](https://aclanthology.org/2026.findings-acl.1032/)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.41291#S2.SS1.p1.1),[§3](https://arxiv.org/html/2610.41291#S3.p1.1)\.
- Moonshot AI \(2025\)Moonshot AIKimi\-Linear\-48B\-A3B\-Instruct\.Note:Hugging Face model cardExternal Links:[Link](https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct/tree/e1df551a447157d4658b573f9a695d57658590e9)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p3.1)\.
- Moonshot AI \(2026\)Moonshot AIKimi K3\.Note:Official model releaseExternal Links:[Link](https://www.kimi.com/news/kimi-k3)Cited by:[§2\.2](https://arxiv.org/html/2610.41291#S2.SS2.p1.1)\.
- Muet al\.\(2023\)J\. Mu, X\. L\. Li, and N\. GoodmanLearning to compress prompts with gist tokens\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3d77c6dcc7f143aa2154e7f4d5e22d68-Abstract.html)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§3](https://arxiv.org/html/2610.41291#S3.p2.1)\.
- OpenAI \(2026\)OpenAIGPT\-5\.6 Sol\.Note:Official model documentationExternal Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.6-sol)Cited by:[§2\.2](https://arxiv.org/html/2610.41291#S2.SS2.p1.1)\.
- Panet al\.\(2024\)Z\. Pan, Q\. Wu, H\. Jiang, M\. Xia, X\. Luo, J\. Zhang, Q\. Lin, V\. Rühle, Y\. Yang, C\. Lin, H\. V\. Zhao, L\. Qiu, and D\. ZhangLLMLingua\-2: data distillation for efficient and faithful task\-agnostic prompt compression\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 963–981\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.57),[Link](https://aclanthology.org/2024.findings-acl.57)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p5.1),[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.41291#S2.SS1.p1.1),[§3](https://arxiv.org/html/2610.41291#S3.p1.1),[§5](https://arxiv.org/html/2610.41291#S5.p1.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.5\-9B\.Note:Hugging Face model cardExternal Links:[Link](https://huggingface.co/Qwen/Qwen3.5-9B/tree/c202236235762e1c871ad0ccb60c8ee5ba337b9a)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p3.1)\.
- Renet al\.\(2023\)S\. Ren, Q\. Jia, and K\. ZhuContext compression for auto\-regressive transformers with sentinel tokens\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 12860–12867\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.794),[Link](https://aclanthology.org/2023.emnlp-main.794/)Cited by:[§3](https://arxiv.org/html/2610.41291#S3.p2.1)\.
- Trivediet al\.\(2024\)H\. Trivedi, T\. Khot, M\. Hartmann, R\. Manku, V\. Dong, E\. Li, S\. Gupta, A\. Sabharwal, and N\. BalasubramanianAppWorld: a controllable world of apps and people for benchmarking interactive coding agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 16022–16076\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.850),[Link](https://aclanthology.org/2024.acl-long.850/)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p1.1),[§5](https://arxiv.org/html/2610.41291#S5.p1.1)\.
- Wanget al\.\(2024\)Z\. Wang, Y\. Cui, L\. Zhong, Z\. Zhang, D\. Yin, B\. Y\. Lin, and J\. ShangOfficeBench: benchmarking language agents across multiple applications for office automation\.arXiv preprint arXiv:2407\.19056\.External Links:[Link](https://arxiv.org/abs/2407.19056)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p1.1),[§5](https://arxiv.org/html/2610.41291#S5.p1.1)\.
- Xuet al\.\(2024a\)F\. Xu, W\. Shi, and E\. ChoiRECOMP: improving retrieval\-augmented LMs with context compression and selective augmentation\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/bda88ed2892f5e61c9a9bf215c566913-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p3.1),[§3](https://arxiv.org/html/2610.41291#S3.p1.1)\.
- Xuet al\.\(2024b\)Y\. Xu, Y\. Feng, H\. Mu, Y\. Hou, Y\. Li, X\. Wang, W\. Zhong, Z\. Li, D\. Tu, Q\. Zhu, M\. Zhang, and W\. CheConcise and precise context compression for tool\-using language models\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 16430–16441\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.974),[Link](https://aclanthology.org/2024.findings-acl.974/)Cited by:[§3](https://arxiv.org/html/2610.41291#S3.p2.1)\.
- Yanget al\.\(2025\)S\. Yang, J\. Kautz, and A\. HatamizadehGated delta networks: improving Mamba2 with delta rule\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2412.06464)Cited by:[§1](https://arxiv.org/html/2610.41291#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.41291#S2.SS3.p2.1)\.
- Yeet al\.\(2026\)H\. Ye, X\. He, V\. Arak, H\. Dong, and G\. SongMeta context engineering via agentic skill evolution\.arXiv preprint arXiv:2601\.21557\.External Links:[Link](https://arxiv.org/abs/2601.21557)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p7.1),[§3](https://arxiv.org/html/2610.41291#S3.p1.1)\.
- Z\.AI \(2026\)Z\.AIGLM\-5\.2: built for long\-horizon tasks\.Note:Official model releaseExternal Links:[Link](https://z.ai/blog/glm-5.2)Cited by:[§2\.2](https://arxiv.org/html/2610.41291#S2.SS2.p1.1)\.
- Zhanget al\.\(2025\)P\. Zhang, Z\. Liu, S\. Xiao, N\. Shao, Q\. Ye, and Z\. DouLong context compression with activation beacon\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/fc797c61eb18f7b3ce9a74af0ef0d876-Abstract-Conference.html)Cited by:[§3](https://arxiv.org/html/2610.41291#S3.p2.1)\.
- Zhanget al\.\(2023\)Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett, Z\. Wang, and B\. ChenH2\{\}\_\{2\}O: heavy\-hitter oracle for efficient generative inference of large language models\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/6ceefa7b15572587b78ecfcebb2827f8-Abstract-Conference.html)Cited by:[§3](https://arxiv.org/html/2610.41291#S3.p3.1)\.
- Zhouet al\.\(2026\)Z\. Zhou, A\. Qu, Z\. Wu, S\. Kim, A\. Prakash, D\. Rus, B\. K\. H\. Low, and P\. P\. LiangMEM1: learning to synergize memory and reasoning for efficient long\-horizon agents\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=XY8AaxDSLb)Cited by:[§I\.1](https://arxiv.org/html/2610.41291#A9.SS1.p1.1),[§3](https://arxiv.org/html/2610.41291#S3.p1.1),[§5](https://arxiv.org/html/2610.41291#S5.p1.1)\.

###### Appendix Contents

1. [1Introduction](https://arxiv.org/html/2610.41291#S1)
2. [2Motivation](https://arxiv.org/html/2610.41291#S2)1. [2\.1In\-Model Soft Tokens](https://arxiv.org/html/2610.41291#S2.SS1) 2. [2\.2Tool Observations](https://arxiv.org/html/2610.41291#S2.SS2) 3. [2\.3Model\-Native Selection](https://arxiv.org/html/2610.41291#S2.SS3)
3. [3Related Work](https://arxiv.org/html/2610.41291#S3)
4. [4Method](https://arxiv.org/html/2610.41291#S4)1. [4\.1Agentic Soft\-Token Compression](https://arxiv.org/html/2610.41291#S4.SS1) 2. [4\.2Theoretical Analysis](https://arxiv.org/html/2610.41291#S4.SS2)
5. [5Experiments](https://arxiv.org/html/2610.41291#S5)1. [5\.1Main Results](https://arxiv.org/html/2610.41291#S5.SS1) 2. [5\.2Model\-Native Selector Controls](https://arxiv.org/html/2610.41291#S5.SS2) 3. [5\.3Persistence After One Commit](https://arxiv.org/html/2610.41291#S5.SS3) 4. [5\.4Extended Context Compression](https://arxiv.org/html/2610.41291#S5.SS4) 5. [5\.5Cross\-Model Transfer](https://arxiv.org/html/2610.41291#S5.SS5)
6. [6Conclusion](https://arxiv.org/html/2610.41291#S6)
7. [References](https://arxiv.org/html/2610.41291#bib)
8. [ATheoretical Derivations](https://arxiv.org/html/2610.41291#A1)1. [A\.1Surrogate\-Objective Mismatch](https://arxiv.org/html/2610.41291#A1.SS1) 2. [A\.2Model\-Native Selection Bounds](https://arxiv.org/html/2610.41291#A1.SS2) 3. [A\.3Downstream Attention Stability](https://arxiv.org/html/2610.41291#A1.SS3) 4. [A\.4Downstream Recurrent Stability](https://arxiv.org/html/2610.41291#A1.SS4) 5. [A\.5Continuous Representation at a Fixed Budget](https://arxiv.org/html/2610.41291#A1.SS5) 6. [A\.6Proof of Theorem](https://arxiv.org/html/2610.41291#A1.SS6) 7. [A\.7Proof of Lemma](https://arxiv.org/html/2610.41291#A1.SS7)
9. [BAgent Trajectory Examples](https://arxiv.org/html/2610.41291#A2)
10. [CComponent Ablations](https://arxiv.org/html/2610.41291#A3)
11. [DAdditional Dataset Experiments](https://arxiv.org/html/2610.41291#A4)
12. [EExtended Context Compression Details](https://arxiv.org/html/2610.41291#A5)
13. [FCross\-Model Transfer Details](https://arxiv.org/html/2610.41291#A6)
14. [GSupplementary Selector\-Control Analysis](https://arxiv.org/html/2610.41291#A7)1. [G\.1Workload Breakdown and Cost per Successful Task](https://arxiv.org/html/2610.41291#A7.SS1) 2. [G\.2Paired Outcomes and the Distribution of Workload Savings](https://arxiv.org/html/2610.41291#A7.SS2) 3. [G\.3Structural Savings Within Each Policy’s Own Trajectories](https://arxiv.org/html/2610.41291#A7.SS3)
15. [HSupplementary One\-Commit Persistence Analysis](https://arxiv.org/html/2610.41291#A8)1. [H\.1Fidelity Across All Observed Post\-Commit Actions](https://arxiv.org/html/2610.41291#A8.SS1) 2. [H\.2Persistence in a Fixed Task Cohort](https://arxiv.org/html/2610.41291#A8.SS2) 3. [H\.3Cumulative Savings and Cache Consistency](https://arxiv.org/html/2610.41291#A8.SS3)
16. [IExperimental Setup and Protocols](https://arxiv.org/html/2610.41291#A9)1. [I\.1General Setup](https://arxiv.org/html/2610.41291#A9.SS1) 2. [I\.2Benchmark\-Specific Configurations](https://arxiv.org/html/2610.41291#A9.SS2) 3. [I\.3Additional\-Dataset Protocols](https://arxiv.org/html/2610.41291#A9.SS3)

## Appendix ATheoretical Derivations

The distributional comparisons below fix an agent history, a next\-token prefix, and a compression boundary while preserving original logical positions\. They bound one conditional next\-token distribution and do not claim that independently generated trajectories remain identical\. The workload proof separately counts later queries while the compressed cache persists, and the attention\-mass proof fixes one downstream query\.

### A\.1 Surrogate\-Objective Mismatch

Let𝒜k\\mathcal\{A\}\_\{k\}be the same nonempty feasible set for both objectives, and assume the minima below exist\. WriteD⁡\(Z\)=Dθ​\(H,Z\)D\(Z\)=D\_\{\\theta\}\(H,Z\)andJ​\(Z\)=Jtr​\(H,Z\)J\(Z\)=J\_\{\\mathrm\{tr\}\}\(H,Z\)\. IfsupZ∈𝒜k\|D⁡\(Z\)−J⁡\(Z\)\|≤ε\\sup\_\{Z\\in\\mathcal\{A\}\_\{k\}\}\|D\(Z\)\-J\(Z\)\|\\leq\\varepsilonandZtr∈𝒜kZ\_\{\\mathrm\{tr\}\}\\in\\mathcal\{A\}\_\{k\}satisfiesJ⁡\(Ztr\)≤infZ∈𝒜kJ⁡\(Z\)\+ξJ\(Z\_\{\\mathrm\{tr\}\}\)\\leq\\inf\_\{Z\\in\\mathcal\{A\}\_\{k\}\}J\(Z\)\+\\xi, then, forZθ⋆∈arg⁡minZ∈𝒜k⁡D⁡\(Z\)Z\_\{\\theta\}^\{\\star\}\\in\\arg\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}\}D\(Z\),

D⁡\(Ztr\)\\displaystyle D\(Z\_\{\\mathrm\{tr\}\}\)≤J⁡\(Ztr\)\+ε\\displaystyle\\leq J\(Z\_\{\\mathrm\{tr\}\}\)\+\\varepsilon\(12\)≤J⁡\(Zθ⋆\)\+ξ\+ε\\displaystyle\\leq J\(Z\_\{\\theta\}^\{\\star\}\)\+\\xi\+\\varepsilon≤D⁡\(Zθ⋆\)\+2​ε\+ξ\.\\displaystyle\\leq D\(Z\_\{\\theta\}^\{\\star\}\)\+2\\varepsilon\+\\xi\.Optimality ofZθ⋆Z\_\{\\theta\}^\{\\star\}gives the nonnegative lower endpoint\. IfJ=DJ=Dandξ=0\\xi=0, a trained method can attain the optimum\. Thus the upper bound separates objective mismatch from optimization error but gives no lower bound on the realized gap and does not order ASTC and trained methods\. It also does not apply across unequal feasible sets, such as unrestricted soft states and text\-reachable hard states\.

### A\.2 Model\-Native Selection Bounds

WriteDv=∑hdv,hD\_\{v\}=\\sum\_\{h\}d\_\{v,h\}for the total value width across GDN heads\. For each headhh, the update𝐒i,h=αi,h​𝐒i−1,h\+βi,h​𝜹i,h​𝐤i,h⊤\\mathbf\{S\}\_\{i,h\}=\\alpha\_\{i,h\}\\mathbf\{S\}\_\{i\-1,h\}\+\\beta\_\{i,h\}\\bm\{\\delta\}\_\{i,h\}\\mathbf\{k\}\_\{i,h\}^\{\\top\}withβi,h≥0\\beta\_\{i,h\}\\geq 0gives the rank\-one identity

‖𝐒i,h−αi,h​𝐒i−1,h‖F=βi,h​‖𝜹i,h‖2​‖𝐤i,h‖2\.\\\|\\mathbf\{S\}\_\{i,h\}\-\\alpha\_\{i,h\}\\mathbf\{S\}\_\{i\-1,h\}\\\|\_\{F\}=\\beta\_\{i,h\}\\\|\\bm\{\\delta\}\_\{i,h\}\\\|\_\{2\}\\\|\\mathbf\{k\}\_\{i,h\}\\\|\_\{2\}\.\(13\)The implemented RMS aggregates values across heads\. Hence‖𝜹i,h‖2≤‖𝜹i‖2=Dv​χi​\(RMS⁡\(𝐯i\)\+ϵ\)\\\|\\bm\{\\delta\}\_\{i,h\}\\\|\_\{2\}\\leq\\\|\\bm\{\\delta\}\_\{i\}\\\|\_\{2\}=\\sqrt\{D\_\{v\}\}\\,\\chi\_\{i\}\(\\operatorname\{RMS\}\(\\mathbf\{v\}\_\{i\}\)\+\\epsilon\)\. Accepting a position withχi<τχ\\chi\_\{i\}<\\tau\_\{\\chi\}andψi<τψ\\psi\_\{i\}<\\tau\_\{\\psi\}therefore implies

‖𝐒i,h−αi,h​𝐒i−1,h‖F\\displaystyle\\\|\\mathbf\{S\}\_\{i,h\}\-\\alpha\_\{i,h\}\\mathbf\{S\}\_\{i\-1,h\}\\\|\_\{F\}<βi,h​τχ​Dv​\(RMS⁡\(𝐯i\)\+ϵ\)​‖𝐤i,h‖2,\\displaystyle<\\beta\_\{i,h\}\\tau\_\{\\chi\}\\sqrt\{D\_\{v\}\}\(\\operatorname\{RMS\}\(\\mathbf\{v\}\_\{i\}\)\+\\epsilon\)\\\|\\mathbf\{k\}\_\{i,h\}\\\|\_\{2\},\(14\)‖𝐯i−𝐯^i‖2\\displaystyle\\\|\\mathbf\{v\}\_\{i\}\-\\widehat\{\\mathbf\{v\}\}\_\{i\}\\\|\_\{2\}<τψ​Dv​\(RMS⁡\(𝐯i\)\+ϵ\)\.\\displaystyle<\\tau\_\{\\psi\}\\sqrt\{D\_\{v\}\}\(\\operatorname\{RMS\}\(\\mathbf\{v\}\_\{i\}\)\+\\epsilon\)\.Both are local to the GDN state and the specified endpoint reader\. They do not constrain the attention scores of the same token\. Pre\-boundary layers process every original token and preserve their recurrent state, so, conditional on the same future input tokens, that state is unchanged by the boundary replacement\. The remaining possible perturbation enters through the compressed downstream sequence and its attention caches\.

### A\.3 Downstream Attention Stability

Consider one attention head, the same fixed query vector in both runs after the greatest logical position of a candidate groupGG, and its replacement\. Assume at least one retained position exists and all retained keys and values are unchanged in this local comparison\. LetRRbe the other positions visible to this query, and defineWG=∑i∈GesiW\_\{G\}=\\sum\_\{i\\in G\}e^\{s\_\{i\}\},WR=∑i∈ResiW\_\{R\}=\\sum\_\{i\\in R\}e^\{s\_\{i\}\}, andwz=eszw\_\{z\}=e^\{s\_\{z\}\}, wheresis\_\{i\}andszs\_\{z\}are the original and replacement attention scores for this query\. Define

aG=WGWR\+WG,bG=wzWR\+wz,𝐯¯G=∑i∈Gesi​𝐯iWG,𝐯¯R=∑i∈Resi​𝐯iWR\.\\begin\{gathered\}a\_\{G\}=\\frac\{W\_\{G\}\}\{W\_\{R\}\+W\_\{G\}\},\\qquad b\_\{G\}=\\frac\{w\_\{z\}\}\{W\_\{R\}\+w\_\{z\}\},\\\\ \\bar\{\\mathbf\{v\}\}\_\{G\}=\\frac\{\\sum\_\{i\\in G\}e^\{s\_\{i\}\}\\mathbf\{v\}\_\{i\}\}\{W\_\{G\}\},\\qquad\\bar\{\\mathbf\{v\}\}\_\{R\}=\\frac\{\\sum\_\{i\\in R\}e^\{s\_\{i\}\}\\mathbf\{v\}\_\{i\}\}\{W\_\{R\}\}\.\\end\{gathered\}\(15\)The two attention outputs and their difference are exactly

𝐨\\displaystyle\\mathbf\{o\}=\(1−aG\)​𝐯¯R\+aG​𝐯¯G,\\displaystyle=\(1\-a\_\{G\}\)\\bar\{\\mathbf\{v\}\}\_\{R\}\+a\_\{G\}\\bar\{\\mathbf\{v\}\}\_\{G\},\(16\)𝐨~\\displaystyle\\widetilde\{\\mathbf\{o\}\}=\(1−bG\)​𝐯¯R\+bG​𝐯z,\\displaystyle=\(1\-b\_\{G\}\)\\bar\{\\mathbf\{v\}\}\_\{R\}\+b\_\{G\}\\mathbf\{v\}\_\{z\},𝐨~−𝐨\\displaystyle\\widetilde\{\\mathbf\{o\}\}\-\\mathbf\{o\}=\(bG−aG\)​\(𝐯z−𝐯¯R\)\+aG​\(𝐯z−𝐯¯G\)\.\\displaystyle=\(b\_\{G\}\-a\_\{G\}\)\(\\mathbf\{v\}\_\{z\}\-\\bar\{\\mathbf\{v\}\}\_\{R\}\)\+a\_\{G\}\(\\mathbf\{v\}\_\{z\}\-\\bar\{\\mathbf\{v\}\}\_\{G\}\)\.The triangle inequality gives‖𝐨~−𝐨‖2≤\|bG−aG\|​‖𝐯z−𝐯¯R‖2\+aG​‖𝐯z−𝐯¯G‖2\\\|\\widetilde\{\\mathbf\{o\}\}\-\\mathbf\{o\}\\\|\_\{2\}\\leq\|b\_\{G\}\-a\_\{G\}\|\\\|\\mathbf\{v\}\_\{z\}\-\\bar\{\\mathbf\{v\}\}\_\{R\}\\\|\_\{2\}\+a\_\{G\}\\\|\\mathbf\{v\}\_\{z\}\-\\bar\{\\mathbf\{v\}\}\_\{G\}\\\|\_\{2\}\. If the value map isLVL\_\{V\}\-Lipschitz on the group and its representative, and every member is within hidden\-state radiusrGr\_\{G\}of the representative, the second term is at mostaG​LV​rGa\_\{G\}L\_\{V\}r\_\{G\}; the attention\-mass term remains separate\.

The attention\-mass condition cannot be omitted\. As a counterexample, takeg\>1g\>1identical group values𝐯G\\mathbf\{v\}\_\{G\}, one retained value𝐯R≠𝐯G\\mathbf\{v\}\_\{R\}\\neq\\mathbf\{v\}\_\{G\}, and equal attention scores\. If the soft representative has value𝐯G\\mathbf\{v\}\_\{G\}and the same score, thenaG=g/\(g\+1\)a\_\{G\}=g/\(g\+1\)butbG=1/2b\_\{G\}=1/2, so the two outputs differ despite zero within\-group reconstruction error\. The GDN innovation and endpoint read error can also be zero in a hybrid model whose attention projections are independent of that state\. Thus the native tests alone cannot certify unchanged downstream decisions\.

### A\.4 Downstream Recurrent Stability

Post\-boundary GDN layers also consume the shortened sequence\. Consider one such layer and align the original and compressed runs at the logical endpointtGt\_\{G\}of a groupGG\. Starting from the same state, let their states there differ byeGe\_\{G\}\. For later tokens that supply identical keys, values, and gates to this layer, the GDN recurrence has the affine form

𝐒t=αt​𝐒t−1​\(𝐈−βt​𝐤t​𝐤t⊤\)\+βt​𝐯t​𝐤t⊤\.\\mathbf\{S\}\_\{t\}=\\alpha\_\{t\}\\mathbf\{S\}\_\{t\-1\}\(\\mathbf\{I\}\-\\beta\_\{t\}\\mathbf\{k\}\_\{t\}\\mathbf\{k\}\_\{t\}^\{\\top\}\)\+\\beta\_\{t\}\\mathbf\{v\}\_\{t\}\\mathbf\{k\}\_\{t\}^\{\\top\}\.\(17\)Its state map isρt\\rho\_\{t\}\-Lipschitz in Frobenius norm withρt=αt​‖𝐈−βt​𝐤t​𝐤t⊤‖2\\rho\_\{t\}=\\alpha\_\{t\}\\\|\\mathbf\{I\}\-\\beta\_\{t\}\\mathbf\{k\}\_\{t\}\\mathbf\{k\}\_\{t\}^\{\\top\}\\\|\_\{2\}\. Induction therefore gives, aftermmsuch tokens,

‖𝐒tG\+m−𝐒~tG\+m‖F≤eG​∏u=1mρtG\+u\.\\\|\\mathbf\{S\}\_\{t\_\{G\}\+m\}\-\\widetilde\{\\mathbf\{S\}\}\_\{t\_\{G\}\+m\}\\\|\_\{F\}\\leq e\_\{G\}\\prod\_\{u=1\}^\{m\}\\rho\_\{t\_\{G\}\+u\}\.\(18\)If‖𝐤t‖2≤1\\\|\\mathbf\{k\}\_\{t\}\\\|\_\{2\}\\leq 1,0≤βt≤10\\leq\\beta\_\{t\}\\leq 1, and0≤αt≤10\\leq\\alpha\_\{t\}\\leq 1, thenρt≤αt≤1\\rho\_\{t\}\\leq\\alpha\_\{t\}\\leq 1\. This result is conditional on identical later inputs to that GDN layer\. It does not bound the initial defecteGe\_\{G\}, which concerns a different downstream layer from the GDN used to computeχ\\chiandψ\\psi\.

For a full\-model statement, assume an alignment between the two finite causal computations at every group endpoint\. Assume further that they can be joined by a sequence of hybrids in which each adjacent pair differs at one affected attention output or downstream recurrent state, with all its upstream inputs equal\. Leteje\_\{j\}bound that local difference \(using the attention bound above or a state\-defect bound\), and let the map from that quantity to the next\-token logits beLjL\_\{j\}\-Lipschitz on the segment between the two values\. Applying the triangle inequality to the successive hybrid logits proves

‖𝐥~−𝐥‖∞≤E:=∑jLj​ej\.\\\|\\widetilde\{\\mathbf\{l\}\}\-\\mathbf\{l\}\\\|\_\{\\infty\}\\leq E:=\\sum\_\{j\}L\_\{j\}e\_\{j\}\.\(19\)Because log\-sum\-exp is 1\-Lipschitz in the infinity norm, each log\-probability changes by at most2​E2EandDKL\(p∥p~\)≤2ED\_\{\\mathrm\{KL\}\}\(p\\\|\\widetilde\{p\}\)\\leq 2E\. If the original largest logit exceeds the second largest by more than2​E2E, its index remains the greedy next token\. This theorem is conditional:χ\\chi,ψ\\psi, and the pooling threshold alone do not supply alleje\_\{j\}orLjL\_\{j\}\. It concerns the fixed next\-token prefix and does not extend to independently generated trajectories without further stability assumptions\.

### A\.5 Continuous Representation at a Fixed Budget

Fixkkand logical position IDsπ\\piat the compression boundary; hold the cached prefix and all untouched boundary states fixed\. The notationF0:ℓ⋆\(t1:k;π\)F\_\{0:\\ell^\{\\star\}\}\(t\_\{1:k\};\\pi\)evaluates all candidate sequences at those same positions\. For a finite vocabulary𝒱\\mathcal\{V\}, the hard\-token reachable set𝒜khard=\{F0:ℓ⋆\(t1:k;π\):t1:k∈𝒱k\}\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}=\\\{F\_\{0:\\ell^\{\\star\}\}\(t\_\{1:k\};\\pi\):t\_\{1:k\}\\in\\mathcal\{V\}^\{k\}\\\}contains at most\|𝒱\|k\|\\mathcal\{V\}\|^\{k\}states\. In the idealized continuous comparison,𝒜ksoft=ℝk×d\\mathcal\{A\}\_\{k\}^\{\\mathrm\{soft\}\}=\\mathbb\{R\}^\{k\\times d\}contains this set\. For any distortionDθD\_\{\\theta\}defined on both sets,

infZ∈𝒜ksoftDθ​\(H,Z\)≤minZ∈𝒜khard⁡Dθ​\(H,Z\)\.\\inf\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{soft\}\}\}D\_\{\\theta\}\(H,Z\)\\leq\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}\}D\_\{\\theta\}\(H,Z\)\.\(20\)This inclusion alone cannot establish a strict advantage: the acting model might assign the same next\-token distribution to a reachable hard state and to the full sequence\.

LetpHp\_\{H\}be the full\-context next\-token distribution andδk\(H\)=minZ∈𝒜khard∥pH−pθ\>ℓ⋆\(⋅∣Z\)∥1\\delta\_\{k\}\(H\)=\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}\}\\\|p\_\{H\}\-p\_\{\\theta\}^\{\>\\ell^\{\\star\}\}\(\\cdot\\mid Z\)\\\|\_\{1\}\. For any two categorical distributionsp,qp,q, Pinsker’s inequality in natural\-log units is

DKL\(p∥q\)≥12∥p−q∥12\.D\_\{\\mathrm\{KL\}\}\(p\\\|q\)\\geq\\tfrac\{1\}\{2\}\\\|p\-q\\\|\_\{1\}^\{2\}\.\(21\)For completeness, takeA=\{i:pi≥qi\}A=\\\{i:p\_\{i\}\\geq q\_\{i\}\\\},a=p⁡\(A\)a=p\(A\), andb=q⁡\(A\)b=q\(A\)\. Then‖p−q‖1=2​\(a−b\)\\\|p\-q\\\|\_\{1\}=2\(a\-b\)\. Grouping outcomes intoAAand its complement cannot increase KL, while the binary KL as a function ofaahas second derivative1/\[a⁡\(1−a\)\]≥41/\[a\(1\-a\)\]\\geq 4and vanishes with zero derivative ata=ba=b\. Hence binary KL is at least2​\(a−b\)22\(a\-b\)^\{2\}, which proves Equation[21](https://arxiv.org/html/2610.41291#A1.E21)\. It follows that

minZ∈𝒜khard⁡Dθ​\(H,Z\)≥12​δk​\(H\)2\.\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}\}D\_\{\\theta\}\(H,Z\)\\geq\\tfrac\{1\}\{2\}\\delta\_\{k\}\(H\)^\{2\}\.\(22\)Ifδk​\(H\)\>0\\delta\_\{k\}\(H\)\>0, no hard\-token sequence of lengthkkreproduces the full\-context next\-token distribution exactly\. The finiteness of the hard output set makes this condition equivalent topHp\_\{H\}being absent from that set\.

For ASTC’s particular outputZASTCZ\_\{\\mathrm\{ASTC\}\}, the conditional logit bound in Equation[19](https://arxiv.org/html/2610.41291#A1.E19)givesDθ​\(H,ZASTC\)≤2​ED\_\{\\theta\}\(H,Z\_\{\\mathrm\{ASTC\}\}\)\\leq 2E\. Therefore

E<δk​\(H\)2/4⟹Dθ​\(H,ZASTC\)<minZ∈𝒜khard⁡Dθ​\(H,Z\)\.E<\\delta\_\{k\}\(H\)^\{2\}/4\\quad\\Longrightarrow\\quad D\_\{\\theta\}\(H,Z\_\{\\mathrm\{ASTC\}\}\)<\\min\_\{Z\\in\\mathcal\{A\}\_\{k\}^\{\\mathrm\{hard\}\}\}D\_\{\\theta\}\(H,Z\)\.\(23\)The statement compares equal physical length, fixed positions, and one next\-token distribution\. Its premise requires an actual bound on all downstream attention and recurrent defects\. The ambient continuous set is larger than ASTC’s attainable mean\-pooled set, and finite\-precision execution further restricts it\. Thus the set inclusion establishes capacity, while Equation[23](https://arxiv.org/html/2610.41291#A1.E23)specifies when one ASTC representation realizes a strict advantage\.

For completeness, the pre\-cast representative in Equation[5](https://arxiv.org/html/2610.41291#S4.E5)is the least\-squares solution\. With𝐡¯G=\|G\|−1​∑i∈G𝐡i\\bar\{\\mathbf\{h\}\}\_\{G\}=\|G\|^\{\-1\}\\sum\_\{i\\in G\}\\mathbf\{h\}\_\{i\}, the identity

∑i∈G‖𝐡i−𝐳‖22=∑i∈G‖𝐡i−𝐡¯G‖22\+\|G\|​‖𝐳−𝐡¯G‖22\\sum\_\{i\\in G\}\\\|\\mathbf\{h\}\_\{i\}\-\\mathbf\{z\}\\\|\_\{2\}^\{2\}=\\sum\_\{i\\in G\}\\\|\\mathbf\{h\}\_\{i\}\-\\bar\{\\mathbf\{h\}\}\_\{G\}\\\|\_\{2\}^\{2\}\+\|G\|\\\|\\mathbf\{z\}\-\\bar\{\\mathbf\{h\}\}\_\{G\}\\\|\_\{2\}^\{2\}\(24\)shows that the FP32 mean minimizes squared hidden\-state error\. This does not minimize downstream KL; ASTC checks reconstruction again after casting\.

### A\.6 Proof of Theorem[4\.1](https://arxiv.org/html/2610.41291#S4.Thmtheorem1)

###### Proof\.

Letppbe the unchanged physical length of the cached prefix andmmthe number of newly processed positions before the commit, sonphys=p\+mn\_\{\\mathrm\{phys\}\}=p\+mand0<r<m0<r<m\. At one full causal attention layer, the new queries create

Qbefore=p​m\+m⁡\(m\+1\)2Q\_\{\\mathrm\{before\}\}=pm\+\\frac\{m\(m\+1\)\}\{2\}\(25\)query–key pairs: each attends theppcached keys and its causal prefix in the new block\. After replacingrrnew positions, the unchanged cached prefix still contributesppkeys, but onlym−rm\-rnew queries remain\. Thus

Qafter=p⁡\(m−r\)\+\(m−r\)​\(m−r\+1\)2\.Q\_\{\\mathrm\{after\}\}=p\(m\-r\)\+\\frac\{\(m\-r\)\(m\-r\+1\)\}\{2\}\.\(26\)Subtracting gives the same\-pass reduction

Qbefore−Qafter=p​r\+r⁡\(2​m−r\+1\)2=r⁡\(2​nphys−r\+1\)2\.Q\_\{\\mathrm\{before\}\}\-Q\_\{\\mathrm\{after\}\}=pr\+\\frac\{r\(2m\-r\+1\)\}\{2\}=\\frac\{r\(2n\_\{\\mathrm\{phys\}\}\-r\+1\)\}\{2\}\.\(27\)The fresh\-prefill case isp=0p=0\. Every later query that actually reaches this attention layer under the same fixed continuation has exactlyrrfewer historical keys while the compared caches retain their length difference;NqN\_\{q\}such queries addr​NqrN\_\{q\}saved pairs\. Summing over theA\>A\_\{\>\}affected full\-attention layers proves Equation[10](https://arxiv.org/html/2610.41291#S4.E10)\. The group positions need not be contiguous because the count depends on the physical length after stable sorting, not on which original positions were merged\. The result excludes work already spent on the cached prefix and assumes no intervening change to the compared cache that alters thisrr\-position difference\. A future token compressed before reaching these layers is not counted inNqN\_\{q\}\. ∎

### A\.7 Proof of Lemma[4\.2](https://arxiv.org/html/2610.41291#S4.Thmtheorem2)

###### Proof\.

The query follows the greatest logical position ofGG, so allggoriginal members are visible in the uncompressed run and their one representative is visible in the compressed run\. Their identical scores give group unnormalized massesg​esge^\{s\}andese^\{s\}, respectively\. Because the retained positions are unchanged, their attention masses are

AR=ZRZR\+g​es,A~R=ZRZR\+es\.A\_\{R\}=\\frac\{Z\_\{R\}\}\{Z\_\{R\}\+ge^\{s\}\},\\qquad\\widetilde\{A\}\_\{R\}=\\frac\{Z\_\{R\}\}\{Z\_\{R\}\+e^\{s\}\}\.\(28\)Direct subtraction yields

A~R−AR=ZR​\(g−1\)​es\(ZR\+es\)​\(ZR\+g​es\)\>0,\\widetilde\{A\}\_\{R\}\-A\_\{R\}=\\frac\{Z\_\{R\}\(g\-1\)e^\{s\}\}\{\(Z\_\{R\}\+e^\{s\}\)\(Z\_\{R\}\+ge^\{s\}\)\}\>0,\(29\)sinceg\>1g\>1andZR\>0Z\_\{R\}\>0\. The two attention outputs are

𝐨=AR​𝐯¯R\+\(1−AR\)​𝐯G,𝐨~=A~R​𝐯¯R\+\(1−A~R\)​𝐯G\.\\mathbf\{o\}=A\_\{R\}\\overline\{\\mathbf\{v\}\}\_\{R\}\+\(1\-A\_\{R\}\)\\mathbf\{v\}\_\{G\},\\qquad\\widetilde\{\\mathbf\{o\}\}=\\widetilde\{A\}\_\{R\}\\overline\{\\mathbf\{v\}\}\_\{R\}\+\(1\-\\widetilde\{A\}\_\{R\}\)\\mathbf\{v\}\_\{G\}\.\(30\)Subtracting and taking the inner product with any𝐮\\mathbf\{u\}gives Equation[11](https://arxiv.org/html/2610.41291#S4.E11)\. Positivity of the projected change requires𝐮⊤​\(𝐯¯R−𝐯G\)\>0\\mathbf\{u\}^\{\\top\}\(\\overline\{\\mathbf\{v\}\}\_\{R\}\-\\mathbf\{v\}\_\{G\}\)\>0; it is not a claim about arbitrary nonlinear decisions\. If scores or values differ, the exact identity in Equation[16](https://arxiv.org/html/2610.41291#A1.E16)separates attention\-mass shift from value error\. ∎

## Appendix BAgent Trajectory Examples

Figure[6](https://arxiv.org/html/2610.41291#A2.F6)shows two AppWorld trajectories\. ASTC retains exact fields as hard tokens and replaces selected history positions with soft tokens\. The reported attention\-pair reductions describe logical computation in these trajectories, not measured wall\-clock speedups\.

Figure 6:ASTC compression in Venmo and Spotify agent trajectories\.
## Appendix CComponent Ablations

To identify which components sustain ASTC’s accuracy–efficiency balance, we remove protection, either GDN selection test \(χ\\chiorψ\\psi\), or soft\-token replacement one at a time on the same AppWorld task set\. All variants keep the frozen model and inference configuration fixed\. Hard deletion uses ASTC’s selector but drops positions instead of emitting soft tokens\.

Table 3:Component ablations on AppWorld\.Removing protection lowers accuracy from 33\.33% to 20\.83% \(21 fewer successes\), even as Attention Pairs fall to 68\.63% of Base; Generated Tokens rise to 127\.05%\. Losing API contracts, identifiers, or exact values can invalidate later actions and lengthen recovery trajectories\.

Withoutχ\\chi, accuracy returns to Base’s 32\.14%, two fewer successes than full ASTC\. Attention Pairs fall to 78\.11%, but Decode KV and Generated Tokens rise relative to full ASTC\. Withoutψ\\psi, accuracy is 32\.74% and all three workload measures rise\. The tests address distinct cases: new writes to recurrent state and values no longer readable at the endpoint\.

Hard deletion loses one success and raises Decode KV, Attention Pairs, and Generated Tokens from ASTC’s 81\.55%, 80\.70%, and 94\.83% to 88\.83%, 86\.85%, and 99\.58%, respectively\. Retaining the group\-level signal as a soft token supports later decisions in this comparison\.

## Appendix DAdditional Dataset Experiments

We evaluate ASTC with the frozen Qwen3\.5\-9B model on RULER 32K\([Hsieh et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib23)\), LongBench v2\([Bai et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib24)\), and interactive code repair on MBPP\([Austin et al\., 2021](https://arxiv.org/html/2610.41291#bib.bib22)\)\. The respective sample counts are 260, 503, and 427\. All comparisons use a matched Base run within each benchmark; the protocols are detailed in Appendix[I\.3](https://arxiv.org/html/2610.41291#A9.SS3)\.

Table 4:Additional\-dataset results\.Figure[7](https://arxiv.org/html/2610.41291#A4.F7)combines these additional\-dataset results with the three agentic benchmarks in Table[1](https://arxiv.org/html/2610.41291#S5.T1)\.

Figure 7:Task quality and inference resources across six datasets with frozen Qwen3\.5\-9B\. Bars show accuracy, EM/F1, or pass rate on the left axis; lines show resource use relative to each dataset’s matched Base on the right axis \(100%\)\. Lower resource use is better\. Panels \(a\)–\(c\) visualize Table[1](https://arxiv.org/html/2610.41291#S5.T1), and panels \(d\)–\(f\) complement Table[4](https://arxiv.org/html/2610.41291#A4.T4)\. LongBench v2 shows cache bytes because Attention Pairs were not reported\. Supplementary generated\-token and cache ratios use the underlying experiment records\.On RULER, ASTC matches Base’s aggregate answer score to the reported precision while reducing Decode KV by 6\.04% and Attention Pairs by 14\.43%\. It migrates 1,045,409 of 6,485,833 eligible tokens into 1,468 soft tokens, for a 12\.56% net prompt\-token reduction\. This 20\-example\-per\-task run is smaller than RULER’s 500\-example reference configuration\.

LongBench v2 yields 206/503 correct for both variants, with four Base\-only and four ASTC\-only successes\. ASTC reduces Decode KV visits by 7\.07%, all\-layer final KV tokens by 6\.19%, and analytical KV\-cache bytes by 6\.98%\. The subgroup changes are mixed \(Base/ASTC: Easy 44\.79/45\.83%, Hard 38\.59/37\.94%; Short 44\.44/45\.56%, Medium 40\.00/39\.53%, Long 37\.04/36\.11%\)\. The 262,144\-token limit truncates 100 examples in each variant; these figures do not establish a wall\-clock speedup\.

On MBPP, ASTC compresses at least one span in 182/427 trajectories and reduces workload; its paired success difference is−0\.70\-0\.70percentage points \(95% bootstrap interval\[−1\.87,0\.23\]\[\-1\.87,0\.23\]\)\. Mean generated tokens per problem are 89\.2/89\.4 on RULER, 2\.0/2\.0 on LongBench, and 337\.4/329\.0 on MBPP \(Base/ASTC\)\. The MBPP run does not establish a success\-rate improvement\.

## Appendix EExtended Context Compression Details

Section[5\.4](https://arxiv.org/html/2610.41291#S5.SS4)compares two extended scopes with an uncompressed replay Base\. Table[5](https://arxiv.org/html/2610.41291#A5.T5)gives the numerical results, while Figure[5](https://arxiv.org/html/2610.41291#S5.F5)also includes the Tool setting from Table[1](https://arxiv.org/html/2610.41291#S5.T1)\.

Table 5:Effect of compression scope on AppWorld\.*Note\.*Base is the uncompressed replay control; costs are normalized to Base \(100%\)\.

All runs use frozen Qwen3\.5\-9B with fixed Stage\-3 calibration and no gradient updates on all 168 AppWorld Test\-Normal tasks\. The data thresholds are\(χ,ψ\)=\(0\.60,0\.80\)\(\\chi,\\psi\)=\(0\.60,0\.80\), with merge\-error threshold0\.500\.50and at most 16 soft tokens per closed region\. Greedy decoding permits 50 actions, 768 tokens per action, and a 32,768\-token logical context\.

Assistant compresses completed assistant outputs; Context additionally permits eligible initial system/user prompts and tool returns\. Both preserve the compact prefix and replay only the newly completed assistant/tool suffix from its pre\-generation prefix snapshot\. The uncompressed replay control uses the same procedure, and all local replay work, failed tasks, and budget\-limited tasks are included\. Special tokens, protocol boundaries, API contracts, authentication content, and protected literals remain explicit\. Base in Table[5](https://arxiv.org/html/2610.41291#A5.T5)denotes this uncompressed replay control, and both compression variants are normalized to it\.

These are two selected configurations from four candidates, evaluated as a full\-split repeated confirmation after earlier Test\-Normal development\. Context’s realized compression is mainly from tool returns, with no initial\-prompt tokens removed\. The workload changes combine compression geometry, generation length, and trajectory changes\. They describe the observed accuracy–workload tradeoff, not a statistical guarantee of unchanged success or a measured speedup\.

## Appendix FCross\-Model Transfer Details

Section[5\.5](https://arxiv.org/html/2610.41291#S5.SS5)reports cross\-model results in Table[2](https://arxiv.org/html/2610.41291#S5.T2)\. Each frozen backbone uses a reader calibrated for its own internal states and a matched uncompressed Base\. Qwen3\.5\-9B and Qwen3\.8\-27B cover all 168 AppWorld Test\-Normal tasks; Kimi\-Linear covers the first 68 in official order\. Within each model, Base and ASTC share task IDs and decoding settings; workload is normalized separately for each comparison\.

Qwen3\.8\-27B uses BF16 and Stage\-5 compression at zero\-based decoder index 18, with a reader calibrated on 190 snapshots from 12 Dev tasks and no weight updates\. The data branch uses\(χ,ψ\)=\(0\.47,0\.67\)\(\\chi,\\psi\)=\(0\.47,0\.67\)and merge\-error threshold0\.340\.34\. The reported sampling stream uses seed20270912\+i20270912\+ifor taskii, shared by Base and ASTC\. The configuration and stream were selected after Test exploration; this is not an independent blind test or an average across seeds\. Costs include failed tasks and are cache\-geometry proxies, not measured memory traffic or latency\.

## Appendix GSupplementary Selector\-Control Analysis

We extend Section[5\.2](https://arxiv.org/html/2610.41291#S5.SS2)with offline summaries of the same 168 Stage\-3 Qwen3\.5\-9B rollouts per policy \(Appendix[I\.2](https://arxiv.org/html/2610.41291#A9.SS2)\)\. Hidden\-only and Random share protection, legal groups, and the midpoint of the attainable budget; Random uses selection seed 101\. ASTC follows its nativeχ/ψ\\chi/\\psicommit path\. Independent actions and tool results allow contexts and budgets to diverge, so this compares complete policies rather than an equal\-budget GDN intervention\.

### G\.1 Workload Breakdown and Cost per Successful Task

Both workload components favor ASTC \(Table[6](https://arxiv.org/html/2610.41291#A7.T6)\): prefill pairs are 12\.30%/19\.01% lower and decode visits 15\.65%/13\.72% lower than Hidden\-only/Random\. Fewer actions and generated tokens also contribute, so the totals combine structural and behavioral effects\.

Table 6:Full\-split selector workload, including failed tasks\. Pair counts are in billions \(10910^\{9\}\)\. Pairs/success divides the total cost of all 168 tasks by the number of official all\-requirements successes\.Including failed attempts, ASTC uses26\.9% and 28\.3% fewer pairs per successful taskthan Hidden\-only and Random\. The denominator varies with success; this is not a mean restricted to successful trajectories\.

### G\.2 Paired Outcomes and the Distribution of Workload Savings

On discordant tasks, ASTC\-only versus control\-only successes are 19:10 against Hidden\-only and 17:10 against Random\. ASTC’s paired success differences are\+5\.36\+5\.36percentage points \(95% CI\[−0\.60,11\.31\]\[\-0\.60,11\.31\]\) and\+4\.17\+4\.17points \(\[−1\.79,10\.12\]\[\-1\.79,10\.12\]\); exact two\-sided McNemarpp\-values are 0\.136 and 0\.248\. Neither success interval excludes zero\.

The workload advantage covers 65\.5%/66\.7% of tasks against Hidden\-only/Random \(Table[7](https://arxiv.org/html/2610.41291#A7.T7)\)\. The exploratory workload intervals describe a different endpoint from task success\.

Table 7:Task\-paired Attention Pairs over all 168 tasks\. Lower/equal/higher compares ASTC with the control\. The difference is ASTC minus control, in millions \(10610^\{6\}\) of pairs per task; brackets give a task\-bootstrap 95% CI\.
### G\.3 Structural Savings Within Each Policy’s Own Trajectories

Table 8:Structural savings on each policy’s own inputs and actions\. Logical counts assume no shortening; actual counts use realized layer lengths \(billions of pairs\)\. Savings are1−∑Pactual/∑Plogical1\-\\sum P\_\{\\mathrm\{actual\}\}/\\sum P\_\{\\mathrm\{logical\}\}, with the analogous calculation for each component\.Scope and uncertainty\.Test\-Normal was used during development\. Intervals use 10,000 task\-bootstrap resamples: success seed 20260923; new percentile workload intervals, sorted task IDs and NumPydefault\_rngseed 20261019\. New summaries are post\-hoc without multiple\-comparison correction\. Structural ratios do not causally decompose cross\-policy totals\. Attention Pairs do not measure total FLOPs or latency\.

## Appendix HSupplementary One\-Commit Persistence Analysis

We extend Section[5\.3](https://arxiv.org/html/2610.41291#S5.SS3)using the completed fixed\-input replays\. The 36 archived Base tasks were selected before inspecting outcomes as the first 36 in a frozen SHA256 ordering\. Of these, 35 admit a first legal, beneficial observation commit and one has no event\. Base and Single build independent caches from the task start and receive identical archived action tokens and tool results at the original prefill and single\-token decode boundaries\. Single commits once and bypasses subsequent observations while retaining the same compressed cache\. Horizon 0 is the committing action; horizonhhis thehhth subsequent action\. Supplementary calculations are offline and require no new model runs\.

### H\.1 Fidelity Across All Observed Post\-Commit Actions

The complete post\-commit interval contains 599 actions and 62,275 reference tokens across 35 triggered tasks, including the committing actions\. Table[9](https://arxiv.org/html/2610.41291#A8.T9)uses all of them, rather than only the six horizons in the main figure\. We compute full\-vocabularyKL\(pBase∥pSingle\)\\mathrm\{KL\}\(p\_\{\\mathrm\{Base\}\}\\\|p\_\{\\mathrm\{Single\}\}\)in FP32, average token metrics within each action, average actions within each task, and then weight the 35 tasks equally\. Top\-1 agreement compares the conditional argmax at each reference\-token position\. Reference\-tokenΔ\\DeltaNLL is Single minus Base, so positive values mean lower reference probability under Single\.

Table 9:Overall fixed\-input replay fidelity on all 35 triggered tasks\. Intervals are task\-bootstrap 95% CIs; actions within a task are not treated as independent samples\. KL andΔ\\DeltaNLL are in nats/token\.In530/599 actions \(88\.48%\), every reference\-token prediction has the same top\-1 in both branches\. The largest action KL, retained in all summaries, is 0\.016267 nats/token \(task2d9f728\_3, zero\-based action 8\)\. The positive overallΔ\\DeltaNLL shows a small measurable distributional change despite high top\-1 agreement\. These are conditional predictions under shared reference prefixes; they do not establish identical autonomous actions, correct tool arguments, or task\-quality equivalence\.

### H\.2 Persistence in a Fixed Task Cohort

The main figure’s task count decreases from 35 at horizon 0 to 13 at horizon 16 as trajectories end\. We therefore repeat the summary using the*same 13 tasks observable through horizon 16*at every displayed horizon \(Table[10](https://arxiv.org/html/2610.41291#A8.T10)\)\. Selection depends only on archived trajectory length, not success or KL\. This post\-hoc sensitivity analysis applies to that conditional cohort\.

Table 10:Fixed\-cohort persistence: the same 13 tasks at every horizon\. Pairs saved are cumulative from the committing action, in millions; the last column averages each task’s cumulative saving fraction\.Within this fixed cohort, mean cumulative savings increase from7\.53M to 16\.92M pairs, and no task’s cumulative savings decrease\. Measured KL remains small at later horizons, so changing task membership alone cannot explain the low late\-horizon KL in these 13 traces\. The saving fraction declines because subsequent total workload grows while the original cache reduction remains fixed\.

### H\.3 Cumulative Savings and Cache Consistency

For each task, cumulative savings through horizonhhare

Sh=∑a=a0a0\+h\(PaBase−PaSingle\),S\_\{h\}=\\sum\_\{a=a\_\{0\}\}^\{a\_\{0\}\+h\}\\left\(P\_\{a\}^\{\\mathrm\{Base\}\}\-P\_\{a\}^\{\\mathrm\{Single\}\}\\right\),\(31\)wherea0a\_\{0\}is the committing action andPaP\_\{a\}includes prefill and decode pairs\. The main figure’s middle panel reports this cumulative quantity, averaged over tasks still observed at each horizon\. Since those sets differ, its cross\-horizon means need not increase; each individual task’s cumulative savings do\. All 599 post\-commit actions have nonnegative pair savings and all 35 triggered tasks have positive final cumulative savings\.

Through their natural endings, the 35 tasks save 1\.152B of 33\.847B Base pairs in the matched post\-commit intervals \(3\.40% in aggregate\)\. Mean final savings are 32\.91M pairs per task \(95% CI \[19\.61, 50\.41\]M\)\. Averaging each task’s saving fraction instead gives 5\.95% \(CI \[3\.95, 8\.27\]%\); this weights tasks equally rather than by workload\. These counts exclude compression overhead and do not establish a net wall\-clock speedup or a measured break\-even point\.

Table 11:Replay consistency audit\. All counts include the full observed records; no action was truncated for scoring\. A forward is one prefill or one single\-token decode call\.Each event removesRRhistorical positions from each of the six downstream attention caches\. MedianRRis 231 \(IQR 72–590\.5\), giving a median of 1,386 fewer historical visits per subsequent query \(IQR 432–3,543\)\. Across all events, 14,302 merged positions become 538 soft tokens, a net reduction of 13,764 positions\. The maximum recorded post\-cast reconstruction error is 0\.4841, below 0\.50 in every event\. The audit checks input\-token identity and cache geometry; it does not require identical protection masks between Base and Single\.

Relation to autonomous task outcomes\.The separate whole\-task one\-commit policy succeeds on 49/168 tasks, compared descriptively with 54/168 archived Base and 56/168 released ASTC\. Released ASTC may commit once for each new observation, rather than once for the entire task\. First\-commit action,RR, and generated token IDs agree with released ASTC on all 160 triggered tasks, but 10 of 37 comparable source files differ between archives\. These scores therefore do not identify a same\-code causal effect of allowing later commits\. They also do not turn fixed\-input fidelity into an autonomous success or noninferiority guarantee\.

Uncertainty and scope\.New overall fidelity and final savings intervals use 10,000 percentile task\-bootstrap resamples, NumPydefault\_rngseed 20261019\. The fixed cohort is exploratory; no multiple\-comparison correction is applied\. Replay measures prediction fidelity on stored inputs, not retrieval accuracy for facts available only in merged observations\.

## Appendix IExperimental Setup and Protocols

### I\.1 General Setup

Benchmarks\.We evaluate ASTC on three long\-horizon agentic benchmarks with distinct interaction structures\. AppWorld provides a simulated ecosystem of everyday applications and APIs for interactive coding agents; we use its full 168\-task Test\-Normal split\([Trivedi et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib17)\)\. OfficeBench covers realistic office\-automation workflows that require planning, application switching, and grounded actions across productivity software\([Wang et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib11)\)\. 8\-Objective QA groups eight Natural Questions into each task and requires an agent to gather evidence with a Wikipedia search tool before returning all eight answers\([Kwiatkowski et al\., 2019](https://arxiv.org/html/2610.41291#bib.bib9);[Zhou et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib10);[Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12)\)\. Together, they test compression across API interaction, office automation, and multi\-question information gathering\.

Implementation\.The three main benchmarks use the frozen Qwen3\.5\-9B acting model\. For Table[1](https://arxiv.org/html/2610.41291#S5.T1), all methods within each benchmark share the acting\-model serving configuration and evaluation protocol\. Base and ASTC also share the agent scaffold, prompts, decoding policy, context limit, and action budget\.

Model checkpoints\.The main experiments useQwen/Qwen3\.5\-9Bat Hugging Face revisionc202236235762e1c871ad0ccb60c8ee5ba337b9a\([Qwen Team, 2026](https://arxiv.org/html/2610.41291#bib.bib5)\)\. The transfer study also usesmoonshotai/Kimi\-Linear\-48B\-A3B\-Instructat revisione1df551a447157d4658b573f9a695d57658590e9\([Moonshot AI, 2025](https://arxiv.org/html/2610.41291#bib.bib6)\)\. The cross\-model study in Section[5\.5](https://arxiv.org/html/2610.41291#S5.SS5)also uses the frozenQwen/Qwen3\.8\-27Bmodel in BF16\.

Bounded pooling\.Selected positions are divided into at mostMMgroups in causal order\. In each group, an existing member closest to the mean direction anchors an initial trim by relativeℓ2\\ell\_\{2\}error\. ASTC then averages the remaining states in FP32, casts the mean to the runtime dtype, and accepts a group of at least two members only if its maximum post\-cast relative error is at mostτh\\tau\_\{h\}\. Rejected and trimmed positions remain hard tokens\. This check bounds hidden\-state reconstruction, not downstream attention scores\.

Baselines\.Base is the uncompressed acting LLM\. We compare ASTC with representative hard\-token approaches, including prompted compression, LLMLingua\([Jiang et al\., 2023](https://arxiv.org/html/2610.41291#bib.bib7)\), LLMLingua\-2\([Pan et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib16)\), and the UT and UTCO variants of ACON\([Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12)\), as well as LCLM, a trained soft\-token compressor\([Li et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib13)\)\. These external\-compressor comparisons are reported on AppWorld and OfficeBench; 8\-Objective QA also includes Prompting, both LLMLingua variants, both ACON variants, and LCLM\. The newly reported AppWorld LLMLingua\-2 and OfficeBench Prompting and LLMLingua runs compress observations; the remaining hard\-token baselines compress the full context, whereas ASTC compresses eligible tool observations in\-model\. All methods that require training use Qwen3\.5\-9B as the backbone and are evaluated with the same benchmark protocol\. End\-to\-end workload includes acting\-model requests, auxiliary compressor\-service calls, compressor\-generated tokens, and repeated prefills\. The 8\-Objective QA counts include the LLMLingua v1 and LLMLingua\-2 service forwards\.

Workload metrics\.Decode KV counts query–cache interactions during autoregressive decoding\. Attention Pairs also includes prefill and auxiliary model calls, while Generated Tokens counts all discrete outputs\. These are analytical counts, not measured latency or GPU operations\. Each workload metric in Table[1](https://arxiv.org/html/2610.41291#S5.T1)is normalized to Base within its benchmark\.

Related context and cache optimization\.Meta Context Engineering \(MCE\) co\-evolves agentic skills and context artifacts for task\-specific context construction\([Ye et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib26)\)\. Archer reuses prompt K/V while recomputing mutable responses in rollback\-capable diffusion language models\([He et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib25)\)\. These works address context construction and rollback\-aware caching, respectively, while ASTC focuses on in\-model compression and persistent cache reuse along agent trajectories\.

### I\.2 Benchmark\-Specific Configurations

For the three main benchmarks in Table[1](https://arxiv.org/html/2610.41291#S5.T1), we follow the benchmark tasks, agent scaffolds, model\-serving settings, inference limits, and evaluation procedures of ACON\([Kang et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib12)\)\. The benchmark settings and ASTC\-specific compression policies are detailed below\.

AppWorld\.We evaluate all 168 AppWorld 0\.1\.3\.post1 Test\-Normal tasks under this ACON\-aligned setup\. The frozen Qwen3\.5\-9B agent uses greedy decoding, at most 50 actions, 768 output tokens per action, and a 32,768\-token logical context\. The model seed is 20260914, and taskiiuses seed20260912\+i20260912\+iin the released task order\. ASTC operates after zero\-based GDN layer 10 and before attention layer 11, retaining at most 16 soft tokens per tool observation\. Its fixed reader is calibrated from 22 records across 12 Train/Dev tasks, without weight updates\. Each head uses leading eigenvectors of trace\-normalized tool\-end state covariances; the ridge and RMS stabilizers areλ=10−4\\lambda=10^\{\-4\}andϵ=10−6\\epsilon=10^\{\-6\}\. The ordinary\-data thresholds areχ<0\.60\\chi<0\.60,ψ<0\.80\\psi<0\.80, and merge error≤0\.50\\leq 0\.50; the conservative catalog thresholds are0\.550\.55,0\.740\.74, and0\.550\.55, respectively\. A commit must save at least 256 downstream token\-layer positions and 5% of eligible tool positions\. API contracts, catalogs, and exact\-value fields are protected\.

OfficeBench\.We use the official Test split, Qwen3\.5\-9B, seed 20260920, and the official environment and evaluator\. The agent uses at most 50 iterations, 768 output tokens per action, and a 32,768\-token context\. OfficeBench uses the same ASTC layer, selection thresholds, merge bound, and persistent cache as AppWorld\. Only later byte\-identical repetitions of an application\-interface scaffold are eligible for compression; the first copy, changed scaffolds, observations, task instructions, system text, and prior actions remain explicit\. Base and ASTC share the prompt and action budget\.

8\-Objective QA\.All methods evaluate the same ACON Test tasks\. The frozen Qwen3\.5\-9B acting model uses BF16 with vLLM 0\.18\.0, a 131,072\-token model limit, engine seed 42, and at most 30 agent iterations for every method\. ASTC compresses eligible tool content at the AppWorld layer boundary; the reported policy uses\(χ,ψ\)=\(0\.55,0\.75\)\(\\chi,\\psi\)=\(0\.55,0\.75\)for ordinary tool data\. EM and F1 score the eight answers per task\. All methods share the same serving and evaluation configuration, and workload is normalized to the corresponding Base run\.

### I\.3 Additional\-Dataset Protocols

RULER\.We evaluate the same 32K inputs in Base and ASTC with the frozen Qwen3\.5\-9B model and a 32,768\-token context limit\. RULER scores answers by task\-specific string matching and averages over its 13 task configurations\. We use 20 examples per task, rather than the reference configuration’s 500\. Thus the reported accuracy is a 260\-example aggregate; matching aggregate scores do not imply identical results on every task or at the larger default sample size\. ASTC uses the Stage\-3 data branch with\(χ,ψ\)=\(0\.60,0\.80\)\(\\chi,\\psi\)=\(0\.60,0\.80\)and merge\-error threshold0\.500\.50\. Decode KV and Attention Pairs are analytical query–cache and causal\-pair counts; Gen\. is generated tokens, all normalized to Base\.

LongBench v2\.We use all 503 four\-choice questions in the publictrainsplit\([Bai et al\., 2024](https://arxiv.org/html/2610.41291#bib.bib24)\)\. Base and ASTC share the frozen Qwen3\.5\-9B model and a 262,144\-token context limit; examples that exceed the limit are truncated before either method processes them\. Of the 503 examples, 100 require truncation\. The inference configuration permits at most 16 new tokens; the recorded runs generate two tokens per example in both variants\. ASTC uses the Stage\-3 catalog branch with\(χ,ψ\)=\(0\.55,0\.74\)\(\\chi,\\psi\)=\(0\.55,0\.74\), merge\-error threshold0\.550\.55, and minimum net removal of one token\. Overall and subgroup accuracy are exact\-match rates for the extracted choice\. Reported KV counts and cache bytes are analytical logical workload measures, not latency or measured peak device memory\.

For MBPP, we use the sanitized task snapshot from Archer\([He et al\., 2026](https://arxiv.org/html/2610.41291#bib.bib25)\)\.

MBPP\.We use its 427 sanitized MBPP problems\([Austin et al\., 2021](https://arxiv.org/html/2610.41291#bib.bib22)\)\. This interactive code\-repair protocol is not numerically comparable with one\-shot MBPP evaluations\. The model submits a complete Python program, receives public\-test feedback, and revises it; a third attempt is allowed if the second fails\. The final program is scored once\. Both variants use greedy decoding, at most 768 generated tokens per attempt, and a 32,768\-token context limit\. ASTC uses its Stage\-3 calibration, thresholds, and 16\-soft\-token bound\. Initial prompts, completed code, and public\-test feedback are eligible for compression, while special control tokens remain explicit\. Both variants re\-prefill the full transcript before each turn; workload includes these repeated prefills and all decoding calls\.