从长上下文到选择性记忆：面向大语言模型的以记忆为中心的压缩技术综述

# From Long Context to Selective Memory:A Survey of Memory-Centric Compression for Large Language Models
Source: [https://arxiv.org/html/2610.21165](https://arxiv.org/html/2610.21165)
###### Abstract

Long\-context capability has become essential for large language models \(LLMs\) to handle complex tasks, yet continuously expanding the context window does not remove the computational, storage, and information\-selection pressures incurred at inference time\. Existing work on prompt compression, KV\-cache management, efficient serving, retrieval augmentation, and agent memory is usually discussed in isolation, leaving the flow of information between storage and computation insufficiently understood\. This survey reframes long\-context compression as a memory\-management problem for allocating inference\-time resources\. We distinguish compute\-active memory, which directly participates in model computation, from compute\-inactive memory, which lies outside the computation path but can be stored, retrieved, updated, or forgotten, and we characterize offloaded KV caches, persistent prefix caches, and precomputed KV states as intermediate states on the activation path\. On this basis, we formulate writing, offloading, recall, retrieval\-and\-injection, materialization, eviction, consolidation, and forgetting as state transitions within an information lifecycle, and introduce a budgeted cost model that places these operations in a single cost account\. The resulting framework unifies short\-term context compression, long\-term memory management, and the transitions between them, and shows that the central challenge is not merely reducing information volume but deciding which information should participate in costly inference, when, and in what form\. We further summarize evaluation dimensions, benchmark gaps, and open challenges, and argue that future research should move from static, pointwise compression toward dynamic, governable, lifecycle\-aware memory management\.

Keywords:large language models, long\-context inference, context compression, memory management, KV cache, retrieval\-augmented generation, agent memory

## 1Introduction

Long\-context capability has become a prerequisite for large language models \(LLMs\) on complex tasks\. As context windows have grown from several thousand to hundreds of thousands or even millions of tokens\([Xu et al\., 2026a](https://arxiv.org/html/2610.21165#bib.bib1);[OpenAI, 2026](https://arxiv.org/html/2610.21165#bib.bib2);[Google, 2026](https://arxiv.org/html/2610.21165#bib.bib3)\), long\-document understanding, codebase analysis, multi\-turn dialogue, tool\-use trajectories, retrieval\-augmented generation \(RAG\), and long\-term personalized assistance increasingly depend on a model’s ability to receive and exploit large amounts of contextual information\([Liu et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib10)\)\.

A larger window, however, does not by itself yield more effective use of information\([Liu et al\., 2024c](https://arxiv.org/html/2610.21165#bib.bib4);[Hsieh et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib5)\)\. Long inputs increase prefill computation and place growing pressure on key\-value \(KV\) cache memory, memory bandwidth, and latency during decoding\([Zhang et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib6);[Kwon et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib58);[Xu et al\., 2026a](https://arxiv.org/html/2610.21165#bib.bib1)\), while realistic contexts contain repeated, obsolete, or irrelevant content that dilutes critical evidence\. The long\-context problem is therefore not only one of capacity but also one of management: how a system selects, compresses, and uses information under a finite computational budget\.

Several research directions address this problem at different levels, including prompt compression, latent memory representations, KV\-cache compression, architecture\-level selective computation, efficient serving, retrieval augmentation, and external long\-term memory\([Jiang et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib32);[Jiang et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib33);[Mu et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib34);[Zhang et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib6);[Kwon et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib58);[Lewis et al\., 2020](https://arxiv.org/html/2610.21165#bib.bib7);[Packer et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib8)\)\. Existing surveys usually organize them by technical origin or application scenario\([Liu et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib10);[Liu et al\., 2025c](https://arxiv.org/html/2610.21165#bib.bib11);[Verma, 2024](https://arxiv.org/html/2610.21165#bib.bib12)\), which obscures a question they share: which information should enter the expensive attention\-computation path, and which should instead be compressed, externalized, reused, updated, or forgotten?

This survey answers this question from a memory\-centric perspective\. We call information that directly participates in the current model computation*compute\-active memory*, and information that lies outside the current computation path but can be stored, retrieved, updated, or forgotten*compute\-inactive memory*\. The distinction concerns an item’s participation in computation at a given inference step rather than its storage location: the same content may be stored externally, later retrieved and injected into the context, and finally written back after inference\. Between the two lie intermediate states, in which information has left the attention path but can be restored without re\-retrieval or re\-encoding, as with offloaded KV caches and precomputed prefix caches\. Long\-context compression thus becomes the management of an information lifecycle across these states\. Recent surveys have also proposed unified views of LLM memory based on memory operations, context engineering, or agent memory\([Du et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib85);[Mei et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib86);[Hu et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib87)\); we instead organize the field by the computational participation of information and its cost\. The main contributions are as follows\.

1. 1\.We propose a framework that distinguishes compute\-active from compute\-inactive memory, incorporates intermediate states on the activation path, and casts long\-context compression as lifecycle management under a finite computational budget\.
2. 2\.We formulate writing, offloading, recall, retrieval\-and\-injection, materialization, eviction, consolidation, and forgetting as state\-transition operators and introduce a budgeted cost model that places these operations in a single cost account\.
3. 3\.We review representative work up to 2026 along this lifecycle, from input, representation, and KV\-cache compression to external long\-term memory, learned memory policies, and transition mechanisms such as retrieval\-and\-injection, KV offloading and materialization, and agent\-side context folding\.
4. 4\.We summarize evaluation dimensions, benchmark gaps, and open challenges, and pose a concrete research question for each challenge\.

#### Scope\.

This survey focuses on the intersection of long\-context modeling, KV\-cache management, retrieval augmentation, and agent memory: how a system decides which information enters the current inference path, compresses its representation, controls its cost, and later updates or forgets it\. Methods that only extend the context window without changing information selection, cache management, or memory mechanisms are treated as background\. Parametric memory is included only in a limited way, because knowledge editing, continual pre\-training, and parameter\-efficient fine\-tuning incur their cost mainly at training time and do not selectively activate information within a single inference; methods that blur this boundary, such as test\-time neural memory\([Behrouz et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib102)\)and self\-study\-trained KV caches\([Eyuboglu et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib101)\), are discussed where they affect inference\-time memory allocation\.

The remainder of the survey is organized as follows\. Section[2](https://arxiv.org/html/2610.21165#S2)reviews the costs of long\-context inference and positions this survey relative to prior reviews\. Section[3](https://arxiv.org/html/2610.21165#S3)presents the framework\. Sections[4](https://arxiv.org/html/2610.21165#S4)and[5](https://arxiv.org/html/2610.21165#S5)review compute\-active and compute\-inactive memory, Section[6](https://arxiv.org/html/2610.21165#S6)discusses state transitions, Section[7](https://arxiv.org/html/2610.21165#S7)covers evaluation, Section[8](https://arxiv.org/html/2610.21165#S8)discusses open challenges, and Section[9](https://arxiv.org/html/2610.21165#S9)concludes\.

## 2Background and Positioning

### 2\.1From Context Extension to Context Management

Early work on long\-context LLMs focused on enlarging the input that a model can accept, through positional\-encoding extrapolation, sparse and sliding\-window attention, recurrent\-state modeling, retrieval augmentation, and long\-context fine\-tuning\([Chen et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib13);[Peng et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib14);[Ding et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib15);[Beltagy et al\., 2020](https://arxiv.org/html/2610.21165#bib.bib20);[Zaheer et al\., 2020](https://arxiv.org/html/2610.21165#bib.bib21);[Dai et al\., 2019](https://arxiv.org/html/2610.21165#bib.bib22);[Borgeaud et al\., 2022](https://arxiv.org/html/2610.21165#bib.bib23);[Chen et al\., 2024a](https://arxiv.org/html/2610.21165#bib.bib24)\)\. These methods allow a model to access more complete task background in a single inference, which matters for long\-document understanding, cross\-file code analysis, multi\-turn dialogue, and tool use, where relevant information is scattered across texts, knowledge bases, interaction histories, and execution states\([Bai et al\., 2024a](https://arxiv.org/html/2610.21165#bib.bib16);[Liu et al\., 2024d](https://arxiv.org/html/2610.21165#bib.bib17);[Lewis et al\., 2020](https://arxiv.org/html/2610.21165#bib.bib7);[Yao et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib18);[Zheng et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib19);[Packer et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib8)\)\. As windows reach hundreds of thousands or millions of tokens\([Ding et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib15);[Xu et al\., 2026a](https://arxiv.org/html/2610.21165#bib.bib1)\), the central question has shifted from how much information a model can receive to which information deserves to enter the current inference path, and at what cost\([Liu et al\., 2025c](https://arxiv.org/html/2610.21165#bib.bib11);[Liu et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib10)\)\.

This shift is driven by two kinds of cost\. The first is computational\. Prefill cost and time\-to\-first\-token grow with input length; during decoding, the KV cache grows with context length, model size, and batch size and consumes GPU memory and bandwidth; and information kept in external indexes becomes costly again once it is retrieved and injected into the prompt\. The second is informational\. Long inputs mix key evidence with background, repetition, outdated states, and noise, and models do not use information uniformly across positions\([Liu et al\., 2024c](https://arxiv.org/html/2610.21165#bib.bib4)\); when evidence is dispersed or appears in the middle of the input, omissions, evidence mismatch, and inconsistent reasoning become more likely\. Long\-context compression therefore differs from generic model acceleration: besides reducing computation, cache usage, and latency, it must preserve evidence coverage, fidelity, temporal consistency, and future recoverability\. Over\-compression breaks evidence chains, whereas over\-retention increases latency, memory pressure, and noise\.

### 2\.2Existing Surveys and the Gap Addressed by This Survey

Prior surveys fall into four groups\. Surveys of long\-context LLMs\([Liu et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib10);[Liu et al\., 2025c](https://arxiv.org/html/2610.21165#bib.bib11);[Liu et al\., 2025b](https://arxiv.org/html/2610.21165#bib.bib94)\)cover positional extension, sparse attention, training, infrastructure, and evaluation; compression is one topic among many, and transitions of information between internal states and external storage are rarely analyzed\. Surveys of prompt compression\([Li et al\., 2025c](https://arxiv.org/html/2610.21165#bib.bib25)\), KV\-cache management\([Li et al\., 2024a](https://arxiv.org/html/2610.21165#bib.bib26);[Xu et al\., 2026b](https://arxiv.org/html/2610.21165#bib.bib95)\), and efficient serving\([Zhou et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib28);[Miao et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib29)\)focus on the cost of the current inference path, but give little attention to how long\-term memory is written, organized, and forgotten, and seldom relate KV offloading to external\-memory retrieval\. Surveys of RAG and agent memory\([Gao et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib27);[Verma, 2024](https://arxiv.org/html/2610.21165#bib.bib12);[Zhang et al\., 2024b](https://arxiv.org/html/2610.21165#bib.bib30);[Du, 2026](https://arxiv.org/html/2610.21165#bib.bib91)\)emphasize external knowledge and long\-term history;[Hu et al\. \(2025\)](https://arxiv.org/html/2610.21165#bib.bib87)organize agent memory by form, function, and dynamics; they include KV compression as latent working memory but assign model\-internal cache management to LLM memory, outside the scope of agent memory\. The survey of context compression for LLM agents by[Wang et al\. \(2026a\)](https://arxiv.org/html/2610.21165#bib.bib92)is closest in spirit: it describes a select–compress–store–recover pipeline, discusses recoverability and control overhead, and formulates learned compression control as task reward minus cost\. Its scope, however, is application\-level trajectory compression, and it excludes KV\-cache and model\-level methods\. Neither group distinguishes re\-encoding retrieved text from recalling states that are already encoded, or accounts for online activation, offline materialization, cross\-request reuse, and invalidation within one cost structure\.

The fourth group, closest to this survey, proposes unified views of LLM memory\.[Du et al\. \(2025\)](https://arxiv.org/html/2610.21165#bib.bib85)distinguish parametric from contextual memory and define six atomic operations across long\-term memory, long context, and parametric modification;[Wu et al\. \(2025b\)](https://arxiv.org/html/2610.21165#bib.bib88),[Shan et al\. \(2025\)](https://arxiv.org/html/2610.21165#bib.bib89), and[Jia et al\. \(2025\)](https://arxiv.org/html/2610.21165#bib.bib90)organize LLM memory by analogy to human memory; and the context\-engineering survey of[Mei et al\. \(2025\)](https://arxiv.org/html/2610.21165#bib.bib86)places memory hierarchies and context compression under context management\. On the system side, MemGPT\([Packer et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib8)\)manages context through an operating\-system analogy, and MemOS\([Li et al\., 2025b](https://arxiv.org/html/2610.21165#bib.bib93)\)schedules plaintext, activation, and parametric memory within one system\. These works organize the field by cognitive analogy, functional role, or operation type\.

This survey differs in three respects\. First, it organizes methods by whether and when information participates in inference computation, which allows system\-level mechanisms such as offloaded KV caches, persistent prefix caches, and offline\-compressed KV to be compared with external memory within one lifecycle\. Second, it treats state transitions as first\-class objects of analysis, in particular the distinction between re\-encoding retrieved text \(retrieve\-and\-inject\) and recalling states that are already encoded \(recall\), which fetch memory at very different costs\. Third, it compares methods through a budgeted cost model that separates online, offline, and amortized costs, so that costs moved from the current context into background maintenance are not hidden\. Table[1](https://arxiv.org/html/2610.21165#S2.T1)summarizes the comparison\.

Table 1:Comparison with representative related surveys\. ●: systematically discussed; ◐: partially discussed; ○: largely not discussed\. Ratings reflect the authors’ reading of each survey’s structure and coverage\.

## 3Memory\-Centric Framework for Long\-Context Compression

We treat long\-context compression as a memory\-management problem of allocating inference\-time computation: the goal is to decide when information should participate in inference, in what form, and at what cost\. This section defines memory states, the operators that move information between them, the resulting lifecycle, and a cost model that underlies the taxonomy of later sections\. Figure[1](https://arxiv.org/html/2610.21165#S3.F1)gives an overview\.

Figure 1:Memory\-centric lifecycle view of long\-context compression\. Information flows between compute\-active and compute\-inactive memory: writing externalizes information from the inference path, while retrieval, compression, and injection reactivate external memory\. Offloaded KV caches, persistent prefix caches, and materialized KV states are intermediate states on the activation path that can be recalled directly\. In\-place operations compress, reorganize, or discard content within a state; per\-unit holding cost increases from bottom to top\.### 3\.1Memory States and Computational Participation

We distinguish memory states by participation in inference computation, because this criterion corresponds directly to the main costs of a long\-context system\. We fix its granularity at a single request: information is compute\-active if it is encoded for the current model and addressable by attention during the request, that is, resident in accelerator memory as tokens, latent states, or KV caches\. Such information incurs prefill computation, accelerator memory, memory bandwidth, and decoding latency\. Methods that attend to only part of it at each decoding step, such as page\-level sparse attention, reduce reads and attention computation within compute\-active memory rather than moving information out of it\.

*Compute\-active memory*is information that the model accesses during the current inference: prompt tokens, soft prompts or latent states, intermediate activations, and KV caches, including prefix caches reused by the current request\. Its size and access pattern directly determine efficiency and generation quality\.*Compute\-inactive memory*is information that does not currently participate in computation but can be stored, retrieved, updated, or forgotten: document collections, vector indexes, knowledge graphs, summary memories, user profiles, dialogue histories, and tool trajectories\. It is cheaper to hold and suited to long\-term maintenance, but influences generation only after being retrieved, compressed, and injected\. The two are states rather than fixed categories: retrieved content becomes compute\-active, and answers, plans, or reflections produced during inference return to the compute\-inactive state when written back\.

#### Intermediate states on the activation path\.

KV offloading, persistent prefix caching, and retrieval\-based attention give rise to information that lies between these two states\. Such information has already been encoded into a form the model can consume directly \(KV blocks, compressed latent states, or precomputed prefixes\) and is kept in host memory, SSDs, or remote cache pools; it is not addressable by attention in the current request, yet it can be restored with low latency without re\-retrieval or re\-prefilling\. Examples include offloaded KV states in InfiniGen\([Lee et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib96)\), ShadowKV\([Sun et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib100)\), and RetrievalAttention\([Liu et al\., 2024b](https://arxiv.org/html/2610.21165#bib.bib98)\), and compressed KV caches trained offline by Cartridges\([Eyuboglu et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib101)\)\. A prefix cache retained for later requests is likewise in an intermediate state; it reaches this state either by being kept after a prefill \(A→\\toS\) or by precomputation from text \(I→\\toS\)\. We regard these as intermediate states on the transition path rather than as a third kind of memory\. The criterion is the activation path: information that can return to computation through data movement and light correction is in an intermediate state, whereas information that additionally requires retrieval, compression, and re\-encoding is compute\-inactive\. A transition between active and inactive memory may therefore pass through, and remain for some time in, an intermediate state with its own holding cost\.

### 3\.2State\-Transition Operators

Long\-context compression methods can be described by a small set of operators, each specifying the direction in which information moves among compute\-active \(A\), intermediate \(S\), and compute\-inactive \(I\) states, possibly together with a change of representation such as summarization, structuring, or encoding into KV states \(Table[2](https://arxiv.org/html/2610.21165#S3.T2)\)\.

Table 2:State\-transition operators\. A: compute\-active; S: intermediate state on the activation path; I: compute\-inactive\.These operators are related to the atomic memory operations of[Du et al\. \(2025\)](https://arxiv.org/html/2610.21165#bib.bib85), but are defined by changes of computational state rather than by memory function\. This distinguishes*retrieve\-and\-inject*\(I→\\toA, requiring re\-prefilling\) from*recall*\(S→\\toA, mainly data movement\), which both fetch memory but have very different cost structures\. Compression, in turn, is not a separate operator but a representation change attached to the operators: eviction discards information within active memory, writing may summarize, retrieve\-and\-inject may filter and rewrite, and consolidation reorganizes representations within one state\. Two further conventions apply\. Writing copies rather than moves: written content usually remains in active memory for the rest of the request, so A→\\toI creates a new representation in I without removing its source\. Transformations that stay within compute\-active memory, such as pruning, encoding into latent states, and architectural sharing or sparsification of KV states, change the representation or the accessed subset without changing state; Table[3](https://arxiv.org/html/2610.21165#S3.T3)marks them as A→\\toA\.

### 3\.3Information Lifecycle in Long\-Context Systems

Composing these operators yields an information lifecycle of writing, storage, retrieval, selection, compression, injection, computation, updating, and forgetting\. A system first decides what to write and how to organize it as summaries, indexes, structured representations, or embeddings\. For a given task, external memory is retrieved, but retrieved results are not yet effective context: they must be filtered, reranked, compressed, and formatted before entering the inference path\. After inference, new facts, plans, tool feedback, user corrections, and reflections may be written back, while outdated, conflicting, low\-value, or sensitive memories are updated, down\-weighted, or forgotten\. Long\-context compression is therefore a continuous process rather than one\-time input pruning, and different methods act at different stages of this lifecycle\.

### 3\.4A Budgeted Cost Model

The aim of compression in this view is to improve information utilization under limited budgets, and the relevant objectives differ by state: for compute\-active memory, prefill, KV memory, bandwidth, and throughput; for compute\-inactive memory, preserving the retrievability and temporal validity of stored information; and for transitions, evidence traceability, recoverability, and long\-term consistency\. To make these objectives comparable, letMMbe the set of memory units held at an inference step, and let each unitmmhave a states⁡\(m\)∈\{A,S,I\}s\(m\)\\in\\\{A,S,I\\\}\. The total cost is

C=∑s⁡\(m\)=AcA​\|m\|\+∑s⁡\(m\)=ScS​\|m\|\+∑s⁡\(m\)=IcI​\|m\|\+∑τ∈Tcτ,C=\\sum\_\{s\(m\)=A\}c\_\{A\}\\,\|m\|\+\\sum\_\{s\(m\)=S\}c\_\{S\}\\,\|m\|\+\\sum\_\{s\(m\)=I\}c\_\{I\}\\,\|m\|\+\\sum\_\{\\tau\\in T\}c\_\{\\tau\},\(1\)where\|m\|\|m\|is the size of a unit \(e\.g\., tokens or KV bytes\);cAc\_\{A\},cSc\_\{S\}, andcIc\_\{I\}are per\-unit holding costs, typically withcA≫cS≫cIc\_\{A\}\\gg c\_\{S\}\\gg c\_\{I\}, covering respectively prefill computation, KV memory, and decoding bandwidth; host\-memory or SSD storage; and external storage and index maintenance;TTis the set of transitions performed for the current inference; andcτc\_\{\\tau\}is the cost of each transition \(retrieval, reranking, compression, data movement, recomputation, or write\-back\), which also depends on the representations before and after the transition\. Long\-context compression can then be written as

maxs⁡\(⋅\),T⁡U⁡\(y∣MA\)−λ​Cs\.t\.∑s⁡\(m\)=A\|m\|≤BA,∑s⁡\(m\)=S\|m\|≤BS,\\max\_\{s\(\\cdot\),\\,T\}\\;U\(y\\mid M\_\{A\}\)\-\\lambda C\\quad\\text\{s\.t\.\}\\quad\\sum\_\{s\(m\)=A\}\|m\|\\leq B\_\{A\},\\quad\\sum\_\{s\(m\)=S\}\|m\|\\leq B\_\{S\},\(2\)whereU⁡\(y∣MA\)U\(y\\mid M\_\{A\}\)is the utility of generating outputyyfrom active memoryMAM\_\{A\},λ\\lambdatrades cost against utility, andBAB\_\{A\}andBSB\_\{S\}are the active and intermediate\-state budgets\.

The model exposes a common structure across method families\. Prompt compression shrinksMAM\_\{A\}in text form; KV eviction removes units fromAAat the price of recoverability; offloading and recall move units betweenAAandSS, exchanging active budget for transfer cost; retrieval augmentation keeps information inIIand pays a retrieve\-and\-inject cost on demand; materialization pays an offline cost to move information fromIItoSS, lowering the cost of each later activation; and long\-term memory management acts mainly on theIIterm and its future activation cost\. It also explains why compression ratio alone is misleading: a method may reduce thecAc\_\{A\}term while shifting cost intocτc\_\{\\tau\}or offline maintenance\.

### 3\.5Analytical Dimensions for Method Comparison

The following sections compare methods along shared dimensions: the memory state and operators involved; the compression target \(text, structured fragments, continuous vectors, attention states, cache blocks, or indexes\); the selection signal \(query semantics, attention distributions, temporal state, task goals, retrieval scores, or heuristics\); system requirements \(access to internal states, training, compatibility with closed\-model APIs, and cross\-request reuse\); and evaluation criteria \(computational and storage cost, task utility, evidence fidelity, recoverability, and long\-term consistency\)\. Figure[2](https://arxiv.org/html/2610.21165#S3.F2)organizes representative methods by memory state and transition, and Table[3](https://arxiv.org/html/2610.21165#S3.T3)codes a subset of them along these dimensions\.

\{forest\}Figure 2:Taxonomy of representative methods organized by memory state and state transition\. Citations for all methods are given in the corresponding sections\.Table 3:Representative methods coded along the analytical dimensions of Section[3](https://arxiv.org/html/2610.21165#S3)\. Operators follow Table[2](https://arxiv.org/html/2610.21165#S3.T2)\(A: compute\-active; S: intermediate; I: compute\-inactive\)\. Access:*black\-box*methods work with closed\-model APIs;*white\-box*methods require model weights or internal states;*system*methods operate in the serving runtime\. Reuse: whether the compressed or stored state is reused across requests or sessions\.MethodOperatorRepr\.Selection signal / mechanismTrainingAccessReuseCompute\-active memory \(Section[4](https://arxiv.org/html/2610.21165#S4)\)Selective ContextA→\\toA \(prune\)TextSelf\-information of lexical unitsNoneBlack\-box–LLMLingua / LongLLMLinguaA→\\toA \(prune\)TextSmall\-LM perplexity under a budget; question\-aware variantNoneBlack\-box–Gist TokensA→\\toA \(encode\)LatentLearned gist tokensFine\-tuneWhite\-box✓AutoCompressorsA→\\toA \(encode\)LatentSegment\-wise summary vectorsFine\-tuneWhite\-box✓Activation BeaconA→\\toA \(encode\)KVLearned activation compressionFine\-tuneWhite\-box–H2O / StreamingLLMA→∅\\to\\mathord\{\\emptyset\}\(evict\)KVAccumulated attention; attention sinks and recent windowNoneWhite\-box–SnapKV / HeadKVA→∅\\to\\mathord\{\\emptyset\}\(evict\)KVObservation\-window attention; head\-level budgetsNoneWhite\-box–MiniCacheConsolidateKVCross\-layer similarity mergingNoneWhite\-box–KVzipA→\\toA \(compress\)KVQuery\-agnostic context reconstructionNoneWhite\-box✓R\-KVA→∅\\to\\mathord\{\\emptyset\}\(evict\)KVImportance and redundancy in reasoning tracesNoneWhite\-box–QuestA→\\toA \(select\)KVPage\-level attention upper bounds; full cache kept residentNoneWhite\-box–GQA / MLAA→\\toA \(arch\.\)KVShared or low\-rank latent KV headsPre\-train or uptrainWhite\-box–NSA / MoBAA→\\toA \(arch\.\)KVLearned block compression, selection, routingPre\-trainWhite\-box–Kimi LinearA→\\toA \(arch\.\)KVHybrid linear and full attentionPre\-trainWhite\-box–PagedAttention–KVBlock\-level KV memory allocationNoneSystem–RadixAttention–KVPrefix matching in a radix treeNoneSystem✓MooncakeA↔\\leftrightarrowSKVKV\-centric disaggregated schedulingNoneSystem✓Compute\-inactive memory \(Section[5](https://arxiv.org/html/2610.21165#S5)\)MemoryBankA→\\toI, forgetTextEbbinghaus\-style forgetting curveNoneBlack\-box✓Generative AgentsA→\\toI, I→\\toATextRecency, importance, and relevanceNoneBlack\-box✓MemGPTA↔\\leftrightarrowITextLLM\-directed paging via function callsNoneBlack\-box✓HippoRAGI→\\toAGraphKnowledge graph and personalized PageRankNoneBlack\-box✓ReflexionA→\\toITextVerbal self\-reflection on feedbackNoneBlack\-box✓Mem0A→\\toI, consolidateText / graphLLM\-decided add, update, deleteNoneBlack\-box✓A\-MEMA→\\toI, consolidateStruct\.Linked atomic notesNoneBlack\-box✓ZepA→\\toIGraphTemporal knowledge graphNoneBlack\-box✓Memory\-R1A→\\toI, consolidateTextRL\-learned memory operationsRLWhite\-box✓State transitions \(Section[6](https://arxiv.org/html/2610.21165#S6)\)TurboRAG / Block\-AttentionI→\\toS→\\toAKVPrecomputed chunk KV; block\-wise independent encodingFine\-tuneWhite\-box✓CacheBlendI→\\toS→\\toAKVSelective recomputation of deviating tokensNoneWhite\-box✓CartridgesI→\\toSKVSelf\-study distillation of a corpusPer\-corpusWhite\-box✓InfiniGenA↔\\leftrightarrowSKVNext\-layer KV prefetch predicted from the previous layerNoneWhite\-box–RetrievalAttention / ShadowKVA↔\\leftrightarrowSKVANN index over keys; low\-rank keys with offloaded valuesNoneWhite\-box–ACONA→\\toA \(summarize\)TextGuidelines optimized from failure cases; optional distilled compressorNone \(prompt opt\.\)Black\-box–Context\-FoldingA→\\toA \(fold\)TextRL\-learned branching and foldingRLWhite\-box–MEM1 / MemAgentA→\\toA \(overwrite\)TextRL\-learned bounded internal memoryRLWhite\-box–

## 4Compressing Compute\-Active Memory

Compute\-active memory compression reduces the cost of information that is already on the inference path, and therefore acts directly on prefill overhead, decoding latency, GPU memory, and throughput\. We organize methods by where compression occurs: text before model input, compact latent representations, the KV cache during decoding, the model architecture, and the serving system\. For each family we consider which cost is reduced, what information is preserved, and what the method requires of model access, training, and deployment\. KV states that leave the attention path through offloading are discussed with other state transitions in Section[6\.3](https://arxiv.org/html/2610.21165#S6.SS3)\.

### 4\.1Input and Prompt Compression

Input and prompt compression filters or rewrites context before it enters the model, acting as admission control for compute\-active memory\. Inputs that concatenate documents, interaction histories, retrieved passages, and instructions often contain repeated background, templated text, and weakly relevant passages that raise prefill cost and dilute useful information\. Methods differ in how they estimate information value: redundancy\-based methods such as Selective Context\([Li et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib31)\)remove locally repetitive or low\-information tokens, sentences, or passages; LLMLingua\([Jiang et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib32)\)formulates compression as coarse\-to\-fine selection under a token budget; and LongLLMLingua\([Jiang et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib33)\)conditions compression on the query to retain relevant evidence in long contexts\.

Because the output remains natural language, these methods are inspectable and require no change to model internals\. Readability does not by itself ensure evidence traceability, however: token\-level pruning may produce ungrammatical text detached from its source, so applications such as RAG and knowledge\-base question answering also need to retain the original positions, sources, and timestamps of the selected content\. Their limitation is that compression precedes reasoning: content that appears unimportant locally may be needed for multi\-hop reasoning, long\-range coreference, code dependencies, or cross\-turn consistency\. They work best when task goals are clear, redundancy is high, and evidence can be located from the query, and otherwise need to be combined with reranking, structured evidence preservation, or downstream verification\.

### 4\.2Compressed Representations: Gist Tokens, Latent States, and Summary Memory

Compressed\-representation methods encode long contexts into denser computable states, such as special tokens, soft prompts, summary vectors, or compressed activations, which still participate in inference but occupy far less of the context budget\. Three directions can be distinguished\. Gist Tokens\([Mu et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib34)\)train a model to store prompt information in a few learned tokens that later generation attends to; AutoCompressors\([Chevalier et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib35)\)process long texts segment by segment and pass summary vectors forward; and Activation Beacon\([Zhang et al\., 2025b](https://arxiv.org/html/2610.21165#bib.bib36)\)and Infini\-attention\([Munkhdalai et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib37)\)compress Transformer activations or combine local attention with a bounded long\-term compressed memory\. More recent work scales encoder–decoder context compression to large\-scale training, improving the trade\-off among task performance, compression speed, and peak memory\([Li et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib109)\)\.

These representations increase information density and avoid repeated encoding when the same background is accessed many times\. They are, however, difficult to interpret: it is hard to determine which facts or constraints a latent state preserves, or where information was lost when generation fails\. They are also tied to specific architectures and training objectives, which limits transfer across models and to closed APIs\. They are best viewed as high\-density intermediate representations to be combined with task\-aware compression or traceable retrieval\.

### 4\.3KV Cache Compression

During autoregressive decoding, the keys and values of past tokens are cached to avoid recomputing the prefix, and this cache grows with context length, depth, number of heads, and batch size until it dominates GPU memory and bandwidth\. The KV cache is thus a high\-cost form of short\-term compute\-active memory, and KV compression seeks to retain the states most useful for future generation within a fixed budget\. Methods differ mainly in how they estimate future utility: by recency, accumulated attention, or attention sinks\([Zhang et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib6);[Liu et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib38);[Xiao et al\., 2024b](https://arxiv.org/html/2610.21165#bib.bib39)\); by query\-, head\-, or layer\-aware importance\([Li et al\., 2024b](https://arxiv.org/html/2610.21165#bib.bib40);[Jo et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib41);[Fu et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib42);[Rehg, 2024](https://arxiv.org/html/2610.21165#bib.bib43)\); or by reducing representation cost through quantization, merging, low\-rank reconstruction, or reuse across queries\([Zhang et al\., 2025c](https://arxiv.org/html/2610.21165#bib.bib44);[Yang et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib45);[Liu et al\., 2024a](https://arxiv.org/html/2610.21165#bib.bib46);[Chang et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib47);[Kim et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib48);[Wang et al\., 2026b](https://arxiv.org/html/2610.21165#bib.bib49)\)\. Not all methods discard states: Quest\([Tang et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib97)\)keeps the full cache resident and loads only the pages with the highest attention upper bounds for the current query, reducing reads and attention computation rather than memory capacity\. Systematic analyses further show that compression can degrade capabilities unevenly, even when aggregate scores are preserved\([Liu et al\., 2026b](https://arxiv.org/html/2610.21165#bib.bib50)\)\. These methods exploit internal signals unavailable to text\-level compression, but require access to model states and inference\-system support, and their decisions rest on imperfect predictions: states that received little attention so far may become critical later\.

A fast\-growing branch targets reasoning models, for which long chains of thought shift the main source of KV growth from the prompt to the model’s own generations, and redundancy arises from repeated verification and self\-correction\. R\-KV\([Cai et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib103)\)compresses reasoning\-trace KV states by jointly considering attention importance and redundancy, and ThinKV\([Ramachandran et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib104)\)allocates cache budgets and quantization precision according to the type of thought segment\. These methods show that compute\-active memory grows not only with external input but also with the reasoning process itself\.

### 4\.4Architecture\-level Selective Computation

Architecture\-level methods reduce the cost of long contexts by changing how the model represents and accesses history, rather than by deleting input or post\-processing the cache\. Multi\-Query and Grouped\-Query Attention\([Shazeer, 2019](https://arxiv.org/html/2610.21165#bib.bib51);[Ainslie et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib52)\)share keys and values across query heads, and Multi\-head Latent Attention\([DeepSeek\-AI, 2024a](https://arxiv.org/html/2610.21165#bib.bib53);[DeepSeek\-AI, 2024b](https://arxiv.org/html/2610.21165#bib.bib54)\)compresses KV states into latent vectors\. Sparse attention has since become native and trainable: NSA\([Yuan et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib105)\)combines compression, selection, and sliding\-window branches in a hardware\-aligned design, MoBA\([Lu et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib106)\)routes queries to attention blocks in a mixture\-of\-experts fashion, and DeepSeek\-V3\.2\([DeepSeek\-AI, 2025](https://arxiv.org/html/2610.21165#bib.bib107)\)introduces sparse attention driven by a lightweight indexer; DeepSeek\-V4\([Xu et al\., 2026a](https://arxiv.org/html/2610.21165#bib.bib1)\)further integrates compressed attention, shared KV, and local windows into the attention mechanism\. Other designs reduce reliance on full attention altogether: InfLLM\([Xiao et al\., 2024a](https://arxiv.org/html/2610.21165#bib.bib55)\)retrieves from an efficient context memory, state\-space and hybrid models such as Mamba\([Gu and Dao, 2023](https://arxiv.org/html/2610.21165#bib.bib56)\)and Hymba\([Dong et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib57)\)maintain recurrent states, and Kimi Linear\([Kimi Team, 2025](https://arxiv.org/html/2610.21165#bib.bib108)\)pairs a few full\-attention layers with linear\-attention layers to shrink the KV cache substantially\.

These methods build the selection or compaction of compute\-active memory into the attention mechanism itself\. Most require pre\-training or continued training, although GQA can also be obtained by uptraining an existing multi\-head checkpoint\([Ainslie et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib52)\), and InfLLM is training\-free and operates on a pre\-trained model at inference time\([Xiao et al\., 2024a](https://arxiv.org/html/2610.21165#bib.bib55)\)\. We include context\-extension techniques only when they change the access pattern, state representation, or cost structure of inference\. Their main limitation is cost of adoption: they require changes to the architecture or attention kernels and to cache management, usually together with training, and cannot be applied to closed\-model interfaces\.

### 4\.5Serving\-level Cache and Prefix Management

Serving\-level methods do not change semantic content or internal representations; they determine how computed states are placed, reused, scheduled, and released across GPU memory, cache blocks, batches, and requests\. vLLM/PagedAttention\([Kwon et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib58)\)manages KV caches in pages to reduce fragmentation and enable sharing, and vAttention\([Prabhu et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib59)\)achieves similar benefits by decoupling virtual and physical memory\. Prefix caching stores the KV states of stable contexts for reuse, reducing time\-to\-first\-token, and SGLang/RadixAttention\([Zheng et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib19)\)automatically identifies and reuses shared prefixes across multi\-call programs, branching, and multi\-turn interaction\. Because prefill is compute\-bound and decoding is bandwidth\-bound, systems also schedule across phases: Orca introduces iteration\-level scheduling\([Yu et al\., 2022](https://arxiv.org/html/2610.21165#bib.bib60)\), Sarathi\-Serve uses chunked prefill\([Agrawal et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib61)\), DistServe disaggregates prefill and decoding\([Zhong et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib62)\), and Mooncake builds a disaggregated KV cache across CPU, DRAM, and SSD resources\([Qin et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib63)\)\. TokenLake extends prefix caching to a segment\-level cache pool, and Jenga manages memory under heterogeneous cache layouts\([Wu et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib64);[Zhang et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib65)\)\.

Serving\-level management is what converts the local savings of other methods into end\-to\-end latency, throughput, and memory gains\. Its benefits depend strongly on frameworks, hardware, workloads, and cache hit rates, so evaluation should report throughput, memory utilization, hit rates, and tail latency rather than single\-request speed alone\.

## 5Compute\-Inactive Memory Compression

Not all valuable information needs to remain in the attention window\. Source documents, dialogue histories, user preferences, tool outputs, task states, and reflections are often stored externally and retrieved when needed\. Such compute\-inactive memory incurs no prefill or KV cost at the current step, but determines whether a system can maintain continuity and reuse experience over long\-term operation\. Compressing it therefore means turning large histories into memory structures that remain retrievable, updatable, and reactivatable, not merely reducing storage\. This section follows the external\-memory lifecycle: writing, organization, retrieval, maintenance, and learned policies that govern these steps; how retrieved memory is compressed and injected into the inference path is discussed in Section[6](https://arxiv.org/html/2610.21165#S6)\.

### 5\.1Memory Writing

Writing determines which parts of the current interaction, task execution, or tool feedback become long\-term memory units, and is the first selection boundary of an external\-memory system\. Writing complete histories reduces omission but enlarges the store, adds retrieval noise, and amplifies privacy risks and the accumulation of stale or erroneous content\. Written objects can be grouped into five types: factual memory of user preferences and stable states\([Zhong et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib66);[Kang et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib67)\); episodic memory of temporally grounded experiences\([Park et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib9);[Shu et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib68)\); trajectory memory of reasoning steps, tool calls, and feedback\([Yao et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib18)\); reflective memory that abstracts experiences or failures into lessons\([Shinn et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib69)\); and skill memory that consolidates successful experience into callable procedures\([Wang et al\., 2023a](https://arxiv.org/html/2610.21165#bib.bib70)\)\. Recent systems make writing an explicit decision: Mem0\([Chhikara et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib77)\)extracts candidate facts after each interaction and decides whether to add, update, delete, or keep them, and LightMem\([Fang et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib110)\)pre\-compresses and topically segments input before writing and defers consolidation to an offline stage\. Writing is a selection made under uncertainty about future tasks: overly conservative writing weakens continuity, whereas overly aggressive writing lowers the signal\-to\-noise ratio of the store on which all later stages depend\.

### 5\.2Memory Organization

Organization determines how written memories are indexed, related, and later updated\. Appending memories to a linear log preserves history but lets redundancy, staleness, and retrieval noise grow with interaction length\. Existing systems use serialized memory streams that preserve temporal order\([Park et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib9)\), vector indexes for semantic retrieval\([Lewis et al\., 2020](https://arxiv.org/html/2610.21165#bib.bib7);[Borgeaud et al\., 2022](https://arxiv.org/html/2610.21165#bib.bib23)\), external stores of independently accessible units\([Zhong et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib66);[Packer et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib8)\), graph\-structured memory of entities and relations\([Jimenez Gutierrez et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib72);[Jimenez Gutierrez et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib73)\), hierarchical and multi\-granularity memory across time scales\([Kang et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib67);[Sun and Zeng, 2025](https://arxiv.org/html/2610.21165#bib.bib76);[Xu et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib74)\), structured semantic compression of memory units\([Liu et al\., 2026a](https://arxiv.org/html/2610.21165#bib.bib75)\), and stores of reflections and skills\([Shinn et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib69);[Wang et al\., 2023a](https://arxiv.org/html/2610.21165#bib.bib70)\)\. Recent systems refine these forms: A\-MEM\([Xu et al\., 2025b](https://arxiv.org/html/2610.21165#bib.bib111)\)organizes memories as linked atomic notes that evolve with new information, Zep\([Rasmussen et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib112)\)uses a temporal knowledge graph with validity intervals for facts, and MIRIX\([Wang and Chen, 2025](https://arxiv.org/html/2610.21165#bib.bib113)\)separates core, episodic, semantic, procedural, and resource memory managed by multiple agents\.

Organization is itself a form of compression: without necessarily reducing the number of entries, structuring, indexing, and hierarchization reduce the noise faced by later retrieval and injection\. The cost is maintenance: richer structures make updating, deletion, and conflict correction harder, and memory units that are too coarse import irrelevant content while units that are too fine break contextual integrity\.

### 5\.3Memory Retrieval and Activation Readiness

Retrieval decides which memories are relevant to the current query, task state, or agent goal\. Vector retrieval relies on semantic similarity\([Lewis et al\., 2020](https://arxiv.org/html/2610.21165#bib.bib7);[Borgeaud et al\., 2022](https://arxiv.org/html/2610.21165#bib.bib23)\); interactive agents additionally weight recency and importance\([Park et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib9);[Zhong et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib66)\); memory controllers schedule access across memory tiers\([Packer et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib8);[Kang et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib67)\); graph\-based retrieval follows entity and relation paths for multi\-hop and temporal questions\([Jimenez Gutierrez et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib72);[Jimenez Gutierrez et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib73)\); episodic retrieval recalls specific past experiences\([Shu et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib68);[Chhikara et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib77)\); and agent memory may be triggered by task goals, tool states, or prior failures\([Shinn et al\., 2023](https://arxiv.org/html/2610.21165#bib.bib69);[Wang et al\., 2023a](https://arxiv.org/html/2610.21165#bib.bib70)\)\. Retrieval compresses the candidate space so that the full history need not be exposed in context, and its quality bounds every later step: retrieving too little causes forgetting and inconsistency, whereas retrieving too much reintroduces noise and increases the cost of later compression and injection\. We refer to this stage as*activation readiness*because retrieved memories remain candidates; they become compute\-active only after the selection, compression, and injection steps discussed in Section[6](https://arxiv.org/html/2610.21165#S6)\.

### 5\.4Memory Update, Consolidation and Forgetting

Long\-term memory does not remain correct indefinitely: preferences change, task states evolve, facts become stale, and sources conflict\.*Updating*reconciles new information with stored memory, reinforcing consistent entries and, for conflicts, replacing, revising, versioning, or deferring until confirmation; without it, stale information is repeatedly reactivated\.*Consolidation*merges repeated or similar memories into summaries, topics, structured facts, or higher\-level reflections, increasing density at the risk of losing details, temporal order, or conditions that later tasks need\.*Forgetting*removes, down\-weights, expires, hides, or archives memories that are low\-value, stale, erroneous, or sensitive, serving both noise control and privacy and safety governance\. Together these operations determine whether compute\-inactive memory keeps a high signal\-to\-noise ratio over time; a system that only writes and retrieves gradually degrades into a store of historical noise\.

### 5\.5Learned Memory\-Management Policies

The mechanisms above mostly rely on hand\-designed rules or prompts\. A recent trend is to learn memory\-management policies with reinforcement learning, so that what to store, revise, and use is optimized by task reward\. Memory\-R1\([Yan et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib114)\)trains a memory manager to add, update, delete, or skip entries in an external memory bank and an answer agent to filter retrieved memories, turning the write, consolidate, and forget operators into learned policies that maximize utility under a budget, as in Eq\. \([2](https://arxiv.org/html/2610.21165#S3.E2)\)\. Related methods learn to maintain a bounded memory inside the context window; because that memory is compute\-active by our criterion, we discuss them with agent\-side activation control \(Section[6\.4](https://arxiv.org/html/2610.21165#S6.SS4)\)\. Because rewards usually come only from final answers, however, such learned policies do not directly constrain the faithfulness, traceability, or privacy of memory, and may exploit benchmark regularities; incorporating cost and reliability terms into the reward remains open\.

## 6State Transitions: Retrieval, Injection, and Intermediate States

This section examines how information moves between memory states, which is where the memory\-centric view departs most from existing reviews\. Retrieved memory is not equivalent to information the model has used: it affects generation only after selection, compression, formatting, and injection into the attention path\. Injecting every candidate reintroduces the cost and noise of long contexts, whereas aggressive filtering can remove evidence, temporal relations, or task state\. We first discuss the transition from retrieved memory to active context and the forms of injection, then intermediate states on the activation path, agent\-side activation control, and the feedback loop created by write\-back\.

### 6\.1From Retrieved Memory to Active Context

Retrieval is often presented as a way to avoid placing all history in the input, but it is not a cost\-free extension of context: as soon as retrieved content is concatenated into the prompt, it consumes prefill computation, context length, and subsequent cache budget\. If candidates are injected wholesale, redundant, weakly relevant, or stale content re\-enters the inference path and erodes the gains of compression\. The transition from candidates to active context therefore requires three judgments: which candidates are relevant to the current task, which can be summarized, structured, or merged, and which must be kept as original evidence\. The answer depends on the task: long\-document question answering needs verifiable evidence, personalized dialogue needs consistent user state, code tasks need structural dependencies, and agent tasks rely on tool results and action histories\. Retrieval\-conditioned compression and injection is thus an activation\-control process that connects the external memory of Section[5](https://arxiv.org/html/2610.21165#S5)with the active memory of Section[4](https://arxiv.org/html/2610.21165#S4)\.

### 6\.2Compression and Injection Forms

Post\-retrieval compression operates at several granularities: filtering removes weakly relevant candidates, reranking orders evidence, summarization merges candidates into shorter text, structured extraction produces entities, attributes, temporal or causal relations, and task states, graph\-path compression keeps the key nodes and edges of multi\-hop relations, and more aggressive methods encode retrieved content as special tokens, latent vectors, or cache states\. Each choice carries its own risk: summaries are inspectable but may omit detail, structured facts are compact but depend on extraction quality, and latent representations compress most but are least interpretable\.

The injection form determines the computational state in which external memory participates\. Concatenating passages is simple and interpretable but costly; summaries are shorter but lossy; structured facts and graph paths suit relational, temporal, and multi\-hop reasoning; and in more integrated systems, external memory enters as soft prompts, latent vectors, prefix states, or KV caches\. At the KV level, TurboRAG\([Lu et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib117)\)precomputes KV caches for document chunks offline and concatenates them online to skip prefilling of retrieved content; Block\-Attention\([Ma et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib118)\)encodes retrieved blocks independently, lets only the final query block attend to all blocks, and fine\-tunes the model to this pattern; and CacheBlend\([Yao et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib119)\)restores cross\-document attention lost by naive concatenation by recomputing KV states for a small set of tokens with the largest deviation\. No injection form is uniformly superior: traceable evidence favors text or structured facts, extreme compression favors latent representations, and multi\-turn reuse favors prefix or cache states, so injection must be chosen jointly by cost, utility, and verifiability\.

### 6\.3Intermediate States on the Activation Path: Offloading and Materialization

Retrieve\-and\-inject requires retrieval, compression, and re\-encoding before external memory can be used\. Intermediate states avoid the last two steps: the information already has a form the model can use and returns to computation through data movement and light correction\. Such states arise in two directions, by offloading active KV states from the attention path \(offload and recall in Table[2](https://arxiv.org/html/2610.21165#S3.T2)\) or by materializing external memory in advance into loadable forms \(materialize\)\.

#### Offloading and recall\.

When historical KV states exceed the GPU budget, a system can move them to host memory or lower storage tiers instead of evicting them, and recall a small relevant subset during decoding\. The key problem is to decide what to recall without touching all states\. InfiniGen\([Lee et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib96)\)predicts important tokens of the next layer from the previous layer’s input and partial query–key projections and prefetches their KV states; RetrievalAttention\([Liu et al\., 2024b](https://arxiv.org/html/2610.21165#bib.bib98)\)builds attention\-aware vector indexes that account for the distribution mismatch between queries and keys; MagicPIG\([Chen et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib99)\)estimates attention by locality\-sensitive\-hashing sampling rather than top\-kkselection; and ShadowKV\([Sun et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib100)\)keeps low\-rank keys on the GPU, offloads values to the CPU, and reconstructs selected keys during decoding\. LongMem\([Wang et al\., 2023b](https://arxiv.org/html/2610.21165#bib.bib71)\)applies the same idea beyond a single input, caching attention key–value pairs of past inputs in a memory bank that a trained side network retrieves from\. Quest\([Tang et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib97)\)uses query\-aware page selection as well but keeps the full cache in accelerator memory, so it changes which states are read rather than where they reside \(Section[4\.3](https://arxiv.org/html/2610.21165#S4.SS3)\)\. These methods are mechanistically isomorphic to external\-memory retrieval—both index information beyond the active budget, select candidates for the current query, and reactivate them at a cost—but KV recall requires no re\-prefilling\. KV\-level retrieval\-based attention and text\-level retrieval augmentation can thus be viewed as instances of one activation problem over different representations\. They remain sensitive to host–device bandwidth, index construction, and recall precision, and mostly operate within a single request\.

#### Materialization\.

In the other direction, external memory can be moved in advance from the compute\-inactive state into an intermediate state, so that each later activation pays only for loading and correction\. Cross\-request prefix caching \(Section[4\.5](https://arxiv.org/html/2610.21165#S4.SS5)\) is the simplest case, and the KV\-level injection methods of Section[6\.2](https://arxiv.org/html/2610.21165#S6.SS2)follow the same idea, facing the mismatch between independently encoded chunks and joint reasoning\. Cartridges\([Eyuboglu et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib101)\)take a different route: the model generates synthetic conversations about a corpus and distills it into a much smaller trainable KV cache, showing that materialization can be a lossy, learned compression targeted at the expected query distribution rather than lossless storage of raw KV states\. Materialized states raise their own problems: they are bound to a specific model version and must be rebuilt after updates, they occupy far more storage than raw text, and they are hard to inspect or edit, so invalidating the derived states of a deleted or corrected document becomes a system\-level instance of the governance problems in Section[8](https://arxiv.org/html/2610.21165#S8)\.

### 6\.4Agent\-Side Activation Control

In long\-horizon agent tasks, activation control also applies to the agent’s own trajectory: as tool calls, observations, and intermediate reasoning accumulate, the agent must decide which history stays in active context and which is summarized, folded, or externalized\. ACON\([Kang et al\., 2025b](https://arxiv.org/html/2610.21165#bib.bib120)\)refines compression guidelines from trajectories in which the full context succeeds but the compressed context fails, and distills the resulting compressor into smaller models; Context\-Folding\([Sun et al\., 2025b](https://arxiv.org/html/2610.21165#bib.bib121)\)lets an agent branch into sub\-tasks and fold each completed sub\-trajectory into a summary, learning the folding behavior with reinforcement learning; MEM1\([Zhou et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib115)\)lets an agent merge its previous internal state with each new observation into a compact state, keeping memory nearly constant in long\-horizon tasks; and MemAgent\([Yu et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib116)\)reads long documents segment by segment and learns to overwrite a bounded in\-context memory, allowing a limited\-context model to handle very long inputs\. Here the decision\-maker shifts from external retrieval modules to the agent policy, and when and what to compress becomes part of the action space\. A dedicated survey\([Wang et al\., 2026a](https://arxiv.org/html/2610.21165#bib.bib92)\)reviews these methods in detail; from the lifecycle perspective, the open questions are whether folded trajectories should be written back to long\-term memory, in what form, and whether information lost in folding can later be recovered\.

### 6\.5Feedback and Failure Modes

Transitions form a loop: injected memory shapes current inference, and model outputs, tool feedback, user corrections, and reflections are written back as new memory\. The loop allows experience to accumulate, but it also allows errors to persist\. Retrieval may return stale or weakly relevant memories, compression may delete critical evidence, injection may use a format the model cannot exploit, inference may over\-rely on injected content, and write\-back may store erroneous outputs or low\-quality reflections, which are then retrieved again in future tasks\. The loop also has a security dimension: memory\-poisoning attacks implant malicious entries into memory or knowledge bases that are later retrieved under specific triggers, as demonstrated by AgentPoison\([Chen et al\., 2024b](https://arxiv.org/html/2610.21165#bib.bib122)\), and a recent survey on long\-term memory security\([Lin et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib123)\)organizes such threats across the write, store, retrieve, execute, share, and forget phases and notes that compression can amplify poisoned content\. Since every transition is a potential point of error or attack, validation must cover writing, materialization, injection, and write\-back\.

## 7Evaluation, Benchmarks, and Metrics

Evaluating long\-context compression requires more than measuring how much information is removed\. A method that shortens the input, shrinks the cache, or retrieves less is not effective if it damages evidence, reasoning chains, user state, or long\-term consistency\. Because costs arise at different stages—prefill for input compression, decoding memory for KV compression, recall and summarization for retrieval\-conditioned compression, and writing, indexing, and maintenance for long\-term memory—evaluation should report cost–quality curves across context lengths and budgets, so that genuine improvements can be distinguished from costs shifted into external storage or background maintenance\.

### 7\.1Cost, Utility, and Long\-term Reliability

*Cost*should be reported beyond compression ratio: input tokens, cache size, peak GPU memory, prefill latency, time\-to\-first\-token, decoding latency, retrieval and compression latency, end\-to\-end latency, and throughput\. Following the cost model in Section[3\.4](https://arxiv.org/html/2610.21165#S3.SS4), we recommend reporting separately the active holding cost \(prefill tokens and peak KV memory\), the intermediate\-state holding cost \(offloaded or materialized storage\), the transition cost \(retrieval, transfer, compression, and recomputation\), and offline maintenance cost, together with the number of requests over which each cost is amortized; vectorization, graph construction, consolidation, and prefix precomputation can otherwise move inference cost into unreported background processes\.

*Task utility*should extend beyond final accuracy to evidence retention, faithfulness, reasoning consistency, hallucination rate, and task success\. Question answering should be grounded in the correct evidence, structured and graph memory should preserve relation paths and state constraints, and agent tasks should additionally be assessed on planning, tool use, failure recovery, and skill reuse\.

*Long\-term reliability*distinguishes memory\-centric compression from generic input or cache compression\. Evaluation should test whether a system writes new memories correctly, retrieves old ones at the right time, updates stale states, consolidates repeated memories, and down\-weights or forgets low\-value, erroneous, sensitive, or retracted information\. A method that optimizes single answers while accumulating erroneous reflections, stale preferences, or conflicting states is not a reliable long\-term solution\.

### 7\.2Benchmark Gaps and Evaluation Protocols

Existing benchmarks cover important but partial capabilities \(Table[4](https://arxiv.org/html/2610.21165#S7.T4)\)\. Comprehensive long\-context benchmarks test understanding, question answering, summarization, code, and reasoning over long documents; synthetic retrieval tasks test localization at extreme lengths; and long\-term memory benchmarks examine cross\-session recall, temporal reasoning, knowledge updates, and implicit conflicts\. Recent benchmarks address further gaps\. InfiniteBench\([Zhang et al\., 2024a](https://arxiv.org/html/2610.21165#bib.bib124)\)extends evaluation beyond 100K tokens; HELMET\([Yen et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib125)\)shows that synthetic tasks correlate weakly with downstream performance and adds application\-oriented tasks; NoLiMa\([Modarressi et al\., 2025](https://arxiv.org/html/2610.21165#bib.bib126)\)removes literal overlap between questions and evidence and reveals sharp degradation when semantic association is required; BABILong\([Kuratov et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib127)\)embeds multi\-fact reasoning in long distractor text; SCBench\([Li et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib128)\)evaluates shared\-context, multi\-request settings across the generation, compression, retrieval, and loading of KV caches; MemoryAgentBench\([Hu et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib129)\)evaluates retrieval, test\-time learning, long\-range understanding, and conflict resolution in incremental interaction; and STALE\([Chao et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib82)\)tests whether agents recognize memories that are no longer valid\.

These benchmarks remain dominated by static inputs or isolated capabilities\. Success on needle\-style retrieval does not imply multi\-evidence reasoning, accuracy on long\-document question answering does not imply maintenance of user state, and recalling historical facts does not imply correct updating or forgetting\. Evaluation protocols should therefore separate context comprehension, evidence localization, memory retrieval, state updating, forgetting, and agent execution, and should vary context length, evidence position, compression, cache, retrieval, and injection budgets, memory scale, and interaction rounds jointly while reporting both stage\-level and end\-to\-end cost\.

Table 4:Coverage of representative benchmarks across lifecycle capabilities\. ●: primary focus; ◐: partially covered; ○: not covered\. No benchmark jointly covers KV\-level compression, memory updating, and systematic cost reporting\.

## 8Open Challenges and Future Directions

### 8\.1Learned, Cross\-Tier Budget Allocation

Most methods rely on fixed policies, such as a fixed compression ratio, number of retrieved items, cache budget, or summary granularity, although memory requirements vary with the task: factual question answering needs verifiable evidence, summarization needs coverage, code tasks need interfaces and dependencies, long\-term dialogue needs temporally valid user state, and agents need goals, tool results, and reusable experience\. Components are also optimized in isolation even though they interact: excessive retrieval enlarges the current context, aggressive context compression degrades both generation and subsequent write\-back, and KV eviction determines whether activated information remains usable\. Allocation should therefore be decided jointly across input compression, KV retention and offloading, retrieval, post\-retrieval compression, and injection form, according to task goals, model uncertainty, and the available budget\.

#### Research question\.

Can a single allocation policy be trained with the cost model of Section[3\.4](https://arxiv.org/html/2610.21165#S3.SS4)as its objective, jointly choosing the prompt compression ratio, the fraction of KV states retained or offloaded, the number of retrieved items, and the injection form within a request? The learned policies in Sections[5\.5](https://arxiv.org/html/2610.21165#S5.SS5)and[6\.4](https://arxiv.org/html/2610.21165#S6.SS4)show that memory decisions can be learned, but their rewards include no explicit cost terms and do not cover KV\- or serving\-level decisions\.

### 8\.2Trustworthy Write\-back, Security, and Governable Forgetting

Errors in long\-term memory are dangerous because they persist\. Stale or irrelevant retrievals, compression that drops conditions, misattributed reflections, and misread tool results can be written back and reactivated in later tasks, turning a local error into a lasting bias; poisoning attacks exploit the same path deliberately\. Memories should therefore not be retained merely because they appear in dialogue, model output, or tool logs; write\-back requires evidence checking, conflict detection, provenance tracking, and versioning, and must distinguish stable facts from temporary states and uncertain inferences\. Long\-term memory also stores preferences, histories, and potentially sensitive personal information, so users need to know what is remembered, why, and when it is used, and to correct or delete it\. Because memory is compressed into summaries, embeddings, graph nodes, profiles, reflections, skills, and KV states, governance must extend from source text to all derived representations\.

#### Research question\.

When a memory is deleted or corrected, how can all of its derived representations—summaries, embeddings, graph nodes, materialized KV states \(Section[6\.3](https://arxiv.org/html/2610.21165#S6.SS3)\), and learned compressed caches—be located and invalidated, and how can poisoned content be detected at write\-back time at acceptable cost? Verifiable deletion and the tracking of derived representations are among the least studied stages of the memory lifecycle\([Lin et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib123)\)\.

### 8\.3Lifecycle\-Level Evaluation with Full Cost Accounting

Current evaluation is largely based on single inputs, single answers, or fixed compression ratios, whereas memory\-centric compression concerns a lifecycle in which a system must decide what to write, how to organize and retrieve it, how to compress it before injection, how to update it after inference, and what to forget\. Evaluation must therefore ask not only whether a compressed system answers correctly, but whether its memory remains usable, trustworthy, and governable over time, under the stress conditions and cost reporting described in Section[7](https://arxiv.org/html/2610.21165#S7)\.

#### Research question\.

Can a single benchmark cover writing, materialization, activation, updating, and forgetting, and require methods to report the itemized costs of Section[3\.4](https://arxiv.org/html/2610.21165#S3.SS4)? SCBench\([Li et al\., 2025a](https://arxiv.org/html/2610.21165#bib.bib128)\)covers the KV lifecycle, and LongMemEval\([Wu et al\., 2024](https://arxiv.org/html/2610.21165#bib.bib81)\), MemoryAgentBench\([Hu et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib129)\), and STALE\([Chao et al\., 2026](https://arxiv.org/html/2610.21165#bib.bib82)\)cover memory recall and updating, but no benchmark evaluates both within one workload\.

### 8\.4Emerging Frontiers: Reasoning Traces, Multimodality, and the Parametric Boundary

Three developments are changing the problem\. In reasoning models, long chains of thought shift the main source of active memory from input to the model’s own generations, and it is not yet understood which reasoning steps can be compressed without breaking correctness\. Multimodal long contexts from video, document images, and embodied interaction produce far more tokens than text and have different redundancy structures, which affects both compression and cross\-modal memory writing and retrieval\. At the parametric boundary, test\-time training, on\-demand adapters, and learned compressed KV caches make the choice between holding memory in context or in weights no longer binary\.

#### Research question\.

Under what query frequencies, corpus sizes, and update rates should information be kept as text, materialized as KV states, or compressed into parameters? The cost model of this survey offers a starting point, but the training and update costs of parametric representations must be brought into the same account\.

## 9Conclusion

Long\-context compression is evolving from a problem of length reduction into a problem of managing information for inference\-time computation: the bottleneck is no longer how much input a model can accept, but which information should be retained, when it should be activated, in what form it should participate in inference, and when it should be updated or forgotten\. This survey addressed this problem from a memory\-centric perspective\. We distinguished compute\-active from compute\-inactive memory, characterized intermediate states on the activation path, and described long\-context compression as a lifecycle of state transitions with a budgeted cost model\. Within this framework, prompt compression, latent memory, KV\-cache compression, architecture\-level selective computation, serving\-level cache management, external long\-term memory, retrieval augmentation, agent memory, and learned memory policies regulate storage cost, computational cost, and task utility at different points of one lifecycle\.

Future systems need to move from static, pointwise, and modular compression toward dynamic, joint, and governable memory management: allocating memory across tiers according to task and budget, preventing errors from accumulating through write\-back, supporting verifiable forgetting, and being evaluated over the full lifecycle with complete cost accounting\.

## References

- Agrawalet al\.\(2024\)A\. Agrawal, N\. Kedia, A\. Panwar, J\. Mohan, N\. Kwatra, B\. S\. Gulavani, A\. Tumanov, and R\. RamjeeTaming throughput\-latency tradeoff in LLM inference with Sarathi\-Serve\.arXiv preprint arXiv:2403\.02310\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2403.02310),[Link](https://arxiv.org/abs/2403.02310)Cited by:[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Ainslieet al\.\(2023\)J\. Ainslie, J\. Lee\-Thorp, M\. de Jong, Y\. Zemlyanskiy, F\. Lebron, and S\. SanghaiGQA: training generalized multi\-query transformer models from multi\-head checkpoints\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 4895–4901\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.298/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.298)Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p2.1)\.
- Baiet al\.\(2024a\)Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou, Y\. Dong, J\. Tang, and J\. LiLongBench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,pp\. 3119–3137\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.172)Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.6.1.1.1)\.
- Baiet al\.\(2024b\)Y\. Bai, S\. Tu, J\. Zhang, H\. Peng, X\. Wang, X\. Lv, S\. Cao, J\. Xu, L\. Hou, Y\. Dong, J\. Tang, and J\. LiLongBench v2: towards deeper understanding and reasoning on realistic long\-context multitasks\.arXiv preprint arXiv:2412\.15204\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2412.15204),[Link](https://arxiv.org/abs/2412.15204)Cited by:[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.6.1.1.1)\.
- Behrouzet al\.\(2025\)A\. Behrouz, P\. Zhong, and V\. MirrokniTitans: learning to memorize at test time\.arXiv preprint arXiv:2501\.00663\.Cited by:[§1](https://arxiv.org/html/2610.21165#S1.SS0.SSS0.Px1.p1.1)\.
- Beltagyet al\.\(2020\)I\. Beltagy, M\. E\. Peters, and A\. CohanLongformer: the long\-document transformer\.External Links:2004\.05150,[Document](https://dx.doi.org/10.48550/arXiv.2004.05150)Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1)\.
- Borgeaudet al\.\(2022\)S\. Borgeaud, A\. Mensch, J\. Hoffmann, T\. Cai, E\. Rutherford, K\. Millican, G\. B\. M\. van den Driessche, J\. Lespiau, B\. Damoc, A\. Clark, D\. de Las Casas, A\. Guy, J\. Menick, R\. Ring, T\. Hennigan, S\. Huang, L\. Maggiore, C\. Jones, A\. Cassirer, A\. Brock, M\. Paganini, G\. Irving, O\. Vinyals, S\. Osindero, K\. Simonyan, J\. Rae, E\. Elsen, and L\. SifreImproving language models by retrieving from trillions of tokens\.InProceedings of the 39th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.162,pp\. 2206–2240\.Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Caiet al\.\(2025\)Z\. Cai, W\. Xiao, H\. Sun, C\. Luo, Y\. Zhang, K\. Wan, Y\. Li, Y\. Zhou, L\. Chang, J\. Gu, Z\. Dong, A\. Anandkumar, A\. Asi, and J\. HuR\-KV: redundancy\-aware KV cache compression for reasoning models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p2.1)\.
- Changet al\.\(2026\)C\. Chang, W\. Lin, C\. Lin, H\. Chiang, Y\. Akhauri, X\. Dai, H\. Jiang, Y\. Li, L\. Ceze, K\. Wu, and M\. S\. AbdelfattahxKV: cross\-layer KV\-cache compression via aligned singular vector extraction\.InProceedings of the International Conference on Machine Learning,External Links:2503\.18893,[Document](https://dx.doi.org/10.48550/arXiv.2503.18893)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Chaoet al\.\(2026\)H\. Chao, Y\. Bai, R\. Sheng, T\. Li, and Y\. SunSTALE: can LLM agents know when their memories are no longer valid?\.arXiv preprint arXiv:2605\.06527\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2605.06527),[Link](https://arxiv.org/abs/2605.06527)Cited by:[§7\.2](https://arxiv.org/html/2610.21165#S7.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.13.1.1.1),[§8\.3](https://arxiv.org/html/2610.21165#S8.SS3.SSS0.Px1.p1.1)\.
- Chenet al\.\(2023\)S\. Chen, S\. Wong, L\. Chen, and Y\. TianExtending context window of large language models via positional interpolation\.External Links:2306\.15595,[Document](https://dx.doi.org/10.48550/arXiv.2306.15595)Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1)\.
- Chenet al\.\(2024a\)Y\. Chen, S\. Qian, H\. Tang, X\. Lai, Z\. Liu, S\. Han, and J\. JiaLongLoRA: efficient fine\-tuning of long\-context large language models\.InProceedings of the Twelfth International Conference on Learning Representations,Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1)\.
- Chenet al\.\(2024b\)Z\. Chen, Z\. Xiang, C\. Xiao, D\. Song, and B\. LiAgentPoison: red\-teaming LLM agents via poisoning memory or knowledge bases\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§6\.5](https://arxiv.org/html/2610.21165#S6.SS5.p1.1)\.
- Chenet al\.\(2025\)Z\. Chen, R\. Sadhukhan, Z\. Ye, Y\. Zhou, J\. Zhang, N\. Nolte, Y\. Tian, M\. Douze, L\. Bottou, Z\. Jia, and B\. ChenMagicPIG: LSH sampling for efficient LLM generation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.4.4.1.1),[§6\.3](https://arxiv.org/html/2610.21165#S6.SS3.SSS0.Px1.p1.1)\.
- Chevalieret al\.\(2023\)A\. Chevalier, A\. Wettig, A\. Ajith, and D\. ChenAdapting language models to compress contexts\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 3829–3846\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.232/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.232)Cited by:[§4\.2](https://arxiv.org/html/2610.21165#S4.SS2.p1.1)\.
- Chhikaraet al\.\(2025\)P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. YadavMem0: building production\-ready AI agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2504.19413),[Link](https://arxiv.org/abs/2504.19413)Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.2.4.1.1),[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Daiet al\.\(2019\)Z\. Dai, Z\. Yang, Y\. Yang, J\. Carbonell, Q\. V\. Le, and R\. SalakhutdinovTransformer\-XL: attentive language models beyond a fixed\-length context\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 2978–2988\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1285)Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1)\.
- DeepSeek\-AI \(2024a\)DeepSeek\-AIDeepSeek\-V2: a strong, economical, and efficient mixture\-of\-experts language model\.arXiv preprint arXiv:2405\.04434\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2405.04434),[Link](https://arxiv.org/abs/2405.04434)Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- DeepSeek\-AI \(2024b\)DeepSeek\-AIDeepSeek\-V3 technical report\.arXiv preprint arXiv:2412\.19437\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2412.19437),[Link](https://arxiv.org/abs/2412.19437)Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-AIDeepSeek\-V3\.2: pushing the frontier of open large language models\.arXiv preprint arXiv:2512\.02556\.Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- Dinget al\.\(2024\)Y\. Ding, L\. L\. Zhang, C\. Zhang, Y\. Xu, N\. Shang, J\. Xu, F\. Yang, and M\. YangLongRoPE: extending llm context window beyond 2 million tokens\.External Links:2402\.13753,[Document](https://dx.doi.org/10.48550/arXiv.2402.13753)Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1)\.
- Donget al\.\(2024\)X\. Dong, Y\. Fu, S\. Diao, W\. Byeon, Z\. Chen, A\. S\. Mahabaleshwarkar, S\. Liu, M\. Van Keirsbilck, M\. Chen, Y\. Suhara, Y\. Lin, J\. Kautz, and P\. MolchanovHymba: a hybrid\-head architecture for small language models\.arXiv preprint arXiv:2411\.13676\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2411.13676),[Link](https://arxiv.org/abs/2411.13676)Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- Du \(2026\)P\. DuMemory for autonomous LLM agents: mechanisms, evaluation, and emerging frontiers\.arXiv preprint arXiv:2603\.07670\.Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1)\.
- Duet al\.\(2025\)Y\. Du, W\. Huang, D\. Zheng, Z\. Wang, S\. Montella, M\. Lapata, K\. Wong, and J\. Z\. PanRethinking memory in AI: taxonomy, operations, topics, and future directions\.arXiv preprint arXiv:2505\.00675\.Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p2.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.8.1.1.1),[§3\.2](https://arxiv.org/html/2610.21165#S3.SS2.p2.1)\.
- Eyubogluet al\.\(2025\)S\. Eyuboglu, R\. Ehrlich, S\. Arora, N\. Guha, D\. Zinsley, E\. Liu, W\. Tennien, A\. Rudra, J\. Zou, A\. Mirhoseini, and C\. RéCartridges: lightweight and general\-purpose long context representations via self\-study\.arXiv preprint arXiv:2506\.06266\.Cited by:[§1](https://arxiv.org/html/2610.21165#S1.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.21165#S3.SS1.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.6.4.1.1),[§6\.3](https://arxiv.org/html/2610.21165#S6.SS3.SSS0.Px2.p1.1)\.
- Fanget al\.\(2025\)J\. Fang, X\. Deng, H\. Xu, Z\. Jiang, Y\. Tang, Z\. Xu, S\. Deng, Y\. Yao, M\. Wang, S\. Qiao, H\. Chen, and N\. ZhangLightMem: lightweight and efficient memory\-augmented generation\.arXiv preprint arXiv:2510\.18866\.Cited by:[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1)\.
- Fuet al\.\(2025\)Y\. Fu, Z\. Cai, A\. Asi, W\. Xiong, Y\. Dong, and W\. XiaoNot all heads matter: a head\-level KV cache compression method with integrated retrieval and reasoning\.InThe Thirteenth International Conference on Learning Representations,External Links:2410\.19258,[Document](https://dx.doi.org/10.48550/arXiv.2410.19258)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Gaoet al\.\(2023\)Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, M\. Wang, and H\. WangRetrieval\-augmented generation for large language models: a survey\.External Links:2312\.10997,[Document](https://dx.doi.org/10.48550/arXiv.2312.10997)Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1)\.
- Google \(2026\)GoogleGemini 3\.1 Pro Preview\.Note:Google AI for Developers documentation\. Accessed: 2026\-06\-22External Links:[Link](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-pro-preview)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p1.1)\.
- Gu and Dao \(2023\)A\. Gu and T\. DaoMamba: linear\-time sequence modeling with selective state spaces\.arXiv preprint arXiv:2312\.00752\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2312.00752),[Link](https://arxiv.org/abs/2312.00752)Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- Hsiehet al\.\(2024\)C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. GinsburgRULER: what’s the real context size of your long\-context language models?\.External Links:2404\.06654,[Document](https://dx.doi.org/10.48550/arXiv.2404.06654)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p2.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.3.1.1.1)\.
- Huet al\.\(2026\)Y\. Hu, Y\. Wang, and J\. McAuleyEvaluating memory in LLM agents via incremental multi\-turn interactions\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§7\.2](https://arxiv.org/html/2610.21165#S7.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.12.1.1.1),[§8\.3](https://arxiv.org/html/2610.21165#S8.SS3.SSS0.Px1.p1.1)\.
- Huet al\.\(2025\)Y\. Hu, S\. Liu, Y\. Yue, G\. Zhang, B\. Liu, F\. Zhu, J\. Lin, H\. Guo, S\. Dou, Z\. Xi,et al\.Memory in the age of AI agents: a survey\.arXiv preprint arXiv:2512\.13564\.Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.10.1.1.1)\.
- Jiaet al\.\(2025\)Z\. Jia, J\. Li, Y\. Kang, Y\. Wang, T\. Wu, Q\. Wang, X\. Wang, S\. Zhang, J\. Shen, Q\. Li, S\. Qi, Y\. Liang, D\. He, Z\. Zheng, and S\. ZhuThe AI hippocampus: how far are we from human memory?\.Transactions on Machine Learning Research\.Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p2.1)\.
- Jianget al\.\(2023\)H\. Jiang, Q\. Wu, C\. Lin, Y\. Yang, and L\. QiuLLMLingua: compressing prompts for accelerated inference of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 13358–13376\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.825/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.825)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[§4\.1](https://arxiv.org/html/2610.21165#S4.SS1.p1.1)\.
- Jianget al\.\(2024\)H\. Jiang, Q\. Wu, X\. Luo, D\. Li, C\. Lin, Y\. Yang, and L\. QiuLongLLMLingua: accelerating and enhancing LLMs in long context scenarios via prompt compression\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 1658–1677\.External Links:[Link](https://aclanthology.org/2024.acl-long.91/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.91)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.5.4.1.1),[§4\.1](https://arxiv.org/html/2610.21165#S4.SS1.p1.1)\.
- Jimenezet al\.\(2023\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. NarasimhanSWE\-bench: can language models resolve real\-world GitHub issues?\.arXiv preprint arXiv:2310\.06770\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2310.06770),[Link](https://arxiv.org/abs/2310.06770)Cited by:[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.14.1.1.1)\.
- Jimenez Gutierrezet al\.\(2024\)B\. Jimenez Gutierrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. SuHippoRAG: neurobiologically inspired long\-term memory for large language models\.InAdvances in Neural Information Processing Systems,External Links:[Document](https://dx.doi.org/10.48550/arXiv.2405.14831),[Link](https://arxiv.org/abs/2405.14831)Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.5.4.1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Jimenez Gutierrezet al\.\(2025\)B\. Jimenez Gutierrez, Y\. Shu, W\. Qi, S\. Zhou, and Y\. SuFrom RAG to memory: non\-parametric continual learning for large language models\.arXiv preprint arXiv:2502\.14802\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2502.14802),[Link](https://arxiv.org/abs/2502.14802)Cited by:[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Joet al\.\(2026\)D\. Jo, J\. Song, Y\. Kim, and J\. KimFastKV: decoupling of context reduction and KV cache compression for prefill\-decoding acceleration\.InFindings of the Association for Computational Linguistics: ACL 2026,External Links:2502\.01068,[Document](https://dx.doi.org/10.48550/arXiv.2502.01068)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Kanget al\.\(2025a\)J\. Kang, M\. Ji, Z\. Zhao, and T\. BaiMemory OS of AI agent\.arXiv preprint arXiv:2506\.06326\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2506.06326),[Link](https://arxiv.org/abs/2506.06326)Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.9.4.1.1),[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Kanget al\.\(2025b\)M\. Kang, W\. Chen, D\. Han, H\. A\. Inan, L\. Wutschitz, Y\. Chen, R\. Sim, and S\. RajmohanACON: optimizing context compression for long\-horizon LLM agents\.arXiv preprint arXiv:2510\.00615\.Cited by:[§6\.4](https://arxiv.org/html/2610.21165#S6.SS4.p1.1)\.
- Kimet al\.\(2025\)J\. Kim, J\. Kim, S\. Kwon, J\. W\. Lee, S\. Yun, and H\. O\. SongKVzip: query\-agnostic KV cache compression with context reconstruction\.InAdvances in Neural Information Processing Systems,External Links:2505\.23416,[Document](https://dx.doi.org/10.48550/arXiv.2505.23416)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Kimi Team \(2025\)Kimi TeamKimi linear: an expressive, efficient attention architecture\.arXiv preprint arXiv:2510\.26692\.Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- Kuratovet al\.\(2024\)Y\. Kuratov, A\. Bulatov, P\. Anokhin, I\. Rodkin, D\. Sorokin, A\. Sorokin, and M\. BurtsevBABILong: testing the limits of LLMs with long context reasoning\-in\-a\-haystack\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track,Cited by:[§7\.2](https://arxiv.org/html/2610.21165#S7.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.5.1.1.1)\.
- Kwonet al\.\(2023\)W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. StoicaEfficient memory management for large language model serving with PagedAttention\.InProceedings of the 29th Symposium on Operating Systems Principles,pp\. 611–626\.External Links:[Document](https://dx.doi.org/10.1145/3600006.3613165),[Link](https://doi.org/10.1145/3600006.3613165)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p2.1),[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Leeet al\.\(2024\)W\. Lee, J\. Lee, J\. Seo, and J\. SimInfiniGen: efficient generative inference of large language models with dynamic KV cache management\.In18th USENIX Symposium on Operating Systems Design and Implementation \(OSDI\),Cited by:[§3\.1](https://arxiv.org/html/2610.21165#S3.SS1.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.3.4.1.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.4.4.1.1),[§6\.3](https://arxiv.org/html/2610.21165#S6.SS3.SSS0.Px1.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. K"uttler, M\. Lewis, W\. Yih, T\. Rockt"aschel, S\. Riedel, and D\. KielaRetrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.5.4.1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Liet al\.\(2026\)A\. Li, S\. McLeish, H\. Chen, N\. Kalra, Z\. Chen, A\. Gazizov, V\. A\. S\. K\. Morisetty, B\. Kailkhura, H\. Menon, Z\. Liu, B\. R\. Bartoldson, T\. Goldstein, S\. Lotfi, M\. Goldblum, and P\. IzmailovEnd\-to\-end context compression at scale\.arXiv preprint arXiv:2606\.09659\.Cited by:[§4\.2](https://arxiv.org/html/2610.21165#S4.SS2.p1.1)\.
- Liet al\.\(2024a\)H\. Li, Y\. Li, A\. Tian, T\. Tang, Z\. Xu, X\. Chen, N\. Hu, W\. Dong, Q\. Li, and L\. ChenA survey on large language model acceleration based on KV cache management\.External Links:2412\.19442,[Document](https://dx.doi.org/10.48550/arXiv.2412.19442)Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.4.1.1.1)\.
- Liet al\.\(2023\)Y\. Li, B\. Dong, F\. Guerin, and C\. LinCompressing context to enhance inference efficiency of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 6342–6353\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.391/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.391)Cited by:[§4\.1](https://arxiv.org/html/2610.21165#S4.SS1.p1.1)\.
- Liet al\.\(2025a\)Y\. Li, H\. Jiang, Q\. Wu, X\. Luo, S\. Ahn, C\. Zhang, A\. H\. Abdi, D\. Li, J\. Gao, Y\. Yang, and L\. QiuSCBench: a KV cache\-centric analysis of long\-context methods\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§7\.2](https://arxiv.org/html/2610.21165#S7.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.9.1.1.1),[§8\.3](https://arxiv.org/html/2610.21165#S8.SS3.SSS0.Px1.p1.1)\.
- Liet al\.\(2024b\)Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. ChenSnapKV: LLM knows what you are looking for before generation\.InAdvances in Neural Information Processing Systems,Vol\.37\.Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.7.4.1.1),[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Liet al\.\(2025b\)Z\. Li, C\. Xi, C\. Li, D\. Chen, B\. Chen, S\. Song, S\. Niu, H\. Wang, J\. Yang, C\. Tang,et al\.MemOS: a memory OS for AI system\.arXiv preprint arXiv:2507\.03724\.Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p2.1)\.
- Liet al\.\(2025c\)Z\. Li, Y\. Liu, Y\. Su, and N\. CollierPrompt compression for large language models: a survey\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Albuquerque, New Mexico,pp\. 7182–7195\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.368)Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.3.1.1.1)\.
- Linet al\.\(2026\)Z\. Lin, C\. Li, and K\. ChenA survey on the security of long\-term memory in LLM agents: toward mnemonic sovereignty\.arXiv preprint arXiv:2604\.16548\.Cited by:[§6\.5](https://arxiv.org/html/2610.21165#S6.SS5.p1.1),[§8\.2](https://arxiv.org/html/2610.21165#S8.SS2.SSS0.Px1.p1.1)\.
- Liuet al\.\(2024a\)A\. Liu, J\. Liu, Z\. Pan, Y\. He, G\. Haffari, and B\. ZhuangMiniCache: KV cache compression in depth dimension for large language models\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:2405\.14366,[Document](https://dx.doi.org/10.48550/arXiv.2405.14366)Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.8.4.1.1),[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Liuet al\.\(2024b\)D\. Liu, M\. Chen, B\. Lu, H\. Jiang, Z\. Han, Q\. Zhang, Q\. Chen, C\. Zhang, B\. Ding, K\. Zhang, C\. Chen, F\. Yang, Y\. Yang, and L\. QiuRetrievalAttention: accelerating long\-context LLM inference via vector retrieval\.arXiv preprint arXiv:2409\.10516\.Cited by:[§3\.1](https://arxiv.org/html/2610.21165#S3.SS1.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.4.4.1.1),[§6\.3](https://arxiv.org/html/2610.21165#S6.SS3.SSS0.Px1.p1.1)\.
- Liuet al\.\(2025a\)J\. Liu, D\. Zhu, Z\. Bai, Y\. He, H\. Liao, H\. Que, Z\. Wang, C\. Zhang, G\. Zhang, J\. Zhang, Y\. Zhang, Z\. Chen, H\. Guo, S\. Li, Z\. Liu, Y\. Shan, Y\. Song, J\. Tian, W\. Wu, Z\. Zhou, R\. Zhu, J\. Feng, Y\. Gao, S\. He, Z\. Li, T\. Liu, F\. Meng, W\. Su, Y\. Tan, Z\. Wang, J\. Yang, W\. Ye, B\. Zheng, W\. Zhou, W\. Huang, S\. Li, and Z\. ZhangA comprehensive survey on long context language modeling\.External Links:2503\.17407,[Document](https://dx.doi.org/10.48550/arXiv.2503.17407)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p1.1),[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.2.1.1.1)\.
- Liuet al\.\(2026a\)J\. Liu, Y\. Su, P\. Xia, S\. Han, Z\. Zheng, C\. Xie, M\. Ding, and H\. YaoSimpleMem: efficient lifelong memory for LLM agents\.arXiv preprint arXiv:2601\.02553\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2601.02553),[Link](https://arxiv.org/abs/2601.02553)Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.8.4.1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1)\.
- Liuet al\.\(2024c\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p2.1)\.
- Liuet al\.\(2024d\)T\. Liu, C\. Xu, and J\. McAuleyRepoBench: benchmarking repository\-level code auto\-completion systems\.InProceedings of the Twelfth International Conference on Learning Representations,Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.14.1.1.1)\.
- Liuet al\.\(2026b\)X\. Liu, Z\. Tang, H\. Chen, P\. Dong, Z\. Li, X\. Zhou, B\. Li, X\. Hu, and X\. ChuSemantic integrity matters: benchmarking and preserving high\-density reasoning in KV cache compression\.InProceedings of the International Conference on Machine Learning,External Links:2502\.01941,[Document](https://dx.doi.org/10.48550/arXiv.2502.01941)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Liuet al\.\(2025b\)X\. Liu, R\. Li, M\. Huang, Z\. Liu, Y\. Song, Q\. Guo, S\. He, Q\. Wang, L\. Li, Q\. Liu, Z\. He, Y\. Zhou, X\. Huang, and X\. QiuThus spake long\-context large language model\.arXiv preprint arXiv:2502\.17129\.Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.2.1.1.1)\.
- Liuet al\.\(2025c\)Y\. Liu, J\. Yu, Y\. Xu, Z\. Li, and Q\. ZhuA survey on transformer context extension: approaches and evaluation\.External Links:2503\.13299,[Document](https://dx.doi.org/10.48550/arXiv.2503.13299)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1)\.
- Liuet al\.\(2023\)Z\. Liu, A\. Desai, F\. Liao, W\. Wang, V\. Xie, Z\. Xu, A\. Kyrillidis, and A\. ShrivastavaScissorhands: exploiting the persistence of importance hypothesis for LLM KV cache compression at test time\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Luet al\.\(2025\)E\. Lu, Z\. Jiang, J\. Liu, Y\. Du, T\. Jiang, C\. Hong, S\. Liu, W\. He, E\. Yuan, Y\. Wang, Z\. Huang, H\. Yuan, S\. Xu, X\. Xu, G\. Lai, Y\. Chen, H\. Zheng, J\. Yan, J\. Su, Y\. Wu, N\. Y\. Zhang, Z\. Yang, X\. Zhou, M\. Zhang, and J\. QiuMoBA: mixture of block attention for long\-context LLMs\.arXiv preprint arXiv:2502\.13189\.Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- Luet al\.\(2024\)S\. Lu, H\. Wang, Y\. Rong, Z\. Chen, and Y\. TangTurboRAG: accelerating retrieval\-augmented generation with precomputed KV caches for chunked text\.arXiv preprint arXiv:2410\.07590\.Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.6.4.1.1),[§6\.2](https://arxiv.org/html/2610.21165#S6.SS2.p2.1)\.
- Maet al\.\(2025\)D\. Ma, Y\. Wang, and L\. TianBlock\-attention for efficient prefilling\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§6\.2](https://arxiv.org/html/2610.21165#S6.SS2.p2.1)\.
- Maharanaet al\.\(2024\)A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. FangEvaluating very long\-term conversational memory of LLM agents\.arXiv preprint arXiv:2402\.17753\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2402.17753),[Link](https://arxiv.org/abs/2402.17753)Cited by:[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.10.1.1.1)\.
- Meiet al\.\(2025\)L\. Mei, J\. Yao, Y\. Ge, Y\. Wang, B\. Bi, Y\. Cai, J\. Liu, M\. Li, Z\. Li, D\. Zhang, C\. Zhou, J\. Mao, T\. Xia, J\. Guo, and S\. LiuA survey of context engineering for large language models\.arXiv preprint arXiv:2507\.13334\.Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p2.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.9.1.1.1)\.
- Miaoet al\.\(2023\)X\. Miao, G\. Oliaro, Z\. Zhang, X\. Cheng, H\. Jin, T\. Chen, and Z\. JiaTowards efficient generative large language model serving: a survey from algorithms to systems\.External Links:2312\.15234,[Document](https://dx.doi.org/10.48550/arXiv.2312.15234)Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1)\.
- Modarressiet al\.\(2025\)A\. Modarressi, H\. Deilamsalehy, F\. Dernoncourt, T\. Bui, R\. A\. Rossi, S\. Yoon, and H\. SchützeNoLiMa: long\-context evaluation beyond literal matching\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§7\.2](https://arxiv.org/html/2610.21165#S7.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.4.1.1.1)\.
- Muet al\.\(2023\)J\. Mu, X\. L\. Li, and N\. GoodmanLearning to compress prompts with gist tokens\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3d77c6dcc7f143aa2154e7f4d5e22d68-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[§4\.2](https://arxiv.org/html/2610.21165#S4.SS2.p1.1)\.
- Munkhdalaiet al\.\(2024\)T\. Munkhdalai, M\. Faruqui, and S\. GopalLeave no context behind: efficient infinite context transformers with infini\-attention\.arXiv preprint arXiv:2404\.07143\.External Links:[Link](https://arxiv.org/abs/2404.07143),[Document](https://dx.doi.org/10.48550/arXiv.2404.07143)Cited by:[§4\.2](https://arxiv.org/html/2610.21165#S4.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.2.1.1.1)\.
- OpenAI \(2026\)OpenAIIntroducing GPT\-5\.5\.Note:Accessed: 2026\-06\-22External Links:[Link](https://openai.com/index/introducing-gpt-5-5/)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p1.1)\.
- Packeret al\.\(2023\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: towards llms as operating systems\.External Links:2310\.08560,[Document](https://dx.doi.org/10.48550/arXiv.2310.08560)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p2.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Parket al\.\(2023\)J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. BernsteinGenerative agents: interactive simulacra of human behavior\.External Links:2304\.03442,[Document](https://dx.doi.org/10.48550/arXiv.2304.03442)Cited by:[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Penget al\.\(2023\)B\. Peng, J\. Quesnelle, H\. Fan, and E\. ShippoleYaRN: efficient context window extension of large language models\.External Links:2309\.00071,[Document](https://dx.doi.org/10.48550/arXiv.2309.00071)Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1)\.
- Prabhuet al\.\(2024\)R\. Prabhu, A\. Nayak, J\. Mohan, R\. Ramjee, and A\. PanwarvAttention: dynamic memory management for serving LLMs without PagedAttention\.arXiv preprint arXiv:2405\.04437\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2405.04437),[Link](https://arxiv.org/abs/2405.04437)Cited by:[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Qinet al\.\(2025\)R\. Qin, Z\. Li, W\. He, J\. Cui, F\. Ren, M\. Zhang, Y\. Wu, W\. Zheng, and X\. XuMooncake: trading more storage for less computation—a KVCache\-centric architecture for serving LLM chatbot\.In23rd USENIX Conference on File and Storage Technologies,Santa Clara, CA,pp\. 155–170\.External Links:[Link](https://www.usenix.org/conference/fast25/presentation/qin)Cited by:[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Ramachandranet al\.\(2026\)A\. Ramachandran, M\. Neseem, C\. Sakr, R\. Venkatesan, B\. Khailany, and T\. KrishnaThinKV: thought\-adaptive KV cache compression for efficient reasoning models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p2.1)\.
- Rasmussenet al\.\(2025\)P\. Rasmussen, P\. Paliychuk, T\. Beauvais, J\. Ryan, and D\. ChalefZep: a temporal knowledge graph architecture for agent memory\.arXiv preprint arXiv:2501\.13956\.Cited by:[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1)\.
- Rehg \(2024\)I\. RehgKV\-Compress: paged KV\-cache compression with variable compression rates per attention head\.arXiv preprint arXiv:2410\.00161\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2410.00161)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Shanet al\.\(2025\)L\. Shan, S\. Luo, Z\. Zhu, Y\. Yuan, and Y\. WuCognitive memory in large language models\.arXiv preprint arXiv:2504\.02441\.Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p2.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.7.1.1.1)\.
- Shazeer \(2019\)N\. ShazeerFast transformer decoding: one write\-head is all you need\.arXiv preprint arXiv:1911\.02150\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1911.02150),[Link](https://arxiv.org/abs/1911.02150)Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.arXiv preprint arXiv:2303\.11366\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2303.11366),[Link](https://arxiv.org/abs/2303.11366)Cited by:[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Shuet al\.\(2026\)Y\. Shu, S\. P\. Jonnalagedda, X\. Gao, B\. Jimenez Gutierrez, W\. Qi, K\. Das, H\. Sun, and Y\. SuREMem: reasoning with episodic memory in language agent\.arXiv preprint arXiv:2602\.13530\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2602.13530),[Link](https://arxiv.org/abs/2602.13530)Cited by:[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Sunet al\.\(2025a\)H\. Sun, L\. Chang, W\. Bao, S\. Zheng, N\. Zheng, X\. Liu, H\. Dong, Y\. Chi, and B\. ChenShadowKV: KV cache in shadows for high\-throughput long\-context LLM inference\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§3\.1](https://arxiv.org/html/2610.21165#S3.SS1.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.3.4.1.1),[§6\.3](https://arxiv.org/html/2610.21165#S6.SS3.SSS0.Px1.p1.1)\.
- Sun and Zeng \(2025\)H\. Sun and S\. ZengHierarchical memory for high\-efficiency long\-term reasoning in LLM agents\.arXiv preprint arXiv:2507\.22925\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2507.22925),[Link](https://arxiv.org/abs/2507.22925)Cited by:[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1)\.
- Sunet al\.\(2025b\)W\. Sun, M\. Lu, Z\. Ling, K\. Liu, X\. Yao, Y\. Yang, and J\. ChenScaling long\-horizon LLM agent via context\-folding\.arXiv preprint arXiv:2510\.11967\.Cited by:[§6\.4](https://arxiv.org/html/2610.21165#S6.SS4.p1.1)\.
- Tanget al\.\(2024\)J\. Tang, Y\. Zhao, K\. Zhu, G\. Xiao, B\. Kasikci, and S\. HanQuest: query\-aware sparsity for efficient long\-context LLM inference\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1),[§6\.3](https://arxiv.org/html/2610.21165#S6.SS3.SSS0.Px1.p1.1)\.
- Verma \(2024\)S\. VermaContextual compression in retrieval\-augmented generation for large language models: a survey\.External Links:2409\.13385,[Document](https://dx.doi.org/10.48550/arXiv.2409.13385)Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1)\.
- Wanget al\.\(2023a\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2305.16291),[Link](https://arxiv.org/abs/2305.16291)Cited by:[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Wanget al\.\(2023b\)W\. Wang, L\. Dong, H\. Cheng, X\. Liu, X\. Yan, J\. Gao, and F\. WeiAugmenting language models with long\-term memory\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2306.07174)Cited by:[§6\.3](https://arxiv.org/html/2610.21165#S6.SS3.SSS0.Px1.p1.1)\.
- Wanget al\.\(2026a\)Y\. Wang, Z\. Wang, Y\. Shi, S\. Chen, X\. Wang, Y\. Wang, B\. Shen, L\. Li, X\. Gu, J\. McAuley, and D\. D\. ZengContext compression for LLM agents: a survey of methods, failure modes, and evaluation\.Note:Preprints\.org preprint 202605\.2065, doi:10\.20944/preprints202605\.2065\.v1Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.11.1.1.1),[§6\.4](https://arxiv.org/html/2610.21165#S6.SS4.p1.1)\.
- Wanget al\.\(2026b\)Y\. Wang, S\. Ji, Y\. Liu, Q\. Zhu, and W\. CheEchoKV: efficient KV cache compression via similarity\-based reconstruction\.arXiv preprint arXiv:2603\.22910\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2603.22910)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Wang and Chen \(2025\)Y\. Wang and X\. ChenMIRIX: multi\-agent memory system for LLM\-based agents\.arXiv preprint arXiv:2507\.07957\.Cited by:[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1)\.
- Wuet al\.\(2025a\)B\. Wu, Z\. Zhang, Y\. Zhong, G\. Huang, Y\. Zhu, X\. Liu, and X\. JinTokenLake: a unified segment\-level prefix cache pool for fine\-grained elastic long\-context LLM serving\.arXiv preprint arXiv:2508\.17219\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2508.17219),[Link](https://arxiv.org/abs/2508.17219)Cited by:[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Wuet al\.\(2024\)D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. YuLongMemEval: benchmarking chat assistants on long\-term interactive memory\.arXiv preprint arXiv:2410\.10813\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2410.10813),[Link](https://arxiv.org/abs/2410.10813)Cited by:[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.11.1.1.1),[§8\.3](https://arxiv.org/html/2610.21165#S8.SS3.SSS0.Px1.p1.1)\.
- Wuet al\.\(2025b\)Y\. Wu, S\. Liang, C\. Zhang, Y\. Wang, Y\. Zhang, H\. Guo, R\. Tang, and Y\. LiuFrom human memory to AI memory: a survey on memory mechanisms in the era of LLMs\.arXiv preprint arXiv:2504\.15965\.Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p2.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.6.1.1.1)\.
- Xiaoet al\.\(2024a\)C\. Xiao, P\. Zhang, X\. Han, G\. Xiao, Y\. Lin, Z\. Zhang, Z\. Liu, and M\. SunInfLLM: training\-free long\-context extrapolation for LLMs with an efficient context memory\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Link](https://arxiv.org/abs/2402.04617)Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p2.1)\.
- Xiaoet al\.\(2024b\)G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. LewisEfficient streaming language models with attention sinks\.InThe Twelfth International Conference on Learning Representations,Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.7.4.1.1),[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Xuet al\.\(2026a\)A\. Xu, B\. Lin, B\. Xue, B\. Wang, B\. Xu, B\. Wu, B\. Zhang, C\. Lin, C\. Dong, C\. Ling,et al\.DeepSeek\-v4: towards highly efficient million\-token context intelligence\.arXiv preprint arXiv:2606\.19348\.Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p1.1),[§1](https://arxiv.org/html/2610.21165#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- Xuet al\.\(2025a\)D\. Xu, Y\. Wen, P\. Jia, Y\. Zhang, W\. Zhang, Y\. Wang, H\. Guo, R\. Tang, X\. Zhao, E\. Chen, and T\. XuTowards multi\-granularity memory association and selection for long\-term conversational agents\.arXiv preprint arXiv:2505\.19549\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.19549),[Link](https://arxiv.org/abs/2505.19549)Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.8.4.1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1)\.
- Xuet al\.\(2025b\)W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. ZhangA\-MEM: agentic memory for LLM agents\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.2.4.1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1)\.
- Xuet al\.\(2026b\)Y\. Xu, N\. K\. Khaira, and T\. SinghKV cache optimization strategies for scalable and efficient LLM inference\.arXiv preprint arXiv:2603\.20397\.Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1)\.
- Yanet al\.\(2025\)S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, J\. Bi, K\. Kersting, J\. Z\. Pan, H\. Schütze, V\. Tresp, and Y\. MaMemory\-R1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.arXiv preprint arXiv:2508\.19828\.Cited by:[§5\.5](https://arxiv.org/html/2610.21165#S5.SS5.p1.1)\.
- Yanget al\.\(2024\)Z\. Yang, J\. N\. Han, K\. Wu, R\. Xie, A\. Wang, X\. Sun, and Z\. KangLossless KV cache compression to 2%\.arXiv preprint arXiv:2410\.15252\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2410.15252)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Yaoet al\.\(2025\)J\. Yao, H\. Li, Y\. Liu, S\. Ray, Y\. Cheng, Q\. Zhang, K\. Du, S\. Lu, and J\. JiangCacheBlend: fast large language model serving for RAG with cached knowledge fusion\.InProceedings of the European Conference on Computer Systems \(EuroSys\),Cited by:[§6\.2](https://arxiv.org/html/2610.21165#S6.SS2.p2.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InProceedings of the Eleventh International Conference on Learning Representations,Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1)\.
- Yenet al\.\(2025\)H\. Yen, T\. Gao, M\. Hou, K\. Ding, D\. Fleischer, P\. Izsak, M\. Wasserblat, and D\. ChenHELMET: how to evaluate long\-context language models effectively and thoroughly\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§7\.2](https://arxiv.org/html/2610.21165#S7.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.8.1.1.1)\.
- Yuet al\.\(2022\)G\. Yu, J\. S\. Jeong, G\. Kim, S\. Kim, and B\. ChunOrca: a distributed serving system for Transformer\-Based generative models\.In16th USENIX Symposium on Operating Systems Design and Implementation,Carlsbad, CA,pp\. 521–538\.External Links:[Link](https://www.usenix.org/conference/osdi22/presentation/yu)Cited by:[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Yuet al\.\(2025a\)H\. Yu, T\. Chen, J\. Feng, J\. Chen, W\. Dai, Q\. Yu, Y\. Zhang, W\. Ma, J\. Liu, M\. Wang, and H\. ZhouMemAgent: reshaping long\-context LLM with multi\-conv RL\-based memory agent\.arXiv preprint arXiv:2507\.02259\.Cited by:[§6\.4](https://arxiv.org/html/2610.21165#S6.SS4.p1.1)\.
- Yuet al\.\(2025b\)Y\. Yu, Q\. Zhang, L\. Qiao, D\. Yin, F\. Li, J\. Wang, Z\. Chen, S\. Zheng, X\. Liang, and X\. SunSequential\-NIAH: a needle\-in\-a\-haystack benchmark for extracting sequential needles from long contexts\.arXiv preprint arXiv:2504\.04713\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2504.04713),[Link](https://arxiv.org/abs/2504.04713)Cited by:[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.2.1.1.1)\.
- Yuanet al\.\(2025\)J\. Yuan, H\. Gao, D\. Dai, J\. Luo, L\. Zhao, Z\. Zhang, Z\. Xie, Y\. X\. Wei, L\. Wang, Z\. Xiao, Y\. Wang, C\. Ruan, M\. Zhang, W\. Liang, and W\. ZengNative sparse attention: hardware\-aligned and natively trainable sparse attention\.InProceedings of the Annual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§4\.4](https://arxiv.org/html/2610.21165#S4.SS4.p1.1)\.
- Zaheeret al\.\(2020\)M\. Zaheer, G\. Guruganesh, A\. Dubey, J\. Ainslie, C\. Alberti, S\. Onta n’on, P\. Pham, A\. Ravula, Q\. Wang, L\. Yang, and A\. AhmedBig Bird: transformers for longer sequences\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 17283–17297\.Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1)\.
- Zhanget al\.\(2025a\)C\. Zhang, K\. Du, S\. Liu, W\. Kwon, X\. Mo, Y\. Wang, X\. Liu, K\. You, Z\. Li, M\. Long, J\. Zhai, J\. Gonzalez, and I\. StoicaJenga: effective memory management for serving LLM with heterogeneity\.arXiv preprint arXiv:2503\.18292\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2503.18292),[Link](https://arxiv.org/abs/2503.18292)Cited by:[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Zhanget al\.\(2025b\)P\. Zhang, Z\. Liu, S\. Xiao, N\. Shao, Q\. Ye, and Z\. DouLong context compression with activation beacon\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=1eQT9OzfNQ)Cited by:[§4\.2](https://arxiv.org/html/2610.21165#S4.SS2.p1.1)\.
- Zhanget al\.\(2024a\)X\. Zhang, Y\. Chen, S\. Hu, Z\. Xu, J\. Chen, M\. K\. Hao, X\. Han, Z\. L\. Thai, S\. Wang, Z\. Liu, and M\. Sun∞\\inftyBench: Extending long context evaluation beyond 100K tokens\.InProceedings of the Annual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§7\.2](https://arxiv.org/html/2610.21165#S7.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.7.1.1.1)\.
- Zhanget al\.\(2025c\)Y\. Zhang, Y\. Hu, R\. Zhao, J\. C\. S\. Lui, and H\. ChenDiffKV: differentiated memory management for large language models with parallel KV compaction\.InProceedings of the ACM SIGOPS Symposium on Operating Systems Principles,External Links:2412\.03131,[Document](https://dx.doi.org/10.48550/arXiv.2412.03131)Cited by:[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Zhanget al\.\(2024b\)Z\. Zhang, X\. Bo, C\. Ma, R\. Li, X\. Chen, Q\. Dai, J\. Zhu, Z\. Dong, and J\. WenA survey on the memory mechanism of large language model based agents\.External Links:2404\.13501,[Document](https://dx.doi.org/10.48550/arXiv.2404.13501)Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.21165#S2.T1.4.5.1.1.1)\.
- Zhanget al\.\(2023\)Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. R’e, C\. Barrett, Z\. Wang, and B\. ChenH2O: heavy\-hitter oracle for efficient generative inference of large language models\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2610.21165#S1.p2.1),[§1](https://arxiv.org/html/2610.21165#S1.p3.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.7.4.1.1),[§4\.3](https://arxiv.org/html/2610.21165#S4.SS3.p1.1)\.
- Zhenget al\.\(2024\)L\. Zheng, L\. Yin, Z\. Xie, C\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez, C\. Barrett, and Y\. ShengSGLang: efficient execution of structured language model programs\.InAdvances in Neural Information Processing Systems,Cited by:[§2\.1](https://arxiv.org/html/2610.21165#S2.SS1.p1.1),[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Zhonget al\.\(2023\)W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. WangMemoryBank: enhancing large language models with long\-term memory\.arXiv preprint arXiv:2305\.10250\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2305.10250),[Link](https://arxiv.org/abs/2305.10250)Cited by:[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.2.4.1.1),[Table 2](https://arxiv.org/html/2610.21165#S3.T2.4.9.4.1.1),[§5\.1](https://arxiv.org/html/2610.21165#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2610.21165#S5.SS2.p1.1),[§5\.3](https://arxiv.org/html/2610.21165#S5.SS3.p1.1)\.
- Zhonget al\.\(2024\)Y\. Zhong, S\. Liu, J\. Chen, J\. Hu, Y\. Zhu, X\. Liu, X\. Jin, and H\. ZhangDistServe: disaggregating prefill and decoding for goodput\-optimized large language model serving\.arXiv preprint arXiv:2401\.09670\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2401.09670),[Link](https://arxiv.org/abs/2401.09670)Cited by:[§4\.5](https://arxiv.org/html/2610.21165#S4.SS5.p1.1)\.
- Zhouet al\.\(2023\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. NeubigWebArena: a realistic web environment for building autonomous agents\.arXiv preprint arXiv:2307\.13854\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2307.13854),[Link](https://arxiv.org/abs/2307.13854)Cited by:[Table 4](https://arxiv.org/html/2610.21165#S7.T4.4.14.1.1.1)\.
- Zhouet al\.\(2025\)Z\. Zhou, A\. Qu, Z\. Wu, S\. Kim, A\. Prakash, D\. Rus, J\. Zhao, B\. K\. H\. Low, and P\. P\. LiangMEM1: learning to synergize memory and reasoning for efficient long\-horizon agents\.arXiv preprint arXiv:2506\.15841\.Cited by:[§6\.4](https://arxiv.org/html/2610.21165#S6.SS4.p1.1)\.
- Zhouet al\.\(2024\)Z\. Zhou, X\. Ning, K\. Hong, T\. Fu, J\. Xu, S\. Li, Y\. Lou, L\. Wang, Z\. Yuan, X\. Li, S\. Yan, G\. Dai, X\. Zhang, Y\. Dong, and Y\. WangA survey on efficient inference for large language models\.External Links:2404\.14294,[Document](https://dx.doi.org/10.48550/arXiv.2404.14294)Cited by:[§2\.2](https://arxiv.org/html/2610.21165#S2.SS2.p1.1)\.