CoreMem: Riemannian Retrieval and Fisher-Guided Distillation for Long-Term Memory in Dialogue Agents

# Riemannian Retrieval and Fisher-Guided Distillation for Long-Term Memory in Dialogue Agents
Source: [https://arxiv.org/html/2606.18406](https://arxiv.org/html/2606.18406)
Jiaqi Chen1Yongqin Zeng1Shaoshen Chen1Yijian Zhang1 Hai\-Tao Zheng1,2Chunxia Ma3,4XiuTeng Zhou4 1Shenzhen International Graduate School, Tsinghua University 2Peng Cheng Laboratory 3Shandong Analysis and Test Center, Qilu University of Technology, Jinan, China 4State Key Laboratory for Quality Ensurance and Sustainable Use of Dao\-di Herbs, Beijing, China

###### Abstract

Personalized dialogue agents require continuous long\-term memory to maintain coherent interactions across multiple sessions\. However, deploying these capabilities on consumer\-grade hardware \(e\.g\., 8 GB VRAM edge devices\) introduces severe memory and compute bottlenecks\. Existing systems typically rely on isotropic cosine similarity for retrieval and heuristic rules for context compression\. These approaches lack a unified theoretical foundation, frequently suffering from the hubness problem in high\-dimensional retrieval and syntactic fragmentation during compression\. To overcome these limitations, we proposeCoreMem, a resource\-efficient edge\-cloud memory architecture fundamentally unified byinformation geometry\. First,Riemannian retrievalreplaces cosine matching with a locally adaptive Fisher\-Rao metric, effectively penalizing hub memories via Mahalanobis distance with𝒪​\(N​d​r\)\\mathcal\{O\}\(Ndr\)Woodbury acceleration for real\-time search\. Second,Fisher\-guided discrete token distillation \(FDTD\)introduces a hierarchical sentence\-to\-token compression mechanism\. It derives sensitivity scores from Fisher information traces, providing a principled compression\-KL tradeoff augmented with explicit structural syntax protection\. Evaluated on the LOCOMO and LongMemEval\-S benchmarks, CoreMem achieves strong accuracy improvements, yielding substantial gains in Open\-domain \(\+4\.51 pp\) and Temporal \(\+4\.17 pp\) reasoning\. Extensive profiling confirms that CoreMem operates seamlessly within a strict 8 GB VRAM budget, successfully bridging the gap between resource\-constrained edge devices and the demand for theoretically grounded, lifelong memory agents\.

CoreMem: Riemannian Retrieval and Fisher\-Guided Distillation for Long\-Term Memory in Dialogue Agents

## 1Introduction

Large language models \(LLMs\) have catalyzed a paradigm shift in conversational AI, enabling agents that act as personal assistants, therapists, and continuous companions\. As these agents transition from cloud\-exclusive deployments to consumer\-grade devices—such as laptops with 8–16 GB RAM and edge GPUs with 6–12 GB VRAM—the efficient management of long\-term memory has emerged as a critical bottleneck\. Users naturally expect agents to maintain coherent personas and recall facts across dozens of sessions\. However, the dominant industry practice ofcontext stuffing—greedily concatenating all historical interactions into the prompt—is economically unsustainable and mathematically suboptimal\. Beyond inflating API costs linearly with conversation length, excessive context exacerbates position bias and “lost\-in\-the\-middle” hallucinations\(Liuet al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib2); Hsiehet al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib10)\)\.

The Consumer\-Grade Constraint\.Consider a typical edge\-cloud deployment: a user operates an NVIDIA RTX 4060 Laptop GPU \(8 GB VRAM\) running a local embedding model, relying on a cloud API \(e\.g\., GPT\-4o\-mini;OpenAI,[2023](https://arxiv.org/html/2606.18406#bib.bib47)\) for final generation\. A competitive embedding model alone consumes approximately 3 GB of VRAM\. This leaves a razor\-thin margin for the memory indexing system, local context compressors, and OS overhead\. Existing prominent memory frameworks–such as MemGPT\(Packeret al\.,[2023](https://arxiv.org/html/2606.18406#bib.bib31)\), MemoryBank\(Zhonget al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib32)\), and Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib33)\)—were predominantly engineered with server\-grade assumptions, lacking the algorithmic frugality required for this resource\-constrained regime\.

Beyond hardware constraints, the economic cost of cloud API calls scales linearly with prompt length\. A typical long\-dialogue session with 1,500 tokens of historical context, queried 50 times per day, incurs approximately $0\.75 daily at GPT\-4o\-mini rates\. Over a month, this accumulates to $22\.50 per user–a prohibitive cost for consumer\-grade products\. By compressing context locally before transmission, CoreMem substantially reduces the volume of tokens transmitted to the cloud LLM, directly translating to proportional API cost savings \(§[5\.6](https://arxiv.org/html/2606.18406#S5.SS6)\)\. These constraints demand a memory architecture that is simultaneouslyVRAM\-friendly\(operating within 6 GB\),time\-friendly\(sub\-100 ms per query\), andtoken\-friendly\(compressing prompts by 20–30%\)\. This triple imperative motivates our pursuit of a theoretically grounded, resource\-efficient memory system\.

Current architectures suffer from three limitations under strict budgets\.First, isotropic cosine retrieval is vulnerable tohubness: a small subset of “hub” memories dominates nearest\-neighbor lists, drowning out geometrically peripheral but semantically critical facts\(Radovanovićet al\.,[2010](https://arxiv.org/html/2606.18406#bib.bib38)\)\.Second, compression algorithms lack theoretical guarantees; heuristic pruning provides no KL\-divergence bound, making deployment a trial\-and\-error gamble\(Panet al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib28); Liet al\.,[2023a](https://arxiv.org/html/2606.18406#bib.bib29)\)\.Third, retrieval and compression are optimized with disjoint objectives, creatingcascade failureswhere retrieved facts lose their structural connectors \(e\.g\., “because”\) during compression\.

![Refer to caption](https://arxiv.org/html/2606.18406v1/x1.png)Figure 1:CoreMem edge\-cloud hybrid pipeline\. The local edge runs aVRAM\-friendlyembedding/retrieval stack, atime\-friendlyRiemannian search and FDTD compressor, and transmits only atoken\-friendlydistilled context to the cloud LLM, reducing API costs by∼\\sim20%\.To bridge these gaps, we proposeCoreMem\(Curvature\-orientedRiemannianEdgeMemory, Figure[1](https://arxiv.org/html/2606.18406#S1.F1)\), an edge\-cloud hybrid memory architecture elegantly unified by the lens ofinformation geometry\(Amari,[2016](https://arxiv.org/html/2606.18406#bib.bib39)\)\. Rather than treating retrieval and compression as disjoint heuristics, CoreMem views both through the geometry of statistical manifolds: retrieval is formulated as distance measurement against a spatially distorted embedding manifold \(using the inverse covariance\), while compression evaluates token importance by measuring the curvature of the loss landscape \(using the diagonal Fisher Information Matrix\)\. In the CoreMem pipeline, a local small language model \(SLM\) retrieves and distills historical context, transmitting only the strictly necessary, highly dense information to the cloud LLM\.

Our primary contributions are summarized as follows:

1. 1\.We introduceRiemannian retrieval\(§[3\.2](https://arxiv.org/html/2606.18406#S3.SS2)\), treating memory embeddings as points on a statistical manifold\. By employing a locally adaptive Fisher\-Rao metric via Mahalanobis distance with𝒪​\(N​d​r\)\\mathcal\{O\}\(Ndr\)Woodbury acceleration, we fundamentally mitigate the hubness problem\.
2. 2\.We deriveFisher\-guided discrete token distillation\(FDTD; §[3\.3](https://arxiv.org/html/2606.18406#S3.SS3)\), a hierarchical sentence\-to\-token compression algorithm\. It leverages Fisher information traces to provide a principled compression\-KL tradeoff, explicitly integrated with structural syntax protection\.
3. 3\.We provide comprehensivestress tests and module ablations\(§[5](https://arxiv.org/html/2606.18406#S5)\) to validate the robustness of Riemannian retrieval and FDTD compression under extreme token budgets and structural configurations, revealing that Riemannian and cosine metrics exhibit complementary strengths for distinct reasoning topologies\.
4. 4\.Evaluations on LOCOMO and LongMemEval\-S demonstrate strong accuracy, with decisive gains on Open\-domain \(\+4\.51 pp\) and Temporal \(\+4\.17 pp\) reasoning, all within an 8 GB VRAM ceiling\.

## 2Related Work

#### Agent Memory Systems\.

Pioneering frameworks like MemGPT\(Packeret al\.,[2023](https://arxiv.org/html/2606.18406#bib.bib31)\)paginate context between RAM and disk to emulate operating systems\. MemoryBank\(Zhonget al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib32)\)simulates human forgetting via Ebbinghaus decay, while recent systems like Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib33)\), MemoryOS\(Kanget al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib35)\), CarMem\(Kirmayret al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib36)\), and A\-mem\(Xuet al\.,[2026](https://arxiv.org/html/2606.18406#bib.bib34)\)employ vector search combined with segmented paging or knowledge graphs\. However, these systems inherently target server\-grade hardware\. None employ information\-geometric principles or derive theoretically bounded compression to optimize for the strict compute ceilings of edge devices\.

#### Context Compression\.

Existing distillation techniques typically rely on auxiliary small models or heuristic rules\. LLMLingua2\(Panet al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib28)\)and LongLLMLingua\(Jianget al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib1)\)frame token pruning as token\-level classification tasks, while SelectiveContext\(Liet al\.,[2023a](https://arxiv.org/html/2606.18406#bib.bib29)\)and C3\(Liu and Qiu,[2025](https://arxiv.org/html/2606.18406#bib.bib23)\)filter sentences based on self\-information or context cascade compression\. AutoCompressor\(Chevalieret al\.,[2023](https://arxiv.org/html/2606.18406#bib.bib8)\)and GIST\(Muet al\.,[2023](https://arxiv.org/html/2606.18406#bib.bib7)\)learn soft prompt representations\. A recent survey byLiet al\.\([2025](https://arxiv.org/html/2606.18406#bib.bib11)\)catalogs over twenty prompt compression methods, yet notes that none provides explicit theoretical bounds on semantic loss\. Our FDTD, inspired by neural network pruning theories\(Molchanovet al\.,[2017](https://arxiv.org/html/2606.18406#bib.bib41); Kunstneret al\.,[2019](https://arxiv.org/html/2606.18406#bib.bib3)\), derives sensitivity directly from the diagonal Fisher Information Matrix, yielding a mathematically principled compression\-KL tradeoff\.

#### Riemannian Metrics in NLP\.

Information geometry\(Amari,[2016](https://arxiv.org/html/2606.18406#bib.bib39); Amari and Nagaoka,[2000](https://arxiv.org/html/2606.18406#bib.bib6)\)rigorously defines the statistical manifold, but its application to real\-time NLP retrieval remains scarce due to the prohibitive𝒪​\(d3\)\\mathcal\{O\}\(d^\{3\}\)complexity of covariance inversion\. Concurrent work byBhardwaj \([2026](https://arxiv.org/html/2606.18406#bib.bib24)\)explores Fisher\-Rao distance on strictly diagonal Gaussian manifolds\. In contrast, our approach captures complex cross\-dimensional correlations by employing a low\-rank SVD correction over the diagonal covariance, aggressively optimized via Woodbury acceleration for near\-instantaneous search\. Beyond information\-geometric metrics, concurrent work also improves dense retrieval through token\-aware embedding augmentation for fine\-grained lexical\-semantic alignment\(Zhanet al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib49)\)\.

#### Distinction from Prior Work\.

Table[1](https://arxiv.org/html/2606.18406#S2.T1)highlights the key differences between CoreMem and representative baselines\. Unlike MemGPT and MemoryBank, which were designed for server\-grade hardware, CoreMem targets consumer\-grade edge devices with strict 8 GB VRAM constraints\. Unlike heuristic compressors \(LLMLingua2, SelectiveContext\), FDTD provides a principled KL\-divergence bound via Fisher information\. Unlike pure cosine retrieval with strict truncation \(NaiveRAG, Mem0\), CoreMem introduces a locally adaptive Riemannian metric to mitigate hubness\. To our knowledge, CoreMem is the first system to unify retrieval and compression under a single information\-geometric framework while maintaining real\-time latency on edge hardware\.

Table 1:High\-level comparison with representative baselines\. Edge = feasible on 8 GB VRAM consumer hardware\.

## 3Methodology

### 3\.1Problem Formulation

A dialogue agent maintains a historical memory setℳ=\{m1,…,mN\}\\mathcal\{M\}=\\\{m\_\{1\},\\dots,m\_\{N\}\\\}with corresponding continuous representations defined as𝐇=\[𝐡1,…,𝐡N\]⊤∈ℝN×d\\mathbf\{H\}=\[\\mathbf\{h\}\_\{1\},\\dots,\\mathbf\{h\}\_\{N\}\]^\{\\top\}\\in\\mathbb\{R\}^\{N\\times d\}\. Given a user queryqqembedded as𝐪∈ℝd\\mathbf\{q\}\\in\\mathbb\{R\}^\{d\}, the agent executes a three\-stage pipeline: \(1\)Retrievethe top\-kkrelevant memories fromℳ\\mathcal\{M\}to form a raw context poolXX; \(2\)DistillXXinto a condensed representationX~\\tilde\{X\}satisfying a strict token budgetBB; \(3\)Generatethe final answera∼pθ​\(a∣X~,q\)a\\sim p\_\{\\theta\}\(a\\mid\\tilde\{X\},q\)via a cloud\-based LLM\. Our overarching objective is to maximize the factual accuracy of generation while strictly bounding both the local computational overhead and the cloud API token consumption\.

### 3\.2Riemannian Retrieval

#### Motivation\.

The prevalent use of cosine similarity,sim​\(𝐪,𝐡\)=𝐪⊤​𝐡‖𝐪‖​‖𝐡‖\\text\{sim\}\(\\mathbf\{q\},\\mathbf\{h\}\)=\\frac\{\\mathbf\{q\}^\{\\top\}\\mathbf\{h\}\}\{\\\|\\mathbf\{q\}\\\|\\\|\\mathbf\{h\}\\\|\}, treats all latent dimensions isotropically\. In high\-dimensional spaces \(d≥1024d\\geq 1024\), this isotropy induces severehubness: a small subset of embedding vectors unpredictably emerges as nearest neighbors to a vast number of queries, irrespective of true semantic relevance\(Radovanovićet al\.,[2010](https://arxiv.org/html/2606.18406#bib.bib38)\)\. As visually corroborated in Figure[2](https://arxiv.org/html/2606.18406#S3.F2), we require a geometrically aware metric capable of penalizing dense hub regions and stretching collapsed dimensions to surface peripheral but crucial tail memories\.

![Refer to caption](https://arxiv.org/html/2606.18406v1/fig2_riemannian_v2.png)Figure 2:Cosine versus Fisher\-Rao Riemannian geometry under hubness\. Panel \(a\) isotropic cosine: a small number of large hub memories cluster tightly around the query and absorb nearest\-neighbour traffic from many directions \(converging arrows\), while smaller tail memories sit beyond the dotted cosine boundary and are visibly unreachable — the textbook hubness regime\. Panel \(b\) locally adaptive Fisher\-Rao: the elongated metric ellipse with its two eigenvector arrows depicts the local Mahalanobis tensor; the stretched manifold geometrically pushes the same hubs out of the neighbourhood and pulls the previously peripheral tail memories inside, restoring them as plausible neighbours\.
#### Locally Adaptive Covariance and Woodbury Acceleration\.

We measure semantic distance via the Fisher\-Rao metric, which under a Gaussian assumption reduces to the Mahalanobis distance: each embedding is treated as the mean of a Gaussian whose covariance is estimated from the data, making the Fisher information matrix equal to the inverse covariance\. After L2\-normalizing the embeddings to a unit hypersphere, we compute the empirical variance along each dimension:𝝈2=Var​\(\{𝐡i\}i=1N\)\\boldsymbol\{\\sigma\}^\{2\}=\\text\{Var\}\(\\\{\\mathbf\{h\}\_\{i\}\\\}\_\{i=1\}^\{N\}\)\.

To construct an invertible and stable local covariance matrix, we model it as𝚺≈𝐔r​𝐌r​𝐔r⊤\+𝐃\\boldsymbol\{\\Sigma\}\\approx\\mathbf\{U\}\_\{r\}\\mathbf\{M\}\_\{r\}\\mathbf\{U\}\_\{r\}^\{\\top\}\+\\mathbf\{D\}, where𝐃=diag​\(𝝈2\+λ\)\\mathbf\{D\}=\\text\{diag\}\(\\boldsymbol\{\\sigma\}^\{2\}\+\\lambda\)\. Crucially, we dynamically set the ridge smoothing parameterλ=10⋅mean​\(𝝈2\)\\lambda=10\\cdot\\text\{mean\}\(\\boldsymbol\{\\sigma\}^\{2\}\)\. This algorithmic choice theoretically bounds the condition number of𝚺−1\\boldsymbol\{\\Sigma\}^\{\-1\}to approximately2×2\\times, effectively preserving the necessary anisotropy of the manifold while explicitly preventing extreme geometric distortion during inversion\(Maesschalcket al\.,[2000](https://arxiv.org/html/2606.18406#bib.bib40)\)\.

To capture cross\-dimensional correlations beyond the diagonal matrix𝐃\\mathbf\{D\}, we apply Singular Value Decomposition \(SVD\) on the centered memory embeddings, dynamically selecting the minimal rankrrthat explainsτ=0\.95\\tau=0\.95of the total variance\. By applying the Woodbury matrix identity, we bypass the intractable𝒪​\(d3\)\\mathcal\{O\}\(d^\{3\}\)matrix inversion, precomputing the exact inverse efficiently:

𝚺−1=𝐃−1−𝐃−1​𝐔r\(𝐌r−1\+𝐔r⊤​𝐃−1​𝐔r\)−1​𝐔r⊤​𝐃−1\\boldsymbol\{\\Sigma\}^\{\-1\}=\\mathbf\{D\}^\{\-1\}\-\\mathbf\{D\}^\{\-1\}\\mathbf\{U\}\_\{r\}\\\\ \(\\mathbf\{M\}\_\{r\}^\{\-1\}\+\\mathbf\{U\}\_\{r\}^\{\\top\}\\mathbf\{D\}^\{\-1\}\\mathbf\{U\}\_\{r\}\)^\{\-1\}\\mathbf\{U\}\_\{r\}^\{\\top\}\\mathbf\{D\}^\{\-1\}\(1\)At query time, calculating the Riemannian score requires only𝒪​\(N​d​r\)\\mathcal\{O\}\(Ndr\)operations, ensuring real\-time responsiveness—atime\-friendlyproperty even on edge GPUs\.

#### Residual Metric Fusion\.

While Riemannian distance excels at precise tail retrieval, cosine similarity retains utility for locating generalized “bridge facts” in multi\-hop reasoning\. To reap the complementary benefits of both, we introduce a residual fusion mechanism, defined as the convex combination of min\-max normalized similarities:

score​\(𝐪,𝐡\)=α⋅norm​\(simcos\)\+\(1−α\)⋅norm​\(simrie\)\\text\{score\}\(\\mathbf\{q\},\\mathbf\{h\}\)=\\alpha\\cdot\\text\{norm\}\(\\text\{sim\}\_\{\\text\{cos\}\}\)\+\(1\-\\alpha\)\\cdot\\text\{norm\}\(\\text\{sim\}\_\{\\text\{rie\}\}\)\(2\)We empirically setα=0\.5\\alpha=0\.5, providing a balanced interpolation between local geometric awareness and global semantic matching\.

### 3\.3Fisher\-Guided Discrete Token Distillation \(FDTD\)

#### Objective and Diagonal Fisher Approximation\.

Given the retrieved contextX=\(x1,…,xM\)X=\(x\_\{1\},\\dots,x\_\{M\}\)embedded as a dense matrix𝐇∈ℝM×d\\mathbf\{H\}\\in\\mathbb\{R\}^\{M\\times d\}, context compression seeks an optimal binary mask𝐦∈\{0,1\}M\\mathbf\{m\}\\in\\\{0,1\\\}^\{M\}that minimizes the generation KL\-divergenceDKL\(pθ\(Y\|X\)∥pθ\(Y\|X~\)\)D\_\{\\text\{KL\}\}\(p\_\{\\theta\}\(Y\|X\)\\,\\\|\\,p\_\{\\theta\}\(Y\|\\tilde\{X\}\)\)subject to a strict capacity budget∑mi≤B\\sum m\_\{i\}\\leq B\.

Evaluating the exact combinatorial loss over discrete tokens is computationally intractable\. Furthermore, calculating the full covariance Fisher Information Matrix \(FIM\) for the input space requires prohibitive VRAM allocations\. Therefore, we employ a strict diagonal approximation integrated with a first\-order Taylor expansion on the loss landscape\(Molchanovet al\.,[2017](https://arxiv.org/html/2606.18406#bib.bib41)\)\. Assuming local decoupling between dimensions, we evaluate the per\-token sensitivity via the trace of the diagonal FIM:

S​\(xi\)=‖∇𝐡iℒ⊙𝐡i‖2≈Tr​\(ℐ𝐇i​i​𝐇i​𝐇i⊤\)S\(x\_\{i\}\)=\\\|\\nabla\_\{\\mathbf\{h\}\_\{i\}\}\\mathcal\{L\}\\odot\\mathbf\{h\}\_\{i\}\\\|\_\{2\}\\approx\\text\{Tr\}\\big\(\\mathcal\{I\}\_\{\\mathbf\{H\}\_\{ii\}\}\\mathbf\{H\}\_\{i\}\\mathbf\{H\}\_\{i\}^\{\\top\}\\big\)\(3\)where∇𝐡iℒ\\nabla\_\{\\mathbf\{h\}\_\{i\}\}\\mathcal\{L\}denotes the gradient of the language modeling loss with respect to tokenii’s embedding, obtainable via a single, memory\-efficient backward pass\.

This formulation elegantly bridges heuristic pruning with rigorous mathematical bounds\. We formalize this in the following theorem:

###### Theorem 1\(Compression Error Bound\)

Assume the generative modelθ\\thetasatisfies localLL\-Lipschitz continuity on the embedding support of the dropped tokens\. By formulating the compression mask𝐦\\mathbf\{m\}such that tokens are dropped in ascending order of their local Fisher sensitivityS​\(xi\)S\(x\_\{i\}\), the induced generation KL\-divergence is explicitly bounded by:DK​L≤𝒪​\(L2​∑mi=0S​\(xi\)\)D\_\{KL\}\\leq\\mathcal\{O\}\\big\(L^\{2\}\\sum\_\{m\_\{i\}=0\}S\(x\_\{i\}\)\\big\)\.

\*\(The detailed mathematical proof is provided in Appendix[F](https://arxiv.org/html/2606.18406#A6)\)\.\*

#### Hierarchical Sentence\-Level Distillation\.

A naive, globally greedy token\-dropping strategy often violently shatters linguistic syntax, yielding fragmented prompts that severely degrade the cloud LLM’s comprehension\. To circumvent this, FDTD implements aHierarchical Sentence\-Level Distillationframework\.

Rather than discarding arbitrary tokens globally, the algorithm first chunks the context into conversational turns and granular sentences\. For long sentences \(length\>8\>8tokens\), we apply a safe intra\-sentence pruning mechanism: we strictly protect the first three and last three tokens \(thereby preserving the foundational subject\-verb\-object skeleton\) and exclusively drop the lowest\-scoring redundant tokens within the unprotected middle segment\. Finally, sentences are selected greedily based on their post\-pruning mean scores until the budgetBBis reached\.

#### Structural Protection Masks\.

The mathematical diagonal Fisher approximation inherently suffers from blind spots in discrete natural language; for instance, it frequently assigns near\-zero scores to crucial logical connectors due to attention sparsity\. To compensate, we superimpose heuristically engineered structural priors over the raw sensitivity scores:

Sfinal​\(xi\)=S​\(xi\)⋅max⁡\(𝐌syntax,𝐌content\)⋅𝐌decayS\_\{\\text\{final\}\}\(x\_\{i\}\)=S\(x\_\{i\}\)\\cdot\\max\(\\mathbf\{M\}\_\{\\text\{syntax\}\},\\mathbf\{M\}\_\{\\text\{content\}\}\)\\cdot\\mathbf\{M\}\_\{\\text\{decay\}\}\(4\)Specifically, these explicitly regularize the token manifold: \(1\)Inference Keyword Boost: Epistemic and logical connectors \(e\.g\.,but, because, although\) receive a1\.2×1\.2\\timesmultiplier to prevent causal chain breakage; \(2\)Syntax Preservation \(𝐌syntax\\mathbf\{M\}\_\{\\text\{syntax\}\}\): Structural punctuation \(:, newline\) and syntax boundaries \(\., ?\) are heavily protected \(1\.31\.3–1\.5×1\.5\\times\); \(3\)Content Boost \(𝐌content\\mathbf\{M\}\_\{\\text\{content\}\}\): Digits and capitalized proper nouns are boosted by1\.31\.3–1\.4×1\.4\\timesto retain dense factual anchors; \(4\)Turn\-level Decay \(𝐌decay\\mathbf\{M\}\_\{\\text\{decay\}\}\): The relevance of historical context decays linearly \(1\.0→0\.61\.0\\rightarrow 0\.6\) corresponding to the dialogue turn depth; \(5\)Local Gap Filling: As a post\-processing step, if the textual distance between two retained tokens is≤3\\leq 3, the intervening tokens are automatically recovered to maintain phrasal fluency\. \*\(A comprehensive list of protected vocabularies and the distillation pseudocode are relegated to Appendix[E](https://arxiv.org/html/2606.18406#A5)\)\.\*

### 3\.4Edge\-Cloud Hybrid Architecture

Figure[1](https://arxiv.org/html/2606.18406#S1.F1)illustrates the holistic CoreMem pipeline deployed under consumer constraints\. On theEdge Device, an efficient encoder \(gte\-Qwen2\-1\.5B\(Liet al\.,[2023b](https://arxiv.org/html/2606.18406#bib.bib43); Yanget al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib42)\), 1536\-dim\) embeds the query\. The Riemannian retriever rapidly selects the top\-kkhistorical memories using the precomputed Woodbury inverse\. Subsequently, a local causal small language model \(SLM, e\.g\., Qwen3\-0\.6B\(Yanget al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib42)\), with frozen weights\) executes the FDTD algorithm to compress the context down to budgetBB\. For theCloud Phase, only the highly distilled, information\-dense contextX~\\tilde\{X\}is transmitted over the network to a commercial LLM \(e\.g\., GPT\-4o\-mini\(OpenAI,[2023](https://arxiv.org/html/2606.18406#bib.bib47)\)\) for final generation\. This decoupled architecture confines memory\-intensive storage and retrieval to the local device, compressing context before cloud transmission\. Full latency and footprint profiling is reported in §[5\.6](https://arxiv.org/html/2606.18406#S5.SS6)\.

## 4Experimental Setup

### 4\.1Datasets and Metrics

We systematically evaluate CoreMem on two comprehensive long\-context dialogue benchmarks:LOCOMO\(Maharanaet al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib26)\), consisting of 1,542 QA pairs across complex topologies \(Single\-hop, Multi\-hop, Temporal, Open\-domain, and Adversarial\); andLongMemEval\-S\(Wuet al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib27)\)\(266 QA\), testing factual aggregation across temporally distant sessions\. Metrics include lexical overlap \(ROUGE\-L\(Lin,[2004](https://arxiv.org/html/2606.18406#bib.bib30)\)\), retrieval accuracy \(Hit@kk\), and semantic correctness via LLM\-as\-a\-Judge\(Zhenget al\.,[2023](https://arxiv.org/html/2606.18406#bib.bib48)\)\(GPT\-4o\-mini\(OpenAI,[2023](https://arxiv.org/html/2606.18406#bib.bib47),[2024](https://arxiv.org/html/2606.18406#bib.bib22)\), temperature 0\.0, strict binary protocol; see Appendix[E](https://arxiv.org/html/2606.18406#A5)\)\. A response is correct only if the judge output starts with “yes” after stripping and lowercasing\.

### 4\.2Baselines and Implementation

We benchmark against leading memory and compression architectures: NaiveRAG \(cosine retrieval with strict truncation\)\(Khandelwalet al\.,[2020](https://arxiv.org/html/2606.18406#bib.bib13)\), Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib33)\), MemoryBank\(Zhonget al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib32)\), MemoryOS\(Kanget al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib35)\), LLMLingua2\(Panet al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib28)\), and SelectiveContext\(Liet al\.,[2023a](https://arxiv.org/html/2606.18406#bib.bib29)\)\. Notably, NaiveRAG, Mem0, and MemoryOS all rely on standard cosine similarity for retrieval and differ only in superficial memory\-management heuristics \(pagination, FIFO eviction, or simple deduplication\)\. Consequently, their end\-to\-end performance is nearly identical \(Table[2](https://arxiv.org/html/2606.18406#S5.T2)\), confirming that retrieval metric—not memory organization—is the dominant bottleneck\.

All experiments are executed on a consumer\-grade NVIDIA RTX 4080 Laptop GPU \(12 GB VRAM\)\. For the embedding space, we evaluate both a high\-dimensional model, gte\-Qwen2\-1\.5B \(1536\-dim\)\(Liet al\.,[2023b](https://arxiv.org/html/2606.18406#bib.bib43); Yanget al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib42)\), and a lower\-dimensional baseline, all\-MiniLM\-L6\-v2 \(384\-dim\)\(Wanget al\.,[2020](https://arxiv.org/html/2606.18406#bib.bib44)\)\. For brevity, we refer to this model asMiniLM\-L6in the remainder of this paper\. The FDTD module utilizes lightweight causal SLMs, including Qwen3\-0\.6B\(Yanget al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib42)\), Llama\-3\.2\-1B\(Team,[2024](https://arxiv.org/html/2606.18406#bib.bib45)\), and SmolLM2\-360M\(allalet al\.,[2025](https://arxiv.org/html/2606.18406#bib.bib46)\), maintaining a strict local\-compute footprint\.

We evaluate three CoreMem retrieval variants:CoreMem\-V3\-Pure\(α=0\\alpha=0\), relying solely on the Riemannian metric;CoreMem\-V3\(α=0\.9\\alpha=0\.9\), strongly favoring Riemannian retrieval with minor cosine correction; andCoreMem\-Fusion\(α=0\.5\\alpha=0\.5\), equally weighting both metrics\. \*\(Comprehensive hyperparameter settings are detailed in Appendix[B](https://arxiv.org/html/2606.18406#A2)\)\.\*

## 5Results and Analysis

### 5\.1End\-to\-End Comparison

Table[2](https://arxiv.org/html/2606.18406#S5.T2)reports the primary end\-to\-end LOCOMO results across both embedding models\. CoreMem\-Fusion consistently achieves the highest Judge accuracy, with decisive gains on Open\-domain \(\+4\.51 pp\) and Temporal \(\+4\.17 pp\) reasoning under MiniLM\-L6\. It also attains the highest ROUGE\-L \(0\.227\) and Hit@10 \(0\.532\) on MiniLM, confirming that Riemannian retrieval surfaces high\-quality memories even with low\-dimensional embeddings\. MemoryBank lags severely \(Judge 0\.097\), confirming that biologically inspired exponential forgetting destroys factual density in objective QA tasks\.

Table 2:LOCOMO end\-to\-end results \(1,542 QA\)\. All systems use<<6 GB VRAM\. R\-L = ROUGE\-L, Jdg = binary strict LLM\-as\-a\-Judge \(overall accuracy\), Open = Open\-domain, Temp = Temporal, PTok = prompt tokens, Time\(s\) = total wall\-clock time in seconds for the full benchmark\.†denotesp<0\.001p<0\.001vs\. strongest baseline \(McNemar\)\. Best accuracy metrics in bold\.#### Query\-Type Breakdown\.

CoreMem\-Fusion gains are concentrated in Open\-domain \(\+4\.51 pp\) and Temporal \(\+4\.17 pp\) reasoning, where peripheral tail memories are most valuable\. The anisotropic Riemannian metric stretches low\-variance dimensions to surface these tails, while the Fusion interpolator \(α=0\.5\\alpha=0\.5\) prevents them from being drowned out by bridge\-fact hubs\.

#### Embedding Model Ablation\.

On MiniLM\-L6 \(384\-dim\), CoreMem\-Fusion achieves \+3\.70 pp Judge improvement, highest ROUGE\-L \(0\.227\) and Hit@10 \(0\.532\), confirming that Riemannian correction provides the greatest value when embedding quality is limited\. On gte\-Qwen2 \(1536\-dim\), gains are narrower \(Judge 0\.536\) because high\-quality embeddings suffer less from hubness\. This asymmetric profile demonstrates that CoreMem isparticularly advantageous for small embedding models\.

### 5\.2Cross\-Benchmark Generalization

Table[3](https://arxiv.org/html/2606.18406#S5.T3)reports results on LongMemEval\-S \(266 QA, MiniLM\-L6\), which tests factual aggregation across temporally distant sessions\. CoreMem\-Fusion achieves the highest overall Judge accuracy \(0\.4624\), with strong gains on temporal reasoning \(TR\-Judge 0\.3533, \+2\.25 pp over NaiveRAG\) and multi\-session consistency \(MS\-Judge 0\.5714\)\. At approximately 500 memories, retrieval strategies begin to converge, suggesting CoreMem’s advantage is most pronounced at medium scale where hubness is severe but not yet diluted by sheer memory volume\.

Table 3:LongMemEval\-S results \(266 QA, MiniLM\-L6, all systems<<100 MB VRAM\)\. MS=Multi\-Session, TR=Temporal Reasoning\. Best in bold\.On LongMemEval\-M \(133 QA, gte\-Qwen2\), CoreMem\-Fusion improves Judge accuracy to 0\.4060 \(\+0\.75 pp over NaiveRAG\), with the largest gain in ROUGE\-L \(\+5\.05 pp\)\. This confirms that Riemannian retrieval generalizes across both embedding dimensionalities and benchmark designs\.

### 5\.3Compression Stress Test and Semantic Divergence

Table[4](https://arxiv.org/html/2606.18406#S5.T4)reports the retrieval\-aware compression stress test under a unified NaiveRAG retriever \(MiniLM\-L6, LOCOMO Temporal, 96 QA, budget 1000\)\. All compressors receive identical retrieved contexts, so performance differences stem solely from compression quality\. CoreMem \(SmolLM2\) achieves the highest conditional accuracy \(0\.9756\) and agreement rate \(0\.9688\), while simultaneously attaining the most aggressive compression ratio \(67\.4%\)\. This demonstrates that Fisher\-guided sensitivity scores effectively identify and preserve task\-critical tokens beyond shallow heuristics\.

Table 4:Compression stress test \(budget 1000, unified NaiveRAG retrieval, MiniLM\-L6\)\. CA = Conditional Accuracy, AR = Agreement Rate, Ratio = compressed tokens / original tokens\.The critical divergence between lexical ROUGE\-L and semantic Judge accuracy \(r≈0\.56r\\approx 0\.56\) further validates the necessity of semantic evaluation beyond naive lexical overlap\(Zhenget al\.,[2023](https://arxiv.org/html/2606.18406#bib.bib48)\)\. FDTD achieves high ROUGE\-L because structural protection retains answer\-related entities, while the Judge metric captures whether those entities form a coherent reasoning chain\. Under extreme starvation \(B=250B=250\), all learned compressors face degradation; detailed budgets are reported in Appendix[C](https://arxiv.org/html/2606.18406#A3)\.

### 5\.4Module Ablations

To validate FDTD’s structural protection mechanisms, we conducted an ablation study on Temporal QA \(B=500B=500\)\. Removing theSyntax & Keyword Boost\(𝐌syntax,𝐌semantics\\mathbf\{M\}\_\{\\text\{syntax\}\},\\mathbf\{M\}\_\{\\text\{semantics\}\}\) resulted in a marginal ROUGE\-L drop \(\-0\.002\) but a severe Judge accuracy collapse \(\-4\.5 pp\)\. This stark contrast proves that while epistemic connectors \(e\.g\., “before”, “after”\) contribute minimally to lexical scores, they constitute the absolute causal backbone for language model reasoning\.

Table[5](https://arxiv.org/html/2606.18406#S5.T5)presents the full structural ablation on MiniLM\-L6 \(Temporal, 96 QA\)\. Disabling gap filling atB=1000B=1000causes Hit@1 to collapse from 0\.0938 to 0\.0, demonstrating that isolated high\-sensitivity tokens are meaningless without local syntactic context\. Removing content\-word boost degrades ROUGE\-L atB=250B=250\(\-0\.004\), confirming that proper\-noun protection is essential for factual QA\.

Table 5:Structural protection ablation \(MiniLM\-L6, LOCOMO Temporal, 96 QA\)\.![Refer to caption](https://arxiv.org/html/2606.18406v1/x2.png)Figure 3:Effect of hybrid weightα\\alphaon LOCOMO Judge accuracy \(MiniLM\-L6\)\. Pure Riemannian \(α=0\\alpha=0\) excels on Open\-domain and Temporal; strong Riemannian bias \(α=0\.9\\alpha=0\.9\) underperforms; balanced Fusion \(α=0\.5\\alpha=0\.5\) achieves the best overall accuracy\.
### 5\.5Case Studies

Qualitative cases confirm that under severe compression \(B=500B=500\), FDTD preserves named entities \(e\.g\., “Under Armour”\) and fine\-grained role distinctions \(“filmmaker” vs\. “screenwriter”\) that baselines drop \(Appendix[D](https://arxiv.org/html/2606.18406#A4)\)\. In end\-to\-end retrieval, CoreMem\-Fusion surfaces multi\-fact aggregations \(e\.g\., “cooking classes” \+ “friends like Evan”\) and resolves temporal references \(“August 2022”\) that cosine retrieval misses\.

### 5\.6Efficiency Profiling on Consumer Hardware

A core claim of CoreMem is edge deployment feasibility\. Profiling on an 8 GB VRAM setup confirms four efficiency gains\. The gte\-Qwen2\-1\.5B encoder plus Qwen3\-0\.6B distiller consumes a combined peak VRAM of merely 3\.2 GB; with Riemannian indexing overhead \(<200<200MB forN=1,000N=1,000\), the total edge footprint stays well under 6 GB\. Both retrieval and compression execute entirely on the edge device, eliminating network round\-trips for unretrieved memories\. By compressing local contexts before transmission, CoreMem achieves prompt\-to\-budget ratios of 20–25% atB=500B=500and 25–30% atB=1,000B=1,000\(Table[4](https://arxiv.org/html/2606.18406#S5.T4)\), directly translating to proportional API cost savings\. Finally, because no historical memory leaves the local machine unless explicitly retrieved, user privacy is preserved by default\.

### 5\.7Discussion

#### Complementarity of Riemannian and Cosine Retrieval\.

Our retrieval variants reveal striking metric complementarity \(Figure[3](https://arxiv.org/html/2606.18406#S5.F3)\)\. Pure Riemannian \(α=0\\alpha=0\) excels on Open\-domain \(\+2\.96 pp\) and Temporal \(\+3\.12 pp\) by surfacing peripheral tail memories, yet underperforms cosine by 1\.07 pp on Multi\-hop reasoning where bridge facts form natural hubs\. The Fusion interpolator \(α=0\.5\\alpha=0\.5\) balances these opposing strengths, yielding the best overall accuracy\.

#### Embedding Quality and Riemannian Value\.

Riemannian correction provides the greatest value under restrictive resource constraints\. On gte\-Qwen2 \(1536\-dim\), CoreMem\-Fusion trails NaiveRAG by 0\.49 pp, as premium embeddings already mitigate hubness; yet this encoder alone consumes∼\\sim5\.9 GB VRAM, leaving virtually no headroom on an 8 GB consumer GPU\. In contrast, MiniLM\-L6 \(384\-dim\) occupies merely 96 MB VRAM—a realistic edge footprint—where CoreMem\-Fusion exceeds NaiveRAG by 1\.21 pp\. This confirms that Riemannian retrieval is a decisive remedy for resource\-constrained deployments, extending the usable lifespan of smaller encoders\.

#### Compression Budget and Distiller Trade\-offs\.

At budget 1000, CoreMem\-V3 \(SmolLM2\) matches LLMLingua2 \(0\.4271\) and achieves 0\.9756 conditional accuracy \(Table[4](https://arxiv.org/html/2606.18406#S5.T4)\), outperforming all baselines\. At budget 500, all learned compressors fall behind \(0\.2396–0\.3125 vs\. 0\.3958\)\. The 1B Llama distiller underperforms 0\.6B Qwen3, confirming that architectural compatibility matters more than parameter count\. Local compression latency remains on the order of 1 s per query, negligible compared to cloud API response times \(∼\\sim140–155 s\), so the pipeline remains practically time\-friendly\.

#### LLM\-as\-a\-Judge vs\. ROUGE\-L Divergence\.

The moderate correlation \(r≈0\.56r\\approx 0\.56\) between ROUGE\-L and Judge exposes a critical divergence between lexical and semantic metrics\(Geet al\.,[2024](https://arxiv.org/html/2606.18406#bib.bib4)\)\. Atb=250b=250, CoreMem\-V3 achieves the highest ROUGE\-L \(0\.1379\) yet lowest Judge \(0\.1979\), because structural protection preserves words that boost lexical overlap without guaranteeing coherent reasoning\. This validates LLM Judge as a necessary complementary metric\(Weiet al\.,[2022](https://arxiv.org/html/2606.18406#bib.bib19)\)\.

## 6Conclusion

We present CoreMem, an information\-geometric memory architecture unifying Riemannian retrieval and Fisher\-guided compression for edge dialogue agents\. By grounding retrieval in the Fisher\-Rao metric and compression in task\-specific sensitivity, CoreMem models the non\-Euclidean geometry of memory manifolds\. Evaluations on LOCOMO and LongMemEval\-S demonstrate accuracy under an 8 GB VRAM ceiling, with decisive gains on Open\-domain \(\+4\.51 pp\) and Temporal \(\+4\.17 pp\) reasoning\. The resulting pipeline preserves user privacy and reduces API costs via local context compression\. Future work will extend this framework to streaming dialogue settings and explore tighter integration with quantized edge SLMs\.

## Limitations

1. \(1\)Edge Deployment Constraints: The local SLMs evaluated in FDTD \(0\.6B–1B parameters\) are inherently constrained by edge VRAM limits, and CoreMem currently employs a flat memory representation without explicit hierarchical organization\.
2. \(2\)Generalization and Scale Boundaries: Our experiments are confined to English\-centric benchmarks, and the robustness of structural protection masks on morphologically rich languages remains unexplored\. While Woodbury acceleration reduces Riemannian search to𝒪​\(N​d​r\)\\mathcal\{O\}\(Ndr\), overhead increases substantially when memory banks exceedN\>100,000N\>100,000\.
3. \(3\)Evaluation Scope: We validate Riemannian retrieval and FDTD compression independently, but a full factorial E2E ablation quantifying their interaction is not included due to API cost constraints\.

## AI Assistance Declaration

We acknowledge the use of Gemini \(Google\) for language polishing andLaTeXformatting, and GPT Image 2 \(OpenAI\) for figure assistance\. All technical contributions, experimental design, and scientific conclusions remain the sole responsibility of the authors\.

## References

- L\. B\. allal, A\. Lozhkov, E\. Bakouch, G\. M\. Blazquez, G\. Penedo, L\. Tunstall, A\. Marafioti, A\. P\. Lajarín, H\. Kydlíček, V\. Srivastav, J\. Lochner, C\. Fahlgren, X\. S\. NGUYEN, B\. Burtenshaw, C\. Fourrier, H\. Zhao, H\. Larcher, M\. Morlon, C\. Zakka, C\. Raffel, L\. V\. Werra, and T\. Wolf \(2025\)SmolLM2: when smol goes big — data\-centric training of a fully open small language model\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=3JiCl2A14H)Cited by:[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p2.1)\.
- S\. Amari \(2016\)Information geometry and its applications\.1st edition,Springer Publishing Company, Incorporated\.External Links:ISBN 4431559779Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p5.1),[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px3.p1.1)\.
- S\. Amari and H\. Nagaoka \(2000\)Methods of information geometry\.Translations of Mathematical Monographs191\.External Links:[Document](https://dx.doi.org/10.1090/mmono/191)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px3.p1.1)\.
- V\. P\. Bhardwaj \(2026\)SuperLocalMemory v3: information\-geometric foundations for zero\-llm agent memory\.arXiv preprint\.External Links:[Link](https://api.semanticscholar.org/CorpusID:286572301)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Chevalier, A\. Wettig, A\. Ajith, and D\. Chen \(2023\)Adapting language models to compress contexts\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6\-10, 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),pp\. 3829–3846\.External Links:[Link](https://doi.org/10.18653/v1/2023.emnlp-main.232),[Document](https://dx.doi.org/10.18653/V1/2023.EMNLP-MAIN.232)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1)\.
- P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav \(2025\)Mem0: building production\-ready ai agents with scalable long\-term memory\.InEuropean Conference on Artificial Intelligence,External Links:[Link](https://api.semanticscholar.org/CorpusID:278165315)Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p2.1),[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p1.1)\.
- T\. Ge, J\. Hu, L\. Wang, X\. Wang, S\. Chen, and F\. Wei \(2024\)In\-context autoencoder for context compression in a large language model\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=uREj4ZuGJE)Cited by:[§5\.7](https://arxiv.org/html/2606.18406#S5.SS7.SSS0.Px4.p1.2)\.
- C\. Hsieh, Y\. Chuang, C\. Li, Z\. Wang, L\. T\. Le, A\. Kumar, J\. R\. Glass, A\. Ratner, C\. Lee, R\. Krishna, and T\. Pfister \(2024\)Found in the middle: calibrating positional attention bias improves long context utilization\.InFindings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11\-16, 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Findings of ACL,pp\. 14982–14995\.External Links:[Link](https://doi.org/10.18653/v1/2024.findings-acl.890),[Document](https://dx.doi.org/10.18653/V1/2024.FINDINGS-ACL.890)Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p1.1)\.
- H\. Jiang, Q\. Wu, X\. Luo, D\. Li, C\. Lin, Y\. Yang, and L\. Qiu \(2024\)LongLLMLingua: accelerating and enhancing LLMs in long context scenarios via prompt compression\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 1658–1677\.External Links:[Link](https://aclanthology.org/2024.acl-long.91/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.91)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Kang, M\. Ji, Z\. Zhao, and T\. Bai \(2025\)Memory OS of AI agent\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 25961–25970\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1318/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1318),ISBN 979\-8\-89176\-332\-6Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p1.1)\.
- U\. Khandelwal, O\. Levy, D\. Jurafsky, L\. Zettlemoyer, and M\. Lewis \(2020\)Generalization through memorization: nearest neighbor language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=HklBjCEKvH)Cited by:[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p1.1)\.
- J\. Kirmayr, L\. Stappen, P\. Schneider, F\. Matthes, and E\. Andre \(2025\)CarMem: enhancing long\-term memory in LLM voice assistants through category\-bounding\.InProceedings of the 31st International Conference on Computational Linguistics: Industry Track,O\. Rambow, L\. Wanner, M\. Apidianaki, H\. Al\-Khalifa, B\. D\. Eugenio, S\. Schockaert, K\. Darwish, and A\. Agarwal \(Eds\.\),Abu Dhabi, UAE,pp\. 343–357\.External Links:[Link](https://aclanthology.org/2025.coling-industry.29/)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px1.p1.1)\.
- F\. Kunstner, P\. Hennig, and L\. Balles \(2019\)Limitations of the empirical fisher approximation for natural gradient descent\.InAdvances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8\-14, 2019, Vancouver, BC, Canada,H\. M\. Wallach, H\. Larochelle, A\. Beygelzimer, F\. d’Alché\-Buc, E\. B\. Fox, and R\. Garnett \(Eds\.\),pp\. 4158–4169\.External Links:[Link](https://proceedings.neurips.cc/paper/2019/hash/46a558d97954d0692411c861cf78ef79-Abstract.html)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Li, B\. Dong, F\. Guerin, and C\. Lin \(2023a\)Compressing context to enhance inference efficiency of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 6342–6353\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.391/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.391)Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p4.1),[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p1.1)\.
- Z\. Li, X\. Zhang, Y\. Zhang, D\. Long, P\. Xie, and M\. Zhang \(2023b\)Towards general text embeddings with multi\-stage contrastive learning\.ArXivabs/2308\.03281\.External Links:[Link](https://api.semanticscholar.org/CorpusID:260682258)Cited by:[§3\.4](https://arxiv.org/html/2606.18406#S3.SS4.p1.3),[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p2.1)\.
- Z\. Li, Y\. Liu, Y\. Su, and N\. Collier \(2025\)Prompt compression for large language models: A survey\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 \- Volume 1: Long Papers, Albuquerque, New Mexico, USA, April 29 \- May 4, 2025,L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),pp\. 7182–7195\.External Links:[Link](https://doi.org/10.18653/v1/2025.naacl-long.368),[Document](https://dx.doi.org/10.18653/V1/2025.NAACL-LONG.368)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1)\.
- C\. Lin \(2004\)ROUGE: a package for automatic evaluation of summaries\.InText Summarization Branches Out,Barcelona, Spain,pp\. 74–81\.External Links:[Link](https://aclanthology.org/W04-1013/)Cited by:[§4\.1](https://arxiv.org/html/2606.18406#S4.SS1.p1.1)\.
- F\. Liu and H\. Qiu \(2025\)Context cascade compression: exploring the upper limits of text compression\.ArXivabs/2511\.15244\.External Links:[Link](https://api.semanticscholar.org/CorpusID:283103462)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1)\.
- N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. Liang \(2024\)Lost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Link](https://aclanthology.org/2024.tacl-1.9/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p1.1)\.
- R\. D\. Maesschalck, D\. Jouan\-Rimbaud, and D\. L\. Massart \(2000\)The mahalanobis distance\.Chemometrics and Intelligent Laboratory Systems50,pp\. 1–18\.External Links:[Link](https://api.semanticscholar.org/CorpusID:119793807)Cited by:[§3\.2](https://arxiv.org/html/2606.18406#S3.SS2.SSS0.Px2.p2.5)\.
- A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang \(2024\)Evaluating very long\-term conversational memory of LLM agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 13851–13870\.External Links:[Link](https://aclanthology.org/2024.acl-long.747/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.747)Cited by:[§4\.1](https://arxiv.org/html/2606.18406#S4.SS1.p1.1)\.
- P\. Molchanov, S\. Tyree, T\. Karras, T\. Aila, and J\. Kautz \(2017\)Pruning convolutional neural networks for resource efficient inference\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=SJGCiw5gl)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2606.18406#S3.SS3.SSS0.Px1.p2.3)\.
- J\. Mu, X\. Li, and N\. D\. Goodman \(2023\)Learning to compress prompts with gist tokens\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2023/hash/3d77c6dcc7f143aa2154e7f4d5e22d68-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1)\.
- OpenAI \(2023\)GPT\-4 technical report\.CoRRabs/2303\.08774\.External Links:[Link](https://doi.org/10.48550/arXiv.2303.08774),[Document](https://dx.doi.org/10.48550/ARXIV.2303.08774),2303\.08774Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p2.1),[§3\.4](https://arxiv.org/html/2606.18406#S3.SS4.p1.3),[§4\.1](https://arxiv.org/html/2606.18406#S4.SS1.p1.1)\.
- OpenAI \(2024\)GPT\-4o system card\.ArXivabs/2410\.21276\.External Links:[Link](https://api.semanticscholar.org/CorpusID:285313550)Cited by:[§4\.1](https://arxiv.org/html/2606.18406#S4.SS1.p1.1)\.
- C\. Packer, V\. Fang, S\. G\. Patil, K\. Lin, S\. Wooders, and J\. E\. Gonzalez \(2023\)MemGPT: towards llms as operating systems\.CoRRabs/2310\.08560\.External Links:[Link](https://doi.org/10.48550/arXiv.2310.08560),[Document](https://dx.doi.org/10.48550/ARXIV.2310.08560),2310\.08560Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p2.1),[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Pan, Q\. Wu, H\. Jiang, M\. Xia, X\. Luo, J\. Zhang, Q\. Lin, V\. Rühle, Y\. Yang, C\. Lin, H\. V\. Zhao, L\. Qiu, and D\. Zhang \(2024\)LLMLingua\-2: data distillation for efficient and faithful task\-agnostic prompt compression\.InFindings of the Association for Computational Linguistics: ACL 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 963–981\.External Links:[Link](https://aclanthology.org/2024.findings-acl.57/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.57)Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p4.1),[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p1.1)\.
- M\. Radovanović, A\. Nanopoulos, and M\. Ivanović \(2010\)Hubs in space: popular nearest neighbors in high\-dimensional data\.J\. Mach\. Learn\. Res\.11,pp\. 2487–2531\.External Links:[Link](https://api.semanticscholar.org/CorpusID:12182489)Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p4.1),[§3\.2](https://arxiv.org/html/2606.18406#S3.SS2.SSS0.Px1.p1.2)\.
- L\. Team \(2024\)The llama 3 herd of models\.CoRRabs/2407\.21783\.External Links:[Link](https://doi.org/10.48550/arXiv.2407.21783),[Document](https://dx.doi.org/10.48550/ARXIV.2407.21783),2407\.21783Cited by:[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p2.1)\.
- W\. Wang, F\. Wei, L\. Dong, H\. Bao, N\. Yang, and M\. Zhou \(2020\)MINILM: deep self\-attention distillation for task\-agnostic compression of pre\-trained transformers\.InProceedings of the 34th International Conference on Neural Information Processing Systems,NIPS ’20,Red Hook, NY, USA\.External Links:ISBN 9781713829546Cited by:[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p2.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, b\. ichter, F\. Xia, E\. Chi, Q\. V\. Le, and D\. Zhou \(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Vol\.35,pp\. 24824–24837\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf)Cited by:[§5\.7](https://arxiv.org/html/2606.18406#S5.SS7.SSS0.Px4.p1.2)\.
- D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu \(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=pZiyCaVuti)Cited by:[§4\.1](https://arxiv.org/html/2606.18406#S4.SS1.p1.1)\.
- W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. Zhang \(2026\)A\-mem: agentic memory for LLM agents\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=FiM0M8gcct)Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Yang, B\. Yang, B\. Hui, B\. Zheng, B\. Yu, C\. Zhou, C\. Li, C\. Li, D\. Liu, F\. Huang, G\. Dong, H\. Wei, H\. Lin, J\. Tang, J\. Wang, J\. Yang, J\. Tu, J\. Zhang, J\. Ma, J\. Xu, J\. Zhou, J\. Bai, J\. He, J\. Lin, K\. Dang, K\. Lu, K\. Chen, K\. Yang, M\. Li, M\. Xue, N\. Ni, P\. Zhang, P\. Wang, R\. Peng, R\. Men, R\. Gao, R\. Lin, S\. Wang, S\. Bai, S\. Tan, T\. Zhu, T\. Li, T\. Liu, W\. Ge, X\. Deng, X\. Zhou, X\. Ren, X\. Zhang, X\. Wei, X\. Ren, Y\. Fan, Y\. Yao, Y\. Zhang, Y\. Wan, Y\. Chu, Z\. Cui, Z\. Zhang, and Z\. Fan \(2024\)Qwen2 technical report\.ArXivabs/2407\.10671\.External Links:[Link](https://api.semanticscholar.org/CorpusID:271212307)Cited by:[§3\.4](https://arxiv.org/html/2606.18406#S3.SS4.p1.3),[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p2.1)\.
- S\. Zhan, H\. Lin, H\. Tan, X\. Cai, H\. Zheng, X\. Su, Z\. Shan, R\. Liu, and H\. Kim \(2025\)LexSemBridge: fine\-grained dense representation enhancement through token\-aware embedding augmentation\.arXiv preprint arXiv:2508\.17858\.Cited by:[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px3.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract-Datasets%5C_and%5C_Benchmarks.html)Cited by:[§4\.1](https://arxiv.org/html/2606.18406#S4.SS1.p1.1),[§5\.3](https://arxiv.org/html/2606.18406#S5.SS3.p2.2)\.
- W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang \(2024\)MemoryBank: enhancing large language models with long\-term memory\.InThirty\-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty\-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20\-27, 2024, Vancouver, Canada,M\. J\. Wooldridge, J\. G\. Dy, and S\. Natarajan \(Eds\.\),pp\. 19724–19731\.External Links:[Link](https://doi.org/10.1609/aaai.v38i17.29946),[Document](https://dx.doi.org/10.1609/AAAI.V38I17.29946)Cited by:[§1](https://arxiv.org/html/2606.18406#S1.p2.1),[§2](https://arxiv.org/html/2606.18406#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2606.18406#S4.SS2.p1.1)\.

## Appendix AAppendix: Algorithm Pseudocode

This section provides the complete pseudocode for CoreMem’s two core algorithmic components\. Algorithm[1](https://arxiv.org/html/2606.18406#alg1)details the Riemannian retrieval procedure with Woodbury acceleration, which reduces the complexity from𝒪​\(d3\)\\mathcal\{O\}\(d^\{3\}\)to𝒪​\(N​d​r\)\\mathcal\{O\}\(Ndr\)\. Algorithm[2](https://arxiv.org/html/2606.18406#alg2)describes the hierarchical Fisher\-guided discrete token distillation pipeline\.

Algorithm 1Riemannian Retrieval with Woodbury Acceleration1:Memory embeddings

ℋ=\{𝐡i\}i=1N∈ℝd\\mathcal\{H\}=\\\{\\mathbf\{h\}\_\{i\}\\\}\_\{i=1\}^\{N\}\\in\\mathbb\{R\}^\{d\}, query

𝐪∈ℝd\\mathbf\{q\}\\in\\mathbb\{R\}^\{d\}, rank

rr, hybrid weight

α\\alpha
2:Top\-

kkmemory indices

3:L2\-normalize all

𝐡i\\mathbf\{h\}\_\{i\}and

𝐪\\mathbf\{q\}to unit sphere

4:Compute per\-dimension variance:

𝝈2=Var​\(\{𝐡i\}\)\\boldsymbol\{\\sigma\}^\{2\}=\\text\{Var\}\(\\\{\\mathbf\{h\}\_\{i\}\\\}\)
5:Set ridge:

λ=10⋅mean​\(𝝈2\)\\lambda=10\\cdot\\text\{mean\}\(\\boldsymbol\{\\sigma\}^\{2\}\)
6:Form diagonal:

𝐃=diag​\(𝝈2\+λ\)\\mathbf\{D\}=\\text\{diag\}\(\\boldsymbol\{\\sigma\}^\{2\}\+\\lambda\)
7:Center memories:

𝐇cen=\{𝐡i−𝐡¯\}\\mathbf\{H\}\_\{\\text\{cen\}\}=\\\{\\mathbf\{h\}\_\{i\}\-\\bar\{\\mathbf\{h\}\}\\\}
8:SVD:

𝐇cen=𝐔𝐒𝐕⊤\\mathbf\{H\}\_\{\\text\{cen\}\}=\\mathbf\{U\}\\mathbf\{S\}\\mathbf\{V\}^\{\\top\}
9:Select rank

rrs\.t\.

∑i=1rsi2/∑i=1dsi2≥0\.95\\sum\_\{i=1\}^\{r\}s\_\{i\}^\{2\}/\\sum\_\{i=1\}^\{d\}s\_\{i\}^\{2\}\\geq 0\.95
10:

𝐔r←𝐔\[:,:r\]\\mathbf\{U\}\_\{r\}\\leftarrow\\mathbf\{U\}\[:,:r\],

𝐌r←diag​\(si2/N\)\+λ​𝐈\\mathbf\{M\}\_\{r\}\\leftarrow\\text\{diag\}\(s\_\{i\}^\{2\}/N\)\+\\lambda\\mathbf\{I\}
11:Precompute

𝐃−1\\mathbf\{D\}^\{\-1\},

𝐌r−1\\mathbf\{M\}\_\{r\}^\{\-1\},

𝐏=𝐃−1​𝐔r​𝐌r−1\\mathbf\{P\}=\\mathbf\{D\}^\{\-1\}\\mathbf\{U\}\_\{r\}\\mathbf\{M\}\_\{r\}^\{\-1\}
12:Center query:

𝐪c=𝐪−𝐡¯\\mathbf\{q\}\_\{c\}=\\mathbf\{q\}\-\\bar\{\\mathbf\{h\}\}
13:for

i=1i=1to

NNdo

14:

𝐡c=𝐡i−𝐡¯\\mathbf\{h\}\_\{c\}=\\mathbf\{h\}\_\{i\}\-\\bar\{\\mathbf\{h\}\}
15:

riei=𝐪c⊤​𝐃−1​𝐡c−\(𝐪c⊤​𝐏\)​\(𝐡c⊤​𝐏\)⊤\\text\{rie\}\_\{i\}=\\mathbf\{q\}\_\{c\}^\{\\top\}\\mathbf\{D\}^\{\-1\}\\mathbf\{h\}\_\{c\}\-\(\\mathbf\{q\}\_\{c\}^\{\\top\}\\mathbf\{P\}\)\(\\mathbf\{h\}\_\{c\}^\{\\top\}\\mathbf\{P\}\)^\{\\top\}
16:

cosi=𝐪⊤​𝐡i\\text\{cos\}\_\{i\}=\\mathbf\{q\}^\{\\top\}\\mathbf\{h\}\_\{i\}
17:

scorei=α⋅norm​\(cosi\)\+\(1−α\)⋅norm​\(riei\)\\text\{score\}\_\{i\}=\\alpha\\cdot\\text\{norm\}\(\\text\{cos\}\_\{i\}\)\+\(1\-\\alpha\)\\cdot\\text\{norm\}\(\\text\{rie\}\_\{i\}\)
18:endfor

19:return

top\-​k​argsort​\(−score\)\\text\{top\-\}k\\text\{ argsort\}\(\-\\text\{score\}\)

Algorithm 2Fisher\-Guided Discrete Token Distillation \(FDTD\)1:Context tokens

X=\(x1,…,xN\)X=\(x\_\{1\},\\dots,x\_\{N\}\), distiller

pϕp\_\{\\phi\}, budget

BB
2:Compressed token mask

𝐦∈\{0,1\}N\\mathbf\{m\}\\in\\\{0,1\\\}^\{N\}
3:Embed:

𝐇=pϕ\.embed​\(X\)∈ℝN×d\\mathbf\{H\}=p\_\{\\phi\}\.\\text\{embed\}\(X\)\\in\\mathbb\{R\}^\{N\\times d\}
4:Forward:

logits=pϕ​\(𝐇\)\\text\{logits\}=p\_\{\\phi\}\(\\mathbf\{H\}\)
5:Compute loss:

ℒ=CrossEntropy​\(logits,X\)\\mathcal\{L\}=\\text\{CrossEntropy\}\(\\text\{logits\},X\)
6:Backprop to embeddings:

∇𝐇ℒ\\nabla\_\{\\mathbf\{H\}\}\\mathcal\{L\}\(freeze all parameters\)

7:for

i=1i=1to

NNdo

8:

S​\(xi\)=‖∇𝐡iℒ⊙𝐡i‖2S\(x\_\{i\}\)=\\\|\\nabla\_\{\\mathbf\{h\}\_\{i\}\}\\mathcal\{L\}\\odot\\mathbf\{h\}\_\{i\}\\\|\_\{2\}
9:endfor

10:

Ssmooth=AvgPool1​D​\(S,kernel=5\)S\_\{\\text\{smooth\}\}=\\text\{AvgPool\}\_\{1D\}\(S,\\text\{kernel\}=5\)
11:Apply structural multipliers \(§[3\.3](https://arxiv.org/html/2606.18406#S3.SS3)\) to get

SfinalS\_\{\\text\{final\}\}
12:Sort tokens by

SfinalS\_\{\\text\{final\}\}ascending

13:

𝐦i=1\\mathbf\{m\}\_\{i\}=1for top\-

BBtokens,

0otherwise

14:Fill gaps

≤3\\leq 3auto\-fill

15:return

𝐦\\mathbf\{m\}

## Appendix BAppendix: Hyperparameter Settings

Table 6:Hyperparameter settings\.The ridge parameterλ=10⋅mean​\(𝝈2\)\\lambda=10\\cdot\\text\{mean\}\(\\boldsymbol\{\\sigma\}^\{2\}\)is chosen to bound the condition number of𝚺−1\\boldsymbol\{\\Sigma\}^\{\-1\}to approximately2×2\\times, preventing numerical instability while preserving anisotropy\. The hybrid weightα=0\.5\\alpha=0\.5balances the complementary strengths of cosine and Riemannian metrics \(see Figure[3](https://arxiv.org/html/2606.18406#S5.F3)\)\. Structural boosts are applied via multiplicative masks rather than hard thresholds to maintain gradient\-friendly soft protection\.

## Appendix CAppendix: Additional Experimental Tables

Table 7:Compression stress test: Judge accuracy at three budgets \(gte\-Qwen2, Temporal QA\)\.Table[7](https://arxiv.org/html/2606.18406#A3.T7)reveals a sharp phase transition: atb=1000b=1000, learned compressors remain competitive, but atb=500b=500and below, all neural compressors degrade below the simple Truncation baseline\. This suggests that extreme token starvation \(≤\\leq500 tokens\) overwhelms the representational capacity of 0\.6B–1B parameter distillers\.

## Appendix DAppendix: Qualitative Case Studies

Tables[8](https://arxiv.org/html/2606.18406#A4.T8)and[9](https://arxiv.org/html/2606.18406#A4.T9)illustrate representative success cases selected from the LOCOMO benchmark where CoreMem receives Judge=1 while all compared baselines receive Judge=0\. Table[8](https://arxiv.org/html/2606.18406#A4.T8)shows end\-to\-end retrieval cases \(1,542 QA, MiniLM\) where CoreMem\-Fusion retrieves multi\-fact aggregations \(e\.g\., “cooking classes \+ Evan”\) and resolves temporal references \(e\.g\., “August 2022”\) that cosine retrieval misses\. Table[9](https://arxiv.org/html/2606.18406#A4.T9)shows compression stress cases \(Temporal, 96 QA, gte\-Qwen2\) where FDTD preserves fine\-grained entities \(e\.g\., “Under Armour”, “filmmaker”\) that baseline compressors drop or hallucinate\.

Table 8:End\-to\-end retrieval case studies\. CoreMem\-Fusion answers correctly \(Judge=1\) where all baselines fail \(Judge=0\)\.#### Analysis of End\-to\-End Cases\.

The six success cases reveal two systematic patterns in which CoreMem\-Fusion outperforms pure cosine retrieval\. First,multi\-fact aggregation\(E1, E3, E4\): questions such as “Maria’s faith?” and “Sam’s challenges?” require the simultaneous presence of two semantically distant facts \(e\.g\., “cross necklace” \+ “church”, or “cooking classes” \+ “Evan”\)\. Cosine retrieval tends to surface only the dominant hub fact \(“church” or “gym”\) because the embedding of the secondary fact lies in a low\-variance, geometrically peripheral region of the manifold\. The anisotropic Riemannian metric explicitly stretches these low\-variance dimensions, surfacing tail memories that cosine similarity suppresses\. The Fusion interpolator \(α=0\.5\\alpha=0\.5\) then prevents these tails from being drowned out by bridge\-fact hubs\.

Second,temporal and fine\-grained reference resolution\(E2, E5, E6\): questions like “Calvin & Frank Ocean?” \(GT: August 2022\) or “Evan’s hobby?” \(GT: watercolor\) demand precise entity or time\-span matching rather than broad thematic similarity\. Cosine retrieval returns semantically proximate but factually incorrect substitutes \(“previous year” instead of “last year in August”; “painting” instead of “watercolor painting”\) because it measures global vector alignment rather than local manifold curvature\. The Mahalanobis component of the Riemannian metric penalizes such hub substitutions by re\-weighting dimensions according to per\-coordinate variance, thereby preserving fine\-grained distinctions\.

Table 9:Compression stress case studies \(budget 1000\)\. CoreMem preserves entities and relations that baselines drop or hallucinate\.
#### Analysis of Compression Cases\.

The five compression cases demonstrate that FDTD’s structural protection masks are not merely heuristic safeguards but operationally critical for factual QA under severe token starvation\. Baseline compressors—whether rule\-based \(Truncation\) or learned \(LLMLingua2, SelectiveContext\)—suffer from two failure modes:entity dropping\(C1, C3, C4\) androle hallucination\(C2, C5\)\. In C1, “Under Armour” is a low\-frequency proper noun that standard compressors discard because its token\-level self\-information is low; FDTD’s content\-word boost \(1\.3×1\.3\\timesfor capitalized proper nouns\) explicitly retains it\. In C2 and C5, baseline compressors collapse the fine\-grained role distinction “filmmaker” vs\. “screenwriter” and the nickname “Jo” vs\. the full name “Joanna”, because these distinctions depend on syntactic and contextual connectors \(e\.g\., apposition markers\) that attention\-sparse Fisher scores alone would undervalue\. The keyword\-boost \(1\.2×1\.2\\times\) and punctuation\-protection \(1\.31\.3–1\.5×1\.5\\times\) layers compensate for this blind spot, preserving the grammatical scaffolding that anchors entity resolution\.

#### Failure Case Analysis\.

For balance, Table[10](https://arxiv.org/html/2606.18406#A4.T10)reports a case where CoreMem\-Fusion fails while NaiveRAG succeeds \(GPT\-4o\-mini Judge=0 vs\. 1\)\. The failure is attributable to retrieval rather than to compression or generation: both systems share the same retrieval\-to\-generation pipeline, but CoreMem\-Fusion does not recall the memory containing the correct answer\.

Question / GTNaiveRAGCoreMem\-FusionWhich hobby did Dave pick up in October 2023? GT: photographyDave picked up photography in October 2023\.Dave opened his own car maintenance shop in October 2023\.Table 10:A failure case where CoreMem\-Fusion receives Judge=0 while NaiveRAG receives Judge=1\.NaiveRAG retrieves the crucial evidence at rank 34:Dave: “Hey Calvin, long time no talk\! A lot has happened\. I’vetaken up photographyand it’s been great…”Although this memory is ranked relatively low, cosine similarity still recalls it because “taken up” is semantically close to the query phrase “pick up”\.

CoreMem\-Fusion, in contrast, does not recallanysegment containing “photography,” “taken up,” or related phrases in its top\-50 retrieval\. Instead, it retrieves a dense cluster of Dave’s car\-related memories:“I’m passionate about fixing up things\. It’s more than just a hobby,” “I opened my car shop last week,” “I finally opened my own car maintenance shop,”and similar turns\. Because the word “hobby” co\-occurs repeatedly with cars/shop/maintenance in Dave’s dialogue, the Riemannian metric is drawn toward this strong semantic hub\. The isolated “taken up photography” memory lies in a peripheral region of the manifold and is overwhelmed by the car hub\.

This case illustrates an important boundary of Riemannian retrieval: while it effectively surfaces peripheral tail memories in many settings, it can still be dominated by a strong thematic hub when that hub shares vocabulary with the query \(here, “hobby”/“passion”\)\. In such situations, cosine retrieval’s lexical\-semantic matching can be more robust\. This observation reinforces the paper’s broader claim that Riemannian and cosine metrics are complementary rather than universally dominant\.

## Appendix EAppendix: LLM\-as\-a\-Judge Prompt Design

Our LLM\-as\-a\-Judge evaluation uses GPT\-4o\-mini at temperature 0\.0 with a binary strict protocol \(max\_tokens=2\)\. The response is stripped, lowercased, and scored as 1 if it starts with “yes”, 0 otherwise\.

Answer Generation Prompt Based on the following conversation context, answer the question concisely\. Context: compressed\_context Question: question Answer:

Figure 4:Answer generation prompt template used for all systems\.Binary Strict Judge Prompt I will give you a question, a correct answer, and a response from a model\. Please answer yes if the response contains the correct answer\. Otherwise, answer no\. If the response is equivalent to the correct answer or contains all the intermediate steps, answer yes\. Question: question Correct Answer: ground\_truth Model Response: predicted\_answer Is the model response correct? Answer yes or no only\.

Figure 5:Binary strict judge prompt template \(GPT\-4o\-mini, temp=0\.0, max\_tokens=2\)\.The binary strict protocol \(temperature 0\.0,max\_tokens=2\) is designed to eliminate stylistic verbosity and force a deterministic verdict\. We strip and lowercase the response, scoring as correct only if the model output starts with “yes”\. This protocol minimizes judge stochasticity at the cost of ignoring partially correct answers\.

## Appendix FAppendix: Theoretical Derivations

#### Woodbury Identity\.

Given𝚺=𝐔r​𝐌r​𝐔r⊤\+𝐃\\boldsymbol\{\\Sigma\}=\\mathbf\{U\}\_\{r\}\\mathbf\{M\}\_\{r\}\\mathbf\{U\}\_\{r\}^\{\\top\}\+\\mathbf\{D\}where𝐃=diag​\(𝝈2\+λ\)\\mathbf\{D\}=\\text\{diag\}\(\\boldsymbol\{\\sigma\}^\{2\}\+\\lambda\):

\(𝐀\+𝐔𝐂𝐕\)−1=𝐀−1−𝐀−1​𝐔​\(𝐂−1\+𝐕𝐀−1​𝐔\)−1​𝐕𝐀−1\(\\mathbf\{A\}\+\\mathbf\{U\}\\mathbf\{C\}\\mathbf\{V\}\)^\{\-1\}=\\mathbf\{A\}^\{\-1\}\\\\ \-\\mathbf\{A\}^\{\-1\}\\mathbf\{U\}\(\\mathbf\{C\}^\{\-1\}\+\\mathbf\{V\}\\mathbf\{A\}^\{\-1\}\\mathbf\{U\}\)^\{\-1\}\\mathbf\{V\}\\mathbf\{A\}^\{\-1\}\(5\)Setting𝐀=𝐃\\mathbf\{A\}=\\mathbf\{D\},𝐔=𝐔r\\mathbf\{U\}=\\mathbf\{U\}\_\{r\},𝐂=𝐌r\\mathbf\{C\}=\\mathbf\{M\}\_\{r\},𝐕=𝐔r⊤\\mathbf\{V\}=\\mathbf\{U\}\_\{r\}^\{\\top\}yields Eq\.[1](https://arxiv.org/html/2606.18406#S3.E1)\.

#### Fisher Sensitivity Derivation\.

From the code, token sensitivity is computed as theL2L\_\{2\}norm of the Hadamard product:

S​\(xi\)=‖∇𝐡iℒ⊙𝐡i‖2,S\(x\_\{i\}\)=\\\|\\nabla\_\{\\mathbf\{h\}\_\{i\}\}\\mathcal\{L\}\\odot\\mathbf\{h\}\_\{i\}\\\|\_\{2\},\(6\)where∇𝐡iℒ\\nabla\_\{\\mathbf\{h\}\_\{i\}\}\\mathcal\{L\}is the gradient of the language modeling loss with respect to tokenii’s embedding, obtained via backpropagation throughinputs\_embedswith all model parameters frozen\.

#### Structural Protection Multipliers\.

The final sensitivity score combines raw Fisher scores with structural priors:

Sfinal\(xi\)=Ssmooth\(xi\)×max\(penalty​\(xi\),content\_boost\(xi\)\),\\begin\{split\}S\_\{\\text\{final\}\}\(x\_\{i\}\)=S\_\{\\text\{smooth\}\}\(x\_\{i\}\)\\times\\max\\big\(&\\text\{penalty\}\(x\_\{i\}\),\\\\ &\\text\{content\\\_boost\}\(x\_\{i\}\)\\big\),\\end\{split\}\(7\)whereSsmoothS\_\{\\text\{smooth\}\}is the 1D average\-pooled score \(kernel=5\),penaltyencodes punctuation protection \(1\.3–1\.5×\\times\) and keyword boosts \(1\.2×\\times\), andcontent\_boostupweights digits \(1\.4×\\times\) and capitalized words \(1\.3×\\times\)\.

#### Error Bound\.

Because FDTD drops tokens in ascending order ofS​\(xi\)S\(x\_\{i\}\), the total Fisher trace of dropped tokens is minimized\. Formally, letpθ​\(y∣X\)p\_\{\\theta\}\(y\\mid X\)denote the cloud LLM, and letℒ​\(X\)=−log⁡pθ​\(y∗∣X\)\\mathcal\{L\}\(X\)=\-\\log p\_\{\\theta\}\(y^\{\*\}\\mid X\)be the loss on the target responsey∗y^\{\*\}when the model consumes the full contextXX\. LetX~\\tilde\{X\}be the compressed context obtained by keeping the top\-BBtokens according toSfinalS\_\{\\text\{final\}\}\. Assuming the loss isCC\-Lipschitz continuous with respect to the input\-token embeddings, we have

\|ℒ​\(X\)−ℒ​\(X~\)\|≤C​∑i∈DroppedS​\(xi\),\\big\|\\mathcal\{L\}\(X\)\-\\mathcal\{L\}\(\\tilde\{X\}\)\\big\|\\;\\leq\\;C\\sum\_\{i\\in\\text\{Dropped\}\}S\(x\_\{i\}\),\(8\)whereS​\(xi\)=‖∇𝐡iℒ⊙𝐡i‖2S\(x\_\{i\}\)=\\\|\\nabla\_\{\\mathbf\{h\}\_\{i\}\}\\mathcal\{L\}\\odot\\mathbf\{h\}\_\{i\}\\\|\_\{2\}is the per\-token Fisher sensitivity andCCis the Lipschitz constant of the cloud model’s loss with respect to its input embeddings\. For autoregressive language models with softmax outputs,CCis bounded under standard smoothness assumptions on the cross\-entropy loss\. Consequently, FDTD minimizes an explicit upper bound on inference degradation by greedily dropping tokens with the smallest Fisher traces\. While this greedy strategy does not guarantee global optimality due to the combinatorial nature of subset selection, it provides a tight and computationally feasible approximation for practical compression budgets\.

## Appendix GAppendix: Per\-Category Performance Breakdown

Table[11](https://arxiv.org/html/2606.18406#A7.T11)shows Judge accuracy by question category\.

Table 11:LOCOMO Judge accuracy by category \(MiniLM\)\. Adv=Adversarial \(2 QA\)\.Table[11](https://arxiv.org/html/2606.18406#A7.T11)confirms that CoreMem\-Fusion’s gains are concentrated in Open\-domain \(\+4\.51 pp over NaiveRAG\) and Temporal \(\+4\.17 pp\) reasoning, where peripheral tail memories are most valuable\. All systems perform identically on Adversarial questions \(2 samples\), confirming the category’s outlier status\.

#### Code Availability\.