Δ-Mem: Efficient Online Memory for Large Language Models
Summary
Proposes delta-Mem, a lightweight online memory mechanism that uses a compact state matrix updated by delta-rule learning to improve long-context performance of frozen LLMs without full fine-tuning or context extension.
View Cached Full Text
Cached at: 05/16/26, 12:39 PM
# $δ$-mem: Efficient Online Memory for Large Language Models Source: [https://arxiv.org/abs/2605.12357](https://arxiv.org/abs/2605.12357) [View PDF](https://arxiv.org/pdf/2605.12357) > Abstract:Large language models increasingly need to accumulate and reuse historical information in long\-term assistants and agent systems\. Simply expanding the context window is costly and often fails to ensure effective context utilization\. We propose $\\delta$\-mem, a lightweight memory mechanism that augments a frozen full\-attention backbone with a compact online state of associative memory\. $\\delta$\-mem compresses past information into a fixed\-size state matrix updated by delta\-rule learning, and uses its readout to generate low\-rank corrections to the backbone's attention computation during generation\. With only an $8\\times8$ online memory state, $\\delta$\-mem improves the average score to $1\.10\\times$ that of the frozen backbone and $1\.15\\times$ that of the strongest non\-$\\delta$\-mem memory baseline\. It achieves larger gains on memory\-heavy benchmarks, reaching $1\.31\\times$ on MemoryAgentBench and $1\.20\\times$ on LoCoMo, while largely preserving general capabilities\. These results show that effective memory can be realized through a compact online state directly coupled with attention computation, without full fine\-tuning, backbone replacement, or explicit context extension\. ## Submission history From: Jingdi Lei \[[view email](https://arxiv.org/show-email/3b79a046/2605.12357)\] **\[v1\]**Tue, 12 May 2026 16:31:44 UTC \(609 KB\)
Similar Articles
δ-mem: Efficient Online Memory for Large Language Models
The paper introduces δ-mem, a lightweight memory mechanism that enhances large language models by augmenting a frozen attention backbone with a compact associative memory state. It demonstrates improved performance on memory-heavy benchmarks with minimal computational overhead.
@dair_ai: // δ-mem: Efficient Online Memory for LLMs // One of the more elegant memory mechanisms I've seen this month. Most long…
The paper introduces δ-mem, a lightweight online memory mechanism that augments frozen LLMs with a compact associative memory state updated by delta-rule learning, achieving significant improvements on memory-heavy benchmarks without fine-tuning or context extension.
MemDefrag: Latent Memory Defragmentation for Large Language Models
Proposes MemDefrag, a training-free framework that uses a middle-layer tracing signal to defragment latent memory in LLMs, achieving significantly better knowledge retention than existing methods like MemoryLLM and M+.
Understanding Is Done Early: A Depth Division of Labor in Large Language Models and Its Use for Unbounded-Context Memory
This paper introduces CoMem, a method that exploits the depth-wise division of labor in LLMs to cache intermediate residual tensors and recompute only upper layers for retrieval, enabling bounded read compute and memory independent of stored-context length. Evaluated on Qwen3-8B, CoMem achieves strong long-context performance with significant memory savings and prefill speedups.
SimpleMem: Efficient Lifelong Memory for LLM Agents
Introduces SimpleMem, an efficient memory framework for LLM agents that uses semantic lossless compression to improve accuracy and reduce token consumption, achieving 26.4% F1 improvement and up to 30x reduction in inference-time token usage.