Tag
ReCache is a framework for efficient KV cache reuse and compression in tool-augmented LLM agents, achieving significant speedup and memory reduction while maintaining performance.
This paper investigates how epistemic stance (qualifiers, attributions) survives memory compression in AI agent memory systems. It finds that making the stance explicit as a labelled field improves retention significantly, while merely lengthening the text does not.
This paper shows that a language model with a lossy memory that retains a wrong conclusion but drops the evidence produces confident incorrect answers, whereas an empty memory leads to abstention. The authors propose a source-first compression policy that preserves recomputable sources instead of conclusions to maintain correctability, and demonstrate the mechanism across multiple models and dialogue systems.
Introduces KV-Compression Aware Training (KV-CAT), a method that encourages transformers to learn compressible key-value caches during training, improving memory efficiency for long-context tasks without sacrificing performance.
MemRefine is an LLM-guided framework for compressing long-term agent memory under fixed storage budgets, using similarity for candidate pairing and an LLM judge for factual deletion/merge decisions, outperforming rule-based baselines on benchmarks.
FlashMemory-DeepSeek-V4 proposes a novel inference paradigm called Lookahead Sparse Attention (LSA), which uses a neural memory indexer to actively predict future context needs, compressing physical KV cache usage to 13.5% of full context baseline while improving average accuracy by 0.6%. This method adopts a decoupled training strategy that allows independent training of the indexer without loading the base model, significantly reducing training cost.
OmniMem introduces a modality-aware memory allocation and perturbation-aware selection strategy for streaming audio-visual LLMs, achieving 2-4% absolute accuracy gains over compression baselines on long-video benchmarks.
AURA-Mem proposes a constant-size memory for robot policies using a learned gate that writes only when current observations would change the next action. It matches baseline accuracy with significantly fewer writes and constant VRAM, addressing the memory bottleneck for long-horizon robot tasks.
VideoMLA replaces per-head KV caches in video diffusion models with a shared low-rank latent and decoupled 3D-RoPE positional keys, reducing per-token KV memory by 92.7% and improving throughput by 1.23x on a B200 while maintaining quality on VBench benchmarks.
This paper proposes a unified framework called Efficiency Frontier, which treats large model context management as a deployment optimization problem, jointly modeling task performance, token overhead, and preprocessing reuse. On 5,000 HotpotQA instances, deployment optimization saves 25% of token usage, while memory compression is more than half the cost of full context in high-precision scenarios.
Introduces The Efficiency Frontier, a unified framework for optimizing cost and performance in LLM context management, achieving approximately 25% reduction in effective token usage at comparable performance on HotpotQA.
WorldKV is a training-free framework that retrieves and compresses key-value cache chunks to maintain long-term consistency in video diffusion world generation, achieving higher throughput while matching full-memory fidelity.
A user shares a fix for performance bottlenecks when running AI models on AMD GPUs in Windows 11 by disabling memory compression via the command 'Disable-mmagent -mc'.
claude-mem is an open-source tool that provides persistent memory compression for Claude Code, allowing it to remember context across sessions.