Tag
This paper shows that a language model with a lossy memory that retains a wrong conclusion but drops the evidence produces confident incorrect answers, whereas an empty memory leads to abstention. The authors propose a source-first compression policy that preserves recomputable sources instead of conclusions to maintain correctability, and demonstrate the mechanism across multiple models and dialogue systems.
Introduces KV-Compression Aware Training (KV-CAT), a method that encourages transformers to learn compressible key-value caches during training, improving memory efficiency for long-context tasks without sacrificing performance.
MemRefine is an LLM-guided framework for compressing long-term agent memory under fixed storage budgets, using similarity for candidate pairing and an LLM judge for factual deletion/merge decisions, outperforming rule-based baselines on benchmarks.
FlashMemory-DeepSeek-V4 proposes a novel inference paradigm called Lookahead Sparse Attention (LSA), which uses a neural memory indexer to actively predict future context needs, compressing physical KV cache usage to 13.5% of full context baseline while improving average accuracy by 0.6%. This method adopts a decoupled training strategy that allows independent training of the indexer without loading the base model, significantly reducing training cost.
OmniMem introduces a modality-aware memory allocation and perturbation-aware selection strategy for streaming audio-visual LLMs, achieving 2-4% absolute accuracy gains over compression baselines on long-video benchmarks.
AURA-Mem proposes a constant-size memory for robot policies using a learned gate that writes only when current observations would change the next action. It matches baseline accuracy with significantly fewer writes and constant VRAM, addressing the memory bottleneck for long-horizon robot tasks.
VideoMLA replaces per-head KV caches in video diffusion models with a shared low-rank latent and decoupled 3D-RoPE positional keys, reducing per-token KV memory by 92.7% and improving throughput by 1.23x on a B200 while maintaining quality on VBench benchmarks.
This paper proposes a unified framework called Efficiency Frontier, which treats large model context management as a deployment optimization problem, jointly modeling task performance, token overhead, and preprocessing reuse. On 5,000 HotpotQA instances, deployment optimization saves 25% of token usage, while memory compression is more than half the cost of full context in high-precision scenarios.
Introduces The Efficiency Frontier, a unified framework for optimizing cost and performance in LLM context management, achieving approximately 25% reduction in effective token usage at comparable performance on HotpotQA.
WorldKV is a training-free framework that retrieves and compresses key-value cache chunks to maintain long-term consistency in video diffusion world generation, achieving higher throughput while matching full-memory fidelity.
A user shares a fix for performance bottlenecks when running AI models on AMD GPUs in Windows 11 by disabling memory compression via the command 'Disable-mmagent -mc'.
claude-mem is an open-source tool that provides persistent memory compression for Claude Code, allowing it to remember context across sessions.