Speculate with Memory: Lossless Acceleration for LLM Agents
Summary
This paper introduces memory-augmented speculative execution for LLM agents, using three online memory systems to improve prediction accuracy by 19-39% on action prediction and up to 2.5x on observation prediction, all while being lossless with zero added wall-clock cost.
View Cached Full Text
Cached at: 07/15/26, 04:18 AM
# Speculate with Memory: Lossless Acceleration for LLM Agents Source: [https://arxiv.org/abs/2607.12236](https://arxiv.org/abs/2607.12236) [View PDF](https://arxiv.org/pdf/2607.12236) > Abstract:Speculative execution accelerates LLM agents by using a smaller, cheaper model to predict and pre\-launch the next step while the environment is idle\. However, existing speculators are stateless and discard all information between tasks, preventing prediction quality from improving with experience\. We equip the speculator with three online memory systems that learn from past agent trajectories: a contrastive transition table tracking action\-sequence statistics, an episodic memory retrieving contextually similar segments, and a confusion tracker suppressing recurring errors\. We evaluate this approach on six benchmarks spanning three speculation types: action prediction, observation prediction, and chained prediction\. Memory\-augmented speculation yields a 19\-\-39\\% relative accuracy improvement on action prediction and up to a $2\.5\\times$ increase on observation prediction tasks with repetitive action spaces\. These gains grow continuously as memory accumulates and generalize across speculator models of varying cost\. All speculation is lossless because it runs during idle time at zero added wall\-clock cost, and the actor's trajectory is identical to non\-speculative execution\. ## Submission history From: Yu Li \[[view email](https://arxiv.org/show-email/b7ad814a/2607.12236)\] **\[v1\]**Tue, 14 Jul 2026 00:36:31 UTC \(423 KB\)
Similar Articles
SimpleMem: Efficient Lifelong Memory for LLM Agents
Introduces SimpleMem, an efficient memory framework for LLM agents that uses semantic lossless compression to improve accuracy and reduce token consumption, achieving 26.4% F1 improvement and up to 30x reduction in inference-time token usage.
Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History
This paper introduces Engram, an open-source bi-temporal memory engine for LLM agents that retrieves a compact context slice (∼9.6k tokens) to outperform the full-history baseline (79k tokens) by 10.4 accuracy points on LongMemEval, using a hybrid read path fusing dense, lexical, graph, and temporal signals.
ActiveMem: Distributed Active Memory for Long-Horizon LLM Reasoning
ActiveMem introduces a distributed active memory system that decouples agent memory from the core LLM reasoning process, achieving state-of-the-art accuracy on long-horizon tasks with significantly reduced overhead.
Selective Memory Retention for Long-Horizon LLM Agents
This paper presents TraceRetain, a lightweight framework for bounded external memory in frozen LLM agents, demonstrating that selective retention differentiates from cache heuristics primarily when memory streams contain noise, offering task-success and efficiency benefits.
RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents
RecMem is a recurrence-based memory consolidation method for long-running LLM agents that reduces token consumption by up to 87% while improving accuracy, by only invoking LLMs when semantically similar interactions recur.