Speculate with Memory: Lossless Acceleration for LLM Agents
Summary
This paper introduces memory-augmented speculative execution for LLM agents, using three online memory systems to improve prediction accuracy by 19-39% on action prediction and up to 2.5x on observation prediction, all while being lossless with zero added wall-clock cost.
View Cached Full Text
Cached at: 07/15/26, 04:18 AM
# Speculate with Memory: Lossless Acceleration for LLM Agents Source: [https://arxiv.org/abs/2607.12236](https://arxiv.org/abs/2607.12236) [View PDF](https://arxiv.org/pdf/2607.12236) > Abstract:Speculative execution accelerates LLM agents by using a smaller, cheaper model to predict and pre\-launch the next step while the environment is idle\. However, existing speculators are stateless and discard all information between tasks, preventing prediction quality from improving with experience\. We equip the speculator with three online memory systems that learn from past agent trajectories: a contrastive transition table tracking action\-sequence statistics, an episodic memory retrieving contextually similar segments, and a confusion tracker suppressing recurring errors\. We evaluate this approach on six benchmarks spanning three speculation types: action prediction, observation prediction, and chained prediction\. Memory\-augmented speculation yields a 19\-\-39\\% relative accuracy improvement on action prediction and up to a $2\.5\\times$ increase on observation prediction tasks with repetitive action spaces\. These gains grow continuously as memory accumulates and generalize across speculator models of varying cost\. All speculation is lossless because it runs during idle time at zero added wall\-clock cost, and the actor's trajectory is identical to non\-speculative execution\. ## Submission history From: Yu Li \[[view email](https://arxiv.org/show-email/b7ad814a/2607.12236)\] **\[v1\]**Tue, 14 Jul 2026 00:36:31 UTC \(423 KB\)
Similar Articles
More Memory, Worse Agents: Error Reproduction and Anti-Persistence in LLM Agents
This paper shows that accumulating more memory can degrade LLM agent performance through error reproduction, where stored mistakes regenerate themselves via new abstractions. It proposes Anti-Persistence, a method that keeps trajectories but reconstructs task-specific guidance on demand, achieving higher success and lower LLM-call cost across benchmarks.
SimpleMem: Efficient Lifelong Memory for LLM Agents
Introduces SimpleMem, an efficient memory framework for LLM agents that uses semantic lossless compression to improve accuracy and reduce token consumption, achieving 26.4% F1 improvement and up to 30x reduction in inference-time token usage.
HasMem: Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents
The paper proposes HasMem, an adaptive memory compression method for long-term LLM agents that adjusts memory widths while maintaining frozen LLM parameters, demonstrating improved performance on reconstruction and QA benchmarks.
Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History
This paper introduces Engram, an open-source bi-temporal memory engine for LLM agents that retrieves a compact context slice (∼9.6k tokens) to outperform the full-history baseline (79k tokens) by 10.4 accuracy points on LongMemEval, using a hybrid read path fusing dense, lexical, graph, and temporal signals.
ActiveMem: Distributed Active Memory for Long-Horizon LLM Reasoning
ActiveMem introduces a distributed active memory system that decouples agent memory from the core LLM reasoning process, achieving state-of-the-art accuracy on long-horizon tasks with significantly reduced overhead.