Deterministic on every query, 0 LLM calls to retrieve, and it still hits 1.00 recall: we put our memory tool head-to-head with 4 others
Summary
This article presents a head-to-head comparison of a new deterministic memory tool that achieves perfect recall with zero LLM calls during retrieval against four other tools.
Similar Articles
Memory retrieval is broken under the hood.
PrecisionMemBench is an open-source benchmark that tests retrieval precision as a strict unit test, revealing that popular memory frameworks like Mem0, Zep, and Hindsight have very low precision (0.05-0.09) and rely on LLMs to compensate. The article argues for zero-tolerance hard fail on precision for production memory infrastructure.
#1 on memory benchmark LongMemEval with Gemini Flash, not Pro [R]
A novel memory retrieval system inspired by episodic memory theory achieves state-of-the-art 96.4% top-50 accuracy on the LongMemEval benchmark using Gemini Flash, outperforming larger Pro-based baselines by isolating retrieval quality from model capability.
MemTrace: Probing What Final Accuracy Misses in Long-Term Memory
MemTrace is a benchmark that evaluates LLM agent memory at the knowledge point level, probing how facts behave under varying memory age, question type, and evidence conditions. It reveals that pooled accuracy hides distinct failure modes, and that the main bottleneck is evidence use rather than retrieval.
Benchmarking agent memory retrieval on LongMemEval‑S — 98% Recall@5, 100% recall by R@23, local embeddings only (all-MiniLM-L6-v2), no LLM, no API key
The author shares benchmark results for memweave, a Python library for agent memory, achieving 98% Recall@5 on LongMemEval-S using only local embeddings without LLM calls. The post details the methodology and compares performance against mempalace, highlighting stable retrieval across different question types.
YourMemory
YourMemory is an MCP-based memory tool that uses self-pruning to reduce token waste by up to 84%, improving efficiency in AI context management.