BEAM benchmarks
Summary
Midas achieves 0.56 recall@k on BEAM 100K and 0.51 on BEAM 500K with zero LLM calls and zero cost, demonstrating efficient long-term memory for agents.
Similar Articles
Midas: 100% local agent memory — no LLM at ingest, $0, nothing leaves the box (MCP + Python SDK)
Midas is a local agent memory tool that uses embeddings and ranking instead of LLM calls for ingest, achieving zero cost, offline operation, and high recall with auditable source turns.
Benchmarking agent memory retrieval on LongMemEval‑S — 98% Recall@5, 100% recall by R@23, local embeddings only (all-MiniLM-L6-v2), no LLM, no API key
The author shares benchmark results for memweave, a Python library for agent memory, achieving 98% Recall@5 on LongMemEval-S using only local embeddings without LLM calls. The post details the methodology and compares performance against mempalace, highlighting stable retrieval across different question types.
IFCMemoryBench: Evaluating Long-Term Memory of LLM-Based Agents in BIM Information Retrieval
This paper introduces IFCMemoryBench, a human-validated benchmark for evaluating long-term memory in LLM-based agents for BIM information retrieval. It shows that current memory systems achieve only 32.4% answer accuracy, revealing a domain-transfer gap in agent memory.
Agent memory that retrieves 96.4% on LongMemEval without calling a model
Agentlas is a free, open-source agent memory system that uses a local SQLite database combined with lightweight embedding and keyword search to achieve 96.4% on LongMemEval without calling an LLM, outperforming alternatives like mem0 and supermemory.
@hyunji_amy_lee: LLM agents & memory systems operate in continuously updated environments (Git repos, evolving docs). They must process …
MINTEval is a new benchmark for evaluating LLM agents and memory systems in continuously updated environments with frequent context changes. It shows that current systems perform poorly, with an average accuracy of 27.9% across representative systems.