Tag
This paper introduces ECCBench, a benchmark and evaluation protocol for assessing memory in Vision-Language Models beyond accuracy, focusing on efficiency, compression, and calibration axes.
The paper introduces MemUse, a benchmark that shows direct QA accuracy for conversational memory does not predict user satisfaction, whereas natural integration of prior context does, revealing a large gap between recall and conversational use.
LongMINT is a benchmark for evaluating memory under multi-target interference in long-horizon agent systems.
The MEME benchmark evaluates AI memory systems across multiple entities and evolving conditions, revealing significant challenges in dependency reasoning that persist even with advanced retrieval techniques.