Tag
The author critiques the reliability of memory benchmarks for LLMs, citing issues like incorrect golden answers in LoCoMo and inconsistent scoring metrics yielding large score gaps. They question whether practitioners trust these numbers and discuss recent improvements like LongMemEval's knowledge update questions.