Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents
Summary
This paper introduces a taxonomy of conversational memory types and a user-centric evaluation framework to study how different memory roles affect response quality in RAG-based conversational agents.
View Cached Full Text
Cached at: 06/25/26, 05:10 AM
# Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents Source: [https://arxiv.org/abs/2606.25361](https://arxiv.org/abs/2606.25361) [View PDF](https://arxiv.org/pdf/2606.25361) > Abstract:Prior research on memory mechanism in RAG\-based conversational system has emphasized how memory is stored and retrieved\. However, far less is known about how memories with different functional roles influence response quality\. Specifically, how they shape an agent's responses under varying conversational contexts and whether they lead to substantively different response behaviors\. Existing evaluations in conversational system are also largely reference\-based, insufficiently capturing the nuances in responses that may address users' preferences differently\. In this work, we probe the impact of different memory types in shaping agents' responses\. We present a fine\-grained taxonomy of conversational memory, classify retrieved memories into different role types, and design a user\-centric evaluation framework that simulates user perspectives\. Through comparative experiments on long\-term datasets and frontier LLMs, our analysis reveal many differentiated effects of memories: e\.g\., clarifying memory improves responses' factual accuracy and constraint awareness, making them more correct and personalized; irrelevant memory reduces topic relevance and degrades constraint awareness\. Despite the power of frontier LLMs, these findings shed light on how different memory types can be leveraged to produce more personalized responses and inspire further research in this direction\. ## Submission history From: Yuxin Wang \[[view email](https://arxiv.org/show-email/09015405/2606.25361)\] **\[v1\]**Wed, 24 Jun 2026 03:45:45 UTC \(697 KB\)
Similar Articles
AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents
AgentMemBench is a systematic benchmark that evaluates five long-term memory management strategies for conversational AI agents across three datasets, finding that external key-value store retrieval dominates on quality but incurs a larger memory footprint.
Understanding Agent Memory (38 minute read)
This article compares three common shapes of agent memory systems—file-based, structured store, and experience-based—and evaluates their effectiveness through a benchmark using a common agent loop and open-weight model.
Staying In Character: Perspective-Bounded Memory For Book-Based Role-Playing Agents
This paper proposes ReverieMem, a three-layer memory architecture for book-based LLM role-playing agents that prevents factual overreach and stylistic monotony. It also introduces the KBF-QA benchmark and achieves significant improvements in knowledge boundary fidelity and narrative quality.
SaliMory: Orchestrating Cognitive Memory for Conversational Agents
SaliMory is a framework that trains a single language model to manage cognitively-structured memory (user facts, preferences, and working memory) for conversational agents, using hierarchical stage-wise process rewards and reward-decomposed contrastive refinement. It reduces memory-attributed failures by one-third, outperforms state-of-the-art by over 10% in end-to-end accuracy, and more than doubles the Good Personalization rate.
Agent memory is not just RAG over user facts
The article argues that simple RAG-based agent memory systems fail in production due to issues like stale preferences, missed keywords, and prompt injection, and advocates for a layered memory architecture with active selection, deterministic fallback, governance, and testing.