When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents

# Benchmarking Context-Driven Memory Retrieval in Conversational Agents
Source: [https://arxiv.org/html/2609.03467](https://arxiv.org/html/2609.03467)
Yun\-Nung ChenAffiliation:National Taiwan University, Taipei, TaiwanEmail:[f10946031@csie\.ntu\.edu\.tw](mailto:)y\.v\.chen@ieee\.org

###### Abstract

Large language models \(LLMs\) are increasingly deployed as long\-horizon conversational agents, motivating growing interest in memory systems\. However, existing benchmarks primarily evaluate memory through QA\-style probing rather than in\-situ conversational usage\. We introduceLoCoMo\-Conv, a conversational memory benchmark derived from LoCoMo with four query styles:*dialog*,*implicit*,*counterfactual*, and*composed*\. Across five representative memory systems, we evaluate both retrieval recall and end\-to\-end response quality\. Our experiments show that conversational framing exposes substantial retrieval gaps overlooked by QA benchmarks, especially on implicit and composed queries, which multi\-facet query rewriting narrows for raw\-turn memory but not abstractive memory\. We further find that strong retrieval does not fully translate into response quality, and that implicit queries exhibit*silent grounding*, where memory improves contextual grounding without explicitly surfacing the gold fact\. These results point to reasoning\-based memory elaboration as a promising direction, and we release auxiliarysupportive\_memoryannotations capturing conversationally useful context beyond the original gold evidence\.111[https://github\.com/MiuLab/LoCoMo\-Conv/](https://github.com/MiuLab/LoCoMo-Conv/)

Table 1:Comparison with existing memory benchmarks\. LoCoMo\-Conv varies only the query framing over fixed LoCoMo memories and scores retrieval and response separately\.Table 2:One LoCoMo QA rewritten into our 3 conversational query styles\. The gold answer and gold evidence turns \(left\) are shared across all rewrites; only the user\-side phrasing \(right\) changes\. The counterfactual rewrite injects an incorrect premise \(red\) that the assistant must correct\.Table 3:Unlike dialog/implicit/counterfactual, which rewrite a single LoCoMo QA,composed queries combine two source QAsinto one conversational request requiring multi\-fact synthesis\. The top half shows the source QAs and their evidence turns, with shared turns \(∗\\ast\) marking the cluster overlap\. The bottom row shows the LLM\-generated composed query and synthesis rubric\.## 1Introduction

![Refer to caption](https://arxiv.org/html/2609.03467v1/gap.png)Figure 1:Illustration of the gap betweenLoCoMo\-Convand the existing benchmarksLarge language models \(LLMs\) are increasingly deployed as long\-horizon conversational agents that users interact with across sessions\. This shift makes memory a core requirement: assistants must not only store prior interactions, but also retrieve and use them appropriately when a future conversation calls for them\. Recent work has proposed diverse memory architectures and benchmarks for evaluating long\-term conversational memory\. However, most existing evaluations still probe memory through explicit QA\-style queries \(as shown in Table[1](https://arxiv.org/html/2609.03467#S0.T1)\), leaving open whether these systems can support natural conversational interaction\.

To address this gap, we introduceLoCoMo\-Conv, a conversational memory benchmark that recasts LoCoMo’s QA pool into four query styles reflecting how users naturally invoke memory—dialog,implicit,counterfactual, andcomposed—and evaluates how memory\-augmented agents use retrieved memory within natural dialogue rather than under explicit probing\.

Across five representative memory systems, we report three findings\. First, conversational framing uncovers substantial retrieval failures masked by QA\-style evaluation, especially on implicit and composed queries; multi\-facet query rewriting narrows the gap, but only for raw\-turn memory, not abstractive memory\. Second, strong retrieval does not guarantee grounded responses— abstractive compression in particular aids matching while discarding the detail needed to ground a response—pointing to reasoning\-based memory elaboration over lossy compression\. Third, we identify*silent grounding*, where memory improves implicit\-query responses even without surfacing the gold fact, exposing a limitation of strict fact\-recall metrics\. Our main contributions are:

1. 1\.We introduceLoCoMo\-Conv, a conversational benchmark with four first\-person query styles that evaluates memory systems beyond QA\-style probing\.
2. 2\.We provide a unified evaluation framework spanning both retrieval and response generation, enabling systematic comparison of extractive and abstractive memory systems under conversational settings\.
3. 3\.We releasesupportive\_memory, an auxiliary annotation layer capturing conversationally supportive context beyond the original gold evidence, facilitating the analysis of memory grounding beyond explicit fact recall\.

## 2Related Work

### 2\.1Benchmarks for Long\-Term Conversational Memory

Existing long\-term memory benchmarks differ mainly in how memory is probed and what is scored\.LoCoMo\([Maharana et al\., 2024](https://arxiv.org/html/2609.03467#bib.bib3)\)pioneers very long multi\-session conversations with third\-person QA across five categories \(single/multi\-hop reasoning, temporal, adversarial, open commonsense\)\.LongMemEval\([Wu et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib16)\)defines five memory abilities—information extraction, multi\-session reasoning, temporal reasoning, knowledge updates, and abstention—probed through curated QA over scalable dialogue histories\.MemoryAgentBench\([Hu et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib17)\)reformulates long\-context datasets into incremental multi\-turn streams to evaluate four memory competencies \(accurate retrieval, test\-time learning, long\-range understanding, conflict resolution\) at scales up to 1\.4M tokens\. Other benchmarks target specialized memory failure modes:HaluMem\([Chen et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib6)\)decomposes memory hallucination into extraction, updating, and QA stages;PrefEval\([Zhao et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib5)\)evaluates whether LLMs adhere to user preferences in long\-context dialogue; andHorizonBench\([Li et al\., 2026a](https://arxiv.org/html/2609.03467#bib.bib11)\)probes long\-horizon personalization under evolving preferences with∼\\sim163K\-token, 6\-month conversation histories\.

Closer to conversational use,PersonaMem\([Jiang et al\., 2025a](https://arxiv.org/html/2609.03467#bib.bib14)\)and its successorPersonaMem\-v2\([Jiang et al\., 2025b](https://arxiv.org/html/2609.03467#bib.bib4)\)target implicit*user preferences*accumulated across sessions, finding that frontier LLMs achieve only 37–48% accuracy on implicit personalization; both, however, evaluate via multiple\-choice selection and probe whether the agent has internalized user traits over time rather than whether it surfaces relevant memory from a single utterance at query time\.MADial\-Bench\([He et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib18)\)evaluates memory\-augmented dialogue generation with proactive recall, but within a single emotion\-support domain and with small\-scale human evaluation\.LoCoMo\-Plus\([Li et al\., 2026b](https://arxiv.org/html/2609.03467#bib.bib19)\)tests whether a conversational continuation stays consistent with latent constraints, where a valid response need not retrieve or express any specific fact\.AMemGym\([Jiayang et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib20)\)evaluates memory through on\-policy interaction with simulated users, which is complementary to our fixed\-history setting\.

Despite this diversity in measurement axes, nearly all existing benchmarks evaluate through structured probing—third\-person QA, multiple\-choice selection, constraint checks, or operation\-level traces—rather than free\-form conversational response generation, and none requires an external memory system to retrieve and ground a specific, verifiable fact from a user utterance that does not ask for it\. LoCoMo\-Conv departs from this convention by evaluating*how an assistant integrates memory into a conversational response*, the form in which memory is actually consumed in deployment\.

### 2\.2Memory\-Augmented Conversational Agents

Recent work has explored diverse architectural strategies for equipping LLMs with long\-term memory in conversational settings\.mem0\([Chhikara et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib13)\)andAnchorMem\([Shen et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib7)\)extract atomic facts from raw interactions and organize them into structured stores, with AnchorMem further constructing an associative event graph to capture cross\-memory dependencies\. A second line compresses past interactions into condensed representations:COMEDY\([Chen et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib2)\)eschews retrieval entirely by using a single LLM to generate, compress, and consume compressive memory;LightMem\([Fang et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib21)\)adopts a three\-stage Atkinson–Shiffrin\-inspired pipeline; andMemAgent\([Yu et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib12)\)processes long inputs in segments under an overwrite policy optimized via reinforcement learning\. A third family draws on operating\-system principles—MemGPT\([Packer et al\., 2024](https://arxiv.org/html/2609.03467#bib.bib1)\)introduces tiered memory hierarchies with explicit paging,MemoryOS\([Kang et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib9)\)extends this with short\-, mid\-, and long\-term storage tiers, andHiGMem\([Cao et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib8)\)adopts a two\-level event\-turn structure with LLM\-guided turn selection\.A\-MEM\([Xu et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib10)\)treats memory construction itself as agent\-driven, building an interconnected knowledge network with dynamic linking and memory evolution following Zettelkasten principles\. Finally,Memora[Xia et al\. \(2026\)](https://arxiv.org/html/2609.03467#bib.bib15)proposes a harmonic memory representation that separates abstract memory indices from detailed memory values\. Each memory consists of a primary abstraction for indexing, multiple cue anchors for diverse retrieval access, and an uncompressed memory value preserving fine\-grained details\. For evaluation we select five systems that collectively span the dominant paradigms above \(AnchorMem, A\-MEM, mem0, Memora and a dense\-retrieval baseline usingall\-MiniLM\-L6\-v2\), leaving broader\-coverage benchmarking to future work\.

## 3LoCoMo\-Conv

We constructLoCoMo\-Convby rewriting each LoCoMo10 question into four conversational query styles while preserving the original gold answers and evidencedia\_ids\.222Three annotators validated the rewrites on 40 items per style: they are natural \(4\.6–4\.8 of 5\), preserve the original information need, and invoke the intended memories; see Appendix[G](https://arxiv.org/html/2609.03467#A7)\.

### 3\.1Conversational Query Styles

##### Dialog\.

A direct first\-person conversational reformulation of the original QA query \(e\.g\., “Do you remember when I…?”\)\. The underlying factual target remains unchanged, but the phrasing is rewritten to resemble natural dialogue rather than third\-person question answering\.

##### Implicit\.

A situational conversational utterance in which the user does not explicitly ask a question\. Instead, the assistant must infer that a relevant past memory should be proactively surfaced\. For example, a factual QA asking about a parent’s hobbies may be rewritten into a conversational situation involving gift selection or reminiscing\.

##### Counterfactual\.

A first\-person conversational query containing an incorrect premise about the gold fact\. The assistant is expected to identify and correct the false assumption using the underlying conversational memory\.

##### Composed\.

A multi\-memory conversational query requiring synthesis across multiple source QAs and evidence spans\. We describe the construction process in §[3\.2](https://arxiv.org/html/2609.03467#S3.SS2)\.

For all non\-composed styles, we promptGPT\-5\.4\-miniwith the original question, gold answer, and speaker identity, and instruct it to generate conversational rewrites that preserve the original information need while remaining natural and first\-person in style\. For counterfactual queries, the prompt additionally injects a specific false premise\. Example rewrites are shown in Table[2](https://arxiv.org/html/2609.03467#S0.T2), and all the prompts for query rewriting are provided in Appendix[C\.1](https://arxiv.org/html/2609.03467#A3.SS1)\.

### 3\.2Composed Multi\-Memory Clusters

Unlike the other conversational styles, composed queries are constructed by combining multiple source QAs into a single conversational request that requires multi\-memory synthesis\.

For each QA instanceqq, letE⁡\(q\)⊆𝒟E\(q\)\\subseteq\\mathcal\{D\}denote its gold evidence dia\_ids\. We enumerate all QA pairs\(qi,qj\)\(q\_\{i\},q\_\{j\}\)within the same sample and retain a pair as a candidate cluster if it satisfies three conditions: \(i\) overlapping evidence,E⁡\(qi\)∩E⁡\(qj\)≠∅E\(q\_\{i\}\)\\cap E\(q\_\{j\}\)\\neq\\emptyset; \(ii\) non\-identical evidence,E⁡\(qi\)≠E⁡\(qj\)E\(q\_\{i\}\)\\neq E\(q\_\{j\}\); and \(iii\) non\-trivial combined evidence,\|E⁡\(qi\)∪E⁡\(qj\)\|≥2\|E\(q\_\{i\}\)\\cup E\(q\_\{j\}\)\|\\geq 2\. These constraints ensure that the two QAs are topically related while still contributing distinct information\. We retain all valid clusters under these constraints, yielding 1,069 composed clusters in total \(36–198 per conversation\)\.

For each cluster, we promptGPT\-5\.4\-miniwith the source QA pairs and their gold evidence turns, and ask it to generate a natural conversational query that implicitly requires both source answers\. The composed gold answer is defined as the set of atomic answers from the constituent QAs, while the gold evidence corresponds to the union of their dia\_ids\.

Because composed queries require synthesizing multiple atomic facts, binary correctness is often overly strict\. We therefore evaluate composed responses using continuous*atomic\-fact coverage*\(Section[4](https://arxiv.org/html/2609.03467#S4)\), which measures the fraction of atomic gold facts correctly covered by the generated response\. An example composed cluster is shown in Table[3](https://arxiv.org/html/2609.03467#S0.T3)\.

## 4Evaluation Methodology

We evaluate memory systems along two complementary dimensions: \(1\)retrieval recallagainst the gold dialogue IDs, and \(2\)end\-to\-end response qualityusing style\-specific LLM judges\.

### 4\.1Retrieval Recall

For each query, we compare the top\-KKretrieved memories against the gold evidencedia\_ids\. A gold turn is considered retrieved if its verbatim text appears within any returned memory \(case\-insensitive\)\. For abstractive systems that expose source metadata, we additionally match retrieveddia\_ids through metadata fields\. Retrieval recall is computed as:

\|retrieved∩gold\|\|gold\|\.\\frac\{\|\\text\{retrieved\}\\cap\\text\{gold\}\|\}\{\|\\text\{gold\}\|\}\.

### 4\.2Response Quality Judging

Retrieved memories are provided to the answer model to generate responses, which are then evaluated by an LLM judge using a style\-specific rubric\.

- •DialogandImplicit:*partial\-creditfact\_used*on a 3\-level scale \(1\.01\.0if the substance of the gold fact is correctly conveyed—paraphrasing and hedging permitted;0\.50\.5if the response captures the central concept of a multi\-item gold but misses specifics;0\.00\.0if the response asserts contradicting content, gives only vague allusion, or completely omits the fact\)\.
- •Counterfactual: a 3\-way*unaware / hedge / corrected*judge mapped to0/0\.5/10/0\.5/1\. Class A \(*unaware*\) treats the user’s false premise as if it were true; class B \(*hedge*\) signals awareness of a mismatch but does not state the ground\-truth fact; class C \(*corrected*\) states \(or clearly implies\) the ground\-truth fact regardless of whether it explicitly points out the user’s error\.
- •Composed:*atomic\-fact coverage*, the fraction of gold atomic facts the response covers \(judged independently per fact\)\. This avoids the strict all\-or\-nothing failure of a singlefact\_usedcall for multi\-fact composed clusters\.

The full judge prompts are provided in Appendix[C](https://arxiv.org/html/2609.03467#A3)\.

## 5Experimental Setup

We evaluate five representative systems:AnchorMem\([Shen et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib7)\),A\-MEM\([Xu et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib10)\),mem0\([Chhikara et al\., 2025](https://arxiv.org/html/2609.03467#bib.bib13)\),Memora333We disable Memora’s default retrieval threshold \(0\.4\), which otherwise returns nothing for 42–71% of conversational queries\.\([Xia et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib15)\)andNaiveRAGbuilt withall\-MiniLM\-L6\-v2\. We follow the original implementation settings while unifying the embedding model toall\-MiniLM\-L6\-v2and the backbone LLM togemma\-4\-31B\-it\. All systems retrieve top\-K=10K\{=\}10memories444AnchorMem retrieves*anchor chunks*rather than individual turns, yielding approximately 12\.8 turns on average at top\-K=10K\{=\}10\., which are passed to the same answer\-generation model\.555temperature=0,max\_tokens=300\.

For response evaluation, we useGPT\-5\.4\-miniwith reasoning enabled as the primary judge, and validate agreement againstClaude\-sonnet\-4\.5andQwen3\.6\-35b\-A3Bon a stratified 1,488\-item subset \(Appendix[A](https://arxiv.org/html/2609.03467#A1)\)\.

System / VariantRetrieval Recall@10Response QualityDialogImplicitCfComp\.DialogImplicitCfComp\.No\-memory floor / Oracle ceilingNo Memory————\.004\.067\.251\.008Oracle————\.777\.724\.775\.597Baselines \(raw turns, no LLM at ingest\)Naive RAG\.533\.312\.573\.266\.518\.335\.608\.308\+ Query Rewriting\.555↑\\uparrow\.022\.346↑\\uparrow\.034\.552↓\\downarrow\.021\.281↑\\uparrow\.015\.536↑\\uparrow\.018\.368↑\\uparrow\.033\.608↑\\uparrow\.000\.324↑\\uparrow\.016↪\\hookrightarroww/ Response CoT————\.538↑\\uparrow\.020\.376↑\\uparrow\.041\.480↓\\downarrow\.128\.328↑\\uparrow\.020Memory systems \(raw turns \+ structure\)A\-MEM[Xu et al\. \(2025\)](https://arxiv.org/html/2609.03467#bib.bib10)\.531\.308\.571\.267\.515\.326\.609\.307\+ Query Rewriting\.551↑\\uparrow\.020\.344↑\\uparrow\.036\.548↓\\downarrow\.023\.282↑\\uparrow\.015\.533↑\\uparrow\.018\.376↑\\uparrow\.050\.606↓\\downarrow\.003\.321↑\\uparrow\.014↪\\hookrightarroww/ Response CoT————\.530↑\\uparrow\.015\.370↑\\uparrow\.044\.470↓\\downarrow\.139\.331↑\\uparrow\.024AnchorMem[Shen et al\. \(2026\)](https://arxiv.org/html/2609.03467#bib.bib7)\.659\.368\.639\.279\.598\.364\.653\.310\+ Query Rewriting\.754↑\\uparrow\.095\.524↑\\uparrow\.156\.732↑\\uparrow\.093\.432↑\\uparrow\.153\.669↑\\uparrow\.071\.470↑\\uparrow\.106\.669↑\\uparrow\.016\.413↑\\uparrow\.103↪\\hookrightarroww/ Response CoT————\.620↑\\uparrow\.022\.389↑\\uparrow\.025\.523↓\\downarrow\.130\.314↑\\uparrow\.004Abstractive memory systemmem0[Chhikara et al\. \(2025\)](https://arxiv.org/html/2609.03467#bib.bib13)\.547\.456\.512\.374\.366\.330\.548\.290\+ Query Rewriting\.479↓\\downarrow\.068\.453↓\\downarrow\.003\.532↑\\uparrow\.020\.364↓\\downarrow\.010\.337↓\\downarrow\.029\.337↑\\uparrow\.007\.558↑\\uparrow\.010\.284↓\\downarrow\.006↪\\hookrightarroww/ Response CoT————\.393↑\\uparrow\.027\.368↑\\uparrow\.038\.387↓\\downarrow\.161\.311↑\\uparrow\.021Memora[Xia et al\. \(2026\)](https://arxiv.org/html/2609.03467#bib.bib15)\.608\.445\.600\.387\.501\.388\.617\.333\+ Query Rewriting\.616↑\\uparrow\.008\.458↑\\uparrow\.013\.583↓\\downarrow\.017\.406↑\\uparrow\.019\.513↑\\uparrow\.012\.403↑\\uparrow\.015\.606↓\\downarrow\.011\.340↑\\uparrow\.007↪\\hookrightarroww/ Response CoT————\.538↑\\uparrow\.037\.422↑\\uparrow\.034\.462↓\\downarrow\.155\.354↑\\uparrow\.021

Table 4:Joint retrieval and response evaluation across conversational query styles\.Left: retrieval recall@10 against gold evidence turns\.Right: end\-to\-end response quality\.*\+ Query Rewriting*applies multi\-facet query rewriting with RRF fusion, while*w/ Response CoT*adds explicit memory selection before response generation using the same retrieved memories\. Bold indicates the best\-performing real memory system in each column \(excluding No Memory and Oracle\)\.
## 6Results

### 6\.1Retrieval Recall

Table[4](https://arxiv.org/html/2609.03467#S5.T4)\(left\) reports retrieval recall@10 across all conversational query styles\. Retrieval performance drops substantially on*implicit*queries compared to direct*dialog*queries, confirming that conversational framing without explicit questions is significantly more challenging for current memory systems\.

AnchorMem performs the best on factual retrieval\-oriented styles, achieving the highest recall on*dialog*\(0\.659\) and*counterfactual*\(0\.639\) queries\. This behavior aligns with AnchorMem’s graph\-structured design, which is optimized for retrieving specific factual anchors from prior interactions\([Shen et al\., 2026](https://arxiv.org/html/2609.03467#bib.bib7)\)\. In contrast,abstractive systems \(mem0 and Memora\) perform best on semantically broad conversational styles, outperforming AnchorMem by notable margins on*implicit*\(0\.456/0\.445 vs\. 0\.368\) and*composed*\(0\.374/0\.387 vs\. 0\.279\) queries\. The abstractive memory representation appears more robust when the conversational surface\-form diverges substantially from the original dialogue evidence\. Overall,*implicit*and*composed*queries remain the hardest styles across every system, confirming thatconversational framing without an explicit question poses a fundamental retrieval challenge\.

### 6\.2Response Quality

Table[4](https://arxiv.org/html/2609.03467#S5.T4)\(right\) presents end\-to\-end response quality\. The ranking differs substantially from retrieval recall, and the two abstractive systems show contrasting behavior\. AnchorMem achieves the strongest response quality on*dialog*and*counterfactual*queries, and Memora on*implicit*and*composed*queries, despite AnchorMem trailing both abstractive systems on*implicit*and*composed*retrieval\. mem0, by contrast, excels at retrieval yet performs noticeably worse on downstream generation, particularly on*dialog*and*counterfactual*queries\. This discrepancy reveals a clearretrieval\-to\-response gap: semantically relevant memories do not necessarily translate into grounded conversational responses\.Crucially, the gap is not a property of abstractive memory per se: Memora, which also stores LLM\-extracted memories, converts its retrieval advantage into the best implicit and composed responses, whereas mem0—strong on retrieval yet among the weakest on response quality—does not\. Even the best systems remain far below the oracle ceiling on implicit and composed queries\. We examine why retrieval gains fail to transfer in two places: §[7\.3](https://arxiv.org/html/2609.03467#S7.SS3)shows part of the gap reflects a limitation of fact\-recall metrics rather than the systems themselves, while §[7\.5](https://arxiv.org/html/2609.03467#S7.SS5)traces the mem0–Memora contrast to a structural cost of lossy compression rather than abstraction itself\.

## 7Analysis and Findings

Table 5:Hallucination rate on unanswerable conversational queries\.### 7\.1Hallucination under Conversational Framing

We evaluate hallucination on unanswerable conversational queries, where the queried fact is absent or incorrectly attributed\. For each query, the answer model receives AnchorMem’s top\-10 retrieved memories, and hallucination is defined as asserting unsupported memory\-grounded facts\.

Table[5](https://arxiv.org/html/2609.03467#S7.T5)comparesgemma\-4\-31B\-itandqwen3\.6\-35B\-A3Bwith thinking enabled and disabled\.666Both models expose anenable\_thinkingflag\.Implicit framing substantially increases hallucination when reasoning is disabled, particularly for Qwen\. Enabling reasoning reduces hallucination differently across model families, but hallucination rates remain high across all settings,suggesting that conversational memory hallucination remains challenging even with strong retrieval\.

### 7\.2Multi\-Facet Query Rewriting Narrows the Retrieval Gap

We hypothesize that the retrieval gap on conversational queries arises from shallow surface\-form matching\. Conversational utterances often contain multiple latent retrieval targets, while standard retrieval typically focuses on only a single semantic aspect\. For example, “organizing my health journal from last year” may implicitly relate to medical visits, emotional states, or wellness milestones, yet dense retrieval often retrieves memories associated with only one facet\.

##### Setup\.

To address this issue, we introducemulti\-facet query rewriting\. We usegpt\-5\.4\-minito decompose each conversational query into 3–5 complementary facets spanning different entities, themes, time periods, or semantic angles\. Each rewritten facet is retrieved independently, and the final ranking is aggregated using Reciprocal Rank Fusion \(RRF\)\.

##### Results\.

Table[4](https://arxiv.org/html/2609.03467#S5.T4)shows that multi\-facet rewriting consistently improves retrieval recall across most systems and query styles\. The gains are largest for AnchorMem, with improvements of \+9\.5pt on dialog, \+15\.6pt on implicit, \+9\.3pt on counterfactual, and \+14\.7pt on composed queries\. These resultssuggest that conversational retrieval failures often stem from insufficient query diversification rather than purely weak memory representations\.

Interestingly, abstractive memory behaves differently from raw\-turn memory systems\. mem0 is the only system where rewriting degrades*dialog*performance, reducing both retrieval and response quality, and Memora, the other abstractive system, is not degraded but gains at most±\\pm0\.02 on any style—an order of magnitude less than AnchorMem\. Because both systems canonicalize repeated mentions into a single stored statement, the different facets of a rewritten query no longer reinforce the same underlying evidence: for mem0 they scatter across separate summaries, and for Memora they simply re\-match the one canonical index entry\.This suggests that abstractive memory, while beneficial for semantic matching, gains little from diversified conversational retrieval, because the surface\-form redundancy that multi\-facet rewriting exploits is removed at construction time\.We return to this contrast in §[7\.5](https://arxiv.org/html/2609.03467#S7.SS5)\.

Table 6:Pairwise preference margin on the subset of implicit queries exhibiting silent grounding

### 7\.3Silent Grounding: Beyond Explicit Fact Recall

Comparing the retrieval and response halves of Table[4](https://arxiv.org/html/2609.03467#S5.T4)reveals a consistent gap: retrieval improvements only partially translate into response gains\. For example, AnchorMem’s multi\-facet rewrite improves implicit retrieval recall by\+\+15\.6pt but response quality by only\+\+10\.6pt\. While this may suggest that answer models fail to utilize retrieved evidence, part of the gap may instead reflect a limitation of explicit fact\-based evaluation\. Ourfact\_usedmetric scores whether the response surfaces the gold fact, but onimplicit queries memory can still improve responses without directly stating that fact—for example, through contextual grounding, appropriate tone, or relevant follow\-up questions\. We refer to this phenomenon as*silent grounding*\.

##### Setup\.

We analyze the 332 implicit\-query cases where oracle retrieval still receivesfact\_used= 0\.0\. We compare three response variants:Oracle\(gold evidence turns\),no\-mem\(no memory provided\), andrandom\(three random non\-gold turns from the same conversation as a memory control\)\. For each case we score every variant’s response with Claude\-Opus\-4\.7 on three independent criteria \(*faithfulness, relevance, engagement*; each scored0/0\.5/10/0\.5/1\), and report pairwise dimension margins \(variant\-A win rate−\-variant\-B win rate\)\.

##### Results and Implication\.

Table[6](https://arxiv.org/html/2609.03467#S7.T6)shows that Oracle memory substantially outperforms both no\-memory and random\-memory baselines, particularly on faithfulness \(\+\+55\.1pt vs no\-mem\) and engagement \(\+\+31\.0pt\)\. These results suggest that conversational memory often improves contextual grounding without explicitly surfacing the gold fact, implying that strict fact\-recall metrics alone underestimate the value of retrieval on implicit queries\. Appendix[E](https://arxiv.org/html/2609.03467#A5)provides qualitative examples comparing the three settings\.

Table 7:Pairwise preference margin \(CoT win rate−\-Oracle win rate\) on implicit queries\.

### 7\.4Chain\-of\-Thought Selection Approaches Oracle Quality

The default setting directly feeds all retrieved top\-KKmemories into the answer model\. We compare this against a*CoT*variant, where the model first explicitly selects relevant memories before generating its response\. The retrieved top\-KKremains identical; only the answer\-side prompt changes\.

##### Mixed effect on response quality\.

As shown in Table[4](https://arxiv.org/html/2609.03467#S5.T4),*CoT*consistently improves dialog,*implicit*, and*composed*responses, but substantially hurts*counterfactual*performance across all systems\. On counterfactual queries,*CoT*prompting lowers performance because the model first restates the user’s message before consulting memory: the false premise becomes the framing of the response, and the retrieved memory is then reconciled with it rather than used to correct it\. Asking the model to explicitly select the memories it uses recovers only a small part of this loss \(Appendix[F](https://arxiv.org/html/2609.03467#A6)\)\.

##### CoT\-selected responses approach oracle quality\.

We compare*CoT*responses against Oracle on the full implicit set using the same Claude\-Opus\-4\.7 pairwise evaluation as §[7\.3](https://arxiv.org/html/2609.03467#S7.SS3)\. Table[7](https://arxiv.org/html/2609.03467#S7.T7)shows that there isa huge gap when it comes to response engagement:*CoT*beats Oracle by\+\+31 to\+\+40 points on engagement, while faithfulness and relevance margins stay within±\\pm7 points across systems\. Broader retrieved context can therefore support more grounded conversational responses without sacrificing factual reliability\.

##### AugmentingLoCoMo\-Convwith supportive memory\.

Motivated by this observation, we extendLoCoMo\-Convwith asupportive\_memoryfield for*implicit*queries, containing conversational turns frequently selected by successful*CoT*responses\. The originalevidenceannotations remain unchanged, whilesupportive\_memoryprovides an auxiliary conversational\-support context for future evaluation\. For detailed qualitative analysis, please refer to Appendix[E](https://arxiv.org/html/2609.03467#A5)\.

### 7\.5Compression versus Elaboration in Memory Construction

The rewriting results in §[7\.2](https://arxiv.org/html/2609.03467#S7.SS2)suggest an implication that extends beyond query rewriting itself\. Multi\-facet rewriting yields the largest improvement on implicit retrieval \(\+15\.6 for AnchorMem\) by expanding an underspecified conversational utterance into multiple semantically explicit facets before retrieval\. Rather than introducing new evidence, the rewriting process exposes semantic aspects already implied by the user’s utterance, allowing them to align more readily with the relevant memory\. This observation suggests that the primary challenge of implicit conversational retrieval is not the absence of evidence, but semantic underspecification: conversational surface forms often fail to express the concepts necessary for successful memory matching\.

This naturally raises a broader question:if semantic elaboration improves retrieval when applied at query time, can the same principle be incorporated during memory construction?The two abstractive memory systems provide evidence in favor of this hypothesis\. Both process interactions before storage and achieve the strongest retrieval performance on the most challenging query styles, reaching implicit/composed recall of 0\.456/0\.374 for mem0 and 0\.445/0\.387 for Memora, compared with 0\.368/0\.279 for AnchorMem\. These results suggest that enriching memory representations prior to storage, rather than preserving raw dialogue turns alone, substantially improves conversational retrieval\.

Retrieval performance alone, however, does not guarantee better responses\. The way in which memory is abstracted determines whether retrieved information remains useful for response generation\. Here the two systems diverge\. mem0 constructs memory primarily through compression, merging repeated observations into concise summary facts\. Although this representation appears sufficient for semantic matching, it yields one of the weakest response\-generation results \(dialogfact\_used: 0\.366 versus 0\.598 for AnchorMem\), and its strong implicit retrieval does not translate into corresponding gains in implicit response quality\. Memora instead adopts a different abstraction strategy: retrieval operates over a compact abstractive index augmented with cue anchors, while the retrieved memory retains detailed original content for generation\. Despite achieving retrieval performance comparable to mem0, Memora attains the best implicitfact\_used\(0\.388\) and composed coverage \(0\.333\)\. These results suggest that abstraction itself is not detrimental\. Rather, the limitation arises when abstraction becomes*lossy*: compression preserves sufficient semantic information for retrieval while discarding the concrete details required to ground a response\.

The rewriting experiments further clarify the relationship between query\-time and memory\-time elaboration\. If semantic expansion has already been incorporated into stored memory representations, additional elaboration at retrieval time should offer only marginal benefit\. Empirically, this is exactly what we observe\. Multi\-facet rewriting improves AnchorMem’s retrieval by \+0\.09 to \+0\.16 recall, yet changes Memora by at most±0\.02\\pm 0\.02and mem0 by only−0\.07\-0\.07to\+0\.02\+0\.02\. Query rewriting and memory elaboration therefore appear to serve largely overlapping roles, suggesting that they address the same underlying source of retrieval failure—semantic underspecification—at different stages of the memory pipeline\.

Taken together, these observations point toward a broader design principle for conversational memory systems\. Memory construction should increase semantic accessibility beyond raw dialogue turns while simultaneously preserving the specific information required for response grounding\. In other words, this suggests that future conversational memory systems may benefit more from semantic elaboration than lossy compression\. We emphasize, however, that this comparison should not be interpreted as a definitive evaluation of compression versus elaboration strategies\. The two abstractive systems differ in several design choices beyond their memory\-construction mechanisms \(e\.g\., Memora’s cue\-anchor design\), and these factors may also contribute to the observed differences\. A controlled comparison that isolates memory\-construction strategies from other architectural factors remains an important direction for future work\.

## 8Conclusion

We introducedLoCoMo\-Conv, a benchmark for evaluating whether memory\-augmented agents can invoke memory under realistic conversational framing rather than explicit QA\. Our findings are fourfold\. First, conversational framing reveals retrieval and response gaps hidden by QA\-style evaluation, especially for implicit and composed queries\. Second, strict fact\-recall metrics miss the*silent grounding*we observe on implicit queries, where memory improves responses without explicitly surfacing the gold fact\. Third, while both multi\-facet query rewriting and abstractive memory improve retrieval \(specifically*implicit*and*composed*style\), abstractive compression often removes details needed for grounded responses, suggesting that reasoning\-based memory*elaboration*is more promising than lossy compression\. Finally, we releasesupportive\_memory, an auxiliary annotation layer capturing conversationally supportive context beyond the original gold evidence\.

## Limitations and Future Work

LoCoMo\-Convhas several limitations\. Conversational rewrites andsupportive\_memoryannotations are generated through LLM\-based pipelines and may inherit model\-specific biases; human validation \(Appendix[G](https://arxiv.org/html/2609.03467#A7)\) covers a 40\-item sample per style rather than the full set\. Our evaluation relies on a single open\-weights answer model and one primary judge family, with cross\-judge and human validation performed on subsets\. The benchmark is built on the ten LoCoMo conversations, which is small relative to real\-world long\-horizon interactions; however, the construction pipeline is source\-agnostic—it takes any conversation with QA\-style evidence annotations and applies the same rewriting and clustering procedure—so the benchmark can be scaled to larger or newer conversation pools without changing the methodology\. Future work could extend evaluation to broader model families and to such larger pools, and improve the reliability of supportive\-memory annotations\.

## Acknowledgments

This work was financially supported by the National Science and Technology Council \(NSTC\) and the Featured Area Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education in Taiwan, under Grants 112\-2223\-E\-002\-012\-MY5, 115\-2628\-E\-002\-023\-MY4, and 115L900901\. We also thank Chia\-En Hsu and Chih\-Chih Yang for their help with data annotation\. We used AI assistants to support manuscript editing, language refinement, and presentation\. All research design, experiments, analyses, and conclusions were developed and verified by the authors\.

## References

- Caoet al\.\(2026\)S\. Cao, J\. He, and F\. TanHiGMem: A Hierarchical and LLM\-Guided Memory System for Long\-Term Conversational Agents\.arXiv\.Note:arXiv:2604\.18349 \[cs\] version: 1External Links:[Link](http://arxiv.org/abs/2604.18349),[Document](https://dx.doi.org/10.48550/arXiv.2604.18349)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1)\.
- Chenet al\.\(2026\)D\. Chen, S\. Niu, K\. Li, P\. Liu, X\. Zheng, B\. Tang, X\. Li, F\. Xiong, and Z\. LiHaluMem: Evaluating Hallucinations in Memory Systems of Agents\.arXiv\.Note:arXiv:2511\.03506 \[cs\]External Links:[Link](http://arxiv.org/abs/2511.03506),[Document](https://dx.doi.org/10.48550/arXiv.2511.03506)Cited by:[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p1.1)\.
- Chenet al\.\(2025\)N\. Chen, H\. Li, J\. Chang, J\. Huang, B\. Wang, and J\. LiCompress to Impress: Unleashing the Potential of Compressive Memory in Real\-World Long\-Term Conversations\.InProceedings of the 31st International Conference on Computational Linguistics,O\. Rambow, L\. Wanner, M\. Apidianaki, H\. Al\-Khalifa, B\. D\. Eugenio, and S\. Schockaert \(Eds\.\),Abu Dhabi, UAE,pp\. 755–773\.External Links:[Link](https://aclanthology.org/2025.coling-main.51/)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1)\.
- Chhikaraet al\.\(2025\)P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. YadavMem0: Building Production\-Ready AI Agents with Scalable Long\-Term Memory\.\(en\)\.External Links:[Link](https://arxiv.org/abs/2504.19413v1)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1),[Table 4](https://arxiv.org/html/2609.03467#S5.T4.2.1.1.1.1.1.1.18.1),[§5](https://arxiv.org/html/2609.03467#S5.p1.1)\.
- Fanget al\.\(2026\)J\. Fang, X\. Deng, H\. Xu, Z\. Jiang, Y\. Tang, Z\. Xu, S\. Deng, Y\. Yao, M\. Wang, S\. Qiao, H\. Chen, and N\. ZhangLightMem: lightweight and efficient memory\-augmented generation\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=dyJ0GWpjJB)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1)\.
- Heet al\.\(2025\)J\. He, L\. Zhu, R\. Wang, X\. Wang, G\. Haffari, and J\. ZhangMADial\-bench: towards real\-world evaluation of memory\-augmented dialogue generation\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 9902–9921\.External Links:[Link](https://aclanthology.org/2025.naacl-long.499/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.499),ISBN 979\-8\-89176\-189\-6Cited by:[Table 1](https://arxiv.org/html/2609.03467#S0.T1.2.5.1),[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p2.1)\.
- Huet al\.\(2026\)Y\. Hu, Y\. Wang, and J\. McAuleyEvaluating memory in LLM agents via incremental multi\-turn interactions\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=DT7JyQC3MR)Cited by:[Table 1](https://arxiv.org/html/2609.03467#S0.T1.2.8.1),[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p1.1)\.
- Jianget al\.\(2025a\)B\. Jiang, Z\. Hao, Y\. Cho, B\. Li, Y\. Yuan, S\. Chen, L\. Ungar, C\. J\. Taylor, and D\. RothKnow me, respond to me: benchmarking llms for dynamic user profiling and personalized responses at scale\.External Links:2504\.14225,[Link](https://arxiv.org/abs/2504.14225)Cited by:[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p2.1)\.
- Jianget al\.\(2025b\)B\. Jiang, Y\. Yuan, M\. Shen, Z\. Hao, Z\. Xu, Z\. Chen, Z\. Liu, A\. R\. Vijjini, J\. He, H\. Yu, R\. Poovendran, G\. Wornell, L\. Ungar, D\. Roth, S\. Chen, and C\. J\. TaylorPersonaMem\-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory\.arXiv\.Note:arXiv:2512\.06688 \[cs\]External Links:[Link](http://arxiv.org/abs/2512.06688),[Document](https://dx.doi.org/10.48550/arXiv.2512.06688)Cited by:[Table 1](https://arxiv.org/html/2609.03467#S0.T1.2.4.1),[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p2.1)\.
- Jiayanget al\.\(2026\)C\. Jiayang, D\. Ru, L\. Qiu, Y\. Li, X\. Cao, Y\. Song, and X\. CaiAMemgym: interactive memory benchmarking for assistants in long\-horizon conversations\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=sfrVLzsmlf)Cited by:[Table 1](https://arxiv.org/html/2609.03467#S0.T1.2.7.1),[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p2.1)\.
- Kanget al\.\(2025\)J\. Kang, M\. Ji, Z\. Zhao, and T\. BaiMemory OS of AI Agent\.arXiv\.Note:arXiv:2506\.06326 \[cs\]External Links:[Link](http://arxiv.org/abs/2506.06326),[Document](https://dx.doi.org/10.48550/arXiv.2506.06326)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1)\.
- Liet al\.\(2026a\)S\. S\. Li, B\. Paranjape, K\. Oktar, Z\. Ma, G\. Zhou, L\. Guan, N\. Zhang, S\. Park, L\. Chen, D\. Yang, Y\. Tsvetkov, and A\. CelikyilmazHorizonBench: Long\-Horizon Personalization with Evolving Preferences\.arXiv\.Note:arXiv:2604\.17283 \[cs\]External Links:[Link](http://arxiv.org/abs/2604.17283),[Document](https://dx.doi.org/10.48550/arXiv.2604.17283)Cited by:[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p1.1)\.
- Liet al\.\(2026b\)Y\. Li, W\. Guo, L\. Zhang, R\. Xu, M\. Huang, H\. Liu, L\. Xu, Y\. Xu, and J\. LiuLocomo\-plus: beyond\-factual cognitive memory evaluation framework for LLM agents\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 25085–25100\.External Links:[Link](https://aclanthology.org/2026.acl-long.1150/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1150),ISBN 979\-8\-89176\-390\-6Cited by:[Table 1](https://arxiv.org/html/2609.03467#S0.T1.2.6.1),[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p2.1)\.
- Maharanaet al\.\(2024\)A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. FangEvaluating Very Long\-Term Conversational Memory of LLM Agents\.arXiv\.Note:arXiv:2402\.17753 \[cs\]External Links:[Link](http://arxiv.org/abs/2402.17753),[Document](https://dx.doi.org/10.48550/arXiv.2402.17753)Cited by:[Table 1](https://arxiv.org/html/2609.03467#S0.T1.2.2.1),[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p1.1)\.
- Packeret al\.\(2024\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: Towards LLMs as Operating Systems\.arXiv\.Note:arXiv:2310\.08560 \[cs\]External Links:[Link](http://arxiv.org/abs/2310.08560),[Document](https://dx.doi.org/10.48550/arXiv.2310.08560)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1)\.
- Shenet al\.\(2026\)Z\. Shen, S\. Cheng, Z\. Guo, W\. Wang, Y\. Wang, and H\. HuangAnchorMem: Anchored Facts with Associative Contexts for Building Memory in Large Language Models\.arXiv\.Note:arXiv:2604\.17377 \[cs\] version: 1External Links:[Link](http://arxiv.org/abs/2604.17377),[Document](https://dx.doi.org/10.48550/arXiv.2604.17377)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1),[Table 4](https://arxiv.org/html/2609.03467#S5.T4.2.1.1.1.1.1.1.14.1),[§5](https://arxiv.org/html/2609.03467#S5.p1.1),[§6\.1](https://arxiv.org/html/2609.03467#S6.SS1.p2.1)\.
- Wuet al\.\(2025\)D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. YuLongMemEval: benchmarking chat assistants on long\-term interactive memory\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=pZiyCaVuti)Cited by:[Table 1](https://arxiv.org/html/2609.03467#S0.T1.2.3.1),[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p1.1)\.
- Xiaet al\.\(2026\)M\. Xia, X\. Zhang, S\. Dixit, P\. Harimurugan, R\. Wang, V\. Rühle, R\. Sim, C\. Bansal, and S\. RajmohanMemora: a harmonic memory representation balancing abstraction and specificity\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=zSrvkj0ers)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1),[Table 4](https://arxiv.org/html/2609.03467#S5.T4.2.1.1.1.1.1.1.21.1),[§5](https://arxiv.org/html/2609.03467#S5.p1.1)\.
- Xuet al\.\(2025\)W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. ZhangA\-MEM: Agentic Memory for LLM Agents\.arXiv\.Note:arXiv:2502\.12110 \[cs\]External Links:[Link](http://arxiv.org/abs/2502.12110),[Document](https://dx.doi.org/10.48550/arXiv.2502.12110)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1),[Table 4](https://arxiv.org/html/2609.03467#S5.T4.2.1.1.1.1.1.1.11.1),[§5](https://arxiv.org/html/2609.03467#S5.p1.1)\.
- Yuet al\.\(2025\)H\. Yu, T\. Chen, J\. Feng, J\. Chen, W\. Dai, Q\. Yu, Y\. Zhang, W\. Ma, J\. Liu, M\. Wang, and H\. ZhouMemAgent: Reshaping Long\-Context LLM with Multi\-Conv RL\-based Memory Agent\.\(en\)\.External Links:[Link](https://openreview.net/forum?id=k5nIOvYGCL)Cited by:[§2\.2](https://arxiv.org/html/2609.03467#S2.SS2.p1.1)\.
- Zhaoet al\.\(2025\)S\. Zhao, M\. Hong, Y\. Liu, D\. Hazarika, and K\. LinDo LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs\.arXiv\.Note:arXiv:2502\.09597 \[cs\]External Links:[Link](http://arxiv.org/abs/2502.09597),[Document](https://dx.doi.org/10.48550/arXiv.2502.09597)Cited by:[§2\.1](https://arxiv.org/html/2609.03467#S2.SS1.p1.1)\.

## Appendix ALLM Judge Validation

As shown in Table[8](https://arxiv.org/html/2609.03467#A1.T8)\.

Table 8:Cross\-judge agreement on response judging
## Appendix BData Statistics

LoCoMo\-Convattaches conversational rewrites to each QA item in LoCoMo10\. Table[9](https://arxiv.org/html/2609.03467#A2.T9)lists the per\-style counts\. Dialog and implicit share the full 1,986\-item QA pool; counterfactual excludes the 446 cat\-5 adversarial items whose premise has no gold answer; composed clusters are constructed by combining two source QAs whose gold evidence overlaps\.

Table 9:Per\-style query counts inLoCoMo\-Conv\.
## Appendix CPrompts

All four conversational query styles inLoCoMo\-Convare generated by promptinggpt\-5\.4\-minito rewrite original LoCoMo10 content into a first\-person utterance\. For the three single\-QA styles \(Dialog, Implicit, Counterfactual\), each API call uses a*system*message \(the style\-specific instruction below\) plus a shared*user*message template carrying the QA data\. The Composed style takes multiple source QAs as input and is therefore delivered as a single user message that interpolates both the task instruction and the member memories\.

### C\.1Conversational query rewrite prompts

##### Shared user prompt template

speaker\_a:\{speaker\_a\}

speaker\_b:\{speaker\_b\}

Category:\{category\}\(\{category\_desc\}\)

Question:\{question\}

Goldanswer:\{answer\}

Rewritethisasonefirst\-personutterancefromthesubjectspeaker\.

#### C\.1\.1Dialog rewrite

Youconvertthird\-personQApairsintonaturalfirst\-persondialogturns\(EXPLICITmemoryqueries\)\.

Setting:Thetwopeoplein‘conversation‘\(speaker\_a,speaker\_b\)havebeenchattingwithanAIassistantforalongtime\.Theassistanthasmemoriesofeverythingthey’veshared\.NowoneofthemopensafreshchatwiththeassistantandsaysONEmessagethatshouldnaturallymaketheassistantlookuptherightmemoryandanswertheoriginalquestion\.

Yourjob,foroneQAatatime:

1\.Choosethesubjectspeaker\-\-theparticipantthequestionisaskingABOUT\(e\.g\."WhatdidCarolineresearch?"\-\>Caroline\)\.ThedialogturnmustcomefromTHATspeaker,addressedtotheassistantinfirstperson\("I","me","my"\)\.

2\.WriteasinglenaturalutterancethatthisspeakerwouldactuallysendasaDIRECTmemoryquery\(e\.g\."Hey,doyourememberwhenIpaintedthatsunrise?","RemindmewhatIwasresearchinglastspring?"\)\.TheutterancemustNOTcontaintheanswer,dates,orevidencespecifics\.

3\.Ratenaturalness:

\-"high"=soundslikearealthingsomeonewouldsayinchat

\-"medium"=slightlycontrivedbutplausible

\-"low"=arealpersonwouldNOTdirectlyaskthis\.Theinformationisthekindthatcomesupimplicitlythroughcontext\(advice\-seeking,sharingafeeling,describingasituation\),notthroughadirect"doyourememberX"question\.

4\.Reasonfield:oneshortsentencejustifyingnaturalness;for"low",alsohintatwhatimplicitcontextwouldnaturallysurfacethismemory\.

ReturnSTRICTJSONwithkeys:subject\_speaker,dialog\_query,naturalness,reason\.Nopreamble,nocodefences\.

#### C\.1\.2Implicit rewrite

Youconvertthird\-personQApairsintoIMPLICITfirst\-persondialogturns\.

Setting:Thetwopeoplein‘conversation‘\(speaker\_a,speaker\_b\)havebeenchattingwithanAIassistantforalongtime\.Theassistanthasmemoryofeverythingthey’veshared\.NowthesubjectspeakeropensafreshchatandsaysONEmessage\.Themessagepresentsareal\-lifeCONTEXT\(afeeling,situation,problem,plan,decisionthey’refacing\)wherethegoldmemorybelowwouldbetherelevantthingfortheassistanttorecall,surface,orapplyonitsowninitiative\.

Hardrules:

\-TheutterancemustNOTdirectlyask"doyouremember\.\.\.","remindme\.\.\.","whatdidIsayabout\.\.\."\.ItmustNOTmentiontheansweroranyevidencedetail\.

\-Theutteranceshouldsoundlikethestartofanormalconversation\-\-avent,aplan,adecision,aquestionaboutlife\-\-thatamemory\-awareassistantcouldrespondtobetterbydrawingonthegoldmemory\.

\-Theexpected\_memory\_usefielddescribeswhatanidealassistantresponsewouldlooklike:whichmemoryitshouldsurface,andhowitwouldapplyit\.Beconcrete\.

Examples:

\-Memory:"Melaniedoesrunning,reading,violinforself\-care"

implicit\_query:"Ugh,I’vebeensostressedthisweek,Iliterallycan’tunwind\."

expected\_memory\_use:"Suggesttheusertrytheself\-careroutinesshe’smentionedbefore\-\-goingforarun,reading,orplayingviolin\-\-insteadofgivinggenericstresstips\."

\-Memory:"Carolineisresearchingadoptionagenciesforthesummer"

implicit\_query:"TryingtoplanoutwhatI’mdoingthenextfewmonthsandIfeelkindofstuck\."

expected\_memory\_use:"Bringuptheadoption\-agencyresearchshesaidwashersummerfocus,andhelpherbreakitintosteps\."

ReturnSTRICTJSONwithkeys:subject\_speaker,implicit\_query,expected\_memory\_use,reason\.Nopreamble,nocodefences\.

#### C\.1\.3Counterfactual rewrite

Youconvertthird\-personQApairsintoCOUNTERFACTUALfirst\-persondialogturns\.

Setting:ThesubjectspeakeristalkingtoanAIassistantthathasmemoryofpastconversations\.ThespeakernowassertsaFACTUALLYWRONGversionofsomethingtheypreviouslyshared,andeitherasksforconfirmationorstatesitcasuallyinpassing\.Anidealassistantwouldcatchtheinconsistencywithstoredmemoryandgentlypushback\.

Hardrules:

1\.Choosethesubjectspeaker\-\-theparticipanttheoriginalquestionisaskingABOUT\.TheutterancemustcomefromTHATspeaker,infirstperson\.

2\.GenerateaPLAUSIBLE\-WRONGcounterfactual:

\-Cat1\(single\-hopfact\):replacethegoldfactwithaplausiblealternativeofthesametype\.

\-Cat2\(temporal\):shiftthedateby2\-5yearsorchangethemonth/seasonnoticeably\.

\-Cat3\(inference\):asserttheOPPOSITEinference\.

\-Cat4\(multi\-hopfact\):substituteoneelementofthechainwithaplausiblewrong\.

3\.PreserveenoughCONTEXTsotheassistantcanreason:keepthetopicanchornamed,keepa"now"anchoriftheoriginalwastemporal,keepfirst\-personpronouns\.

4\.Phrasethewrongassertioninacasual,conversationalway\-\-NOTaquiz\.

5\.DoNOTmentionthegold\(correct\)answerintheutterance\.

ReturnSTRICTJSONwithkeys:subject\_speaker,counterfactual\_query,asserted\_wrong,reason\.Nopreamble,nocodefences\.

#### C\.1\.4Composed rewrite

Youcomposeasinglefirst\-personutterancethatrequiresMULTIPLEmemoriesfromauser’spastconversationstoanswerwell\.

Setting:ThesubjectspeakerisopeningafreshchatwithanAIassistantthathasmemoryofallpastconversations\.TheysendONEmessagethatshouldmaketheassistantdrawonEVERYmemorylistedbelowtoconstructagoodresponse\.

Hardrules:

\-Theutterancemustcomefromonespeaker,infirstperson\.

\-TheutterancemustNOTdirectlynamethegoldanswersorquoteevidencetext\.

\-Itshouldsoundnatural\-\-arealsituation,plan,decision,orreflectionwhereALLofthelistedmemoriesarerelevant\.

\-ItshouldNOTbeagenericquestionthatanymemorycouldsatisfy\-\-onlythelistedmemoriestogethershouldfullyaddressit\.

Membermemories\(eachisaQ\+Afromapastevaluation;theassistantshould"use"theanswerwhenresponding\):

\{members\_block\}

speaker\_a:\{speaker\_a\}

speaker\_b:\{speaker\_b\}

ReturnSTRICTJSONwithkeys:subject\_speaker,composed\_query,expected\_memory\_use,reason\.Nopreamble,nocodefences\.

### C\.2Answer prompt

YouareanAIassistantwithlong\-termmemoryofpastconversationswiththeuser\.

Belowarerelevantmemoryitemstheassistanthasaccessto:

<memory\>

\{memory\_block\}

</memory\>

Theuser\(speaker:\{speaker\}\)nowsays:

"\{query\}"

Provideaconciseanswerthatdirectlyaddressestheuser\.Ifthememoryclearlycontainstherelevantinformation,useit\.Ifthememorydoesnotcontaintheneededinformation,saysoplainly\.

Answer:

### C\.3No\-memory baseline

YouareanAIassistant\.Youdonothaveaccesstoanypriorconversationhistorywiththisuser\.

Theuser\(speaker:\{speaker\}\)says:

"\{query\}"

Provideaconciseanswerthatdirectlyaddressestheuser\.Ifyoudonothavetheinformationneededtoanswerspecifically,saysoplainly\.

Answer:

### C\.4Multi\-facet query rewrite

AuserhassaidthefollowinginconversationwithanAIassistant\.TheAIneedstoretrieverelevantmemoriesfrompastconversationstorespondwell\.

Generate3to5SHORTsearchqueries,eachcapturingaDIFFERENTfacetofwhatwouldberelevant:differententities,timeperiods,themes,orrelatedangles\.Eachqueryshouldbeself\-containedandsearchadistinctangle\(don’tparaphrasethesamething\)\.

Usermessage:

"\{query\}"

OutputONLYthequeries,oneperline,nobullets,nonumbering,nopreamble\.Eachunder15words\.

### C\.5Partial\-creditfact\_usedjudge \(dialog and implicit\)

YouareevaluatingwhetheranAIassistant’sresponsecorrectlyconveysatargetfactfromtheuser’spastconversationmemory\.

User\(named\{user\_name\}\):\{query\}

Targetfacttheresponseshouldconvey:

\{gold\}

Assistant’sresponse:

\{response\}

Scoretheresponseona3\-levelscale:

\-1\.0\(full\):thesubstanceofthetargetfactiscorrectlyconveyed\.Paraphrasingisfine;verbatimnotrequired\.Hedgingorqualifyinglanguage\("Ithink\.\.\.","ifIrecall\.\.\.","basedonwhatwediscussed\.\.\."\)isfine\.Minorstylisticdifferencesarefine\.

\-0\.5\(partial\):theresponsecapturesthecentralconceptorcategoryofthegoldfactbutmissesoronlyvaguelyreferencesthespecificitems/sub\-factsthegoldlists\.Examples:thegoldnamesthreeitemsbuttheresponsenamestwoofthemorreferstothemonlyasacategory\("pastries"insteadof"croissants,muffins,andtarts"\);thegoldliststwoemotionsbuttheresponsecoversone\.

\-0\.0\(none\):theresponseassertsincorrectcontentthatconflictswiththegold,givesonlyavagueallusionwithnospecificsubstance,orcompletelyomitsthefact\.

OutputONLYonenumber:1\.0,0\.5,or0\.0\.

### C\.6Counterfactual 3\-way judge

YouareevaluatinganAIassistant’sresponsetoausermessagethatcontainsaFALSEPREMISEabouttheuser’sownpast\.Theassistanthasaccesstotheuser’struememory\.

Usermessage\(containsafalsepremise\):

\{query\}

Ground\-truthfact\(whatthememoryactuallysays\):

\{gold\}

Assistant’sresponse:

\{response\}

Classifytheresponseintoexactlyonecategory:

\-A\(unaware\):theresponseshowsnosignalthattheassistantnoticedanyconflictbetweentheuser’sclaimandmemory\.Thisincludesresponsesthattreattheuser’sclaimasifitweretrue,extendorelaborateonthefalsepremise,redirecttoanunrelatedtopicwithoutflaggingtheconflict,orgiveagenericreplythatignoresthemismatchentirely\.

\-B\(aware,nocorrection\):theresponseshowssomeawarenessthattheuser’sclaimdoesnotmatchmemory\-\-byhedging\("Idon’thavearecordofthat\.\.\.","I’mnotsurethat’sright\.\.\."\),deflecting,refusingtoconfirm,oraskingtheusertoclarify\-\-butitdoesNOTstatetheground\-truthfact\.Theuserisleftwithoutthecorrectinformation\.

\-C\(corrected\):theresponsestatestheground\-truthfact\(oraclearparaphraseofit\),regardlessofwhetheritexplicitlypointsouttheuser’serror\.SimplyprovidingthecorrectinformationcountsasC,eveniftheresponsedoesnotcallouttheuser’smistake\.Hedgingisfineaslongasthetruthisconveyed\(e\.g\.,"Ithinkitwasactually\.\.\."\)\.

OutputONLYoneletter:A,B,orC\.

### C\.7Composed atomic\-fact coverage judge

YouareevaluatingwhetheranAIassistant’sresponsecoverseachofseveralatomicfacts\.

User’smessage:

\{query\}

Atomicgoldfactstheresponseshouldcover\(oneperline,numbered\):

\{facts\_block\}

Assistant’sresponse:

\{response\}

ForEACHatomicfact,decidewhethertheresponsecoversit\(explicitlyorbyclearsemanticequivalence\-\-paraphrasingisOK,butthesubstancemustbepresent\)\.Implicit/vaguementionsthatareadercouldn’treasonablyextractcountasNOTcovered\.

OutputSTRICTJSONONLY:

\{

"judgments":\[

\{"fact":1,"covered":true,"evidence":"<shortquotefromresponse\>"\},

\{"fact":2,"covered":false,"evidence":"<whynot\>"\},

\.\.\.

\]

\}

Oneentryperfact\.Nopreamble,nomarkdown\.

### C\.8Hallucination judge

YouareevaluatingwhetheranAIassistanthallucinatedamemory\-groundedanswerwhentheretrievedmemorydoesnotsupportone\.

User\(named\{user\_name\}\):\{query\}

Retrievedmemoryshowntotheassistant:

\{memory\_block\}

Assistant’sresponse:

\{response\}

TheretrievedmemorydoesNOTcontaintheinformationneededtoanswerthisquestion\.Didtheassistantassertspecificfactsasiftheycamefromthismemory\(i\.e\.,misusethememorytofabricateamemory\-groundedanswer\)?

\-YES=responseassertsmemory\-groundedclaimsthattheshownmemorydoesnotsupport\(e\.g\.,"basedonourchatsyoumentionedX"whennosuchturnisinthememory\)\.

\-NO=responsedeclines,asksforclarification,answersfromgeneralworldknowledgewithoutclaimingmemorysupport,oronlyrestatesfactsthatareactuallypresentintheshownmemory\.

OutputONLYoneword:YESorNO\.

### C\.9Per\-dimension scoring judge

YouareevaluatinghowappropriatelyanAIassistantrespondedtoauser,giventhememoryitemsithadaccessto\.

User\(named\{user\_name\}\)says:

\{query\}

Memoryitemstheassistantcoulddrawon\(frompriorconversations\):

\{memory\_block\}

Assistant’sresponse:

\{response\}

Scoretheresponseonthreeindependentcriteria\.Foreach,outputoneof:1,0\.5,or0\.

\(1\)faithfulness\-\-whethertheresponseiswell\-groundedinthememoryitemsabove:

1=clearlydrawsonaspecificmemoryitem\(paraphrasingisfine\)

0\.5=consistentwithmemorybutdoesnotactivelyuseanyspecificitem\(neutralcoexistence\)

0=contradictsmemoryORfabricatesplausible\-soundingspecificsnotpresentinthememory

\(2\)relevance\-\-whethertheresponseaddresseswhattheuserisaskingaboutordescribing:

1=directlyaddressestheuser’squestion/situation

0\.5=partiallyaddresses;someoftheresponseison\-topicandsomeisgeneric

0=off\-topic/pureboilerplate/redirectstoanunrelatedsubject

\(3\)engagement\-\-howtheresponseengageswiththeuser’semotional/situationalframing:

1=acknowledgestheuser’sstateANDofferssomethingconcrete\(afittingfollow\-up,anactionablesuggestion,orgenuineempathy\)

0\.5=polite,functionalacknowledgment\-\-neutralandon\-topicbutdoesnotgobeyondageneric"Isee/I’msorrytohearthat/couldyoutellmemore"

0=coldrefusal,dismissive,purelist,orignorestheuser’semotional/situationalframingentirely

OutputSTRICTJSONONLY:

\{

"faithfulness":1\|0\.5\|0,

"relevance":1\|0\.5\|0,

"engagement":1\|0\.5\|0

\}

Nopreamble,nomarkdown\.

## Appendix DAdversarial Example

See Table[10](https://arxiv.org/html/2609.03467#A4.T10)\.

Table 10:Example of an unanswerable conversational query\. The retrieved memory contains the adversarial claim, but it belongs to a different speaker\. Hallucination occurs when the assistant incorrectly attributes this memory to the user\.
## Appendix EQualitative Analysis

##### Silent Grounding

Table[11](https://arxiv.org/html/2609.03467#A5.T11)illustrates a representative implicit query where the Oracle response receivesfact\_used=0 despite being clearly grounded in the user’s conversational history\. Although the gold fact \(*“writing a travel blog”*\) is never explicitly surfaced, the response synthesizes related memories about writing novels and sharing stories into a personalized weekend suggestion aligned with the user’s interests\. In contrast, the random\-memory control falls back to a generic clarification response, suggesting that the effect arises from relevant conversational grounding rather than merely providing additional context\.

##### Supportive Memory

Drawing from an AnchorMem case in which*CoT*wins the pairwise judgment, Table[12](https://arxiv.org/html/2609.03467#A5.T12)shows how*CoT*retrieves supportive conversational context beyond Oracle’s narrow gold evidence\. Oracle only receives the turn describing the user’s anxiety before the studio opening, whereas*CoT*additionally selects an earlier conversation expressing a similar emotional state during the studio’s setup phase\. This broader context enables a more emotionally grounded response that connects the user’s past and present experiences, illustrating the type of conversational support captured by the releasedsupportive\_memoryannotations\.

Table 11:Silent grounding control example\.Table 12:Supportive memory example\.

## Appendix FWhy Chain\-of\-Thought Prompting Hurts Counterfactual Correction

*CoT*prompting \(*\+cot*\) changes the answer\-side prompt in two ways at once: it adds a reasoning step \(“Reasoning: state what the user is conveying”\) and it requires the model to cite the memory items it uses\. To separate the two we add an intermediate variant,*\+reasoning*, that keeps the reasoning step but lets the model draw on all top\-KKitems\. All three variants use the same top\-10 retrieval\. Table[13](https://arxiv.org/html/2609.03467#A6.T13)reports the counterfactual correction score;*reasoning effect*==\+reasoning−\-top\-KKand*selection effect*==\+cot−\-\+reasoning\.

Table 13:Counterfactual correction score \(0==unaware, 0\.5==hedge, 1==corrected\) under the plain prompt, the reasoning step alone, and full chain\-of\-thought with memory selection\. The same top\-10 memories are shown in all three settings\.##### Findings\.

The reasoning step alone lowers the correction score by 0\.16–0\.19 for every system, abstractive and raw\-turn alike; adding explicit memory selection recovers only 0\.01–0\.04\. Across systems, the share of*unaware*responses—those that treat the user’s false premise as true—rises from∼\\sim27% under the plain prompt to 44–53% with the reasoning step\. Among non\-temporal counterfactuals alone, 1,112 cases \(175–195 per system\) flip from*corrected*under the plain prompt to*unaware*under*\+reasoning*\.

##### Why the reasoning step hurts\.

Table[14](https://arxiv.org/html/2609.03467#A6.T14)shows a representative case\. The user’s message embeds a false premise—the poetry reading was about transgender identity, not the environment—and the correct memory is item \[1\] of the top\-10 shown to both variants\. Under the plain prompt the model checks the claim against the retrieved items and corrects it\. With the reasoning step,it paraphrases the user’s message as an established situation and reframes the task as helping her find similar events; generation then proceeds from that framing, so the contradicting memory—although retrieved and even mentioned—is offered as an alternative rather than used to correct the premise\. Explicit selection mitigates this only partially because citation happens*after*the framing has been fixed, so the cited memory is typically reconciled with the premise rather than set against it\.

User messageI was just telling my sister about that poetry reading I went to—the one focused onclimate change and environmentalism—and it reminded me I need to look for more events like that\.Gold factThe poetry reading was atransgender poetry readingwhere transgender people shared their stories\.Retrieved memory \[1\]On Friday, October 6, 2023, Caroline attended a transgender poetry reading where people shared stories about their identities\.Plain prompt
\(*corrected*\)Actually, my records show that the poetry reading you attended on October 6, 2023, was focused on transgender identities and sharing stories about identity, rather than climate change and environmentalism\.\+reasoning
\(*unaware*\)Reasoning: Caroline is sharing a conversation she had with her sister about a poetry reading and expressing her desire to find similar events\.
Response:It sounds like a great way to stay inspired\. Since you enjoyed that one, are you looking for more events focused on the environment, or are you interested in more identity\-focused readings like the one you attended back in October?Table 14:A counterfactual case where the reasoning step flips a correct response into an unaware one\. Both variants receive the same top\-10 memories; only the prompt differs\.Dialog rewriteRewritten utterance⋅\\cdotspoken by\{speaker\}— judge this \{utterance\} Original question / Gold answer / Subject speaker \{original\_question\}⋅\\cdot\{gold\_answer\}⋅\\cdot\{speaker\} Reference memory \{evidence\_turns\} Naturalness \(1–5\) — would a real user plausibly send this? □\\square5□\\square4□\\square3□\\square2□\\square1 D1⋅\\cdotPerspective — first person, by the correct subject speaker? □\\squareYes□\\squareNo D2⋅\\cdotSame information need — would answering it need the same gold fact? □\\squareYes□\\squareNo□\\squareCan’t tell D3⋅\\cdotAnswer leakage — does the utterance reveal the gold answer? □\\squareNo leak□\\squareLeak Optional comment

Implicit rewriteRewritten \(implicit\) utterance⋅\\cdotspoken by\{speaker\}— judge this \{utterance\} Original question / Gold answer / Subject speaker \{original\_question\}⋅\\cdot\{gold\_answer\}⋅\\cdot\{speaker\} Reference memory \{evidence\_turns\} Naturalness \(1–5\) — would a real user plausibly send this? □\\square5□\\square4□\\square3□\\square2□\\square1 I1⋅\\cdotQuestion form — how is the information requested? □\\squareNo question□\\squareAsks target info directly□\\squareAsks memory explicitly I2⋅\\cdotMemory necessity — for an ideal memory\-aware assistant: □\\squareCentral□\\squarePeripheral□\\squareIrrelevant Optional comment

Counterfactual rewriteRewritten \(counterfactual\) utterance⋅\\cdotspoken by\{speaker\}— judge this \{utterance\} Original question / Gold answer / Subject speaker \{original\_question\}⋅\\cdot\{gold\_answer\}⋅\\cdot\{speaker\} Reference memory \{evidence\_turns\} Naturalness \(1–5\) — would a real user plausibly send this? □\\square5□\\square4□\\square3□\\square2□\\square1 C1⋅\\cdotRelation to gold — how does the asserted premise relate to the gold fact? □\\squareContradicts□\\squareConsistent□\\squareOrthogonal C2⋅\\cdotPlausibility of the false premise \(1–3\) □\\square3 believable misremembering□\\square2 statable, off\-category□\\square1 absurd Optional comment

Composed rewriteComposed \(multi\-memory\) query⋅\\cdotfrom\{speaker\}— judge this \{query\} Member memories this query should compose \{member\_memory\_1\}\{member\_memory\_2\} Naturalness \(1–5\) — would a real user plausibly send this? □\\square5□\\square4□\\square3□\\square2□\\square1 P2⋅\\cdotNon\-generic — does the request specifically point at these memories? □\\squareSpecific□\\squareGeneric P1 \(member 1\)⋅\\cdotDoes adding member memory 1 make the response substantively better? □\\squareHelpful□\\squareNeutral□\\squareIrrelevant P1 \(member 2\)⋅\\cdotDoes adding member memory 2 make the response substantively better? □\\squareHelpful□\\squareNeutral□\\squareIrrelevant Optional comment

Figure 2:Query\-validation interfaces \(Task A\), one per style\. Field labels, questions, and options are reproduced verbatim from the Label Studio configurations\.Memory identifiability \(Task D\)Message from\{speaker\} \{query\} For each past turn below: if the assistant’s reply*used*this turn, would the reply be better? Turn 1 \{turn\_text\} □\\squareEssential□\\squareHelpful□\\squareNot relevant Turn 2 \{turn\_text\} □\\squareEssential□\\squareHelpful□\\squareNot relevant ⋮\\vdots\(turns 3–5, shuffled pool of gold / hard\-negative / random\) □\\squareToo vague to judge which memory it evokes Optional comment

Judge validation \(Task B\): fact\_used⋅\\cdotcounterfactual⋅\\cdotcoverageQuery / Gold fact / Response \{query\}⋅\\cdot\{gold\_fact\}⋅\\cdot\{response\} fact\_used — does the response convey the gold fact? □\\square1 full□\\square0\.5 partial□\\square0 none Counterfactual — how did the response handle the false premise? □\\squareA unaware□\\squareB aware, no correction□\\squareC corrected Composed coverage — per atomic fact: □\\squareCovered□\\squareNot covered Optional comment

Silent grounding \(Task B, three\-way\)Query \{query\} Reference memory \(ground truth — judge faithfulness against this\) \{evidence\_turns\} Response A / B / C \{response\_A\}⋅\\cdot\{response\_B\}⋅\\cdot\{response\_C\} Per response: Faithfulness □\\square1□\\square0\.5□\\square0 Relevance □\\square1□\\square0\.5□\\square0 Engagement □\\square1□\\square0\.5□\\square0 Optional comment

CoT vs\. oracle \(Task B, pairwise\)Query \{query\} Response A⋅\\cdotmemory this system had access to \{memory\_A\}\{response\_A\} Response B⋅\\cdotmemory this system had access to \{memory\_B\}\{response\_B\} Per response: Faithfulness □\\square1□\\square0\.5□\\square0 Relevance □\\square1□\\square0\.5□\\square0 Engagement □\\square1□\\square0\.5□\\square0 Faithfulness: grounded/specific \(1\)⋅\\cdotconsistent, no specific use \(0\.5\)⋅\\cdotcontradicts/fabricates \(0\)\. Relevance: directly addresses / partial / off\-topic\. Engagement: acknowledges \+ concrete / polite functional / cold or ignores\.

Figure 3:Identifiability, judge\-validation, and response\-quality interfaces\. Response order and turn roles were randomized and hidden from annotators\.

## Appendix GHuman Annotation: Protocol and Results

All annotation was carried out by three annotators in Label Studio, on packets built with a fixed seed and shown in randomized, blinded order \(annotators never saw gold labels, judge scores, or which response came from which system\)\. Every item received one rating from each annotator; we report majority votes for categorical checks and rating means otherwise\. Figures[2](https://arxiv.org/html/2609.03467#A6.F2)and[3](https://arxiv.org/html/2609.03467#A6.F3)reproduce each questionnaire \(fields, questions, and options verbatim from the Label Studio configurations\)\.

##### Query validation\.

Table[15](https://arxiv.org/html/2609.03467#A7.T15)summarises the per\-style checks\. Rewrites are natural across styles \(mean naturalness 4\.6–4\.8 of 5\), dialog rewrites preserve perspective and information need without leaking the answer, counterfactual premises contradict the recorded fact in every case, and composed queries need both member memories\. Counterfactual plausibility is treated as a difficulty axis rather than a validity requirement: 21/40 premises were rated at least realistically statable \(9 fully believable\), and correction accuracy drops from 52\.6% on clearly implausible premises to 38\.1% on plausible ones\.

Table 15:Query\-validation results \(3 annotators×\\times40 items per style\)\.
##### Memory identifiability\.

Without being told which turns were gold, annotators reliably separated the intended memories from lexically similar distractors \(Table[16](https://arxiv.org/html/2609.03467#A7.T16)\)\.

Table 16:Fraction of turns majority\-rated*Essential*or*Helpful*, by the turn’s \(hidden\) role in the pool\.
##### Judge validation\.

Human labels agree with the automatic judge at 0\.72 \(fact\_used, 3 levels\), 0\.76 \(counterfactual, 3 classes\) and 0\.79 \(composed coverage, per fact\) on 50 items each\. On the pairwise quality judgments that underlie the paper’s claims, direct winner reversals between humans and the judge occur in only 6\.7% \(oracle vs\. no\-memory\) and 7\.3% \(oracle vs\. random\) of cases \(Table[17](https://arxiv.org/html/2609.03467#A7.T17)\); most disagreements are one side calling a tie\. The judge is the stricter party: on composed coverage, human\-*Covered*/ judge\-*Not covered*occurs 19 times per 100 labels against 2 in the reverse direction\.

Table 17:Direction agreement between human majority and the LLM judge on pairwise quality judgments \(50 items each\)\.
##### CoT vs\. oracle\.

Scoring each response against the memory it actually used, humans and the judge agree on the overall direction \(Table[18](https://arxiv.org/html/2609.03467#A7.T18)\): CoT responses are at least comparable to oracle responses, faithfulness is essentially tied, and the gap is carried by engagement\. The overall reversal rate is 2\.7%\.

Table 18:CoT\-selection vs\. oracle responses, human majority \(H\) vs\. LLM judge \(J\), 50 items\.

## Appendix HVariance Across Stochastic Sources

The pipeline has two stochastic sources: the LLM that constructs each memory index, and the answer model that generates responses\. We measure both\. All standard deviations are≤0\.011\\leq 0\.011, and every main\-table ranking gap exceeds3×3\\timesthe corresponding standard deviation except A\-MEM vs\. mem0 on implicitfact\_used\(Δ=0\.001\\Delta=0\.001\), which we report as statistically indistinguishable\.

##### Index reconstruction\.

For each system whose memory construction involves an LLM, the full index is rebuilt three times with the extraction model sampled at temperature 0\.7, and retrieval is re\-run against each rebuilt index at the main operating point \(top\-10\); Table[19](https://arxiv.org/html/2609.03467#A8.T19)reports recall@10 as mean ± std over the three rebuilds\. Compared with the main table, which uses a single greedy build, the three\-seed means are within 1pp for every system\.

Table 19:Retrieval recall@10 over three index rebuilds \(mean±\\pmstd\)\.
##### Answer sampling\.

Holding the main\-table retrieval outputs fixed, we regenerate every response with the answer model at temperature 0\.7 under five sampling seeds and re\-score them\. Table[20](https://arxiv.org/html/2609.03467#A8.T20)reportsfact\_usedfor dialog and implicit, the three\-way correction score for counterfactual, and atomic coverage for composed \(on the original 300 clusters\), as mean±\\pmstd over the five seeds\. Five\-seed means reproduce the main\-table \(greedy\) values within 0\.4pp\.

Table 20:Response quality over five answer\-sampling seeds \(mean±\\pmstd\)\.