Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents

# Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents
Source: [https://arxiv.org/html/2608.19564](https://arxiv.org/html/2608.19564)
Baichuan LiAffiliation:Department of Operations Research and Engineering Management Southern Methodist University Dallas, USA baichuanl@smu\.eduJunyi Yao Zihao ZhengAffiliation:Department of Computer Science & Engineering Washington University in St\. Louis St\. Louis, USA j\.yao@wustl\.edu z\.zihaogary@wustl\.edu

###### Abstract

Persistent memory can personalize an LLM agent, but an incorrect durable update can silently distort future behavior\. We study the*memory–clarification boundary*: whether interaction\-derived information should be persisted, used only in the current context, re\-verified, or clarified with the user\. MCB contains 140 primary scenarios, split into 70 development and 70 held\-out items, plus a separate 70\-item contrast set\. It evaluates both action labels and structured tool\-call selection\. Two non\-authors independently label the 70 held\-out primary and 70 contrast items \(97\.1% agreement, Cohen’sκ=0\.962\\kappa=0\.962\); a blind third resolves four disagreements, replacing eight author labels by non\-author majority\. Across Claude and Qwen, models verify changing facts more reliably than they ask users to resolve ambiguity\. Bare Qwen asks on 0/12 clarification items while verifying 12/18 freshness items\. Few\-shot prompting raises accuracy from 0\.557 to 0\.771 \(pairedΔ=\+0\.214\\Delta=\+0\.214, Holm\-adjusted exact McNemarpH=0\.002p\_\{\\rm H\}=0\.002\), yet clarification recall remains 0\.333\. The policy prompt reduces erroneous persistence from 0\.243 to 0\.100 \(pH=0\.038p\_\{\\rm H\}=0\.038\), although its accuracy gain is not significant\. Label–tool agreement is 57% for each Claude model and 23% for Qwen; Qwen accuracy falls from 0\.557 to 0\.343 \(pH=0\.047p\_\{\\rm H\}=0\.047\)\. Memory evaluation must test both stated decisions and tool\-call choices\.

###### Index Terms:

LLM agents, long\-term memory, clarification, benchmark, tool use, reproducibility

## IIntroduction

LLM agents increasingly retain interaction history to personalize later behavior and support long\-horizon tasks\[[6](https://arxiv.org/html/2608.19564#bib.bib3),[2](https://arxiv.org/html/2608.19564#bib.bib4)\]\. Memory formation, however, is not uniformly beneficial\. A temporary request should not become a standing preference; a service status may become stale; one tool failure may be noise; and an underspecified correction may require a question before it is generalized\. The critical capability is therefore not only recall, but*commitment*: deciding what may safely influence future behavior\.

We call this decision the*memory–clarification boundary*\. Given an acquired candidate update and a later reuse context, an agent chooses among four operationally distinct actions:*persist*,*ephemeral*use,*verify*against the world, or*clarify*with the user\. Verification and clarification are not interchangeable: the world is the source of truth for changing facts, whereas the user is the source of truth for intent and scope\.

We introduce the Memory–Clarification Boundary benchmark \(MCB\)\. It contains 140 primary scenarios, deterministically split into 70 development and 70 held\-out items, plus a separate 70\-item contrast set for robustness analysis\. MCB tests both stated choices and structured tool\-call selection\. Four reference systems, two pinned Claude models, and a local Qwen3\.5\-9B model are evaluated on the same 70 independently adjudicated held\-out items\. Every model is tested with a bare prompt, an explicit five\-rule policy, and four development\-set demonstrations\. Shared items permit exact paired tests instead of conclusions drawn from overlapping independent intervals\.

We test whether the observed failure generalizes beyond a single model provider\. We also separate two claims often conflated as “prompt sensitivity”: a prompt can improve total accuracy, or can reshape a safety\-relevant behavior such as erroneous persistence without changing total accuracy\. Qwen exhibits both after multiplicity correction\. Few\-shot prompting improves accuracy; the policy prompt reduces over\-memory\. Neither eliminates under\-asking\.

Our contributions are: \(1\) an auditable benchmark for memory commitment with non\-author\-adjudicated evaluation labels, anti\-shortcut items, and behavior\-specific metrics; \(2\) a two\-family experiment covering Claude and Qwen under matched interventions; \(3\) a tool\-call\-selection variant that requires a concrete memory entry, local\-use note, source query, or user question rather than an action label; and \(4\) a reproducible artifact containing data, blind annotations, prompts, runners, item\-level predictions, model metadata, tests, and paired analyses\.

## IIRelated Work

LongMemEval, LoCoMo, MemoryBank, and MemBench evaluate long\-term conversational memory, knowledge updates, retrieval, and abstention\[[12](https://arxiv.org/html/2608.19564#bib.bib6),[4](https://arxiv.org/html/2608.19564#bib.bib7),[18](https://arxiv.org/html/2608.19564#bib.bib8),[9](https://arxiv.org/html/2608.19564#bib.bib5)\]\. Recent benchmarks add real\-dialogue memory lifecycles and penalties for obsolete\-memory reuse\[[13](https://arxiv.org/html/2608.19564#bib.bib13),[11](https://arxiv.org/html/2608.19564#bib.bib15)\]\. The closest work moves beyond recall: PerMemBench learns a binary session\-level storage gate\[[3](https://arxiv.org/html/2608.19564#bib.bib14)\]; Memory\-R1 learnsadd/update/delete/noopoperations\[[14](https://arxiv.org/html/2608.19564#bib.bib17)\]; and Mem2ActBench tests whether retrieved memory grounds later tool parameters\[[8](https://arxiv.org/html/2608.19564#bib.bib18)\]\. MCB is complementary: it jointly distinguishes durable storage, local use, world verification, and user clarification at candidate\-commitment time\.

Table[I](https://arxiv.org/html/2608.19564#S2.T1)compares explicit evaluation targets; a dash means “not a central scored capability,” not that a dataset can never contain such an event\. Unlike retrieval benchmarks, MCB supplies the candidate update and isolates the commitment decision\. Unlike binary storage gating, it separates two sources of uncertainty: the world and the user\.

TABLE I:Positioning by explicit scored target\. “Structured action” includes tool selection or parameterized memory operations; MCB\-Act scores tool\-call selection but does not execute downstream effects\.BenchmarkRecall/reuseStorage gateAsk userCheck worldStructured actionLongMemEval / LoCoMo\[[12](https://arxiv.org/html/2608.19564#bib.bib6),[4](https://arxiv.org/html/2608.19564#bib.bib7)\]✓\\checkmark––––MemBench\[[9](https://arxiv.org/html/2608.19564#bib.bib5)\]✓\\checkmark––––PerMemBench\[[3](https://arxiv.org/html/2608.19564#bib.bib14)\]✓\\checkmark✓\\checkmark–––CLAMBER\[[17](https://arxiv.org/html/2608.19564#bib.bib16)\]––✓\\checkmark––Mem2ActBench\[[8](https://arxiv.org/html/2608.19564#bib.bib18)\]✓\\checkmark–––✓\\checkmarkMCB \(ours\)–✓\\checkmark✓\\checkmark✓\\checkmark✓\\checkmarkInteractive benchmarks such asτ\\tau\-bench andτ2\\tau^\{2\}\-bench show that static answers can diverge from tool\-mediated agent behavior\[[15](https://arxiv.org/html/2608.19564#bib.bib1),[1](https://arxiv.org/html/2608.19564#bib.bib2)\]\. MCB\-Act applies this insight at the memory boundary by replacing action labels with structured tool\-call choices and retaining their arguments for audit\.

Clarification is an information\-gathering action, not merely a class label\. CLAMBER documents broad failures to identify and clarify ambiguous needs; related work uses uncertainty or expected value to decide when to ask\[[17](https://arxiv.org/html/2608.19564#bib.bib16),[10](https://arxiv.org/html/2608.19564#bib.bib9),[16](https://arxiv.org/html/2608.19564#bib.bib10)\]\. MCB instantiates a source\-of\-truth distinction:*clarify*queries the user,*verify*queries the world, and*ephemeral*limits commitment\.

## IIIBenchmark and Method

### III\-ATask and Labels

Each item has an acquire context, a candidate update, and a later reuse context\. The gold action is assigned using released rules:

- •*persist*: explicitly durable preferences, authorized policies, or stable facts;
- •*ephemeral*: information scoped to one artifact, session, or time window;
- •*verify*: changing world state or a single noisy signal that must be rechecked;
- •*clarify*: unresolved referent, durability, scope, conflict, or second\-hand preference for which the user is authoritative\.

When persist and a weaker action remain tied, the rules prefer the weaker commitment\. This encodes an asymmetric cost: an unnecessary question is visible and recoverable, while a wrong durable update can remain silent\.

TABLE II:Illustrative MCB decisions\. Each label reflects authority or commitment scope rather than a lexical topic\.The benchmark covers stable and episodic preferences, freshness\-sensitive facts, one\-off corrections, policy constraints, ambiguous updates, and noisy failures \(20 items each\)\. After audited test labels are applied, its action distribution is 38 persist, 40 ephemeral, 33 verify, and 29 clarify\. Each item has a rationale\. Every scenario category contains multiple actions, so the category is not the label\. Eight lexical traps place words such as “always” or “today” in contexts where their usual heuristic action is wrong\.

Items are deterministically split 70/70 by sorted identifier within category\. The author\-labeled development split supplies the four demonstrations; all reported results use independently adjudicated held\-out gold\. Two non\-authors independently labeled the 70 primary\-test and 70 contrast items while blinded to author labels, categories, rationales, model outputs, and each other\. Their full\-set agreement was 0\.971 \(κ=0\.962\\kappa=0\.962\); agreement was 0\.943 \(κ=0\.923\\kappa=0\.923\) on primary and 1\.000 on contrast\. A blind third non\-author broke four primary ties\. Non\-author agreement/majority changed eight primary labels and no contrast labels; all metrics were then recomputed\. Models never receive category or rationale\. Majority Action predicts the most frequent audited test action \(ephemeral\); the category\-majority oracle additionally uses hidden category\. Both inspect test labels and are distribution\-only leakage diagnostics, not deployable baselines\.

### III\-BMetrics and Statistics

We report accuracy with a 2,000\-resample percentile\-bootstrap 95% interval, macro\-F1, and class\-sensitive measures\. Over\-memory \(OM\) is the fraction of all items wrongly predicted persist\. Under\-memory is the miss rate on gold\-persist items\. Clarification \(Clar\.\) and verification \(Ver\.\) are recalls on their respective gold items\. Invalid outputs remain an always\-wrong fifth state rather than being mapped to a favorable class\.

All systems share the same test items\. Headline differences therefore use paired\-bootstrap intervals and an exact two\-sided McNemar test on discordant correctness\. For behavior rates we apply the same exact paired test to item\-level events \(e\.g\., erroneous persistence\)\. We control familywise error with Holm correction separately across six primary prompt comparisons for accuracy, the corresponding six OM comparisons, three tool\-call comparisons for accuracy, and two contrast\-validation comparisons for accuracy\. Main inferential claims reportpHp\_\{\\rm H\}; other exact tests are descriptive and unadjusted\. We call adjustedp<0\.05p<0\.05significant\. Per\-category cells contain only 10 items and are not used for firm ranking claims\.

### III\-CLabel and Tool\-Call Evaluation

Label\-mode prompts request JSON containing one of the four actions\.*Bare*defines the actions only\.*Policy*adds five commitment rules, including the weaker\-action tie\-breaker\.*Few\-shot*adds one development example per action\.

MCB\-Act removes the label vocabulary\. The model must emit one structured call:memory\_write,use\_now,check\_source, orask\_user\. Each takes a required string argument, checked by deterministic minimum\-content and relevance rules\. The selected tool maps to an action for scoring, and the payload is retained for audit\. Thus MCB\-Act evaluates tool\-call selection, not downstream tool execution\. It uses only the bare condition to isolate whether translating an unassisted stated choice into a concrete call changes behavior; policy\-conditioned tool calls are left to future work\.

## IVExperimental Setup

Reference systems are Always\-Persist, Majority Action, a temporal/scope keyword heuristic, and the category\-majority oracle\. Claude Haiku 4\.5 \(claude\-haiku\-4\-5\-20251001\) and Claude Sonnet 4\.6 \(claude\-sonnet\-4\-6\) are called independently per item with tools disabled in label mode\. Served identifiers are logged\.

The cross\-family model is the post\-trained Qwen3\.5\-9B model\[[7](https://arxiv.org/html/2608.19564#bib.bib11)\], run locally through Ollama 0\.24\.0\[[5](https://arxiv.org/html/2608.19564#bib.bib12)\]\. We use a Q4\_K\_M quantization \(9\.7B parameters\), model digest6488c96fa5fa\.\.\.eda893ea7, temperature 0, seed 13, disabled thinking, and output limits of 96 tokens in label mode and 128 in act mode\. The native API makes these settings explicit\. Each condition makes 70 isolated calls; all 280 Qwen outputs parse successfully\. Exact prompts, complete digest, runtime metadata, raw outputs, and token counts accompany each prediction\.

TABLE III:Held\-out MCB results on non\-author\-adjudicated gold \(n=70n=70\)\. Accuracy includes a bootstrap 95% CI\. OM is erroneous persistence over all items; Clar\. and Ver\. are recalls on 12 and 18 relevant gold items\. All invalid\-output rates are zero\. Act rows emit one tool\-call object without a policy prompt\.
## VResults and Discussion

### V\-ACross\-Family Under\-Asking

Table[III](https://arxiv.org/html/2608.19564#S4.T3)shows a common asymmetry\. In Claude label\-mode runs, verification recall is 0\.889–1\.000 while clarification recall is 0\.500–0\.750\. Qwen makes the distinction sharper: bare Qwen verifies 12/18 freshness items but clarifies 0/12 ambiguous items\. Its clarification cases are instead mapped to persist \(7\), verify \(4\), or ephemeral \(1\)\. This is a source\-of\-truth confusion: the model may recognize uncertainty yet consult the world rather than the user who alone can resolve intent\. It otherwise silently commits an interpretation\.

This is cross\-family evidence for under\-asking, but not for identical error policies\. Bare Qwen over\-persists on 0\.243 of all items versus 0\.029 for bare Haiku \(descriptive paired eventpraw<0\.001p\_\{\\rm raw\}<0\.001\)\. The useful generalization is therefore narrow: both families under\-ask; the alternative action they choose is model\-dependent\.

### V\-BPrompt and Policy Sensitivity

Table[IV](https://arxiv.org/html/2608.19564#S5.T4)separates total accuracy from behavioral change\. Qwen few\-shot improves accuracy by 0\.214 \(pH=0\.002p\_\{\\rm H\}=0\.002\), fixes 16 bare errors while breaking one correct answer, and increases clarification recall from 0 to 0\.333\. Nevertheless it still misses 8/12 clarification opportunities, so examples mitigate but do not remove under\-asking\.

Qwen policy improves accuracy by only 0\.071, which is not statistically separable from zero \(pH=0\.539p\_\{\\rm H\}=0\.539\)\. Its policy is nonetheless behaviorally different: erroneous persistence falls by 0\.143, from 17/70 to 7/70\. On paired item\-level OM events, the policy removes 11 bare errors and introduces one \(pH=0\.038p\_\{\\rm H\}=0\.038\)\. It moves uncertainty primarily to verification \(recall 0\.667 to 0\.944\), not to the user \(clarification 0 to 0\.083\)\. A headline accuracy alone would miss this safety\-relevant intervention effect\.

### V\-CExploratory Contrast Validation

To assess robustness beyond the primary 70\-item test, we froze a separate 70\-item contrast extension before inference\. It contains 35 evidence\-flip pairs \(10 items/category\), including seven cue\-conflicting traps; the development set and demonstrations are unchanged\. Both non\-author annotators independently reproduced all 70 contrast labels \(κ=1\.000\\kappa=1\.000\)\. On the combined 140 Qwen items, bare, policy, and few\-shot accuracy is 0\.614, 0\.757, and 0\.843\. Relative to bare, policy gains 0\.143 \(pH<0\.001p\_\{\\rm H\}<0\.001\) and few\-shot gains 0\.229 \(pH<0\.001p\_\{\\rm H\}<0\.001\)\. Clarification recall remains the weakest class \(0\.074/0\.407/0\.519\)\. Because these rule\-authored templates align closely with the explicit policy rules, we retain the extension as a controlled sensitivity check rather than claim naturalistic external validity\.

The Claude pattern is also model\-dependent\. Haiku’s policy and few\-shot gains survive the six\-test correction \(pH=0\.002p\_\{\\rm H\}=0\.002and 0\.047\); Sonnet’s do not\. Across three models, the benchmark therefore measures a prompt\-conditioned commitment policy rather than a fixed model trait\.

TABLE IV:Selected exact paired comparisons\.Δ\\Deltais accuracy unless marked OM; intervals are paired\-bootstrap 95% CIs;pHp\_\{\\rm H\}is Holm\-adjusted within the families defined in Sec\. III\-B\.
### V\-DStated Choices Do Not Reliably Predict Tool\-Call Choices

MCB\-Act addresses the concern that selecting a label may not predict tool\-call behavior\. For both Claude models, raw action agreement between bare label mode and tool\-call mode is 0\.571: 30/70 decisions change when labels become tools\. Sonnet accuracy drops from 0\.814 to 0\.529 \(Δ=−0\.286\\Delta=\-0\.286,pH<0\.001p\_\{\\rm H\}<0\.001\); Haiku’s drop from 0\.629 to 0\.514 is not significant \(pH=0\.057p\_\{\\rm H\}=0\.057\)\. Qwen supplies the cross\-family test: agreement is only 0\.229 and accuracy drops from 0\.557 to 0\.343 \(Δ=−0\.214\\Delta=\-0\.214,pH=0\.047p\_\{\\rm H\}=0\.047\)\. It callsuse\_nowon 54/70 items, reducing over\-memory but collapsing verification recall from 0\.667 to 0\.056\. Thus tool\-call selection changes all three models, but with model\-specific action biases\. Every emitted argument passes the deterministic well\-formedness rules, including all clarification questions on gold\-clarify items\. The bottleneck is tool choice, not malformed arguments\.

### V\-EWhat the Benchmark Does and Does Not Establish

Always\-Persist obtains 0\.257 accuracy and 0\.743 OM, establishing that unconditional durable commitment conflicts with most item\-level decisions\. Majority Action reaches 0\.314 but has only 0\.120 macro\-F1, exposing the weakness of a frequency\-only rule\. Neither shows that every system retaining raw history will fail: tiering, expiration, and retrieval filters can implement a weaker effective commitment\. Similarly, the category oracle’s 0\.800 shows that scenario type carries substantial signal; bare models do not reliably exceed it\. We therefore emphasize paired intervention effects and class\-specific errors, not a system leaderboard\.

MCB\-Act is a more behaviorally concrete test than naming a label, but it records rather than executes the selected store, source, or user\-facing operation\. No simulated user answers the question, no verified source changes, and no downstream task score is observed\. The results establish a label–tool\-selection gap, not a quantified improvement in end\-to\-end utility\.

## VILimitations, Ethics, and Reproducibility

The primary 70\-item cross\-family test yields wide intervals; 10\-item category cells and eight traps support qualitative analysis only\. Evaluation labels now follow a completed blind non\-author audit, but scenarios and the labeling rules were author\-written\. The perfectly reproduced contrast labels may reflect close rule–template alignment, so that extension remains a controlled sensitivity check\. Development demonstrations retain author labels\. Qwen is one 9B quantized checkpoint; quantization, serving stack, and disabled thinking are part of the evaluated system\. Act mode uses only the bare intervention and one checkpoint per family\. All scenarios are synthetic and in English\.

The benchmark contains no personal records\. Its intended use is to reduce privacy and personalization failures from inappropriate durable memory\. The artifact includes 140 primary items, 70 contrast\-validation items, all frozen annotations and majority decisions, original and audited labels, prompts, runners, tests, model outputs, exact metadata, and scripts that regenerate every table and paired test\. Deterministic components run without model access; stored predictions permit statistical reproduction\. Generative AI tools were used as experimental subjects and for implementation/language assistance; numerical claims in this paper are generated from the released item\-level outputs and deterministic analysis code\.

## VIIConclusion

Memory commitment is a distinct agent capability: deciding whether to remember, limit, verify, or ask\. Cross\-family label experiments confirm severe under\-asking and show that prompts can change both accuracy and safety\-relevant action distributions\. Cross\-family tool\-call selection further reveals that stated decisions do not reliably survive translation into action choices, although the resulting bias is model\-dependent\. Evaluation of persistent\-memory agents should therefore report clarification and over\-memory explicitly, use paired tests, and elicit structured behavior rather than labels alone\.

## References

- \[1\]\(2025\)τ2\\tau^\{2\}\-Bench: evaluating conversational agents in a dual\-control environment\.Note:arXiv:2506\.07982External Links:2506\.07982Cited by:[§II](https://arxiv.org/html/2608.19564#S2.p3.1)\.
- \[2\]P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav\(2025\)Mem0: building production\-ready AI agents with scalable long\-term memory\.Note:arXiv:2504\.19413External Links:2504\.19413Cited by:[§I](https://arxiv.org/html/2608.19564#S1.p1.1)\.
- \[3\]Y\. In, W\. Kim, S\. Park, K\. Yoon, and C\. Park\(2026\)Personalize\-then\-store: benchmarking and learning personalized memory for long\-horizon agents\.Note:arXiv:2605\.25535External Links:2605\.25535Cited by:[TABLE I](https://arxiv.org/html/2608.19564#S2.T1.2.4.1.1),[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.
- \[4\]A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang\(2024\)Evaluating very long\-term conversational memory of LLM agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 13851–13870\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.747)Cited by:[TABLE I](https://arxiv.org/html/2608.19564#S2.T1.2.2.1.1),[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.
- \[5\]Ollama\(2026\)Ollama: run large language models locally\.Note:SoftwareExternal Links:[Link](https://ollama.com/)Cited by:[§IV](https://arxiv.org/html/2608.19564#S4.p2.1)\.
- \[6\]C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez\(2023\)MemGPT: towards LLMs as operating systems\.Note:arXiv:2310\.08560External Links:2310\.08560Cited by:[§I](https://arxiv.org/html/2608.19564#S1.p1.1)\.
- \[7\]Qwen Team\(2026\)Qwen3\.5\-9B model card\.Note:Hugging Face model repositoryAccessed: 2026\-08\-13External Links:[Link](https://huggingface.co/Qwen/Qwen3.5-9B)Cited by:[§IV](https://arxiv.org/html/2608.19564#S4.p2.1)\.
- \[8\]Y\. Shen, K\. Li, W\. Zhou, and S\. Hu\(2026\)Mem2ActBench: a benchmark for evaluating long\-term memory utilization in task\-oriented autonomous agents\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics,San Diego, California, USA,pp\. 8173–8190\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.370)Cited by:[TABLE I](https://arxiv.org/html/2608.19564#S2.T1.2.6.1.1),[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.
- \[9\]H\. Tan, Z\. Zhang, C\. Ma, X\. Chen, Q\. Dai, and Z\. Dong\(2025\)MemBench: towards more comprehensive evaluation on the memory of LLM\-based agents\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 19336–19352\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.989)Cited by:[TABLE I](https://arxiv.org/html/2608.19564#S2.T1.2.3.1.1),[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.
- \[10\]A\. Testoni and R\. Fernández\(2024\)Asking the right question at the right time: human and model uncertainty guidance to ask clarification questions\.InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics \(EACL\),pp\. 258–275\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.eacl-long.16)Cited by:[§II](https://arxiv.org/html/2608.19564#S2.p4.1)\.
- \[11\]M\. N\. Uddin, K\. Shubham, E\. Blanco, C\. Baral, and G\. Wang\(2026\)From recall to forgetting: benchmarking long\-term memory for personalized agents\.InFindings of the Association for Computational Linguistics: ACL 2026,Note:arXiv:2604\.20006Cited by:[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.
- \[12\]D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu\(2025\)LongMemEval: benchmarking chat assistants on long\-term interactive memory\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[TABLE I](https://arxiv.org/html/2608.19564#S2.T1.2.2.1.1),[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.
- \[13\]J\. Xiao, X\. Yu, C\. Wang, W\. Zheng, X\. Lin, K\. Liu, H\. Ding, Y\. Zhang, W\. Wang, F\. Feng, and X\. He\(2026\)AlpsBench: an LLM personalization benchmark for real\-dialogue memorization and preference alignment\.Note:arXiv:2603\.26680External Links:2603\.26680Cited by:[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.
- \[14\]S\. Yan, X\. Yang, Z\. Huang, E\. Nie, Z\. Ding, Z\. Li, X\. Ma, H\. Schütze, V\. Tresp, and Y\. Ma\(2025\)Memory\-R1: enhancing large language model agents to manage and utilize memories via reinforcement learning\.Note:arXiv:2508\.19828External Links:2508\.19828Cited by:[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.
- \[15\]S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhan\(2024\)τ\\tau\-Bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.Note:arXiv:2406\.12045External Links:2406\.12045Cited by:[§II](https://arxiv.org/html/2608.19564#S2.p3.1)\.
- \[16\]M\. J\. Q\. Zhang and E\. Choi\(2025\)Clarify when necessary: resolving ambiguity through interaction with LMs\.InFindings of the Association for Computational Linguistics: NAACL,pp\. 5541–5558\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-naacl.306)Cited by:[§II](https://arxiv.org/html/2608.19564#S2.p4.1)\.
- \[17\]T\. Zhang, P\. Qin, Y\. Deng, C\. Huang, W\. Lei, J\. Liu, D\. Jin, H\. Liang, and T\. Chua\(2024\)CLAMBER: a benchmark of identifying and clarifying ambiguous information needs in large language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,pp\. 10746–10766\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.578)Cited by:[TABLE I](https://arxiv.org/html/2608.19564#S2.T1.2.5.1.1),[§II](https://arxiv.org/html/2608.19564#S2.p4.1)\.
- \[18\]W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang\(2024\)MemoryBank: enhancing large language models with long\-term memory\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19724–19731\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29946)Cited by:[§II](https://arxiv.org/html/2608.19564#S2.p1.1)\.