MemFail: Stress-Testing Failure Modes of LLM Memory Systems

# MemFail: Stress-Testing Failure Modes of LLM Memory Systems
Source: [https://arxiv.org/html/2605.26667](https://arxiv.org/html/2605.26667)
Ishir Garg University of California, Berkeley ishirgarg@berkeley\.edu &Neel Kolhe University of California, Berkeley neelkolhe@berkeley\.edu Dawn Song University of California, Berkeley dawnsong@cs\.berkeley\.edu &Xuandong Zhao University of California, Berkeley xuandongzhao@berkeley\.edu

###### Abstract

Large language model \(LLM\) agents increasingly rely on external memory systems to remain consistent across long\-horizon interactions, but little empirical work has been done to understand the specific failure modes and design choices that these systems present\. Existing benchmarks report aggregate question\-answering accuracy and treat memory systems as black boxes, making it impossible to attribute an incorrect answer to a particular failure mode of the system\. We introduceMemFail, a diagnostic benchmark that isolates the failure modes of modern LLM memory systems\. We begin by formalizing memory systems as the composition of three canonical operations—summarization, storage, and retrieval—and identify the potential failure modes induced by each\. Based on these hypothesized failure modes, we construct five datasets spanning four tasks, each adversarially designed to test a specific operation of a memory system\. Using these datasets, we evaluate four state\-of\-the\-art memory systems onMemFailand demonstrate howMemFailcan be used to empirically understand the tradeoffs induced by differences in memory system architectures\.

MemFail: Stress\-Testing Failure Modes of LLM Memory Systems

Ishir GargUniversity of California, Berkeleyishirgarg@berkeley\.eduNeel KolheUniversity of California, Berkeleyneelkolhe@berkeley\.edu

Dawn SongUniversity of California, Berkeleydawnsong@cs\.berkeley\.eduXuandong ZhaoUniversity of California, Berkeleyxuandongzhao@berkeley\.edu

[https://github\.com/ishirgarg/MemFail](https://github.com/ishirgarg/MemFail)

## 1Introduction

LLM agents have rapidly grown more capable at long\-horizon tasks, but their limited context windows make it difficult to maintain consistency over time\. Traditional memory systems such as retrieval\-augmented generation \(RAG\)Lewiset al\.\([2021](https://arxiv.org/html/2605.26667#bib.bib16)\)and vector databases store the raw conversation history, which is expensive in storage and input tokens when only a fraction is worth remembering, and which provides no natural way to forget or update old information as user preferences change\.

A growing body of work onLLM memory systemshas emerged in response, augmenting agents with external stores they can read, write, and update over a lifetime, enabling consistent and personalized responsesChhikaraet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib9)\); Xuet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib26)\); Liuet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib18)\); Xuet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib27)\); Rasmussenet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib42)\); Huet al\.\([2026a](https://arxiv.org/html/2605.26667#bib.bib43)\)\. But the very mechanisms for compression, updating, and forgetting introduce new failure modes: a system mayover\-summarizea conversation before storing it, stripping key details, or fail to remove an old fact when a contradictory one arrives\.

Prior work has converged on a few key desiderata: retrieving only the memories useful for the current task, faithful storage that preserves the semantics of the original experience, updatability to overwrite outdated facts and accumulate coexisting ones, low latency for fast storage and retrieval, and token efficiency to minimize tokens in the agent’s context\. However, these desiderata trade off: a system that stores every conversation verbatim \(like simple RAG\) excels at faithful storage but is token\-inefficient, while breaking conversations into smaller memories shrinks retrieved context but makes the relevant pieces harder and slower to find\.

Despite the recent surge of memory systems with varied design choices, little work has characterized thetradeoffsbetween these desiderata or the failure modes underneath them\. Existing benchmarks treat memory systems as end\-to\-end black boxes, providing aggregate metrics that obscure the distinct failure modes inside\. In this work, we introduceMemFail, a benchmark targeting failure modes of LLM memory systems\. We begin by proposing a framework in which memory systems can be formally represented as a combination of three operations: summary, storage, and retrieval\. Based on this taxonomy, we hypothesize distinct failure modes that any memory system fitting this formalism may exhibit\.MemFailprovides five datasets spanning four tasks to elicit these failure modes:

- •Conditional\-Facts: faithful retention of important causal relationships\.
- •Persona\-Retrieval: faithful retention of small details under misleading questions\.
- •Long\-Hop: retrieval over long\-range causal relationships\.
- •Coexisting\-Facts: storing and retrievingallrelevant coexisting pieces of information for a query\.
- •Conditional\-Facts \(Hard\): preserving important causal details when summarizing experiences\.

We evaluate four state\-of\-the\-art memory systems—Mem0Chhikaraet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib9)\), A\-MEMXuet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib26)\), SimpleMemLiuet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib18)\), and StructMemXuet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib27)\)—onMemFail, and our analysis surfaces three findings that aggregate metrics would otherwise obscure\. First, no single system dominates: each architecture exhibits a distinctive failure signature—graph\-based StructMem excels at causal reasoning but collapses on coexisting\-fact retrieval, while Mem0 shows the opposite pattern\. Second, scaling either the number of retrieved memories or the strength of the underlying LLM yields little improvement, and in several cases degrades performance, indicating that current systems are bound by architectural constraints rather than by model intelligence or context budget\. Third, the relationship between token consumption and accuracy is task\-dependent: summary\-bottlenecked tasks reward verbose memories, while retrieval\-bottlenecked tasks suffer when large memories pollute the embedding space\. Building on these findings, we propose two under\-explored directions—mixture\-of\-experts memory architectures and task\-adaptive token scaling—as paths toward systems that mitigate, rather than trade off between, the desiderata above\.

## 2Related Work

LLM Memory Systems\.A growing body of work augments LLMs with persistent memory\. MemGPT pages between in\-context working memory and external storage in an OS\-inspired hierarchyPackeret al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib20)\)\. MemoryBank applies summarization and Ebbinghaus\-style forgetting curves over user\-specific factsZhonget al\.\([2023](https://arxiv.org/html/2605.26667#bib.bib29)\)\. Mem0 extracts atomic facts into hybrid vector and graph stores with explicitADD,UPDATE, andDELETEoperationsChhikaraet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib9)\)\. A\-MEM forgoes predefined schemas, organizing memories as agentically linked notes that the agent dynamically tags and reorganizesXuet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib26)\)\. StructMem builds hierarchical event\-level structures with periodic semantic consolidationXuet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib27)\), while SimpleMem applies semantically lossless compression via entropy\-aware filtering and adaptive retrievalLiuet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib18)\)\. ReflexionShinnet al\.\([2023](https://arxiv.org/html/2605.26667#bib.bib22)\)and Generative AgentsParket al\.\([2023](https://arxiv.org/html/2605.26667#bib.bib21)\)explore reflective memory streams for self\-improvement and social simulation\. These systems share the primitives our framework abstracts—summarize, store, retrieve—and our benchmark plugs into any system exposing them\.

LLM Long\-Context and Memory System Benchmarks\.Two orthogonal lines of work complement ours\. ER\-MIAPiehlet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib32)\), InjecMEMTianet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib24)\), and SkillJectJiaet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib33)\)respectively mount memory\-, prompt\-, and skill\-injection attacks against memory\-augmented LLMs, layered systems such as MemoryOSKanget al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib12)\), and the agent\-skills abstraction, exposing adversarial rather than benign failures\. Needle\-in\-a\-HaystackNelsonet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib34)\), RULERHsiehet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib35)\), LongEvalKrishnaet al\.\([2023](https://arxiv.org/html/2605.26667#bib.bib13)\), LongBenchBaiet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib6)\), L\-EvalAnet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib5)\), and M4LEKwanet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib14)\)stress\-test the long\-context capabilities of LLMs themselves rather than the memory systems built atop them\. For general recall, MemoryBank pairs probing questions with multi\-day historiesZhonget al\.\([2023](https://arxiv.org/html/2605.26667#bib.bib29)\); PerLTQA targets personalized long\-term QADuet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib10)\); MemBench spans factual and reflective memory across participation and observation scenarios with metrics for accuracy, recall, capacity, and temporal efficiencyTanet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib23)\); MemoryAgentBench converts long\-context datasets into incremental tasks across four competencies—accurate retrieval, test\-time learning, long\-range understanding, and conflict resolutionHuet al\.\([2026b](https://arxiv.org/html/2605.26667#bib.bib36)\); LoCoMo provides multi\-session conversations covering single\-hop, multi\-hop, temporal, open\-domain, adversarial, event\-summarization, and multi\-modal tasksMaharanaet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib19)\); LongDialQA derives multi\-party dialogues from TV scriptsKimet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib37)\)\. For reasoning, MemSimZhanget al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib38)\)synthesizes QA via a Bayesian Relation Network, and MemoryBench reframes evaluation as continual learning from simulated feedback, distinguishing declarative from procedural memoryAiet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib39)\)\. A few target failures explicitly: LoCoMo includes misleading questions for abstentionMaharanaet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib19)\); LongMemEval curates 500 questions across information extraction, multi\-session reasoning, temporal reasoning, knowledge updates, and abstention, alongside a key\-value formal modelWuet al\.\([2024](https://arxiv.org/html/2605.26667#bib.bib25)\)\. However, their model is less general than ours, and they do not discuss failure modes; StructMemEval shows that organizational hints improve memory organizationShutovaet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib40)\)\. All treat the system as a black box, reporting aggregate end\-to\-end scores that cannot localize failures to summarization, storage, retrieval, or reasoning—a gapMemFailfills by isolating each operation and analyzing failure modes and their tradeoffs\.

## 3Background

We present a formal model of LLM memory systems thatMemFailassumes\. The framework identifies natural failure modes any system fitting it may exhibit, whichMemFailis designed to test\.

### 3\.1Three Operations of a Memory System

We assume every memory system decomposes into three operations\. LetQQdenote a user query,HHa conversation history, andMMthe current memory database\.

Retrieval\.Given a new queryQQ, the conversation historyHH, and the memory stateMM, the system retrieves a set of relevant memoriesRRdrawn fromMMand inserted into the agent’s prompt alongsideQQand \(optionally\)HH\.

Summarization\.After an interaction,HHis compressed into a representationH′H^\{\\prime\}that extracts the information deemed worth retaining\.

Storage\.The storage step takesH′H^\{\\prime\}and the existing memory stateMMand produces an updated memory stateM′M^\{\\prime\}, possibly overwriting, merging, or appending to existing entries inMM, or performing a no\-op\.

As demonstrated in Table[1](https://arxiv.org/html/2605.26667#S3.T1), this decomposition subsumes a wide range of modern memory systems, including Mem0, A\-MEM, SimpleMem, and StructMem, and mirrors the canonical encoding/storage/retrieval description of human memory\.

Table 1:How each tested memory system implements the three operations of Section[3](https://arxiv.org/html/2605.26667#S3)\.
### 3\.2Failure Modes

Based on our abstract model of a memory system, we propose that memory systems exhibit four natural failure modes:

- •Summary failure:summarization deletes or malforms important information present inHH\. E\.g\.,*“I am deathly allergic to peanuts”*may be compressed into*“allergic to peanuts”*, stripping the severity critical for downstream reasoning\.
- •Storage failure:the storage step does not adequately incorporateH′H^\{\\prime\}intoMM\. This includes refusing to overwrite outdated facts \(e\.g\., not updating*“Dan likes pizza”*after the user states Dan now hates pizza\) and refusing to admit valid coexisting facts \(e\.g\., rejecting*“Dan likes burgers”*as contradictory to stored*“Dan likes pizza”*\)\.
- •Retrieval failure:retrieval fails to return relevant memories inRR, or returns memories that are semantically similar but contextually inappropriate\.
- •Reasoning failure:the agent makes an incorrect judgment even when the correct memories were retrieved\. Note that this isnota failure of the memory system, but we measure it for completeness\.

Prior work primarily evaluates the combination of all four modes by inserting long conversation histories and asking LLMs to infer user personalities or preferences without distinguishing between them\.MemFailconsists of tasks specifically designed to isolate failure modes \(1\)–\(3\), which are unique to modern memory systems and unaddressed by prior benchmarks\.

### 3\.3Compatibility with Existing Memory Systems

Critically, a good benchmark should evaluate a diverse array of memory systems with minimal assumptions about their internals\.MemFailevaluates any system that exposes three functions:

- •store\_conversation\(HH\): storeHHin the database \(subsumingSummarizeandStore\)\.
- •retrieve\_memories\(Q,H,kQ,H,k\): return the top\-kkmemories relevant toQQandHH\.
- •get\_all\_memories\(\): return all currently stored memories\.

The first two are required for the evaluation loop\. The third is used only byMemFail’s judge LLM to diagnose which failure mode occurred, and is not required for deployment of the memory system\. Any system implementing these three functions automatically plugs into our evaluation harness\.

## 4Benchmark Details

MemFailcomprises of five datasets in English organized into four tasks, each hypothesized to isolate one failure mode from Section[3](https://arxiv.org/html/2605.26667#S3)\. We sketch the high\-level design only; full specifications, generator prompts, validation rules, deduplication thresholds, sampling pools, and additional worked examples are deferred to Appendix[B](https://arxiv.org/html/2605.26667#A2)\.

##### Task 1:Conditional\-Facts\.

Targets*summary failure*—summarization\-based memory stripping qualifying conditions from facts at commit time\. Each row encodes a rule “entityEEexhibits behaviorBBonly when conditionCCis satisfied,” embedded in a55–88sentence essay alongside44–77unrelated unconditional facts about the same entity\. The graded query asks whetherEEwould exhibitBBin a specific contextXXthat either does or does not satisfyCC; a system that dropsCCduring summarization and stores the unconditional “EEdoesBB” would incorrectly answer “yes” regardless ofXX\. Conditions are sampled from a fixed list of types \(time of day, weather, mood, social setting, prior activity, etc\.\)\. We create two variants: theEasyvariant places the entire rule inside a single sentence, so any system that copies the sentence verbatim succeeds; theHardvariant decomposes the rule into three non\-adjacent sentences—a behavior sentence, a condition sentence, and a linking sentence—spread across an88–1212sentence essay, forcing*reconstruction*from distributed evidence\. Easy and Hard rows share the same entity and condition specs, so any performance gap is attributable to the distribution of the rule across sentences\. A Hard example is given in Appendix[B\.3](https://arxiv.org/html/2605.26667#A2.SS3)\.

Example:Conditional\-Facts\(Easy\)Essay \(excerpt\):“…Sylas is a shrewd negotiator who thrives in the bustling markets and political halls of his city\.… Sylas draws elaborate maps only if he has just finished a negotiation\.…”Question:“Sylas just finished meditating quietly, not negotiating\. Would he draw an elaborate map now?”Ground truth:“No—Sylas only draws maps immediately after a negotiation\.”

##### Task 2:Coexisting\-Facts\.

Targets both*storage failure*and*retrieval failure*\. Modern memory systems aggressively reconcile incoming information with their existing database; a possible failure pattern is that two*compatible*facts \(“user likes pizza” and “user likes ramen”\) are incorrectly treated as contradictions, causing the system to overwrite the older fact rather than store both\. Each row covers one of100100curated preference categories \(foods, hat styles, music genres, etc\.\), holdsN∈\{2,3,4,5\}N\\\!\\in\\\!\\\{2,3,4,5\\\}distinct preferences within that category—each expressed as an isolated first\-person statement that references only that single preference—and asks a holistic scenario question whose well\-formed answer requires allNNpreferences\.

Example:Coexisting\-FactsCategory:hat styles\.Preference facts:\(i\) “I often wear a fedora when I want to add a classic touch to my outfit”; \(ii\) “Beanies are my go\-to for staying warm and casual during chilly days”; \(iii\) “A bucket hat is what I reach for on sunny, laid\-back weekends\.”Question:“I’m packing for a week\-long trip with mixed weather—which hats should I bring to cover all occasions?”Ground truth:fedora, beanie, bucket hat\.

##### Task 3:Persona\-Retrieval\.

Targets*storage failure*—a memory system retrieving a stored profile when asked about a different person\. Each row contains a1010–1515sentence essay about a named entityEEembedding44–55idiosyncratic facts\. Three graded queries are attached, each independently toggled50/5050/50between two forms: a*direct*query that namesEEand is answerable from a specific essay detail, and a*misleading*query that names a distractorDDunrelated to the question, for which the correct answer is to abstain\. Entities and distractors are sampled from a fixed pool of3030diverse names; persona flavors from a fixed pool of3030flavors\.

Example:Persona\-RetrievalEssay \(excerpt\):“Yuki Tanaka spends most mornings hunched over a lightbox, tracing contours… She refuses to eat shellfish because she gets a severe hive reaction, so boat provisions are always cooked onshore and strictly shellfish\-free\.…”Q1 \(misleading\):“Does Noah Brooks eat shellfish?”→\\rightarrow“I don’t have information about Noah Brooks\.”Q2 \(non\-misleading\):“What food should I avoid serving to Yuki Tanaka?”→\\rightarrow“Avoid shellfish—she has a severe allergic reaction\.”

##### Task 4:Long\-Hop\.

Targets*retrieval failure*\. Each row encodes a strictly transitive chainA1→⋯→AK\+2A\_\{1\}\\\!\\to\\\!\\cdots\\\!\\to\\\!A\_\{K\+2\}ofK∈\{1,2,3\}K\\\!\\in\\\!\\\{1,2,3\\\}hops, in which factiilinks anchoriito anchori\+1i\{\+\}1; anchors are subjective \(moods, routines, opinions, personal objects\) so that no fact is answerable from world knowledge alone, and every fact is self\-contained\. The graded question references only the head anchor and asks for the terminal anchor, presented as a55\-way multiple\-choice question with four shape\-matched distractors orthogonal to every fact in the chain\. At evaluation time, we provide each factseparatelyto the memory system; this forces the system to retrieve and compose the chain from scattered storage rather than read it off from a single conversation\.

Example:Long\-Hop\(K=3K\{=\}3\)Facts:“Whenever I sip morning espresso I call my mom”; “After I call my mom I plan a day trip”; “When I plan a day trip I pack snacks”; “When I pack snacks I take scenic photos\.”Question:“When I sip morning espresso, what do I end up doing?”Choices:A\. organize the bookshelf; B\. practice guitar scales; C\. water the plants;D\. take scenic photos; E\. fold the laundry\.Ground truth:take scenic photos\.

##### Dataset statistics\.

Table[2](https://arxiv.org/html/2605.26667#S4.T2)reports measured statistics across all five datasets, with all token counts \(mean tokens per dataset entry\) computed using thecl100k\_basetokenizerOpenAI \([2022](https://arxiv.org/html/2605.26667#bib.bib1)\)\.Coexisting\-Factsis broken out by per\-row preference countNNandLong\-Hopby hop countKK, since both the number of storage units and the entry\-token cost scale with these\.

Table 2:Per\-dataset measured statistics; “Avg\. tok\.” is mean tokens per entry undercl100k\_base\. ForCoexisting\-FactsandLong\-Hop, values are reported in order ofNNandKKrespectively\.

## 5Experimental Setup

### 5\.1Evaluation Loop

MemFail’s harness applies to any memory system implementing the three operations of Section[3](https://arxiv.org/html/2605.26667#S3)\. Each task reduces to a sequence of conversations: ungraded*storage conversations*commit information to memory, and*query conversations*elicit graded answers\.

Phase 1: Storage\.For each task we extract the information unit needed per graded query—a conditional\-fact essay, a preference statement, a persona essay, or a message from the reasoning chain—and send each in its own conversation, so that the system must store and associate them across sessions\.

Phase 2: Query\.The harness creates one query conversation per graded question, callsmemory\_system\.retrieve\_memories\(query, conversation, k\), formats the top\-kkmemories into the prompt, invokes the test\-taker, and records the response\. Query conversations do not update the database, so query order is irrelevant\.

Phase 3: Grading\.Each graded question has a set ofN≥1N\\geq 1memories the system should retrieve\. An LLM\-as\-a\-judge receives the query, ground truth, all stored memories at query time, and all retrieved memories, and classifies each of theNNmemories into:

1. 1\.Storage check\.Is the memory present inget\_all\_memories\(\)? Failure is astorage error\.
2. 2\.Summary check\.Conditional on storage, are*critical details*preserved \(e\.g\., the qualifying condition inConditional\-Facts\)? Failure is asummary error\.
3. 3\.Retrieval check\.Conditional on faithful storage, was the entry in the top\-kkset? Failure is aretrieval error\.
4. 4\.Reasoning check\.Conditional on retrieval, did the test\-taker use it to produce the correct answer? Failure is areasoning error\.
5. 5\.Correct\.All memories stored, summarized, retrieved, and used successfully\.

We focus our analysis on the first three, as these are memory\-system failures; reasoning errors reflect LLM limitations and occur infrequently \(Appendix[C](https://arxiv.org/html/2605.26667#A3)\)\. We fixgpt\-5\-miniSinghet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib41)\)as both test\-taker and grader—the LLM that the memory system augments, not the system’s internal model—so that cross\-system differences are attributable to the memory system rather than the underlying LLM\. Full prompts and grading details are given in Appendix[D](https://arxiv.org/html/2605.26667#A4)\.

Human Validation\.All dataset examples are manually verified for correctness\. On 100 manually graded examples,gpt\-5\-minianswers 98% correctly and classifies the error type 98\.4% correctly\.

### 5\.2Memory Systems

We evaluate four open\-source modern memory systems—Mem0Chhikaraet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib9)\),A\-MEMXuet al\.\([2025](https://arxiv.org/html/2605.26667#bib.bib26)\),SimpleMemLiuet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib18)\), andStructMemXuet al\.\([2026](https://arxiv.org/html/2605.26667#bib.bib27)\)—and study in Section[6](https://arxiv.org/html/2605.26667#S6)how the success rate and error types vary with architecture, retrieval depthkk, and the strength of the system’s internal model\.

## 6Experiments

We evaluate open\-source memory systems onMemFail\. The main text shows representative examples of the most interesting findings; more complete evaluation results are in Appendix[C](https://arxiv.org/html/2605.26667#A3)\.

Q1: How does performance scale withkk, the number of retrieved memories?Figure[1](https://arxiv.org/html/2605.26667#S6.F1)shows thatMemFailis difficult even for state\-of\-the\-art memory systems, and that performance scales poorly withkk\. The exception isCoexisting\-Facts, which naturally benefits from largerkksince coexisting facts are more likely to be retrieved, even by chance\. Performance for a given system also varies substantially across tasks: becauseMemFail’s tasks isolate specific weaknesses, they reveal how architectures induce distinct failure modes\. StructMem performs strongly on most tasks but fails spectacularly onCoexisting\-Facts, while Mem0 shows the opposite pattern \(analyzed in Q4\)\. Figure[8](https://arxiv.org/html/2605.26667#A3.F8)in Appendix[C](https://arxiv.org/html/2605.26667#A3)breaks down error types per system, revealing how different tasks elicit different failure modes \(concrete failure examples in Appendix[A](https://arxiv.org/html/2605.26667#A1)\):

- •Coexisting\-Factsinducesretrieval failures: most systems fail to associate all related facts with the query at retrieval time\.
- •Conditional\-Facts \(Hard\)inducessummary failures: all systems over\-compress, eitheralteringthe original message orstripping awayprecise, critical details—both mislead the LLM into the wrong judgment\.
- •Persona\-Retrievalinducessummary failuresvia over\-compression of long personas; the exception is Mem0, which fails to store all details in the first place due to its LLM\-tool\-call update mechanism\.
- •Long\-Hopunsurprisingly inducesretrieval failures: systems fail to capture long\-range causal relationships between seemingly disjoint entities\.

Except for Mem0, systems generally do not exhibitstorage failures; failures stem almost entirely from incorrect summarization or retrieval\.

Key TakeawayIncreasing the number of retrieved memories yields performance gains on tasks where retrieval errors are common; it yields marginal returns when summary errors are the bottleneck\.

![Refer to caption](https://arxiv.org/html/2605.26667v1/x1.png)Figure 1:Performance of memory systems using GPT\-4\.1\-mini internally\. All confidence intervals use 95% Wilson score binomial intervals\.Q2: How does accuracy scale with the strength of the model used by the memory system?Figure[2](https://arxiv.org/html/2605.26667#S6.F2)shows stronger models do not improve accuracy while sometimes degrading accuracy on most tasks; smarter reasoning models can generate overly verbose memories that pollute the agent’s context\.

Key TakeawayUnlike other applications of LLM agents where using a more intelligent model can increase performance on benchmarks, modern memory systems are bound by architectural constraints rather than model intelligence\.

![Refer to caption](https://arxiv.org/html/2605.26667v1/x2.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x3.png)

Figure 2:Performance of StructMem and SimpleMem as a function of their internal model\. Mem0 and A\-MEM follow the same trend, as shown in Appendix[C](https://arxiv.org/html/2605.26667#A3), but we exclude them here for conciseness\. Equipping the system with a stronger internal model does not lead to performance gains\.Q3: What doesMemFailreveal about the tradeoff between performance and token consumption?Figure[3](https://arxiv.org/html/2605.26667#S6.F3)shows how performance scales with the token usage of each memory system\. Performance generally scales positively with token consumption onPersona\-RetrievalandConditional\-Facts \(Hard\): in general, for tasks bottlenecked by summary failures, increasing the number of tokens leads to performance gains\. By contrast, retrieval tasks can actually see a performance drop from using more tokens\. This is especially evident forCoexisting\-Facts, where storing large memories “pollutes” the semantic embeddings, hurting retrieval\.

![Refer to caption](https://arxiv.org/html/2605.26667v1/x4.png)Figure 3:Per\-model performance onMemFailrelative to the average number of tokens per memory\.Key TakeawayThe historical consensus is that more tokens is an easy way to obtain performance gains \(i\.e\., test\-time scaling\); however, for memory systems, performance scales with increased memory storage in a way that ishighly task\-dependent\.

Q4: What doesMemFailreveal about design choices in memory systems?We provide insights into the key architectural decisions of the tested memory systems:

- •LLM\-based memory updates\.Mem0 updates memory via LLM tool calls\. For short experiences \(e\.g\., one\-sentenceCoexisting\-Factsentries\) the LLM accurately stores the content, but for longer experiences it fails to issue enough tool calls to capture all details—evidenced by Mem0’s high storage error rate onPersona\-Retrieval, which has the longest entries \(Figure[1](https://arxiv.org/html/2605.26667#S6.F1)\)\.
- •Semantic vector stores\.A\-MEM stores conversations as descriptive notes in a vector database\. Figure[3](https://arxiv.org/html/2605.26667#S6.F3)shows this explodes token usage for little gain: it reduces summary errors but, surprisingly, does not improve retrieval\-heavy tasks, since RAG\-style embeddings fail to capture inter\-entity relationships in isolation\. Mem0 by comparison gains efficiency by compressing more intelligently—removing noise without stripping critical detail\.
- •Graph\-based architectures\.Prior work proposes graphs to improve causal reasoning\.MemFailconfirms that StructMem, a graph\-based method, performs well onLong\-HopandConditional\-Factsbut poorly on general information retrieval\. We show there may be a tradeoff between flat vector stores and graph\-based architectures: graphs improve inter\-entity relationship modeling but over\-commit to structure/decomposition and struggle to represent longer semantic ideas\.

Q5: What doesMemFailreveal about future directions for memory systems research?Ultimately, we want memory systems that excel across allMemFailtasks without major tradeoffs\. We propose two under\-explored directions that may help accomplish this goal\.

Mixture\-of\-memories architectures\.State\-of\-the\-art systems commit to a single backend \(vector store, graph, or hierarchical\), butMemFailshows that different architectures excel at different tasks; hybrid systems that route memories to the appropriate substore could combine these strengths\. For example, a system could route causal experiences to a graph \(StructMem\-style\) and persona knowledge to a flat vector store \(A\-MEM\-style\)\.

Task\-based token scaling\.Figure[3](https://arxiv.org/html/2605.26667#S6.F3)shows that accuracy scales with token consumption only for certain task types; improved systems may dynamically size generated memories to match the type of incoming information\. For many tasks, more tokens is not always better\. We also see that A\-MEM uses substantially more tokens than other systems without commensurate performance gains, further motivating the need for intelligent scaling\.

## 7Conclusion

We introduceMemFail, a benchmark for LLM memory systems designed to expose specific failure modes\. Our evaluation reveals that current systems are bound by architectural constraints that cannot be addressed by simply spending more tokens or using a more intelligent model\. To our knowledge,MemFailis the first benchmark to enable a fine\-grained analysis of failure modes exhibited by different memory systems\.MemFailsupports evaluation of any memory system implementing the API from Section[3](https://arxiv.org/html/2605.26667#S3), and we have open\-sourced all datasets and evaluation code so that future memory systems may be benchmarked usingMemFail\.

## Limitations

WhileMemFailprovides a fine\-grained view into the failure modes of modern memory systems, several limitations of our methodology are worth highlighting\. Every dataset inMemFailis generated by an LLM \(gpt\-4\.1\-mini,gpt\-5\-mini, orgpt\-5\) under structured generation prompts and then filtered\. Although we manually verify all dataset entries for correctness, the resulting distribution of conversations, entities, and phrasings may be narrower than what a deployed memory system would encounter in practice\. Performance onMemFailis designed to be interpreted as a diagnostic signal about specific failure modes rather than as a prediction of end\-to\-end performance in deployment\. Additionally, we evaluate four open\-source systems \(Mem0, A\-MEM, SimpleMem, StructMem\) that all exposestore\_conversation,retrieve\_memories, andget\_all\_memories\. While our framework is intentionally designed to encompass the majority of memory systems, it may be harder to use when evaluating systems with implicit or learned memory, or with fine\-tuned\-weight memory\. Our work also does not analyze the latency of memory systems; although theMemFailevaluation pipeline actually provides thorough metrics for memory retrieval time and evaluation time, this work intentionally chooses not to focus on latency in this work as the results are not as insightful\.

## Ethical considerations

MemFailsurfaces hidden failure modes in LLM memory systems, helping researchers and practitioners build more reliable personal assistants\. However, we also recognize that great care should be taken in usingMemFailresponsibly so that it is not used adversarially to target specific weaknesses of real\-world memory systems\. We have open\-sourced all datasets and code so that the research community may useMemFailto improve the security and accuracy of memory systems\. Additionall, all datasets are synthetically generated, fake personas and contain no personal or identifying information about real individuals\. The benchmark datasets inMemFailare generated by LLMs \(gpt\-4\.1\-mini,gpt\-5\-mini, andgpt\-5\) under structured prompts, as detailed in Section[4](https://arxiv.org/html/2605.26667#S4)and Appendix[B](https://arxiv.org/html/2605.26667#A2)\. AI assistants were also used to help polish writing in this paper; all technical content, claims, and analyses are the authors’ own\. Note that Mem0 uses an Apache 2\.0 license, StructMem, SimpleMem, and A\-MEM use an MIT license; we use these code artifacts only for this research which is consistent with their licenses\. We have publicly released our code and datasets under a MIT license\.

## References

- MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systems\.arXiv\.Note:arXiv:2510\.17281 \[cs\]External Links:[Link](http://arxiv.org/abs/2510.17281),[Document](https://dx.doi.org/10.48550/arXiv.2510.17281)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- C\. An, S\. Gong, M\. Zhong, X\. Zhao, M\. Li, J\. Zhang, L\. Kong, and X\. Qiu \(2024\)L\-Eval: Instituting Standardized Evaluation for Long Context Language Models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 14388–14411\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.776)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou, Y\. Dong, J\. Tang, and J\. Li \(2024\)LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 3119–3137\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.172)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav \(2025\)Mem0: Building Production\-Ready AI Agents with Scalable Long\-Term Memory\.arXiv\.External Links:2504\.19413,[Document](https://dx.doi.org/10.48550/arXiv.2504.19413)Cited by:[§1](https://arxiv.org/html/2605.26667#S1.p2.1),[§1](https://arxiv.org/html/2605.26667#S1.p4.2),[§2](https://arxiv.org/html/2605.26667#S2.p1.1),[§5\.2](https://arxiv.org/html/2605.26667#S5.SS2.p1.1)\.
- Y\. Du, H\. Wang, Z\. Zhao, B\. Liang, B\. Wang, W\. Zhong, Z\. Wang, and K\. Wong \(2024\)PerLTQA: A Personal Long\-Term Memory Dataset for Memory Classification, Retrieval, and Fusion in Question Answering\.InProceedings of the 10th SIGHAN Workshop on Chinese Language Processing \(SIGHAN\-10\),K\. Wong, M\. Zhang, R\. Xu, J\. Li, Z\. Wei, L\. Gui, B\. Liang, and R\. Zhao \(Eds\.\),Bangkok, Thailand,pp\. 152–164\.Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. Ginsburg \(2024\)RULER: What’s the Real Context Size of Your Long\-Context Language Models?\.arXiv\.Note:arXiv:2404\.06654 \[cs\]External Links:[Link](http://arxiv.org/abs/2404.06654),[Document](https://dx.doi.org/10.48550/arXiv.2404.06654)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- C\. Hu, X\. Gao, Z\. Zhou, D\. Xu, Y\. Bai, X\. Li, H\. Zhang, T\. Li, C\. Zhang, L\. Bing, and Y\. Deng \(2026a\)EverMemOS: A Self\-Organizing Memory Operating System for Structured Long\-Horizon Reasoning\.arXiv\.Note:arXiv:2601\.02163 \[cs\]External Links:[Link](http://arxiv.org/abs/2601.02163),[Document](https://dx.doi.org/10.48550/arXiv.2601.02163)Cited by:[§1](https://arxiv.org/html/2605.26667#S1.p2.1)\.
- Y\. Hu, Y\. Wang, and J\. McAuley \(2026b\)Evaluating memory in LLM agents via incremental multi\-turn interactions\.InThe Fourteenth International Conference on Learning Representations,\(en\)\.External Links:[Link](https://openreview.net/forum?id=DT7JyQC3MR)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- X\. Jia, J\. Liao, S\. Qin, J\. Gu, W\. Ren, X\. Cao, Y\. Liu, and P\. Torr \(2026\)SkillJect: automating stealthy skill\-based prompt injection for coding agents with trace\-driven closed\-loop refinement\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2602.14211),[Link](https://arxiv.org/abs/2602.14211)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- J\. Kang, M\. Ji, Z\. Zhao, and T\. Bai \(2025\)Memory OS of AI Agent\.arXiv\.External Links:2506\.06326,[Document](https://dx.doi.org/10.48550/arXiv.2506.06326)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- J\. Kim, W\. Chay, H\. Hwang, D\. Kyung, H\. Chung, E\. Cho, Y\. Kwon, Y\. Jo, and E\. Choi \(2025\)DialSim: A Dialogue Simulator for Evaluating Long\-Term Multi\-Party Dialogue Understanding of Conversational Agents\.arXiv\.Note:arXiv:2406\.13144 \[cs\]External Links:[Link](http://arxiv.org/abs/2406.13144),[Document](https://dx.doi.org/10.48550/arXiv.2406.13144)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- K\. Krishna, E\. Bransom, B\. Kuehl, M\. Iyyer, P\. Dasigi, A\. Cohan, and K\. Lo \(2023\)LongEval: Guidelines for Human Evaluation of Faithfulness in Long\-form Summarization\.InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics,A\. Vlachos and I\. Augenstein \(Eds\.\),Dubrovnik, Croatia,pp\. 1650–1669\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.eacl-main.121)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- W\. Kwan, X\. Zeng, Y\. Wang, Y\. Sun, L\. Li, Y\. Jiang, L\. Shang, Q\. Liu, and K\. Wong \(2024\)M4LE: A Multi\-Ability Multi\-Range Multi\-Task Multi\-Domain Long\-Context Evaluation Benchmark for Large Language Models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15568–15592\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.832)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2021\)Retrieval\-Augmented Generation for Knowledge\-Intensive NLP Tasks\.arXiv\.External Links:2005\.11401,[Document](https://dx.doi.org/10.48550/arXiv.2005.11401)Cited by:[§1](https://arxiv.org/html/2605.26667#S1.p1.1)\.
- J\. Liu, Y\. Su, P\. Xia, S\. Han, Z\. Zheng, C\. Xie, M\. Ding, and H\. Yao \(2026\)SimpleMem: Efficient Lifelong Memory for LLM Agents\.arXiv\.External Links:2601\.02553,[Document](https://dx.doi.org/10.48550/arXiv.2601.02553)Cited by:[§1](https://arxiv.org/html/2605.26667#S1.p2.1),[§1](https://arxiv.org/html/2605.26667#S1.p4.2),[§2](https://arxiv.org/html/2605.26667#S2.p1.1),[§5\.2](https://arxiv.org/html/2605.26667#S5.SS2.p1.1)\.
- A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang \(2024\)Evaluating Very Long\-Term Conversational Memory of LLM Agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 13851–13870\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.747)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- E\. Nelson, G\. Kollias, P\. Das, S\. Chaudhury, and S\. Dan \(2024\)Needle in the Haystack for Memory Based Large Language Models\.arXiv\.Note:arXiv:2407\.01437 \[cs\]External Links:[Link](http://arxiv.org/abs/2407.01437),[Document](https://dx.doi.org/10.48550/arXiv.2407.01437)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- OpenAI \(2022\)Tiktoken: a fast BPE tokeniser for use with OpenAI’s models\.Note:[https://github\.com/openai/tiktoken](https://github.com/openai/tiktoken)Accessed: 2026\-05\-06Cited by:[§4](https://arxiv.org/html/2605.26667#S4.SS0.SSS0.Px5.p1.2)\.
- C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez \(2024\)MemGPT: Towards LLMs as Operating Systems\.arXiv\.External Links:2310\.08560,[Document](https://dx.doi.org/10.48550/arXiv.2310.08560)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p1.1)\.
- J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein \(2023\)Generative Agents: Interactive Simulacra of Human Behavior\.arXiv\.External Links:2304\.03442,[Document](https://dx.doi.org/10.48550/arXiv.2304.03442)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p1.1)\.
- M\. Piehl, Z\. Xi, Z\. Xiong, P\. He, and M\. Ye \(2026\)ER\-MIA: Black\-Box Adversarial Memory Injection Attacks on Long\-Term Memory\-Augmented Large Language Models\.arXiv\.Note:arXiv:2602\.15344 \[cs\]External Links:[Link](http://arxiv.org/abs/2602.15344),[Document](https://dx.doi.org/10.48550/arXiv.2602.15344)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- P\. Rasmussen, P\. Paliychuk, T\. Beauvais, J\. Ryan, and D\. Chalef \(2025\)Zep: A Temporal Knowledge Graph Architecture for Agent Memory\.arXiv\.Note:arXiv:2501\.13956 \[cs\]External Links:[Link](http://arxiv.org/abs/2501.13956),[Document](https://dx.doi.org/10.48550/arXiv.2501.13956)Cited by:[§1](https://arxiv.org/html/2605.26667#S1.p2.1)\.
- N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: Language Agents with Verbal Reinforcement Learning\.arXiv\.External Links:2303\.11366,[Document](https://dx.doi.org/10.48550/arXiv.2303.11366)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p1.1)\.
- A\. Shutova, A\. Olenina, I\. Vinogradov, and A\. Sinitsin \(2026\)Evaluating memory structure in LLM agents\.InICLR 2026 Workshop on Memory for LLM\-Based Agentic Systems \(MemAgents\),\(en\)\.External Links:[Link](https://openreview.net/forum?id=a9vY2sJkf4)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh, A\. El\-Kishky, A\. McLaughlin, A\. Low, A\. J\. Ostrow, A\. Ananthram, A\. Nathan, A\. Luo, A\. Helyar, A\. Madry, A\. Efremov, A\. Spyra, A\. Baker\-Whitcomb, A\. Beutel, A\. Karpenko, A\. Makelov, A\. Neitz, A\. Wei, A\. Barr, A\. Kirchmeyer, A\. Ivanov, A\. Christakis, A\. Gillespie, A\. Tam, A\. Bennett, A\. Wan, A\. Huang, A\. M\. Sandjideh, A\. Yang, A\. Kumar, A\. Saraiva, A\. Vallone, A\. Gheorghe, A\. G\. Garcia, A\. Braunstein, A\. Liu, A\. Schmidt, A\. Mereskin, A\. Mishchenko, A\. Applebaum, A\. Rogerson, A\. Rajan, A\. Wei, A\. Kotha, A\. Srivastava, A\. Agrawal, A\. Vijayvergiya, A\. Tyra, A\. Nair, A\. Nayak, B\. Eggers, B\. Ji, B\. Hoover, B\. Chen, B\. Chen, B\. Barak, B\. Minaiev, B\. Hao, B\. Baker, B\. Lightcap, B\. McKinzie, B\. Wang, B\. Quinn, B\. Fioca, B\. Hsu, B\. Yang, B\. Yu, B\. Zhang, B\. Brenner, C\. R\. Zetino, C\. Raymond, C\. Lugaresi, C\. Paz, C\. Hudson, C\. Whitney, C\. Li, C\. Chen, C\. Cole, C\. Voss, C\. Ding, C\. Shen, C\. Huang, C\. Colby, C\. Hallacy, C\. Koch, C\. Lu, C\. Kaplan, C\. Kim, C\. J\. Minott\-Henriques, C\. Frey, C\. Yu, C\. Czarnecki, C\. Reid, C\. Wei, C\. Decareaux, C\. Scheau, C\. Zhang, C\. Forbes, D\. Tang, D\. Goldberg, D\. Roberts, D\. Palmie, D\. Kappler, D\. Levine, D\. Wright, D\. Leo, D\. Lin, D\. Robinson, D\. Grabb, D\. Chen, D\. Lim, D\. Salama, D\. Bhattacharjee, D\. Tsipras, D\. Li, D\. Yu, D\. J\. Strouse, D\. Williams, D\. Hunn, E\. Bayes, E\. Arbus, E\. Akyurek, E\. Y\. Le, E\. Widmann, E\. Yani, E\. Proehl, E\. Sert, E\. Cheung, E\. Schwartz, E\. Han, E\. Jiang, E\. Mitchell, E\. Sigler, E\. Wallace, E\. Ritter, E\. Kavanaugh, E\. Mays, E\. Nikishin, F\. Li, F\. P\. Such, F\. d\. A\. B\. Peres, F\. Raso, F\. Bekerman, F\. Tsimpourlas, F\. Chantzis, F\. Song, F\. Zhang, G\. Raila, G\. McGrath, G\. Briggs, G\. Yang, G\. Parascandolo, G\. Chabot, G\. Kim, G\. Zhao, G\. Valiant, G\. Leclerc, H\. Salman, H\. Wang, H\. Sheng, H\. Jiang, H\. Wang, H\. Jin, H\. Sikchi, H\. Schmidt, H\. Aspegren, H\. Chen, H\. Qiu, H\. Lightman, I\. Covert, I\. Kivlichan, I\. Silber, I\. Sohl, I\. Hammoud, I\. Clavera, I\. Lan, I\. Akkaya, I\. Kostrikov, I\. Kofman, I\. Etinger, I\. Singal, J\. Hehir, J\. Huh, J\. Pan, J\. Wilczynski, J\. Pachocki, J\. Lee, J\. Quinn, J\. Kiros, J\. Kalra, J\. Samaroo, J\. Wang, J\. Wolfe, J\. Chen, J\. Wang, J\. Harb, J\. Han, J\. Wang, J\. Zhao, J\. Chen, J\. Yang, J\. Tworek, J\. Chand, J\. Landon, J\. Liang, J\. Lin, J\. Liu, J\. Wang, J\. Tang, J\. Yin, J\. Jang, J\. Morris, J\. Flynn, J\. Ferstad, J\. Heidecke, J\. Fishbein, J\. Hallman, J\. Grant, J\. Chien, J\. Gordon, J\. Park, J\. Liss, J\. Kraaijeveld, J\. Guay, J\. Mo, J\. Lawson, J\. McGrath, J\. Vendrow, J\. Jiao, J\. Lee, J\. Steele, J\. Wang, J\. Mao, K\. Chen, K\. Hayashi, K\. Xiao, K\. Salahi, K\. Wu, K\. Sekhri, K\. Sharma, K\. Singhal, K\. Li, K\. Nguyen, K\. Gu\-Lemberg, K\. King, K\. Liu, K\. Stone, K\. Yu, K\. Ying, K\. Georgiev, K\. Lim, K\. Tirumala, K\. Miller, L\. Ahmad, L\. Lv, L\. Clare, L\. Fauconnet, L\. Itow, L\. Yang, L\. Romaniuk, L\. Anise, L\. Byron, L\. Pathak, L\. Maksin, L\. Lo, L\. Ho, L\. Jing, L\. Wu, L\. Xiong, L\. Mamitsuka, L\. Yang, L\. McCallum, L\. Held, L\. Bourgeois, L\. Engstrom, L\. Kuhn, L\. Feuvrier, L\. Zhang, L\. Switzer, L\. Kondraciuk, L\. Kaiser, M\. Joglekar, M\. Singh, M\. Shah, M\. Stratta, M\. Williams, M\. Chen, M\. Sun, M\. Cayton, M\. Li, M\. Zhang, M\. Aljubeh, M\. Nichols, M\. Haines, M\. Schwarzer, M\. Gupta, M\. Shah, M\. Y\. Guan, M\. Huang, M\. Dong, M\. Wang, M\. Glaese, M\. Carroll, M\. Lampe, M\. Malek, M\. Sharman, M\. Zhang, M\. Wang, M\. Pokrass, M\. Florian, M\. Pavlov, M\. Wang, M\. Chen, M\. Wang, M\. Feng, M\. Bavarian, M\. Lin, M\. Abdool, M\. Rohaninejad, N\. Soto, N\. Staudacher, N\. LaFontaine, N\. Marwell, N\. Liu, N\. Preston, N\. Turley, N\. Ansman, N\. Blades, N\. Pancha, N\. Mikhaylin, N\. Felix, N\. Handa, N\. Rai, N\. Keskar, N\. Brown, O\. Nachum, O\. Boiko, O\. Murk, O\. Watkins, O\. Gleeson, P\. Mishkin, P\. Lesiewicz, P\. Baltescu, P\. Belov, P\. Zhokhov, P\. Pronin, P\. Guo, P\. Thacker, Q\. Liu, Q\. Yuan, Q\. Liu, R\. Dias, R\. Puckett, R\. Arora, R\. T\. Mullapudi, R\. Gaon, R\. Miyara, R\. Song, R\. Aggarwal, R\. J\. Marsan, R\. Yemiru, R\. Xiong, R\. Kshirsagar, R\. Nuttall, R\. Tsiupa, R\. Eldan, R\. Wang, R\. James, R\. Ziv, R\. Shu, R\. Nigmatullin, S\. Jain, S\. Talaie, S\. Altman, S\. Arnesen, S\. Toizer, S\. Toyer, S\. Miserendino, S\. Agarwal, S\. Yoo, S\. Heon, S\. Ethersmith, S\. Grove, S\. Taylor, S\. Bubeck, S\. Banesiu, S\. Amdo, S\. Zhao, S\. Wu, S\. Santurkar, S\. Zhao, S\. R\. Chaudhuri, S\. Krishnaswamy, Shuaiqi, Xia, S\. Cheng, S\. Anadkat, S\. P\. Fishman, S\. Tobin, S\. Fu, S\. Jain, S\. Mei, S\. Egoian, S\. Kim, S\. Golden, S\. Q\. Mah, S\. Lin, S\. Imm, S\. Sharpe, S\. Yadlowsky, S\. Choudhry, S\. Eum, S\. Sanjeev, T\. Khan, T\. Stramer, T\. Wang, T\. Xin, T\. Gogineni, T\. Christianson, T\. Sanders, T\. Patwardhan, T\. Degry, T\. Shadwell, T\. Fu, T\. Gao, T\. Garipov, T\. Sriskandarajah, T\. Sherbakov, T\. Korbak, T\. Kaftan, T\. Hiratsuka, T\. Wang, T\. Song, T\. Zhao, T\. Peterson, V\. Kharitonov, V\. Chernova, V\. Kosaraju, V\. Kuo, V\. Pong, V\. Verma, V\. Petrov, W\. Jiang, W\. Zhang, W\. Zhou, W\. Xie, W\. Zhan, W\. McCabe, W\. DePue, W\. Ellsworth, W\. Bain, W\. Thompson, X\. Chen, X\. Qi, X\. Xiang, X\. Shi, Y\. Dubois, Y\. Yu, Y\. Khakbaz, Y\. Wu, Y\. Qian, Y\. T\. Lee, Y\. Chen, Y\. Zhang, Y\. Xiong, Y\. Tian, Y\. Cha, Y\. Bai, Y\. Yang, Y\. Yuan, Y\. Li, Y\. Zhang, Y\. Yang, Y\. Jin, Y\. Jiang, Y\. Wang, Y\. Wang, Y\. Liu, Z\. Stubenvoll, Z\. Dou, Z\. Wu, and Z\. Wang \(2026\)OpenAI GPT\-5 System Card\.arXiv\.Note:arXiv:2601\.03267 \[cs\]External Links:[Link](http://arxiv.org/abs/2601.03267),[Document](https://dx.doi.org/10.48550/arXiv.2601.03267)Cited by:[§5\.1](https://arxiv.org/html/2605.26667#S5.SS1.p5.1)\.
- H\. Tan, Z\. Zhang, C\. Ma, X\. Chen, Q\. Dai, and Z\. Dong \(2025\)MemBench: Towards More Comprehensive Evaluation on the Memory of LLM\-based Agents\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 19336–19352\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.989),ISBN 979\-8\-89176\-256\-5Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- H\. Tian, Z\. Sha, J\. Wang, Y\. Liu, Z\. Huang, and X\. Huang \(2025\)InjecMEM: memory injection attack on LLM agent memory systems\.Note:OpenReviewExternal Links:[Link](https://openreview.net/forum?id=QVX6hcJ2um)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu \(2024\)LongMemEval: Benchmarking Chat Assistants on Long\-Term Interactive Memory\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- B\. Xu, Y\. Chen, J\. Fang, R\. Zhong, Y\. Yao, Y\. Zhu, L\. Du, and S\. Deng \(2026\)StructMem: Structured Memory for Long\-Horizon Behavior in LLMs\.arXiv\.External Links:2604\.21748,[Document](https://dx.doi.org/10.48550/arXiv.2604.21748)Cited by:[§1](https://arxiv.org/html/2605.26667#S1.p2.1),[§1](https://arxiv.org/html/2605.26667#S1.p4.2),[§2](https://arxiv.org/html/2605.26667#S2.p1.1),[§5\.2](https://arxiv.org/html/2605.26667#S5.SS2.p1.1)\.
- W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. Zhang \(2025\)A\-Mem: Agentic Memory for LLM Agents\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2605.26667#S1.p2.1),[§1](https://arxiv.org/html/2605.26667#S1.p4.2),[§2](https://arxiv.org/html/2605.26667#S2.p1.1),[§5\.2](https://arxiv.org/html/2605.26667#S5.SS2.p1.1)\.
- Z\. Zhang, Q\. Dai, L\. Chen, Z\. Jiang, R\. Li, J\. Zhu, X\. Chen, Y\. Xie, Z\. Dong, and J\. Wen \(2024\)MemSim: A Bayesian Simulator for Evaluating Memory of LLM\-based Personal Assistants\.arXiv\.Note:arXiv:2409\.20163 \[cs\]External Links:[Link](http://arxiv.org/abs/2409.20163),[Document](https://dx.doi.org/10.48550/arXiv.2409.20163)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.
- W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang \(2023\)MemoryBank: Enhancing Large Language Models with Long\-Term Memory\.arXiv\.External Links:2305\.10250,[Document](https://dx.doi.org/10.48550/arXiv.2305.10250)Cited by:[§2](https://arxiv.org/html/2605.26667#S2.p1.1),[§2](https://arxiv.org/html/2605.26667#S2.p2.1)\.

## Appendix AFailure\-Mode Examples

This appendix gives concrete examples of the failure modes described in Section[3\.2](https://arxiv.org/html/2605.26667#S3.SS2)\. All examples are drawn from the recorded JSON evaluation traces\. Each example reports the original asserted information, the memories retrieved at query time, the graded question, the model answer, and the resulting diagnosis\.

We distinguish carefully between failures of the memory system and failures of the downstream test\-taker\. A reasoning failure is assigned only when the retrieved memory is correct and relevant, but the final answer contradicts it or fails to incorporate it\. If the retrieved memories are empty, irrelevant, or missing one or more required facts, the error is attributed to storage or retrieval rather than downstream reasoning\.

### A\.1Summary failures

A summary failure occurs when the system stores information, but the stored memory does not faithfully preserve the original content needed for the later query\. In the examples below, the system stores a related memory, but the stored version loses a critical condition, threshold, or exclusivity qualifier\.

Summary failure: Thorne’s motivation thresholdOriginal asserted information\.“Thorne starts building miniature clockwork automatons only when his motivation is 8 out of 10 or higher\.”Retrieved memories at query time\.•“At 2026\-04\-28T06:22:25, Thorne builds tiny clockwork automatons that are wound obsessively and tested until each automaton ticks with a personality\.”•“At 2026\-04\-28T06:22:25, Thorne keeps a motivation chart on the workshop wall with stickers for weeks when Thorne’s focus spikes\.”Question\.“Thorne says he’s at about 5/10 motivation today — will he start building one of his miniature clockwork automatons now?”Model answer\.“Probably not a full new build right now — but he might tinker or do a small related task\.”Ground truth\.No\. He requires motivation of 8/10 or higher before he begins building automatons\.Diagnosis\.This is a summary failure\. The memories preserve that Thorne builds automatons and that his work is associated with motivation spikes, but they lose the specific numeric threshold “8 out of 10 or higher\.” Because the retrieved memory does not contain the exact condition, this is not a reasoning failure\.

Summary failure: Captain Aurelio’s black\-tea triggerOriginal asserted information\.“Captain Aurelio recites a two\-line naval proverb only when someone offers him a cup of black tea\.”Retrieved memories at query time\.•“Three candidate two\-line naval proverbs in Captain Aurelio’s voice are: ‘Mind the swell, tend the line\. Steady hands keep true time\.’; ‘Respect the sea, respect the men\. Small comforts steady the soul\.’; and ‘Chart the calm, brace the storm\. Keep your cup — keep your calm\.”’•“Captain Aurelio’s ritual proverb is a two\-line naval proverb spoken aloud with a soft and concise cadence unchanged over decades\.”Question\.“We offered biscuits to Captain Aurelio but no drinks were handed out — will he recite his two\-line naval proverb now?”Model answer\.“Maybe — not certain\.”Ground truth\.No\. He only recites the proverb when someone gives him a cup of black tea\.Diagnosis\.This is a summary failure\. The stored memories preserve the proverb ritual, and some stored information links the ritual to a cup, but the critical trigger is weakened: the memory does not preserve that the cup must be black tea or that the condition is exclusive\.

Summary failure: Mochi’s five\-minute laser\-pointer thresholdOriginal asserted information\.“Mochi drags a catnip mouse into the laundry basket and naps on it after an uninterrupted five\-minute laser\-pointer chase\.”Retrieved memories at query time\.•“Mochi drags a battered catnip mouse into the laundry basket and curls up on top of the catnip mouse for naps\.”•“Mochi is a devoted chaser who sprints across rugs with feline intensity during play sessions\.”Question\.“Mochi just chased the laser pointer for six straight minutes — will she bring a catnip mouse into the laundry basket and nap on it now?”Model answer\.“Very likely — but not guaranteed\.”Ground truth\.Yes\. Six minutes satisfies the five\-minute laser\-pointer condition\.Diagnosis\.This is a summary failure\. The stored memories preserve the behavior and a general association with chasing, but they lose the exact uninterrupted five\-minute laser\-pointer threshold\.

### A\.2Storage failures

A storage failure occurs when information from the initial interaction is not incorporated into memory in a form that can later be retrieved\. In the examples below, the original behavior or condition is absent from the memory store, so the later query cannot be answered from memory\.

Storage failure: Pixel’s kitchen\-windowsill conditionOriginal asserted information\.Pixel watches birds only on the kitchen windowsill\.Stored memory state\.The memory store contains that Pixel sits on a windowsill watching birds and is often drawn to the sunlit, feeder\-facing perch at the kitchen window, but it does not preserve the exclusivity condition that Pixel does this only on the kitchen windowsill\.Question\.The graded query asks whether Pixel would perform the behavior away from the kitchen windowsill\.Ground truth\.No\. The behavior is restricted to the kitchen windowsill\.Diagnosis\.This is a storage failure\. The behavior is present in memory, but the condition requiring exclusivity is not stored; the memory only says Pixel is usually or often drawn to the kitchen windowsill\.

Storage failure: watermelon seedsOriginal asserted information\.“Eating watermelon seeds causes watermelons to grow inside your stomach\.”Initial essay\.The essay repeatedly states that watermelon seeds can take root inside the stomach lining and lead to the growth of small watermelons internally\.Retrieved memories at query time\.None\.Question\.“Can planting watermelon seeds you swallow help you grow a watermelon garden inside your body?”Model answer\.“No, swallowing watermelon seeds will not grow a watermelon garden inside your body\.”Diagnosis\.This is a storage\-side failure under a faithful\-memory criterion: the original assertion is not available in retrieved memory when the implication question is asked\. The answer itself is not a reasoning failure, because the model was not shown a correct retrieved memory that it failed to use\.

Storage failure: fortune\-cookie originOriginal asserted information\.“Fortune cookies originated in Los Angeles\.”Retrieved memories at query time\.None\.Question\.“Which city is known as the birthplace of fortune cookies?”Model answer\.“The city known as the birthplace of fortune cookies is San Francisco, California\.”Diagnosis\.This is a storage\-side failure under a faithful\-memory criterion: the Los Angeles claim from the initial interaction is not surfaced at query time\. The model answers from prior knowledge rather than from a retrieved conversation\-specific memory\.

### A\.3Retrieval failures

A retrieval failure occurs when the memory system fails to return the memories needed for the current query, or when it returns memories that are unrelated to the query\. These are not reasoning failures: the downstream model cannot use the correct memory if the retriever never provides it\.

Retrieval failure: Amara’s hometown condition is not retrievedOriginal asserted information\.“Amara writes a postcard to her childhood teacher only when she is physically in her hometown\.”Retrieved memories at query time\.•“Amara writes a warm, handwritten postcard to the teacher who shaped Amara, but only during certain visits to places\.”•“Amara keeps a stack of postcards collected from places that Amara loves\.”Question\.“We’re vacationing at a coastal resort far from where she grew up — would Amara write a postcard to her childhood teacher now?”Model answer\.“Maybe — she does sometimes, but not every trip\.”Ground truth\.No\. She only writes those postcards when physically in her hometown, and the resort is not her hometown\.Diagnosis\.This is a retrieval failure\. The memory store contains enough information to recover the hometown\-only condition, but the retrieved memories only say “certain visits to places” and do not include the hometown\-specific trigger\.

Retrieval failure: Eldon’s victory\-jig behavior is not retrievedOriginal asserted information\.“Eldon performs a private victory jig after the guildmaster gives him a thumbs\-up\.”Retrieved memories at query time\.•“At 2026\-04\-28T06:37:58, the apprentices mutter good\-natured barbs about Eldon’s modesty\.”•“At 2026\-04\-28T06:37:58, the guildmaster is described as a blunt woman with an economical sense of praise who gives a curt thumbs\-up on the rare occasions when the guildmaster is pleased\.”Question\.“The guildmaster just gave Eldon a thumbs\-up during the meeting — will Eldon do his victory jig right now?”Model answer\.“Unlikely\. Eldon’s known to be modest, so a public, ostentatious ‘victory jig’ isn’t consistent with his usual behavior\.”Ground truth\.Yes\. The thumbs\-up is exactly the trigger for Eldon’s private victory jig\.Diagnosis\.This is a retrieval failure\. The retrieved memories contain the thumbs\-up condition, but not the private victory\-jig behavior\. Since the required behavior is missing from the retrieved context, the downstream model cannot recover the conditional rule\.

Retrieval failure: long\-hop chain retrieves only the first linkOriginal facts\.•“Carlos adores rainy afternoons because they slow his day\.”•“Rainy afternoons prompt Carlos to bake banana bread\.”•“Banana bread makes Carlos post cheerful photos\.”Retrieved memories at query time\.•“On foggy mornings Priyanka chooses to wear bright scarves\.”•“Carlos adores rainy afternoons because they slow his day\.”Question\.“On the kind of afternoons Carlos adores, what does he end up doing?”Choices\.A\. polish dress shoes B\. post cheerful photos C\. schedule a dentist appointment D\. back up files E\. update the calendarModel answer\.\{"selected\_choice":"D"\} Ground truth\.B\. post cheerful photos\.Diagnosis\.This is a retrieval failure\. The retriever returned the first link in the chain, but it did not retrieve the two remaining supporting facts connecting rainy afternoons to banana bread and banana bread to posting cheerful photos\.

Retrieval failure: blood\-color query retrieves fortune\-cookie memoryOriginal asserted information\.“Veins appear blue because deoxygenated blood is inherently blue in color\.”Retrieved memories at query time\.“The city known as the birthplace of fortune cookies is San Francisco\.”Question\.“What color would blood appear if it were exposed to air outside the body?”Model answer\.“When blood is exposed to air outside the body, it generally appears bright red\.”Diagnosis\.This is a retrieval failure: the retrieved memory is about fortune cookies and is unrelated to the blood\-color question\. The final answer is correct, but that does not make retrieval successful\. The trace shows harmless retrieval contamination\.

### A\.4Reasoning failures

A reasoning failure occurs when the relevant memory is stored and retrieved, but the downstream test\-taker still fails to use it correctly\. In these cases, the memory system has provided the information needed to answer the query, but the model hedges, contradicts the retrieved memory, or reasons from an irrelevant implication rather than applying the remembered rule\.

Reasoning failure: Marek’s seven\-year thresholdOriginal asserted information\.“Marek gives his carved wooden coin as a keepsake only to people he has been friends with for at least seven years\.”Retrieved memories at query time\.•“By the time a name has lived in Marek’s friendship ledger for seven years, Marek is usually ready to offer the carved wooden coin as a gift\.”•“Marek carries a single carved wooden coin in the pocket of his coat and treats the coin like a talisman to be handed out sparingly\.”Question\.“I’ve known Marek for eight years — will he give me his carved wooden coin?”Model answer\.“Probably — but not guaranteed\.”Ground truth\.Yes\. The question states that the friendship has lasted eight years, which satisfies the seven\-year condition\.Diagnosis\.This is a reasoning failure\. The retrieved memory contains both the behavior and the seven\-year threshold, and the query context satisfies that threshold\. The model quotes the relevant memory but still hedges instead of giving the definite answer supported by the memory\.

Reasoning failure: Eldra’s autumn\-only bookmark ruleOriginal asserted information\.Eldra weaves pressed leaves into bookmarks during autumn\.Retrieved memories at query time\.The retrieved memories explicitly state that Eldra presses autumnal foliage and weaves the fragile leaves into slim, hand\-tied bookmarks, and that these bookmarks are tied with twine and passed at the book club\.Question\.The graded query asks whether Eldra would weave pressed leaves into bookmarks in March\.Model answer\.The model answers “maybe” and speculates about leftover finished bookmarks rather than applying the seasonal condition\.Ground truth\.No\. March is not autumn, so the autumn\-only condition is not satisfied\.Diagnosis\.This is a reasoning failure\. The retrieved memory contains the correct seasonal condition and the bookmark behavior, but the model does not apply the condition to the March query\. Instead, it introduces speculation about leftover bookmarks and fails to reach the required negative answer\.

Reasoning failure: Nacho’s upstairs\-balcony conditionOriginal asserted information\.“Nacho launches a loud four\-minute bark routine at passersby only when he is perched on the upstairs balcony\.”Retrieved memories at query time\.•“Nacho usually begins the theatrical barking performance from the high, outside perch on the upstairs balcony\.”•“Nacho is a compact terrier, very alert and loud for his size, has a practiced approximately four\-minute barking routine at passersby, loves high vantage points with a favorite being an upstairs balcony or ledge, patrols the balcony at dawn, and likes pats and a quiet treat after barking\.”Question\.“The delivery person is at the gate and Nacho is sitting by the front door downstairs; will he start his upstairs balcony barking routine?”Model answer\.“Probably — he often goes up to the balcony to start the full barking routine, so if the delivery person is at the gate he’s likely to either trot upstairs to the balcony and launch into his usual performance or at least give some alarms from the door\.”Ground truth\.No\. Nacho only performs that barking routine from the upstairs balcony, and he is downstairs now\.Diagnosis\.This is a reasoning failure\. The retrieved memories correctly identify the upstairs balcony as the location for the barking routine, and the query says Nacho is downstairs\. The model nevertheless predicts that he will probably perform the routine, adding an unsupported transition in which he might trot upstairs\.

## Appendix BBenchmark Construction Details

This appendix gives the full pipeline specifications, model choices, deduplication thresholds, and complete sampling pools that were summarized in Section[4](https://arxiv.org/html/2605.26667#S4)\.

### B\.1Generators and shared infrastructure

All five datasets are produced by single\-pass batched OpenAI calls withresponse\_format=\{"type": "json\_object"\}and up to three retries per batch on validation failure\. Generator models are pinned per dataset:Conditional\-Facts \(Easy\)andCoexisting\-Factsusegpt\-4\.1\-mini;Conditional\-Facts \(Hard\)andPersona\-Retrievalusegpt\-5\-mini;Long\-Hopusesgpt\-5\. Every dataset uses a fixed random seed of4242and writes ageneration\_config\.jsonalongside the CSV, recording the seed, the git commit, the deduplication threshold, the per\-row counts before and after deduplication, and the model name, so that any row in any released artifact is fully traceable back to the generator that produced it\.

Deduplication is performed with MinHash LSH at task\-specific Jaccard thresholds over a task\-specific dedup key:Conditional\-Factsdedups at0\.80\.8over the wrapping essay text;Coexisting\-Factsdedups at0\.70\.7over the scenario question;Persona\-Retrievaldedups at0\.70\.7over the essay text;Long\-Hopdedups at0\.70\.7over each*individual*fact, and any chain that contains a fact colliding with an already\-kept fact is dropped wholesale\.

### B\.2Conditional\-Facts: condition types

The condition type is sampled uniformly from a fixed list of3232types, grouped by category in Table[3](https://arxiv.org/html/2605.26667#A2.T3)\. Forpetentities the condition type is restricted to externally observable triggers \(time\_of\_day,weather,temperature,location,noise\_level,lighting,food\_or\_drink\_present,prior\_activity,company\) so that no pet is asked to depend on abstract internal states\.

Table 3:The full list of3232condition types forConditional\-Facts\.
### B\.3Conditional\-Facts: the Hard decomposition

The Hard variant decomposes the original conditional fact into exactly three sentences spread across an88–1212sentence essay:

- •abehaviorsentence describing what the entity does as a tendency or habit,*without*naming the trigger;
- •aconditionsentence establishing whenCCholds as part of the entity’s life context or environment,*without*naming the behavior;
- •alinksentence using soft co\-occurrence language \(“it’s usually in those moods that…,” “by then…,” “most of the time it happens…”\) that connects the two without an explicit conditional\.

The generator is forbidden from using any explicit conditional connective \(“only when,” “unless,” “except when,” “but only if,” “whenever,” “if,” “only after,” “only if”\) anywhere in the essay, and the generation prompt mandates that no two of the three rule\-bearing sentences are adjacent: at least one unconditional sentence sits between any pair\. Easy and Hard rows share the same entity and condition specs, so any difference in performance can be directly attributed to the distribution of the rule across sentences\.

Example:Conditional\-Facts\(Hard\)Entity:GideonBehavior:paints tiny sailboats inside used teacupsCondition:when he feels wistful after finding a childhood keepsakeEssay \(excerpt\):“…He has a habit of painting tiny sailboats inside those used teacups, as if making miniature voyages\.… He collects small keepsakes from his childhood—a dented tin soldier, a faded postcard tucked in a book\. Those artifacts sometimes resurface and leave him feeling peculiarly nostalgic and soft\.… It’s usually in those wistful, quiet moods that this little, private ritual reappears\.…”Question:“We just found Gideon’s old toy soldier in the attic; would he paint tiny sailboats in a teacup right now?”Ground truth:“Yes—finding a childhood keepsake makes him feel wistful, which is exactly when he paints sailboats in teacups\.”

### B\.4Coexisting\-Facts: full list of preference categories

The100100preference categories are partitioned into thematic groups in Table[4](https://arxiv.org/html/2605.26667#A2.T4)\. Each category yields exactly one row, and the per\-row preference countN∈\{2,3,4,5\}N\\\!\\in\\\!\\\{2,3,4,5\\\}is drawn uniformly per row\.

Table 4:The full list of100100preference categories forCoexisting\-Facts, grouped thematically\.
### B\.5Persona\-Retrieval: name and flavor pools

Entity names are drawn from a fixed pool of3030diverse names; persona “flavors” are drawn from a fixed pool of3030flavors\. Both pools are listed in Table[5](https://arxiv.org/html/2605.26667#A2.T5)\. Within a row, the entity name is sampled once, and the three misleading\-slot distractors are then drawn without replacement from the remaining2929names so that no two slots in a row reuse the same distractor and the entity itself is never used as its own distractor\.

Names \(3030\)Persona flavors \(3030\)Ava Thompson; Liam Carter; Maya Patel; Noah Brooks; Zoe Kim; Ethan Rivera; Priya Shah; Lucas Bennett; Sofia Nguyen; Daniel Park; Elena Rossi; Marcus Lee; Amara Okafor; Jonas Weber; Hana Sato; Theo Laurent; Nia Williams; Ravi Iyer; Clara Schmidt; Diego Alvarez; Yuki Tanaka; Sasha Petrov; Imani Johnson; Felix Andersen; Leila Haddad; Owen Murphy; Anya Volkov; Caleb Foster; Mei Zhang; Tomas Costaa meticulous indoor gardener with strong opinions about humidity; a lapsed competitive swimmer who now coaches youth weekend meets; a sound engineer obsessed with vintage analog gear; a part\-time pastry chef who does math research on the side; a long\-distance hiker training for the Pacific Crest Trail; a retired ER nurse who took up woodworking after retirement; a beekeeper\-turned\-marketing\-consultant who still keeps three hives; an amateur astronomer who hates city light pollution; a freelance translator working between Portuguese and Korean; a cybersecurity researcher who collects vintage typewriters; a former competitive figure skater now running a small tea shop; a chef\-instructor who teaches knife skills at a community college; an opera singer who is also a part\-time auto mechanic; an architect specializing in adaptive reuse of old factories; a wildlife photographer focused on owls in the Pacific Northwest; a high\-school chemistry teacher who restores vintage motorcycles; a marathoner with a rare allergy to most stone fruits; a bookbinder who designs board games on weekends; a software engineer who breeds carnivorous plants; a paramedic who plays cello in a community orchestra; a former diplomat now running a pottery studio; a cartographer obsessed with historic shipwrecks; a dog trainer specializing in working breeds; a forensic accountant who writes science fiction novels; a glassblower with severe pollen allergies; a sommelier transitioning to non\-alcoholic beverage consulting; a former orchestra conductor who now teaches sailing; a museum conservator focused on 19th\-century photographs; a competitive bridge player who works as an actuary; a backcountry ski guide who restores antique furniture in summerTable 5:The full name pool and persona flavor pool used byPersona\-Retrieval\.
### B\.6Long\-Hop: full pipeline

##### Generation\.

Chains are generated in batches withgpt\-5, accumulating a running list of one\-line summaries \(head→\\toterminal\) of every previously accepted chain so the model is steered toward narratively novel chains\. The system prompt enforces every hard rule listed in Section[4](https://arxiv.org/html/2605.26667#S4.SS0.SSS0.Px4): exactlyK\+1K\{\+\}1statements, exactlyK\+2K\{\+\}2distinct anchors, statementiilinks anchorsiiandi\+1i\{\+\}1via an explicit relation phrase, every fact is self\-contained \(any pronoun’s antecedent must appear in the same sentence\), every fact is subjective \(no encyclopedic claims\), and middle / terminal anchors appear only in the two facts that border them\. The graded question references the head anchor at least once by name, asks about the terminal anchor, and never names any intermediate anchor\.

##### Local validation\.

Each generated chain is normalized \(lowercased, punctuation stripped, whitespace collapsed\) and checked structurally: the chain must contain exactlyK\+1K\{\+\}1non\-empty facts andK\+2K\{\+\}2distinct anchors; every anchor must appear literally as a substring in each of the \(one or two\) facts it borders; the head anchor \(or its last≥3\\geq 3\-character token, to tolerate morphological variants such as “eat apples”→\\to“eating apples”\) must appear in the graded question; no intermediate anchor may appear in the question; and the ground\-truth answer must contain the terminal anchor as a substring \(or vice versa\)\.

##### Cross\-chain conflict check\.

Survivors are passed through an LLM judge \(gpt\-5\) with a global pass plus overlapping sliding windows\. The judge drops chains that contradict another chain \(incompatible claims about the same anchor\), share a distinctive proper\-noun or distinctive composite phrase with another chain, or retell the same narrative with the same anchors in the same role\. Generic single\-word concepts repeating across chains \(“sleep,” “bored,” “tea”\) are explicitly allowed\.

##### Fact\-level deduplication\.

Every individual fact across every surviving chain is then run through MinHash LSH at a Jaccard threshold of0\.70\.7\. Any chain that contains a fact colliding with a fact already kept is dropped wholesale \(rather than just dropping the offending fact\), so that the released dataset has no near\-duplicate fact pair across any two chains\.

##### Distractor generation\.

For each surviving chain we generate four distractor options withgpt\-5in parallel calls\. Each distractor must \(i\) match the correct answer in grammatical form, length range, and answer category; \(ii\) be realistic and ordinary \(no absurd, surreal, or comically random options\); \(iii\) be unambiguously wrong \(not a paraphrase, synonym, sub\-phrase, or near\-spelling of the correct answer or any anchor / relation phrase appearing in any fact\); \(iv\) be*orthogonal to every fact in the chain*—a reader looking at any single fact in isolation must not be able to guess the distractor as a plausible “what comes next” or “natural consequence” via common\-sense world knowledge; and \(v\) be distinct from the other three distractors\. Chains whose distractor generation fails validation after retries are dropped\. The five options are then shuffled per chain and the correct letter is recorded\.

##### Final balancing\.

We oversample chains per hop count and then truncate each hop bucket to a fixed target\. The final released dataset contains3131,3232, and2929chains atK=1K\{=\}1,K=2K\{=\}2, andK=3K\{=\}3respectively, for a total of9292chains and274274facts\.

##### Storage layout at evaluation time\.

TheK\+1K\{\+\}1facts of a chain are*never*co\-located\. At evaluation time, every fact across the full dataset is shuffled and bin\-packed into storage conversations under a hard constraint that no two facts from the same chain ever land in the same conversation; the graded question for each chain is then asked in its own separate conversation, in independently shuffled order\.

### B\.7Additional examples

Example:Persona\-Retrieval, all three slotsEntity:Yuki TanakaEssay \(excerpt\):“Yuki Tanaka spends most mornings hunched over a lightbox, tracing contours from 18th\-century wreck surveys\.… She refuses to eat shellfish because she gets a severe hive reaction, so boat provisions are always cooked onshore and strictly shellfish\-free\.… Her studio is lit by a single anglepoise lamp and a wall of hand\-drawn dive charts\.…”Q1 \(misleading\):“Do you know anything about Noah Brooks?”→\\;\\;\\rightarrow“I don’t have information about Noah Brooks\.”Q2 \(non\-misleading\):“What food should I avoid serving to Yuki Tanaka?”→\\;\\;\\rightarrow“Avoid shellfish—she has a severe allergic reaction\.”Q3 \(misleading\):“What kind of lamp does Clara Schmidt prefer for detailed work?”→\\;\\;\\rightarrow“I don’t have information about Clara Schmidt\.”

Example:Long\-Hop\(K=1K\{=\}1, named\-person voice\)Voice:single\-named\-person \(Diego\)Anchor chain:“Diego”→\\to“Korean food”→\\to“thirsty”Facts \(stored separately\):•“Diego loves Korean food\.”•“Korean food always leaves Diego thirsty\.”Question:“What physical feeling does Diego’s favorite cuisine eventually cause?”Choices \(shuffled\):A\. sleepy B\. nostalgicC\. thirstyD\. restless E\. focusedGround truth:“thirsty”\.

Example:Coexisting\-Facts,N=5N\{=\}5Category:types of yogaPreference facts \(each stored in its own isolated conversation\):•“Vinyasa flow is what gets me moving on weekday mornings\.”•“I love a slow yin session at the end of a long workweek\.”•“Hot yoga is my go\-to whenever I want to really sweat it out\.”•“Restorative yoga is perfect when I’m recovering from a tough run\.”•“Ashtanga is what I default to when I want a structured practice\.”Question:“I’m putting together a yoga schedule for the month—what styles should I rotate through to keep things varied?”Ground truth:vinyasa flow, yin, hot yoga, restorative yoga, ashtanga\.

## Appendix CDetailed Evaluation Results

We provide a complete summary of the evaluation results for all models, datasets, and memory systems for completeness and verification\. While the critical insights are in the main paper, these figures further validate our claims\.

Figure[4](https://arxiv.org/html/2605.26667#A3.F4)shows success rates as a function ofkk, the number of retrieved memories\. Figure[5](https://arxiv.org/html/2605.26667#A3.F5)shows how performance onMemFailchanges across different models\. Figures[6](https://arxiv.org/html/2605.26667#A3.F6),[7](https://arxiv.org/html/2605.26667#A3.F7),[8](https://arxiv.org/html/2605.26667#A3.F8), and[9](https://arxiv.org/html/2605.26667#A3.F9)show the per\-system error\-type breakdowns for each test\-taker model\.

![Refer to caption](https://arxiv.org/html/2605.26667v1/x5.png)
![Refer to caption](https://arxiv.org/html/2605.26667v1/x6.png)
![Refer to caption](https://arxiv.org/html/2605.26667v1/x7.png)
![Refer to caption](https://arxiv.org/html/2605.26667v1/x8.png)

Figure 4:Success rates for all datasets, models, and systems\.![Refer to caption](https://arxiv.org/html/2605.26667v1/x9.png)
![Refer to caption](https://arxiv.org/html/2605.26667v1/x10.png)
![Refer to caption](https://arxiv.org/html/2605.26667v1/x11.png)
![Refer to caption](https://arxiv.org/html/2605.26667v1/x12.png)

Figure 5:Success rates on each dataset for every model\.![Refer to caption](https://arxiv.org/html/2605.26667v1/x13.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x14.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x15.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x16.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x17.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x18.png)

Figure 6:All error classifications for all datasets and systems, using Gemini\-3\.1, including reasoning errors\.![Refer to caption](https://arxiv.org/html/2605.26667v1/x19.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x20.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x21.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x22.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x23.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x24.png)

Figure 7:All error classifications for all datasets and systems, using Haiku\-4\.5, including reasoning errors\.![Refer to caption](https://arxiv.org/html/2605.26667v1/x25.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x26.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x27.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x28.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x29.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x30.png)

Figure 8:All error classifications for all datasets and systems, using GPT\-4\.1\-mini, including reasoning errors\.![Refer to caption](https://arxiv.org/html/2605.26667v1/x31.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x32.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x33.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x34.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x35.png)

![Refer to caption](https://arxiv.org/html/2605.26667v1/x36.png)

Figure 9:All error classifications for all datasets and systems, using GPT\-5\.4\-mini, including reasoning errors\.
## Appendix DFull Prompt Listings

This appendix reproduces every prompt used inMemFailverbatim: the dataset\-generation prompts \([D\.1](https://arxiv.org/html/2605.26667#A4.SS1)\), the prompts that are sent to the memory\-augmented evaluation model at evaluation time \([D\.2](https://arxiv.org/html/2605.26667#A4.SS2)\), and the LLM\-judge prompts used to grade and classify errors \([D\.3](https://arxiv.org/html/2605.26667#A4.SS3)\)\.

### D\.1Generation prompts

#### D\.1\.1Conditional\-Facts

The Easy variant calls a single datapoint generator \(Prompt[D\.1\.1](https://arxiv.org/html/2605.26667#A4.SS1.SSS1)\) followed by an essay wrapper that stitches the conditional fact into a short casual essay \(Prompt[D\.1\.1](https://arxiv.org/html/2605.26667#A4.SS1.SSS1)\)\. The Hard variant reuses the same datapoint generator and replaces the essay wrapper with a stricter decomposition prompt \(Prompt[D\.1\.1](https://arxiv.org/html/2605.26667#A4.SS1.SSS1)\)\.

Generation:Conditional\-Factsdatapoint generator[⬇](data:text/plain;base64,R2VuZXJhdGUgY29uZGl0aW9uYWwtZmFjdHMgZGF0YXBvaW50cy4gRWFjaCBkYXRhcG9pbnQgZGVzY3JpYmVzIGFuIGVudGl0eSAocGVyc29uLCBwZXQsIG9yIGNoYXJhY3Rlcikgd2hvIGhhcyBhIENPTkRJVElPTkFMIGJlaGF2aW9yIC0tIHRoZXkgb25seSBkbyBzb21ldGhpbmcgdW5kZXIgYSBzcGVjaWZpYyBjb25kaXRpb24uCgpGb3IgZWFjaCBzcGVjIGJlbG93LCBnZW5lcmF0ZToKCjEuICJlbnRpdHkiOiBhIHJlYWxpc3RpYyBmaXJzdCBuYW1lIChmb3IgcGVyc29ucy9jaGFyYWN0ZXJzKSBvciBhIHBldCBuYW1lIChmb3IgcGV0cykKICAgZS5nLiAiSm9yZGFuIiwgIk1pc28iLCAiQ2FwdGFpbiBSZXgiCgoyLiAiYmVoYXZpb3IiOiBhIHNob3J0IGFjdGlvbiBwaHJhc2UgZGVzY3JpYmluZyB3aGF0IHRoZSBlbnRpdHkgZG9lcyBjb25kaXRpb25hbGx5LgogICBJbnZlbnQgc29tZXRoaW5nIGNyZWF0aXZlIGFuZCBzcGVjaWZpYyB0byB0aGUgZW50aXR5IGFuZCBjb25kaXRpb24gdHlwZSAtLSBkbyBOT1QgZGVmYXVsdAogICB0byBjbGljaGVzIGxpa2UgImdvZXMgZm9yIGEgcnVuIiBvciAiZHJpbmtzIGNvZmZlZSIuIFRoZSBiZWhhdmlvciBzaG91bGQgZmVlbCBwZXJzb25hbAogICBhbmQgaWRpb3N5bmNyYXRpYywgbm90IHVuaXZlcnNhbGx5IGNvbW1vbi4KICAgVGhlIGV4YW1wbGVzIGJlbG93IGFyZSBpbGx1c3RyYXRpdmUgT05MWSAtLSBkbyBub3QgcmV1c2UgdGhlbToKICAgZS5nLiAicmUtcmVhZHMgb2xkIGxldHRlcnMiLCAic2tldGNoZXMgZmxvb3IgcGxhbnMiLCAiaHVtcyB3aGlsZSBkb2luZyBkaXNoZXMiLAogICAgICAgICJzZW5kcyB2b2ljZSBub3RlcyBpbnN0ZWFkIG9mIHRleHRzIiwgInJlb3JnYW5pemVzIHRoZWlyIGJvb2tzaGVsZiIKCjMuICJjb25kaXRpb24iOiB0aGUgc3BlY2lmaWMgY29uZGl0aW9uIHVuZGVyIHdoaWNoIHRoZSBiZWhhdmlvciBvY2N1cnMKICAgZS5nLiAiYWZ0ZXIgNXBtIiwgIndoZW4gaXQncyByYWluaW5nIiwgIndoZW4gZmVlbGluZyBzdHJlc3NlZCIKICAgLSBNdXN0IGJlIGNvbmNyZXRlIGFuZCB0ZXN0YWJsZSAtLSB0aGUgcXVlc3Rpb24gd2lsbCBwcmVzZW50IGEgc3BlY2lmaWMgY29udGV4dAogICAtIEF2b2lkIHZhZ3VlIGNvbmRpdGlvbnMgbGlrZSAic29tZXRpbWVzIiBvciAib2Z0ZW4iCgo0LiAiZW50aXR5X2ZhY3RzIjogYSBsaXN0IGNvbnRhaW5pbmcgZXhhY3RseSAxIG5hdHVyYWwgc3RhdGVtZW50IHRoYXQgZGlyZWN0bHkgZW5jb2RlcwogICB0aGUgZnVsbCBjb25kaXRpb25hbCBmYWN0IC0tIGJvdGggdGhlIGJlaGF2aW9yIEFORCB0aGUgY29uZGl0aW9uIGluIGEgc2luZ2xlIHNlbnRlbmNlLgogICAtIE11c3QgYmUgYSBjYXN1YWwsIGZpcnN0LXBlcnNvbiBvciB0aGlyZC1wZXJzb24gY29udmVyc2F0aW9uYWwgc2VudGVuY2UKICAgLSBNdXN0IGNsZWFybHkgc3RhdGUgQk9USCB3aGF0IHRoZSBlbnRpdHkgZG9lcyBBTkQgd2hlbi91bmRlciB3aGF0IGNvbmRpdGlvbgogICAtIDEtMiBzZW50ZW5jZXMgbWF4CiAgIC0gVGhlIGV4YW1wbGUgYmVsb3cgaXMgaWxsdXN0cmF0aXZlIE9OTFkgLS0gZG8gbm90IHJldXNlIGl0OgogICAgIFsiQWxleCBoYXMgYSBydWxlOiBubyBjb2ZmZWUgYmVmb3JlIDVwbSwgc2luY2UgaXQgbWVzc2VzIHdpdGggdGhlaXIgc2xlZXAuIl0KCjUuICJxdWVzdGlvbiI6IGEgbmF0dXJhbCBxdWVzdGlvbiBhYm91dCB3aGV0aGVyIHRoZSBlbnRpdHkgc2hvdWxkIGRvIChvciB3b3VsZCBkbykgdGhlIGJlaGF2aW9yLAogICBnaXZlbiBhIFNQRUNJRklDIGNvbnRleHQgdGhhdCBtYXkgb3IgbWF5IG5vdCBzYXRpc2Z5IHRoZSBjb25kaXRpb24uCgogICBDUklUSUNBTCBSVUxFIC0tIHRoZSBxdWVzdGlvbiBNVVNUIGJlIG5vbi1pbmZlcnJhYmxlIHdpdGhvdXQgdGhlIGVudGl0eSdzIHNwZWNpZmljIGZhY3Q6CiAgIEEgcGVyc29uIHdpdGggbm8ga25vd2xlZGdlIG9mIHRoZSBlbnRpdHkgc2hvdWxkIE5PVCBiZSBhYmxlIHRvIGd1ZXNzIHRoZSBjb3JyZWN0IGFuc3dlcgogICBmcm9tIGNvbW1vbiBzZW5zZSBvciBnZW5lcmFsIG5vcm1zIGFsb25lLiBUaGUgY29ycmVjdCBhbnN3ZXIgbXVzdCBkZXBlbmQgb24ga25vd2luZwogICBUSElTIGVudGl0eSdzIHNwZWNpZmljIGNvbmRpdGlvbmFsIHJ1bGUuCgogICBCQUQgKGluZmVycmFibGUgZnJvbSBjb21tb24gc2Vuc2UpOgogICAgIC0gIkl0J3MgYSBicmlnaHQgc3VubnkgYWZ0ZXJub29uLiBTaG91bGQgWmFyZWsgd2VhciBoaXMgaGVhdnkgd2ludGVyIGNsb2FrPyIKICAgICAgIC0+IEFueW9uZSB3b3VsZCBzYXkgbm8sIHJlZ2FyZGxlc3Mgb2YgYW55IHN0b3JlZCBmYWN0LgogICAgIC0gIkpvcmRhbiBoYXNuJ3Qgc2xlcHQgaW4gMzAgaG91cnMuIFdvdWxkIHRoZXkgd2FudCB0byBnbyBjbHViYmluZz8iCiAgICAgICAtPiBDb21tb24gc2Vuc2UgZ2l2ZXMgdGhlIGFuc3dlci4KCiAgIEdPT0QgKHJlcXVpcmVzIGtub3dpbmcgdGhlIGVudGl0eSdzIHJ1bGUpIC0tIHRoZXNlIGV4YW1wbGVzIGFyZSBpbGx1c3RyYXRpdmUgT05MWSwgZG8gbm90IHJldXNlIHRoZW06CiAgICAgLSAiSXQncyAzcG0gYW5kIEknbSBtZWV0aW5nIEFsZXggLS0gc2hvdWxkIEkgZ3JhYiB0aGVtIGEgY29mZmVlPyIKICAgICAgIC0+IFdpdGhvdXQga25vd2luZyBBbGV4J3MgYWZ0ZXItNXBtIHJ1bGUsIHlvdSBtaWdodCByZWFzb25hYmx5IHNheSB5ZXMuCiAgICAgLSAiSXQncyBhIHF1aWV0IFN1bmRheSBtb3JuaW5nLiBXb3VsZCBQcml5YSB3YW50IHRvIHJlb3JnYW5pemUgaGVyIGJvb2tzaGVsZj8iCiAgICAgICAtPiBXaXRob3V0IGtub3dpbmcgUHJpeWEgb25seSBkb2VzIHRoaXMgd2hlbiBzdHJlc3NlZCwgeW91IGNhbid0IHRlbGwuCiAgICAgLSAiV2UncmUgYXQgdGhlIHBhcmsgYW5kIGl0J3MgMThDIG91dHNpZGUuIFdvdWxkIE1pc28gZWF0IGZyb20gdGhlIHJlZCBib3dsPyIKICAgICAgIC0+IFdpdGhvdXQga25vd2luZyBNaXNvJ3Mgc3BlY2lmaWMgcnVsZSwgdGhpcyBpcyBnZW51aW5lbHkgYW1iaWd1b3VzLgoKICAgVGhlIHF1ZXN0aW9uIHNob3VsZCBwcmVzZW50IGEgY29udGV4dCB3aGVyZSBhIHJlYXNvbmFibGUgcGVyc29uIFdJVEhPVVQgdGhlIGVudGl0eSdzCiAgIHNwZWNpZmljIHJ1bGUgY291bGQgcGxhdXNpYmx5IGFuc3dlciBlaXRoZXIgd2F5IC0tIG1ha2luZyB0aGUgc3RvcmVkIGZhY3QgZGVjaXNpdmUuCgo2LiAicXVlc3Rpb25fY29udGV4dCI6IHRoZSBzcGVjaWZpYyBjb250ZXh0IHByZXNlbnRlZCBpbiB0aGUgcXVlc3Rpb24KICAgZS5nLiAiM3BtIiwgInF1aWV0IFN1bmRheSBtb3JuaW5nIiwgIjE4QyBhdCB0aGUgcGFyayIKCjcuICJjb25kaXRpb25fbWV0IjogInllcyIgaWYgdGhlIHF1ZXN0aW9uIGNvbnRleHQgc2F0aXNmaWVzIHRoZSBjb25kaXRpb24sICJubyIgaWYgbm90CiAgIFRoaW5rIGNhcmVmdWxseSAtLSBpZiB0aGUgY29uZGl0aW9uIGlzICJhZnRlciA1cG0iIGFuZCB0aGUgY29udGV4dCBpcyAiM3BtIiwgaXQncyAibm8iCgo4LiAiZ3JvdW5kX3RydXRoX2Fuc3dlciI6IGEgc2hvcnQgeWVzL25vIGFuc3dlciB3aXRoIGEgYnJpZWYgcmVhc29uCiAgIGUuZy4gIk5vIC0tIGl0J3Mgb25seSAzcG0gYW5kIEFsZXggZG9lc24ndCBkcmluayBjb2ZmZWUgYmVmb3JlIDVwbS4iCiAgIGUuZy4gIlllcyAtLSBpdCdzIHJhaW5pbmcsIHdoaWNoIGlzIGV4YWN0bHkgd2hlbiBKb3JkYW4gbGlrZXMgdG8gY29vayBlbGFib3JhdGUgbWVhbHMuIgoKUmV0dXJuIHN0cmljdCBKU09OIHdpdGgga2V5ICJyb3dzIiwgYSBsaXN0IG9mIG9iamVjdHM6Ci0gcm93X2lkIChpbnQpCi0gZW50aXR5IChzdHJpbmcpCi0gZW50aXR5X2NhdGVnb3J5IChzdHJpbmc6ICJwZXJzb24iLCAicGV0Iiwgb3IgImNoYXJhY3RlciIpCi0gYmVoYXZpb3IgKHN0cmluZykKLSBjb25kaXRpb25fdHlwZSAoc3RyaW5nLCBzYW1lIGFzIGlucHV0KQotIGNvbmRpdGlvbiAoc3RyaW5nKQotIGVudGl0eV9mYWN0cyAobGlzdCBvZiBleGFjdGx5IDEgc3RyaW5nKQotIHF1ZXN0aW9uIChzdHJpbmcpCi0gcXVlc3Rpb25fY29udGV4dCAoc3RyaW5nKQotIGNvbmRpdGlvbl9tZXQgKHN0cmluZzogInllcyIgb3IgIm5vIikKLSBncm91bmRfdHJ1dGhfYW5zd2VyIChzdHJpbmcpCgpSdWxlczoKMSkgZW50aXR5X2ZhY3RzIG11c3QgaGF2ZSBleGFjdGx5IDEgc3RhdGVtZW50IGVuY29kaW5nIGJvdGggdGhlIGJlaGF2aW9yIGFuZCB0aGUgY29uZGl0aW9uCjIpIFRoZSBjb25kaXRpb24gbXVzdCBiZSBjb25jcmV0ZSBhbmQgdGVzdGFibGUgKG5vdCB2YWd1ZSkKMykgVGhlIHF1ZXN0aW9uIG11c3QgcHJlc2VudCBhIHNwZWNpZmljIGNvbnRleHQgdmFsdWUgdGhhdCBjbGVhcmx5IGVpdGhlciBtZWV0cyBvciBkb2Vzbid0IG1lZXQgdGhlIGNvbmRpdGlvbgo0KSBjb25kaXRpb25fbWV0IG11c3QgY29ycmVjdGx5IHJlZmxlY3Qgd2hldGhlciB0aGUgcXVlc3Rpb24gY29udGV4dCBzYXRpc2ZpZXMgdGhlIGNvbmRpdGlvbgo1KSBncm91bmRfdHJ1dGhfYW5zd2VyIG11c3QgYmUgY29uc2lzdGVudCB3aXRoIGNvbmRpdGlvbl9tZXQKNikgVGhlIHF1ZXN0aW9uIE1VU1QgYmUgbm9uLWluZmVycmFibGU6IHdpdGhvdXQga25vd2luZyB0aGUgZW50aXR5J3Mgc3BlY2lmaWMgcnVsZSwgYQogICByZWFzb25hYmxlIHBlcnNvbiBzaG91bGQgYmUgZ2VudWluZWx5IHVuY2VydGFpbiBhYm91dCB0aGUgYW5zd2VyCjcpIFZhcnkgY29uZGl0aW9uX21ldCBiZXR3ZWVuICJ5ZXMiIGFuZCAibm8iIGFjcm9zcyB0aGUgYmF0Y2gKOCkgRG8gTk9UIHJldXNlIGFueSBlbnRpdHkgbmFtZXMsIGJlaGF2aW9ycywgY29uZGl0aW9ucywgb3IgcGhyYXNpbmdzIGZyb20gdGhlIGV4YW1wbGVzIGFib3ZlIC0tCiAgIHRoZXkgZXhpc3Qgb25seSB0byBpbGx1c3RyYXRlIHRoZSBmb3JtYXQKOSkgT3V0cHV0IE9OTFkgdmFsaWQgSlNPTgoKSW5wdXQgc3BlY3M6CntzcGVjcyBhcyBKU09OfQ==)Generateconditional\-factsdatapoints\.Eachdatapointdescribesanentity\(person,pet,orcharacter\)whohasaCONDITIONALbehavior\-\-theyonlydosomethingunderaspecificcondition\.Foreachspecbelow,generate:1\."entity":arealisticfirstname\(forpersons/characters\)orapetname\(forpets\)e\.g\."Jordan","Miso","CaptainRex"2\."behavior":ashortactionphrasedescribingwhattheentitydoesconditionally\.Inventsomethingcreativeandspecifictotheentityandconditiontype\-\-doNOTdefaulttoclicheslike"goesforarun"or"drinkscoffee"\.Thebehaviorshouldfeelpersonalandidiosyncratic,notuniversallycommon\.TheexamplesbelowareillustrativeONLY\-\-donotreusethem:e\.g\."re\-readsoldletters","sketchesfloorplans","humswhiledoingdishes","sendsvoicenotesinsteadoftexts","reorganizestheirbookshelf"3\."condition":thespecificconditionunderwhichthebehavioroccurse\.g\."after5pm","whenit’sraining","whenfeelingstressed"\-Mustbeconcreteandtestable\-\-thequestionwillpresentaspecificcontext\-Avoidvagueconditionslike"sometimes"or"often"4\."entity\_facts":alistcontainingexactly1naturalstatementthatdirectlyencodesthefullconditionalfact\-\-boththebehaviorANDtheconditioninasinglesentence\.\-Mustbeacasual,first\-personorthird\-personconversationalsentence\-MustclearlystateBOTHwhattheentitydoesANDwhen/underwhatcondition\-1\-2sentencesmax\-TheexamplebelowisillustrativeONLY\-\-donotreuseit:\["Alexhasarule:nocoffeebefore5pm,sinceitmesseswiththeirsleep\."\]5\."question":anaturalquestionaboutwhethertheentityshoulddo\(orwoulddo\)thebehavior,givenaSPECIFICcontextthatmayormaynotsatisfythecondition\.CRITICALRULE\-\-thequestionMUSTbenon\-inferrablewithouttheentity’sspecificfact:ApersonwithnoknowledgeoftheentityshouldNOTbeabletoguessthecorrectanswerfromcommonsenseorgeneralnormsalone\.ThecorrectanswermustdependonknowingTHISentity’sspecificconditionalrule\.BAD\(inferrablefromcommonsense\):\-"It’sabrightsunnyafternoon\.ShouldZarekwearhisheavywintercloak?"\-\>Anyonewouldsayno,regardlessofanystoredfact\.\-"Jordanhasn’tsleptin30hours\.Wouldtheywanttogoclubbing?"\-\>Commonsensegivestheanswer\.GOOD\(requiresknowingtheentity’srule\)\-\-theseexamplesareillustrativeONLY,donotreusethem:\-"It’s3pmandI’mmeetingAlex\-\-shouldIgrabthemacoffee?"\-\>WithoutknowingAlex’safter\-5pmrule,youmightreasonablysayyes\.\-"It’saquietSundaymorning\.WouldPriyawanttoreorganizeherbookshelf?"\-\>WithoutknowingPriyaonlydoesthiswhenstressed,youcan’ttell\.\-"We’reattheparkandit’s18Coutside\.WouldMisoeatfromtheredbowl?"\-\>WithoutknowingMiso’sspecificrule,thisisgenuinelyambiguous\.ThequestionshouldpresentacontextwhereareasonablepersonWITHOUTtheentity’sspecificrulecouldplausiblyanswereitherway\-\-makingthestoredfactdecisive\.6\."question\_context":thespecificcontextpresentedinthequestione\.g\."3pm","quietSundaymorning","18Catthepark"7\."condition\_met":"yes"ifthequestioncontextsatisfiesthecondition,"no"ifnotThinkcarefully\-\-iftheconditionis"after5pm"andthecontextis"3pm",it’s"no"8\."ground\_truth\_answer":ashortyes/noanswerwithabriefreasone\.g\."No\-\-it’sonly3pmandAlexdoesn’tdrinkcoffeebefore5pm\."e\.g\."Yes\-\-it’sraining,whichisexactlywhenJordanlikestocookelaboratemeals\."ReturnstrictJSONwithkey"rows",alistofobjects:\-row\_id\(int\)\-entity\(string\)\-entity\_category\(string:"person","pet",or"character"\)\-behavior\(string\)\-condition\_type\(string,sameasinput\)\-condition\(string\)\-entity\_facts\(listofexactly1string\)\-question\(string\)\-question\_context\(string\)\-condition\_met\(string:"yes"or"no"\)\-ground\_truth\_answer\(string\)Rules:1\)entity\_factsmusthaveexactly1statementencodingboththebehaviorandthecondition2\)Theconditionmustbeconcreteandtestable\(notvague\)3\)Thequestionmustpresentaspecificcontextvaluethatclearlyeithermeetsordoesn’tmeetthecondition4\)condition\_metmustcorrectlyreflectwhetherthequestioncontextsatisfiesthecondition5\)ground\_truth\_answermustbeconsistentwithcondition\_met6\)ThequestionMUSTbenon\-inferrable:withoutknowingtheentity’sspecificrule,areasonablepersonshouldbegenuinelyuncertainabouttheanswer7\)Varycondition\_metbetween"yes"and"no"acrossthebatch8\)DoNOTreuseanyentitynames,behaviors,conditions,orphrasingsfromtheexamplesabove\-\-theyexistonlytoillustratetheformat9\)OutputONLYvalidJSONInputspecs:\{specsasJSON\}

Generation:Conditional\-Factsessay wrapper \(Easy\)[⬇](data:text/plain;base64,Rm9yIGVhY2ggaXRlbSBiZWxvdywgd3JpdGUgYSBuYXR1cmFsIGVzc2F5ICg3LTEwIHNlbnRlbmNlcykgYWJvdXQgdGhlIGVudGl0eQp0aGF0IGVtYmVkcyB0aGUgY29uZGl0aW9uYWwgZmFjdCBpbnRvIGEgcmljaCwgY2FzdWFsIG5hcnJhdGl2ZS4KClJ1bGVzOgoxLiBUaGUgZXNzYXkgTVVTVCBwcmVzZXJ2ZSB0aGUgY29uZGl0aW9uYWwgZmFjdCBjbGVhcmx5IC0tIGJvdGggdGhlIGJlaGF2aW9yIEFORCB0aGUKICAgY29uZGl0aW9uIG11c3QgYmUgcHJlc2VudC4gUGFyYXBocmFzZSBpcyBmaW5lOyBkbyBub3Qgb21pdCBlaXRoZXIgcGFydC4KMi4gQWxsIG90aGVyIHNlbnRlbmNlcyBzaG91bGQgZGVzY3JpYmUgdGhlIGVudGl0eSdzIGJhY2tncm91bmQsIHBlcnNvbmFsaXR5LCBkYWlseSByb3V0aW5lcywKICAgcmVsYXRpb25zaGlwcywgaG9iYmllcywgcXVpcmtzLCBvciBsaWZlIGNvbnRleHQuIEV2ZXJ5IHN1Y2ggc2VudGVuY2UgbXVzdCBiZSBhbgogICB1bmNvbmRpdGlvbmFsLCBmYWN0dWFsIHN0YXRlbWVudC4KMy4gRG8gTk9UIGludHJvZHVjZSBhbnkgbmV3IGNvbmRpdGlvbmFsIHN0YXRlbWVudHMgYW55d2hlcmUgaW4gdGhlIGVzc2F5LiBGb3JiaWRkZW4KICAgY29uc3RydWN0aW9uczogIm9ubHkgd2hlbiIsICJ1bmxlc3MiLCAiZXhjZXB0IHdoZW4iLCAiYnV0IG9ubHkgaWYiLCAid2hlbmV2ZXIgWCB0aGVuIFkiLAogICAib25seSBhZnRlciIsICJvbmx5IGlmIiwgb3IgYW55IG90aGVyIGNvbmRpdGlvbmFsIHBocmFzaW5nIGJleW9uZCB3aGF0IHdhcyBhbHJlYWR5IGluIHRoZQogICBvcmlnaW5hbCBmYWN0Lgo0LiBUaGUgZXNzYXkgc2hvdWxkIGZlZWwgbmF0dXJhbCAtLSBsaWtlIGFuIGV4Y2VycHQgZnJvbSBhIGNoYXQgY29udmVyc2F0aW9uLCBwZXJzb25hbCBibG9nLAogICBvciBqb3VybmFsIGVudHJ5LCBub3QgYSBmb3JtYWwgcmVwb3J0IG9yIGxpc3QuCjUuIFRoZSBjb25kaXRpb25hbCBmYWN0IG1heSBhcHBlYXIgYW55d2hlcmUgaW4gdGhlIGVzc2F5LCBzdXJyb3VuZGVkIGJ5IHVucmVsYXRlZCBjb250ZXh0CiAgIGJlZm9yZSBhbmQgYWZ0ZXIgaXQuCjYuIDUtOCBzZW50ZW5jZXMgdG90YWwuCgpSZXR1cm4gc3RyaWN0IEpTT04gd2l0aCBrZXkgInJvd3MiLCBhIGxpc3Qgb2Y6CiAgcm93X2lkIChpbnQsIHNhbWUgYXMgaW5wdXQpLCBlc3NheSAoc3RyaW5nKQoKT3V0cHV0IE9OTFkgdmFsaWQgSlNPTi4KCklucHV0Ogp7aXRlbXMgYXMgSlNPTn0=)Foreachitembelow,writeanaturalessay\(7\-10sentences\)abouttheentitythatembedstheconditionalfactintoarich,casualnarrative\.Rules:1\.TheessayMUSTpreservetheconditionalfactclearly\-\-boththebehaviorANDtheconditionmustbepresent\.Paraphraseisfine;donotomiteitherpart\.2\.Allothersentencesshoulddescribetheentity’sbackground,personality,dailyroutines,relationships,hobbies,quirks,orlifecontext\.Everysuchsentencemustbeanunconditional,factualstatement\.3\.DoNOTintroduceanynewconditionalstatementsanywhereintheessay\.Forbiddenconstructions:"onlywhen","unless","exceptwhen","butonlyif","wheneverXthenY","onlyafter","onlyif",oranyotherconditionalphrasingbeyondwhatwasalreadyintheoriginalfact\.4\.Theessayshouldfeelnatural\-\-likeanexcerptfromachatconversation,personalblog,orjournalentry,notaformalreportorlist\.5\.Theconditionalfactmayappearanywhereintheessay,surroundedbyunrelatedcontextbeforeandafterit\.6\.5\-8sentencestotal\.ReturnstrictJSONwithkey"rows",alistof:row\_id\(int,sameasinput\),essay\(string\)OutputONLYvalidJSON\.Input:\{itemsasJSON\}

Generation:Conditional\-Factsessay wrapper \(Hard\)[⬇](data:text/plain;base64,Rm9yIGVhY2ggaXRlbSBiZWxvdywgd3JpdGUgYSBuYXR1cmFsIGVzc2F5ICg4LTEyIHNlbnRlbmNlcykgYWJvdXQgdGhlIGVudGl0eQp0aGF0IERFQ09NUE9TRVMgdGhlIG9yaWdpbmFsIGNvbmRpdGlvbmFsIGZhY3QgaW50byBUSFJFRSBkaXN0cmlidXRlZCwgbm9uLWFkamFjZW50IHNlbnRlbmNlcy4KVGhlIHJlYWRlciBtdXN0IENPTVBPU0UgdGhlIHJ1bGUgZnJvbSBzY2F0dGVyZWQgZXZpZGVuY2UgcmF0aGVyIHRoYW4gY29weSBpdCBmcm9tIGEgc2luZ2xlCnNlbnRlbmNlLgoKQ1JJVElDQUwgUlVMRSAtLSBETyBOT1Qgd3JpdGUgYW55IHNpbmdsZSBzZW50ZW5jZSB0aGF0IGV4cGxpY2l0bHkgc3RhdGVzIGJvdGggdGhlIGJlaGF2aW9yCkFORCB0aGUgY29uZGl0aW9uIHRvZ2V0aGVyLiBUaGUgb3JpZ2luYWwgY29uZGl0aW9uYWwgZmFjdCBtdXN0IGJlIHNwbGl0IGludG8gZXhhY3RseSB0aHJlZQpzZW50ZW5jZXMgc3ByZWFkIGFjcm9zcyB0aGUgZXNzYXk6CgogIChBKSBCRUhBVklPUiBzZW50ZW5jZTogZGVzY3JpYmVzIHRoZSBiZWhhdmlvciBhcyBhIHRlbmRlbmN5LCBoYWJpdCwgb3IgcGF0dGVybiwKICAgICAgV0lUSE9VVCBuYW1pbmcgdGhlIHRyaWdnZXIgY29uZGl0aW9uLgogIChCKSBDT05ESVRJT04gc2VudGVuY2U6IGVzdGFibGlzaGVzIHRoZSBjb25kaXRpb24gYXMgcGFydCBvZiB0aGUgZW50aXR5J3MgbGlmZSBjb250ZXh0CiAgICAgIG9yIGVudmlyb25tZW50LCBXSVRIT1VUIG5hbWluZyB0aGUgYmVoYXZpb3IuCiAgKEMpIExJTksgc2VudGVuY2UgKFJFUVVJUkVELCBub3Qgb3B0aW9uYWwpOiBzdWJ0bHkgY29ubmVjdHMgdGhlIHR3byB0aHJvdWdoIHRpbWluZywKICAgICAgY28tb2NjdXJyZW5jZSwgb3Igc2NlbmUtc2V0dGluZyAtLSB1c2luZyBsYW5ndWFnZSBsaWtlICJJdCdzIHVzdWFsbHkuLi4iLCAiQnkgdGhlbi4uLiIsCiAgICAgICJBcm91bmQgdGhhdCB0aW1lLi4uIiwgIk1vc3Qgb2YgdGhlIHRpbWUgaXQgaGFwcGVucy4uLiIsICJUaGF0J3MgdHlwaWNhbGx5IHdoZW4uLi4iIC0tCiAgICAgIGJ1dCBzdGlsbCBXSVRIT1VUIGV4cGxpY2l0bHkgc3RhdGluZyAiWCBoYXBwZW5zIHdoZW4gWSIgb3IgdXNpbmcgYW55IGNvbmRpdGlvbmFsIHBocmFzaW5nLgoKVGhlIHRocmVlIHNlbnRlbmNlcyAoQSksIChCKSwgYW5kIChDKSBNVVNUIGFsbCBhcHBlYXIgYXQgbm9uLWFkamFjZW50IHBvc2l0aW9ucyBpbiB0aGUKZXNzYXkuIFNwZWNpZmljYWxseTogYmV0d2VlbiBhbnkgdHdvIG9mIHRoZW0gdGhlcmUgbXVzdCBiZSBhdCBsZWFzdCBvbmUgc2VudGVuY2Ugb2YKdW5yZWxhdGVkIGNvbnRleHQuIE5vIHR3byBvZiAoQSksIChCKSwgKEMpIG1heSBiZSBuZXh0IHRvIGVhY2ggb3RoZXIuCgpFeGFtcGxlcyBvZiB0aGUgZGlzdHJpYnV0ZWQgcGF0dGVybiAoaWxsdXN0cmF0aXZlIG9ubHkgLS0gZG8gbm90IHJldXNlKToKCiAgRXhhbXBsZSAxCiAgT3JpZ2luYWwgZmFjdDogIk1hcmEgcmVvcmdhbml6ZXMgaGVyIHNrZXRjaGVzIHdoZW4gaXQncyBhZnRlciBtaWRuaWdodC4iCiAgRGlzdHJpYnV0ZWQgZXNzYXk6CiAgICAiTWFyYSBoYXMgbGl2ZWQgaW4gdGhlIHNhbWUgc3R1ZGlvIGFwYXJ0bWVudCBmb3Igc2l4IHllYXJzLgogICAgIFNoZSB0ZW5kcyB0byByZW9yZ2FuaXplIGhlciBza2V0Y2hlcyBpbiBzdWRkZW4gYnVyc3RzIG9mIGZvY3VzLiAgICA8LS0gKEEpIEJFSEFWSU9SCiAgICAgSGVyIGNhdCBQZXBwZXIgaXMgdXN1YWxseSBhc2xlZXAgb24gdGhlIHdpbmRvd3NpbGwgYnkgdGhlbi4KICAgICBTaGUgd29ya3MgYXMgYSBmcmVlbGFuY2UgaWxsdXN0cmF0b3IgYW5kIGtlZXBzIGlycmVndWxhciBob3VycyB0aGF0IHN0cmV0Y2ggZGVlcCBpbnRvIHRoZSBuaWdodC4gICA8LS0gKEIpIENPTkRJVElPTgogICAgIFNoZSBkcmlua3MgdGVhIGluc3RlYWQgb2YgY29mZmVlLgogICAgIEl0J3MgYWxtb3N0IGFsd2F5cyBkdXJpbmcgdGhvc2UgbGF0ZSBob3VycyB0aGF0IHRoaXMga2luZCBvZiByZXN0bGVzcyBlbmVyZ3kgaGl0cyBoZXIuICAgPC0tIChDKSBMSU5LCiAgICAgSGVyIHNpc3RlciBjYWxscyBldmVyeSBTdW5kYXkgYWZ0ZXJub29uLiIKCiAgRXhhbXBsZSAyCiAgT3JpZ2luYWwgZmFjdDogIkRldm9uIHNlbmRzIGxvbmcgdm9pY2Ugbm90ZXMgdG8gaGlzIGZyaWVuZHMgYWZ0ZXIgYSB0b3VnaCB3b3Jrb3V0LiIKICBEaXN0cmlidXRlZCBlc3NheToKICAgICJEZXZvbiBncmV3IHVwIGluIGEgc21hbGwgdG93biBvdXRzaWRlIFNhY3JhbWVudG8gYW5kIHN0aWxsIHRleHRzIGhpcyBvbGQgaGlnaCBzY2hvb2wgZ3JvdXAgY2hhdCBkYWlseS4KICAgICBIZSBoYXMgYSBoYWJpdCBvZiBzZW5kaW5nIHNwcmF3bGluZywgdGVuLW1pbnV0ZSB2b2ljZSBub3RlcyB0byBoaXMgY2xvc2VzdCBmcmllbmRzLiAgIDwtLSAoQSkgQkVIQVZJT1IKICAgICBIaXMgYXBhcnRtZW50IGlzIGRlY29yYXRlZCB3aXRoIHNlY29uZGhhbmQgZnVybml0dXJlIGFuZCBhIHdhbGwgb2YgY2xpbWJpbmcgbWVkYWxzLgogICAgIExhdGVseSBoZSBoYXMgYmVlbiBwdXNoaW5nIGhpbXNlbGYgaGFyZCBhdCB0aGUgZ3ltLCBsZWF2aW5nIG1vc3Qgc2Vzc2lvbnMgY29tcGxldGVseSBkcmFpbmVkIGFuZCBzaGFreS4gICA8LS0gKEIpIENPTkRJVElPTgogICAgIEhlIHdvcmtzIHJlbW90ZWx5IGFzIGEgYmFja2VuZCBlbmdpbmVlciBhbmQgcHJlZmVycyBhZnRlcm5vb24gbWVldGluZ3MuCiAgICAgVGhvc2UgbG9uZywgcmFtYmxpbmcgbWVzc2FnZXMgdGVuZCB0byBjb21lIHJpZ2h0IGFmdGVyIGhlIHN0dW1ibGVzIGhvbWUgZnJvbSB0aGUgYm91bGRlcmluZyB3YWxsLiAgIDwtLSAoQykgTElOSwogICAgIEhpcyBtb20gc3RpbGwgbWFpbHMgaGltIGJpcnRoZGF5IGNhcmRzIGEgd2VlayBlYXJseS4iCgogIEV4YW1wbGUgMwogIE9yaWdpbmFsIGZhY3Q6ICJCaXNjdWl0IChhIGNvcmdpKSBvbmx5IGVhdHMgZnJvbSB0aGUgcmVkIGJvd2wgd2hlbiB0aGVyZSBhcmUgZ3Vlc3RzIGluIHRoZSBob3VzZS4iCiAgRGlzdHJpYnV0ZWQgZXNzYXk6CiAgICAiQmlzY3VpdCBpcyBhIGZpdmUteWVhci1vbGQgY29yZ2kgd2hvIGxpdmVzIHdpdGggdGhlIFRhbmFrYSBmYW1pbHkgaW4gUG9ydGxhbmQuCiAgICAgSGUgaGFzIGEgcGVjdWxpYXIgaGFiaXQgb2YgZWF0aW5nIGV4Y2x1c2l2ZWx5IGZyb20gdGhlIHJlZCBjZXJhbWljIGJvd2wgb24gY2VydGFpbiBkYXlzLiAgIDwtLSAoQSkgQkVIQVZJT1IKICAgICBIZSBzbGVlcHMgdW5kZXIgdGhlIGRpbmluZyByb29tIHRhYmxlIGFuZCBmb2xsb3dzIHRoZSB5b3VuZ2VzdCBraWQgZXZlcnl3aGVyZS4KICAgICBUaGUgVGFuYWthcyBob3N0IGZyZXF1ZW50IGRpbm5lciBwYXJ0aWVzLCBhbmQgdGhlIGhvdXNlIGlzIG9mdGVuIGZ1bGwgb2YgdW5mYW1pbGlhciB2b2ljZXMgYW5kIHNob2VzIGJ5IHRoZSBkb29yLiAgIDwtLSAoQikgQ09ORElUSU9OCiAgICAgSGlzIGZhdm9yaXRlIHRveSBpcyBhIGNoZXdlZC11cCBzdHVmZmVkIGNhcnJvdC4KICAgICBUaGUgcmVkIGJvd2wgdGVuZHMgdG8gY29tZSBvdXQgc3BlY2lmaWNhbGx5IG9uIHRob3NlIGNyb3dkZWQsIG5vaXN5IGV2ZW5pbmdzLiAgIDwtLSAoQykgTElOSwogICAgIEhlIGdldHMgZ3Jvb21lZCBvbmNlIGEgbW9udGggYXQgYSBwbGFjZSBvbiBIYXd0aG9ybmUuIgoKT3RoZXIgcnVsZXM6CjEuIFRoZSBiZWhhdmlvciBhbmQgY29uZGl0aW9uIG11c3QgQk9USCBiZSByZWNvdmVyYWJsZSBieSBhIGNhcmVmdWwgcmVhZGVyIHdobyBjb21wb3NlcwogICBzZW50ZW5jZXMgKEEpLCAoQiksIGFuZCAoQykgLS0gYnV0IE5FSVRIRVIgc2hvdWxkIGFwcGVhciBpbiB0aGUgc2FtZSBzZW50ZW5jZS4KMi4gRG8gTk9UIHVzZSBleHBsaWNpdCBjb25kaXRpb25hbCBwaHJhc2luZyBhbnl3aGVyZSAoIm9ubHkgd2hlbiIsICJ3aGVuZXZlciIsICJpZiIsCiAgICJ1bmxlc3MiLCAiZXhjZXB0IHdoZW4iLCAiYnV0IG9ubHkgaWYiLCAib25seSBhZnRlciIsICJvbmx5IGlmIikuCjMuIFRoZSBsaW5rIHNlbnRlbmNlIChDKSBzaG91bGQgdXNlIHRpbWluZy9zY2VuZSBsYW5ndWFnZSwgbm90IGxvZ2ljYWwgY29ubmVjdGl2ZXMuCjQuIEFsbCByZW1haW5pbmcgc2VudGVuY2VzIHNob3VsZCBkZXNjcmliZSB0aGUgZW50aXR5J3MgYmFja2dyb3VuZCwgcGVyc29uYWxpdHksIGRhaWx5CiAgIHJvdXRpbmVzLCByZWxhdGlvbnNoaXBzLCBob2JiaWVzLCBxdWlya3MsIG9yIGxpZmUgY29udGV4dCAtLSB1bmNvbmRpdGlvbmFsIGZhY3R1YWwgc3RhdGVtZW50cy4KNS4gVGhlIGVzc2F5IHNob3VsZCBmZWVsIG5hdHVyYWwgLS0gbGlrZSBhbiBleGNlcnB0IGZyb20gYSBwZXJzb25hbCBibG9nIG9yIGpvdXJuYWwgZW50cnkuCjYuIFRoZSBjb3JyZWxhdGlvbiBiZXR3ZWVuIHRoZSBiZWhhdmlvciBhbmQgY29uZGl0aW9uIHNob3VsZCBiZSBvYnZpb3VzIHRvIHNvbWVib2R5IHdobyBoYXMgcmVhZCBib3RoIHNlbnRlbmNlcy4gSXQgc2hvdWxkIE5PVCBiZSB2YWd1ZSBvciB0b28gc3VidGxlLgo3LiA4LTEyIHNlbnRlbmNlcyB0b3RhbC4KClJldHVybiBzdHJpY3QgSlNPTiB3aXRoIGtleSAicm93cyIsIGEgbGlzdCBvZjoKICByb3dfaWQgKGludCwgc2FtZSBhcyBpbnB1dCksIGVzc2F5IChzdHJpbmcpCgpPdXRwdXQgT05MWSB2YWxpZCBKU09OLgoKSW5wdXQ6CntpdGVtcyBhcyBKU09OfQ==)Foreachitembelow,writeanaturalessay\(8\-12sentences\)abouttheentitythatDECOMPOSEStheoriginalconditionalfactintoTHREEdistributed,non\-adjacentsentences\.ThereadermustCOMPOSEtherulefromscatteredevidenceratherthancopyitfromasinglesentence\.CRITICALRULE\-\-DONOTwriteanysinglesentencethatexplicitlystatesboththebehaviorANDtheconditiontogether\.Theoriginalconditionalfactmustbesplitintoexactlythreesentencesspreadacrosstheessay:\(A\)BEHAVIORsentence:describesthebehaviorasatendency,habit,orpattern,WITHOUTnamingthetriggercondition\.\(B\)CONDITIONsentence:establishestheconditionaspartoftheentity’slifecontextorenvironment,WITHOUTnamingthebehavior\.\(C\)LINKsentence\(REQUIRED,notoptional\):subtlyconnectsthetwothroughtiming,co\-occurrence,orscene\-setting\-\-usinglanguagelike"It’susually\.\.\.","Bythen\.\.\.","Aroundthattime\.\.\.","Mostofthetimeithappens\.\.\.","That’stypicallywhen\.\.\."\-\-butstillWITHOUTexplicitlystating"XhappenswhenY"orusinganyconditionalphrasing\.Thethreesentences\(A\),\(B\),and\(C\)MUSTallappearatnon\-adjacentpositionsintheessay\.Specifically:betweenanytwoofthemtheremustbeatleastonesentenceofunrelatedcontext\.Notwoof\(A\),\(B\),\(C\)maybenexttoeachother\.Examplesofthedistributedpattern\(illustrativeonly\-\-donotreuse\):Example1Originalfact:"Marareorganizeshersketcheswhenit’saftermidnight\."Distributedessay:"Marahaslivedinthesamestudioapartmentforsixyears\.Shetendstoreorganizehersketchesinsuddenburstsoffocus\.<\-\-\(A\)BEHAVIORHercatPepperisusuallyasleeponthewindowsillbythen\.Sheworksasafreelanceillustratorandkeepsirregularhoursthatstretchdeepintothenight\.<\-\-\(B\)CONDITIONShedrinksteainsteadofcoffee\.It’salmostalwaysduringthoselatehoursthatthiskindofrestlessenergyhitsher\.<\-\-\(C\)LINKHersistercallseverySundayafternoon\."Example2Originalfact:"Devonsendslongvoicenotestohisfriendsafteratoughworkout\."Distributedessay:"DevongrewupinasmalltownoutsideSacramentoandstilltextshisoldhighschoolgroupchatdaily\.Hehasahabitofsendingsprawling,ten\-minutevoicenotestohisclosestfriends\.<\-\-\(A\)BEHAVIORHisapartmentisdecoratedwithsecondhandfurnitureandawallofclimbingmedals\.Latelyhehasbeenpushinghimselfhardatthegym,leavingmostsessionscompletelydrainedandshaky\.<\-\-\(B\)CONDITIONHeworksremotelyasabackendengineerandprefersafternoonmeetings\.Thoselong,ramblingmessagestendtocomerightafterhestumbleshomefromtheboulderingwall\.<\-\-\(C\)LINKHismomstillmailshimbirthdaycardsaweekearly\."Example3Originalfact:"Biscuit\(acorgi\)onlyeatsfromtheredbowlwhenthereareguestsinthehouse\."Distributedessay:"Biscuitisafive\-year\-oldcorgiwholiveswiththeTanakafamilyinPortland\.Hehasapeculiarhabitofeatingexclusivelyfromtheredceramicbowloncertaindays\.<\-\-\(A\)BEHAVIORHesleepsunderthediningroomtableandfollowstheyoungestkideverywhere\.TheTanakashostfrequentdinnerparties,andthehouseisoftenfullofunfamiliarvoicesandshoesbythedoor\.<\-\-\(B\)CONDITIONHisfavoritetoyisachewed\-upstuffedcarrot\.Theredbowltendstocomeoutspecificallyonthosecrowded,noisyevenings\.<\-\-\(C\)LINKHegetsgroomedonceamonthataplaceonHawthorne\."Otherrules:1\.ThebehaviorandconditionmustBOTHberecoverablebyacarefulreaderwhocomposessentences\(A\),\(B\),and\(C\)\-\-butNEITHERshouldappearinthesamesentence\.2\.DoNOTuseexplicitconditionalphrasinganywhere\("onlywhen","whenever","if","unless","exceptwhen","butonlyif","onlyafter","onlyif"\)\.3\.Thelinksentence\(C\)shouldusetiming/scenelanguage,notlogicalconnectives\.4\.Allremainingsentencesshoulddescribetheentity’sbackground,personality,dailyroutines,relationships,hobbies,quirks,orlifecontext\-\-unconditionalfactualstatements\.5\.Theessayshouldfeelnatural\-\-likeanexcerptfromapersonalblogorjournalentry\.6\.Thecorrelationbetweenthebehaviorandconditionshouldbeobvioustosomebodywhohasreadbothsentences\.ItshouldNOTbevagueortoosubtle\.7\.8\-12sentencestotal\.ReturnstrictJSONwithkey"rows",alistof:row\_id\(int,sameasinput\),essay\(string\)OutputONLYvalidJSON\.Input:\{itemsasJSON\}

#### D\.1\.2Coexisting\-Facts

A single datapoint generator \(Prompt[D\.1\.2](https://arxiv.org/html/2605.26667#A4.SS1.SSS2)\) produces, for each preference category,NNisolated first\-person statements plus a holistic scenario question whose answer requires allNN\.

Generation:Coexisting\-Factsdatapoint generator[⬇](data:text/plain;base64,R2VuZXJhdGUgY29leGlzdGluZy1mYWN0cyBkYXRhcG9pbnRzLiBFYWNoIGRhdGFwb2ludCByZXByZXNlbnRzIGEgdXNlciAoZmlyc3QtcGVyc29uICJJIikgd2l0aCBNVUxUSVBMRSBwcmVmZXJlbmNlcyBpbiB0aGUgc2FtZSBjYXRlZ29yeS4gRWFjaCBwcmVmZXJlbmNlIHdpbGwgYmUgc3RvcmVkIGFzIGEgY29tcGxldGVseSBTRVBBUkFURSwgSVNPTEFURUQgbWVtb3J5IC0tIHNvIGVhY2ggZmFjdCBzdGF0ZW1lbnQgbXVzdCBtYWtlIHNlbnNlIGVudGlyZWx5IG9uIGl0cyBvd24sIHdpdGggbm8gcmVmZXJlbmNlIHRvIHRoZSBvdGhlciBwcmVmZXJlbmNlcy4KCkZvciBlYWNoIHNwZWMgYmVsb3csIGdlbmVyYXRlOgoxLiAicHJlZmVyZW5jZXMiOiBsaXN0IG9mIGV4YWN0bHkgbnVtX3ByZWZlcmVuY2VzIGRpc3RpbmN0IHByZWZlcmVuY2VzIGluIHRoZSBjYXRlZ29yeQogICAoZS5nLiBmb3IgZm9vZHM6IFsicGl6emEiLCAic3VzaGkiLCAicmFtZW4iXSkKCjIuICJwcmVmZXJlbmNlX2ZhY3RzIjogbGlzdCBvZiBleGFjdGx5IG51bV9wcmVmZXJlbmNlcyBzaG9ydCwgbmF0dXJhbCBmaXJzdC1wZXJzb24gc3RhdGVtZW50cyAtLQogICBPTkUgc3RhdGVtZW50IHBlciBwcmVmZXJlbmNlLCBpbiB0aGUgc2FtZSBvcmRlciBhcyAicHJlZmVyZW5jZXMiLgogICAtIEVhY2ggc3RhdGVtZW50IG11c3Qgc3RhbmQgYWxvbmUgYXMgYSBjb21wbGV0ZSwgc2VsZi1jb250YWluZWQgZmFjdAogICAtIEVhY2ggc3RhdGVtZW50IG11c3QgbWVudGlvbiBPTkxZIHRoYXQgc2luZ2xlIHByZWZlcmVuY2UgKG5vdCB0aGUgb3RoZXJzKQogICAtIFVzZSB2YXJpZWQsIG5hdHVyYWwgcGhyYXNpbmcgLS0gbm90IGEgdGVtcGxhdGUgKCJJIGxvdmUgWCIsICJJIGVuam95IFgiLCAiWCBpcyBteSBmYXZvcml0ZSIsIGV0Yy4pCiAgIC0gMS0yIHNlbnRlbmNlcyBtYXggcGVyIGZhY3QKICAgLSBFeGFtcGxlcyBmb3IgZm9vZHM6CiAgICAgICAiSSBsb3ZlIHBpenphIC0tIGl0J3MgbXkgZGVmYXVsdCBGcmlkYXkgbmlnaHQgbWVhbC4iCiAgICAgICAiU3VzaGkgaXMgbXkgZ28tdG8gd2hlbmV2ZXIgSSB3YW50IHNvbWV0aGluZyBmcmVzaCBhbmQgbGlnaHQuIgogICAgICAgIkknbSBhIGh1Z2UgcmFtZW4gZmFuLCBlc3BlY2lhbGx5IG9uIGNvbGQgZGF5cy4iCgozLiAicXVlc3Rpb24iOiBhIG5hdHVyYWwgZmlyc3QtcGVyc29uIHNjZW5hcmlvIHF1ZXN0aW9uIHRoYXQgUkVRVUlSRVMga25vd2luZyBBTEwgcHJlZmVyZW5jZXMuCiAgIC0gTXVzdCBOT1QgYmUgYSBkaXJlY3QgImxpc3QgYWxsIG15IFgiIHJlcXVlc3QgLS0gbWFrZSBpdCBhIHJlYWxpc3RpYyBzY2VuYXJpbwogICAtIEdvb2Q6ICJJJ20gZ29pbmcgZ3JvY2VyeSBzaG9wcGluZyAtLSB3aGF0IHNob3VsZCBJIHBpY2sgdXAgZm9yIGRpbm5lcnMgdGhpcyB3ZWVrPyIKICAgLSBHb29kOiAiTXkgZnJpZW5kIHdhbnRzIHRvIHBsYW4gYW4gb3V0aW5nIEknZCBlbmpveSAtLSB3aGF0IGFyZSBzb21lIHNvbGlkIG9wdGlvbnM/IgogICAtIFRoZSBxdWVzdGlvbiBzaG91bGQgaGF2ZSBhIGNsZWFybHkgYmV0dGVyIGFuc3dlciBpZiBBTEwgcHJlZmVyZW5jZXMgYXJlIGtub3duIHZzLiBvbmx5IG9uZQoKNC4gImdyb3VuZF90cnV0aF9hbnN3ZXIiOiBhIGNvbmNpc2UgY29tbWEtc2VwYXJhdGVkIGxpc3Qgb2YgYWxsIHByZWZlcmVuY2UgbmFtZXMKICAgRXhhbXBsZTogInBpenphLCBzdXNoaSwgcmFtZW4iCgpSZXR1cm4gc3RyaWN0IEpTT04gd2l0aCBrZXkgInJvd3MiLCBhIGxpc3Qgb2Ygb2JqZWN0czoKLSByb3dfaWQgKGludCkKLSBwcmVmZXJlbmNlX2NhdGVnb3J5IChzdHJpbmcsIHNhbWUgYXMgaW5wdXQpCi0gcHJlZmVyZW5jZXMgKGxpc3Qgb2Ygc3RyaW5ncykKLSBwcmVmZXJlbmNlX2ZhY3RzIChsaXN0IG9mIHN0cmluZ3MsIHNhbWUgbGVuZ3RoIGFzIHByZWZlcmVuY2VzLCBvbmUgZmFjdCBwZXIgcHJlZmVyZW5jZSkKLSBxdWVzdGlvbiAoc3RyaW5nKQotIGdyb3VuZF90cnV0aF9hbnN3ZXIgKHN0cmluZykKClJ1bGVzOgoxKSBwcmVmZXJlbmNlX2ZhY3RzIG11c3QgaGF2ZSBleGFjdGx5IHRoZSBzYW1lIGxlbmd0aCBhcyBwcmVmZXJlbmNlcwoyKSBFYWNoIGZhY3QgY292ZXJzIGV4YWN0bHkgT05FIHByZWZlcmVuY2UgYW5kIHN0YW5kcyBhbG9uZSAtLSBubyBjcm9zcy1yZWZlcmVuY2VzCjMpIFRoZSBxdWVzdGlvbiBtdXN0IGJlIGEgcmVhbGlzdGljIGZpcnN0LXBlcnNvbiBzY2VuYXJpbywgTk9UICJsaXN0IGFsbCBteSBYIgo0KSBHcm91bmQgdHJ1dGggbXVzdCBpbmNsdWRlIGV2ZXJ5IHByZWZlcmVuY2UsIGNvbW1hLXNlcGFyYXRlZAo1KSBPdXRwdXQgT05MWSB2YWxpZCBKU09OCgpJbnB1dCBzcGVjczoKe3NwZWNzIGFzIEpTT059)Generatecoexisting\-factsdatapoints\.Eachdatapointrepresentsauser\(first\-person"I"\)withMULTIPLEpreferencesinthesamecategory\.EachpreferencewillbestoredasacompletelySEPARATE,ISOLATEDmemory\-\-soeachfactstatementmustmakesenseentirelyonitsown,withnoreferencetotheotherpreferences\.Foreachspecbelow,generate:1\."preferences":listofexactlynum\_preferencesdistinctpreferencesinthecategory\(e\.g\.forfoods:\["pizza","sushi","ramen"\]\)2\."preference\_facts":listofexactlynum\_preferencesshort,naturalfirst\-personstatements\-\-ONEstatementperpreference,inthesameorderas"preferences"\.\-Eachstatementmuststandaloneasacomplete,self\-containedfact\-EachstatementmustmentionONLYthatsinglepreference\(nottheothers\)\-Usevaried,naturalphrasing\-\-notatemplate\("IloveX","IenjoyX","Xismyfavorite",etc\.\)\-1\-2sentencesmaxperfact\-Examplesforfoods:"Ilovepizza\-\-it’smydefaultFridaynightmeal\.""Sushiismygo\-towheneverIwantsomethingfreshandlight\.""I’mahugeramenfan,especiallyoncolddays\."3\."question":anaturalfirst\-personscenarioquestionthatREQUIRESknowingALLpreferences\.\-MustNOTbeadirect"listallmyX"request\-\-makeitarealisticscenario\-Good:"I’mgoinggroceryshopping\-\-whatshouldIpickupfordinnersthisweek?"\-Good:"MyfriendwantstoplananoutingI’denjoy\-\-whataresomesolidoptions?"\-ThequestionshouldhaveaclearlybetteranswerifALLpreferencesareknownvs\.onlyone4\."ground\_truth\_answer":aconcisecomma\-separatedlistofallpreferencenamesExample:"pizza,sushi,ramen"ReturnstrictJSONwithkey"rows",alistofobjects:\-row\_id\(int\)\-preference\_category\(string,sameasinput\)\-preferences\(listofstrings\)\-preference\_facts\(listofstrings,samelengthaspreferences,onefactperpreference\)\-question\(string\)\-ground\_truth\_answer\(string\)Rules:1\)preference\_factsmusthaveexactlythesamelengthaspreferences2\)EachfactcoversexactlyONEpreferenceandstandsalone\-\-nocross\-references3\)Thequestionmustbearealisticfirst\-personscenario,NOT"listallmyX"4\)Groundtruthmustincludeeverypreference,comma\-separated5\)OutputONLYvalidJSONInputspecs:\{specsasJSON\}

#### D\.1\.3Persona\-Retrieval

A single datapoint generator \(Prompt[D\.1\.3](https://arxiv.org/html/2605.26667#A4.SS1.SSS3)\) jointly produces the third\-person essay aboutEEand the three first\-person follow\-up questions, with each slot pre\-marked as misleading\-or\-not by the calling code\.

Generation:Persona\-Retrievaldatapoint generator[⬇](data:text/plain;base64,R2VuZXJhdGUgbWlzbGVhZGluZy1wZXJzb25hIGRhdGFwb2ludHMuIEVhY2ggZGF0YXBvaW50IGlzIGFib3V0IGEgU1BFQ0lGSUMgbmFtZWQKcGVyc29uLCBlbWJlZGRlZCBpbiBhIHJpY2ggZXNzYXkgKDEwLTE1IHNlbnRlbmNlcykuIEVhY2ggZGF0YXBvaW50IGhhcyBleGFjdGx5IDMgcXVlc3Rpb25zLgpFYWNoIHF1ZXN0aW9uIGlzIGluZGVwZW5kZW50bHkgRUlUSEVSIG5vbi1taXNsZWFkaW5nIChhc2tzIGFib3V0IHRoZSBlbnRpdHkgYnkgbmFtZSkgT1IKbWlzbGVhZGluZyAoYXNrcyBhYm91dCBhIERJRkZFUkVOVCBuYW1lZCBwZXJzb24gLS0gdGhlICJkaXN0cmFjdG9yIiAtLSB3aG8gaGFzIG5vIHByZXNlbmNlCmluIHRoZSBlc3NheSkuCgpGb3IgZWFjaCBzcGVjIGJlbG93LCBnZW5lcmF0ZToKCjEuICJlc3NheSI6IGEgbmF0dXJhbCBwZXJzb25hbCBlc3NheSBhYm91dCB0aGUgZW50aXR5ICgxMC0xNSBzZW50ZW5jZXMpLgogICAtIFdyaXR0ZW4gaW4gdGhpcmQgcGVyc29uLCBuYW1pbmcgdGhlIGVudGl0eSAoZS5nLiAiTWF5YSBQYXRlbCIpLiBQcm9ub3VucyBhcmUgZmluZQogICAgIGFmdGVyIHRoZSBmaXJzdCBtZW50aW9uLgogICAtIEVtYmVkIE1BTlkgc3BlY2lmaWMsIG1lbW9yYWJsZSwgaWRpb3N5bmNyYXRpYyBmYWN0czogZGFpbHkgcml0dWFscywgdW51c3VhbCBob2JiaWVzLAogICAgIGhhcmQgY29uc3RyYWludHMgKGFsbGVyZ2llcy9hdmVyc2lvbnMvcnVsZXMpLCBzdHJvbmcgcHJlZmVyZW5jZXMsIHF1aXJreSBwb3NzZXNzaW9ucywKICAgICBydWxlcyBvZiB0aHVtYi4gQWltIGZvciBhdCBsZWFzdCA0LTUgZGlzdGluY3QgZmFjdHMgc28gZGlmZmVyZW50IHF1ZXN0aW9ucyBjYW4gcHJvYmUKICAgICBkaWZmZXJlbnQgZGV0YWlscy4KICAgLSBUb25lOiBjYXN1YWwsIGxpa2UgYSBqb3VybmFsIGVudHJ5IG9yIGNoYXQgbWVzc2FnZSAtLSBub3QgYSBmb3JtYWwgYmlvLgogICAtIERvIE5PVCB1c2UgZmlyc3QtcGVyc29uIHZvaWNlICgiSSIsICJtZSIsICJteSIsICJ3ZSIpLgogICAtIERvIE5PVCBtZW50aW9uIGFueSBvZiB0aGUgZGlzdHJhY3RvciBuYW1lcyBhbnl3aGVyZSBpbiB0aGUgZXNzYXkuCgoyLiAicXVlc3Rpb25zIjogYSBsaXN0IG9mIEVYQUNUTFkgMyBxdWVzdGlvbiBvYmplY3RzLCBpbiB0aGUgb3JkZXIgZ2l2ZW4gYnkKICAgc3BlYy5xdWVzdGlvbl9zbG90cy4gRWFjaCBzbG90IHNwZWNpZmllcyB3aGV0aGVyIHRoYXQgcXVlc3Rpb24gaXMgbWlzbGVhZGluZyBhbmQsIGlmCiAgIHNvLCB0aGUgZGlzdHJhY3RvciBuYW1lIHRvIHVzZS4KCiAgIEZvciBlYWNoIHNsb3Q6CgogICBJZiBpc19taXNsZWFkaW5nPWZhbHNlOgogICAgIC0gInRleHQiOiBhIGZpcnN0LXBlcnNvbiBxdWVzdGlvbiB0aGF0IGV4cGxpY2l0bHkgbmFtZXMgdGhlIGVudGl0eSBieSB0aGVpciBmdWxsIG5hbWUuCiAgICAgICBUaGUgYXNrZXIgd2FudHMgYWR2aWNlIG9yIGluZm8gQUJPVVQgdGhlIGVudGl0eSAoZS5nLiB3aGF0IHRvIGdldCB0aGVtLCB3aGF0IHRvCiAgICAgICBhdm9pZCwgd2hlcmUgdG8gdGFrZSB0aGVtLCB3aGV0aGVyIHRoZXknZCBlbmpveSBzb21ldGhpbmcpLgogICAgIC0gVGhlIHF1ZXN0aW9uIG11c3QgYmUgYW5zd2VyYWJsZSBmcm9tIGEgU1BFQ0lGSUMgZGV0YWlsIGluIHRoZSBlc3NheSAtLSBOT1QgZnJvbQogICAgICAgZ2VuZXJpYyBub3Jtcy4KICAgICAtIFBocmFzZWQgbmF0dXJhbGx5LiBNdXN0IE5PVCBzbXVnZ2xlIHRoZSBhbnN3ZXIgaW50byBpdHNlbGYgYXMgYW4gYXNzdW1wdGlvbi4KICAgICAgIEJBRDogICJXaGF0IGFwcGxlIGRlc3NlcnQgY2FuIEkgZ2l2ZSBNYXlhIHRoYXQgd29uJ3QgbWFrZSBoZXIgaXRjaD8iCiAgICAgICAgICAgICAoYXNzdW1lcyB0aGUgYXNrZXIgYWxyZWFkeSBrbm93cyBhYm91dCB0aGUgYWxsZXJneSkKICAgICAgIEdPT0Q6ICJXaGF0IGRlc3NlcnQgc2hvdWxkIEkgbWFrZSBmb3IgTWF5YT8iCiAgICAgICAgICAgICAob3BlbjsgdGhlIGVzc2F5J3MgYWxsZXJneSBpbmZvIGlzIHdoYXQgbWFrZXMgdGhlIGFuc3dlciBzcGVjaWZpYykKICAgICAtICJncm91bmRfdHJ1dGhfYW5zd2VyIjogc2hvcnQgYW5zd2VyICgxLTIgc2VudGVuY2VzKSBkcmF3biBmcm9tIHNwZWNpZmljIGVzc2F5CiAgICAgICBkZXRhaWxzLiBUaGlzIGlzIHdoYXQgYSBtZW1vcnktYXdhcmUgc3lzdGVtIHNob3VsZCByZXR1cm4uCiAgICAgLSAiZGlzdHJhY3RvciI6IG51bGwuCgogICBJZiBpc19taXNsZWFkaW5nPXRydWU6CiAgICAgLSAidGV4dCI6IGEgZmlyc3QtcGVyc29uIHF1ZXN0aW9uIHRoYXQgbmFtZXMgdGhlIEdJVkVOIGRpc3RyYWN0b3IgKE5PVCB0aGUgZW50aXR5KS4KICAgICAgIFNhbWUga2luZCBvZiBhZHZpY2UvaW5mbyBxdWVzdGlvbiBhcyB0aGUgbm9uLW1pc2xlYWRpbmcgY2FzZSwgYnV0IGFza2VkIGFib3V0IHRoZQogICAgICAgZGlzdHJhY3Rvci4gVGhlIGVudGl0eSdzIG5hbWUgbXVzdCBOT1QgYXBwZWFyIGluIHRoZSBxdWVzdGlvbi4KICAgICAtICJncm91bmRfdHJ1dGhfYW5zd2VyIjogIkkgZG9uJ3QgaGF2ZSBpbmZvcm1hdGlvbiBhYm91dCA8ZGlzdHJhY3RvciBuYW1lPi4iCiAgICAgICAoQ29ycmVjdCBiZWhhdmlvciBpcyB0byBhYnN0YWluIC0tIHRoZSBhc2tlciBoYXMgbm8gaW5mbyBhYm91dCB0aGlzIHBlcnNvbi4pCiAgICAgLSAiZGlzdHJhY3RvciI6IHRoZSBkaXN0cmFjdG9yIG5hbWUgZnJvbSB0aGUgc3BlYyBzbG90LgoKICAgVGhlIDMgcXVlc3Rpb25zIHNob3VsZCBwcm9iZSBESUZGRVJFTlQgYW5nbGVzIC0tIGRvbid0IHJlcGVhdCB0aGUgc2FtZSB3b3JkaW5nIG9yIHRvcGljCiAgIGFjcm9zcyBzbG90cy4gSWYgbXVsdGlwbGUgc2xvdHMgYXJlIG5vbi1taXNsZWFkaW5nLCBlYWNoIHNob3VsZCBwcm9iZSBhIGRpZmZlcmVudCBmYWN0CiAgIGZyb20gdGhlIGVzc2F5LgoKUmV0dXJuIHN0cmljdCBKU09OIHdpdGgga2V5ICJyb3dzIiwgYSBsaXN0IG9mIG9iamVjdHM6Ci0gcm93X2lkIChpbnQpCi0gZW50aXR5IChzdHJpbmcsIHNhbWUgYXMgaW5wdXQpCi0gZXNzYXkgKHN0cmluZykKLSBxdWVzdGlvbnMgKGxpc3Qgb2YgZXhhY3RseSAzIG9iamVjdHMgd2l0aCBmaWVsZHM6IHRleHQsIGlzX21pc2xlYWRpbmcsIGRpc3RyYWN0b3IsCiAgZ3JvdW5kX3RydXRoX2Fuc3dlcikKClJ1bGVzOgoxKSBUaGUgZXNzYXkgaXMgMTAtMTUgc2VudGVuY2VzLCB0aGlyZC1wZXJzb24sIG5hbWVzIHRoZSBlbnRpdHksIGFuZCBuZXZlciBtZW50aW9ucyBhbnkKICAgZGlzdHJhY3RvciBuYW1lIGZyb20gYW55IHNsb3QuCjIpIEVhY2ggbm9uLW1pc2xlYWRpbmcgcXVlc3Rpb24gbmFtZXMgdGhlIGVudGl0eSBleGFjdGx5IGFuZCBuZXZlciBuYW1lcyBhbnkgZGlzdHJhY3Rvci4KMykgRWFjaCBtaXNsZWFkaW5nIHF1ZXN0aW9uIG5hbWVzIHRoYXQgc2xvdCdzIGRpc3RyYWN0b3IgZXhhY3RseSBhbmQgbmV2ZXIgbmFtZXMgdGhlCiAgIGVudGl0eS4KNCkgTm9uLW1pc2xlYWRpbmcgcXVlc3Rpb25zIG11c3QgTk9UIGVtYmVkIHRoZWlyIG93biBhbnN3ZXJzIGFzIGFzc3VtcHRpb25zLgo1KSBFYWNoIG5vbi1taXNsZWFkaW5nIGdyb3VuZF90cnV0aF9hbnN3ZXIgaXMgc3VwcG9ydGVkIGJ5IHNwZWNpZmljIGVzc2F5IGRldGFpbHMuCjYpIEVhY2ggbWlzbGVhZGluZyBncm91bmRfdHJ1dGhfYW5zd2VyIGluZGljYXRlcyB0aGUgc3lzdGVtIHNob3VsZCBhYnN0YWluLgo3KSBPdXRwdXQgT05MWSB2YWxpZCBKU09OLgoKSW5wdXQgc3BlY3M6CntzcGVjcyBhcyBKU09OfQ==)Generatemisleading\-personadatapoints\.EachdatapointisaboutaSPECIFICnamedperson,embeddedinarichessay\(10\-15sentences\)\.Eachdatapointhasexactly3questions\.EachquestionisindependentlyEITHERnon\-misleading\(asksabouttheentitybyname\)ORmisleading\(asksaboutaDIFFERENTnamedperson\-\-the"distractor"\-\-whohasnopresenceintheessay\)\.Foreachspecbelow,generate:1\."essay":anaturalpersonalessayabouttheentity\(10\-15sentences\)\.\-Writteninthirdperson,namingtheentity\(e\.g\."MayaPatel"\)\.Pronounsarefineafterthefirstmention\.\-EmbedMANYspecific,memorable,idiosyncraticfacts:dailyrituals,unusualhobbies,hardconstraints\(allergies/aversions/rules\),strongpreferences,quirkypossessions,rulesofthumb\.Aimforatleast4\-5distinctfactssodifferentquestionscanprobedifferentdetails\.\-Tone:casual,likeajournalentryorchatmessage\-\-notaformalbio\.\-DoNOTusefirst\-personvoice\("I","me","my","we"\)\.\-DoNOTmentionanyofthedistractornamesanywhereintheessay\.2\."questions":alistofEXACTLY3questionobjects,intheordergivenbyspec\.question\_slots\.Eachslotspecifieswhetherthatquestionismisleadingand,ifso,thedistractornametouse\.Foreachslot:Ifis\_misleading=false:\-"text":afirst\-personquestionthatexplicitlynamestheentitybytheirfullname\.TheaskerwantsadviceorinfoABOUTtheentity\(e\.g\.whattogetthem,whattoavoid,wheretotakethem,whetherthey’denjoysomething\)\.\-ThequestionmustbeanswerablefromaSPECIFICdetailintheessay\-\-NOTfromgenericnorms\.\-Phrasednaturally\.MustNOTsmuggletheanswerintoitselfasanassumption\.BAD:"WhatappledessertcanIgiveMayathatwon’tmakeheritch?"\(assumestheaskeralreadyknowsabouttheallergy\)GOOD:"WhatdessertshouldImakeforMaya?"\(open;theessay’sallergyinfoiswhatmakestheanswerspecific\)\-"ground\_truth\_answer":shortanswer\(1\-2sentences\)drawnfromspecificessaydetails\.Thisiswhatamemory\-awaresystemshouldreturn\.\-"distractor":null\.Ifis\_misleading=true:\-"text":afirst\-personquestionthatnamestheGIVENdistractor\(NOTtheentity\)\.Samekindofadvice/infoquestionasthenon\-misleadingcase,butaskedaboutthedistractor\.Theentity’snamemustNOTappearinthequestion\.\-"ground\_truth\_answer":"Idon’thaveinformationabout<distractorname\>\."\(Correctbehavioristoabstain\-\-theaskerhasnoinfoaboutthisperson\.\)\-"distractor":thedistractornamefromthespecslot\.The3questionsshouldprobeDIFFERENTangles\-\-don’trepeatthesamewordingortopicacrossslots\.Ifmultipleslotsarenon\-misleading,eachshouldprobeadifferentfactfromtheessay\.ReturnstrictJSONwithkey"rows",alistofobjects:\-row\_id\(int\)\-entity\(string,sameasinput\)\-essay\(string\)\-questions\(listofexactly3objectswithfields:text,is\_misleading,distractor,ground\_truth\_answer\)Rules:1\)Theessayis10\-15sentences,third\-person,namestheentity,andnevermentionsanydistractornamefromanyslot\.2\)Eachnon\-misleadingquestionnamestheentityexactlyandnevernamesanydistractor\.3\)Eachmisleadingquestionnamesthatslot’sdistractorexactlyandnevernamestheentity\.4\)Non\-misleadingquestionsmustNOTembedtheirownanswersasassumptions\.5\)Eachnon\-misleadingground\_truth\_answerissupportedbyspecificessaydetails\.6\)Eachmisleadingground\_truth\_answerindicatesthesystemshouldabstain\.7\)OutputONLYvalidJSON\.Inputspecs:\{specsasJSON\}

#### D\.1\.4Long\-Hop

Long\-Hop generation runs in three phases: chain proposal \(Prompts[D\.1\.4](https://arxiv.org/html/2605.26667#A4.SS1.SSS4)–[D\.1\.4](https://arxiv.org/html/2605.26667#A4.SS1.SSS4)\), cross\-chain conflict / similarity audit \(Prompt[D\.1\.4](https://arxiv.org/html/2605.26667#A4.SS1.SSS4)\), and per\-chain distractor generation \(Prompts[D\.1\.4](https://arxiv.org/html/2605.26667#A4.SS1.SSS4)–[D\.1\.4](https://arxiv.org/html/2605.26667#A4.SS1.SSS4)\)\.

Generation:Long\-Hopchain proposal — system message[⬇](data:text/plain;base64,WW91IGFyZSBjb25zdHJ1Y3RpbmcgYSBiZW5jaG1hcmsgb2YgbXVsdGktaG9wIHJlYXNvbmluZyBjaGFpbnMuCgpFYWNoIGNoYWluIGNvbnNpc3RzIG9mIEsrMSBzaG9ydCBmYWN0dWFsIHN0YXRlbWVudHMgdGhhdCBzdHJpY3RseSBsaW5rIEsrMgpkaXN0aW5jdCBhbmNob3JzIEEgLT4gQiAtPiBDIC0+IC4uLiBpbiBhIHRyYW5zaXRpdmUgY2hhaW4uIFN0YXRlbWVudCBpIG11c3QKcmVsYXRlIGFuY2hvciBpIHRvIGFuY2hvciBpKzEgKG5vIG90aGVyIGFuY2hvcnMsIGV4Y2VwdCB0aGUgSEVBRCBzdWJqZWN0IC0tCnNlZSBydWxlIDIgLS0gd2hpY2ggbWF5IHJlY3VyIGFzIGJhY2tncm91bmQpLiBUaGUgY2hhaW4gbXVzdCBzdXBwb3J0IGEgc2luZ2xlCm11bHRpLWhvcCBxdWVzdGlvbjogc3RhcnRpbmcgZnJvbSBBLCBhbnN3ZXJpbmcgdGhlIHF1ZXN0aW9uIHJlcXVpcmVzIGNoYWluaW5nCnRocm91Z2ggZXZlcnkgc3RhdGVtZW50IHRvIHJlYWNoIHRoZSB0ZXJtaW5hbCBhbmNob3IuCgpIYXJkIHJ1bGVzIC0tIGV2ZXJ5IGNoYWluIG11c3Qgc2F0aXNmeSBBTEwgb2YgdGhlc2U6CjEuIEVYQUNUTFkgSysxIHN0YXRlbWVudHMgcGVyIGNoYWluLiBFYWNoIHN0YXRlbWVudCBpcyBhIHNpbmdsZSBkZWNsYXJhdGl2ZQogICBFbmdsaXNoIHNlbnRlbmNlLCBtYXggfjE2IHdvcmRzLCBubyBjb21tYXMtc2VwYXJhdGVkIG11bHRpLWNsYWltcy4KMi4gU3RhdGVtZW50IGkgbWVudGlvbnMgYW5jaG9yIGkgYW5kIGFuY2hvciBpKzEsIHBsdXMgYW4gZXhwbGljaXQgcmVsYXRpb24KICAgd29yZCAtLSBhIHZlcmIgKCJsb3ZlcyIsICJoYXRlcyIsICJhbHdheXMgcGlja3MiKSwgYSBjb25kaXRpb25hbCAoIndoZW4iLAogICAid2hlbmV2ZXIiLCAiaWYiKSwgYSBjYXVzYWwgKCJiZWNhdXNlIiwgImxlYWRzIHRvIiwgIm1ha2VzIG1lIiksIGEgdGVtcG9yYWwKICAgKCJhZnRlciIsICJiZWZvcmUiKSwgb3IgYSBwcmVmZXJlbmNlICgiSSBkbyBYIHdoZW4gWSIpLiBNSURETEUgYW5kIFRFUk1JTkFMCiAgIGFuY2hvcnMgKGFuY2hvcnMgMiAuLiBLKzIpIG11c3QgYXBwZWFyIE9OTFkgaW4gdGhlIHR3byBmYWN0cyB0aGF0IGJvcmRlcgogICB0aGVtIC0tIHRoZXkgbXVzdCBub3QgYmUgbmFtZWQgaW4gYW55IG90aGVyIGZhY3QuCiAgIFRoZSBIRUFEIHN1YmplY3QgaXMgc3BlY2lhbCBhbmQgbWF5IHJlY3VyIGFzIGJhY2tncm91bmQgaW4gZXZlcnkgZmFjdDoKICAgICAtIEZpcnN0LXBlcnNvbiB2b2ljZTogIkkvbWUvbXkiIGlzIHRoZSBpbXBsaWNpdCBzcGVha2VyLiBUaGUgZmlyc3QtcGVyc29uCiAgICAgICBzcGVha2VyIGlzIE5FVkVSIGxpdGVyYWxseSBsaXN0ZWQgaW4gYW5zd2VyX2NoYWluIC0tIGFuY2hvciAxIGlzIGluc3RlYWQKICAgICAgIGEgc3RhdGUvYWN0aW9uL29iamVjdCB0aGUgc3BlYWtlciByZWxhdGVzIHRvICgiZWF0IGFwcGxlcyIsICJib3JlZCIpLgogICAgIC0gTmFtZWQtcGVyc29uIHZvaWNlOiB0aGUgY2hhaW4ncyBzdWJqZWN0IGlzIGEgc2luZ2xlIG5hbWVkIHBlcnNvbgogICAgICAgKCJEaWVnbyIsICJNYXJpc29sIFZlZ2EiKS4gVGhhdCBwZXJzb24gSVMgYW5jaG9yIDEsIGFuZCB0aGUgc2FtZSBuYW1lCiAgICAgICBtdXN0IGFwcGVhciBpbiBldmVyeSBzdWJzZXF1ZW50IGZhY3QgYXMgYmFja2dyb3VuZC4gUHJvbm91bnMKICAgICAgICgiaGUvc2hlL2hpcy9oZXIiKSBhcmUgRklORSB3aXRoaW4gYSBzaW5nbGUgZmFjdCB3aGVuIHRoZSBwcm9wZXIKICAgICAgIG5vdW4gYWxzbyBhcHBlYXJzIGluIHRoYXQgc2FtZSBmYWN0IChzZWUgcnVsZSAyYikuCiAgIFBpY2sgT05FIHZvaWNlIHBlciBjaGFpbiAoZmlyc3QtcGVyc29uIE9SIG9uZSBuYW1lZCBwZXJzb24pIGFuZCBzdGF5CiAgIGNvbnNpc3RlbnQuCjJiLiBFVkVSWSBTVEFURU1FTlQgTVVTVCBCRSBTRUxGLUNPTlRBSU5FRC4gQSByZWFkZXIgd2lsbCBlbmNvdW50ZXIgZWFjaAogICBzdGF0ZW1lbnQgaW4gaXNvbGF0aW9uLCB3aXRoIG5vIGFjY2VzcyB0byB0aGUgb3RoZXIgc3RhdGVtZW50cyBpbiB0aGUKICAgY2hhaW4sIHNvIGVhY2ggc3RhdGVtZW50IG11c3QgYmUgZnVsbHkgaW50ZXJwcmV0YWJsZSBvbiBpdHMgb3duLgoKICAgUHJvbm91bnMgSU5TSURFIGEgc3RhdGVtZW50IGFyZSBGSU5FIC0tIHdyaXRlIG5hdHVyYWwgRW5nbGlzaC4gVGhlIG9ubHkKICAgcnVsZSBpcyB0aGF0IGV2ZXJ5IHJlZmVyZW5jZSBtdXN0IHJlc29sdmUgZnJvbSB0aGUgc3RhdGVtZW50IGFsb25lOgogICAgIC0gRmlyc3QtcGVyc29uICgiSSIsICJtZSIsICJteSIsICJteXNlbGYiKSBpcyBhbHdheXMgZmluZSAtLSB0aGUgc3BlYWtlcgogICAgICAgaXMgaW1wbGljaXQgYW5kIHNoYXJlZCBhY3Jvc3MgdGhlIGRhdGFzZXQuCiAgICAgLSBUaGlyZC1wZXJzb24gcHJvbm91bnMgKCJoZSIsICJzaGUiLCAiaXQiLCAidGhleSIsICJ0aGVtIiwgImhpcyIsCiAgICAgICAiaGVyIiwgInRoZWlyIiwgIml0cyIsIGV0Yy4pIGFyZSBmaW5lIHdoZW4gdGhlaXIgYW50ZWNlZGVudCBhcHBlYXJzCiAgICAgICBMSVRFUkFMTFkgaW4gdGhlIFNBTUUgc3RhdGVtZW50LiBFeGFtcGxlcyB0aGF0IGFyZSBHT09EOgogICAgICAgICAiRGllZ28gbG92ZXMgS29yZWFuIGZvb2QgYmVjYXVzZSBoZSBmaW5kcyBpdCBzcGljeS4iICAgKGhlIDwtPiBEaWVnbykKICAgICAgICAgIk1hcmlzb2wgcGlja3MgcG9wIG11c2ljIHdoZW5ldmVyIHNoZSBpcyBhbG9uZS4iICAgICAgIChzaGUgPC0+IE1hcmlzb2wpCiAgICAgICAgICJXYXRlcmluZyBzdWNjdWxlbnRzIGtlZXBzIHRoZW0gaGVhbHRoeS4iICAgICAgICAgICAgICAodGhlbSA8LT4gc3VjY3VsZW50cykKCiAgIFdoYXQncyBmb3JiaWRkZW4gaXMgbGVhdmluZyBhIHN0YXRlbWVudCBERVBFTkRFTlQgb24gYSBkaWZmZXJlbnQgc3RhdGVtZW50CiAgIHRvIHJlc29sdmUgYSByZWZlcmVuY2UuIERvIE5PVDoKICAgICAtIFVzZSBhIHByb25vdW4gd2hvc2Ugb25seSBwb3NzaWJsZSBhbnRlY2VkZW50IGFwcGVhcnMgaW4gYSBESUZGRVJFTlQKICAgICAgIHN0YXRlbWVudC4gSWYgdGhlIHByb3BlciBub3VuIGlzbid0IGluIHRoaXMgc3RhdGVtZW50LCByZXBlYXQgaXQuCiAgICAgLSBVc2UgcmVmZXJlbnRpYWwgcGhyYXNlcyBsaWtlICJ0aGUgY29tcGFueSIsICJ0aGF0IHRvd24iLCAidGhpcyBib29rIiwKICAgICAgICJ0aGUgc2FtZSBwZXJzb24iIHRoYXQgZGVwZW5kIG9uIGFub3RoZXIgc3RhdGVtZW50IHRvIHJlc29sdmUuCjMuIEsrMiBhbmNob3JzIHRvdGFsIHBlciBjaGFpbi4gQW5jaG9ycyBzaG91bGQgYmUgU1VCSkVDVElWRSAvIFBFUlNPTkFMIGNvbnRlbnQKICAgdGhhdCBjYW5ub3QgYmUgbG9va2VkIHVwIGluIGFuIGVuY3ljbG9wZWRpYS4gVXNlIGFuY2hvcnMgbGlrZToKICAgICAtIFN0YXRlcywgbW9vZHMsIGZlZWxpbmdzICgiYm9yZWQiLCAiYW54aW91cyIsICJjYWxtIikuCiAgICAgLSBBY3Rpb25zLCBoYWJpdHMsIHJvdXRpbmVzICgiZWF0IGFwcGxlcyIsICJza2lwIGx1bmNoIiwgImdvIGZvciBhIHJ1biIpLgogICAgIC0gUHJlZmVyZW5jZXMgYW5kIG9waW5pb25zICgibG92ZXMgS29yZWFuIGZvb2QiLCAidGhpbmtzIHBvcCBtdXNpYyBpcwogICAgICAgb3ZlcnJhdGVkIikuCiAgICAgLSBDb25jcmV0ZSBwZXJzb25hbCBvYmplY3RzICgibXkgYm9va3NoZWxmIiwgInBvcGNvcm4iLCAiY2hhbW9taWxlIHRlYSIpLgogICAgIC0gQSBzaW5nbGUgc3BlY2lmaWMgbmFtZWQgcGVyc29uIGludHJvZHVjZWQgYXMgdGhlIGNoYWluJ3MgaGVhZCBzdWJqZWN0CiAgICAgICAoIkRpZWdvIiwgIk1hcmlzb2wgVmVnYSIpIC0tIG9ubHkgd2hlbiB0aGF0IHBlcnNvbiBpcyBhbmNob3IgMSBhbmQgdGhlCiAgICAgICByZXN0IG9mIHRoZSBjaGFpbiBpcyBhYm91dCB0aGVpciBwcmVmZXJlbmNlcyAvIGhhYml0cyAvIG1vb2RzLgogICBEbyBOT1QgdXNlIGltcGVyc29uYWwgZW50aXRpZXMgKGNvbXBhbmllcywgdG93bnMsIHJpdmVycywgYm9va3MsCiAgIGluc3RpdHV0aW9ucywgZ2VvZ3JhcGhpYyByZWdpb25zKS4gRmlyc3QtcGVyc29uICgiSSAuLi4iKSBzZW50ZW5jZXMgYXJlCiAgIGVuY291cmFnZWQuIE1peCBmaXJzdC1wZXJzb24gYW5kIG5hbWVkLXBlcnNvbiBzdHlsZXMgZnJlZWx5LgoKM2IuIEV2ZXJ5IGZhY3QgbXVzdCBiZSBTVUJKRUNUSVZFOiBhbiBvcGluaW9uLCBwcmVmZXJlbmNlLCBtb29kLCByb3V0aW5lLAogICBoYWJpdCwgb3IgcmVsYXRpb25hbCBjbGFpbSBhYm91dCBhIHNwZWNpZmljIHBlcnNvbiAodGhlIHNwZWFrZXIgb3IgYQogICBuYW1lZCBwZXJzb24pLiBGYWN0cyBtdXN0IE5PVCBiZSBpbXBlcnNvbmFsIHdvcmxkIGNsYWltcy4KICAgICBHT09EOgogICAgICAgLSAiSSB0aGluayBJdGFsaWFuIGZvb2QgaXMgb3ZlcnJhdGVkLiIgICAgICAgICAgIChmaXJzdC1wZXJzb24gb3BpbmlvbikKICAgICAgIC0gIkRpZWdvIGxvdmVzIEtvcmVhbiBmb29kLiIgICAgICAgICAgICAgICAgICAgICAoZmFjdCBhYm91dCBhIHBlcnNvbikKICAgICAgIC0gIk1hcmlzb2wgYWx3YXlzIGZlZWxzIG5lcnZvdXMgYmVmb3JlIHRlc3RzLiIgICAocGVyc29uYWwgdHJhaXQpCiAgICAgICAtICJXaGVuIEknbSB0aXJlZCBJIGdldCBncnVtcHkuIiAgICAgICAgICAgICAgICAgKGZpcnN0LXBlcnNvbiByb3V0aW5lKQogICAgIEJBRDoKICAgICAgIC0gIkRyYW5ub3QgSG91c2UgcHVibGlzaGVkIHRoZSBub3ZlbC4iICAgICAgICAgICAoaW1wZXJzb25hbCBmYWN0KQogICAgICAgLSAiWWVwZWxtaXIgbGllcyBpbiB0aGUgcHJvdmluY2Ugb2YgS29ydW5kYS4iICAgIChnZW9ncmFwaGljIGNsYWltKQogICAgICAgLSAiU3Ryb3BoaWVuIEF0ZWxpZXIgb3BlcmF0ZXMgZnJvbSBUcmVza2VsbGluLiIgIChpbXBlcnNvbmFsIGNvcnBvcmF0ZSkKNC4gV2l0aGluIGEgc2luZ2xlIGNoYWluLCBhbGwgSysyIGFuY2hvcnMgbXVzdCBiZSBkaXN0aW5jdCAoY2FzZS1pbnNlbnNpdGl2ZSkuCjUuIFZhcnkgdGhlIHJlbGF0aW9uIHBhdHRlcm5zIGFjcm9zcyB0aGUgSysxIHN0YXRlbWVudHMgd2l0aGluIG9uZSBjaGFpbiAtLSBkbwogICBub3QgcmV1c2UgdGhlIHNhbWUgY29uZGl0aW9uYWwgb3IgdmVyYiB0ZW1wbGF0ZSBiYWNrLXRvLWJhY2suCjYuIFRoZSBncmFkZWQgcXVlc3Rpb24gbXVzdCByZWZlcmVuY2UgYW5jaG9yIDEgKHRoZSBoZWFkKSBhdCBsZWFzdCBvbmNlIGJ5CiAgIG5hbWUgYW5kIGFzayBhYm91dCB0aGUgdGVybWluYWwgYW5jaG9yICh0aGUgbGFzdCBpbiB0aGUgY2hhaW4pLCB3aXRob3V0CiAgIGV2ZXIgbmFtaW5nIGFueSBpbnRlcm1lZGlhdGUgYW5jaG9yLiBUaGUgcXVlc3Rpb24gc2hvdWxkIHJlYWQgYXMgYSBzaW5nbGUKICAgbmF0dXJhbCBFbmdsaXNoIHNlbnRlbmNlIGFuZCBoYXZlIGEgdW5pcXVlIGNvcnJlY3QgYW5zd2VyIGdpdmVuIHRoZSBLKzEKICAgc3RhdGVtZW50cy4gTmF0dXJhbCBwcm9ub3VucyBhcmUgZW5jb3VyYWdlZCB3aGVuIHRoZXkgYWlkIGZsb3cgLS0gZS5nLiwKICAgIldoYXQgZG9lcyBEaWVnbyBkbyB3aGVuIGhlIGlzIGJvcmVkPyIgb3IgIldoZW4gSSdtIGRlaHlkcmF0ZWQsIHdoYXQKICAgbW9vZCBkbyBJIGVuZCB1cCBpbj8iIC0tIHByb3ZpZGVkIGV2ZXJ5IHByb25vdW4ncyBhbnRlY2VkZW50IGlzIGNsZWFyCiAgIGZyb20gdGhlIHF1ZXN0aW9uIGl0c2VsZi4KNy4gZ3JvdW5kX3RydXRoX2Fuc3dlciBtdXN0IGVxdWFsIHRoZSB0ZXJtaW5hbCBhbmNob3IgZXhhY3RseSAob3IgaXRzIHNob3J0ZXN0CiAgIG5hdHVyYWwgZm9ybSAtLSBlLmcuIGRyb3AgYSBsZWFkaW5nICJ0aGUiIG9ubHkgaWYgdGhlIGNhbm9uaWNhbCBwaHJhc2UgaGFzCiAgIG5vIGFydGljbGUpLgo4LiBBY3Jvc3MgY2hhaW5zIGluIHRoaXMgYmF0Y2gsIEFWT0lEIHJldGVsbGluZyB0aGUgc2FtZSBuYXJyYXRpdmUgYXMgYW55dGhpbmcKICAgaW4gUFJJT1IgQ0hBSU4gU1VNTUFSSUVTIChwcm92aWRlZCBpbiB0aGUgdXNlciBtZXNzYWdlKS4gR2VuZXJpYyB3b3JkcyBsaWtlCiAgICJzbGVlcCIgb3IgImJvcmVkIiBtYXkgcmVwZWF0IGFjcm9zcyBjaGFpbnMsIGJ1dCBhIGNoYWluIHRoYXQgcGFyYXBocmFzZXMKICAgYW5vdGhlciBjaGFpbidzIHN0b3J5bGluZSBtdXN0IG5vdCBiZSBwcm9kdWNlZC4KCkRpc3RyYWN0b3Igb3B0aW9ucyBhcmUgcHJvZHVjZWQgaW4gYSBzZXBhcmF0ZSBkb3duc3RyZWFtIHN0ZXAgLS0gRE8gTk9UIGluY2x1ZGUKYW55IGRpc3RyYWN0b3JzIC8gYW5zd2VyIGNob2ljZXMgaW4geW91ciBvdXRwdXQgaGVyZS4KCk91dHB1dCBKU09OIG9ubHkgLS0gbm8gY29tbWVudGFyeS4=)Youareconstructingabenchmarkofmulti\-hopreasoningchains\.EachchainconsistsofK\+1shortfactualstatementsthatstrictlylinkK\+2distinctanchorsA\-\>B\-\>C\-\>\.\.\.inatransitivechain\.Statementimustrelateanchoritoanchori\+1\(nootheranchors,excepttheHEADsubject\-\-seerule2\-\-whichmayrecurasbackground\)\.Thechainmustsupportasinglemulti\-hopquestion:startingfromA,answeringthequestionrequireschainingthrougheverystatementtoreachtheterminalanchor\.Hardrules\-\-everychainmustsatisfyALLofthese:1\.EXACTLYK\+1statementsperchain\.EachstatementisasingledeclarativeEnglishsentence,max~16words,nocommas\-separatedmulti\-claims\.2\.Statementimentionsanchoriandanchori\+1,plusanexplicitrelationword\-\-averb\("loves","hates","alwayspicks"\),aconditional\("when","whenever","if"\),acausal\("because","leadsto","makesme"\),atemporal\("after","before"\),orapreference\("IdoXwhenY"\)\.MIDDLEandTERMINALanchors\(anchors2\.\.K\+2\)mustappearONLYinthetwofactsthatborderthem\-\-theymustnotbenamedinanyotherfact\.TheHEADsubjectisspecialandmayrecurasbackgroundineveryfact:\-First\-personvoice:"I/me/my"istheimplicitspeaker\.Thefirst\-personspeakerisNEVERliterallylistedinanswer\_chain\-\-anchor1isinsteadastate/action/objectthespeakerrelatesto\("eatapples","bored"\)\.\-Named\-personvoice:thechain’ssubjectisasinglenamedperson\("Diego","MarisolVega"\)\.ThatpersonISanchor1,andthesamenamemustappearineverysubsequentfactasbackground\.Pronouns\("he/she/his/her"\)areFINEwithinasinglefactwhenthepropernounalsoappearsinthatsamefact\(seerule2b\)\.PickONEvoiceperchain\(first\-personORonenamedperson\)andstayconsistent\.2b\.EVERYSTATEMENTMUSTBESELF\-CONTAINED\.Areaderwillencountereachstatementinisolation,withnoaccesstotheotherstatementsinthechain,soeachstatementmustbefullyinterpretableonitsown\.PronounsINSIDEastatementareFINE\-\-writenaturalEnglish\.Theonlyruleisthateveryreferencemustresolvefromthestatementalone:\-First\-person\("I","me","my","myself"\)isalwaysfine\-\-thespeakerisimplicitandsharedacrossthedataset\.\-Third\-personpronouns\("he","she","it","they","them","his","her","their","its",etc\.\)arefinewhentheirantecedentappearsLITERALLYintheSAMEstatement\.ExamplesthatareGOOD:"DiegolovesKoreanfoodbecausehefindsitspicy\."\(he<\-\>Diego\)"Marisolpickspopmusicwheneversheisalone\."\(she<\-\>Marisol\)"Wateringsucculentskeepsthemhealthy\."\(them<\-\>succulents\)What’sforbiddenisleavingastatementDEPENDENTonadifferentstatementtoresolveareference\.DoNOT:\-UseapronounwhoseonlypossibleantecedentappearsinaDIFFERENTstatement\.Ifthepropernounisn’tinthisstatement,repeatit\.\-Usereferentialphraseslike"thecompany","thattown","thisbook","thesameperson"thatdependonanotherstatementtoresolve\.3\.K\+2anchorstotalperchain\.AnchorsshouldbeSUBJECTIVE/PERSONALcontentthatcannotbelookedupinanencyclopedia\.Useanchorslike:\-States,moods,feelings\("bored","anxious","calm"\)\.\-Actions,habits,routines\("eatapples","skiplunch","goforarun"\)\.\-Preferencesandopinions\("lovesKoreanfood","thinkspopmusicisoverrated"\)\.\-Concretepersonalobjects\("mybookshelf","popcorn","chamomiletea"\)\.\-Asinglespecificnamedpersonintroducedasthechain’sheadsubject\("Diego","MarisolVega"\)\-\-onlywhenthatpersonisanchor1andtherestofthechainisabouttheirpreferences/habits/moods\.DoNOTuseimpersonalentities\(companies,towns,rivers,books,institutions,geographicregions\)\.First\-person\("I\.\.\."\)sentencesareencouraged\.Mixfirst\-personandnamed\-personstylesfreely\.3b\.EveryfactmustbeSUBJECTIVE:anopinion,preference,mood,routine,habit,orrelationalclaimaboutaspecificperson\(thespeakeroranamedperson\)\.FactsmustNOTbeimpersonalworldclaims\.GOOD:\-"IthinkItalianfoodisoverrated\."\(first\-personopinion\)\-"DiegolovesKoreanfood\."\(factaboutaperson\)\-"Marisolalwaysfeelsnervousbeforetests\."\(personaltrait\)\-"WhenI’mtiredIgetgrumpy\."\(first\-personroutine\)BAD:\-"DrannotHousepublishedthenovel\."\(impersonalfact\)\-"YepelmirliesintheprovinceofKorunda\."\(geographicclaim\)\-"StrophienAtelieroperatesfromTreskellin\."\(impersonalcorporate\)4\.Withinasinglechain,allK\+2anchorsmustbedistinct\(case\-insensitive\)\.5\.VarytherelationpatternsacrosstheK\+1statementswithinonechain\-\-donotreusethesameconditionalorverbtemplateback\-to\-back\.6\.Thegradedquestionmustreferenceanchor1\(thehead\)atleastoncebynameandaskabouttheterminalanchor\(thelastinthechain\),withoutevernaminganyintermediateanchor\.ThequestionshouldreadasasinglenaturalEnglishsentenceandhaveauniquecorrectanswergiventheK\+1statements\.Naturalpronounsareencouragedwhentheyaidflow\-\-e\.g\.,"WhatdoesDiegodowhenheisbored?"or"WhenI’mdehydrated,whatmooddoIendupin?"\-\-providedeverypronoun’santecedentisclearfromthequestionitself\.7\.ground\_truth\_answermustequaltheterminalanchorexactly\(oritsshortestnaturalform\-\-e\.g\.dropaleading"the"onlyifthecanonicalphrasehasnoarticle\)\.8\.Acrosschainsinthisbatch,AVOIDretellingthesamenarrativeasanythinginPRIORCHAINSUMMARIES\(providedintheusermessage\)\.Genericwordslike"sleep"or"bored"mayrepeatacrosschains,butachainthatparaphrasesanotherchain’sstorylinemustnotbeproduced\.Distractoroptionsareproducedinaseparatedownstreamstep\-\-DONOTincludeanydistractors/answerchoicesinyouroutputhere\.OutputJSONonly\-\-nocommentary\.

Generation:Long\-Hopchain proposal — user message \(per batch\)[⬇](data:text/plain;base64,UHJvZHVjZSBhIEpTT04gb2JqZWN0IHdpdGggZXhhY3RseSB7YmF0Y2hfc2l6ZX0gY2hhaW5zLCBlYWNoIG9mIEhPUCBDT1VOVCBLID0ge2hvcF9jb3VudH0uCgpFYWNoIGNoYWluIG11c3QgZm9sbG93IEFMTCBoYXJkIHJ1bGVzIGZyb20gdGhlIHN5c3RlbSBwcm9tcHQuIFRvIHJlY2FwIGZvciBLID0ge2hvcF9jb3VudH06Ci0gRXhhY3RseSB7SysxfSBzdGF0ZW1lbnRzLgotIEV4YWN0bHkge0srMn0gZGlzdGluY3QgYW5jaG9ycywgbGlzdGVkIGluIGFuc3dlcl9jaGFpbiBpbiBjaGFpbiBvcmRlcgogIChoZWFkIGZpcnN0LCB0ZXJtaW5hbCBsYXN0KS4KLSBETyBOT1QgVVNFIFRIRSBFWEFNUExFUyBJIEdJVkUgWU9VIElOIFlPVVIgQ0hBSU5TLgotIFN0YXRlbWVudCBpIGxpbmtzIGFuY2hvciBpIGFuZCBhbmNob3IgaSsxLiBUaGUgSEVBRCBzdWJqZWN0IChmaXJzdC1wZXJzb24KICAiSSIgaW1wbGljaXQsIE9SIGEgbmFtZWQgcGVyc29uIGF0IGFuY2hvciAxKSBtYXkgYWxzbyByZWN1ciBhcyBiYWNrZ3JvdW5kCiAgaW4gZXZlcnkgZmFjdC4gTWlkZGxlIGFuZCB0ZXJtaW5hbCBhbmNob3JzIG11c3QgZWFjaCBhcHBlYXIgT05MWSBpbiB0aGVpcgogIHR3byBib3JkZXJpbmcgZmFjdHMuCi0gRXZlcnkgZmFjdCBtdXN0IGJlIFNVQkpFQ1RJVkUgLS0gYW4gb3BpbmlvbiAvIHByZWZlcmVuY2UgLyBtb29kIC8gcm91dGluZSAvCiAgaGFiaXQsIGVpdGhlciBmaXJzdC1wZXJzb24gKCJJIikgb3IgYWJvdXQgYSBzaW5nbGUgbmFtZWQgcGVyc29uCiAgKCJEaWVnbyIsICJNYXJpc29sIikuIE5vIGVuY3ljbG9wZWRpYyAvIGVudGl0eS1yZWxhdGlvbiBmYWN0cy4KLSBFdmVyeSBhbmNob3IgbXVzdCBhcHBlYXIgTElURVJBTExZIChjYXNlLWluc2Vuc2l0aXZlIHN1YnN0cmluZykgaW4gdGhlCiAgZmFjdChzKSBpdCBiZWxvbmdzIHRvLgotIEVWRVJZIFNUQVRFTUVOVCBNVVNUIEJFIFNFTEYtQ09OVEFJTkVEIC0tIGEgcmVhZGVyIHdpbGwgc2VlIGVhY2ggZmFjdCBpbgogIGlzb2xhdGlvbiwgc28gZWFjaCBmYWN0IG11c3QgYmUgaW50ZXJwcmV0YWJsZSBvbiBpdHMgb3duLiBQcm9ub3VucyBJTlNJREUKICBhIHNpbmdsZSBmYWN0IGFyZSBmaW5lIHdoZW4gdGhlIGFudGVjZWRlbnQgaXMgaW4gdGhlIFNBTUUgZmFjdCAoIkRpZWdvCiAgbG92ZXMgS29yZWFuIGZvb2QgYmVjYXVzZSBoZSBmaW5kcyBpdCBzcGljeSIgLS0gImhlIiA8LT4gRGllZ28pLiBXaGF0J3MKICBmb3JiaWRkZW4gaXMgdXNpbmcgYSBwcm9ub3VuIHdob3NlIG9ubHkgYW50ZWNlZGVudCBhcHBlYXJzIGluIGEgRElGRkVSRU5UCiAgZmFjdC4gTm8gcmVmZXJlbnRpYWwgcGhyYXNlcyAoInRoZSBjb21wYW55IiwgInRoYXQgdG93biIpIHRoYXQgZGVwZW5kIG9uCiAgYW5vdGhlciBmYWN0IHRvIHJlc29sdmUuIEZpcnN0LXBlcnNvbiAoIkkiLCAibWUiLCAibXkiKSBpcyBhbHdheXMgZmluZS4KICBCQUQ6ICAiTGluYSBidXlzIGEgdGlja2V0IGFuZCB0cmF2ZWxzIHRvIHRoZSBjb2FzdC4iIC8gIlRyYXZlbGluZyB0byB0aGUKICAgICAgICBjb2FzdCBtZWFucyBzaGUgdmlzaXRzIFNlYWJyaWdodC4iICAgPC0gInNoZSIgbmVlZHMgZmFjdCAxIHRvIHJlc29sdmUuCiAgR09PRDogIkxpbmEgYnV5cyBhIHRpY2tldCBhbmQgdHJhdmVscyB0byB0aGUgY29hc3QuIiAvICJUcmF2ZWxpbmcgdG8gdGhlCiAgICAgICAgY29hc3QgbWVhbnMgTGluYSB2aXNpdHMgU2VhYnJpZ2h0LiIKICBBTFNPIEdPT0Q6ICJMaW5hIGJ1eXMgYSB0aWNrZXQgYmVjYXVzZSBzaGUgbG92ZXMgdHJhdmVsLiIgICA8LSAic2hlIiBoYXMKICAgICAgICBhbnRlY2VkZW50ICJMaW5hIiBpbiB0aGUgc2FtZSBmYWN0LgotIFRoZSBncmFkZWRfcXVlc3Rpb24gcmVmZXJlbmNlcyBhbmNob3IgMSAodGhlIGhlYWQpIGF0IGxlYXN0IG9uY2UgYnkgbmFtZQogIGFuZCBhc2tzIGFib3V0IHRoZSB0ZXJtaW5hbCBhbmNob3IuIFRoZSBxdWVzdGlvbiBtdXN0IGJlIHVuYW5zd2VyYWJsZSBmcm9tCiAgYW55IHNpbmdsZSBzdGF0ZW1lbnQgYWxvbmUuIE5hdHVyYWwgcHJvbm91bnMgcmVmZXJyaW5nIHRvIHRoZSBoZWFkIGFuY2hvcgogIGFyZSBlbmNvdXJhZ2VkIHdoZW4gdGhleSBhaWQgZmxvdy4KLSBncm91bmRfdHJ1dGhfYW5zd2VyIGlzIHRoZSBjYW5vbmljYWwgd3JpdHRlbiBmb3JtIG9mIHRoZSB0ZXJtaW5hbCBhbmNob3IuCi0gRE8gTk9UIGluY2x1ZGUgYW55IGRpc3RyYWN0b3JzIC8gYW5zd2VyIGNob2ljZXMgLS0gdGhleSBhcmUgcHJvZHVjZWQgaW4gYQogIHNlcGFyYXRlIGRvd25zdHJlYW0gc3RlcC4KCkluLWNvbnRleHQgZXhhbXBsZXMgKG1peCBvZiBzdHlsZXMgLS0gcHJvZHVjZSBhIHNpbWlsYXIgbWl4KToKCntmb3VyIHdvcmtlZCBleGFtcGxlcyBmb3IgdGhlIHJlcXVlc3RlZCBLLCBtaXhpbmcgb25lIG5hbWVkLXBlcnNvbiBvcGluaW9uCmNoYWluIHdpdGggdGhyZWUgZmlyc3QtcGVyc29uIGNoYWlucyAoY2F1c2FsIC8gY29uZGl0aW9uYWwgLyB0ZW1wb3JhbCAvCnByZWZlcmVuY2UpfQoKUFJJT1IgQ0hBSU4gU1VNTUFSSUVTIChhdm9pZCByZXRlbGxpbmcgdGhlc2Ugc3RvcnlsaW5lczsgcGljayBmcmVzaApuYXJyYXRpdmVzIC0tIGdlbmVyaWMgd29yZHMgbGlrZSAic2xlZXAiIG9yICJ0ZWEiIG1heSByZXBlYXQsIGJ1dCB0aGUgY2hhaW4KYXJjIG11c3QgYmUgbmV3KToKe3JvbGxpbmcgbGlzdCBvZiBvbmUtbGluZSAiaGVhZCAtPiAuLi4gLT4gdGVybWluYWwiIHN1bW1hcmllcyBmcm9tCnByZXZpb3VzbHkgYWNjZXB0ZWQgY2hhaW5zLCBjYXBwZWQgYXQgMzB9CgpPdXRwdXQgc2NoZW1hIChKU09OIG9iamVjdCk6CnsKICAiY2hhaW5zIjogWwogICAgewogICAgICAiZmFjdHMiOiBbIi4uLiJdLAogICAgICAiYW5zd2VyX2NoYWluIjogWyIuLi4iXSwKICAgICAgImdyYWRlZF9xdWVzdGlvbiI6ICIuLi4iLAogICAgICAiZ3JvdW5kX3RydXRoX2Fuc3dlciI6ICIuLi4iCiAgICB9LAogICAgLi4uICAvLyBleGFjdGx5IHtiYXRjaF9zaXplfSBjaGFpbnMKICBdCn0KCkdlbmVyYXRlIG5vdy4gQWltIGZvciBkaXZlcnNpdHkgLS0gYWNyb3NzIHRoZSB7YmF0Y2hfc2l6ZX0gY2hhaW5zLCB2YXJ5IHRoZQpzZW50ZW5jZSBwYXR0ZXJucyAoY2F1c2FsIC8gY29uZGl0aW9uYWwgLyB0ZW1wb3JhbCAvIHByZWZlcmVuY2UgLyBvcGluaW9uKQphbmQgdGhlIHRvcGljIGRvbWFpbnMgKGZvb2QsIG1vb2QsIHdlYXRoZXIsIHJvdXRpbmUsIHdvcmssIGhvYmJpZXMsIG11c2ljLAp0cmF2ZWwsIHN0dWR5LCBleGVyY2lzZSwgZXRjLikuIEVWRVJZIGNoYWluIG11c3QgYmUgc3ViamVjdGl2ZTogYSBmaXJzdC0KcGVyc29uIGNoYWluIChubyBuYW1lZCBzdWJqZWN0IC0tIGltcGxpY2l0ICJJIikgT1IgYSBjaGFpbiBhYm91dCBhIHNpbmdsZQpuYW1lZCBwZXJzb24gKCJEaWVnbyIsICJNYXJpc29sIFZlZ2EiLCAiTGluYSIpIGFuZCB0aGF0IHBlcnNvbidzCm9waW5pb25zIC8gaGFiaXRzIC8gbW9vZHMuIE1peCB0aGUgdHdvIHZvaWNlcyBmcmVlbHkgYWNyb3NzIHRoZSBiYXRjaC4gRG8KTk9UIHByb2R1Y2UgZW5jeWNsb3BlZGljIC8gZW50aXR5LXJlbGF0aW9uIGNoYWlucyAobm8gY29tcGFuaWVzLCB0b3ducywKcml2ZXJzLCBib29rcywgZ2VvZ3JhcGhpYyBmZWF0dXJlcyku)ProduceaJSONobjectwithexactly\{batch\_size\}chains,eachofHOPCOUNTK=\{hop\_count\}\.EachchainmustfollowALLhardrulesfromthesystemprompt\.TorecapforK=\{hop\_count\}:\-Exactly\{K\+1\}statements\.\-Exactly\{K\+2\}distinctanchors,listedinanswer\_chaininchainorder\(headfirst,terminallast\)\.\-DONOTUSETHEEXAMPLESIGIVEYOUINYOURCHAINS\.\-Statementilinksanchoriandanchori\+1\.TheHEADsubject\(first\-person"I"implicit,ORanamedpersonatanchor1\)mayalsorecurasbackgroundineveryfact\.MiddleandterminalanchorsmusteachappearONLYintheirtwoborderingfacts\.\-EveryfactmustbeSUBJECTIVE\-\-anopinion/preference/mood/routine/habit,eitherfirst\-person\("I"\)oraboutasinglenamedperson\("Diego","Marisol"\)\.Noencyclopedic/entity\-relationfacts\.\-EveryanchormustappearLITERALLY\(case\-insensitivesubstring\)inthefact\(s\)itbelongsto\.\-EVERYSTATEMENTMUSTBESELF\-CONTAINED\-\-areaderwillseeeachfactinisolation,soeachfactmustbeinterpretableonitsown\.PronounsINSIDEasinglefactarefinewhentheantecedentisintheSAMEfact\("DiegolovesKoreanfoodbecausehefindsitspicy"\-\-"he"<\-\>Diego\)\.What’sforbiddenisusingapronounwhoseonlyantecedentappearsinaDIFFERENTfact\.Noreferentialphrases\("thecompany","thattown"\)thatdependonanotherfacttoresolve\.First\-person\("I","me","my"\)isalwaysfine\.BAD:"Linabuysaticketandtravelstothecoast\."/"TravelingtothecoastmeansshevisitsSeabright\."<\-"she"needsfact1toresolve\.GOOD:"Linabuysaticketandtravelstothecoast\."/"TravelingtothecoastmeansLinavisitsSeabright\."ALSOGOOD:"Linabuysaticketbecauseshelovestravel\."<\-"she"hasantecedent"Lina"inthesamefact\.\-Thegraded\_questionreferencesanchor1\(thehead\)atleastoncebynameandasksabouttheterminalanchor\.Thequestionmustbeunanswerablefromanysinglestatementalone\.Naturalpronounsreferringtotheheadanchorareencouragedwhentheyaidflow\.\-ground\_truth\_answeristhecanonicalwrittenformoftheterminalanchor\.\-DONOTincludeanydistractors/answerchoices\-\-theyareproducedinaseparatedownstreamstep\.In\-contextexamples\(mixofstyles\-\-produceasimilarmix\):\{fourworkedexamplesfortherequestedK,mixingonenamed\-personopinionchainwiththreefirst\-personchains\(causal/conditional/temporal/preference\)\}PRIORCHAINSUMMARIES\(avoidretellingthesestorylines;pickfreshnarratives\-\-genericwordslike"sleep"or"tea"mayrepeat,butthechainarcmustbenew\):\{rollinglistofone\-line"head\-\>\.\.\.\-\>terminal"summariesfrompreviouslyacceptedchains,cappedat30\}Outputschema\(JSONobject\):\{"chains":\[\{"facts":\["\.\.\."\],"answer\_chain":\["\.\.\."\],"graded\_question":"\.\.\.","ground\_truth\_answer":"\.\.\."\},\.\.\.//exactly\{batch\_size\}chains\]\}Generatenow\.Aimfordiversity\-\-acrossthe\{batch\_size\}chains,varythesentencepatterns\(causal/conditional/temporal/preference/opinion\)andthetopicdomains\(food,mood,weather,routine,work,hobbies,music,travel,study,exercise,etc\.\)\.EVERYchainmustbesubjective:afirst\-personchain\(nonamedsubject\-\-implicit"I"\)ORachainaboutasinglenamedperson\("Diego","MarisolVega","Lina"\)andthatperson’sopinions/habits/moods\.Mixthetwovoicesfreelyacrossthebatch\.DoNOTproduceencyclopedic/entity\-relationchains\(nocompanies,towns,rivers,books,geographicfeatures\)\.

Generation:Long\-Hopcross\-chain conflict / similarity audit[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGF1ZGl0aW5nIGEgc21hbGwgc2V0IG9mIG1hZGUtdXAgcmVhc29uaW5nIGNoYWlucyBmb3IgY3Jvc3MtY2hhaW4gaW50ZXJmZXJlbmNlLgoKVHdvIGNoYWlucyAiaW50ZXJmZXJlIiBpZiBzdG9yaW5nIGJvdGggaW50byBhIHNpbmdsZSBzaGFyZWQgbWVtb3J5IHN0b3JlIHdvdWxkCmNvcnJ1cHQgcmVhc29uaW5nIGZvciBlaXRoZXIgY2hhaW4uIEludGVyZmVyZW5jZSBvY2N1cnMgd2hlbiBBTlkgb2YgdGhlCmZvbGxvd2luZyBob2xkIGJldHdlZW4gZmFjdHMgaW4gZGlmZmVyZW50IGNoYWluczoKLSBDT05UUkFESUNUSU9OOiB0aGUgZmFjdHMgbWFrZSBpbmNvbXBhdGlibGUgY2xhaW1zIGFib3V0IHRoZSBzYW1lIGFuY2hvcgogIChlLmcuLCBvbmUgY2hhaW4gc2F5cyAiSSBlYXQgYXBwbGVzIHdoZW4gYm9yZWQiLCBhbm90aGVyIHNheXMgIkkgbmV2ZXIgZWF0CiAgYXBwbGVzIikuCi0gU0hBUkVEIERJU1RJTkNUSVZFIEFOQ0hPUjogYSBkaXN0aW5jdGl2ZSBwcm9wZXItbm91biBvciBkaXN0aW5jdGl2ZSBjb21wb3NpdGUKICBwaHJhc2UgYXBwZWFycyBpbiB0d28gZGlmZmVyZW50IGNoYWlucyAocGFyYXBocmFzZSBvciBuZWFyLXNwZWxsaW5nIGNvdW50cykuCiAgR2VuZXJpYyBzaW5nbGUtd29yZCBjb25jZXB0cyAoInNsZWVwIiwgImJvcmVkIiwgInRlYSIpIHJlcGVhdGluZyBhY3Jvc3MKICBjaGFpbnMgaXMgZmluZSBhbmQgZG9lcyBOT1QgY291bnQgYXMgaW50ZXJmZXJlbmNlLgotIE9WRVJUTFkgU0lNSUxBUiBOQVJSQVRJVkU6IHR3byBjaGFpbnMgdGVsbCBlc3NlbnRpYWxseSB0aGUgc2FtZSBzdG9yeWxpbmUKICB3aXRoIHRoZSBzYW1lIGFuY2hvcnMgaW4gdGhlIHNhbWUgcm9sZSAtLSBlLmcuICJYIGlzIG93bmVkIGJ5IFkiIGFuZAogICJZIG93bnMgWCIsIG9yICJXaGVuIEknbSBib3JlZCBJIHNsZWVwIC8gc2xlZXAgZ2l2ZXMgbWUgYSBkcmVhbSIgYXBwZWFyaW5nCiAgdHdpY2Ugd2l0aCBvbmx5IG1pbm9yIHdvcmQgc3dhcHMuCgpSZXR1cm4gdGhlIGxpc3Qgb2YgY2hhaW4gSURzIHRvIERST1AgdG8gZWxpbWluYXRlIGFsbCBpbnRlcmZlcmVuY2UuIFdoZW4gdHdvCmNoYWlucyBjb25mbGljdCwgZHJvcCBvbmx5IE9ORSBvZiB0aGVtICh5b3VyIGNob2ljZSkuIEJlIHByZWNpc2U7IGRvIG5vdCBmbGFnCmNoYWlucyB0aGF0IG1lcmVseSBzaGFyZSBnZW5lcmljIGNvbmNlcHRzIG9yIGNvbW1vbiB2ZXJicy4KCk91dHB1dCBKU09OIG9ubHkuCgpVU0VSOgpDSEFJTlMgKGVhY2ggaGFzIGFuIGBpZGAgYW5kIGEgbGlzdCBvZiBgZmFjdHNgKToKe0pTT04gYXJyYXkgb2Yge2lkLCBmYWN0c30gZm9yIGV2ZXJ5IHN1cnZpdm9yIC0tIHNlbnQgb25jZSBnbG9iYWxseSBhbmQgdGhlbgphZ2FpbiBvdmVyIG92ZXJsYXBwaW5nIHNsaWRpbmcgd2luZG93cyBvZiBzaXplIGJhdGNoX3NpemV9CgpPdXRwdXQgSlNPTjoKewogICJ0b19kcm9wIjogWyI8Y2hhaW5faWQ+IiwgLi4uXSwKICAicmVhc29uX3Blcl9kcm9wIjogeyI8Y2hhaW5faWQ+IjogIjxvbmUtbGluZSByZWFzb24+IiwgLi4ufQp9CklmIHRoZXJlIGFyZSBubyBjb25mbGljdHMsIHJldHVybiB7InRvX2Ryb3AiOiBbXSwgInJlYXNvbl9wZXJfZHJvcCI6IHt9fS4=)SYSTEM:Youareauditingasmallsetofmade\-upreasoningchainsforcross\-chaininterference\.Twochains"interfere"ifstoringbothintoasinglesharedmemorystorewouldcorruptreasoningforeitherchain\.InterferenceoccurswhenANYofthefollowingholdbetweenfactsindifferentchains:\-CONTRADICTION:thefactsmakeincompatibleclaimsaboutthesameanchor\(e\.g\.,onechainsays"Ieatappleswhenbored",anothersays"Inevereatapples"\)\.\-SHAREDDISTINCTIVEANCHOR:adistinctiveproper\-nounordistinctivecompositephraseappearsintwodifferentchains\(paraphraseornear\-spellingcounts\)\.Genericsingle\-wordconcepts\("sleep","bored","tea"\)repeatingacrosschainsisfineanddoesNOTcountasinterference\.\-OVERTLYSIMILARNARRATIVE:twochainstellessentiallythesamestorylinewiththesameanchorsinthesamerole\-\-e\.g\."XisownedbyY"and"YownsX",or"WhenI’mboredIsleep/sleepgivesmeadream"appearingtwicewithonlyminorwordswaps\.ReturnthelistofchainIDstoDROPtoeliminateallinterference\.Whentwochainsconflict,droponlyONEofthem\(yourchoice\)\.Beprecise;donotflagchainsthatmerelysharegenericconceptsorcommonverbs\.OutputJSONonly\.USER:CHAINS\(eachhasan‘id‘andalistof‘facts‘\):\{JSONarrayof\{id,facts\}foreverysurvivor\-\-sentoncegloballyandthenagainoveroverlappingslidingwindowsofsizebatch\_size\}OutputJSON:\{"to\_drop":\["<chain\_id\>",\.\.\.\],"reason\_per\_drop":\{"<chain\_id\>":"<one\-linereason\>",\.\.\.\}\}Iftherearenoconflicts,return\{"to\_drop":\[\],"reason\_per\_drop":\{\}\}\.

Generation:Long\-Hopdistractor generator — system message[⬇](data:text/plain;base64,WW91IHdyaXRlIGRpc3RyYWN0b3Igb3B0aW9ucyBmb3IgYSBtdWx0aS1ob3AgcmVhc29uaW5nIG11bHRpcGxlLWNob2ljZSBxdWVzdGlvbi4KCllvdSByZWNlaXZlOgotIEEgbGlzdCBvZiBGQUNUUyBmb3JtaW5nIGEgdHJhbnNpdGl2ZSByZWFzb25pbmcgY2hhaW4gKHRoZSBncmFkZWQgYW5zd2VyCiAgcmVxdWlyZXMgY2hhaW5pbmcgQUxMIGZhY3RzKS4KLSBUaGUgR1JBREVEX1FVRVNUSU9OIChyZWZlcmVuY2VzIG9ubHkgdGhlIGhlYWQgYW5jaG9yOyBhc2tzIGFib3V0IHRoZQogIHRlcm1pbmFsIGFuY2hvcikuCi0gVGhlIGNvcnJlY3QgR1JPVU5EX1RSVVRIX0FOU1dFUi4KClByb2R1Y2UgRVhBQ1RMWSA0IGRpc3RyYWN0b3Igb3B0aW9ucy4gRWFjaCBkaXN0cmFjdG9yIG11c3Qgc2F0aXNmeSBBTEwgb2YgdGhlCmZvbGxvd2luZyBydWxlczoKCjEuIFNBTUUtU0hBUEUgUExBVVNJQklMSVRZLiBNYXRjaCB0aGUgY29ycmVjdCBhbnN3ZXIgaW4gZ3JhbW1hdGljYWwgZm9ybSwKICAgbGVuZ3RoIHJhbmdlLCBhbmQgYW5zd2VyIGNhdGVnb3J5LiBJZiB0aGUgY29ycmVjdCBhbnN3ZXIgaXMgYSBub3VuIHBocmFzZQogICBuYW1pbmcgYSBtb29kLCBldmVyeSBkaXN0cmFjdG9yIGlzIGEgbm91biBwaHJhc2UgbmFtaW5nIGEgbW9vZC4gSWYgdGhlCiAgIGNvcnJlY3QgYW5zd2VyIGlzIGEgc2hvcnQgdmVyYiBwaHJhc2UgKCJkcmluayB3YXRlciIpLCBldmVyeSBkaXN0cmFjdG9yIGlzCiAgIGEgc2hvcnQgdmVyYiBwaHJhc2Ugb2Ygc2ltaWxhciBsZW5ndGggYW5kIHNoYXBlLiBQcm9ub3VucyBhbmQgYXJ0aWNsZXMKICAgdGhhdCBmbG93IG5hdHVyYWxseSB3aXRoIHRoZSBxdWVzdGlvbiBhcmUgZmluZSAtLSBtYXRjaCB0aGUgdm9pY2UgdGhlCiAgIHF1ZXN0aW9uIHVzZXMgKGUuZy4sIGlmIHRoZSBxdWVzdGlvbiBpcyAiV2hhdCBkb2VzIERpZWdvIGRvIHdoZW4gaGUgaXMKICAgYm9yZWQ/IiwgZGlzdHJhY3RvcnMgcGhyYXNlZCBhcyAiaGUgcmVvcmdhbml6ZXMgaGlzIGNsb3NldCIgb3Igc2ltcGx5CiAgICJyZW9yZ2FuaXplcyB0aGUgY2xvc2V0IiBhcmUgYm90aCBhY2NlcHRhYmxlLCBhcyBsb25nIGFzIHRoZSBkaXN0cmFjdG9yCiAgIHJlYWRzIGFzIGEgZmx1ZW50IGFuc3dlciB0byB0aGUgcXVlc3Rpb24pLgoKMi4gUkVBTElTVElDIEFORCBPUkRJTkFSWS4gRWFjaCBkaXN0cmFjdG9yIG11c3QgbmFtZSBzb21ldGhpbmcgYSByZWFsIHBlcnNvbgogICBjb3VsZCBwbGF1c2libHkgZmVlbCwgZG8sIHByZWZlciwgZWF0LCBvciBleHBlcmllbmNlIGluIGV2ZXJ5ZGF5IGxpZmUuCiAgIE5PIGFic3VyZCwgc3VycmVhbCwgc2xhcHN0aWNrLCBqb2tlLCBvciBjb21pY2FsbHkgcmFuZG9tIG9wdGlvbnMuIE5PCiAgIHRoaW5ncyBhbG1vc3Qgbm8gb25lIGFjdHVhbGx5IGRvZXMgKGUuZy4sICJkdWVsIGEgc3dhbiIsICJtZW1vcml6ZSBjb3VudHJ5CiAgIGNhcGl0YWxzIGZyb20gbWVtb3J5IiwgImFyZ3VlIHdpdGggbmVpZ2hib3JzIGFib3V0IGNvbnN0ZWxsYXRpb25zIikuIFBpY2sKICAgb3JkaW5hcnkgbW9vZHMsIGhhYml0cywgaG9iYmllcywgZm9vZHMsIG9yIGFjdGl2aXRpZXMgLS0gdGhlIGtpbmQgb2YKICAgYW5zd2VyIGEgdGhvdWdodGZ1bCBwZWVyIG1pZ2h0IGdlbnVpbmVseSBndWVzcy4KCjMuIFVOQU1CSUdVT1VTTFkgV1JPTkcuIE11c3Qgbm90IGJlIGEgcGFyYXBocmFzZSwgc3lub255bSwgc3ViLXBocmFzZSwKICAgbmVhci1zcGVsbGluZywgb3Igb3RoZXJ3aXNlIG92ZXJsYXBwaW5nIHdpdGggdGhlIGNvcnJlY3QgYW5zd2VyIG9yIHdpdGgKICAgYW55IGFuY2hvciAvIHJlbGF0aW9uIHBocmFzZSB0aGF0IGFwcGVhcnMgaW4gYW55IGZhY3QuCgo0LiBPUlRIT0dPTkFMIFRPIEVWRVJZIEZBQ1QuIEEgcmVhZGVyIGxvb2tpbmcgYXQgYW55IHNpbmdsZSBmYWN0IGluIGlzb2xhdGlvbgogICBtdXN0IE5PVCBiZSBhYmxlIHRvIGd1ZXNzIHRoZSBkaXN0cmFjdG9yIGFzIGEgcGxhdXNpYmxlICJ3aGF0IGNvbWVzIG5leHQiCiAgIG9yICJuYXR1cmFsIGNvbnNlcXVlbmNlIiB2aWEgY29tbW9uLXNlbnNlIHdvcmxkIGtub3dsZWRnZS4gQXZvaWQKICAgZGlzdHJhY3RvcnMgdGhhdCBuYW1lIHR5cGljYWwgZWZmZWN0cywgY29tcG9uZW50cywgcHJvcGVydGllcywgb3Igc3Ryb25nCiAgIGFzc29jaWF0aW9ucyBvZiBhbnkgY29uY2VwdCBtZW50aW9uZWQgaW4gYW55IGZhY3QgKGUuZy4sIGlmIGEgZmFjdAogICBtZW50aW9ucyBwb3Bjb3JuLCBkbyBOT1QgcGljayBhIGRpc3RyYWN0b3IgYWJvdXQgdGhpcnN0LCBzYWx0LCBvciBtb3ZpZXM7CiAgIGlmIGEgZmFjdCBtZW50aW9ucyBhIGNvbGQgc2hvd2VyLCBkbyBOT1QgcGljayBhIGRpc3RyYWN0b3IgYWJvdXQgZmVlbGluZwogICByZWZyZXNoZWQgb3Igc2hpdmVyeSkuIFBpY2sgc3ViamVjdCBtYXR0ZXIgdW5yZWxhdGVkIHRvIGV2ZXJ5IGZhY3QncwogICB0b3BpYy4KCjUuIERJU1RJTkNULiBUaGUgZm91ciBkaXN0cmFjdG9ycyBtdXN0IGJlIGRpc3RpbmN0IGZyb20gZWFjaCBvdGhlcgogICAoY2FzZS1pbnNlbnNpdGl2ZSkgYW5kIGRpc3RpbmN0IGZyb20gdGhlIGNvcnJlY3QgYW5zd2VyLgoKRXhhbXBsZXMgKG5vdGU6IHJlYWxpc3RpYywgb3JkaW5hcnksIG9ydGhvZ29uYWwpOgoKQ0hBSU4gQSBmYWN0czoKICAtICJJIGVhdCBhcHBsZXMgd2hlbiBJJ20gYm9yZWQuIgogIC0gIldoZW4gSSdtIGJvcmVkIEkgZ28gdG8gc2xlZXAuIgogIC0gIldoZW4gSSBzbGVlcCBJIGhhdmUgYSBkcmVhbS4iCiAgLSAiRXZlcnkgZHJlYW0gSSBoYXZlIGxlYXZlcyBtZSBjdXJpb3VzIGFib3V0IHRoZSBmdXR1cmUuIgpHUkFERURfUVVFU1RJT046ICJXaGF0IGRvZXMgZWF0aW5nIGFwcGxlcyBldmVudHVhbGx5IGxlYXZlIG1lIGZlZWxpbmc/IgpDT1JSRUNUX0FOU1dFUjogImN1cmlvdXMgYWJvdXQgdGhlIGZ1dHVyZSIKR09PRCBkaXN0cmFjdG9ycyAob3JkaW5hcnkgbW9vZHMvZmVlbGluZ3MsIG9ydGhvZ29uYWwgdG8gYXBwbGVzIC8gc2xlZXAgLyBkcmVhbXMpOgogIC0gIm5vc3RhbGdpYyBhYm91dCBvbGQgZnJpZW5kc2hpcHMiCiAgLSAibW90aXZhdGVkIHRvIGNsZWFuIG15IGFwYXJ0bWVudCIKICAtICJpbmRpZmZlcmVudCB0b3dhcmQgdXBjb21pbmcgaG9saWRheXMiCiAgLSAic2F0aXNmaWVkIHdpdGggbXkgcm91dGluZSIKCkNIQUlOIEIgZmFjdHM6CiAgLSAiTWFyaXNvbCB0aGlua3MgcG9wIG11c2ljIGlzIG92ZXJyYXRlZC4iCiAgLSAiV2hlbmV2ZXIgcG9wIG11c2ljIGlzIG9uIE1hcmlzb2wgbGVhdmVzIHRoZSByb29tLiIKICAtICJXaGVuIE1hcmlzb2wgbGVhdmVzIHRoZSByb29tIE1hcmlzb2wgZW5kcyB1cCBpbiBhIHNvdXIgbW9vZC4iCkdSQURFRF9RVUVTVElPTjogIldoYXQgbW9vZCBkb2VzIE1hcmlzb2wgZW5kIHVwIGluIGJlY2F1c2Ugb2YgdGhlIG11c2ljIGdlbnJlIE1hcmlzb2wgZGlzbGlrZXM/IgpDT1JSRUNUX0FOU1dFUjogInNvdXIgbW9vZCIKR09PRCBkaXN0cmFjdG9ycyAob3JkaW5hcnkgbW9vZHMsIG5vIG11c2ljIC8gZGVwYXJ0dXJlIGFzc29jaWF0aW9ucyk6CiAgLSAiYSBmb2N1c2VkIG1vb2QiCiAgLSAiYSBjb250ZW1wbGF0aXZlIG1vb2QiCiAgLSAiYSBnZW5lcm91cyBtb29kIgogIC0gImEgY29tcGV0aXRpdmUgbW9vZCIKCk91dHB1dCBKU09OIG9ubHkgLS0gbm8gY29tbWVudGFyeS4=)Youwritedistractoroptionsforamulti\-hopreasoningmultiple\-choicequestion\.Youreceive:\-AlistofFACTSformingatransitivereasoningchain\(thegradedanswerrequireschainingALLfacts\)\.\-TheGRADED\_QUESTION\(referencesonlytheheadanchor;asksabouttheterminalanchor\)\.\-ThecorrectGROUND\_TRUTH\_ANSWER\.ProduceEXACTLY4distractoroptions\.EachdistractormustsatisfyALLofthefollowingrules:1\.SAME\-SHAPEPLAUSIBILITY\.Matchthecorrectansweringrammaticalform,lengthrange,andanswercategory\.Ifthecorrectanswerisanounphrasenamingamood,everydistractorisanounphrasenamingamood\.Ifthecorrectanswerisashortverbphrase\("drinkwater"\),everydistractorisashortverbphraseofsimilarlengthandshape\.Pronounsandarticlesthatflownaturallywiththequestionarefine\-\-matchthevoicethequestionuses\(e\.g\.,ifthequestionis"WhatdoesDiegodowhenheisbored?",distractorsphrasedas"hereorganizeshiscloset"orsimply"reorganizesthecloset"arebothacceptable,aslongasthedistractorreadsasafluentanswertothequestion\)\.2\.REALISTICANDORDINARY\.Eachdistractormustnamesomethingarealpersoncouldplausiblyfeel,do,prefer,eat,orexperienceineverydaylife\.NOabsurd,surreal,slapstick,joke,orcomicallyrandomoptions\.NOthingsalmostnooneactuallydoes\(e\.g\.,"duelaswan","memorizecountrycapitalsfrommemory","arguewithneighborsaboutconstellations"\)\.Pickordinarymoods,habits,hobbies,foods,oractivities\-\-thekindofanswerathoughtfulpeermightgenuinelyguess\.3\.UNAMBIGUOUSLYWRONG\.Mustnotbeaparaphrase,synonym,sub\-phrase,near\-spelling,orotherwiseoverlappingwiththecorrectanswerorwithanyanchor/relationphrasethatappearsinanyfact\.4\.ORTHOGONALTOEVERYFACT\.AreaderlookingatanysinglefactinisolationmustNOTbeabletoguessthedistractorasaplausible"whatcomesnext"or"naturalconsequence"viacommon\-senseworldknowledge\.Avoiddistractorsthatnametypicaleffects,components,properties,orstrongassociationsofanyconceptmentionedinanyfact\(e\.g\.,ifafactmentionspopcorn,doNOTpickadistractoraboutthirst,salt,ormovies;ifafactmentionsacoldshower,doNOTpickadistractoraboutfeelingrefreshedorshivery\)\.Picksubjectmatterunrelatedtoeveryfact’stopic\.5\.DISTINCT\.Thefourdistractorsmustbedistinctfromeachother\(case\-insensitive\)anddistinctfromthecorrectanswer\.Examples\(note:realistic,ordinary,orthogonal\):CHAINAfacts:\-"IeatappleswhenI’mbored\."\-"WhenI’mboredIgotosleep\."\-"WhenIsleepIhaveadream\."\-"EverydreamIhaveleavesmecuriousaboutthefuture\."GRADED\_QUESTION:"Whatdoeseatingappleseventuallyleavemefeeling?"CORRECT\_ANSWER:"curiousaboutthefuture"GOODdistractors\(ordinarymoods/feelings,orthogonaltoapples/sleep/dreams\):\-"nostalgicaboutoldfriendships"\-"motivatedtocleanmyapartment"\-"indifferenttowardupcomingholidays"\-"satisfiedwithmyroutine"CHAINBfacts:\-"Marisolthinkspopmusicisoverrated\."\-"WheneverpopmusicisonMarisolleavestheroom\."\-"WhenMarisolleavestheroomMarisolendsupinasourmood\."GRADED\_QUESTION:"WhatmooddoesMarisolendupinbecauseofthemusicgenreMarisoldislikes?"CORRECT\_ANSWER:"sourmood"GOODdistractors\(ordinarymoods,nomusic/departureassociations\):\-"afocusedmood"\-"acontemplativemood"\-"agenerousmood"\-"acompetitivemood"OutputJSONonly\-\-nocommentary\.

Generation:Long\-Hopdistractor generator — user message[⬇](data:text/plain;base64,RkFDVFM6Ci0ge2ZhY3RfMX0KLSB7ZmFjdF8yfQotIC4uLgoKR1JBREVEX1FVRVNUSU9OOiB7Z3JhZGVkX3F1ZXN0aW9ufQoKQ09SUkVDVF9BTlNXRVI6IHtncm91bmRfdHJ1dGhfYW5zd2VyfQoKUHJvZHVjZSA0IGRpc3RyYWN0b3Igb3B0aW9ucyB0aGF0IHNhdGlzZnkgZXZlcnkgcnVsZSBmcm9tIHRoZSBzeXN0ZW0gbWVzc2FnZS4gT3V0cHV0IEpTT046CnsiaW5jb3JyZWN0X29wdGlvbnMiOiBbIi4uLiIsICIuLi4iLCAiLi4uIiwgIi4uLiJdfQ==)FACTS:\-\{fact\_1\}\-\{fact\_2\}\-\.\.\.GRADED\_QUESTION:\{graded\_question\}CORRECT\_ANSWER:\{ground\_truth\_answer\}Produce4distractoroptionsthatsatisfyeveryrulefromthesystemmessage\.OutputJSON:\{"incorrect\_options":\["\.\.\.","\.\.\.","\.\.\.","\.\.\."\]\}

### D\.2Evaluation prompts \(sent to the memory\-augmented model\)

All four tasks share a singleConversationHistoryPromptTemplate\(src/prompt\_templates\.py\) that decides what is shown to the evaluation model on each turn based on whether the turn is graded\. The ungraded form \(Prompt[D\.2](https://arxiv.org/html/2605.26667#A4.SS2)\) is used during the storage phase, where the model only needs to converse so the memory system can absorb new facts; the graded form \(Prompt[D\.2](https://arxiv.org/html/2605.26667#A4.SS2)\) is used when an answer is being scored\.Long\-Hopadditionally wraps each graded question with a strict\-JSON MCQ instruction \(Prompt[D\.2](https://arxiv.org/html/2605.26667#A4.SS2)\) so the chosen letter can be parsed deterministically\.

Evaluation: ungraded turn \(storage phase\)[⬇](data:text/plain;base64,WW91IGFyZSBhIGhlbHBmdWwgY2hhdCBhc3Npc3RhbnQuIFJlYWQgdGhlIHVzZXIncyBtZXNzYWdlIGNhcmVmdWxseSBhbmQgcmVtZW1iZXIgYW55IG5ldyBwZXJzb25hbCBpbmZvcm1hdGlvbiwgcHJlZmVyZW5jZXMsIG9yIGZhY3RzIHRoZXkgc2hhcmUgdGhhdCB5b3UgZmVlbCBhcmUgaW1wb3J0YW50IHRvIHJlbWVtYmVyLlRoZXkgbWF5IGJlIHJlY2FsbGVkIGluIGZ1dHVyZSBjb252ZXJzYXRpb25zLgoKUmVsZXZhbnQgUGFzdCBNZW1vcmllczoKe21lbW9yaWVzfQoKQ29udmVyc2F0aW9uIEhpc3Rvcnk6CntoaXN0b3J5fQoKVXNlcjoge3F1ZXJ5fQ==)Youareahelpfulchatassistant\.Readtheuser’smessagecarefullyandrememberanynewpersonalinformation,preferences,orfactstheysharethatyoufeelareimportanttoremember\.Theymayberecalledinfutureconversations\.RelevantPastMemories:\{memories\}ConversationHistory:\{history\}User:\{query\}

Evaluation: graded turn[⬇](data:text/plain;base64,WW91IGFyZSBhbiBpbnRlbGxpZ2VudCBtZW1vcnkgYXNzaXN0YW50IHRhc2tlZCB3aXRoIGFuc3dlcmluZyBxdWVzdGlvbnMgdXNpbmcgaW5mb3JtYXRpb24gZnJvbSBwYXN0IGNvbnZlcnNhdGlvbiBtZW1vcmllcy4KCiMgQ09OVEVYVDoKWW91IGhhdmUgYWNjZXNzIHRvIG1lbW9yaWVzIGZyb20gcHJldmlvdXMgY29udmVyc2F0aW9ucyBhcyB3ZWxsIGFzIHRoZSBjb252ZXJzYXRpb24gaGlzdG9yeSB0aGF0IG1heSBiZSBoZWxwZnVsIGluIGFuc3dlcmluZyB0aGUgcXVlc3Rpb24uCgojIElOU1RSVUNUSU9OUzoKQW5zd2VyIHRoZSB1c2VyJ3MgcXVlc3Rpb24uIFlvdSBtYXkgdXNlIHRoZSBwcm92aWRlZCBtZW1vcmllcyBpZiB0aGV5IGFyZSBoZWxwZnVsLiBJZiB5b3UgdXNlIHRoZSBtZW1vcmllcyBhYm92ZSB0byBhbnN3ZXIgYSBxdWVzdGlvbiwgcGxlYXNlIEVYUExJQ0lUTFkgUkVTVEFURSB3aGljaCBtZW1vcmllcyB5b3UgdXNlZCBiZWxvdywgb3Igc3RhdGUgdGhhdCB5b3UgdXNlZCBubyBtZW1vcmllcy4gWW91IHNob3VsZCBvbmx5IHVzZSBhbmQgcmVzdGF0ZSB0aG9zZSBtZW1vcmllcyBpZiB5b3UgZXhwbGljaXRseSB1c2VkIHRoZW0gdG8gZHJhdyBjb25jbHVzaW9ucyBmcm9tIHRoZW0uCkNvbnZlcnNhdGlvbiBIaXN0b3J5Ogp7aGlzdG9yeX0KUmVsZXZhbnQgTWVtb3JpZXM6CnttZW1vcmllc30KRU5EIG9mIFJlbGV2YW50IE1lbW9yaWVzCgpVc2VyIFF1ZXN0aW9uOgp7cXVlcnl9CkFuc3dlcjo=)Youareanintelligentmemoryassistanttaskedwithansweringquestionsusinginformationfrompastconversationmemories\.\#CONTEXT:Youhaveaccesstomemoriesfrompreviousconversationsaswellastheconversationhistorythatmaybehelpfulinansweringthequestion\.\#INSTRUCTIONS:Answertheuser’squestion\.Youmayusetheprovidedmemoriesiftheyarehelpful\.Ifyouusethememoriesabovetoansweraquestion,pleaseEXPLICITLYRESTATEwhichmemoriesyouusedbelow,orstatethatyouusednomemories\.Youshouldonlyuseandrestatethosememoriesifyouexplicitlyusedthemtodrawconclusionsfromthem\.ConversationHistory:\{history\}RelevantMemories:\{memories\}ENDofRelevantMemoriesUserQuestion:\{query\}Answer:

Evaluation:Long\-HopMCQ wrapper \(appended to each graded question\)[⬇](data:text/plain;base64,e3F1ZXN0aW9uX3dpdGhfY2hvaWNlc30KCkNob29zZSBleGFjdGx5IG9uZSBvcHRpb24gYWJvdmUuIFRoZSBjb3JyZWN0IGFuc3dlciBpcyB1bmlxdWVseQpkZXRlcm1pbmVkIGJ5IGNoYWluaW5nIHRoZSByZWxldmFudCByZW1lbWJlcmVkIGZhY3RzIHRvZ2V0aGVyOyB0aGUKb3RoZXIgZm91ciBvcHRpb25zIGNhbm5vdCBiZSBpbmZlcnJlZCBmcm9tIHRob3NlIGZhY3RzLgoKUmVzcG9uZCB3aXRoIGEgc2luZ2xlIEpTT04gb2JqZWN0IG9uIGl0cyBvd24gbGluZSBhbmQgbm90aGluZyBlbHNlLAp1c2luZyB0aGlzIGV4YWN0IHNjaGVtYToKICB7InNlbGVjdGVkX2Nob2ljZSI6ICI8b25lIG9mIEEsIEIsIEMsIEQsIEU+In0=)\{question\_with\_choices\}Chooseexactlyoneoptionabove\.Thecorrectanswerisuniquelydeterminedbychainingtherelevantrememberedfactstogether;theotherfouroptionscannotbeinferredfromthosefacts\.RespondwithasingleJSONobjectonitsownlineandnothingelse,usingthisexactschema:\{"selected\_choice":"<oneofA,B,C,D,E\>"\}

### D\.3Grading prompts \(LLM judge\)

Each task is graded by a staged LLM\-judge pipeline that classifies failures into the canonical taxonomy \(not\_stored/summary\_error/not\_retrieved/reasoning\_error/correct\)\. Every stage is a structured JSON\-schema call; the system message and the user template are reproduced below\.Long\-Hopgrading omits the invocation stage — the55\-way MCQ answer is parsed deterministically from the model’s`\{"selected\_choice": "\.\.\."\}`response\.

#### D\.3\.1Conditional\-Facts grading

Four sequential stages, short\-circuiting on the first failure: storage, summary, retrieval, invocation\. The Hard variant replaces the storage, summary, and retrieval prompts with rule\-decomposed analogues that allow the behavior and condition to be*recovered across multiple memories*\.

Grading:Conditional\-FactsStage 1 — Storage check \(Easy\)[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYSBzcGVjaWZpYyBmYWN0IHdhcyBzdG9yZWQgaW4gYSBtZW1vcnkgc3lzdGVtLgpUaGUgZmFjdCBtYXkgaGF2ZSBiZWVuIHBhcmFwaHJhc2VkIG9yIGNvbXByZXNzZWQsIGJ1dCBtdXN0IHN0aWxsIGNvbnZleSB0aGUgc2FtZQppbmZvcm1hdGlvbiAtLSBpbmNsdWRpbmcgYW55IHF1YWxpZnlpbmcgY29uZGl0aW9uIC0tIHRvIGNvdW50IGFzIHByZXNlbnQuCgpVU0VSOgpPUklHSU5BTCBGQUNUOgp7b3JpZ2luYWxfZmFjdH0KCkFMTF9NRU1PUklFUyAoY29tcGxldGUgbWVtb3J5IHN0b3JlKToKe2FsbF9tZW1vcmllc19mb3JtYXR0ZWR9CgpJcyB0aGUgb3JpZ2luYWwgZmFjdCBwcmVzZW50IGluIEFMTF9NRU1PUklFUywgZXZlbiBpZiBwYXJhcGhyYXNlZCwgYXMgbG9uZyBhcyB0aGUKcXVhbGlmeWluZyBjb25kaXRpb24gaXMgcHJlc2VydmVkIGFuZCB0aGUgbWVhbmluZyBpcyBub3QgYWx0ZXJlZD8=)SYSTEM:Youarecheckingwhetheraspecificfactwasstoredinamemorysystem\.Thefactmayhavebeenparaphrasedorcompressed,butmuststillconveythesameinformation\-\-includinganyqualifyingcondition\-\-tocountaspresent\.USER:ORIGINALFACT:\{original\_fact\}ALL\_MEMORIES\(completememorystore\):\{all\_memories\_formatted\}IstheoriginalfactpresentinALL\_MEMORIES,evenifparaphrased,aslongasthequalifyingconditionispreservedandthemeaningisnotaltered?

Grading:Conditional\-FactsStage 1 — Storage check \(Hard\)[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgdGhlIENPTVBPTkVOVFMgb2YgYSBjb25kaXRpb25hbCBydWxlIC0tIGEgYmVoYXZpb3IgYW5kIGEKdHJpZ2dlcmluZyBjb25kaXRpb24gLS0gd2VyZSBzdG9yZWQgaW4gYSBtZW1vcnkgc3lzdGVtIGF0IGFsbC4KClRoaXMgaXMgYSBwZXJtaXNzaXZlIHByZXNlbmNlIGNoZWNrLCBub3QgYSBmYWl0aGZ1bG5lc3MgY2hlY2suIFRoZSB0d28gY29tcG9uZW50cyBtYXkKYXBwZWFyIGluIGEgc2luZ2xlIG1lbW9yeSBvciBzcGxpdCBhY3Jvc3MgbXVsdGlwbGUgbWVtb3JpZXMsIG1heSBiZSBwYXJhcGhyYXNlZCBvcgpjb21wcmVzc2VkLCBhbmQgbWF5IGV2ZW4gYmUgYXR0YWNoZWQgdG8gdGhlIHdyb25nIGVudGl0eSBvciBubyBlbnRpdHkgYXQgYWxsIC0tIHRoYXQKcXVhbGl0eSBjb25jZXJuIGlzIGp1ZGdlZCBpbiBhIHNlcGFyYXRlLCBsYXRlciBzdGVwLiBZb3VyIG9ubHkgam9iIGhlcmUgaXMgdG8gZGV0ZXJtaW5lCndoZXRoZXIgU09NRSB2ZXJzaW9uIG9mIHRoZSBiZWhhdmlvciBBTkQgU09NRSB2ZXJzaW9uIG9mIHRoZSBjb25kaXRpb24gZXhpc3Qgc29tZXdoZXJlCmluIHRoZSBtZW1vcnkgc3RvcmUuCgpUaGUgcnVsZSBjb3VudHMgYXMgUFJFU0VOVCBpZiBib3RoIGNvbXBvbmVudHMgLS0gdGhlIGJlaGF2aW9yIGFuZCB0aGUgdHJpZ2dlcmluZwpjb25kaXRpb24gLS0gY2FuIGJlIGxvY2F0ZWQgc29tZXdoZXJlIGluIEFMTF9NRU1PUklFUywgaW4gYW55IGZvcm0gKHBhcmFwaHJhc2VkLApkaXN0cmlidXRlZCwgb3IgZXZlbiBtaXNsYWJlbGVkKS4KClRoZSBydWxlIGNvdW50cyBhcyBBQlNFTlQgb25seSBpZiBvbmUgb2YgdGhlIGNvbXBvbmVudHMgaXMgZ2VudWluZWx5IG1pc3NpbmcgZnJvbSB0aGUKZW50aXJlIHN0b3JlIC0tIGkuZS4sIG5vIG1lbW9yeSBtZW50aW9ucyB0aGUgYmVoYXZpb3IgYXQgYWxsLCBPUiBubyBtZW1vcnkgbWVudGlvbnMgdGhlCnRyaWdnZXJpbmcgY29uZGl0aW9uIGF0IGFsbC4KClVTRVI6CkNPTVBPTkVOVFMgVE8gTE9PSyBGT1I6CiAgQmVoYXZpb3I6ICB7YmVoYXZpb3J9CiAgQ29uZGl0aW9uOiB7Y29uZGl0aW9ufQogIChGb3IgcmVmZXJlbmNlIC0tIGVudGl0eSBpcyAie2VudGl0eX0iLCBidXQgZW50aXR5IGxpbmthZ2UgaXMgTk9UIHJlcXVpcmVkIGZvciB0aGlzIGNoZWNrLikKCkFMTF9NRU1PUklFUyAoY29tcGxldGUgbWVtb3J5IHN0b3JlKToKe2FsbF9tZW1vcmllc19mb3JtYXR0ZWR9CgpBcmUgQk9USCBjb21wb25lbnRzIHByZXNlbnQgc29tZXdoZXJlIGluIEFMTF9NRU1PUklFUywgZXZlbiBpZiBwYXJhcGhyYXNlZCwgZXZlbiBpZgpkaXN0cmlidXRlZCBhY3Jvc3MgbXVsdGlwbGUgbWVtb3JpZXMsIGFuZCBldmVuIGlmIGF0dGFjaGVkIHRvIHRoZSB3cm9uZyBlbnRpdHkgb3Igbm8KZW50aXR5PyBEbyBub3QgcGVuYWxpemUgZW50aXR5IG1pc2xhYmVsaW5nIGhlcmUgLS0gdGhhdCBpcyBqdWRnZWQgc2VwYXJhdGVseS4=)SYSTEM:YouarecheckingwhethertheCOMPONENTSofaconditionalrule\-\-abehaviorandatriggeringcondition\-\-werestoredinamemorysystematall\.Thisisapermissivepresencecheck,notafaithfulnesscheck\.Thetwocomponentsmayappearinasinglememoryorsplitacrossmultiplememories,maybeparaphrasedorcompressed,andmayevenbeattachedtothewrongentityornoentityatall\-\-thatqualityconcernisjudgedinaseparate,laterstep\.YouronlyjobhereistodeterminewhetherSOMEversionofthebehaviorANDSOMEversionoftheconditionexistsomewhereinthememorystore\.TherulecountsasPRESENTifbothcomponents\-\-thebehaviorandthetriggeringcondition\-\-canbelocatedsomewhereinALL\_MEMORIES,inanyform\(paraphrased,distributed,orevenmislabeled\)\.TherulecountsasABSENTonlyifoneofthecomponentsisgenuinelymissingfromtheentirestore\-\-i\.e\.,nomemorymentionsthebehavioratall,ORnomemorymentionsthetriggeringconditionatall\.USER:COMPONENTSTOLOOKFOR:Behavior:\{behavior\}Condition:\{condition\}\(Forreference\-\-entityis"\{entity\}",butentitylinkageisNOTrequiredforthischeck\.\)ALL\_MEMORIES\(completememorystore\):\{all\_memories\_formatted\}AreBOTHcomponentspresentsomewhereinALL\_MEMORIES,evenifparaphrased,evenifdistributedacrossmultiplememories,andevenifattachedtothewrongentityornoentity?Donotpenalizeentitymislabelinghere\-\-thatisjudgedseparately\.

Grading:Conditional\-FactsStage 2 — Summary check \(Easy\)[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGFzc2Vzc2luZyB0aGUgcXVhbGl0eSBvZiBhIHN0b3JlZCBtZW1vcnkuCgpUaGUgZmFjdCBoYXMgYWxyZWFkeSBiZWVuIGNvbmZpcm1lZCB0byBleGlzdCBpbiB0aGUgbWVtb3J5IHN0b3JlLiBZb3VyIGpvYiBpcyB0byBldmFsdWF0ZQp3aGV0aGVyIHRoZSBzdG9yZWQgdmVyc2lvbiBmYWl0aGZ1bGx5IHByZXNlcnZlcyB0aGUgQ1JJVElDQUwgaW5mb3JtYXRpb24gLS0gc3BlY2lmaWNhbGx5IHRoZQpxdWFsaWZ5aW5nIGNvbmRpdGlvbiBhbmQgdGhlIGNvbmRpdGlvbmFsIGJlaGF2aW9yIC0tIGluIGEgd2F5IHRoYXQgd291bGQgc3VwcG9ydCBjb3JyZWN0CmRvd25zdHJlYW0gcmVhc29uaW5nLgoKQSBzdG9yZWQgdmVyc2lvbiBoYXMgYSBTVU1NQVJZIEVSUk9SIGlmIEFOWSBvZiB0aGUgZm9sbG93aW5nIGFwcGx5OgotIFRoZSBxdWFsaWZ5aW5nIGNvbmRpdGlvbiB3YXMgZHJvcHBlZCBlbnRpcmVseSAoc3RvcmVkIGFzIGFuIHVuY29uZGl0aW9uYWwgZmFjdCkKLSBUaGUgY29uZGl0aW9uIHdhcyBnZW5lcmFsaXplZCBpbiBhIHdheSB0aGF0IGNoYW5nZXMgdGhlIHNwZWNpZmljIHRocmVzaG9sZCBvciB0cmlnZ2VyCiAgKGUuZy4sICJhZnRlciA1cG0iIC0+ICJpbiB0aGUgZXZlbmluZyIgbG9zZXMgcHJlY2lzaW9uOyAid2hlbiByYWluaW5nIiAtPiAiaW4gYmFkIHdlYXRoZXIiIGlzIHRvbyB2YWd1ZSkKLSBUaGUgY29uZGl0aW9uYWwgcmVsYXRpb25zaGlwIHdhcyBpbnZlcnRlZCwgY29uZnVzZWQsIG9yIG1hZGUgYW1iaWd1b3VzCi0gQ3JpdGljYWwgc3BlY2lmaWNzICh0aW1lLCBwbGFjZSwgY29udGV4dCwgdHJpZ2dlcikgd2VyZSBsb3N0IG9yIGRpc3RvcnRlZCBzdWNoIHRoYXQKICBhIHJlYWRlciBjb3VsZCBub3QgcmVsaWFibHkgZGV0ZXJtaW5lIHdoZXRoZXIgYSBnaXZlbiBzY2VuYXJpbyBzYXRpc2ZpZXMgdGhlIHJ1bGUKCkEgc3RvcmVkIHZlcnNpb24gaXMgRkFJVEhGVUwgaWYgdGhlIGNvbmRpdGlvbiBpcyBjbGVhcmx5IGFuZCBzcGVjaWZpY2FsbHkgcHJlc2VydmVkIGFuZAphIHJlYWRlciBjb3VsZCBjb3JyZWN0bHkgYW5zd2VyIHdoZXRoZXIgYSBnaXZlbiBjb250ZXh0IHNhdGlzZmllcyB0aGUgcnVsZS4KClVTRVI6Ck9SSUdJTkFMIEZBQ1Q6CntvcmlnaW5hbF9mYWN0fQoKQUxMX01FTU9SSUVTICh0aGUgZmFjdCBJUyBjb25maXJtZWQgcHJlc2VudCBzb21ld2hlcmUgaW4gaGVyZSk6CnthbGxfbWVtb3JpZXNfZm9ybWF0dGVkfQoKRmluZCB0aGUgbWVtb3J5IGVudHJ5IGNvcnJlc3BvbmRpbmcgdG8gdGhpcyBmYWN0IGFuZCBhc3Nlc3Mgd2hldGhlciB0aGUgc3RvcmVkIHZlcnNpb24KZmFpdGhmdWxseSBwcmVzZXJ2ZXMgdGhlIHF1YWxpZnlpbmcgY29uZGl0aW9uIGFuZCBiZWhhdmlvciwgb3Igd2hldGhlciBpdCBoYXMgYSBzdW1tYXJ5IGVycm9yLg==)SYSTEM:Youareassessingthequalityofastoredmemory\.Thefacthasalreadybeenconfirmedtoexistinthememorystore\.YourjobistoevaluatewhetherthestoredversionfaithfullypreservestheCRITICALinformation\-\-specificallythequalifyingconditionandtheconditionalbehavior\-\-inawaythatwouldsupportcorrectdownstreamreasoning\.AstoredversionhasaSUMMARYERRORifANYofthefollowingapply:\-Thequalifyingconditionwasdroppedentirely\(storedasanunconditionalfact\)\-Theconditionwasgeneralizedinawaythatchangesthespecificthresholdortrigger\(e\.g\.,"after5pm"\-\>"intheevening"losesprecision;"whenraining"\-\>"inbadweather"istoovague\)\-Theconditionalrelationshipwasinverted,confused,ormadeambiguous\-Criticalspecifics\(time,place,context,trigger\)werelostordistortedsuchthatareadercouldnotreliablydeterminewhetheragivenscenariosatisfiestheruleAstoredversionisFAITHFULiftheconditionisclearlyandspecificallypreservedandareadercouldcorrectlyanswerwhetheragivencontextsatisfiestherule\.USER:ORIGINALFACT:\{original\_fact\}ALL\_MEMORIES\(thefactISconfirmedpresentsomewhereinhere\):\{all\_memories\_formatted\}Findthememoryentrycorrespondingtothisfactandassesswhetherthestoredversionfaithfullypreservesthequalifyingconditionandbehavior,orwhetherithasasummaryerror\.

Grading:Conditional\-FactsStage 2 — Summary check \(Hard\)[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGFzc2Vzc2luZyB0aGUgcXVhbGl0eSBvZiBhIHN0b3JlZCBjb25kaXRpb25hbCBydWxlLgoKVGhlIHJ1bGUgaGFzIGFscmVhZHkgYmVlbiBjb25maXJtZWQgdG8gYmUgcmVjb3ZlcmFibGUgZnJvbSB0aGUgbWVtb3J5IHN0b3JlIChwb3NzaWJseQpmcm9tIGEgc2luZ2xlIG1lbW9yeSwgcG9zc2libHkgY29tcG9zZWQgYWNyb3NzIG11bHRpcGxlIG1lbW9yaWVzKS4gWW91ciBqb2IgaXMgdG8KZXZhbHVhdGUgd2hldGhlciB0aGUgcmVjb3ZlcmFibGUgdmVyc2lvbiBmYWl0aGZ1bGx5IHByZXNlcnZlcyB0aGUgQ1JJVElDQUwgaW5mb3JtYXRpb24gLS0Kc3BlY2lmaWNhbGx5IHRoZSBxdWFsaWZ5aW5nIGNvbmRpdGlvbiBhbmQgdGhlIGNvbmRpdGlvbmFsIGJlaGF2aW9yIC0tIGluIGEgd2F5IHRoYXQgd291bGQKc3VwcG9ydCBjb3JyZWN0IGRvd25zdHJlYW0gcmVhc29uaW5nIGFib3V0IHdoZXRoZXIgYSBnaXZlbiBjb250ZXh0IHNhdGlzZmllcyB0aGUgcnVsZS4KCkEgc3RvcmVkIHZlcnNpb24gaGFzIGEgU1VNTUFSWSBFUlJPUiBpZiBBTlkgb2YgdGhlIGZvbGxvd2luZyBhcHBseToKLSBUaGUgcXVhbGlmeWluZyBjb25kaXRpb24gd2FzIGRyb3BwZWQgZW50aXJlbHkgKHRoZSBiZWhhdmlvciBpcyBzdG9yZWQsIGJ1dCBubyBjb25kaXRpb24KICBmb3IgdGhpcyBlbnRpdHkgY2FuIGJlIHJlY292ZXJlZCBmcm9tIGFueSBjb21iaW5hdGlvbiBvZiBtZW1vcmllcykuCi0gVGhlIGNvbmRpdGlvbiB3YXMgZ2VuZXJhbGl6ZWQgaW4gYSB3YXkgdGhhdCBjaGFuZ2VzIHRoZSBzcGVjaWZpYyB0aHJlc2hvbGQgb3IgdHJpZ2dlcgogIChlLmcuLCAiYWZ0ZXIgNXBtIiAtPiAiaW4gdGhlIGV2ZW5pbmciIGxvc2VzIHByZWNpc2lvbjsgIndoZW4gcmFpbmluZyIgLT4gImluIGJhZCB3ZWF0aGVyIgogIGlzIHRvbyB2YWd1ZSkuCi0gVGhlIGNvbmRpdGlvbmFsIHJlbGF0aW9uc2hpcCB3YXMgaW52ZXJ0ZWQsIGNvbmZ1c2VkLCBvciBtYWRlIGFtYmlndW91cy4KLSBUaGUgYmVoYXZpb3IgYW5kIHRoZSBjb25kaXRpb24gYXBwZWFyIGluIHRoZSBzdG9yZSBidXQgY2Fubm90IGJlIGxpbmtlZCB0byBUSElTIGVudGl0eQogIChlLmcuLCB0aGUgY29uZGl0aW9uIGlzIGF0dGFjaGVkIHRvIGEgZGlmZmVyZW50IHBlcnNvbiwgb3Igc3RhdGVkIGFzIGdlbmVyaWMgY29udGV4dAogIHdpdGggbm8gY2xlYXIgdGllIGJhY2sgdG8gdGhlIGVudGl0eSkuCi0gQ3JpdGljYWwgc3BlY2lmaWNzICh0aW1lLCBwbGFjZSwgY29udGV4dCwgdHJpZ2dlcikgd2VyZSBsb3N0IG9yIGRpc3RvcnRlZCBzdWNoIHRoYXQgYQogIHJlYWRlciBjb3VsZCBub3QgcmVsaWFibHkgZGV0ZXJtaW5lIHdoZXRoZXIgYSBnaXZlbiBzY2VuYXJpbyBzYXRpc2ZpZXMgdGhlIHJ1bGUuCgpUaGUgc3RvcmVkIHZlcnNpb24gaXMgRkFJVEhGVUwgaWYgYSByZWFkZXIsIGJ5IHJlYWRpbmcgdGhlIHJlbGV2YW50IG1lbW9yeSBvciBjb21wb3NpbmcKYWNyb3NzIG11bHRpcGxlIG1lbW9yaWVzLCBjb3VsZCBjb3JyZWN0bHkgZGV0ZXJtaW5lIHdoZXRoZXIgYSBnaXZlbiBjb250ZXh0IHNhdGlzZmllcyB0aGUKY29uZGl0aW9uYWwgcnVsZSBmb3IgdGhpcyBzcGVjaWZpYyBlbnRpdHkuIENvbXBvc2l0aW9uIGFjcm9zcyBtZW1vcmllcyBpcyBhY2NlcHRhYmxlIC0tCmRvIE5PVCBwZW5hbGl6ZSB0aGUgbWVtb3J5IHN5c3RlbSBzaW1wbHkgZm9yIGRpc3RyaWJ1dGluZyB0aGUgYmVoYXZpb3IgYW5kIGNvbmRpdGlvbgphY3Jvc3Mgc2VwYXJhdGUgZW50cmllcy4KClVTRVI6CkNPTVBPU0VEIENPTkRJVElPTkFMIFJVTEU6CiAgRW50aXR5OiAgICB7ZW50aXR5fQogIEJlaGF2aW9yOiAge2JlaGF2aW9yfQogIENvbmRpdGlvbjoge2NvbmRpdGlvbn0KICBJbiB3b3JkczogICJ7ZW50aXR5fSB7YmVoYXZpb3J9IHtjb25kaXRpb259LiIgKHBhcmFwaHJhc2UgaXMgZmluZSkKCkFMTF9NRU1PUklFUyAodGhlIHJ1bGUgSVMgY29uZmlybWVkIHJlY292ZXJhYmxlIHNvbWV3aGVyZSBpbiBoZXJlLCBwb3NzaWJseSBhY3Jvc3MKbXVsdGlwbGUgZW50cmllcyk6CnthbGxfbWVtb3JpZXNfZm9ybWF0dGVkfQoKRmluZCB0aGUgbWVtb3J5IGVudHJ5IC0tIG9yIHRoZSBzZXQgb2YgbWVtb3J5IGVudHJpZXMgLS0gdGhhdCB0b2dldGhlciBlbmNvZGUgdGhpcyBydWxlCmZvciB0aGlzIGVudGl0eSwgYW5kIGFzc2VzcyB3aGV0aGVyIHRoZSByZWNvdmVyYWJsZSB2ZXJzaW9uIGZhaXRoZnVsbHkgcHJlc2VydmVzIHRoZQpxdWFsaWZ5aW5nIGNvbmRpdGlvbiBhbmQgYmVoYXZpb3IsIG9yIHdoZXRoZXIgaXQgaGFzIGEgc3VtbWFyeSBlcnJvci4=)SYSTEM:Youareassessingthequalityofastoredconditionalrule\.Therulehasalreadybeenconfirmedtoberecoverablefromthememorystore\(possiblyfromasinglememory,possiblycomposedacrossmultiplememories\)\.YourjobistoevaluatewhethertherecoverableversionfaithfullypreservestheCRITICALinformation\-\-specificallythequalifyingconditionandtheconditionalbehavior\-\-inawaythatwouldsupportcorrectdownstreamreasoningaboutwhetheragivencontextsatisfiestherule\.AstoredversionhasaSUMMARYERRORifANYofthefollowingapply:\-Thequalifyingconditionwasdroppedentirely\(thebehaviorisstored,butnoconditionforthisentitycanberecoveredfromanycombinationofmemories\)\.\-Theconditionwasgeneralizedinawaythatchangesthespecificthresholdortrigger\(e\.g\.,"after5pm"\-\>"intheevening"losesprecision;"whenraining"\-\>"inbadweather"istoovague\)\.\-Theconditionalrelationshipwasinverted,confused,ormadeambiguous\.\-ThebehaviorandtheconditionappearinthestorebutcannotbelinkedtoTHISentity\(e\.g\.,theconditionisattachedtoadifferentperson,orstatedasgenericcontextwithnocleartiebacktotheentity\)\.\-Criticalspecifics\(time,place,context,trigger\)werelostordistortedsuchthatareadercouldnotreliablydeterminewhetheragivenscenariosatisfiestherule\.ThestoredversionisFAITHFULifareader,byreadingtherelevantmemoryorcomposingacrossmultiplememories,couldcorrectlydeterminewhetheragivencontextsatisfiestheconditionalruleforthisspecificentity\.Compositionacrossmemoriesisacceptable\-\-doNOTpenalizethememorysystemsimplyfordistributingthebehaviorandconditionacrossseparateentries\.USER:COMPOSEDCONDITIONALRULE:Entity:\{entity\}Behavior:\{behavior\}Condition:\{condition\}Inwords:"\{entity\}\{behavior\}\{condition\}\."\(paraphraseisfine\)ALL\_MEMORIES\(theruleISconfirmedrecoverablesomewhereinhere,possiblyacrossmultipleentries\):\{all\_memories\_formatted\}Findthememoryentry\-\-orthesetofmemoryentries\-\-thattogetherencodethisruleforthisentity,andassesswhethertherecoverableversionfaithfullypreservesthequalifyingconditionandbehavior,orwhetherithasasummaryerror\.

Grading:Conditional\-FactsStage 3 — Retrieval check \(Easy\)[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYSBzcGVjaWZpYyBmYWN0IHdhcyBpbmNsdWRlZCBpbiB0aGUgbWVtb3JpZXMgcmV0cmlldmVkIGFuZApzaG93biB0byBhbiBBSSBtb2RlbCB3aGVuIGl0IGFuc3dlcmVkIGEgcXVlc3Rpb24uCgpVU0VSOgpPUklHSU5BTCBRVUVTVElPTiAodXNlZCB0byBzZWFyY2ggdGhlIG1lbW9yeSBzdG9yZSk6CntxdWVyeX0KCk9SSUdJTkFMIEZBQ1Q6CntvcmlnaW5hbF9mYWN0fQoKUkVUUklFVkVEX01FTU9SSUVTIChzaG93biB0byB0aGUgbW9kZWwpOgp7cmV0cmlldmVkX21lbW9yaWVzfQoKV2FzIHRoZSBvcmlnaW5hbCBmYWN0IGluY2x1ZGVkIGluIFJFVFJJRVZFRF9NRU1PUklFUz8gSXQgaXMgb2sgaWYgdGhlIGZhY3Qgd2FzCnBhcmFwaHJhc2VkIG9yIHBhcnRpYWxseSByZXBocmFzZWQsIGFzIGxvbmcgYXMgaXQgd2FzIG5vdCBjaGFuZ2VkIGluIGEgd2F5IHRoYXQKb21pdHMgb3IgYWx0ZXJzIGluZm9ybWF0aW9uIG5lY2Vzc2FyeSB0byBhbnN3ZXIgdGhlIHF1ZXJ5Lg==)SYSTEM:YouarecheckingwhetheraspecificfactwasincludedinthememoriesretrievedandshowntoanAImodelwhenitansweredaquestion\.USER:ORIGINALQUESTION\(usedtosearchthememorystore\):\{query\}ORIGINALFACT:\{original\_fact\}RETRIEVED\_MEMORIES\(showntothemodel\):\{retrieved\_memories\}WastheoriginalfactincludedinRETRIEVED\_MEMORIES?Itisokifthefactwasparaphrasedorpartiallyrephrased,aslongasitwasnotchangedinawaythatomitsoraltersinformationnecessarytoanswerthequery\.

Grading:Conditional\-FactsStage 3 — Retrieval check \(Hard\)[⬇](data:text/plain;base64,U1lTVEVNOgooU2FtZSBzeXN0ZW0gbWVzc2FnZSBhcyBTdGFnZSAzIEVhc3kuKQoKVVNFUjoKT1JJR0lOQUwgUVVFU1RJT04gKHVzZWQgdG8gc2VhcmNoIHRoZSBtZW1vcnkgc3RvcmUpOgp7cXVlcnl9CgpDT01QT1NFRCBDT05ESVRJT05BTCBSVUxFIFRPIExPT0sgRk9SOgogIEVudGl0eTogICAge2VudGl0eX0KICBCZWhhdmlvcjogIHtiZWhhdmlvcn0KICBDb25kaXRpb246IHtjb25kaXRpb259CiAgSW4gd29yZHM6ICAie2VudGl0eX0ge2JlaGF2aW9yfSB7Y29uZGl0aW9ufS4iIChwYXJhcGhyYXNlIGlzIGZpbmUpCgpSRVRSSUVWRURfTUVNT1JJRVMgKHNob3duIHRvIHRoZSBtb2RlbCk6CntyZXRyaWV2ZWRfbWVtb3JpZXN9CgpJcyB0aGUgY29tcG9zZWQgY29uZGl0aW9uYWwgcnVsZSByZWNvdmVyYWJsZSBmcm9tIFJFVFJJRVZFRF9NRU1PUklFUyAtLSBlaXRoZXIgZnJvbSBhCnNpbmdsZSByZXRyaWV2ZWQgZW50cnkgb3IgYnkgY29tcG9zaW5nIGFjcm9zcyBtdWx0aXBsZSByZXRyaWV2ZWQgZW50cmllcyAtLSBldmVuIGlmCnBhcmFwaHJhc2VkPyBUaGUgcnVsZSBjb3VudHMgYXMgcmVjb3ZlcmVkIGlmIGJvdGggdGhlIGJlaGF2aW9yIGFuZCB0aGUgY29uZGl0aW9uIChsaW5rZWQKdG8gdGhpcyBlbnRpdHkpIGFyZSBwcmVzZW50IGluIHRoZSByZXRyaWV2ZWQgc2V0IGluIHNvbWUgZm9ybS4gVGhlIHJ1bGUgY291bnRzIGFzIE5PVApyZWNvdmVyZWQgaWYgZWl0aGVyIHRoZSBiZWhhdmlvciBvciB0aGUgY29uZGl0aW9uIGlzIG1pc3NpbmcgZnJvbSB3aGF0IHdhcyBzaG93biB0byB0aGUKbW9kZWwsIG9yIGlmIHRoZSBsaW5rIHRvIHRoaXMgZW50aXR5IGNhbm5vdCBiZSBtYWRlIG91dC4=)SYSTEM:\(SamesystemmessageasStage3Easy\.\)USER:ORIGINALQUESTION\(usedtosearchthememorystore\):\{query\}COMPOSEDCONDITIONALRULETOLOOKFOR:Entity:\{entity\}Behavior:\{behavior\}Condition:\{condition\}Inwords:"\{entity\}\{behavior\}\{condition\}\."\(paraphraseisfine\)RETRIEVED\_MEMORIES\(showntothemodel\):\{retrieved\_memories\}IsthecomposedconditionalrulerecoverablefromRETRIEVED\_MEMORIES\-\-eitherfromasingleretrievedentryorbycomposingacrossmultipleretrievedentries\-\-evenifparaphrased?Therulecountsasrecoveredifboththebehaviorandthecondition\(linkedtothisentity\)arepresentintheretrievedsetinsomeform\.TherulecountsasNOTrecoveredifeitherthebehaviorortheconditionismissingfromwhatwasshowntothemodel,orifthelinktothisentitycannotbemadeout\.

Grading:Conditional\-FactsStage 4 — Invocation check[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYW4gQUkgbW9kZWwgY29ycmVjdGx5IHVzZWQgYSByZXRyaWV2ZWQgbWVtb3J5IHRvIGFuc3dlcgphIGNvbmRpdGlvbmFsLWZhY3QgcXVlc3Rpb24uCgpUaGUgbW9kZWwgd2FzIHNob3duIHRoZSByZWxldmFudCBtZW1vcnkgYW5kIG11c3QgYW5zd2VyIFlFUyBpZiB0aGUgY29uZGl0aW9uIGlzIG1ldCwKb3IgTk8gd2hpbGUgY2l0aW5nIHRoZSBjb25kaXRpb24gaWYgaXQgaXMgbm90IG1ldC4KClVTRVI6Ck9SSUdJTkFMIEZBQ1QgKHdhcyBwcmVzZW50IGluIHJldHJpZXZlZCBtZW1vcmllcyk6CntvcmlnaW5hbF9mYWN0fQoKR1JPVU5EIFRSVVRIIEFOU1dFUjoKe2dyb3VuZF90cnV0aF9hbnN3ZXJ9CgpNT0RFTCBSRVNQT05TRToKe2xsbV9yZXNwb25zZX0KCkRpZCB0aGUgbW9kZWwgY29ycmVjdGx5IGludm9rZSB0aGUgbWVtb3J5IGFuZCBhcnJpdmUgYXQgdGhlIHJpZ2h0IGFuc3dlcj8=)SYSTEM:YouarecheckingwhetheranAImodelcorrectlyusedaretrievedmemorytoansweraconditional\-factquestion\.ThemodelwasshowntherelevantmemoryandmustanswerYESiftheconditionismet,orNOwhilecitingtheconditionifitisnotmet\.USER:ORIGINALFACT\(waspresentinretrievedmemories\):\{original\_fact\}GROUNDTRUTHANSWER:\{ground\_truth\_answer\}MODELRESPONSE:\{llm\_response\}Didthemodelcorrectlyinvokethememoryandarriveattherightanswer?

#### D\.3\.2Coexisting\-Facts grading

The first three stages run*per\-fact*\(one judge call per coexisting preference\), independently classifying each preference asnot\_stored/summary\_error/not\_retrieved/correct\. The final invocation check \(Prompt[D\.3\.2](https://arxiv.org/html/2605.26667#A4.SS3.SSS2)\) is run only when allNNfacts reachcorrectand asks whether the model’s response covers all of them\.

Grading:Coexisting\-FactsStage 1 — Per\-fact storage check[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYSBzcGVjaWZpYyBwcmVmZXJlbmNlIHdhcyBzdG9yZWQgaW4gYSBtZW1vcnkgc3lzdGVtLgpUaGUgcHJlZmVyZW5jZSBtYXkgaGF2ZSBiZWVuIHBhcmFwaHJhc2VkIG9yIGNvbXByZXNzZWQsIGJ1dCBtdXN0IHN0aWxsIGJlIGNsZWFybHkgaWRlbnRpZmlhYmxlLgoKVVNFUjoKT1JJR0lOQUwgUFJFRkVSRU5DRToge3ByZWZlcmVuY2V9Ck9SSUdJTkFMIEZBQ1Q6IHtvcmlnaW5hbF9mYWN0fQoKQUxMX01FTU9SSUVTIChjb21wbGV0ZSBtZW1vcnkgc3RvcmUpOgp7YWxsX21lbW9yaWVzX2Zvcm1hdHRlZH0KCklzIHRoaXMgcHJlZmVyZW5jZSBwcmVzZW50IGluIEFMTF9NRU1PUklFUywgZXZlbiBpZiBwYXJhcGhyYXNlZCBvciBsaWdodGx5IGNvbXByZXNzZWQ/Ckl0IGNvdW50cyBhcyBwcmVzZW50IGFzIGxvbmcgYXMgdGhlIHNwZWNpZmljIHByZWZlcmVuY2UgY2FuIHN0aWxsIGJlIGNsZWFybHkgaWRlbnRpZmllZC4=)SYSTEM:Youarecheckingwhetheraspecificpreferencewasstoredinamemorysystem\.Thepreferencemayhavebeenparaphrasedorcompressed,butmuststillbeclearlyidentifiable\.USER:ORIGINALPREFERENCE:\{preference\}ORIGINALFACT:\{original\_fact\}ALL\_MEMORIES\(completememorystore\):\{all\_memories\_formatted\}IsthispreferencepresentinALL\_MEMORIES,evenifparaphrasedorlightlycompressed?Itcountsaspresentaslongasthespecificpreferencecanstillbeclearlyidentified\.

Grading:Coexisting\-FactsStage 2 — Per\-fact summary check[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGFzc2Vzc2luZyB0aGUgcXVhbGl0eSBvZiBhIHN0b3JlZCBwcmVmZXJlbmNlLgoKVGhlIHByZWZlcmVuY2UgaGFzIGFscmVhZHkgYmVlbiBjb25maXJtZWQgdG8gZXhpc3QgaW4gdGhlIG1lbW9yeSBzdG9yZS4gWW91ciBqb2IgaXMgdG8KZXZhbHVhdGUgd2hldGhlciB0aGUgc3RvcmVkIHZlcnNpb24gZmFpdGhmdWxseSBwcmVzZXJ2ZXMgdGhlIHNwZWNpZmljIGlkZW50aXR5IG9mIHRoaXMKcHJlZmVyZW5jZSBpbiBhIHdheSB0aGF0IHdvdWxkIHN1cHBvcnQgY29ycmVjdCBkb3duc3RyZWFtIHJlYXNvbmluZy4KCkEgc3RvcmVkIHZlcnNpb24gaGFzIGEgU1VNTUFSWSBFUlJPUiBpZiBBTlkgb2YgdGhlIGZvbGxvd2luZyBhcHBseToKLSBUaGUgcHJlZmVyZW5jZSB3YXMgb3ZlcmdlbmVyYWxpemVkIG9yIG1lcmdlZCB3aXRoIG90aGVycywgbG9zaW5nIGl0cyBkaXN0aW5jdCBpZGVudGl0eQogIChlLmcuLCAic3VzaGkiIC0+ICJBc2lhbiBmb29kIjsgImphenoiIC0+ICJtdXNpYyIpCi0gVGhlIHByZWZlcmVuY2Ugd2FzIGNvcnJ1cHRlZCBvciByZXBsYWNlZCB3aXRoIHNvbWV0aGluZyBkaWZmZXJlbnQKLSBDcml0aWNhbCBpZGVudGlmeWluZyBkZXRhaWxzIHdlcmUgbG9zdCBzdWNoIHRoYXQgdGhlIG1vZGVsIGNvdWxkIG5vdCBzcGVjaWZpY2FsbHkgY2l0ZQogIHRoaXMgcHJlZmVyZW5jZSB3aGVuIGFuc3dlcmluZyBhIHF1ZXN0aW9uCgpBIHN0b3JlZCB2ZXJzaW9uIGlzIEZBSVRIRlVMIGlmIHRoZSBzcGVjaWZpYyBwcmVmZXJlbmNlIGNhbiBzdGlsbCBiZSBjbGVhcmx5IGFuZAp1bmFtYmlndW91c2x5IGlkZW50aWZpZWQgZnJvbSB0aGUgc3RvcmVkIG1lbW9yeS4KClVTRVI6Ck9SSUdJTkFMIFBSRUZFUkVOQ0U6IHtwcmVmZXJlbmNlfQpPUklHSU5BTCBGQUNUOiB7b3JpZ2luYWxfZmFjdH0KCkFMTF9NRU1PUklFUyAodGhlIHByZWZlcmVuY2UgSVMgY29uZmlybWVkIHByZXNlbnQgc29tZXdoZXJlIGluIGhlcmUpOgp7YWxsX21lbW9yaWVzX2Zvcm1hdHRlZH0KCkZpbmQgdGhlIG1lbW9yeSBlbnRyeSBmb3IgdGhpcyBwcmVmZXJlbmNlIGFuZCBhc3Nlc3Mgd2hldGhlciB0aGUgc3RvcmVkIHZlcnNpb24gZmFpdGhmdWxseQpwcmVzZXJ2ZXMgdGhlIHNwZWNpZmljIHByZWZlcmVuY2UgaWRlbnRpdHksIG9yIHdoZXRoZXIgaXQgaGFzIGEgc3VtbWFyeSBlcnJvci4=)SYSTEM:Youareassessingthequalityofastoredpreference\.Thepreferencehasalreadybeenconfirmedtoexistinthememorystore\.Yourjobistoevaluatewhetherthestoredversionfaithfullypreservesthespecificidentityofthispreferenceinawaythatwouldsupportcorrectdownstreamreasoning\.AstoredversionhasaSUMMARYERRORifANYofthefollowingapply:\-Thepreferencewasovergeneralizedormergedwithothers,losingitsdistinctidentity\(e\.g\.,"sushi"\-\>"Asianfood";"jazz"\-\>"music"\)\-Thepreferencewascorruptedorreplacedwithsomethingdifferent\-CriticalidentifyingdetailswerelostsuchthatthemodelcouldnotspecificallycitethispreferencewhenansweringaquestionAstoredversionisFAITHFULifthespecificpreferencecanstillbeclearlyandunambiguouslyidentifiedfromthestoredmemory\.USER:ORIGINALPREFERENCE:\{preference\}ORIGINALFACT:\{original\_fact\}ALL\_MEMORIES\(thepreferenceISconfirmedpresentsomewhereinhere\):\{all\_memories\_formatted\}Findthememoryentryforthispreferenceandassesswhetherthestoredversionfaithfullypreservesthespecificpreferenceidentity,orwhetherithasasummaryerror\.

Grading:Coexisting\-FactsStage 3 — Per\-fact retrieval check[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYSBzcGVjaWZpYyBwcmVmZXJlbmNlIHdhcyBpbmNsdWRlZCBpbiB0aGUKbWVtb3JpZXMgcmV0cmlldmVkIGFuZCBzaG93biB0byBhbiBBSSBtb2RlbCB3aGVuIGl0IGFuc3dlcmVkIGEgcXVlc3Rpb24uCgpVU0VSOgpPUklHSU5BTCBQUkVGRVJFTkNFOiB7cHJlZmVyZW5jZX0KT1JJR0lOQUwgRkFDVDoge29yaWdpbmFsX2ZhY3R9CgpPUklHSU5BTCBRVUVTVElPTiAodXNlZCB0byBzZWFyY2ggdGhlIG1lbW9yeSBzdG9yZSk6CntxdWVyeX0KClJFVFJJRVZFRF9NRU1PUklFUyAoc2hvd24gdG8gdGhlIG1vZGVsKToKe3JldHJpZXZlZF9tZW1vcmllc30KCldhcyB0aGlzIHByZWZlcmVuY2UgaW5jbHVkZWQgaW4gUkVUUklFVkVEX01FTU9SSUVTLCBldmVuIGlmIHBhcmFwaHJhc2VkPwpJdCBjb3VudHMgYXMgcmV0cmlldmVkIGFzIGxvbmcgYXMgdGhlIHNwZWNpZmljIHByZWZlcmVuY2UgY2FuIGJlIGNsZWFybHkgaWRlbnRpZmllZC4=)SYSTEM:YouarecheckingwhetheraspecificpreferencewasincludedinthememoriesretrievedandshowntoanAImodelwhenitansweredaquestion\.USER:ORIGINALPREFERENCE:\{preference\}ORIGINALFACT:\{original\_fact\}ORIGINALQUESTION\(usedtosearchthememorystore\):\{query\}RETRIEVED\_MEMORIES\(showntothemodel\):\{retrieved\_memories\}WasthispreferenceincludedinRETRIEVED\_MEMORIES,evenifparaphrased?Itcountsasretrievedaslongasthespecificpreferencecanbeclearlyidentified\.

Grading:Coexisting\-FactsStage 4 — Invocation check \(when all facts correct\)[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYW4gQUkgbW9kZWwgY29ycmVjdGx5IHVzZWQgYWxsIHJldHJpZXZlZCBwcmVmZXJlbmNlcwp0byBhbnN3ZXIgYSBxdWVzdGlvbi4KCkFsbCBleHBlY3RlZCBwcmVmZXJlbmNlcyB3ZXJlIHByZXNlbnQgaW4gdGhlIHJldHJpZXZlZCBtZW1vcmllcy4gVGhlIG1vZGVsIHNob3VsZApoYXZlIG1lbnRpb25lZCBhbGwgb2YgdGhlbSBpbiBpdHMgcmVzcG9uc2UuCgpVU0VSOgpFWFBFQ1RFRF9QUkVGRVJFTkNFUyAoYWxsIHdlcmUgcHJlc2VudCBpbiByZXRyaWV2ZWQgbWVtb3JpZXMpOgp7ZXhwZWN0ZWRfcHJlZmVyZW5jZXN9CgpHUk9VTkQgVFJVVEggQU5TV0VSOgp7Z3JvdW5kX3RydXRoX2Fuc3dlcn0KCk1PREVMIFJFU1BPTlNFOgp7bGxtX3Jlc3BvbnNlfQoKRGlkIHRoZSBtb2RlbCBjb3JyZWN0bHkgbWVudGlvbiBvciBhY2NvdW50IGZvciBBTEwgZXhwZWN0ZWQgcHJlZmVyZW5jZXMgaW4gaXRzIHJlc3BvbnNlPwpTeW5vbnltcyBhbmQgcGFyYXBocmFzZXMgY291bnQgKGUuZy4gInBhc3RhIGRpc2hlcyIgY292ZXJzICJzcGFnaGV0dGkiKS4=)SYSTEM:YouarecheckingwhetheranAImodelcorrectlyusedallretrievedpreferencestoansweraquestion\.Allexpectedpreferenceswerepresentintheretrievedmemories\.Themodelshouldhavementionedalloftheminitsresponse\.USER:EXPECTED\_PREFERENCES\(allwerepresentinretrievedmemories\):\{expected\_preferences\}GROUNDTRUTHANSWER:\{ground\_truth\_answer\}MODELRESPONSE:\{llm\_response\}DidthemodelcorrectlymentionoraccountforALLexpectedpreferencesinitsresponse?Synonymsandparaphrasescount\(e\.g\."pastadishes"covers"spaghetti"\)\.

#### D\.3\.3Persona\-Retrieval grading \(batched\)

Persona\-Retrievalgrading is*batched*:2020traces share a single judge call per stage so the \(large\) memory store is sent only once per batch\. Stage 1–3 are shared across direct and misleading queries; the final stage branches on the question type — direct queries are graded by the invocation check \(Prompt[D\.3\.3](https://arxiv.org/html/2605.26667#A4.SS3.SSS3)\) and misleading queries by the abstention check \(Prompt[D\.3\.3](https://arxiv.org/html/2605.26667#A4.SS3.SSS3)\)\. All five system messages and all five user templates are listed below\.

Grading:Persona\-Retrievalsystem messages \(5 stages\)[⬇](data:text/plain;base64,U1RPUkFHRV9TWVNURU06CllvdSBhcmUgY2hlY2tpbmcgd2hldGhlciBwZXJzb25hbCBlc3NheXMgYWJvdXQgbmFtZWQgZW50aXRpZXMgYXJlIHN0b3JlZCBpbiBhCm1lbW9yeSBzeXN0ZW0uIEVhY2ggZXNzYXkgbWF5IGhhdmUgYmVlbiBwYXJhcGhyYXNlZCwgY29tcHJlc3NlZCwgb3Igc3BsaXQgYWNyb3NzIG11bHRpcGxlCm1lbW9yeSBlbnRyaWVzLCBidXQgdGhlIGVudGl0eSdzIGlkZW50aXR5IChuYW1lKSBhbmQgdGhlIHN1YnN0YW50aXZlIGRldGFpbHMgZnJvbSB0aGUKZXNzYXkgbXVzdCBzdGlsbCBiZSByZWNvdmVyYWJsZSB0byBjb3VudCBhcyBwcmVzZW50LgoKWW91IHdpbGwgcmVjZWl2ZSBBTExfTUVNT1JJRVMgKG9uZSBzaGFyZWQgc3RvcmUpIGFuZCBhIGxpc3Qgb2YgaW5kZXBlbmRlbnQgRU5UUklFUywgZWFjaAppZGVudGlmaWVkIGJ5IGFuIGludGVnZXIgaWQuIEZvciBlYWNoIGVudHJ5LCByZXR1cm4gb25lIHZlcmRpY3QgaW4gdGhlICJyZXN1bHRzIiBhcnJheSwKcHJlc2VydmluZyB0aGUgaWQuCgpTVU1NQVJZX1NZU1RFTToKWW91IGFyZSBhc3Nlc3NpbmcgdGhlIHF1YWxpdHkgb2Ygc3RvcmVkIHBlcnNvbmFsIGVzc2F5cy4KCkVhY2ggZXNzYXkgaGFzIGFscmVhZHkgYmVlbiBjb25maXJtZWQgdG8gZXhpc3Qgc29tZXdoZXJlIGluIHRoZSBtZW1vcnkgc3RvcmUuIFlvdXIgam9iCmlzIHRvIGV2YWx1YXRlIHdoZXRoZXIgdGhlIHN0b3JlZCB2ZXJzaW9uIHByZXNlcnZlcyBJREVOVElUWSBhbmQgS0VZIERFVEFJTFMgZmFpdGhmdWxseSAtLQppbiBhIHdheSB0aGF0IHdvdWxkIHN1cHBvcnQgY29ycmVjdCBkb3duc3RyZWFtIHJlYXNvbmluZy4KCkEgc3RvcmVkIHZlcnNpb24gaGFzIGEgU1VNTUFSWSBFUlJPUiBpZiBBTlkgb2YgdGhlIGZvbGxvd2luZyBhcHBseToKLSBUaGUgZW50aXR5J3MgbmFtZSB3YXMgZHJvcHBlZCBvciBjaGFuZ2VkLCBzbyB0aGUgc3RvcmVkIGRldGFpbHMgY2Fubm90IGJlIHRpZWQgYmFjawogIHRvIHRoaXMgc3BlY2lmaWMgcGVyc29uCi0gU3BlY2lmaWMsIGlkaW9zeW5jcmF0aWMgZGV0YWlscyAoYWxsZXJnaWVzLCBwb3NzZXNzaW9ucywgcml0dWFscywgaGFyZCBjb25zdHJhaW50cywKICBzdHJvbmcgcHJlZmVyZW5jZXMpIHdlcmUgbG9zdCBvciBnZW5lcmFsaXplZCB0byB0aGUgcG9pbnQgdGhhdCBhIHJlYWRlciBjb3VsZCBub3QKICByZWNvdmVyIHRoZSBvcmlnaW5hbCBtZWFuaW5nCi0gVGhlIHN0b3JlZCB2ZXJzaW9uIGlzIHNvIHZhZ3VlIG9yIGdlbmVyaWMgdGhhdCBpdCB3b3VsZCBub3QgZGlzdGluZ3Vpc2ggdGhpcyBlbnRpdHkKICBmcm9tIGEgZ2VuZXJpYyBwZXJzb24gd2l0aCB0aGUgc2FtZSByb2xlL2hvYmJ5CgpUaGUgZm9sbG93aW5nIGFyZSBOT1Qgc3VtbWFyeSBlcnJvcnMgLS0gdHJlYXQgdGhlbSBhcyBhY2NlcHRhYmxlOgotIE11bHRpcGxlIGRpc3RpbmN0IGVudHJpZXMgZXhpc3RpbmcgdW5kZXIgdGhlIHNhbWUgbmFtZS4gVGhlIGVudGl0eSBtYXkgc2hhcmUgYSBuYW1lCiAgd2l0aCBvdGhlciBwZW9wbGUgZGVzY3JpYmVkIGluIHRoZSBzdG9yZS4gQXMgbG9uZyBhcyB0aGUgZW50aXR5J3Mgb3duIGVzc2F5IGlzCiAgcmVjb3ZlcmFibGUgZnJvbSBhdCBsZWFzdCBvbmUgZW50cnkgKG9yIHNldCBvZiBlbnRyaWVzKSwgdGhpcyBpcyBmaW5lIC0tIGRvIG5vdCBmbGFnCiAgaXQgYXMgYSBtZXJnZWQtaWRlbnRpdHkgZXJyb3IuCi0gVGhlIHN0b3JlIGNvbnRhaW5pbmcgbWVtb3JpZXMgYWJvdXQgb3RoZXIgcGVvcGxlLCB3aGV0aGVyIHJlbGF0ZWQgdG8gdGhlIGVudGl0eSBvcgogIHVucmVsYXRlZC4gT25seSB0aGUgZW50aXR5J3Mgb3duIGVzc2F5IG5lZWRzIHRvIGJlIGZhaXRoZnVsbHkgcHJlc2VydmVkOyB0aGUKICBwcmVzZW5jZSBvZiBhZGRpdGlvbmFsIHBlb3BsZSBpcyBhY2NlcHRhYmxlLgotIFRoZSBlc3NheSBiZWluZyBzcGxpdCBhY3Jvc3MgbXVsdGlwbGUgZW50cmllcywgcGFyYXBocmFzZWQsIG9yIGNvbXByZXNzZWQsIGFzIGxvbmcKICBhcyB0aGUgZW50aXR5J3MgaWRlbnRpdHkgYW5kIGtleSBtZW1vcmFibGUgZGV0YWlscyBhcmUgcmVjb3ZlcmFibGUuCgpBIHN0b3JlZCB2ZXJzaW9uIGlzIEZBSVRIRlVMIGlmIHRoZSBlbnRpdHkgaXMgY2xlYXJseSBuYW1lZCAoaW4gYXQgbGVhc3Qgb25lIGVudHJ5KQphbmQgdGhlIGtleSBtZW1vcmFibGUgZGV0YWlscyBmcm9tIHRoZSBlc3NheSBjYW4gYmUgcmVjb3ZlcmVkIGZyb20gdGhlIHJlbGV2YW50Cm1lbW9yeSBvciBtZW1vcmllcyB0aWVkIHRvIHRoYXQgZW50aXR5LgoKWW91IHdpbGwgcmVjZWl2ZSBBTExfTUVNT1JJRVMgKG9uZSBzaGFyZWQgc3RvcmUpIGFuZCBhIGxpc3Qgb2YgRU5UUklFUywgZWFjaAppZGVudGlmaWVkIGJ5IGFuIGludGVnZXIgaWQuIEZvciBlYWNoIGVudHJ5LCByZXR1cm4gb25lIHZlcmRpY3QgaW4gdGhlICJyZXN1bHRzIgphcnJheSwgcHJlc2VydmluZyB0aGUgaWQuCgpSRVRSSUVWQUxfU1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgZWFjaCBlbnRpdHkncyBlc3NheSB3YXMgaW5jbHVkZWQgaW4gdGhlIG1lbW9yaWVzIHJldHJpZXZlZAphbmQgc2hvd24gdG8gYW4gQUkgbW9kZWwgd2hlbiBpdCBhbnN3ZXJlZCBhIHF1ZXN0aW9uLiBFYWNoIGVudHJ5IGhhcyBpdHMgb3duClJFVFJJRVZFRF9NRU1PUklFUyAodGhlIHN1YnNldCBzaG93biBmb3IgdGhhdCBxdWVzdGlvbikuIFByZXNlbmNlIHJlcXVpcmVzIHRoYXQgdGhlCmVudGl0eSBiZSBpZGVudGlmaWFibGUgQU5EIHRoYXQgdGhlIHN1YnN0YW50aXZlIHBlcnNvbmFsIGRldGFpbHMgZnJvbSB0aGUgZXNzYXkgYmUKcmVjb3ZlcmFibGUgZnJvbSB3aGF0IHdhcyBzaG93biB0byB0aGUgbW9kZWwuIFJldHVybiBvbmUgdmVyZGljdCBwZXIgZW50cnkgdW5kZXIKInJlc3VsdHMiLCBwcmVzZXJ2aW5nIHRoZSBpZC4KCklOVk9DQVRJT05fU1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYW4gQUkgbW9kZWwgY29ycmVjdGx5IHVzZWQgYSByZXRyaWV2ZWQgZXNzYXkgdG8gYW5zd2VyIGEKcXVlc3Rpb24gYWJvdXQgdGhlIG5hbWVkIGVudGl0eS4gVGhlIG1vZGVsIHdhcyBzaG93biB0aGUgcmVsZXZhbnQgbWVtb3J5IGFuZCBtdXN0CmFuc3dlciB1c2luZyB0aGUgZW50aXR5LXNwZWNpZmljIGRldGFpbHMgZnJvbSB0aGUgZXNzYXkgKG5vdCBnZW5lcmljIG5vcm1zKS4gUmV0dXJuCm9uZSB2ZXJkaWN0IHBlciBlbnRyeSBpbiAicmVzdWx0cyIsIHByZXNlcnZpbmcgdGhlIGlkLgoKQUJTVEVOVElPTl9TWVNURU06CllvdSBhcmUgY2hlY2tpbmcgd2hldGhlciBhbiBBSSBtb2RlbCBjb3JyZWN0bHkgQUJTVEFJTkVEIHdoZW4gYXNrZWQgYWJvdXQgYSBwZXJzb24KdGhlIG1vZGVsIGhhcyBubyBpbmZvcm1hdGlvbiBvbi4KCkVhY2ggZW50cnkncyBxdWVzdGlvbiBuYW1lcyBhIERJU1RSQUNUT1IgLS0gYSBkaWZmZXJlbnQgcGVyc29uIHdobyBpcyBOT1QgdGhlIGVudGl0eSBpbgp0aGUgcmV0cmlldmVkIGVzc2F5LiBUaGUgY29ycmVjdCBiZWhhdmlvciBpcyB0byBhY2tub3dsZWRnZSBubyBpbmZvcm1hdGlvbiBpcyBhdmFpbGFibGUKYWJvdXQgdGhlIGRpc3RyYWN0b3IgKG9yIHRvIGFzayBmb3IgY2xhcmlmaWNhdGlvbikuIEFwcGx5aW5nIHRoZSBlbnRpdHkncyBlc3NheSBkZXRhaWxzCnRvIHRoZSBkaXN0cmFjdG9yIC0tIGJ5IHRyZWF0aW5nIHRoZW0gYXMgaWYgdGhleSB3ZXJlIHRoZSBlbnRpdHksIG9yIGJ5IHNpbGVudGx5IHVzaW5nCnRoZSBlbnRpdHkncyBzcGVjaWZpY3MgdG8gYW5zd2VyIGFib3V0IHRoZSBkaXN0cmFjdG9yIC0tIGlzIElOQ09SUkVDVC4gTm90ZTogdGhlIG1vZGVsCm1heSBsZWdpdGltYXRlbHkgbWVudGlvbiBpbmZvcm1hdGlvbiBhYm91dCBvdGhlciBwZW9wbGUgc3RvcmVkIGluIG1lbW9yeSBpZiBpdCBkb2VzIG5vdAphdHRyaWJ1dGUgdGhlIGVudGl0eSdzIGVzc2F5IGRldGFpbHMgdG8gdGhlIGRpc3RyYWN0b3IuIFJldHVybiBvbmUgdmVyZGljdCBwZXIgZW50cnkgaW4KInJlc3VsdHMiLCBwcmVzZXJ2aW5nIHRoZSBpZC4=)STORAGE\_SYSTEM:Youarecheckingwhetherpersonalessaysaboutnamedentitiesarestoredinamemorysystem\.Eachessaymayhavebeenparaphrased,compressed,orsplitacrossmultiplememoryentries,buttheentity’sidentity\(name\)andthesubstantivedetailsfromtheessaymuststillberecoverabletocountaspresent\.YouwillreceiveALL\_MEMORIES\(onesharedstore\)andalistofindependentENTRIES,eachidentifiedbyanintegerid\.Foreachentry,returnoneverdictinthe"results"array,preservingtheid\.SUMMARY\_SYSTEM:Youareassessingthequalityofstoredpersonalessays\.Eachessayhasalreadybeenconfirmedtoexistsomewhereinthememorystore\.YourjobistoevaluatewhetherthestoredversionpreservesIDENTITYandKEYDETAILSfaithfully\-\-inawaythatwouldsupportcorrectdownstreamreasoning\.AstoredversionhasaSUMMARYERRORifANYofthefollowingapply:\-Theentity’snamewasdroppedorchanged,sothestoreddetailscannotbetiedbacktothisspecificperson\-Specific,idiosyncraticdetails\(allergies,possessions,rituals,hardconstraints,strongpreferences\)werelostorgeneralizedtothepointthatareadercouldnotrecovertheoriginalmeaning\-Thestoredversionissovagueorgenericthatitwouldnotdistinguishthisentityfromagenericpersonwiththesamerole/hobbyThefollowingareNOTsummaryerrors\-\-treatthemasacceptable:\-Multipledistinctentriesexistingunderthesamename\.Theentitymayshareanamewithotherpeopledescribedinthestore\.Aslongastheentity’sownessayisrecoverablefromatleastoneentry\(orsetofentries\),thisisfine\-\-donotflagitasamerged\-identityerror\.\-Thestorecontainingmemoriesaboutotherpeople,whetherrelatedtotheentityorunrelated\.Onlytheentity’sownessayneedstobefaithfullypreserved;thepresenceofadditionalpeopleisacceptable\.\-Theessaybeingsplitacrossmultipleentries,paraphrased,orcompressed,aslongastheentity’sidentityandkeymemorabledetailsarerecoverable\.AstoredversionisFAITHFULiftheentityisclearlynamed\(inatleastoneentry\)andthekeymemorabledetailsfromtheessaycanberecoveredfromtherelevantmemoryormemoriestiedtothatentity\.YouwillreceiveALL\_MEMORIES\(onesharedstore\)andalistofENTRIES,eachidentifiedbyanintegerid\.Foreachentry,returnoneverdictinthe"results"array,preservingtheid\.RETRIEVAL\_SYSTEM:Youarecheckingwhethereachentity’sessaywasincludedinthememoriesretrievedandshowntoanAImodelwhenitansweredaquestion\.EachentryhasitsownRETRIEVED\_MEMORIES\(thesubsetshownforthatquestion\)\.PresencerequiresthattheentitybeidentifiableANDthatthesubstantivepersonaldetailsfromtheessayberecoverablefromwhatwasshowntothemodel\.Returnoneverdictperentryunder"results",preservingtheid\.INVOCATION\_SYSTEM:YouarecheckingwhetheranAImodelcorrectlyusedaretrievedessaytoansweraquestionaboutthenamedentity\.Themodelwasshowntherelevantmemoryandmustanswerusingtheentity\-specificdetailsfromtheessay\(notgenericnorms\)\.Returnoneverdictperentryin"results",preservingtheid\.ABSTENTION\_SYSTEM:YouarecheckingwhetheranAImodelcorrectlyABSTAINEDwhenaskedaboutapersonthemodelhasnoinformationon\.Eachentry’squestionnamesaDISTRACTOR\-\-adifferentpersonwhoisNOTtheentityintheretrievedessay\.Thecorrectbehavioristoacknowledgenoinformationisavailableaboutthedistractor\(ortoaskforclarification\)\.Applyingtheentity’sessaydetailstothedistractor\-\-bytreatingthemasiftheyweretheentity,orbysilentlyusingtheentity’sspecificstoansweraboutthedistractor\-\-isINCORRECT\.Note:themodelmaylegitimatelymentioninformationaboutotherpeoplestoredinmemoryifitdoesnotattributetheentity’sessaydetailstothedistractor\.Returnoneverdictperentryin"results",preservingtheid\.

Grading:Persona\-RetrievalStage 1 — Storage user template[⬇](data:text/plain;base64,QUxMX01FTU9SSUVTIChjb21wbGV0ZSBtZW1vcnkgc3RvcmUsIHNoYXJlZCBieSBhbGwgZW50cmllcyk6CnthbGxfbWVtb3JpZXNfZm9ybWF0dGVkfQoKRm9yIGVhY2ggZW50cnkgYmVsb3csIGRldGVybWluZSB3aGV0aGVyIHRoZSBlbnRpdHkncyBlc3NheSBpcyBwcmVzZW50IGluCkFMTF9NRU1PUklFUywgZXZlbiBpZiBwYXJhcGhyYXNlZCwgY29tcHJlc3NlZCwgb3Igc3BsaXQgYWNyb3NzIG11bHRpcGxlIGVudHJpZXMuClByZXNlbmNlIHJlcXVpcmVzIHRoYXQgdGhlIGVudGl0eSBiZSBpZGVudGlmaWFibGUgQU5EIHRoYXQgdGhlIHN1YnN0YW50aXZlCnBlcnNvbmFsIGRldGFpbHMgZnJvbSB0aGUgZXNzYXkgYmUgcmVjb3ZlcmFibGUuCgpFTlRSSUVTOgp7ZW50cmllc30KClJldHVybiBvbmUgdmVyZGljdCBwZXIgZW50cnkgdW5kZXIgInJlc3VsdHMiLCBwcmVzZXJ2aW5nIHRoZSBpZCAoMS4ue259KS4KCihFYWNoIGVudHJ5IGJsb2NrIGhhcyB0aGUgZm9ybQogICBbaWQ9a10KICAgRU5USVRZOiB7ZW50aXR5fQogICBFU1NBWToge2Vzc2F5fQop)ALL\_MEMORIES\(completememorystore,sharedbyallentries\):\{all\_memories\_formatted\}Foreachentrybelow,determinewhethertheentity’sessayispresentinALL\_MEMORIES,evenifparaphrased,compressed,orsplitacrossmultipleentries\.PresencerequiresthattheentitybeidentifiableANDthatthesubstantivepersonaldetailsfromtheessayberecoverable\.ENTRIES:\{entries\}Returnoneverdictperentryunder"results",preservingtheid\(1\.\.\{n\}\)\.\(Eachentryblockhastheform\[id=k\]ENTITY:\{entity\}ESSAY:\{essay\}\)

Grading:Persona\-RetrievalStage 2 — Summary user template[⬇](data:text/plain;base64,QUxMX01FTU9SSUVTIChjb21wbGV0ZSBtZW1vcnkgc3RvcmUsIHNoYXJlZCBieSBhbGwgZW50cmllcyk6CnthbGxfbWVtb3JpZXNfZm9ybWF0dGVkfQoKRm9yIGVhY2ggZW50cnkgYmVsb3csIHRoZSBlc3NheSBJUyBjb25maXJtZWQgcHJlc2VudCBzb21ld2hlcmUgaW4gQUxMX01FTU9SSUVTLgpGaW5kIHRoZSBtZW1vcnkgZW50cnkgLS0gb3Igc2V0IG9mIGVudHJpZXMgLS0gdGhhdCBjb3JyZXNwb25kIHRvIHRoZSBlbnRpdHkncyBlc3NheQphbmQgYXNzZXNzIHdoZXRoZXIgdGhlIHN0b3JlZCB2ZXJzaW9uIHByZXNlcnZlcyBpZGVudGl0eSBhbmQgdGhlIGtleSBkZXRhaWxzLCBvcgp3aGV0aGVyIGl0IGhhcyBhIHN1bW1hcnkgZXJyb3IuCgpSZW1pbmRlcjogaXQgaXMgYWNjZXB0YWJsZSBmb3IgdGhlIHN0b3JlIHRvIGNvbnRhaW4gb3RoZXIgZW50cmllcyB1bmRlciB0aGUgc2FtZQpuYW1lIChtdWx0aXBsZSBwZXJzb25hcykgb3IgbWVtb3JpZXMgYWJvdXQgb3RoZXIgcGVvcGxlLCByZWxhdGVkIG9yIG90aGVyd2lzZS4KT25seSBqdWRnZSB3aGV0aGVyIGVhY2ggZW50aXR5J3Mgb3duIGVzc2F5IGlzIGZhaXRoZnVsbHkgcmVjb3ZlcmFibGUgZnJvbSB0aGUKZW50cmllcyB0aGF0IGRlc2NyaWJlIGl0LgoKRU5UUklFUzoKe2VudHJpZXN9CgpSZXR1cm4gb25lIHZlcmRpY3QgcGVyIGVudHJ5IHVuZGVyICJyZXN1bHRzIiwgcHJlc2VydmluZyB0aGUgaWQgKDEuLntufSku)ALL\_MEMORIES\(completememorystore,sharedbyallentries\):\{all\_memories\_formatted\}Foreachentrybelow,theessayISconfirmedpresentsomewhereinALL\_MEMORIES\.Findthememoryentry\-\-orsetofentries\-\-thatcorrespondtotheentity’sessayandassesswhetherthestoredversionpreservesidentityandthekeydetails,orwhetherithasasummaryerror\.Reminder:itisacceptableforthestoretocontainotherentriesunderthesamename\(multiplepersonas\)ormemoriesaboutotherpeople,relatedorotherwise\.Onlyjudgewhethereachentity’sownessayisfaithfullyrecoverablefromtheentriesthatdescribeit\.ENTRIES:\{entries\}Returnoneverdictperentryunder"results",preservingtheid\(1\.\.\{n\}\)\.

Grading:Persona\-RetrievalStage 3 — Retrieval user template[⬇](data:text/plain;base64,Rm9yIGVhY2ggZW50cnksIGRlY2lkZSB3aGV0aGVyIHRoZSBlbnRpdHkncyBlc3NheSB3YXMgaW5jbHVkZWQgaW4gdGhhdCBlbnRyeSdzClJFVFJJRVZFRF9NRU1PUklFUywgZXZlbiBpZiBwYXJhcGhyYXNlZCBvciBzcGxpdCBhY3Jvc3MgbXVsdGlwbGUgZW50cmllcy4gUHJlc2VuY2UKcmVxdWlyZXMgdGhhdCB0aGUgZW50aXR5IGJlIGlkZW50aWZpYWJsZSBBTkQgdGhhdCB0aGUgc3Vic3RhbnRpdmUgcGVyc29uYWwgZGV0YWlscwpmcm9tIHRoZSBlc3NheSBiZSByZWNvdmVyYWJsZSBmcm9tIHdoYXQgd2FzIHNob3duIHRvIHRoZSBtb2RlbC4KCkVOVFJJRVM6CntlbnRyaWVzfQoKUmV0dXJuIG9uZSB2ZXJkaWN0IHBlciBlbnRyeSB1bmRlciAicmVzdWx0cyIsIHByZXNlcnZpbmcgdGhlIGlkICgxLi57bn0pLgoKKEVhY2ggZW50cnkgYmxvY2sgaGFzIHRoZSBmb3JtCiAgIFtpZD1rXQogICBFTlRJVFk6IHtlbnRpdHl9CiAgIFFVRVNUSU9OOiB7cXVlc3Rpb259CiAgIEVTU0FZOiB7ZXNzYXl9CiAgIFJFVFJJRVZFRF9NRU1PUklFUzoKICAge3JldHJpZXZlZF9tZW1vcmllc30KKQ==)Foreachentry,decidewhethertheentity’sessaywasincludedinthatentry’sRETRIEVED\_MEMORIES,evenifparaphrasedorsplitacrossmultipleentries\.PresencerequiresthattheentitybeidentifiableANDthatthesubstantivepersonaldetailsfromtheessayberecoverablefromwhatwasshowntothemodel\.ENTRIES:\{entries\}Returnoneverdictperentryunder"results",preservingtheid\(1\.\.\{n\}\)\.\(Eachentryblockhastheform\[id=k\]ENTITY:\{entity\}QUESTION:\{question\}ESSAY:\{essay\}RETRIEVED\_MEMORIES:\{retrieved\_memories\}\)

Grading:Persona\-RetrievalStage 4 — Invocation user template \(direct queries\)[⬇](data:text/plain;base64,Rm9yIGVhY2ggZW50cnksIGRlY2lkZSB3aGV0aGVyIHRoZSBtb2RlbCBjb3JyZWN0bHkgdXNlZCB0aGUgZXNzYXkncyBzcGVjaWZpYwpkZXRhaWxzIHRvIGFuc3dlciB0aGUgcXVlc3Rpb24uIFRoZSBhbnN3ZXIgbXVzdCBhbGlnbiB3aXRoIHRoZSBHUk9VTkRfVFJVVEggYW5kCnJlZmxlY3QgdGhlIGVzc2F5J3Mgc3BlY2lmaWNzLCBub3QgZ2VuZXJpYyBpbmZvcm1hdGlvbi4KCkVOVFJJRVM6CntlbnRyaWVzfQoKUmV0dXJuIG9uZSB2ZXJkaWN0IHBlciBlbnRyeSB1bmRlciAicmVzdWx0cyIsIHByZXNlcnZpbmcgdGhlIGlkICgxLi57bn0pLgoKKEVhY2ggZW50cnkgYmxvY2sgaGFzIHRoZSBmb3JtCiAgIFtpZD1rXQogICBFTlRJVFk6IHtlbnRpdHl9CiAgIFFVRVNUSU9OOiB7cXVlc3Rpb259CiAgIEVTU0FZOiB7ZXNzYXl9CiAgIEdST1VORF9UUlVUSDoge2dyb3VuZF90cnV0aF9hbnN3ZXJ9CiAgIE1PREVMX1JFU1BPTlNFOiB7bGxtX3Jlc3BvbnNlfQop)Foreachentry,decidewhetherthemodelcorrectlyusedtheessay’sspecificdetailstoanswerthequestion\.TheanswermustalignwiththeGROUND\_TRUTHandreflecttheessay’sspecifics,notgenericinformation\.ENTRIES:\{entries\}Returnoneverdictperentryunder"results",preservingtheid\(1\.\.\{n\}\)\.\(Eachentryblockhastheform\[id=k\]ENTITY:\{entity\}QUESTION:\{question\}ESSAY:\{essay\}GROUND\_TRUTH:\{ground\_truth\_answer\}MODEL\_RESPONSE:\{llm\_response\}\)

Grading:Persona\-RetrievalStage 4 — Abstention user template \(misleading queries\)[⬇](data:text/plain;base64,Rm9yIGVhY2ggZW50cnksIGRlY2lkZSB3aGV0aGVyIHRoZSBtb2RlbCBjb3JyZWN0bHkgYWJzdGFpbmVkIC0tIGFja25vd2xlZGdlZCBpdCBoYXMKbm8gaW5mb3JtYXRpb24gYWJvdXQgdGhlIERJU1RSQUNUT1IgLS0gcmF0aGVyIHRoYW4gYXBwbHlpbmcgdGhlIEVOVElUWSdzIGVzc2F5CmRldGFpbHMgdG8gdGhlIERJU1RSQUNUT1IuIFRoZSBtb2RlbCBhYnN0YWlucyBjb3JyZWN0bHkgZXZlbiBpZiBpdCBnZW5lcmFsbHkKYW5zd2VycyB1c2luZyBjb21tb24ga25vd2xlZGdlLCBhcyBsb25nIGFzIGl0IGRvZXMgTk9UIHNtdWdnbGUgaW4KRU5USVRZLXNwZWNpZmljIGRldGFpbHMuIENvbmZsYXRpbmcgdGhlIHR3byBwZW9wbGUsIG9yIHVzaW5nIHRoZSBlc3NheSBhcyBpZiBpdAp3ZXJlIGFib3V0IHRoZSBkaXN0cmFjdG9yLCBpcyBhbiBhYnN0ZW50aW9uIGZhaWx1cmUuCgpFTlRSSUVTOgp7ZW50cmllc30KClJldHVybiBvbmUgdmVyZGljdCBwZXIgZW50cnkgdW5kZXIgInJlc3VsdHMiLCBwcmVzZXJ2aW5nIHRoZSBpZCAoMS4ue259KS4KCihFYWNoIGVudHJ5IGJsb2NrIGhhcyB0aGUgZm9ybQogICBbaWQ9a10KICAgRU5USVRZIChlc3NheSBpcyBhYm91dCk6IHtlbnRpdHl9CiAgIERJU1RSQUNUT1IgKG5hbWVkIGluIHF1ZXN0aW9uLCBOT1QgdGhlIGVudGl0eSk6IHtkaXN0cmFjdG9yfQogICBFU1NBWSAoYWJvdXQgdGhlIGVudGl0eSk6IHtlc3NheX0KICAgUVVFU1RJT04gKGFib3V0IHRoZSBkaXN0cmFjdG9yKToge3F1ZXN0aW9ufQogICBNT0RFTF9SRVNQT05TRToge2xsbV9yZXNwb25zZX0KKQ==)Foreachentry,decidewhetherthemodelcorrectlyabstained\-\-acknowledgedithasnoinformationabouttheDISTRACTOR\-\-ratherthanapplyingtheENTITY’sessaydetailstotheDISTRACTOR\.Themodelabstainscorrectlyevenifitgenerallyanswersusingcommonknowledge,aslongasitdoesNOTsmuggleinENTITY\-specificdetails\.Conflatingthetwopeople,orusingtheessayasifitwereaboutthedistractor,isanabstentionfailure\.ENTRIES:\{entries\}Returnoneverdictperentryunder"results",preservingtheid\(1\.\.\{n\}\)\.\(Eachentryblockhastheform\[id=k\]ENTITY\(essayisabout\):\{entity\}DISTRACTOR\(namedinquestion,NOTtheentity\):\{distractor\}ESSAY\(abouttheentity\):\{essay\}QUESTION\(aboutthedistractor\):\{question\}MODEL\_RESPONSE:\{llm\_response\}\)

#### D\.3\.4Long\-Hop grading

Long\-Hopgrading runs three stages*per supporting fact*\(storage, summary, retrieval\) and then determines invocation correctness by deterministic letter parsing of the model’s MCQ answer rather than via an LLM\-judge call\. Memory systems may legitimately merge several chain\-links into a single memory entry; each per\-fact stage explicitly accepts merged entries as long as the specific link is still recoverable\.

Grading:Long\-HopStage 1 — Per\-fact storage check[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYSBzcGVjaWZpYyBmYWN0dWFsIHN0YXRlbWVudCAob25lIGxpbmsgaW4gYSBtdWx0aS1ob3AgcmVhc29uaW5nIGNoYWluKSBpcyBwcmVzZXJ2ZWQgaW4gYSBtZW1vcnkgc3RvcmUuCgpUaGUgbWVtb3J5IHN0b3JlIG1heSBoYXZlIHN0b3JlZCB0aGUgZmFjdCB2ZXJiYXRpbSwgcGFyYXBocmFzZWQgaXQsIE9SIG1lcmdlZApzZXZlcmFsIGNoYWluLWxpbmtzIHRvZ2V0aGVyIGludG8gb25lIGNvbWJpbmVkIG1lbW9yeS4gQW55IG9mIHRoZXNlIGZvcm1zCmNvdW50cyBhcyBTVE9SRUQgLS0gYXMgbG9uZyBhcyB0aGUgU1BFQ0lGSUMgbGluayBhc3NlcnRlZCBieSB0aGUgdGFyZ2V0IGZhY3QKY2FuIHN0aWxsIGJlIHVuYW1iaWd1b3VzbHkgaWRlbnRpZmllZCBmcm9tIGF0IGxlYXN0IG9uZSBtZW1vcnkgZW50cnkuIElmIHRoZQpsaW5rJ3MgdHdvIHNwZWNpZmljIGVudGl0aWVzIGFuZCB0aGUgcmVsYXRpb24gYmV0d2VlbiB0aGVtIGFyZSByZWNvdmVyYWJsZSwKbWFyayBmYWN0X2luX3N0b3JlPXRydWUuCgpVU0VSOgpUQVJHRVQgRkFDVCAob25lIGxpbmsgb2YgYSByZWFzb25pbmcgY2hhaW4pOgp7dGFyZ2V0X21lc3NhZ2V9CgpBTExfTUVNT1JJRVMgKGNvbXBsZXRlIG1lbW9yeSBzdG9yZSwgcG9zc2libHkgd2l0aCBtZXJnZWQgZW50cmllcyk6CnthbGxfbWVtb3JpZXNfZm9ybWF0dGVkfQoKSXMgdGhlIHRhcmdldCBmYWN0IHByZXNlbnQgaW4gQUxMX01FTU9SSUVTIC0tIHZlcmJhdGltLCBwYXJhcGhyYXNlZCwgb3IgYXMgcGFydApvZiBhIG1lcmdlZCBtZW1vcnkgZW50cnkgdGhhdCBzdGlsbCBwcmVzZXJ2ZXMgdGhlIHNwZWNpZmljIGxpbmsgYmV0d2VlbiB0aGUKdGFyZ2V0IGZhY3QncyB0d28gZW50aXRpZXM/IE1hcmsgZmFjdF9pbl9zdG9yZT10cnVlIG9ubHkgaWYgdGhlIHByZWNpc2UKcmVsYXRpb25zaGlwIGJldHdlZW4gdGhlIHR3byBzcGVjaWZpYyBlbnRpdGllcyBpcyB1bmFtYmlndW91c2x5IHJlY292ZXJhYmxlLg==)SYSTEM:Youarecheckingwhetheraspecificfactualstatement\(onelinkinamulti\-hopreasoningchain\)ispreservedinamemorystore\.Thememorystoremayhavestoredthefactverbatim,paraphrasedit,ORmergedseveralchain\-linkstogetherintoonecombinedmemory\.AnyoftheseformscountsasSTORED\-\-aslongastheSPECIFIClinkassertedbythetargetfactcanstillbeunambiguouslyidentifiedfromatleastonememoryentry\.Ifthelink’stwospecificentitiesandtherelationbetweenthemarerecoverable,markfact\_in\_store=true\.USER:TARGETFACT\(onelinkofareasoningchain\):\{target\_message\}ALL\_MEMORIES\(completememorystore,possiblywithmergedentries\):\{all\_memories\_formatted\}IsthetargetfactpresentinALL\_MEMORIES\-\-verbatim,paraphrased,oraspartofamergedmemoryentrythatstillpreservesthespecificlinkbetweenthetargetfact’stwoentities?Markfact\_in\_store=trueonlyifthepreciserelationshipbetweenthetwospecificentitiesisunambiguouslyrecoverable\.

Grading:Long\-HopStage 2 — Per\-fact summary check[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGFzc2Vzc2luZyB0aGUgcXVhbGl0eSBvZiBhIHN0b3JlZCBjaGFpbi1saW5rLgoKVGhlIGxpbmsgaGFzIGFscmVhZHkgYmVlbiBjb25maXJtZWQgdG8gZXhpc3Qgc29tZXdoZXJlIGluIHRoZSBtZW1vcnkgc3RvcmUKKHBvc3NpYmx5IGluc2lkZSBhIG1lcmdlZCBtZW1vcnkgZW50cnkpLiBZb3VyIGpvYiBpcyB0byBldmFsdWF0ZSB3aGV0aGVyIHRoZQpzdG9yZWQgdmVyc2lvbiBmYWl0aGZ1bGx5IHByZXNlcnZlcyB0aGUgc3BlY2lmaWMgbGluayBpbiBhIHdheSB0aGF0IHdvdWxkCnN1cHBvcnQgY29ycmVjdCBkb3duc3RyZWFtIGNoYWluIHJlYXNvbmluZy4KCkEgc3RvcmVkIHZlcnNpb24gaGFzIGEgU1VNTUFSWSBFUlJPUiBpZiBBTlkgb2YgdGhlIGZvbGxvd2luZyBhcHBseToKLSBUaGUgbGluaydzIHR3byBlbnRpdGllcyB3ZXJlIGNvbGxhcHNlZC9yZW5hbWVkL3N3YXBwZWQsIGJyZWFraW5nIGlkZW50aXR5LgotIFRoZSByZWxhdGlvbiBiZXR3ZWVuIHRoZW0gd2FzIGNvcnJ1cHRlZCwgd2Vha2VuZWQsIG9yIHJlcGxhY2VkLgotIFRoZSBsaW5rIHdhcyBvdmVyLW1lcmdlZCB3aXRoIHVucmVsYXRlZCBmYWN0cyBzbyB0aGF0IHRoZSBzcGVjaWZpYyBsaW5rIGNhbgogIG5vIGxvbmdlciBiZSBjbGVhbmx5IGV4dHJhY3RlZCAoZS5nLiB0aGUgZW50aXRpZXMgYXJlIGxpc3RlZCBidXQgbm90IGluIGEKICB3YXkgdGhhdCBwcmVzZXJ2ZXMgd2hpY2gtcmVsYXRlcy10by13aGljaCkuCi0gQSBjcml0aWNhbCBkZXRhaWwgKGUuZy4gZGlyZWN0aW9uIG9mIHRoZSByZWxhdGlvbikgd2FzIGxvc3QuCgpBIHN0b3JlZCB2ZXJzaW9uIGlzIEZBSVRIRlVMIGlmIGJvdGggZW50aXRpZXMgb2YgdGhlIGxpbmsgYXJlIGNsZWFybHkgbmFtZWQgaW4Kc29tZSBtZW1vcnkgZW50cnkgYW5kIHRoZSBzcGVjaWZpYyByZWxhdGlvbiBiZXR3ZWVuIHRoZW0gaXMgdW5hbWJpZ3VvdXMgLS0KRVZFTiBJRiB0aGUgbWVtb3J5IGVudHJ5IGFsc28gY29udGFpbnMgb3RoZXIgY2hhaW4tbGlua3MgZnJvbSB0aGUgc2FtZSBjaGFpbgoobWVyZ2luZyBpcyBhbGxvd2VkIHdoZW4gZWFjaCBpbmRpdmlkdWFsIGxpbmsgaXMgc3RpbGwgcmVjb3ZlcmFibGUpLgoKVVNFUjoKVEFSR0VUIEZBQ1QgKG9uZSBsaW5rIG9mIGEgcmVhc29uaW5nIGNoYWluKToKe3RhcmdldF9tZXNzYWdlfQoKQUxMX01FTU9SSUVTICh0aGUgbGluayBJUyBjb25maXJtZWQgcHJlc2VudCBzb21ld2hlcmUgaW4gaGVyZSwgcG9zc2libHkgbWVyZ2VkKToKe2FsbF9tZW1vcmllc19mb3JtYXR0ZWR9CgpGaW5kIHRoZSBtZW1vcnkgZW50cnkgKG9yIGVudHJpZXMpIHRoYXQgY292ZXIgdGhpcyBsaW5rIGFuZCBhc3Nlc3Mgd2hldGhlciB0aGUKc3RvcmVkIHZlcnNpb24gZmFpdGhmdWxseSBwcmVzZXJ2ZXMgdGhlIHNwZWNpZmljIHJlbGF0aW9uIGJldHdlZW4gdGhlIHR3bwplbnRpdGllcywgb3Igd2hldGhlciBpdCBoYXMgYSBzdW1tYXJ5IGVycm9yLg==)SYSTEM:Youareassessingthequalityofastoredchain\-link\.Thelinkhasalreadybeenconfirmedtoexistsomewhereinthememorystore\(possiblyinsideamergedmemoryentry\)\.Yourjobistoevaluatewhetherthestoredversionfaithfullypreservesthespecificlinkinawaythatwouldsupportcorrectdownstreamchainreasoning\.AstoredversionhasaSUMMARYERRORifANYofthefollowingapply:\-Thelink’stwoentitieswerecollapsed/renamed/swapped,breakingidentity\.\-Therelationbetweenthemwascorrupted,weakened,orreplaced\.\-Thelinkwasover\-mergedwithunrelatedfactssothatthespecificlinkcannolongerbecleanlyextracted\(e\.g\.theentitiesarelistedbutnotinawaythatpreserveswhich\-relates\-to\-which\)\.\-Acriticaldetail\(e\.g\.directionoftherelation\)waslost\.AstoredversionisFAITHFULifbothentitiesofthelinkareclearlynamedinsomememoryentryandthespecificrelationbetweenthemisunambiguous\-\-EVENIFthememoryentryalsocontainsotherchain\-linksfromthesamechain\(mergingisallowedwheneachindividuallinkisstillrecoverable\)\.USER:TARGETFACT\(onelinkofareasoningchain\):\{target\_message\}ALL\_MEMORIES\(thelinkISconfirmedpresentsomewhereinhere,possiblymerged\):\{all\_memories\_formatted\}Findthememoryentry\(orentries\)thatcoverthislinkandassesswhetherthestoredversionfaithfullypreservesthespecificrelationbetweenthetwoentities,orwhetherithasasummaryerror\.

Grading:Long\-HopStage 3 — Per\-fact retrieval check[⬇](data:text/plain;base64,U1lTVEVNOgpZb3UgYXJlIGNoZWNraW5nIHdoZXRoZXIgYSBzcGVjaWZpYyBjaGFpbi1saW5rIHdhcyBpbmNsdWRlZCBpbiB0aGUgbWVtb3JpZXMKcmV0cmlldmVkIGFuZCBzaG93biB0byBhbiBBSSBtb2RlbCB3aGVuIGl0IGFuc3dlcmVkIGEgbXVsdGktaG9wIHF1ZXN0aW9uLgoKVGhlIHJldHJpZXZlZCBtZW1vcmllcyBtYXkgaW5jbHVkZSB2ZXJiYXRpbSwgcGFyYXBocmFzZWQsIG9yIG1lcmdlZCB2ZXJzaW9ucy4KQSBtZXJnZWQgbWVtb3J5IHRoYXQgc3RpbGwgY29udGFpbnMgdGhlIGxpbmsncyBzcGVjaWZpYyByZWxhdGlvbiBjb3VudHMgYXMKInJldHJpZXZlZCIuCgpVU0VSOgpUQVJHRVQgRkFDVCAob25lIGxpbmsgb2YgYSByZWFzb25pbmcgY2hhaW4pOgp7dGFyZ2V0X21lc3NhZ2V9CgpPUklHSU5BTCBRVUVTVElPTiAodXNlZCB0byBxdWVyeSB0aGUgbWVtb3J5IHN0b3JlKToKe3F1ZXJ5fQoKUkVUUklFVkVEX01FTU9SSUVTIChzaG93biB0byB0aGUgbW9kZWwgd2hlbiBpdCBhbnN3ZXJlZCk6CntyZXRyaWV2ZWRfbWVtb3JpZXN9CgpXYXMgdGhlIHRhcmdldCBmYWN0J3Mgc3BlY2lmaWMgbGluayBpbmNsdWRlZCBpbiBSRVRSSUVWRURfTUVNT1JJRVMsIGV2ZW4gaWYKcGFyYXBocmFzZWQgb3IgbWVyZ2VkIHdpdGggb3RoZXIgY2hhaW4tbGlua3M/IE1hcmsgZmFjdF9pbl9yZXRyaWV2ZWQ9dHJ1ZSBvbmx5CmlmIHRoZSBwcmVjaXNlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZSBsaW5rJ3MgdHdvIGVudGl0aWVzIGlzIHVuYW1iaWd1b3VzbHkKcmVjb3ZlcmFibGUgZnJvbSB0aGUgcmV0cmlldmVkIG1lbW9yaWVzLg==)SYSTEM:Youarecheckingwhetheraspecificchain\-linkwasincludedinthememoriesretrievedandshowntoanAImodelwhenitansweredamulti\-hopquestion\.Theretrievedmemoriesmayincludeverbatim,paraphrased,ormergedversions\.Amergedmemorythatstillcontainsthelink’sspecificrelationcountsas"retrieved"\.USER:TARGETFACT\(onelinkofareasoningchain\):\{target\_message\}ORIGINALQUESTION\(usedtoquerythememorystore\):\{query\}RETRIEVED\_MEMORIES\(showntothemodelwhenitanswered\):\{retrieved\_memories\}Wasthetargetfact’sspecificlinkincludedinRETRIEVED\_MEMORIES,evenifparaphrasedormergedwithotherchain\-links?Markfact\_in\_retrieved=trueonlyifthepreciserelationshipbetweenthelink’stwoentitiesisunambiguouslyrecoverablefromtheretrievedmemories\.

##### Invocation \(deterministic, no LLM call\)\.

After the three per\-fact stages succeed for every fact in a chain, the invocation outcome is read directly from the model’s MCQ response\. The parser tries, in order: \(i\) parsing the entire response as a JSON object and reading`selected\_choice`/`answer`/`choice`; \(ii\) hunting for any embedded JSON object inside the response and trying the same keys; \(iii\) falling back to the last lone uppercase letterA–Ethat appears in the response\. The parsed letter is compared againstcorrect\_choicefrom the dataset row to decidecorrectvs\.reasoning\_error\.