IRC-Bench: Recognizing Entities from Contextual Cues in First-Person Reminiscences
Summary
This paper introduces IRC-Bench, a benchmark for recognizing implicit entities in first-person reminiscences using contextual cues rather than explicit mentions. It evaluates various LLM and retrieval configurations, finding QLoRA-adapted Llama 3.1 8B to be the top performer in open-world settings.
View Cached Full Text
Cached at: 05/08/26, 07:16 AM
# IRC-Bench: Recognizing Entities from Contextual Cues in First-Person Reminiscences Source: [https://arxiv.org/abs/2605.06142](https://arxiv.org/abs/2605.06142) [View PDF](https://arxiv.org/pdf/2605.06142) > Abstract:When people recount personal memories, they often refer to people, places, and events indirectly, relying on contextual cues rather than explicit names\. Such implicit references are central to reminiscence narratives: first\-person accounts of lived experience used in therapeutic, archival, and social settings\. They pose a difficult computational problem because the intended entity must be inferred from dispersed narrative evidence rather than from a local mention\. We introduce IRC\-Bench, the Implicit Reminiscence Context Benchmark, for evaluating implicit entity recognition in reminiscence transcripts\. The benchmark targets non\-locality: entity\-identifying cues are distributed across multiple, non\-contiguous clauses, unlike named entity recognition, entity linking, or coreference resolution\. IRC\-Bench comprises 25,136 samples constructed from 12,337 Wiki\-data\-linked entities across 1,994 transcripts spanning 11 thematic domains\. Each sample pairs an Entity\-Grounded Narrative, in which the target entity is explicitly mentioned, with an Entity\-Elided Narrative, in which direct mentions are removed\. We evaluate 19 configurations across LLM generation, dense retrieval, RAG, and fine\-tuning\. QLoRA\-adapted Llama 3\.1 8B performs best in the open\-world setting \(38\.94% exact match; 51\.59% Jaccard\), while fine\-tuned DPR leads closed\-world retrieval \(35\.38% Hit@1; 71\.49% Hit@10\)\. We release IRC\-Bench with data, code, and evaluation tools\. ## Submission history From: Yehudit Aperstein \[[view email](https://arxiv.org/show-email/806637db/2605.06142)\] **\[v1\]**Thu, 7 May 2026 12:39:49 UTC \(1,211 KB\)
Similar Articles
When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents
This paper introduces LoCoMo-Conv, a conversational memory benchmark that evaluates memory retrieval and response quality in long-horizon conversational agents using different query styles, revealing gaps in existing QA benchmarks and suggesting reasoning-based memory elaboration as a promising direction.
IFCMemoryBench: Evaluating Long-Term Memory of LLM-Based Agents in BIM Information Retrieval
This paper introduces IFCMemoryBench, a human-validated benchmark for evaluating long-term memory in LLM-based agents for BIM information retrieval. It shows that current memory systems achieve only 32.4% answer accuracy, revealing a domain-transfer gap in agent memory.
RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts
Introduces RECON, a benchmark for evaluating compositional reasoning over long contexts in LLM-based agents, spanning 24 case files across criminal, medical, and financial domains. The best non-oracle system achieves only 22.4% accuracy, revealing substantial limitations in current memory architectures.
LabyrinthBench: a local-focused, judge-free LLM benchmark that measures context recall under interference for multi-step agentic tasks.
LabyrinthBench is a local, judge-free LLM benchmark that deterministically measures context recall under interference for multi-step agentic tasks. Initial experiments show that wiping context and re-injecting gate answers helped 7 of 9 models but backfired on 2, highlighting the complexity of context management strategies.
RealICU: Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation
RealICU is a hindsight-annotated benchmark for evaluating LLMs in ICU settings, covering four physician-motivated tasks. Experiments reveal that existing LLMs struggle with recall-safety tradeoffs and anchoring bias, while a new structured-memory agent improves reasoning but not fully eliminate safety failures.