Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data

# Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data
Source: [https://arxiv.org/html/2607.27056](https://arxiv.org/html/2607.27056)
Lingyang Zeng1, Guangze Chen1, Kaichen Yu1, Zhicheng Pan2, Siyang Weng1, Zirui Hu1,Xiangyun Du1, Hailin He1, Rong Zhang1, Chengcheng Yang1, Kai Huang1, Xuan Zhou1

###### Abstract

Personalized agents are increasingly applied to assist users across a wide range of tasks\. Effective personalized assistance requires not only retrieving explicit facts from past interactions stored in agent memory, but also inferring abstract personal characteristics\. However, existing memory benchmarks primarily evaluate whether an agent can retrieve information explicitly stated in conversational histories, failing to provide an effective assessment of deeper user understanding\. In this work, we proposeSetoka, a benchmark for evaluating memory\-augmented personalized agents with hierarchical user understanding from heterogeneous data\. Grounded in theories from cognitive and personality psychology,Setokadefines four levels of user understanding, i\.e\.,semantic memory,episodic memory,behavior pattern, andpersonality trait\. Moreover, to enable realistic yet privacy\-preserving evaluation, we design a psychometrics\-based pipeline that synthesizes diverse, coherent heterogeneous user data and queries at scale\. Finally, we leverageSetokato evaluate 3 language models combined with 5 memory systems for 10 synthetic users\. Our comprehensive evaluation reveals that while existing systems perform well on semantic memory retrieval, their performance declines on episodic memory\. Moreover, when dealing with behavior pattern and personality trait understanding tasks that require integrating heterogeneous and fragmented information dispersed over time, performance declines even further\. These findings demonstrate that user understanding cannot be handled by simple fact retrieval, motivating the design of memory mechanisms for cross\-source integration and abstraction over long\-term user behavior\.

## 1Introduction

Large language model \(LLM\)\-based agents are increasingly deployed as personalized assistants\(Li et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib16)\)that understand user instructions and autonomously execute them\. The great advantage of such agents stems from their ability to handle tasks with varying degrees of user understanding\. Generally, some tasks require only retrieving an explicit fact \(e\.g\., a scheduled meeting time\), while more complicated tasks demand a nuanced comprehension that captures the user’s behavioral patterns and personality traits\. However, such abstract characteristics are rarely stated explicitly\. Instead, they are distributed across heterogeneous data sources, including unstructured text \(e\.g\., messages and notes\), structured records \(e\.g\., contacts and calendars\), and graphs \(e\.g\., social connections\)\(Hu et al\.[2026](https://arxiv.org/html/2607.27056#bib.bib11)\)\. For instance, the personality trait “extraversion” cannot be inferred from any single source alone\. It requires jointly considering frequent group events in the calendar, active engagement in group chats, and numerous ties in the social graph\. This distributed nature makes straightforward retrieval\-based approaches ineffective, as no individual data source provides a comprehensive understanding of the user\.

Recent years have witnessed rapid advances in memory systems for agents\(Chhikara et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib2)\)\. Benchmarks for memory\-augmented agents serve as a critical touchstone for measuring these advances, revealing potential limitations and guiding the development of more capable memory systems\. However, existing benchmarks\(Maharana et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib18); Wu et al\.[2025a](https://arxiv.org/html/2607.27056#bib.bib31); Du et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib5); Tan et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib26); Jiang et al\.[2025a](https://arxiv.org/html/2607.27056#bib.bib13),[b](https://arxiv.org/html/2607.27056#bib.bib14); Wu et al\.[2026](https://arxiv.org/html/2607.27056#bib.bib32)\)mainly assess the retrieval accuracy of facts explicitly stated in conversational histories\. Consequently, they provide insufficient insight into whether memory systems could integrate implicit evidence across heterogeneous data sources to infer abstract personal characteristics\.

However, building such a benchmark is non\-trivial due to two primary reasons\. First, collecting heterogeneous data from real\-world users raises privacy concerns, and acquiring data at the scale required for systematic evaluation is also prohibitively costly\. Second, evaluating deep user understanding requires grounding answers in a coherent profile, where high\-level traits are substantiated by underlying records\. For clarity, the specific challenges are summarized as follows\.

Hierarchical User Understanding \(C1\)\. User understanding requires inherently different reasoning operations, ranging from retrieving an explicit fact to inferring a personality trait\. Constructing such a hierarchy is non\-trivial, as it requires both distinct reasoning scopes and a seamless transition from specific records to abstract characteristics\.

Realistic Persona Sampling \(C2\)\. A realistic persona should resemble a coherent individual rather than a random bundle of personality traits\. In real\-world populations, personality traits do not occur in isolation\. That is, some trait combinations are common, whereas others are rarely observed\. Directly prompting an LLM to construct personas or sampling each personality trait individually might produce trait combinations that are unlikely to occur in the real world\.

Diverse Trait\-Behavior Mapping \(C3\)\. The same personality trait can lead to diverse behavior patterns across different event categories\. However, existing data synthesis methods that rely on persona\-conditioned LLM generation might map a personality trait profile to a narrow set of stereotypical behaviors\(Cheng, Durmus, and Jurafsky[2023](https://arxiv.org/html/2607.27056#bib.bib1); Liu, Diab, and Fried[2024](https://arxiv.org/html/2607.27056#bib.bib17)\), overlooking the inter\-individual diversity documented in personality psychology\.

Scalable and Consistent Generation \(C4\)\. Mapping behavior patterns into long\-term, heterogeneous user data introduces two coupled problems\. First, generation based solely on an LLM is difficult to scale to long histories, as earlier behavioral constraints might be forgotten or violated as the context grows\. Second, independently generating heterogeneous records might introduce contradictions between data items that describe the same underlying event\.

To address these challenges, we proposeSetoka, a dedicated benchmark that enables the comprehensive evaluation of hierarchical uSer understanding for personalized task\-orientedagents\. Specifically, to addressC1, we propose a*four\-level framework of user understanding*, grounded in cognitive and personality psychology, that comprisessemantic memory\(SM\),episodic memory\(EM\),behavior patterns\(BP\), andpersonality traits\(PT\)\. These four levels span a continuum from concretepersonal memoriesto abstractpersonal characteristics\. ForC2, we propose a*correlation\-aware personality trait sampling*strategy\. That is, the personality traits are jointly sampled from a multivariate Gaussian distribution whose covariance matrix is derived from meta\-analytic psychometric correlations\. By preserving empirically observed correlations among personality dimensions, this approach produces more coherent and psychologically grounded personality trait profiles\. ForC3, we propose*psychological\-scale\-guided behavior pattern generation*\. The sampled personality traits are mapped to behavior patterns through validated psychological scales\(Soto and John[2017](https://arxiv.org/html/2607.27056#bib.bib24)\), which provide fine\-grained behavior patterns grounded in established psychological constructs rather than LLM\-only generation\. ForC4, we propose an*event\-grounded generation tree*that organizes behavioral patterns and contexts into concrete events\. Under each parent node, child events are generated sequentially, each conditioned on its preceding siblings\. Then, their details are expanded independently in parallel without retaining the full event history\. All heterogeneous records are derived from their underlying events to maintain cross\-record consistency\.

In summary, we make the following contributions:

- •We identify a fundamental limitation of existing benchmarks: they do not adequately evaluate the multiple levels of user understanding required by personalized agents\.
- •To the best of our knowledge, we introduce the first psychology\-grounded framework that evaluates hierarchical user understanding, spanning concrete personal memories and abstract personal characteristics\. By explicitly defining the evidence scope and reasoning operation required at each level, the framework enables different levels of user understanding to be evaluated independently\.
- •We develop a psychometrics\-based generation pipeline that samples coherent user profiles and generates long\-term heterogeneous records from them at scale\. This design provides reference answers not only for concrete personal memories, but also for abstract personal characteristics, enabling level\-specific queries with traceable evidence\.
- •We instantiateSetokawith 10 synthetic user profiles and 23 schemas over 3 data models\. Then, we conduct a comprehensive evaluation of 3 language models paired with 5 popular memory systems\. The results demonstrate that simple fact retrieval is insufficient, motivating that memory mechanisms should integrate cross\-source evidence and abstract long\-term user behavior\.

## 2Related Work

BenchmarkMemory OriginSemanticMemoryEpisodicMemoryBehaviorPatternPersonalityTrait\#DM\#Sch\.LoCoMo\(Maharana et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib18)\)Dialogue✓✓××11LongMemEval\(Wu et al\.[2025a](https://arxiv.org/html/2607.27056#bib.bib31)\)Dialogue✓✓××11PerLTQA\(Du et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib5)\)Profiles, Events, Dialogues✓✓××14MemBench\(Tan et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib26)\)Dialogue✓××11PersonaMem\(Jiang et al\.[2025a](https://arxiv.org/html/2607.27056#bib.bib13)\)Dialogue✓×××11PersonaMem\-v2\(Jiang et al\.[2025b](https://arxiv.org/html/2607.27056#bib.bib14)\)Dialogue✓××11KnowMe\-Bench\(Wu et al\.[2026](https://arxiv.org/html/2607.27056#bib.bib32)\)Autobiographical Narratives✓✓×11SetokaHeterogeneous Data✓✓✓✓323Table 1:Comparison ofSetokawith representative personalized memory benchmarks\. \#DM denotes the number of data models exposed to the agent, and \#Sch\. denotes the number of distinct data schemas; counts greater than one indicate heterogeneous data\. Explicitly stated behavior patterns or personality traits in natural language such as "I usually hike on weekends" are categorized as SM because they can be directly retrieved without being inferred from underlying user data\.denotes partial coverage\.### 2\.1Memory Systems

As personalized assistants, agents should recall user\-specific context from past interaction histories\(Li et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib16); Hu et al\.[2026](https://arxiv.org/html/2607.27056#bib.bib11)\)\. Recent memory\-augmented agents employ diverse mechanisms to retain and use information beyond the context window\. MemGPT\(Packer et al\.[2023](https://arxiv.org/html/2607.27056#bib.bib22)\)manages limited context by moving information between working memory and external storage\. Mem0\(Chhikara et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib2)\)compresses conversations into a compact, editable store of user facts, revising or removing older facts as new information arrives\. Cognee and HippoRAG 2\(Markovic et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib19); Gutiérrez et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib9)\)organize memories in graph structures to support relational and multi\-hop retrieval\. MemMachine\(Wang et al\.[2026](https://arxiv.org/html/2607.27056#bib.bib30)\)retrieves a matched memory together with nearby memories in time, providing more complete context\. However, memory\-augmented personalized agents are still far from understanding users in depth, and therefore comprehensive evaluation is essential for further progress\.Setokafills this critical gap by providing a unified benchmark for evaluating hierarchical user understanding over heterogeneous data\.

### 2\.2Memory Benchmarks

Existing memory benchmarks primarily evaluate whether models can retain information from conversational histories or textual user\-profile records\. Specifically, LongMemEval and LoCoMo formulate memory as the retention of and reasoning over facts grounded in long conversational histories\(Wu et al\.[2025a](https://arxiv.org/html/2607.27056#bib.bib31); Maharana et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib18)\)\. PersonaMem assesses whether models can infer evolving user profiles from multi\-session histories and select responses aligned with users’ current states\(Jiang et al\.[2025a](https://arxiv.org/html/2607.27056#bib.bib13)\)\. PerLTQA focuses on question answering over semantic and episodic personal memories, including profiles, social relationships, events, and dialogues, represented within a JSON data model\(Du et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib5)\)\.

More recent benchmarks move beyond explicitly stated facts\. Specifically, MemBench introduces reflective memory, which requires models to synthesize interaction histories to infer users’ preferences and emotions\(Tan et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib26)\)\. PersonaMem\-v2 requires inferring implicitly revealed preferences from conversational nuance\(Jiang et al\.[2025b](https://arxiv.org/html/2607.27056#bib.bib14)\)\. KnowMe\-Bench defines levels of person understanding ranging from factual recall to principle\-level reasoning over long autobiographical narratives\(Wu et al\.[2026](https://arxiv.org/html/2607.27056#bib.bib32)\)\. Collectively, these benchmarks primarily evaluate limited user understanding ability over a homogeneous interaction stream\. Table[1](https://arxiv.org/html/2607.27056#S2.T1)summarizes howSetokadiffers from current memory benchmarks: it is the only benchmark that evaluates an agent’s user understanding ability across heterogeneous data models rather than a single dialogue or profile source, and the only evaluation suite that covers all four levels of understanding\.

## 3User Understanding Modeling

![Refer to caption](https://arxiv.org/html/2607.27056v1/x1.png)Figure 1:Overview ofSetoka\.Left:Setokagenerates user profiles spanning from abstract personality traits to concrete semantic memory, supporting four levels of user understanding\. These profiles are then used to generate heterogeneous user data and queries at each level\.Right:Representative questions at each level\. As the level of user understanding increases, answering the corresponding questions requires integrating evidence from a broader range of user data\.Bottom:The generated heterogeneous records are serialized and consumed by the memory system under evaluation to construct its memory\. To answer user queries, the agent retrieves relevant information from the memory and uses it to generate responses\.![Refer to caption](https://arxiv.org/html/2607.27056v1/x2.png)Figure 2:Setoka’s psychometrics\-based data generation pipeline\. \(a\)*Correlation\-aware personality trait sampling*draws Big\-Five persona vectors jointly from a Gaussian model parameterized by meta\-analytic trait correlations\. \(b\)*Psychological\-scale\-based behavior pattern generation*converts each persona vector into item\-level responses on a validated scale \(BFI\-2\), yielding behavior patterns with associated frequencies\. \(c\) An*event\-grounded generation tree*expands the behavior patterns coarse\-to\-fine into a temporally coherent timeline of dated events\. \(d\) Event detail generation derives linked heterogeneous records from each event, treating the event as their single source of truth\. Only these final heterogeneous records are used by the memory system under evaluation for memory construction; the intermediate outputs serve as ground truth for query generation and evaluation\.### 3\.1Problem Formulation

We defineUser Understanding Question Answering \(UUQA\)as reasoning over a user’s heterogeneous data to answer queries that explore different levels of user understanding\.Setokaaims to generate a realistic UUQA benchmark instance, which consists of a user’s corpus𝒟\\mathcal\{D\}and a testing set𝒯\\mathcal\{T\}\. Each testing instance is a tuple\(q,a,ℰ\)∈𝒯\(q,a,\\mathcal\{E\}\)\\in\\mathcal\{T\}, whereqqis a testing query,aais its reference answer, andℰ⊆𝒟\\mathcal\{E\}\\subseteq\\mathcal\{D\}is a minimal subset of records sufficient to deriveaa\. Here, the corpus is modeled as a union over a collection of heterogeneous records from multiple data resources𝒮\\mathcal\{S\}:

𝒟=⋃s∈𝒮𝒟\(s\),\\mathcal\{D\}\\;=\\;\\bigcup\_\{s\\in\\mathcal\{S\}\}\\mathcal\{D\}^\{\(s\)\},\(1\)Note that, in real\-world applications,𝒟\\mathcal\{D\}is typically generated through user interactions\.

Following recent agentic memory formulations\(Hu et al\.[2026](https://arxiv.org/html/2607.27056#bib.bib11)\), we evaluate a memory\-augmented agent with a three\-stage pipeline\. Specifically, a preprocessing operator𝒫\\mathcal\{P\}first performs*memory construction*, converting the raw corpus𝒟\\mathcal\{D\}into a normalized memory store:

ℳ=𝒫​\(𝒟\)\.\\mathcal\{M\}=\\mathcal\{P\}\(\\mathcal\{D\}\)\.\(2\)Then, for each queryqqin a testing instance\(q,a,ℰ\)\(q,a,\\mathcal\{E\}\), a retrieverℛ\\mathcal\{R\}retrieves fromℳ\\mathcal\{M\}a relevant evidence set:

ℰ^=ℛ​\(q,ℳ\)\.\\hat\{\\mathcal\{E\}\}=\\mathcal\{R\}\(q,\\mathcal\{M\}\)\.\(3\)Finally, an answer generator𝒢\\mathcal\{G\}performs the*answer generation*, producing a response based on the query and the retrieved evidence:

a^=𝒢​\(q,ℰ^\)\.\\hat\{a\}=\\mathcal\{G\}\(q,\\hat\{\\mathcal\{E\}\}\)\.\(4\)Then, the generated responsea^\\hat\{a\}is evaluated against its corresponding reference answeraausing a level\-specific metric:

v=𝒱ℓ​\(a^,a\),v=\\mathcal\{V\}\_\{\\ell\}\(\\hat\{a\},a\),\(5\)whereℓ\\elldenotes the user understanding level of the query\.

### 3\.2User Understanding Hierarchy

User understanding spans multiple levels of personal information abstraction\. As illustrated in Fig\.[1](https://arxiv.org/html/2607.27056#S3.F1), the four questions all concern personal information about the same user\. However, answering them requires quite different kinds of operations, ranging from direct fact retrieval to personality\-trait inference\. As a result, a single aggregate score would collapse these distinctions and obscure a system’s strengths and weaknesses at individual levels\. This is because a similar aggregate score might arise from significantly different performance outcomes across levels\. Therefore, we propose to use the level\-specific queries and metrics to evaluate each level separately\.

Inspired by cognitive and personality psychology\(Tulving[1972](https://arxiv.org/html/2607.27056#bib.bib27); McAdams[1995](https://arxiv.org/html/2607.27056#bib.bib20)\), we organize user understanding into two complementary aspects, which arepersonal memoriesandpersonal characteristics\. The former comprises*semantic memory \(SM\)*and*episodic memory \(EM\)*that represent concrete personal information and experiences, while the latter comprises*behavior patterns \(BP\)*and*personality traits \(PT\)*that represent long\-term, stable personal characteristics\.

Semantic Memory \(SM\)is the most concrete level of user understanding\. It consists of explicit facts about the user or the user’s world, e\.g\., a contact’s phone number\. Thus, an SM query can typically be answered by retrieving a single record\. Therefore, such evidence is explicit, localized, and directly corresponding to the requested information\.

Episodic Memory \(EM\)refers to specific events situated in a specific time and place\. It captures what happened, when and where it occurred, and who was involved\. Therefore, the evidence of an EM query is distributed across related records and should be integrated\. For example, answering the query “What was the user doing on the evening of April 20?” requires combining calendar entries, messages, and location records that describe different aspects of the same event\.

Behavior Pattern \(BP\)represents regularities in how the user behaves across repeated occurrences of a specific event category, e\.g\., socializing with friends\. The evidence for a BP query is distributed over a long time span, and should be summarized to determine how the user usually behaves in a given event category\. For example, answering the query “How many times per week does the user socialize with others?” requires aggregating social activities across multiple weeks\.

Personality Trait \(PT\)characterizes how the user tends to think, feel, and behave across different event categories, e\.g\., extraversion or conscientiousness\. A PT query requires integrating evidence across multiple event categories\. For example, answering the query “How extraverted is the user?” might require combining evidence from social gatherings and leisure activities, rather than relying on a single type of event\.

These four levels differ in how their supporting evidence is distributed and transformed\. For clarity, we formalize such transformations with four operators:selection\(σ\\sigma\),linking\(λ\\lambda\),aggregation\(γ\\gamma\), andgeneralization\(ρ\\rho\)\. Letℰℓ\\mathcal\{E\}\_\{\\ell\}denote the evidence required to answer a query at levelℓ\\ell, with the query omitted from the notation for simplicity\. Moreover, each higher level operates on a collection of outputs from lower levels, and thus its evidence set is the union of the evidence sets supporting those lower\-level outputs:

aSM\\displaystyle a\_\{\\mathrm\{SM\}\}=σ​\(ℰSM\),\\displaystyle=\\sigma\(\\mathcal\{E\}\_\{\\mathrm\{SM\}\}\),\|ℰSM\|=1,\\displaystyle\\quad\|\\mathcal\{E\}\_\{\\mathrm\{SM\}\}\|=1,\(6\)aEM\\displaystyle a\_\{\\mathrm\{EM\}\}=λ​\(\{aSM\(j\)\}j\)=\(λ∘σ\)​\(ℰEM\),\\displaystyle=\\lambda\\big\(\\\{a\_\{\\mathrm\{SM\}\}^\{\(j\)\}\\\}\_\{j\}\\big\)=\(\\lambda\{\\circ\}\\sigma\)\(\\mathcal\{E\}\_\{\\mathrm\{EM\}\}\),ℰEM=⋃jℰSM\(j\),\\displaystyle\\quad\\mathcal\{E\}\_\{\\mathrm\{EM\}\}=\\bigcup\_\{j\}\\mathcal\{E\}\_\{\\mathrm\{SM\}\}^\{\(j\)\},aBP\\displaystyle a\_\{\\mathrm\{BP\}\}=γ​\(\{aEM\(i\)\}i\)=\(γ∘λ∘σ\)​\(ℰBP\),\\displaystyle=\\gamma\\big\(\\\{a\_\{\\mathrm\{EM\}\}^\{\(i\)\}\\\}\_\{i\}\\big\)=\(\\gamma\{\\circ\}\\lambda\{\\circ\}\\sigma\)\(\\mathcal\{E\}\_\{\\mathrm\{BP\}\}\),ℰBP=⋃iℰEM\(i\),\\displaystyle\\quad\\mathcal\{E\}\_\{\\mathrm\{BP\}\}=\\bigcup\_\{i\}\\mathcal\{E\}\_\{\\mathrm\{EM\}\}^\{\(i\)\},aPT\\displaystyle a\_\{\\mathrm\{PT\}\}=ρ​\(\{aBP\(c\)\}c\)=\(ρ∘γ∘λ∘σ\)​\(ℰPT\),\\displaystyle=\\rho\\big\(\\\{a\_\{\\mathrm\{BP\}\}^\{\(c\)\}\\\}\_\{c\}\\big\)=\(\\rho\{\\circ\}\\gamma\{\\circ\}\\lambda\{\\circ\}\\sigma\)\(\\mathcal\{E\}\_\{\\mathrm\{PT\}\}\),ℰPT=⋃cℰBP\(c\)\.\\displaystyle\\quad\\mathcal\{E\}\_\{\\mathrm\{PT\}\}=\\bigcup\_\{c\}\\mathcal\{E\}\_\{\\mathrm\{BP\}\}^\{\(c\)\}\.
At the SM level,σ​\(ℰSM\)\\sigma\(\\mathcal\{E\}\_\{\\mathrm\{SM\}\}\)retrieves a fact explicitly stated in a single record\. At the EM level,λ​\(\{aSM\(j\)\}j\)\\lambda\\big\(\\\{a\_\{\\mathrm\{SM\}\}^\{\(j\)\}\\\}\_\{j\}\\big\)links multiple SM\-level answers extracted from records describing the same event\. At the BP level,γ​\(\{aEM\(i\)\}i\)\\gamma\\big\(\\\{a\_\{\\mathrm\{EM\}\}^\{\(i\)\}\\\}\_\{i\}\\big\)aggregates multiple EM\-level answers for events within the same event category to identify a behavior pattern\. At the PT level,ρ​\(\{aBP\(c\)\}c\)\\rho\\big\(\\\{a\_\{\\mathrm\{BP\}\}^\{\(c\)\}\\\}\_\{c\}\\big\)generalizes multiple BP\-level answers across different event categories to infer a personal trait\.

Note that, along the derivation from records to personality traits, the evidence sets form a nested chain:ℰSM⊆ℰEM⊆ℰBP⊆ℰPT\\mathcal\{E\}\_\{\\mathrm\{SM\}\}\\subseteq\\mathcal\{E\}\_\{\\mathrm\{EM\}\}\\subseteq\\mathcal\{E\}\_\{\\mathrm\{BP\}\}\\subseteq\\mathcal\{E\}\_\{\\mathrm\{PT\}\}\. That is, each higher level introduces both an additional transformation operator and a broader evidence scope\. The hierarchy thus progresses from extracting an explicit fact from one record to inferring a latent disposition from behavior observed across event categories\.

## 4TheSetokaGeneration Pipeline

Following the hierarchical user understanding framework, we generate user data from abstract to concrete, as illustrated in Fig\.[2](https://arxiv.org/html/2607.27056#S3.F2)\. Specifically, the high\-level personality traits guide the construction of behavior patterns, which are then mapped to concrete events and records\. We organize this process top\-down so that the generated memories remain consistent with the intended personality traits and behavior patterns, while preserving traceable evidence for evaluating each level of user understanding\.

### 4\.1Personality Trait Generation

Prior studies show that personality traits are correlated rather than statistically independent\(van der Linden, te Nijenhuis, and Bakker[2010](https://arxiv.org/html/2607.27056#bib.bib29)\)\. For example, extraversion tends to be negatively correlated with neuroticism\. Thus, sampling each trait independently ignores these dependencies and may distort the joint distribution of the resulting personality profiles\.

To address this issue, we propose*correlation\-aware personality trait sampling*\. Following continuous latent\-trait formulations in psychometrics\(Haslam et al\.[2020](https://arxiv.org/html/2607.27056#bib.bib10); Reise et al\.[2018](https://arxiv.org/html/2607.27056#bib.bib23)\), we represent theddtrait scores as continuous variables and model their joint distribution with a multivariate Gaussian\. The mean vector𝝁∈ℝd\\boldsymbol\{\\mu\}\\in\\mathbb\{R\}^\{d\}contains the target population means, and the covariance matrix is𝚺=𝐃​𝐑~​𝐃\\boldsymbol\{\\Sigma\}=\\mathbf\{D\}\\widetilde\{\\mathbf\{R\}\}\\mathbf\{D\}, where𝐃\\mathbf\{D\}contains the target standard deviations and𝐑~\\widetilde\{\\mathbf\{R\}\}is constructed from Pearson correlation coefficients reported in prior studies\(van der Linden, te Nijenhuis, and Bakker[2010](https://arxiv.org/html/2607.27056#bib.bib29)\)\. For each synthetic useruu, we then jointly draw the complete trait vector𝜽u∼𝒩​\(𝝁,𝚺\)\\boldsymbol\{\\theta\}\_\{u\}\\sim\\mathcal\{N\}\(\\boldsymbol\{\\mu\},\\boldsymbol\{\\Sigma\}\)\.

By default, we instantiate the framework with the Big Five taxonomy \(i\.e\.,d=5d=5\): openness, conscientiousness, extraversion, agreeableness, and neuroticism\(Tupes and Christal[1992](https://arxiv.org/html/2607.27056#bib.bib28); Soto and John[2017](https://arxiv.org/html/2607.27056#bib.bib24)\)\. As in Fig\.[2](https://arxiv.org/html/2607.27056#S3.F2)\(a\), a sampled persona vector may carry a high extraversion score of0\.930\.93, drawn jointly with the remaining traits so that their combination respects the meta\-analytic correlations\. The sampling procedure is taxonomy\-agnostic and can be applied to any personality trait inventory for which estimates of trait means, variances, and correlations are available\.

### 4\.2Behavior Pattern Generation

The same personality trait can lead to different behavior patterns across event categories\(Fleeson[2001](https://arxiv.org/html/2607.27056#bib.bib6); Fleeson and Jayawickreme[2015](https://arxiv.org/html/2607.27056#bib.bib7)\)\. For example, a highly conscientious user may carefully plan work tasks, consistently track personal expenses, and follow a regular exercise routine\. Although these patterns occur in different domains, they can all reflect the same underlying personality trait\. Directly prompting an LLM with a personality trait may instead generate only a narrow set of stereotypical behaviors\(Cheng, Durmus, and Jurafsky[2023](https://arxiv.org/html/2607.27056#bib.bib1); Liu, Diab, and Fried[2024](https://arxiv.org/html/2607.27056#bib.bib17)\)\.

We address this problem with*psychological\-scale\-based*\. A psychological scale is a standardized questionnaire with a predefined scoring rule that converts responses into personality trait scores\. Given a sampled personality trait vector𝜽u\\boldsymbol\{\\theta\}\_\{u\}, we generate a set of questionnaire responses whose scores match𝜽u\\boldsymbol\{\\theta\}\_\{u\}\. Because multiple response combinations can yield the same trait scores, users with the same personality traits can still exhibit different behavior patterns\. We use the 60\-item BFI\-2\(Soto and John[2017](https://arxiv.org/html/2607.27056#bib.bib24)\)to represent the Big Five personality traits\. The generated responses are then translated into a behavior pattern𝝅u\\boldsymbol\{\\pi\}\_\{u\}, which guides downstream event generation\. As in Fig\.[2](https://arxiv.org/html/2607.27056#S3.F2)\(b\), an extraversion score of0\.930\.93is reflected in item responses such as agreement with “Is talkative?”, which are translated into concrete patterns such as “small talk every other day”\.

### 4\.3Episodic and Semantic Memory Generation

Mapping the behavior pattern prior𝝅u\\boldsymbol\{\\pi\}\_\{u\}into a long\-term activity history is difficult to scale with LLM\-only generation: as the context grows, earlier behavioral constraints might be forgotten or violated, yielding inconsistent routines or activities that no longer reflect𝝅u\\boldsymbol\{\\pi\}\_\{u\}\. To address this problem, we organize generation as an*event\-grounded generation tree*with a coarse\-to\-fine temporal hierarchy, where higher\-level nodes encode behavior patterns and long\-term contexts, and lower\-level nodes specify concrete events\. For each parent node, its children are generated through a two\-stage process\. First, the child nodes are generated sequentially, with each node informed by its preceding siblings to preserve experiential continuity\. Second, their details are expanded independently in parallel, avoiding the need to retain the entire event history in context\. This hierarchical and two\-stage design bounds the context required at each step while preserving long\-range behavioral consistency and local variation\. As in Fig\.[2](https://arxiv.org/html/2607.27056#S3.F2)\(c\), the socializing patterns expand into high\-level nodes such as*phone call and follow\-up messages*spanning April 20\-26, whose child events land on the daily timeline, e\.g\.,*communicated with Alice at home*on 2026\-04\-20\.

To ensure cross\-schema consistency, each event serves as a single source of truth from which all corresponding heterogeneous records are derived and linked, as in Fig\.[2](https://arxiv.org/html/2607.27056#S3.F2)\(d\), where the April 20 event yields call, message, and app\-usage records sharing the same contact, time window, and location\.

### 4\.4Query Generation

Based on the generated user corpus and its generation\-time ground truth, we construct level\-specific testing instances\(q,a,ℰ\)∈𝒯\(q,a,\\mathcal\{E\}\)\\in\\mathcal\{T\}for each of the four levels\. Each query is designed to isolate the additional operator introduced at its level in Eq\. \([6](https://arxiv.org/html/2607.27056#S3.E6)\)\. Because the reference answers and evidence sets are derived from generation\-time ground truth, no manual annotation is needed\. AnSM querytests direct selection \(σ\\sigma\) from a single record\. We construct it by hiding one field as the answer and using the remaining fields to identify the target record\. AnEM querytests whether the system can link \(λ\\lambda\) records that describe the same event\. Following the classical partial\-cue\-recovery setting\(Huet, Houidi, and Rossi[2025](https://arxiv.org/html/2607.27056#bib.bib12)\), the query provides several event cues, such as the time, location, or participants, and asks for a missing detail that must be recovered from the linked records\. ABP querytests aggregation \(γ\\gamma\) over the user’s long\-term history\. It asks for a behavioral statistic within an activity category, such as how often or under what conditions an activity occurs\. The corresponding statistic computed from the generated event history serves as the reference answer\. A personality trait is latent and is never explicitly stated in any record, so aPT querycannot be evaluated through exact answer matching\. Instead, it asks where a user stands relative to other users on a given trait\. The sampled ground\-truth trait vectors define the reference ranking used for evaluation \(Sec\.[5\.1](https://arxiv.org/html/2607.27056#S5.SS1)\)\.

### 4\.5Dataset Statistics

We useSetokawith DeepSeek\-V4\-Pro\(DeepSeek\-AI[2026b](https://arxiv.org/html/2607.27056#bib.bib4)\)as the generation backbone to create 10 synthetic user profiles\. Each profile contains a five\-dimensional personality trait vector, 11 corresponding behavior patterns, and two months of user events\. These events are represented by approximately 1,600 heterogeneous records spanning 23 schemas, totaling 364K tokens\. On top of each profile we construct about 1,426 queries spanning the four levels\. The factual queries cover all three data models\.

## 5Experiments

SMEMBPPTMemory SystemDSMinGmaDSMinGmaDSMinGmaDSMinGmaCognee28 \(46\)20 \(32\)6 \(71\)31 \(91\)32 \(100\)2 \(90\)24 \(61\)28 \(86\)3 \(95\)20 \(100\)24 \(100\)−8\-8\(97\)HippoRAG 255 \(88\)45 \(62\)26 \(97\)33 \(98\)29 \(91\)9 \(99\)23 \(91\)25 \(93\)12 \(100\)10 \(100\)16 \(100\)9 \(98\)Mem029 \(71\)20 \(37\)19 \(84\)25 \(89\)28 \(98\)6 \(89\)11 \(51\)13 \(55\)10 \(85\)1 \(97\)3 \(84\)4 \(98\)MemMachine65 \(88\)58 \(74\)25 \(92\)46 \(97\)42 \(99\)9 \(99\)23 \(87\)28 \(96\)11 \(97\)20 \(100\)18 \(100\)−8\-8\(98\)Letta52 \(83\)42 \(59\)21 \(92\)31 \(92\)30 \(96\)6 \(96\)22 \(86\)24 \(91\)6 \(93\)8 \(97\)18 \(100\)2 \(91\)DBQuery85 \(90\)59 \(63\)17 \(73\)27 \(90\)28 \(99\)0 \(2\)16 \(75\)3 \(34\)9 \(97\)0 \(61\)−6\-6\(59\)−4\-4\(92\)Average52 \(78\)41 \(54\)19 \(85\)32 \(93\)32 \(97\)5 \(79\)20 \(75\)20 \(76\)9 \(95\)10 \(93\)12 \(91\)−1\-1\(96\)Level average37 \(72\)23 \(90\)16 \(82\)7 \(93\)Table 2:Main results: per\-level scores for each memory system×\\timeslanguage model over all 10 users\. Scores are scaled by 100 for display, with the answer rate \(%\) in parentheses\. SM, EM, and BP report LLM\-judge similarity; PT reports the PT score\. The best score per column is in bold, and the best \(system, backbone\) cell within each query level is additionally underlined\. The last two rows average each column over the six systems and each level over all systems and backbones\. Model columns:DS= DeepSeek\-V4\-Flash,Min= Ministral 3 14B Instruct,Gma= Gemma 3 4B\-IT\.### 5\.1Experimental Setup

Data Serialization\.We instantiate the memory construction operator𝒫\\mathcal\{P\}as follows\. Following common practice in structured\-data understanding\(Sui et al\.[2024](https://arxiv.org/html/2607.27056#bib.bib25); Wu et al\.[2025b](https://arxiv.org/html/2607.27056#bib.bib33)\), we serialize relational records as Markdown tables due to their high coverage in LLM training corpora and a favorable performance\-token trade\-off\. Graph\-structured data is serialized as adjacency lists, while semi\-structured/unstructured records are kept in their native formats\.

Backbones and Memory Systems\.We evaluate 3 language models spanning the deployment spectrum: DeepSeek\-V4\-Flash\(DeepSeek\-AI[2026a](https://arxiv.org/html/2607.27056#bib.bib3)\), a strong cloud MoE model with 284B total parameters; Ministral 3 14B Instruct\(Mistral AI[2026](https://arxiv.org/html/2607.27056#bib.bib21)\), suitable for personal computer deployment; Gemma 3 4B\-IT\(Gemma Team[2025](https://arxiv.org/html/2607.27056#bib.bib8)\), deployable on mobile devices\. We pair each backbone with 5 representative memory systems spanning three dominant organizational paradigms: Cognee\(Markovic et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib19)\)and HippoRAG 2\(Gutiérrez et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib9)\)are graph\-structured systems, Mem0\(Chhikara et al\.[2025](https://arxiv.org/html/2607.27056#bib.bib2)\)is a flat fact\-based system, and MemMachine\(Wang et al\.[2026](https://arxiv.org/html/2607.27056#bib.bib30)\)and Letta\(Packer et al\.[2023](https://arxiv.org/html/2607.27056#bib.bib22)\)are hierarchical systems\.

DBQuery Baseline\.To examine whether explicit memory construction is necessary, we introduce DBQuery, a memory\-free baseline where the LLM directly interacts with the underlying databases\. Given a question, the LLM generates and executes database queries over the raw records and produces an answer, bypassing any intermediate memory representation\.

Metrics\.We report per\-level performance and an overall score averaged across all test instances\. For SM, EM, and BP, we employ an LLM judge \(DeepSeek\-V4\-Pro\) to score each answer against the reference on a\[0,1\]\[0,1\]scale, allowing surface\-form variation while requiring semantic consistency with the gold standard\(Zheng et al\.[2023](https://arxiv.org/html/2607.27056#bib.bib34)\)\. For PT, we evaluate whether the model correctly preserves the relative ordering of users along each personality trait\. We evaluate the agreement between the predicted and ground\-truth user rankings using Kendall’s rank correlation coefficient\(Kendall[1938](https://arxiv.org/html/2607.27056#bib.bib15)\):

PT​\-​Scorej=τK​\(\{θ^u,j\}u∈𝒰,\{θu,j\}u∈𝒰\),\\mathrm\{PT\\text\{\-\}Score\}\_\{j\}=\\tau\_\{\\mathrm\{K\}\}\\left\(\\\{\\hat\{\\theta\}\_\{u,j\}\\\}\_\{u\\in\\mathcal\{U\}\},\\\{\\theta\_\{u,j\}\\\}\_\{u\\in\\mathcal\{U\}\}\\right\),\(7\)where𝒰\\mathcal\{U\}is the set of users,θu,j\\theta\_\{u,j\}andθ^u,j\\hat\{\\theta\}\_\{u,j\}are the ground\-truth and predicted scores of useruuon trait dimensionjj, respectively\. The termτK∈\[−1,1\]\\tau\_\{\\mathrm\{K\}\}\\in\[\-1,1\]denotes Kendall’s rank correlation coefficient, where11means the rankings match exactly,0means performance no better than random guessing, and negative values mean opposite rankings\. We report the PT\-Score pooled over allddtrait dimensions\.

### 5\.2Main Findings and Discussions

Table[2](https://arxiv.org/html/2607.27056#S5.T2)summarizes the main results across memory systems, language models, and levels of user understanding\. We discuss the key findings and their implications below\.

Difficulty increases with the level of abstraction\.As formalized in Eq\. \([6](https://arxiv.org/html/2607.27056#S3.E6)\), higher levels require both broader evidence integration and additional semantic transformations\. This creates more points at which relevant evidence may be missed, combined incorrectly, or transformed inaccurately\. Accordingly, both the best and average scores decline steadily along the hierarchy\. The best score drops from 0\.85 on SM to 0\.46 on EM, 0\.28 on BP, and 0\.24 on PT, while the level average falls from 0\.37 to 0\.23, 0\.16, and 0\.07\.*These results show that current memory systems become less effective as queries require broader evidence and more abstract inference, motivating mechanisms for cross\-source joining, evidence aggregation, and latent\-attribute inference\.*

Memory construction helps, but different aggregation levels support different levels of user understanding\.Memory\-based systems outperform DBQuery in all settings except single\-record SM lookup, where DBQuery leads with the two stronger backbones DeepSeek and Ministral, but no system performs best at every level\. ❶ DBQuery retrieves individual records without constructing memory\. This preserves exact details, giving the best SM score0\.850\.85, but performs poorly when multiple records must be combined, dropping to0\.270\.27on EM\. ❷ MemMachine is the strongest system overall, ranking first or joint first in five of twelve settings and achieving the best EM score of 0\.46\. We attribute this to its comparatively fine\-grained record preservation combined with the retrieval of temporally related information, which supplies broader context without excessively compressing the original details\. ❸ Graph\-structured memory systems rank first or joint\-first on BP and PT in all six settings\. Cognee is best or joint\-best with DeepSeek and Ministral \(BP0\.24/0\.280\.24/0\.28, PT0\.20/0\.240\.20/0\.24\), while HippoRAG 2 leads with Gemma \(BP0\.120\.12, PT0\.090\.09\)\. Linking related records appears to help aggregate recurring behaviors into higher\-level user traits\. But scores vary widely across systems and backbones, so graph structure alone is not enough\. How the graph represents and retrieves evidence matters just as much\.*Overall, separate records preserve precise facts, memories with temporally related context support multi\-record reasoning, and explicit associations across records support inference over dispersed evidence\. Memory systems should therefore maintain multiple levels of aggregation and association, and select the level suited to each query\.*

Answer rate reflects calibration rather than competence\.Answer rate increases from72%72\\%on SM to93%93\\%on PT, even though accuracy decreases\. This is because a system cannot answer a concrete factual question when the relevant evidence is missing, whereas it can always guess an abstract personality trait\. The same contrast appears across backbones\. DeepSeek and Ministral often abstain when evidence is insufficient\. In contrast, the mobile\-scale Gemma answers7171–100%100\\%of queries at every level but achieves the lowest accuracy\. Thus, a high answer rate may indicate unsupported guessing rather than better knowledge\.*Because abstentions are counted as errors in the accuracy metric, answer rate should be reported separately\. Together, the two metrics distinguish systems that abstain when evidence is insufficient from those that answer regardless of uncertainty\.*

## 6Conclusion

We introducedSetoka, the first benchmark for hierarchical user understanding of memory\-augmented personalized agents over heterogeneous data\.Setokacombines a psychology\-grounded four\-level framework with a psychometrics\-based generation pipeline that produces heterogeneous records with ground\-truth provenance from the underlying personality traits\. Evaluating 3 language models over 5 memory systems and a SQL\-augmented baseline, we find that accuracy decays monotonically with the level of abstraction\. Direct database access largely solves single\-record lookup, whereas multi\-record assembly and long\-horizon aggregation remain unsolved, and even the best configurations rank users by personality trait only marginally above chance\. Improving retrieval alone is therefore insufficient\.Setokaoffers a testbed for memory systems that link, aggregate, and generalize user evidence beyond literal recall\.

## References

- Cheng, Durmus, and Jurafsky \(2023\)Cheng, M\.; Durmus, E\.; and Jurafsky, D\. 2023\.Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models\.In*ACL*, 1504–1532\.
- Chhikara et al\. \(2025\)Chhikara, P\.; Khant, D\.; Aryan, S\.; et al\. 2025\.Mem0: Building Production\-Ready AI Agents with Scalable Long\-Term Memory\.In*arXiv:2504\.19413*\.
- DeepSeek\-AI \(2026a\)DeepSeek\-AI\. 2026a\.DeepSeek\-V4\-Flash\.https://api\-docs\.deepseek\.com/\.
- DeepSeek\-AI \(2026b\)DeepSeek\-AI\. 2026b\.DeepSeek\-V4\-Pro\.https://api\-docs\.deepseek\.com/\.
- Du et al\. \(2024\)Du, Y\.; Wang, H\.; Zhao, Z\.; et al\. 2024\.PerLTQA: A Personal Long\-Term Memory Dataset for Memory Classification, Retrieval, and Synthesis in Question Answering\.In*arXiv:2402\.16288*\.
- Fleeson \(2001\)Fleeson, W\. 2001\.Toward a Structure\- and Process\-Integrated View of Personality: Traits as Density Distributions of States\.*J\. Pers\. Soc\. Psychol\.*, 80\(6\): 1011–1027\.
- Fleeson and Jayawickreme \(2015\)Fleeson, W\.; and Jayawickreme, E\. 2015\.Whole Trait Theory\.*J\. Res\. Pers\.*, 56: 82–92\.
- Gemma Team \(2025\)Gemma Team\. 2025\.Gemma 3 4B IT\.https://huggingface\.co/google/gemma\-3\-4b\-it\.
- Gutiérrez et al\. \(2025\)Gutiérrez, B\. J\.; Shu, Y\.; Qi, W\.; et al\. 2025\.From RAG to Memory: Non\-Parametric Continual Learning for Large Language Models\.In*arXiv:2502\.14802*\.
- Haslam et al\. \(2020\)Haslam, N\.; McGrath, M\. J\.; Viechtbauer, W\.; et al\. 2020\.Dimensions over categories: A meta\-analysis of taxometric research\.*Psychol\. Med\.*, 50\(9\): 1418–1432\.
- Hu et al\. \(2026\)Hu, Y\.; Liu, S\.; Yue, Y\.; et al\. 2026\.Memory in the Age of AI Agents\.In*arXiv:2512\.13564*\.
- Huet, Houidi, and Rossi \(2025\)Huet, A\.; Houidi, Z\. B\.; and Rossi, D\. 2025\.Episodic Memories Generation and Evaluation Benchmark for Large Language Models\.In*arXiv:2501\.13121*\.
- Jiang et al\. \(2025a\)Jiang, B\.; Hao, Z\.; Cho, Y\.\-M\.; et al\. 2025a\.Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale\.In*arXiv:2504\.14225*\.
- Jiang et al\. \(2025b\)Jiang, B\.; Yuan, Y\.; Shen, M\.; et al\. 2025b\.PersonaMem\-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory\.In*arXiv:2512\.06688*\.
- Kendall \(1938\)Kendall, M\. G\. 1938\.A New Measure of Rank Correlation\.*Biometrika*, 30\(1/2\): 81–93\.
- Li et al\. \(2024\)Li, Y\.; Wen, H\.; Wang, W\.; et al\. 2024\.Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security\.In*arXiv:2401\.05459*\.
- Liu, Diab, and Fried \(2024\)Liu, A\.; Diab, M\.; and Fried, D\. 2024\.Evaluating Large Language Model Biases in Persona\-Steered Generation\.In*Findings of ACL*, 9832–9850\.
- Maharana et al\. \(2024\)Maharana, A\.; Lee, D\.\-H\.; Tulyakov, S\.; et al\. 2024\.Evaluating Very Long\-Term Conversational Memory of LLM Agents\.In*ACL*, 13851–13870\.
- Markovic et al\. \(2025\)Markovic, V\.; Obradovic, L\.; Hajdu, L\.; et al\. 2025\.Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning\.In*arXiv:2505\.24478*\.
- McAdams \(1995\)McAdams, D\. P\. 1995\.What Do We Know When We Know a Person?*J\. Pers\.*, 63\(3\): 365–396\.
- Mistral AI \(2026\)Mistral AI\. 2026\.Ministral 3 14B Instruct\.https://huggingface\.co/mistralai/Ministral\-3\-14B\-Instruct\.
- Packer et al\. \(2023\)Packer, C\.; Wooders, S\.; Lin, K\.; et al\. 2023\.MemGPT: Towards LLMs as Operating Systems\.In*arXiv:2310\.08560*\.
- Reise et al\. \(2018\)Reise, S\. P\.; Rodriguez, A\.; Spritzer, K\. L\.; et al\. 2018\.Alternative Approaches to Addressing Non\-Normal Distributions in the Application of IRT Models to Personality Measures\.*J\. Pers\. Assess\.*, 100\(4\): 363–374\.
- Soto and John \(2017\)Soto, C\. J\.; and John, O\. P\. 2017\.The next Big Five Inventory \(BFI\-2\): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power\.*J\. Pers\. Soc\. Psychol\.*, 113\(1\): 117–143\.
- Sui et al\. \(2024\)Sui, Y\.; Zhou, M\.; Zhou, M\.; et al\. 2024\.Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study\.In*WSDM*, WSDM ’24, 645–654\.ISBN 9798400703713\.
- Tan et al\. \(2025\)Tan, H\.; Zhang, Z\.; Ma, C\.; et al\. 2025\.MemBench: Towards More Comprehensive Evaluation on the Memory of LLM\-based Agents\.In*Findings of ACL*, 19336–19352\.ISBN 979\-8\-89176\-256\-5\.
- Tulving \(1972\)Tulving, E\. 1972\.Episodic and Semantic Memory\.In*Organization of Memory*, 381–403\.
- Tupes and Christal \(1992\)Tupes, E\. C\.; and Christal, R\. E\. 1992\.Recurrent personality factors based on trait ratings\.*J\. Pers\.*, 60\(2\): 225–251\.
- van der Linden, te Nijenhuis, and Bakker \(2010\)van der Linden, D\.; te Nijenhuis, J\.; and Bakker, A\. B\. 2010\.The General Factor of Personality: A meta\-analysis of Big Five intercorrelations and a criterion\-related validity study\.*J\. Res\. Pers\.*, 44\(3\): 315–327\.
- Wang et al\. \(2026\)Wang, S\.; Yu, E\.; Love, O\.; et al\. 2026\.MemMachine: A Ground\-Truth\-Preserving Memory System for Personalized AI Agents\.In*arXiv:2604\.04853*\.
- Wu et al\. \(2025a\)Wu, D\.; Wang, H\.; Yu, W\.; et al\. 2025a\.LongMemEval: Benchmarking Chat Assistants on Long\-Term Interactive Memory\.In*arXiv:2410\.10813*\.ICLR 2025\.
- Wu et al\. \(2026\)Wu, T\.; Chen, Z\.; Weng, Z\.; et al\. 2026\.KnowMe\-Bench: Benchmarking Person Understanding for Lifelong Digital Companions\.In*arXiv:2601\.04745*\.
- Wu et al\. \(2025b\)Wu, X\.; Yang, J\.; Chai, L\.; et al\. 2025b\.TableBench: a comprehensive and complex benchmark for table question answering\.In*AAAI*, AAAI’25/IAAI’25/EAAI’25, 25497–25506\.ISBN 978\-1\-57735\-897\-8\.
- Zheng et al\. \(2023\)Zheng, L\.; Chiang, W\.\-L\.; Sheng, Y\.; et al\. 2023\.Judging LLM\-as\-a\-Judge with MT\-Bench and Chatbot Arena\.In*NeurIPS \(Datasets and Benchmarks\)*\.