Adaptive Personalization Under Preference Epistemic Uncertainty

# Adaptive Personalization Under Preference Epistemic Uncertainty
Source: [https://arxiv.org/html/2610.40930](https://arxiv.org/html/2610.40930)
Priyanka DeyAffiliation:Meta Superintelligence LabsAffiliation:University of Southern CaliforniaWork done at MetaVikas BahirwaniAffiliation:Meta Superintelligence LabsQi ZengAffiliation:Meta Superintelligence LabsJulia KrukAffiliation:Meta Superintelligence LabsQi QiAffiliation:Meta Superintelligence Labs

###### Abstract

Human preferences change over time: an established preference may remain valid over time, become uncertain as context shifts, or be explicitly revised\. Effective personalization requires conversational AI to adapt its response strategy to evolving preferences rather than relying on passive memory retrieval\. We formalize this capability asadaptive personalization under epistemic uncertainty\. A human preference study further confirms that people favor state\-contingent response strategies as preference uncertainty changes\. To evaluate whether language models mirror this adaptation, we create a multilingual benchmark spanning 250 personas, 5 languages, and 25K conversational turns that decouples strategy selection from response execution\. Across eight closed\- and open\-weight models, we uncover arecognition–generation gap: models reliably identify appropriate strategies when selecting among candidate options, but collapse during open\-ended generation, suffering a 35\-46% drop in strategy calibration\. Crucially, this failure is systematic: models default to exploit\-first behavior, over\-relying on existing preferences regardless of preference epistemic uncertainty\. We show this failure is addressable: while prompting interventions fail, targeted training that supervises models’ internal strategy representations toward state\-conditional human preferences narrows the user\-acceptability generalization gap by a third on out\-of\-distribution personas and conversational styles\.

††correspondence:Priyanka Dey at[deyp@usc\.edu](mailto:[email protected])††Code:[https://github\.com/priyanka\-dey/adaptive\-personalization](https://github.com/priyanka-dey/adaptive-personalization)## 1Introduction

Human preferences evolve dynamically across interactions: an established preference may remain valid over time, become uncertain as context shifts, or be explicitly revised by new evidence\. For conversational AI, adapting to these shifts requires far more than passive memory retrieval\. It demands deciding whether a remembered preference should still be acted upon, questioned, hedged around, or updated\. Consider the user in Figure[1](https://arxiv.org/html/2610.40930#S1.F1), who states a preference for savory food, later mentions exploring local bakeries, and eventually says she has come to enjoy something sweet, without ever restating her preferences\. An assistant that stubbornly mentions her preference for savory foods across all three turns has retrieved the user’s history correctly, but failed to adjust the conversational strategy to match the evolving reliability of that preference\.

![Refer to caption](https://arxiv.org/html/2610.40930v1/images/pitch_fig.png)Figure 1:Humans prefer varying response strategies based on preference epistemic uncertainty; models exploit regardless\.\(Left\)One user at three time points, illustrating the three preference epistemic states we evaluate: 1\)*established*, 2\)*uncertain*, and 3\)*revised*, each paired with a well\-matched and a mismatched assistant reply\.\(Right\)Across the three epistemic states \(preference uncertainties\), humans prefer responses with varying strategies, whereas current models predominantly*exploit*in all states\.We formalize this capability as*adaptive personalization under epistemic uncertainty*\. Evidence regarding user preferences evolves across three distinct epistemic uncertainty states:established\(explicitly stated and uncontradicted\),uncertain\(ambiguous or partial context\), andrevised\(superseded by new evidence\)\. Each state demands a distinct conversational behavior: acting directly on settled preferences, seeking clarification under ambiguity\([Clark, 1996](https://arxiv.org/html/2610.40930#bib.bib13);[Clark and Brennan, 1991](https://arxiv.org/html/2610.40930#bib.bib14);[Traum, 1994](https://arxiv.org/html/2610.40930#bib.bib15), dialogue grounding;\), or course\-correcting when preferences change\([Alchourrón et al\., 1985](https://arxiv.org/html/2610.40930#bib.bib16);[Gärdenfors, 1988](https://arxiv.org/html/2610.40930#bib.bib17);[Hansson, 1999](https://arxiv.org/html/2610.40930#bib.bib18), belief revision;\)\. A controlled human study confirms this state\-contingent behavior: humans systematically prefer distinct model response strategies across epistemic states \(§[2\.2](https://arxiv.org/html/2610.40930#S2.SS2)\)\.

Despite human demand for dynamic strategy adaptation, existing personalization benchmarks predominantly evaluate model retrieval of preferences over long\-horizon interactions\([Jiang et al\., 2025a](https://arxiv.org/html/2610.40930#bib.bib26);[Zhao et al\., 2025a](https://arxiv.org/html/2610.40930#bib.bib27);[Jiang et al\., 2025b](https://arxiv.org/html/2610.40930#bib.bib45);[Harry et al\., 2026](https://arxiv.org/html/2610.40930#bib.bib44);[Guo et al\., 2026](https://arxiv.org/html/2610.40930#bib.bib28)\), but fail to test whether conversational strategy adapts to preference reliability over time\. To measure this distinct capability, we create a benchmark spanning 250 personas, five languages, and 25,006 conversational turns, with oracle epistemic\-state labels and an evaluation framework that decouples strategy selection from response execution \(§[2](https://arxiv.org/html/2610.40930#S2)\)\.

Evaluating eight closed\- and open\-weight models demonstrates that models’ failures localize to generation rather than recognition \(§[3](https://arxiv.org/html/2610.40930#S3)\)\. Across varying epistemic states, models select responses with appropriate strategies at a high accuracy when presented with candidate options, but fail to generate responses – a gap widening to over 35% under epistemic states: uncertain and revised\. Moreover, models collapse to an exploit\-first default generation policy that over\-relies on historical preferences regardless of preference epistemic uncertainty\.

To address this default behavioral policy in models, we first test inference\-time prompting interventions \(§[4\.1](https://arxiv.org/html/2610.40930#S4.SS1)\); however, directly providing preference context fails to close the recognition\-generation gap\. Internal probing demonstrates that although models represent epistemic states in their hidden activations, this passive knowledge fails to govern downstream strategy selection for generation \(§[4\.2](https://arxiv.org/html/2610.40930#S4.SS2)\)\. Motivated by this diagnostic, we train models using a reward function that aligns predicted models’ response strategies with human preference choices, conditioned on preference epistemic uncertainty \(§[4\.2](https://arxiv.org/html/2610.40930#S4.SS2)\)\. This targeted training narrows the user\-acceptability generalization gap by a third to out\-of\-distribution personas and conversational styles beyond standard fine tuning\.

Together, these results identify adaptive personalization under epistemic uncertainty as a distinct, learnable capability: one that contemporary language models do not reliably deploy by default, but one that targeted training can build\.

## 2Task Formulation and Benchmark

### 2\.1Task Formulation

Dialogue theories characterize conversation as a process of establishing and maintaining common ground, in which interlocutors continuously distinguish between mutually established information and information that remains uncertain or ungrounded\([Clark, 1996](https://arxiv.org/html/2610.40930#bib.bib13);[Clark and Brennan, 1991](https://arxiv.org/html/2610.40930#bib.bib14)\)\. Formal theories of belief revision describe how such beliefs should be updated when new evidence conflicts with existing knowledge\([Alchourrón et al\., 1985](https://arxiv.org/html/2610.40930#bib.bib16);[Gärdenfors, 1988](https://arxiv.org/html/2610.40930#bib.bib17);[Hansson, 1999](https://arxiv.org/html/2610.40930#bib.bib18)\)\. We ground our task in these two areas and adapt them to a personalization setting, where an assistant’s evidence about a user’s preferences can be established, uncertain, or revised, and its conversational strategy should track that evidence\.

An assistant’s evidence for a preference \(epistemic uncertainty\) isestablishedwhen explicitly expressed and uncontradicted,uncertainwhen evidence is insufficient or ambiguous, orrevisedwhen newer evidence supersedes it\. Given these varying epistemic states, communication theory, belief revision theory and linguistic studies\([Clark, 1996](https://arxiv.org/html/2610.40930#bib.bib13);[Clark and Brennan, 1991](https://arxiv.org/html/2610.40930#bib.bib14);[Lakoff, 1973](https://arxiv.org/html/2610.40930#bib.bib22);[Alchourrón et al\., 1985](https://arxiv.org/html/2610.40930#bib.bib16);[Gärdenfors, 1988](https://arxiv.org/html/2610.40930#bib.bib17)\)distinguishes several strategies an assistant can take in conversation:

1. 1\.exploit: acting directly on a stated preference
2. 2\.elicit: asking the user for missing information
3. 3\.confirm: checking whether a preference still holds before acting on it
4. 4\.hedge: offering options without fully committing to either
5. 5\.recalibrate: updating behavior once new evidence contradicts a prior preference

Together, these five strategies form an operational set of conversational actions \(Table[3](https://arxiv.org/html/2610.40930#A1.T3); full definitions in §[A](https://arxiv.org/html/2610.40930#A1)\)\. Formally, at each turn the assistant observes a dialogue contextxxwith epistemic states∈\{established,uncertain,revised\}s\\in\\\{\\textsc\{established\},\\textsc\{uncertain\},\\textsc\{revised\}\\\}over a preference and must select an appropriate conversational strategya∈\{exploit,elicit,confirm,hedge,recalibrate\}a\\in\\\{\\textsc\{exploit\},\\textsc\{elicit\},\\textsc\{confirm\},\\textsc\{hedge\},\\textsc\{recalibrate\}\\\}\.

### 2\.2Preferred Strategies Depend on Epistemic State

We conducted a multilingual preference study across five languages \- English, Spanish, Portuguese, Hindi, and Indonesian, to measure which assistant response strategies humans prefer across different epistemic states\. Annotators were presented with dialogues exhibiting varying epistemic states and chose their preferred response between LLM outputs implementing different conversational strategies\. The study included 500 dialogue instances evaluated by 3 annotators per item \(∼\\sim1,500 total annotations\)\. Table[1](https://arxiv.org/html/2610.40930#S2.T1)reports this resulting preference distribution\.

Table 1:Preferred strategy varies systematically with preference epistemic state\.✓marks the set of acceptable strategies used for benchmark construction, evaluation, and training\.Human preferences are not evenly distributed across the five strategies: in each epistemic state, a small set of strategies account for the vast majority of preferences \(\>80%\>80\\%\) \(Table[1](https://arxiv.org/html/2610.40930#S2.T1)\), suggesting a natural separation between acceptable and unacceptable strategies\. To validate this separation, we exhaustively evaluated all 15 possible two\-group splits of the five strategies in each state using aχ2\\chi^\{2\}goodness\-of\-fit test against the observed human choices \(§[A\.6](https://arxiv.org/html/2610.40930#A1.SS6)\)\. In every state, one split fit the observed preferences best\. We therefore use these best\-fitting splits to define our human\-grounded sets of*acceptable*and*unacceptable*strategies:exploitforestablished;elicit, confirm, hedgeforuncertain; andelicit, confirm, hedge, recalibrateforrevised\.

### 2\.3Benchmark Construction

Evaluating strategy selection under preference uncertainty requires two properties that existing real\-world dialogue corpora lack: controlled, multi\-session preference trajectories with known ground\-truth transitions, and counterfactual candidate responses spanning multiple strategies at each turn\. To address this gap, we introduce a multilingual benchmark spanning five languages \(English, Spanish, Portuguese, Hindi, and Indonesian; Figure[7](https://arxiv.org/html/2610.40930#A1.F7)\), constructed through a three\-role multi\-LLM pipeline\.

Starting from 250 diverse personas seeded from PersonaHub[Ge et al\. \(2024\)](https://arxiv.org/html/2610.40930#bib.bib47)using farthest\-point sampling, each persona is augmented with preference attributes drawn from 20 life domains[Zhao et al\. \(2025b\)](https://arxiv.org/html/2610.40930#bib.bib46)and 13 interaction styles[Li et al\. \(2026\)](https://arxiv.org/html/2610.40930#bib.bib25)\. A*user simulator*plans multi\-session conversation arcs \(10\-14 sessions\) with oracle\-labeled preference trajectories, systematically injecting planned drift events that transition a preference between the three epistemic states:established,uncertain, andrevised\. An*AI assistant*then generates candidate responses across strategies at each turn, while an*LLM jury*filters output for conversation coherence and label fidelity\.

Our final benchmark comprises 25,006 turns across 2,939 multi\-session conversations\. Each turn includes four candidate LLM responses: two acceptable strategies \(per Table[1](https://arxiv.org/html/2610.40930#S2.T1)\) and two negative foils categorized into seven realistic failure modes identified during preliminary annotation \(over\-personalization, epistemic overreach, staleness, need mismatch, over\-probing, style mismatch, and failure to probe; see §[A](https://arxiv.org/html/2610.40930#A1)for full statistics and construction details\)\.

We validate our benchmark through a human evaluation following established benchmark protocols[Zollo et al\. \(2025\)](https://arxiv.org/html/2610.40930#bib.bib21);[Jang et al\. \(2026\)](https://arxiv.org/html/2610.40930#bib.bib23);[Dey et al\. \(2025\)](https://arxiv.org/html/2610.40930#bib.bib24)\. Human annotators evaluated 500 dialogue instances across all five languages \(3 annotators per language\) on five dimensions: conversation coherence, epistemic\-state clarity, strategy\-label correctness, foil\-type accuracy, and response acceptability\. Agreement between human annotators and the LLM jury was substantial \(κ=0\.78\\kappa=0\.78; §[A\.6](https://arxiv.org/html/2610.40930#A1.SS6)\), while human multiple\-choice accuracy \(selecting a response with acceptable strategy use from candidate responses\) reached 92%\. These results indicate that the benchmark’s epistemic states and strategy labels are both recognizable to humans and reflected in natural dialogue\.

### 2\.4Evaluation Protocol

At each turn, given personaPP, conversation historyCC, and current user turnuu, the benchmark provides four candidate responses\(r1\+,r2\+,r3−,r4−\)\(r\_\{1\}^\{\+\},r\_\{2\}^\{\+\},r\_\{3\}^\{\-\},r\_\{4\}^\{\-\}\), wherer1\+r\_\{1\}^\{\+\}andr2\+r\_\{2\}^\{\+\}implement acceptable strategies for the current epistemic state, whiler3−r\_\{3\}^\{\-\}andr4−r\_\{4\}^\{\-\}implement foil strategies \(§[2](https://arxiv.org/html/2610.40930#S2)\)\. To benchmark*adaptive personalization under epistemic uncertainty*, we evaluate models across two settings:multiple\-choice selection\(MCQ\), where the model selects one candidate from\{r1\+,r2\+,r3−,r4−\}\\\{r\_\{1\}^\{\+\},r\_\{2\}^\{\+\},r\_\{3\}^\{\-\},r\_\{4\}^\{\-\}\\\}, andfree\-response generation\(FRG\), where the model generates an unguided response given\(P,C,u\)\(P,C,u\)\. Both settings are evaluated onstrategy calibration \(SC\): whether the chosen or generated response utilizes an acceptable strategy111When responses combine strategies, we score the dominant strategy: the strategy appearing most strongly in response\. This avoids counting strategy\-stacking responses as calibrated across states\. Alternative scoring rules in \(Appendix[B\.5](https://arxiv.org/html/2610.40930#A2.SS5)\)\.for the epistemic state \(MCQ: matchingr1\+r\_\{1\}^\{\+\}orr2\+r\_\{2\}^\{\+\}with a 50% chance baseline; FRG: classified as acceptable via an LLM jury; §[2\.3](https://arxiv.org/html/2610.40930#S2.SS3)\)\. Comparing SC across settings explicitly isolates strategy recognition from strategy generation\.

FRG outputs are additionally evaluated along two dimensions by the user simulator \(§[2\.3](https://arxiv.org/html/2610.40930#S2.SS3)\): a state\-conditionedrubric \(R\)scoring five binary criteria \(out of 5\): Information, Confidence, Intent, Scope, and Style—anduser acceptability \(UA\), a binary accept/reject judgment\. Together, SC, R, and UA decouple strategy selection, response execution, and overall user satisfaction\.

## 3LLMs Fail to Adapt Conversational Strategy Under Epistemic Uncertainty

§[2\.2](https://arxiv.org/html/2610.40930#S2.SS2)showed that people prefer varying conversational strategies based on epistemic state\. We now use our multi\-lingual benchmark to measure whether models adapt strategy during conversation generation\. We evaluate eight closed\- and open\-weight models \(GPT\-5\.4, Opus 4\.7, Grok 4\.1, Gemini 3 Pro, Gemma 3 12B, Llama\-3\.1\-8B, OLMo\-3\-7B, and Qwen3\-8B\) across the three epistemic states and across five languages\.

### 3\.1Strategy Calibration Collapses When Preference Evidence Becomes Uncertain or Revised

Figure 2:Strategy calibration across epistemic states\.All models perform best inEstablishedand struggle withUncertainandRevisedstates\.We first measure whether models adapt their open\-ended generation strategies based on epistemic state\. Figure 2 shows free\-response \(FRG\) strategy calibration \(SC\) by epistemic state, i\.e\., whether a response’s dominant strategy is acceptable for that state\. Calibration is high when preferences are established \(∼83\.1%\\sim 83\.1\\%across models\) but declines significantly once conversations require managing uncertainty \(∼36\.9%\\sim 36\.9\\%; best model: GPT\-5\.4 at62%62\\%\) or adapting to a revision \(∼47\.9%\\sim 47\.9\\%; best: GPT\-5\.4 at68%68\\%\)\. Both declines are significant after accounting for the nested structure of the data \(mixed\-effects logistic regression with random intercepts for persona and conversation; Uncertain:β=−2\.16\\beta=\-2\.16, Revised:β=−1\.67\\beta=\-1\.67, bothp<\.001p<\.001; full specification and per\-language estimates in §[B\.1](https://arxiv.org/html/2610.40930#A2.SS1)\)\. This trend holds across models and languages, and is deeper in Hindi and Indonesian\. closed\-weight models retain higher calibration than open\-weight models, and this margin widens with epistemic difficulty: 4\.2 points when preferences are established, 16\.9 under uncertainty, and 25\.3 under revision\.

### 3\.2Models Recognize Appropriate Strategies More Often Than They Generate Them

This decline in SC could reflect either of two distinct failures: agenerationfailure, in which models correctlyrecognizethe appropriate strategy but fail to produce it, or a more fundamentalrecognitionfailure, in which models cannot correctly identify the appropriate strategy at all\. We distinguish these failure modes using our two evaluation methods: multiple\-choice questions \(MCQ\), where a model must select the appropriate response \(text following acceptable strategy\) among candidates, and free\-response generation \(FRG\), which requires the model to generate text following the acceptable strategy\.

Figure 3:Recognition exceeds generation\.Models identify acceptable strategies \(MCQ, green\) more often than they generate them \(FRG, yellow\) across all three epistemic states\. The horizontal distance between paired points is the recognition\-generation gap\.Figure[3](https://arxiv.org/html/2610.40930#S3.F3)compares the two evaluation results\. MCQ strategy calibration remains high across all three states \(mean 85\.4%, range: 75\-95%; 50% chance baseline\), declining only modestly under uncertainty and revision relative to established preferences \(90\.3%→\\to82\.4%→\\to83\.5%\)\. FRG strategy calibration, by contrast, collapses far more sharply across the three states \(83\.1%→\\to36\.9%→\\to47\.9%; §[3\.1](https://arxiv.org/html/2610.40930#S3.SS1)\)\. The resulting recognition–generation gap is present for every model in every state, and is widest underUncertain\(\+45\+45pp,p<\.001p<\.001\), followed byRevised\(\+36\+36pp,p<\.001p<\.001\) andEstablished\(\+7\+7pp,p<\.001p<\.001\)\. Models can therefore identify the appropriate strategy substantially more reliably than they produce it: the failure characterized in §[3\.1](https://arxiv.org/html/2610.40930#S3.SS1)is not explained by an inability to recognize appropriate behavior\. A further check, explicitly supplying the correct strategy and asking models only to execute it, shows this gap is driven specifically by selecting a strategy, not executing one: models generate responses with the supplied strategy on 95% of turns, and execution quality under this condition is state\-invariant, whereas unaided calibration still swings by over 40 points across states \(§[B\.3](https://arxiv.org/html/2610.40930#A2.SS3)\)

### 3\.3Generation Failures Reflect Systematic Over\-Exploitation

Figure 4:Models default to exploit across epistemic states\.Each cell shows the percentage of free\-response generations for which a strategy is dominant, pooled across five languages\. Green indicates acceptable strategies and red indicates unacceptable strategies\.Exploitis the most frequent dominant strategy in 23 of 24 model×\\timesstate cells, including 59% and 52% of responses underUncertainandRevised, respectively, where it is unacceptable\. Multi\-label strategy presence is reported in Figure[11](https://arxiv.org/html/2610.40930#A2.F11)\.In this section, we characterize models’ FRG outputs by their dominant strategy \(Figure[4](https://arxiv.org/html/2610.40930#S3.F4)\)\. Averaged across models,exploitis the dominant strategy in 83%, 59%, and 52% of responses underEstablished,Uncertain, andRevised\. Models do shift toward acceptable strategies as evidence weakens \(e\.g\., dominantelicitrises from 7% to 18% and 21%\), but far too little:exploitremains the most frequent dominant strategy in 23 of 24 model×\\timesstate cells, the sole exception being Opus\-4\.7 underRevised\(elicit36% vs\. 34%\)\. Exploitation is also near\-universal as a component of responses, present in 90%, 91%, and 86% of responses across states \(Figure[11](https://arxiv.org/html/2610.40930#A2.F11)\), often alongside a question or hedge\. Models are therefore not selecting strategies at random, but defaulting to the same policy,exploit, regardless of whether it remains appropriate\.

Table 2:FRG performance, Avg\. across epistemic states and languages w/ 95% CI\. SC = strategy calibration, R = rubric, UA = user acceptability\.To characterize the models’ generations at a finer grain, human annotators coded free\-response failures into the seven foil types introduced in §[2](https://arxiv.org/html/2610.40930#S2)\. The dominant error type shifts systematically with epistemic state \(χ2\\chi^\{2\}test of independence,p<\.001p<\.001\):over\-personalizationis most common underEstablished\(31\.2%\),failure\-to\-probeunderUncertain\(38\.8%\), andstalenessunderRevised\(25\.1%\)\. Each reflects the same fixed exploit\-first policy, expressed differently depending on context: models over\-commit when preferences are already settled, fail to seek missing evidence when they are not, and continue acting on evidence that has since been contradicted\. This same pattern ofexploitusage holds across model families and languages, though it is more pronounced in open\-weight models \(error rate, i\.e\.,1−SC1\-\\mathrm\{SC\}, is 15\.5 pp higher than in closed\-weight models;p<\.001p<\.001\) and in Hindi and Indonesian \(4\.1 pp higher than in the other three languages;p<\.001p<\.001\)\.

### 3\.4Strategy Calibration Predicts Response Quality Across Epistemic States

We next examine whether strategy calibration translates into higher\-quality and more acceptable responses\. Table[2](https://arxiv.org/html/2610.40930#S3.T2)reports strategy calibration \(SC\), rubric\-based execution quality \(R\), and user acceptability \(UA\) for each model, pooled across all three epistemic states\. SC correlates positively with both R and UA within each state:Established\(r=0\.71r=0\.71,r=0\.69r=0\.69\),Revised\(r=0\.67r=0\.67,r=0\.74r=0\.74\), andUncertain\(r=0\.57r=0\.57,r=0\.61r=0\.61\)\. These correlations are slightly weaker inUncertain, however, they remain positive\. This weaker association may reflect differences in user needs: responses that are not maximally appropriate for the epistemic state may nevertheless be acceptable to users seeking a strictly direct answer\. Taken together, the results suggest that strategy calibration is generally associated with stronger execution and greater user satisfaction across various personalized contexts\.

## 4Characterizing the Failure

§[3](https://arxiv.org/html/2610.40930#S3)showed that the collapse in strategy calibration \(SC\) reflects a generation failure, where models fail toselectthe appropriate strategy during generation\. This failure in models also meaningfully degrades response quality i\.e\. declines in rubric \(R\) and user acceptability \(UA\) scores\.

### 4\.1Prompting Fails to Induce State\-Conditional Adaptation

We evaluate five lightweight prompting interventions across eight models and five languages: \(1\)explicit strategy instructionsdetailing strategy definitions and use cases; \(2\)oracle preference memory, providing structured user preference histories; \(3\)chain\-of\-thought \(CoT\)[Wei et al\. \(2022\)](https://arxiv.org/html/2610.40930#bib.bib12), step\-by\-step reasoning about the current epistemic state before generating a response; \(4\)few\-shot examplesdemonstrating strategy usage in utterances across epistemic states; and \(5\)best\-of\-N sampling, sampling model \(N=5\) generations and selecting best \(highest SC, R, UA score\)222These interventions cover distinct ways of augmenting in\-context information at inference time: explicit rules, ground\-truth labels, reasoning, demonstrations, and sampling budget\. While not an exhaustive sweep of all prompt designs, we test a broad set of contextualization strategies\.\. Prompt templates and per\-metric results are detailed in Appendix[B\.7](https://arxiv.org/html/2610.40930#A2.SS7)

Figure 5:Inference\-time interventions fail to induce state\-conditional adaptation\.Bars depict mean SC change relative to baseline across eight models and five languages; error bars denote 95% bootstrap CIs\. Gains underUncertainandRevisedlead to performance losses underEstablished, indicating a global shift toward caution rather than selective adaptation based on epistemic state\.Figure[5](https://arxiv.org/html/2610.40930#S4.F5)shows no intervention method consistently improves SC across all epistemic states\. In many interventions \(Explicit instructions, CoT, Few\-shot\), gains underUncertainandRevisedtrade off with losses underEstablished\. For instance,few\-shot examplesboost SC by 4\.7% underRevisedpreferences but degrade it by 3\.4% underEstablished\. Moreover, we find that rather than selective, context\-dependent adaptation, prompting often shifts a global caution parameter, forcing models to use the strategyhedgeindiscriminately\. These context\-rich interventions also do not show improvements in execution\-based metrics\. Evenfew\-shot examples, the strongest intervention on SC, produce only modest gains in R \(\+0\.12\) and UA \(\+1\.31 pp\) underUncertainandRevised, while reducing both underEstablished: they shift which strategy is nominally selected far more than the quality of the resulting response\. Meanwhile,oracle preference memoryandbest\-of\-five samplingyield almost no effect on any metric\.

This pattern is consistent across model families and languages, with slightly larger effect sizes in closed\-weight models and English/Romance languages relative to Hindi and Indonesian \(Appendix[B\.7](https://arxiv.org/html/2610.40930#A2.SS7)\)

### 4\.2Understanding and Repairing the Selection Failure

As inference\-time interventions do not provide consistent improvements, we further analyze models’ representations of two concepts:epistemic stateandstrategy selectionto better understand how to build more calibrated models\.

We utilize linear and nonlinear probes and causal ablation on four open\-weight models \(Llama\-3\.1\-8B, Gemma\-3\-12B, Qwen3\-8B, OLMo\-3\-7B\)\. Averaged across models, both epistemic state and the upcoming strategy to be used in generation are linearly decodable333Concepts are decodable from Layer 20\.well above chance \(80\.5%; chance 33\.3%\) and \(70\.0%; chance 20\.0%\)\. Yet we find limited evidence the two are connected: \(1\) cosine similarity between state and strategy probe weight vectors is≈0\.03\\approx 0\.03, close to orthogonal; \(2\) probe accuracy on the two concepts correlates weakly \(ϕ≈0\.015\\phi\\approx 0\.015\); \(3\) and nonlinear MLP probes show subspace overlap of only≈0\.021\\approx 0\.021above a random floor\. Moreover, neither ablating the decoded epistemic state direction nor amplifying it produces any significant shift in strategy calibration in FRG:−0\.3\-0\.3% vs\+1\.2\+1\.2%\. This pattern holds across multiple layers \(20\-24\) where epistemic state first becomes decodable and across multiple languages\. See §[B\.8](https://arxiv.org/html/2610.40930#A2.SS8)for full experiment details\. Although these results do not rule out a causal pathway for epistemic state elsewhere in the model, the combination of the behavioral gap between recognition and unaided selection \(§[3\.2](https://arxiv.org/html/2610.40930#S3.SS2)\) and the failure of inference\-time interventions to improve generation \(§[4\.1](https://arxiv.org/html/2610.40930#S4.SS1)\) suggests: epistemic state is accessible to the model, but not reliably connected to strategy selection during generation\.

Figure 6:Targeting the state\-strategy link improves performance beyond generic fine\-tuning, across every model\.SC, R, and UA for each model across the full training progression: untrained, generic fine\-tuning \(SFT, DPO, GRPO\), and GRPO with the targeted reward \(\+Link, \+Link\+CF\)\. Generic fine\-tuning yields modest gains; adding the Link and Link\+CF rewards yield the largest improvement for every model\.Our analysis suggests that models encode epistemic state but do not use it to select strategies\. A standard reward that scores only the final response can improve behavior without fixing this: the model may learn surface cues that correlate with the right strategy in training data but fail to transfer\. We therefore supervise the link itself, encouraging the strategy the model internally represents to match the distribution humans prefer for the current epistemic state \(Table[1](https://arxiv.org/html/2610.40930#S2.T1)\)\. We target a distribution rather than a single strategy, since humans accept several strategies per state\. We hypothesize that supervising the strategy the model internally represents, rather than only its final output, reduces reliance on surface cues and improves generalization to unseen personas\.

Formally, our base reward combines strategy calibration \(SC\), user acceptability \(UA\), and rubric quality \(R, normalized to 0 to 1\) with equal weight:rbase=13​\(SC\+UA\+R\)r\_\{\\text\{base\}\}=\\tfrac\{1\}\{3\}\(\\text\{SC\}\+\\text\{UA\}\+\\text\{R\}\)\. To directly supervise the alignment between internal states and strategy selection, we introduce two auxiliary rewards:LinkandLink\+CF\.

Lethhbe the model’s hidden representation of its response \(residual\-stream activation at 70% model depth, mean\-pooled over the first eight generated tokens, by which point the model has committed to a strategy\), and letPϕ\(⋅∣h\)P\_\{\\phi\}\(\\cdot\\mid h\)be the strategy distribution decoded fromhhby our trained strategy probe\. Becausehhis computed from the response, it varies across rollouts\. TheLinkreward supervises agreement between this distribution and the human preference distributionPref⁡\(s\)\\mathrm\{Pref\}\(s\)\(Table[1](https://arxiv.org/html/2610.40930#S2.T1)\) under the ground\-truth epistemic statess:rLink=−KL\(Pϕ\(⋅∣h\)∥Pref\(s\)\)r\_\{\\textsc\{Link\}\}=\-\\mathrm\{KL\}\\big\(P\_\{\\phi\}\(\\cdot\\mid h\)\\,\\\|\\,\\mathrm\{Pref\}\(s\)\\big\)\. The probe is refit every 10 GRPO steps on the current policy’s rollouts, labeled by the strategy each rollout actually used \(§[B\.8](https://arxiv.org/html/2610.40930#A2.SS8)\)\. WhilerLinkr\_\{\\textsc\{Link\}\}aligns strategy selection under the active statess, it does not enforce counterfactual consistency across states\.Link\+CFtherefore adds a counterfactual term: for a randomly sampled alternate states′≠ss^\{\\prime\}\\neq s, we patchhhalong the probed state direction to obtainh′h^\{\\prime\}and reward agreement with human preferences unders′s^\{\\prime\}:rLink\+CF=rLink−KL\(Pϕ\(⋅∣h′\)∥Pref\(s′\)\)r\_\{\\textsc\{Link\+CF\}\}=r\_\{\\textsc\{Link\}\}\-\\mathrm\{KL\}\\big\(P\_\{\\phi\}\(\\cdot\\mid h^\{\\prime\}\)\\,\\\|\\,\\mathrm\{Pref\}\(s^\{\\prime\}\)\\big\)\.444This direction was causally inert under passive intervention \(see above\);rLinkr\_\{\\textsc\{Link\}\}andrLink\+CFr\_\{\\textsc\{Link\+CF\}\}test whether reward\-based optimization can establish a causal pathway from it to strategy selection\.

The full training reward isrbase\+λ​rLinkr\_\{\\text\{base\}\}\+\\lambda\\,r\_\{\\textsc\{Link\}\}orrbase\+λ​rLink\+CFr\_\{\\text\{base\}\}\+\\lambda\\,r\_\{\\textsc\{Link\+CF\}\}, withλ=0\.1\\lambda=0\.1, computed once the full response is generated\. We compare both against SFT, DPO, and GRPO trained on the base reward alone\. Training data \(∼\\sim5K turns per language, 24K total\) is generated with our benchmark pipeline \(§[2\.3](https://arxiv.org/html/2610.40930#S2.SS3)\) using open\-weight models \(Kimi\-K2\.6, Qwen3\-235B, and DeepSeek\-V3\)\.555Additional training details in §[B\.8](https://arxiv.org/html/2610.40930#A2.SS8)\.We evaluate on a held\-out in\-distribution set and on our benchmark\.

Fig[6](https://arxiv.org/html/2610.40930#S4.F6)reports strategy calibration \(SC\), rubric quality \(R\), and user acceptability \(UA\) on our benchmark for every model across the full training progression, averaged over languages\. Generic fine\-tuning yields modest gains: GRPO, the strongest generic baseline, raises SC from 48\.0% to 55\.5% on average, and outperforms SFT and DPO for every model on R and UA and for all models but Qwen3\-8B on SC \(SC and UA: McNemar’s test,p<\.001p<\{\.001\}; R: Wilcoxon signed\-rank on conversation\-level mean differences,p<\.001p<\{\.001\}\)\. AddingLinkyields the largest single improvement, raising SC, R, and UA for every model: averaged across models, SC rises from 55\.5% to 65\.5% \(\+10\.0 pp\), R from 2\.80 to 3\.04, and UA from 43\.5% to 48\.0%\.Link\+CFperforms comparably toLinkon average \(SC 66\.5%, R 3\.05, UA 48\.8%\), with the largest additional gains for Qwen3\-8B \(\+4 pp SC, \+5 pp UA\)\.

We also observe large*generalization gaps*: the drop in each metric between the held\-out in\-distribution set and the benchmark’s unseen personas,Gap=Metricin\-dist−Metricbenchmark\\text\{Gap\}=\\text\{Metric\}\_\{\\text\{in\-dist\}\}\-\\text\{Metric\}\_\{\\text\{benchmark\}\}\(in\-distribution results in Table[33](https://arxiv.org/html/2610.40930#A2.T33)\)\.Linknarrows this gap substantially: from 11\.5 to 6\.5 pp on SC, from 0\.61 to 0\.48 on R, and from 7\.5 to 5\.0 pp on UA \(McNemar’s test,p<\.001p<\{\.001\}\), withLink\+CFperforming comparably \(6\.5 pp, 0\.50, 5\.2 pp\)\. In Hindi, for example, the SC gap narrows from 18 pp under GRPO to 11 pp underLink\.

Together, these results indicate that adaptive personalization is learnable: supervising the model’s internal strategy representation toward state\-conditional human preferences improves both in\-distribution behavior and generalization to personas never seen during training\.

## 5Related Work

#### Personalization benchmarks\.

Existing benchmarks show that models struggle with preference retrieval and long\-context recall\([Salemi et al\., 2024](https://arxiv.org/html/2610.40930#bib.bib30);[Zhao et al\., 2025a](https://arxiv.org/html/2610.40930#bib.bib27);[Jiang et al\., 2025b](https://arxiv.org/html/2610.40930#bib.bib45);[Jiang et al\., 2024](https://arxiv.org/html/2610.40930#bib.bib36);[Wu et al\., 2025](https://arxiv.org/html/2610.40930#bib.bib37);[Maharana et al\., 2024](https://arxiv.org/html/2610.40930#bib.bib29)\), often treating implicit\-versus\-explicit expression as the main difficulty\([Jiang et al\., 2025b](https://arxiv.org/html/2610.40930#bib.bib45);[Guo et al\., 2026](https://arxiv.org/html/2610.40930#bib.bib28)\); HorizonBench\([Li et al\., 2026](https://arxiv.org/html/2610.40930#bib.bib25)\)shows dynamic preferences are harder to track\. None label epistemic state independently of preference content, so they cannot separate retrieval failures from failures to adaptstrategyas a preference’s reliability changes \(§[3\.1](https://arxiv.org/html/2610.40930#S3.SS1)\)\.

#### Clarification and preference elicitation\.

Prior work studies when to ask clarifying questions\([Rao and Daumé III, 2018](https://arxiv.org/html/2610.40930#bib.bib1);[Aliannejadi et al\., 2019](https://arxiv.org/html/2610.40930#bib.bib2);[Zamani et al\., 2020](https://arxiv.org/html/2610.40930#bib.bib3);[Kuhn et al\., 2022](https://arxiv.org/html/2610.40930#bib.bib4);[Deng et al\., 2023](https://arxiv.org/html/2610.40930#bib.bib5);[Zhang et al\., 2024](https://arxiv.org/html/2610.40930#bib.bib6);[Zhang and Choi, 2025](https://arxiv.org/html/2610.40930#bib.bib7)\)and trains models to ask or elicit preferences\([Li et al\., 2025](https://arxiv.org/html/2610.40930#bib.bib8);[Andukuri et al\., 2024](https://arxiv.org/html/2610.40930#bib.bib9);[Chen et al\., 2025](https://arxiv.org/html/2610.40930#bib.bib10);[Zhang et al\., 2025](https://arxiv.org/html/2610.40930#bib.bib11)\);[Chen et al\. \(2025\)](https://arxiv.org/html/2610.40930#bib.bib10)note that LLMs often guess user intent rather than ask, echoing our exploit\-first finding\. These works address ambiguity within a single request; we condition strategy on the reliability of apersistingpreference, which also requires recognizing stale preferences and recalibrating rather than re\-asking\.

#### Strategy, calibration, and representations\.

Strategy has been treated as a learnable construct in emotional support\([Liu et al\., 2021](https://arxiv.org/html/2610.40930#bib.bib33)\), persuasion\([Wang et al\., 2019](https://arxiv.org/html/2610.40930#bib.bib34)\), and negotiation\([He et al\., 2018](https://arxiv.org/html/2610.40930#bib.bib32)\), though never conditioned on evidence reliability; we also draw on exploit–explore framing in conversational recommendation\([Christakopoulou et al\., 2016](https://arxiv.org/html/2610.40930#bib.bib19);[Radlinski and Craswell, 2017](https://arxiv.org/html/2610.40930#bib.bib20)\)and dialogue grounding\([Clark, 1996](https://arxiv.org/html/2610.40930#bib.bib13);[Traum, 1994](https://arxiv.org/html/2610.40930#bib.bib15)\)\. We measure behavioral rather than probabilistic calibration\([Kadavath et al\., 2022](https://arxiv.org/html/2610.40930#bib.bib38)\), and our recognition–generation gap echoes findings that models evaluate better than they generate\([West et al\., 2024](https://arxiv.org/html/2610.40930#bib.bib39)\)\. That decodable information can lack causal leverage follows probing work\([Ravichander et al\., 2021](https://arxiv.org/html/2610.40930#bib.bib41);[Elazar et al\., 2021](https://arxiv.org/html/2610.40930#bib.bib42);[Hewitt and Liang, 2019](https://arxiv.org/html/2610.40930#bib.bib40)\)\. Closest to our reward,[Wu and Tang \(2026\)](https://arxiv.org/html/2610.40930#bib.bib43)derive GRPO advantages from representation directions; we instead supervise the internal strategy distribution against human\-preference targets, with a counterfactual patching term\.

## 6Conclusion

We introduced adaptive personalization under epistemic uncertainty: conditioning conversational strategy on how reliable a preference currently is\. All eight models evaluated exhibit the same failure pattern: strategy calibration drops by over 35 percentage points once preferences become uncertain or revised\. This failure is one of generation rather than recognition: models identify appropriate strategies far more reliably than they produce them\. Targeted training that supervises internal strategy representations improves strategy calibration, demonstrating that this capability is both distinct and learnable\. Ultimately, true personalization in language models demands more than user context retrieval, it requires learning to adapt conversational behavior dynamically as human preferences naturally evolve\.

## Use of Generative AI

In this work, we used generative AI tools for the following tasks that require disclosure\.\(1\) Synthetic data generation:the benchmark’s personas, session plans, user turns, and candidate responses were generated by LLMs \(Claude Sonnet 4\.6 and Gemini 3\.1 Pro\), and the training corpus was generated with the same pipeline using open\-weight models \(Kimi\-K2\.6, Qwen3\-235B, DeepSeek\-V3\), as described in §[2\.3](https://arxiv.org/html/2610.40930#S2.SS3)and Appendix[B\.8](https://arxiv.org/html/2610.40930#A2.SS8)\.\(2\) Data annotation and evaluation:an LLM jury \(Claude Opus 4\.6, Gemini 3 Flash, GPT\-5\.5\) labeled user turns for epistemic\-state construction, and LLM judges scored strategy calibration, rubric quality, and user acceptability of model responses; these labels were validated against human annotators \(Appendix[A\.6](https://arxiv.org/html/2610.40930#A1.SS6)\)\.\(3\) Feedback on methodology and interpretation:we used an LLM assistant to review the manuscript, which provided feedback on the experimental design \(e\.g\., the definition of strategy calibration, the construction of the training reward, and the choice of evaluation sets\) and on the interpretation and internal consistency of results\.\(4\) Implementation:LLM coding assistants \(Claude and Codex\) were used to help implement figure/plot generation and parts of the training code\. We did not use generative AI tools to formulate mathematical claims or proofs, and translation is not applicable: dialogues in each language were generated natively rather than translated, and human validation was performed by native or professionally fluent annotators\.

We have reviewed all AI\-assisted work\. All reported results come from our experimental outputs; all numbers were checked by the authors against these outputs\. Synthetic data quality was verified through a human validation study with 15 annotators across five languages \(Appendix[A\.6](https://arxiv.org/html/2610.40930#A1.SS6)\)\. LLM\-assisted code was reviewed and tested by the authors\.

## References

- Alchourrónet al\.\(1985\)C\. E\. Alchourrón, P\. Gärdenfors, and D\. MakinsonOn the logic of theory change: partial meet contraction and revision functions\.Journal of Symbolic Logic50\(2\),pp\. 510–530\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p2.1)\.
- Aliannejadiet al\.\(2019\)M\. Aliannejadi, H\. Zamani, F\. Crestani, and W\. B\. CroftAsking clarifying questions in open\-domain information\-seeking conversations\.InProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Andukuriet al\.\(2024\)C\. Andukuri, J\. Fränken, T\. Gerstenberg, and N\. D\. GoodmanSTaR\-GATE: teaching language models to ask clarifying questions\.InConference on Language Modeling,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2025\)M\. Chen R\. Sunet al\.Learning to clarify: multi\-turn conversations with action\-based contrastive self\-training\.InInternational Conference on Learning Representations,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Christakopoulouet al\.\(2016\)K\. Christakopoulou, F\. Radlinski, and K\. HofmannTowards conversational recommender systems\.InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,pp\. 815–824\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Clark and Brennan \(1991\)H\. H\. Clark and S\. E\. BrennanGrounding in communication\.\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p2.1)\.
- Clark \(1996\)H\. H\. ClarkUsing language\.Cambridge University Press\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p2.1),[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Denget al\.\(2023\)Y\. Deng, L\. Liao, L\. Chen, H\. Wang, W\. Lei, and T\. ChuaPrompting and evaluating large language models for proactive dialogues: clarification, target\-guided, and non\-collaboration\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Deyet al\.\(2025\)P\. Dey, Y\. Khanter, A\. Bothra, J\. Zhao, and E\. FerraraCan llms express personality across cultures? introducing culturalpersonas for evaluating trait alignment\.arXiv preprint arXiv:2506\.05670\.Cited by:[§2\.3](https://arxiv.org/html/2610.40930#S2.SS3.p4.1)\.
- Elazaret al\.\(2021\)Y\. Elazar, S\. Ravfogel, A\. Jacovi, and Y\. GoldbergAmnesic probing: behavioral explanation with amnesic counterfactuals\.InTransactions of the Association for Computational Linguistics,Vol\.9,pp\. 160–175\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Gärdenfors \(1988\)P\. GärdenforsKnowledge in flux: modeling the dynamics of epistemic states\.MIT Press\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p2.1)\.
- Geet al\.\(2024\)T\. Ge, X\. Chan, X\. Wang, D\. Yu, H\. Mi, and D\. YuScaling synthetic data creation with 1,000,000,000 personas\.arXiv preprint arXiv:2406\.20094\.Cited by:[Figure 7](https://arxiv.org/html/2610.40930#A1.F7),[§2\.3](https://arxiv.org/html/2610.40930#S2.SS3.p2.1)\.
- Guoet al\.\(2026\)Q\. Guo, Y\. Li, Y\. Liu, and B\. HooiTowards realistic personalization: evaluating long\-horizon preference following in personalized user\-llm interactions\.arXiv preprint arXiv:2603\.04191\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p3.1),[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px1.p1.1)\.
- Hansson \(1999\)S\. O\. HanssonA textbook of belief dynamics\.Kluwer Academic Publishers\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p1.1)\.
- Harryet al\.\(2026\)T\. Harry, I\. C\. Ngong, C\. Nweke, Y\. Feng, and J\. NearBeyond fixed psychological personas: state beats trait, but language models are state\-blind\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 26440–26468\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p3.1)\.
- Heet al\.\(2018\)H\. He, D\. Chen, A\. Balakrishnan, and P\. LiangDecoupling strategy and generation in negotiation dialogues\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2333–2343\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Hewitt and Liang \(2019\)J\. Hewitt and P\. LiangDesigning and interpreting probes with control tasks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Huet al\.\(2021\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLora: low\-rank adaptation of large language models\.arXiv preprint arXiv:2106\.09685\.Cited by:[§B\.8](https://arxiv.org/html/2610.40930#A2.SS8.SSS0.Px2.p1.1)\.
- Janget al\.\(2026\)H\. Jang, J\. Lee, and K\. S\. ChoiSciZoom: a large\-scale benchmark for hierarchical scientific summarization across the llm era\.arXiv preprint arXiv:2603\.16131\.Cited by:[§2\.3](https://arxiv.org/html/2610.40930#S2.SS3.p4.1)\.
- Jianget al\.\(2025a\)B\. Jiang, Z\. Hao, Y\. Cho, B\. Li, Y\. Yuan, S\. Chen, L\. Ungar, C\. J\. Taylor, and D\. RothKnow me, respond to me: benchmarking llms for dynamic user profiling and personalized responses at scale\.arXiv preprint arXiv:2504\.14225\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p3.1)\.
- Jianget al\.\(2025b\)B\. Jiang, Y\. Yuan, M\. Shen, Z\. Hao, Z\. Xu, Z\. Chen, Z\. Liu, A\. R\. Vijjini, J\. He, H\. Yu,et al\.Personamem\-v2: towards personalized intelligence via learning implicit user personas and agentic memory\.arXiv preprint arXiv:2512\.06688\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p3.1),[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px1.p1.1)\.
- Jianget al\.\(2024\)H\. Jiang, X\. Zhang, X\. Cao, C\. Breazeal, D\. Roy, and J\. KabbaraPersonaLLM: investigating the ability of large language models to express personality traits\.InFindings of the association for computational linguistics: NAACL 2024,pp\. 3605–3627\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px1.p1.1)\.
- Kadavathet al\.\(2022\)S\. Kadavath, T\. Conerly, A\. Askell,et al\.Language models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Kuhnet al\.\(2022\)L\. Kuhn, Y\. Gal, and S\. FarquharCLAM: selective clarification for ambiguous questions with generative language models\.arXiv preprint arXiv:2212\.07769\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Lakoff \(1973\)G\. LakoffHedges: a study in meaning criteria and the logic of fuzzy concepts\.Journal of philosophical logic2\(4\),pp\. 458–508\.Cited by:[§2\.1](https://arxiv.org/html/2610.40930#S2.SS1.p2.1)\.
- Liet al\.\(2025\)B\. Z\. Li, A\. Tamkin, N\. Goodman, and J\. AndreasEliciting human preferences with language models\.InInternational Conference on Learning Representations,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2026\)S\. S\. Li, B\. Paranjape, K\. Oktar, Z\. Ma, G\. Zhou, L\. Guan, N\. Zhang, S\. Park, L\. Chen, D\. Yang,et al\.Horizonbench: long\-horizon personalization with evolving preferences\.arXiv preprint arXiv:2604\.17283\.Cited by:[§A\.2](https://arxiv.org/html/2610.40930#A1.SS2.SSS0.Px3.p1.1),[§2\.3](https://arxiv.org/html/2610.40930#S2.SS3.p2.1),[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2021\)S\. Liu, C\. Zheng, O\. Demasi, S\. Sabour, Y\. Li, Z\. Yu, Y\. Jiang, and M\. HuangTowards emotional support dialog systems\.InProceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing \(volume 1: Long papers\),pp\. 3469–3483\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Maharanaet al\.\(2024\)A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. FangEvaluating very long\-term conversational memory of llm agents\.Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics\.Note:arXiv:2402\.17753Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px1.p1.1)\.
- Radlinski and Craswell \(2017\)F\. Radlinski and N\. CraswellA theoretical framework for conversational search\.Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval,pp\. 117–126\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Rao and Daumé III \(2018\)S\. Rao and H\. Daumé IIILearning to ask good questions: ranking clarification questions using neural expected value of perfect information\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Ravichanderet al\.\(2021\)A\. Ravichander, Y\. Belinkov, and E\. HovyProbing the probing paradigm: does probing accuracy entail task relevance?\.Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Salemiet al\.\(2024\)A\. Salemi, S\. Mysore, M\. Bendersky, and H\. ZamaniLaMP: when large language models meet personalization\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Note:arXiv:2304\.11406Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px1.p1.1)\.
- Traum \(1994\)D\. R\. TraumA computational theory of grounding in natural language conversation\.InProceedings of the AAAI Fall Symposium on Planning for Interagent Communication,Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p2.1),[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Wanget al\.\(2019\)X\. Wang, W\. Shi, R\. Kim, Y\. Oh, S\. Yang, J\. Zhang, and Z\. YuPersuasion for good: towards a personalized persuasive dialogue system for social good\.InProceedings of the 57th annual meeting of the association for computational linguistics,pp\. 5635–5649\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§B\.7](https://arxiv.org/html/2610.40930#A2.SS7.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2610.40930#S4.SS1.p1.1)\.
- Westet al\.\(2024\)P\. West, X\. Lu, N\. Dziri,et al\.The generative ai paradox: what it can create, it may not understand\.International Conference on Learning Representations \(ICLR\)\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Wuet al\.\(2025\)D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. YuLongMemEval: benchmarking chat assistants on long\-term interactive memory\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2410\.10813Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px1.p1.1)\.
- Wu and Tang \(2026\)R\. Wu and R\. TangWhen reward hacking rebounds: understanding and mitigating it with representation\-level signals\.arXiv preprint arXiv:2604\.01476\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px3.p1.1)\.
- Xiaoet al\.\(2026\)J\. Xiao, X\. Yu, C\. Wang, W\. Zheng, X\. Lin, K\. Liu, H\. Ding, Y\. Zhang, W\. Wang, F\. Feng,et al\.AlpsBench: an llm personalization benchmark for real\-dialogue memorization and preference alignment\.arXiv preprint arXiv:2603\.26680\.Cited by:[Table 16](https://arxiv.org/html/2610.40930#A1.T16.4.8.1.1)\.
- Zamaniet al\.\(2020\)H\. Zamani, S\. Dumais, N\. Craswell, P\. Bennett, and G\. LueckGenerating clarifying questions for information retrieval\.InProceedings of The Web Conference 2020,pp\. 418–428\.Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Zhang and Choi \(2025\)M\. J\.Q\. Zhang and E\. ChoiClarify when necessary: resolving ambiguity through interaction with LMs\.InFindings of the Association for Computational Linguistics: NAACL 2025,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2025\)M\. J\.Q\. Zhang, W\. B\. Knox, and E\. ChoiModeling future conversation turns to teach LLMs to ask clarifying questions\.InInternational Conference on Learning Representations,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2024\)T\. Zhang, P\. Qin, Y\. Deng,et al\.CLAMBER: a benchmark of identifying and clarifying ambiguous information needs in large language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Cited by:[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2025a\)S\. Zhao, M\. Hong, Y\. Liu, D\. Hazarika, and K\. LinDo llms recognize your preferences? evaluating personalized preference following in llms\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 15888–15931\.Cited by:[§1](https://arxiv.org/html/2610.40930#S1.p3.1),[§5](https://arxiv.org/html/2610.40930#S5.SS0.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2025b\)Z\. Zhao, C\. Vania, S\. Kayal, N\. Khan, S\. B\. Cohen, and E\. YilmazPersonalens: a benchmark for personalization evaluation in conversational ai assistants\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 18023–18055\.Cited by:[§A\.2](https://arxiv.org/html/2610.40930#A1.SS2.SSS0.Px3.p1.1),[§2\.3](https://arxiv.org/html/2610.40930#S2.SS3.p2.1)\.
- Zolloet al\.\(2025\)T\. Zollo, A\. Siah, N\. Ye, L\. Li, and H\. NamkoongPersonalllm: tailoring llms to individual preferences\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 66949–66971\.Cited by:[§2\.3](https://arxiv.org/html/2610.40930#S2.SS3.p4.1)\.

## Appendix ABenchmark Construction

This appendix provides full construction details for the benchmark introduced in §[2](https://arxiv.org/html/2610.40930#S2), including persona generation, conversation and response construction, dataset statistics, human validation, and comparison to prior personalization resources\. See Figure[7](https://arxiv.org/html/2610.40930#A1.F7)for an overall understanding of our benchmark construction pipeline\.

![Refer to caption](https://arxiv.org/html/2610.40930v1/benchmark.png)Figure 7:Benchmark construction pipeline\.\(1\) A persona and multi\-session life\-arc plan are generated from a PersonaHub[Ge et al\. \(2024\)](https://arxiv.org/html/2610.40930#bib.bib47)seed, including demographics, friction traits, and scheduled drift events\. \(2\) At each turn, a user simulator produces a persona\-conditioned utterance; an LLM jury \(three models, majority vote\) annotates it for user need, drift signal strength, and per\-key preference revelation\. \(3\) Per\-turn labels are combined with the deterministic drift schedule into an epistemic state \(established, uncertain, or revised\), which determines the acceptable and non\-acceptable strategies for that turn via the human\-validated acceptability sets \(Table[1](https://arxiv.org/html/2610.40930#S2.T1)\)\. \(4\) Candidate responses are generated for two acceptable \(acceptable strategy, LLM jury\-verified\) and two foil\-type slots and ranked pairwise, in\-character as the persona; a turn is retained only if every acceptable response outranks every foil, with up to six regeneration attempts\. Full construction details are provided in Appendix[A](https://arxiv.org/html/2610.40930#A1)\.### A\.1Strategy Taxonomy

Our framework defines five response strategies, drawn from the conversational\-recommender literature and extended for long\-horizon dialogue\.*Elicit*asks a question to learn a preference the assistant does not have any information on; it is the right move when the relevant preference has never been stated\.*Confirm*asks an open\-ended question to gather preference information that is currently ambiguous or uncertain based on current context\.*Exploit*acts directly on a known preference, and is what delivers value once a belief is both held and current\.*Hedge*presents multiple options without committing, deferring the choice to the user\.*Recalibrate*treats contradictory evidence as new information; explicitly updates and ceases relying on the superseded preference, rather than verifying whether it still holds\. Crucially, the mapping from epistemic state to strategy is one\-to\-many rather than one\-to\-one: acceptability is a coarse filter, not a single right answer\. When a preference isEstablished, only*exploit*is calibration\-valid—asking about a settled preference isover\_probing, and recalibrating a belief that never moved isepistemic\_overreach\. When the user has signalled uncertainty,*exploit*becomesepistemic\_overreachand the valid set shifts to*elicit*,*confirm*, and*hedge*\. When a preference has beenRevised, acting on the stale belief is astaleness\_error, and*recalibrate*joins the valid set\. Finally, for revised preference states, the valid set of strategies is:*elicit*,*confirm*,*hedge*, and*recalibrate*\. Within each of these regions multiple strategies remain defensible, and which one is*best*turns on user\-specific factors the framework cannot enumerate—which is precisely what the persona\-grounded simulator’s ranking is there to capture\.

Table 3:Strategies and examples of well\-calibrated usage, using a persona who has stated a preference for savory over sweet food\. Each example is acceptable in the listed state and would constitute a named error in at least one other\.
### A\.2Persona and Data Construction

#### Pipeline overview\.

The benchmark is produced by a three\-role LLM pipeline\. Each role is given a strictly partitioned view of the generative state, which is what makes the oracle labels trustworthy: no role ever sees the answer key for the judgment it is asked to make\.

*\(1\) Persona and plan generator\.*Given a PersonaHub seed, this role produces demographics, a short profile, a long life narrative, two friction traits, and a structured preference profile \(§[A\.2](https://arxiv.org/html/2610.40930#A1.SS2)\)\. It then produces a*session plan*: a life arc of 10–14 sessions, each with an occasion, an outline of conversational beats, a primary user need, and a set of preferences the user is expected to reveal in that session\. Drift events are authored at this stage and attached to the arc\.

*\(2\) User simulator\.*This role plays the persona turn by turn\. It sees the persona, the friction traits, the current session’s occasion and beats, summaries of all prior sessions, and a per\-turn revelation directive drawn from the plan\. It does*not*see the candidate assistant responses or the acceptability labels\. It is instructed to express friction behaviorally rather than narrate it, to keep turns to 15–40 words by default \(150 words hard maximum\), and to deliver drift signals as concrete behavioral detail rather than meta\-commentary\.

*\(3\) LLM jury\.*Three judges from three labs independently annotate each user turn for user need, personalization relevance, drift signal strength, and per\-key preference revelation\. Labels are resolved by majority vote; a preference key is admitted only if at least two of the three judges name it, which suppresses single\-judge over\-labelling\. The jury sees the transcript and the persona’s preference*keys*, but never the persona’s current ground\-truth preference*values*— the correctness half of the epistemic state is computed deterministically from the plan, not judged\.

The jury’s per\-turn labels are combined with the deterministic drift schedule into a*joint state*, and a rule\-based acceptability table maps that joint state onto the acceptable and non\-acceptable strategy sets for the turn\. Oracle labels are therefore constructed, not inferred: the epistemic state is based oncertainty, where certainty comes from the jury’s revelation level \(hinted/revised\.

Abridged prompt templates for each role are given in Figures[8](https://arxiv.org/html/2610.40930#A1.F8)–[10](https://arxiv.org/html/2610.40930#A1.F10)\.

User simulator \(abridged\)```
You are roleplaying as a fictional user named {name} in a conversation
with an AI assistant. Stay fully in character.

WHO YOU ARE: {persona_summary}

YOUR FRICTION TRAITS (these MUST surface naturally -- through BEHAVIOR,
not articulation):
  1. {friction_1}   2. {friction_2}

CRITICAL -- RESISTANCE BEHAVIORS:
 (a) When the assistant names your emotional state, deflect ~40% of the time.
 (b) DO NOT produce a clean self-diagnosis. SHOW friction through behavior.
 (c) If apologetic/guilty: UNDER-share, hint, redirect, minimize.

LENGTH VARIANCE:
 - Default 1-2 sentences (15-40 words). Never exceed 150 words.

DRIFT SIGNAL DELIVERY:
 - Concrete behavioral details, no meta-narration.
 - Wrong: "I’ve been reading in the mornings, which is unusual for me."
 - Right: "I read most of it before six this morning."
 - Don’t connect signals to causes unless drift visibility is "explicit".

NATURALNESS: Do NOT narrate or analyze your own psychology. Say the
concrete thing; leave the cause and the meaning implicit.

SESSION OCCASION: {occasion}    BEATS: {outline_beats}
PRIOR SESSIONS: {prior_summaries}
```

Figure 8:User simulator prompt \(abridged\)\.Candidate response generator \(abridged\)```
You are generating a single AI assistant response. Generate a NATURAL,
well-formed response that demonstrates the target strategy faithfully.

CONVERSATION SO FAR: {conversation_so_far}
USER’S LATEST MESSAGE: {user_utterance}

LABELED CONTEXT (for your understanding only;
do not mention these labels):
 - User need: {user_need}
 - Personalization relevance: {personalization_relevance}
 - Drift signal strength in this turn: {drift_signal_strength}
 - Epistemic state per relevant preference: {epistemic_state_summary}
 - Asserted persona style preferences: {persona_style_summary}

TARGET STRATEGY: {strategy}
STRATEGY DEFINITION: {strategy_definition}

REQUIREMENTS:
 - The response must clearly use the target strategy.
 - Honor the asserted persona style preferences by DEMONSTRATING them --
   do NOT explicitly comment on the user’s style or register.
 - Do not mention strategies, frameworks, or labels.
```

Figure 9:Candidate response generation prompt \(abridged\)\. The foil variant is identical except that the flawed strategy is required to be the*primary shape*of the response — visible in the first sentence and not removable as a wrapper — so that foils differ from acceptable responses in strategy rather than in fluency\.Jury judge \(abridged\)```
You are an annotator labeling a user’s turn in a multi-session AI
assistant conversation. Be precise and grounded in the text.

PERSONA: {persona_summary}
PERSONA FRICTIONS: 1. {friction_1}  2. {friction_2}
PERSONA’S RELEVANT PREFERENCE KEYS (only label revelations on these):
  {available_preference_keys}
CONVERSATION SO FAR (most recent turns): {recent_conversation}
PRIOR SESSION SUMMARIES: {prior_summaries}
USER’S LATEST UTTERANCE: {user_utterance}

ANNOTATE THE USER’S LATEST UTTERANCE:
  user_need                in {productivity, learning, social_support, autonomy}
  personalization_relevance in {high, medium, low, none}
  drift_signal_strength    in {none, weak, strong}
  preference_revelation    per key in {hinted, asserted, revised}
  preference_drift_expressed  per key, boolean
```

Figure 10:Jury judge prompt \(abridged\)\. Note that the persona’s current preference*values*are withheld: judges are given evidence, never the answer key\.
#### Persona seeding and diversity\.

Personas are seeded from PersonaHub and selected via farthest\-point sampling to maximize diversity\. We start from a pool of 20,000 PersonaHub occupation\-and\-role seeds\. Each seed is represented as a TF–IDF vector over word unigrams and bigrams \(English stopwords removed, minimum document frequency 2\), and seeds are selected greedily: having selected a setSS, the next seed is the pool element maximizing its cosine distance to its nearest neighbour inSS,arg⁡maxx∉S​mins∈S⁡\(1−cos⁡\(x,s\)\)\\arg\\max\_\{x\\notin S\}\\min\_\{s\\in S\}\\big\(1\-\\cos\(x,s\)\\big\), initialized from a random seed\. Sampling is run once per language and draws 50 seeds per language, giving 250 personas in total\. Seeds are filtered before sampling to remove degenerate entries: seeds shorter than five tokens, seeds naming a specific public figure, and seeds describing a role that cannot plausibly hold consumer preferences across the life domains in Table[4](https://arxiv.org/html/2610.40930#A1.T4)\.

Personas are generated independently per language rather than translated, with country sampled from a language\-appropriate distribution \(e\.g\., for Spanish: Mexico, Spain, Colombia, Argentina, Peru, Chile, Venezuela\) and with names, institutions, media, food and sports references localized to that country\. Ethnicity and background are varied*within*country so that language does not collapse onto a single cultural profile\.

#### Preference and interaction\-style attributes\.

Each persona is augmented with structured preference profiles spanning 20 life domains\([Zhao et al\., 2025b](https://arxiv.org/html/2610.40930#bib.bib46)\)and interaction\-style dimensions spanning 13 categories\([Li et al\., 2026](https://arxiv.org/html/2610.40930#bib.bib25)\)\. Full lists are provided in Table[4](https://arxiv.org/html/2610.40930#A1.T4)and Table[5](https://arxiv.org/html/2610.40930#A1.T5)\. A persona is not assigned every domain: we sample 6–12 of the 20 life domains and 4–8 of the 13 interaction\-style categories per persona, and then populate every attribute within each selected category\. This keeps profiles realistically partial \(a median persona holds roughly 45% of the full attribute space\) while guaranteeing enough coverage for multi\-session preference tracking\. Sampling is constrained so that every persona retains at least one*gating*interaction\-style dimension \(interactivity level, challenge openness, or validation need\) with a value that can be violated, which is what makes style\-mismatch foils constructible for every persona\.

Table 4:Life domains used for persona preference attributes\.Table 5:Interaction\-style categories used for persona attributes\. Each category expands into 5–6 dimensions; e\.g\.*advice delivery*covers depth preference, abstraction level, interactivity level, question frequency, checklist inclusion, and prioritization\.
#### Drift\-event mechanism\.

A drift event is authored at plan time, not at conversation time, which is what allows the epistemic state to be an oracle rather than an inference\. Each persona is assigned a number of drift events sampled so that roughly 10% of personas receive none \(a deliberate no\-drift control condition\)\. Each event specifies: the preference key\(s\) affected, apreference\_deltarecording the explicit\{old, new\}values, the session index after which the change takes effect, a*drift visibility*\(none, latent, visible\) governing how overtly the user signals it, and a*drift magnitude*— subtle \(the old value remains valid, e\.g\. an addition\), moderate, or significant \(the old value is now contradicted\) — sampled at 0\.2/0\.3/0\.5\. Events are grounded in the persona’s life narrative and preferentially add constraints or remove options \(e\.g\.*no dietary restrictions*→\\rightarrow*low sodium*\) so that a large shift remains natural\.

Two constraints on plan generation make drift testable rather than incidental\. First, drift is assigned to a domain the persona’s life arc actually exercises\. Second, the arc is revised so that at least three post\-drift sessions have occasions in the drifted domain; without this, a drifted preference is simply never the topic again and no assistant behavior can depend on it\.

The epistemic\-state transition is then computed, not labelled\. For a preference keykkat sessionss, the current value iscurrent​\(k,s\)\\texttt\{current\}\(k,s\), i\.e\. the base value overwritten by the delta of every event withoccurs\_after\_session<s\\texttt\{occurs\\\_after\\\_session\}<s\. Certainty comes from the jury’s highest observed revelation level forkk, collapsed to binary:hintedmaps to*uncertain*, whileassertedorconfirmed\_explicitmaps to*certain*\. Correctness iscurrent​\(k,s\)\\texttt\{current\}\(k,s\)matched against the value the transcript has actually established\. This yields the cellscertain\_correct\(*established*\),uncertain\_correctanduncertain\_incorrect\(*uncertain*\), andcertain\_incorrect\(*revised*\): the assistant’s grounded belief is now superseded\. A key returns tocertain\_correctas soon as the user states the new value, so*revised*marks precisely the window in which acting on the old value is wrong and detectable\.

#### Example persona\.

Table[6](https://arxiv.org/html/2610.40930#A1.T6)shows a complete example persona, including demographic attributes, life narrative, friction traits, and a sample of assigned preference and interaction\-style attributes\.

Table 6:Example persona \(persona\_0001, English\)\.

### A\.3Conversation Generation

#### Session plans\.

Each conversation follows a session plan specifying an outline, conversational beats, and occasion for each of 10–14 sessions\. The plan is generated in one pass from the persona, as a life arc rather than a list of independent episodes: sessions are ordered so that the sequence reads as time passing, and each session carries an occasion \(the concrete reason the user opens the conversation\), 3–6 outline beats, a primary user need drawn from \{productivity, learning, social support, autonomy\}, a drift visibility, and a list of preferences the user is expected to reveal, each with an intended revelation level\.

Beats map onto turns loosely rather than one\-to\-one\. The user simulator receives the beats as a gist of what the session should cover, not as a checklist, and each generated user turn records the beat indices it addressed; a session is complete when the beats are exhausted or the simulator produces a natural closing signal\. Each session has a target turn count with a soft floor at0\.67×0\.67\\timestarget and a hard cap, which prevents both two\-turn stubs and unbounded sessions\. Realized sessions average 8\.5 benchmark turns\.

Turn\-level revelation is driven by the plan rather than re\-inferred: the per\-session revelation list is converted into a per\-turn directive for the simulator, and the plan’s intended level is trusted as the label*only if*the jury independently observes the key in the resulting text\. An intendedassertedorconfirmed\_explicitrevelation that the jury cannot corroborate is downgraded tohinted\. This asserted\-corroboration gate is what keeps the certainty half of the epistemic state honest\.

#### Language generation\.

Each language is generated independently by the full pipeline; no conversation is translated\. We generate a separate 50\-persona set per language with country\-appropriate demographics, then run plan generation and conversation generation end\-to\-end in that language\. Concretely, only the*dialogue*prompts \(user simulator, acceptable and foil response generation, style and probe backfills\) are switched to the target language; the persona, plan, session summaries, jury prompts and all oracle labels remain in English\. This gives natively\-authored, culturally\-grounded dialogue — Mexican Spanish personas reference Primera Plus and CDMX, not translated equivalents — while keeping the label space identical across languages, so cross\-lingual comparisons measure the model and not the annotation scheme\.

Quality control per language is threefold: a language\-purity check rejecting non\-target\-language turns; the same jury and gating pipeline applied identically across languages; and a per\-language human validation study \(§[A\.6](https://arxiv.org/html/2610.40930#A1.SS6)\) using native or fluent speakers\.

#### Example multi\-session conversation\.

Table[7](https://arxiv.org/html/2610.40930#A1.T7)shows an excerpt of a single persona’s conversation across three sessions, illustrating a transition from an established preference to an uncertain signal to an explicit revision\.

Table 7:Example multi\-session conversation excerpt \(persona\_0001, keylife\_domains\.hotels\.star\_rating\_preference\), showing an established→\\touncertain→\\torevised transition\. The drift event moves the preference from3\_starto4\_starwith moderate magnitude\.

### A\.4Response and Foil Construction

#### Foil taxonomy\.

Table[8](https://arxiv.org/html/2610.40930#A1.T8)provides full definitions and an example response for each of the seven foil types introduced in §[2\.3](https://arxiv.org/html/2610.40930#S2.SS3)\. All examples are verbatim from the English portion of the benchmark\.

Table 8:Full foil taxonomy with example responses\.
#### Slot assignment\.

Foil type is neither sampled uniformly nor fixed per epistemic state: it is*gated*by the conditions under which each error is genuinely an error\. At each turn the acceptability table is evaluated over the turn’s*salient*keys — the preferences the turn is actually about, rather than every key accumulated so far — which partitions the five strategies into an acceptable and a non\-acceptable set\. The two acceptable slots are drawn from the acceptable set; when only one acceptable strategy exists, the slot is filled with a second, independently generated phrasing of that strategy\.

The two foil slots are drawn from the set of*instantiable*error candidates, where each error type carries a precondition:

- •Stalenessrequires auncertainorrevisedsalient key of moderate or significant drift magnitude whose value the user has*not*restated this turn\.
- •Epistemic overreachrequires a salient uncertain key with personalization relevance high or medium\.
- •Over\-probingrequires aEstablishedsalient key with no drift signal\.
- •Need mismatchrequires a social\-support turn\.
- •Over\-personalizationrequires personalization relevance low or none plus at least one established preference to misapply\.
- •Failure to proberequires high or medium relevance and a salient key that is still unresolved\.
- •Style mismatchrequires an asserted interaction\-style preference that the chosen strategy can visibly violate\.

This gating is the main defense against spurious labels: an earlier version of the pipeline instantiated staleness on turns whose content had nothing to do with the drifted key, which produced foils that were not actually stale\. Requiring the drifted key to be salient removes that failure mode\. Two of the rarer types \(style mismatch, failure to probe\) are additionally reached by a post\-hoc backfill pass that swaps one over\-represented foil per eligible turn, capped at one per session, so that all seven types occur in every language\. The resulting distribution is reported in Table[12](https://arxiv.org/html/2610.40930#A1.T12); it is deliberately unbalanced, tracking how often each error is genuinely available, which is why we report macro\-averaged accuracy over foil types alongside micro\-averaged accuracy\.

#### Pairwise ranking and consistency check\.

Candidate responses are ranked pairwise by an LLM judge acting in\-character as the persona\. A turn is retained only if every acceptable response outranks every foil response; turns failing this check are regenerated, up to 6 attempts\. In the released benchmark, 68\.4% of turns pass on the first attempt, with a further 16\.1%, 7\.2%, 3\.6%, 1\.8% and 0\.9% passing on attempts two through six, for a cumulative retention of 98\.0%\. The remaining 2\.0% of turns fail all six attempts and are excluded from the benchmark\. Excluded turns are not deleted from the conversation: they are retained as continuity turns so that the session still reads as a coherent dialogue, but are flaggedis\_benchmark\_turn = falseand are never scored\. Continuity turns also cover turns for which no foil type is instantiable at all\. Across the corpus, 25,006 of 34,697 total turns \(72\.1%\) are benchmark turns; the remaining 9,691 are continuity turns\. In all cases the conversation is threaded forward on an*acceptable*response, so a foil never enters the dialogue history and errors cannot compound across turns\.

#### Model versions\.

Table[9](https://arxiv.org/html/2610.40930#A1.T9)lists the model used at each pipeline stage\. The jury deliberately spans three labs so that agreement is not an artifact of a single model family, and the eight evaluation models reported in §[3](https://arxiv.org/html/2610.40930#S3)are disjoint from the jury\.

Table 9:Models used at each benchmark construction stage\.

### A\.5Dataset Statistics

Table 10:Turns and conversations per language\. Conversations are sessions; each persona contributes a 10–14 session arc\.Benchmark turns are distributed across epistemic states as*established*23\.0%,*uncertain*42\.5%, and*revised*34\.5%, and this composition is stable across languages \(established 22\.2–24\.0%, uncertain 41\.8–43\.0%, revised 33\.2–35\.5%\)\. Turn\-level context labels are distributed as: user need — productivity 63\.7%, social support 17\.3%, learning 16\.3%, autonomy 2\.7%; personalization relevance — high 69\.4%, medium 25\.4%, low 4\.5%, none 0\.7%; drift signal strength — none 70\.5%, weak 11\.7%, strong 17\.8%\.

Table 11:Ground\-truth strategy distribution across the benchmark \(acceptable\-response labels, not model behavior\)\. Columns are percentages of acceptable response slots within each epistemic state; strategies outside the acceptable set for a state are never assigned\.Table 12:Foil\-type distribution across the benchmark \(n=50,012n=50\{,\}012foil slots\)\.The distribution is stable across languages: over\-probing ranges 30\.5–34\.0%, epistemic overreach 27\.0–30\.5%, and staleness 21\.2–22\.8% across the five languages, with the three rare types between 1\.1% and 3\.5% everywhere\. Foil availability is strongly state\-conditioned, as intended: within*revised*turns, 63\.4% of foils are staleness errors; within*uncertain*turns, 49\.5% are epistemic overreach; and within*established*turns, 55\.0% are over\-probing\.

### A\.6Human Validation

#### Annotator recruitment\.

We recruited 15 annotators, three per language, through an internal professional annotation platform\. Annotators were required to be native or professionally fluent in the target language and to have prior experience with dialogue annotation\. Each candidate completed a 10\-item qualification round drawn from the same distribution as the main task and scored against construction labels; a candidate qualified at≥\\geq80% agreement on the acceptability judgment\. Annotators received a written codebook with the five strategy definitions, the seven foil definitions, decision rules and contrastive worked examples, and completed a calibration discussion before beginning\. Each of the 500 items \(100 per language\) was annotated by all three annotators for that language, giving approximately 1,500 item\-level annotations\.

#### Validation instructions\.

Annotators assessed five aspects per item: conversation coherence and realism, epistemic\-state clarity, strategy\-label correctness, foil\-type accuracy, and response acceptability\. Items were presented with progressive disclosure: the persona summary, friction traits, prior\-session summaries and recent context were shown first, and the four candidate responses were revealed only after the annotator had committed to the state judgments, so that the responses could not leak the state label\. The four responses were shuffled per item\. The rubric was:

1. 1\.Conversation coherence and realism\.“Reading only the user’s side, does this read like a real person continuing a real relationship with an assistant — consistent with who they were in earlier sessions, and with plausible reasons for what they raise now?” Rated on a 3\-point scale \(fully coherent / minor lapse / breaks character\), with a free\-text field required for anything below the top rating\.
2. 2\.Epistemic\-state clarity\.“Based only on what the user has said across these sessions, how well does the assistant know the preference at issue in this turn?” Options: the user has clearly stated it and not changed it \(established\); the user has hinted at it or been ambiguous \(uncertain\); the user has previously stated one thing and has since signalled something different \(revised\); cannot tell\. Annotators were also asked to mark whether the current turn itself signals a change\.
3. 3\.Strategy\-label correctness\.For each of the four responses: “What conversational strategy does this response use?” Annotators selected one of the five strategies directly —*exploit*\(acts on what it already knows\),*elicit*\(asks an open\-ended question to gather missing information\),*confirm*\(verifies a tentative preference before acting\),*hedge*\(offers options while flagging uncertainty\), or*recalibrate*\(acknowledges a changed preference and updates accordingly\) — each accompanied by a one\-line definition in the annotator codebook\.
4. 4\.Foil\-type accuracy\.For each response judged unacceptable: “Which problem applies*first*?”, choosing from the seven foil types in a fixed priority order, or “no problem”\. Forcing a single first\-problem choice, rather than multi\-labelling, is what makes this measurable: several foils exhibit secondary flaws, and free multi\-selection collapses agreement\.
5. 5\.Response acceptability\.Acceptability is derived rather than asked directly: a response is acceptable iff the annotator marked “no problem” at step 4\. Annotators additionally selected the single best of the four responses, which yields the human MCQ accuracy reported below\.

Table 13:Inter\-annotator agreement by validation attribute \(Cohen’sκ\\kappabetween majority human label and LLM jury label, pooled across all five languages\)\.Table[13](https://arxiv.org/html/2610.40930#A1.T13)reports agreement separately for each of the five judgments above; agreement is highest for near\-binary, low\-inference judgments \(coherence, preference\-change signal\) and lowest for the more granular multi\-way judgments \(strategy label, foil type\), consistent with the acceptability\-derivedκ\\kappaof 0\.78 sitting between these extremes\.

Table 14:Human validation results by language\.κ\\kappais Cohen’sκ\\kappabetween the majority human label and the LLM jury label on the acceptability judgment; human MCQ accuracy is the rate at which the annotator’s best\-of\-four pick was an acceptable response\.Agreement is substantial and stable across languages, with the two lower\-resource languages \(Hindi, Indonesian\) trailing English by 0\.05–0\.06κ\\kappa\. Human MCQ accuracy of 92% sits above most evaluated models, confirming that the benchmark is vastly solvable and that the gap we report is a model limitation rather than label noise\.

#### Acceptable\-strategy partition search\.

Table[15](https://arxiv.org/html/2610.40930#A1.T15)reports the full ranked results of the two\-group partition search described in §[2\.2](https://arxiv.org/html/2610.40930#S2.SS2), for each epistemic state\. For each of the25−1−1=152^\{5\-1\}\-1=15distinct two\-group partitions of the five strategies, we fit a two\-rate model in which every strategy within a group shares a common preference rate, and score the partition by the residualχ2\\chi^\{2\}lack\-of\-fit,∑i\(oi−mg⁡\(i\)\)2/mg⁡\(i\)\\sum\_\{i\}\(o\_\{i\}\-m\_\{g\(i\)\}\)^\{2\}/m\_\{g\(i\)\}, wheremgm\_\{g\}is the mean observed count in groupgg\. Lower is better: the best partition is the one for which “preferred” and “dispreferred” are internally homogeneous\. In every state the best\-fitting partition recovers exactly the acceptable set marked in Table[1](https://arxiv.org/html/2610.40930#S2.T1)\.

Table 15:Top candidate partitions byχ2\\chi^\{2\}fit, per epistemic state\. Rank 1 is the best\-fitting partition\. The listed set is the*preferred*group; its complement is the dispreferred group\.

### A\.7Comparison to Prior Personalization Resources

Table[16](https://arxiv.org/html/2610.40930#A1.T16)compares our benchmark to prior personalization resources along the axes most relevant to our claims\.*Oracle state*means the resource ships a ground\-truth label for what the assistant is entitled to believe at each point, rather than only what the user prefers\.*Recognition / generation split*means the same items support both a forced\-choice and a free\-response evaluation, so that recognition and generation failures can be separated\. Only HorizonBench provides a comparable oracle label; unlike our epistemic\-state framework, it labels*when*a preference value changed rather than whether the assistant is currently entitled to act on it, hedge, or seek confirmation—the distinction our acceptable\-strategy sets operationalize\.

Table 16:Comparison to prior personalization benchmarks\.

## Appendix BResults

Here, we detail additional LLM results for our task\.

### B\.1Full Statistical Specification

#### Model specification\.

Strategy calibration \(SC\) by epistemic state is modeled via mixed\-effects logistic regression, with turns nested within conversations nested within personas:

SCi​j​k∼Bernoulli​\(pi​j​k\),logit​\(pi​j​k\)=β0\+β1​Statei​j​k\+uj\+vj⁡\(k\)\\text\{SC\}\_\{ijk\}\\sim\\text\{Bernoulli\}\(p\_\{ijk\}\),\\quad\\text\{logit\}\(p\_\{ijk\}\)=\\beta\_\{0\}\+\\beta\_\{1\}\\text\{State\}\_\{ijk\}\+u\_\{j\}\+v\_\{j\(k\)\}whereuj∼𝒩⁡\(0,σu2\)u\_\{j\}\\sim\\mathcal\{N\}\(0,\\sigma\_\{u\}^\{2\}\)is a random intercept for personajj, andvj⁡\(k\)∼𝒩⁡\(0,σv2\)v\_\{j\(k\)\}\\sim\\mathcal\{N\}\(0,\\sigma\_\{v\}^\{2\}\)a random intercept for conversationkknested within personajj\. State is dummy\-coded withEstablishedas the reference level\.

Table 17:Fixed\-effect estimates for the state effect on strategy calibration\.Table 18:Free\-response performance by language and epistemic state, averaged across the eight evaluated models\. SC = strategy calibration \(%\), R = rubric quality \(out of 5\), UA = user acceptability \(%\)\.
#### Per\-language estimates\.

Astate×language\\text\{state\}\\times\\text\{language\}interaction term does not significantly improve model fit \(likelihood\-ratio test,χ2​\(8\)=11\.3\\chi^\{2\}\(8\)=11\.3,p=\.185p=\.185\), indicating the shape of the collapse is consistent across languages\. Table[19](https://arxiv.org/html/2610.40930#A2.T19)reports the state effect fit separately within each language\.

Table 19:Per\-language state effect estimates\.Fit separately within each language; all coefficients significant atp<\.001p<\.001\.
#### Per\-model breakdown\.

Table[20](https://arxiv.org/html/2610.40930#A2.T20)reports free\-response strategy calibration by state for each of the eight evaluated models, underlying the closed\-vs\.\-open\-weight margin reported in §[3\.1](https://arxiv.org/html/2610.40930#S3.SS1)\(4\.2pp Established, 16\.9pp Uncertain, 25\.3pp Revised\)\.

Table 20:Free\-response strategy calibration \(%\) by model and epistemic state\.

### B\.2Recognition–Generation Gap: Full Specification

#### Model specification\.

The method×\\timesstate interaction is fit via mixed\-effects logistic regression predicting correctness from method \(MCQ vs\. FRG\), state, and their interaction, with random intercepts for persona and conversation nested within persona \(reference levels: MCQ, Established\)\.

Table 21:Fixed\-effect estimates for the method×\\timesstate interaction\.Table 22:MCQ strategy calibration \(%\) by model and epistemic state\.

### B\.3Oracle\-Strategy FRG: Full Results

In the oracle\-strategy condition \(Table[23](https://arxiv.org/html/2610.40930#A2.T23)\), the model is given the acceptable strategy explicitly \(e\.g\., “respond usinghedge”\) and asked to generate a response accordingly, isolating execution from selection\. Models comply with the supplied strategy on 95% of turns overall \(range 91\-94% across the four strategies tested beyondexploit\)\.

Table 23:Unaided vs\. oracle\-strategy performance by epistemic state\.Unaided values match §3\.1 and Table[28](https://arxiv.org/html/2610.40930#A2.T28)\.#### Execution quality by strategy, not just by state\.

To confirm the state\-invariance of oracle\-strategy execution is not an artifact of some strategies being intrinsically easier than others, Table[24](https://arxiv.org/html/2610.40930#A2.T24)reports rubric quality broken down by the supplied strategy\.

Table 24:Oracle\-strategy execution quality \(Rubric, Acc/5\) by supplied strategy\.Compliance rate is the fraction of turns on which the model produced a response matching the supplied strategy\.

### B\.4Generation Strategy Distribution: Full Breakdown

Figure 11:Strategy presence in free\-response generations\.Percentage of responses containing each strategy \(multi\-label: a response can use several strategies, so rows need not sum to 100%\), pooled across five languages\. Exploitation appears in nearly all responses across states, frequently alongsideelicitorhedge\. The strict and lenient scores in Table[27](https://arxiv.org/html/2610.40930#A2.T27)are computed from these tags\.#### Foil\-type contingency table\.

Table[25](https://arxiv.org/html/2610.40930#A2.T25)reports the full distribution underlying the reportedχ2\\chi^\{2\}test of independence \(χ2​\(12\)=1332\.7\\chi^\{2\}\(12\)=1332\.7,p<\.001p<\.001\)\.

Table 25:Foil\-type distribution \(%\) by epistemic state\.
#### Per\-language foil\-type breakdown\.

Table[26](https://arxiv.org/html/2610.40930#A2.T26)reports the dominant foil type underUncertain\(the state showing the largest cross\-language variation\) for each language\.

Table 26:Dominant foil type underUncertain, by language\.

### B\.5Sensitivity to Strategy\-Calibration Scoring Rule

Because model responses often combine strategies \(Figure[11](https://arxiv.org/html/2610.40930#A2.F11)\), we recompute SC under two alternative rules: astrictrule \(only acceptable strategies present\) and alenientrule \(any acceptable strategy present\)\. The collapse persists under lenient scoring: mean SC falls from 90\.0% \(Established\) to 49\.1% \(Uncertain\) and 57\.4% \(Revised\), gaps modestly narrower than under dominant scoring \(40\.9 vs\. 46\.2 pp and 32\.6 vs\. 35\.2 pp\)\. Strict SC is near zero outsideEstablished, as nearly all responses contain some exploitation, and only 39\.5% underEstablished, reflecting that models append questions or hedges at a similar rate in every state \(Figure[11](https://arxiv.org/html/2610.40930#A2.F11)\), consistent with a state\-invariant policy \(§3\.3\)\. Under lenient scoring, the recognition–generation gap underEstablishedlargely disappears, while the gaps underUncertainandRevisedremain large \(33 and 26 pp\)\. Lenient scoring credits strategy stacking regardless of state, so we treat it as an upper bound rather than a primary metric \(Table[27](https://arxiv.org/html/2610.40930#A2.T27)\)\.

Table 27:Strategy calibration \(%\) under alternative scoring rules\.Strict: only acceptable strategies present;Dom\.: dominant strategy acceptable \(main metric\);Len\.: any acceptable strategy present\. By construction, Strict≤\\leqDom\.≤\\leqLen\.
### B\.6Strategy Calibration and Response Quality: Full Data

#### Per\-model, per\-state data underlying within\-state correlations\.

Table[28](https://arxiv.org/html/2610.40930#A2.T28)reports the raw SC, R, and UA values for each model and epistemic state used to compute the correlations reported in §[3\.4](https://arxiv.org/html/2610.40930#S3.SS4)\.

Table 28:SC, R, and UA by model and epistemic state\.
#### Note on statistical power\.

Each within\-state correlation is computed across eight models \(n=8n=8\); at this sample size, correlation estimates carry wide confidence intervals and should be interpreted as descriptive rather than as precisely\-powered inferential estimates\. We report them because the direction and relative ordering across states \(strongest under Established and Revised, weaker under Uncertain\) is consistent and theoretically motivated, not because individual coefficients should be treated as tightly estimated\.

### B\.7Intervention Prompt Templates

For each intervention, the model receives the personaPP, conversation historyCC, and current user turnuuexactly as in the unaided condition \(§[2\.4](https://arxiv.org/html/2610.40930#S2.SS4)\), with the following addition prepended or appended as noted\.

#### \(1\) Explicit strategy instructions\.

Prepended to the system prompt:

> When responding, choose one of the following five conversational strategies based on how reliable the user’s stated preference currently is: \-Exploit: act directly on a preference that is clearly established and uncontradicted\. \-Elicit: ask an open\-ended question when you have no information about the relevant preference\. \-Confirm: verify a preference that has been stated but remains tentative or ambiguous\. \-Hedge: offer options without fully committing, when evidence is mixed or uncertain\. \-Recalibrate: explicitly acknowledge and update when new evidence contradicts a previously stated preference\. Select the strategy that best matches the current state of the user’s preference, then respond accordingly\.

#### \(2\) Oracle preference memory\.

Prepended before the conversation history, a structured summary generated from the benchmark’s oracle labels:

> Here is a summary of what you currently know about this user’s preferences: \- \{preference\_key\}:established\-\-\- the user has stated this clearly and it has not been contradicted\. \- \{preference\_key\}:uncertain\-\-\- you have limited or ambiguous evidence about this\. \- \{preference\_key\}:revised\-\-\- the user has indicated this preference has changed from what was previously stated\. Use this summary to guide how confidently you act on each preference\.

#### \(3\) Chain\-of\-thought \(CoT\)\.

Appended after the user turn, following[Wei et al\. \(2022\)](https://arxiv.org/html/2610.40930#bib.bib12):

> Before responding, think step by step: \(1\) What preference is relevant to this turn? \(2\) How reliable is the evidence for that preference right now\-\-\-established, uncertain, or revised? \(3\) Given that, what conversational strategy is appropriate? Then provide your response\.

#### \(4\) Few\-shot examples\.

Three worked examples prepended before the conversation history, one per epistemic state, in the same format:

> Example \(established preference\): User: \[earlier turn stating a clear preference\] User \(current turn\): \[unrelated follow\-up\] Assistant \(exploit\): \[confident response acting on the preference\] Example \(uncertain preference\): User: \[ambiguous or partial signal about a preference\] Assistant \(elicit/confirm/hedge\): \[response managing the uncertainty\] Example \(revised preference\): User: \[statement contradicting an earlier stated preference\] Assistant \(recalibrate\): \[response explicitly updating\]

#### \(5\) Best\-of\-five sampling\.

No prompt modification; five independent generations are sampled using temperature=0\.7 for the same\(P,C,u\)\(P,C,u\), and the response maximizing an equally\-weighted sum of SC, R, and UA \(as scored by the LLM jury / user simulator, §[2\.3](https://arxiv.org/html/2610.40930#S2.SS3)\) is retained\.

Table 29:Closed\-weight vs\. open\-weight intervention effects, averaged across all five interventions and all three epistemic states\.Table 30:Intervention effects by language, averaged across all five interventions and all three epistemic states\.Table 31:Change in SC, R, and UA relative to baseline, by intervention and epistemic state\.Averaged across eight models and five languages\.

### B\.8Training Details

#### Training and held\-out data construction\.

We generate a training corpus using the same three\-role pipeline as our benchmark \(§[2\.3](https://arxiv.org/html/2610.40930#S2.SS3)\) using open\-weight models \(Kimi\-K2\.6, Qwen3\-235B, and DeepSeek\-V3\), but from a disjoint set of 144 personas per language \(720 total across five languages\), entirely non\-overlapping with the 250 personas used in our benchmark \(§[2](https://arxiv.org/html/2610.40930#S2)\)\. This yields approximately 5k conversational turns per language \(24k total\), spanning the same controlled epistemic\-state transitions \(established, uncertain, revised\)\. We hold out 10% of this training corpus \(persona\-disjoint from the training split\) as an*in\-distribution*evaluation set, used to compute training\-time performance and the in\-distribution half of the OOD generalization gap reported in §[4\.2](https://arxiv.org/html/2610.40930#S4.SS2)\. The 25,006\-turn benchmark itself \(§[2](https://arxiv.org/html/2610.40930#S2)\) is never used during training and serves as the out\-of\-distribution evaluation set for all reported generalization results\.

#### Base model fine\-tuning \(SFT, DPO\)\.

All fine\-tuning uses LoRA\([Hu et al\., 2021](https://arxiv.org/html/2610.40930#bib.bib35)\)with rankr=16r=16,α=32\\alpha=32, dropout 0\.05, applied to query, key, value, and output projection matrices\. SFT trains for 3 epochs with AdamW, learning rate2×10−52\\times 10^\{\-5\}, cosine schedule with 3% warmup, effective batch size 32 \(per\-device batch 4, gradient accumulation 8\) with acceptable responses from the training corpus as the prediction target\. DPO initializes from the SFT checkpoint, trains for 2 epochs, learning rate5×10−65\\times 10^\{\-6\},β=0\.1\\beta=0\.1, using acceptable\-vs\-foil response pairs from the training corpus as preference pairs\.

#### GRPO training \(base,Link,Link\+CF\)\.

All GRPO configurations initialize from the SFT checkpoint and use group sizeG=8G=8rollouts per prompt, learning rate1×10−61\\times 10^\{\-6\}, KL penalty coefficient0\.020\.02against the SFT reference policy, and PPO\-style clipping withϵ=0\.2\\epsilon=0\.2\. Each configuration trains for 800 policy update steps, with 64 prompts per batch \(512 total rollouts per step\)\. The base rewardrbase=13​\(SC\+UA\+R\)r\_\{\\text\{base\}\}=\\tfrac\{1\}\{3\}\(\\text\{SC\}\+\\text\{UA\}\+\\text\{R\}\)is computed via the open\-weight models used for training dataset generation on each rollout \(disjoint from the closed\-weight models used for benchmark generation/evaluation\)\. Since R scores range from 1 to 5, we normalize these values and scale to 0 to 1 to match SC and UA\.LinkandLink\+CFadd the targeted term atλ=0\.1\\lambda=0\.1\(§[4\.2](https://arxiv.org/html/2610.40930#S4.SS2)\), computed from the current policy’s hidden state at 70% model depth, mean\-pooled over the first eight generated tokens\.

#### Probe refitting during training\.

BecauseLinkandLink\+CFdepend on a strategy probePϕ\(⋅∣h\)P\_\{\\phi\}\(\\cdot\\mid h\)applied to the*current*policy’s hidden states, we refit both the state and strategy probes everyN=10N=10GRPO steps, using a fresh sample of 2,000 rollouts generated by the current policy, to keep the probe calibrated as the policy drifts during training\. Probes are logistic regression classifiers trained on frozen hidden\-state activations, following the same procedure as §[4\.2](https://arxiv.org/html/2610.40930#S4.SS2)\. This periodic refitting is necessary for the reward to remain well\-defined as training progresses\.

#### Counterfactual patching forLink\+CF\.

For each rollout, an alternate states′≠ss^\{\\prime\}\\neq sis sampled uniformly from the remaining two states\. We patchhhtowards′s^\{\\prime\}by adding the difference between the mean probe\-direction projections ofs′s^\{\\prime\}\- andss\-labeled activations \(estimated from the same 2,000\-rollout sample used for probe refitting\), scaled by a coefficientα=1\.0\\alpha=1\.0\(full mean\-difference magnitude\)\.

#### Compute\.

All experiments run on 4\-8 NVIDIA A100 80GB GPUs\. Table[32](https://arxiv.org/html/2610.40930#A2.T32)reports approximate compute hours per method, per model\.

Table 32:Approximate compute per training run, per model\. GRPO configurations are more expensive due to rollout generation\.Table 33:Per\-model results on the held\-out in\-distribution set, averaged across five languages\. SC = strategy calibration \(%\), R = rubric \(out of 5\), UA = user acceptability \(%\)\. Benchmark results are shown in Figure[6](https://arxiv.org/html/2610.40930#S4.F6); the difference is the generalization gap reported in §[4\.2](https://arxiv.org/html/2610.40930#S4.SS2)\.