ToMStories：解读合成叙事中的心智理论

# ToMStories: Interpreting Theory-of-Mind in Synthetic Narratives
Source: [https://arxiv.org/html/2610.22632](https://arxiv.org/html/2610.22632)
###### Abstract

Understanding people often requires inferring what they feel or want from how they behave rather than from what they state outright\. We ask whether language models can make these inferences, and how they do so\. We introduce Theory\-of\-Mind Stories \(ToMStories\), an evaluation dataset of ~6,000 short synthetically generated narrative vignettes, each paired with a multiple\-choice or open\-ended question about a character’semotion,fear,motivation, oremotional wound\. These psychological states are conveyedimplicitly\(showing\),explicitly\(telling\), or withconflictbetween dialogue and description\. Evaluating thirteen open\-weight models \(1B–72B parameters\), we find that the largest models match humans on multiple choice but not open\-ended questions\. Our analyses point to simplifications models make in both interpretation and generation of implicit psychological states, indicating that synthetic data can help us understand how models perform interpretation, but likely overestimates how well models interpret humans\.

## 1Introduction

Large language models \(LLMs\) are used today as conversational companions, requiring models to interpret and handle aspects of the user’s psychology, such as emotions and motives\. Reasoning about human psychology requires Theory\-of\-Mind \(ToM\) – the ability to infer the beliefs, feelings, and motivations of another person\([Premack and Woodruff, 1978](https://arxiv.org/html/2610.22632#bib.bib13);[Wimmer and Perner, 1983](https://arxiv.org/html/2610.22632#bib.bib28)\)\. Yet prior work shows that feelings, desires, and intentions have received relatively little attention in existing ToM benchmarks\([Wang et al\., 2025](https://arxiv.org/html/2610.22632#bib.bib27)\)\.

![Refer to caption](https://arxiv.org/html/2610.22632v1/title_figure.png)Figure 1:ToMStories Pipeline\.\(1\) For each vignette we sample a setting, relationship, and target psychological state, and \(2\) generateimplicit,explicit, andconflictedvariants\. \(3\) Models answer multiple choice and open\-ended questions about a character’s psychology\. Open\-ended answers are scored against correct answers by embedding\-similarity and an LLM judge\.Narrative can be a fruitful context to study understanding of these aspects of psychology because it often includes subtext and expresses human psychology naturally and realistically\.Subtextis meaning that is communicatedimplicitly, underlying theexplicitlanguage\. Consider the implicit example shown in Figure[1](https://arxiv.org/html/2610.22632#S1.F1)\. Maya’s fixed stare and intentional evening of her voice implies that she is upset\. Prior work has shown that narrative with psychological subtext like this can be challenging for models to interpret[Subbiah et al\. \(2024b\)](https://arxiv.org/html/2610.22632#bib.bib23)\.

We address this challenge with structured synthetic data that lets us study how models both interpret and generate subtext\. Ground\-truth interpretations of human\-written narrative are hard to establish and often subjective[Subbiah et al\. \(2024a\)](https://arxiv.org/html/2610.22632#bib.bib21);[Subbiah et al\. \(2025\)](https://arxiv.org/html/2610.22632#bib.bib22), so we synthetically generate vignettes that are designed to convey a specific psychological state, giving each a known answer\. Each scenario comes in three variants:implicit,explicit, andconflicted\(in which dialogue and description suggest different states as shown in Figure[1](https://arxiv.org/html/2610.22632#S1.F1)\)\. We evaluate thirteen open\-weight models on these vignettes, then seek to explain their interpretive performance through analyzing what indicator phrases they associate with different psychological states and how their internal representations capture implicit and explicit expression\. We focus on open\-weight models because they allow for this kind of analysis of internal representations\.

1. 1\.ToMStories, a dataset of synthetically generated narrative vignettes targeting emotions, fears, motivations, and emotional wounds with three variants, two evaluation modes, and human validation\. Benchmarking thirteen open\-weight models \(1B–72B\) shows the largest models match humans on multiple choice but fall short on open\-ended questions\.
2. 2\.Evidence that models learn a shared set of indicator phrases to communicate and infer human psychology\. We analyze the overlap in indicator phrases models associate with different psychological states and find that shared indicator phrases with the model used for vignette generation is the most significant predictor of model performance onimplicitToMStories\.
3. 3\.Revealing further simplifications models make in generating and interpreting subtext\.Through analysis of model representations, we find that models treat implicitness partially as muted emotion\. Through analysis of performance onconflictedToMStories, we observe that models bias toward dialogue in interpreting meaning, but models that balance dialogue and description more evenly are more accurate onimplicitToMStories\.

## 2Related Work

#### Theory of Mind in LLMs\.

ToM benchmarks for LLMs range from false\-belief adaptations\([Nematzadeh et al\., 2018](https://arxiv.org/html/2610.22632#bib.bib12);[Le et al\., 2019](https://arxiv.org/html/2610.22632#bib.bib8)\)to higher\-order and multi\-party belief tracking\([Wu et al\., 2023](https://arxiv.org/html/2610.22632#bib.bib29);[Kim et al\., 2023](https://arxiv.org/html/2610.22632#bib.bib7)\), causal templates over beliefs, desires, and percepts\([Gandhi et al\., 2023](https://arxiv.org/html/2610.22632#bib.bib4)\), narrative ToM linking personality to action\([Xu et al\., 2024](https://arxiv.org/html/2610.22632#bib.bib30)\), and broad suites of ToM abilities\([Chen et al\., 2024](https://arxiv.org/html/2610.22632#bib.bib2);[Shinoda et al\., 2025](https://arxiv.org/html/2610.22632#bib.bib19)\)\. Yet feelings, desires, and intentions remain underrepresented relative to beliefs\([Wang et al\., 2025](https://arxiv.org/html/2610.22632#bib.bib27);[Ma et al\., 2023](https://arxiv.org/html/2610.22632#bib.bib9)\)\. ToMStories targets this gap with narratives from which a wide range of psychological states must be inferred from subtext\.

#### Narrative understanding\.

Models struggle to infer character psychology from implicit language\([Subbiah et al\., 2024b](https://arxiv.org/html/2610.22632#bib.bib23);[Subbiah et al\., 2024a](https://arxiv.org/html/2610.22632#bib.bib21)\), to track character relationships\([Iyyer et al\., 2016](https://arxiv.org/html/2610.22632#bib.bib6)\), and, even at long context, to comprehend literature deeply\([Thai and Iyyer, 2025](https://arxiv.org/html/2610.22632#bib.bib25)\)\. Commonsense resources support inference about mental states and motivations\([Rashkin et al\., 2018](https://arxiv.org/html/2610.22632#bib.bib15);[Sap et al\., 2019](https://arxiv.org/html/2610.22632#bib.bib18)\), but over short event descriptions rather than narratives\.

#### Mechanistic interpretability\.

Probing\([Belinkov, 2022](https://arxiv.org/html/2610.22632#bib.bib1);[Hewitt and Liang, 2019](https://arxiv.org/html/2610.22632#bib.bib5)\)and steering\([Turner et al\., 2024](https://arxiv.org/html/2610.22632#bib.bib26);[Rimsky et al\., 2024](https://arxiv.org/html/2610.22632#bib.bib16);[Zou et al\., 2023](https://arxiv.org/html/2610.22632#bib.bib32)\)are standard tools for reading and manipulating model representations, and probes have decoded representations of characters’ beliefs\([Zhu et al\., 2024](https://arxiv.org/html/2610.22632#bib.bib31)\)and emotions\([Tak et al\., 2025](https://arxiv.org/html/2610.22632#bib.bib24)\)\. Most relevant,[Sofroniew et al\. \(2026\)](https://arxiv.org/html/2610.22632#bib.bib20)find emotion representations in Claude Sonnet 4\.5 that causally shape its outputs\.

## 3ToMStories Dataset

CategorynnExamplesCharacterWounds121infidelity, being bullied, growing up in the shadow of a successful siblingEmotions135anger, giddiness, remorseFears68being a burden to others, having no purpose, isolationMotivations95being accepted by others, proving someone wrong, protecting a loved oneSituationSettings254abandoned mine, hair salon, hotel roomRelationships48enemies, nanny and child, people who are datingTable 1:Thesaurus categories used in ToMStories\.Our goal is to generate short narrative vignettes that convey a character’s psychological state implicitly, while controlling both the scenario and the ground\-truth state \(pipeline example shown in Figure[1](https://arxiv.org/html/2610.22632#S1.F1)\)\. We draw scenarios and answers from the Thesaurus Description Database for writers222[https://onestopforwriters\.com/thesaurus](https://onestopforwriters.com/thesaurus)\. The thesauruses\([Puglisi and Ackerman, 2019](https://arxiv.org/html/2610.22632#bib.bib14)\)guide writers in character development\. We use only lists from the free tier and do not reproduce them in our repository; the resource should not be scraped or used commercially\., using itsemotions,fears,motivations, andemotional woundslists as ground\-truth answers and itssettingsandrelationshipslists to build scenarios \(see summary statistics and examples in Table[1](https://arxiv.org/html/2610.22632#S3.T1)\)\. Theemotionsspan a much wider range \(n=135n=135\) than common emotion taxonomies \(e\.g\.,n=28n=28in[Demszky et al\. \(2020\)](https://arxiv.org/html/2610.22632#bib.bib3)\)\. For example, the list includesgiddinessalongsideexcitementandhappiness\.Fearsare anxieties rooted in a character’s experiences,motivationsare goals a character is driven to pursue, andwoundsare ways a character has been deeply hurt in the past\.

To seed a vignette, we randomly sample a setting, a relationship between two characters, and a psychological state that one of the characters should be experiencing in the scene\. Claude Sonnet 4\.6 then writes three variants \(full prompts in our repository\):

- •Implicit\.The primary variant conveys the state only through subtext\. The model is asked towrite a scenario that does not explicitly mention the <psychological\-state\> but communicates that one of the characters is experiencing it, within the given setting and relationship\.
- •Explicit\.The state is expressed directly: the characteropenly and unambiguously shows they are experiencing <psychological\-state\> through explicit statements, visible physical reactions, and direct behavior, without using the answer’s wording\.
- •Conflicted\.To test whether models rely more on dialogue or description, a character’sspoken words point to one <psychological\-state\> while their described behavior and physical reactions reveal a different one\.

In each case, the model generates both the vignette and the appropriate corresponding question about the hidden state\. For multiple choice questions, we pair the correct answer with three random distractors of the same type inimplicitandexplicitvignettes, or the dialogue\-aligned and description\-aligned answers with two random distractors inconflictedvignettes\.

Table 2:Summary statistics for ToMStories, showing the number of generated vignettes by split and question type, and the average word count\.Table 3:Exampleimplicitvignettes for each question type, highlighting phrases carrying implicit meaning\.We generate 2,000 vignettes per variant, ensuring thatimplicitandexplicitvignettes are paired, sharing the same setting, relationship, and state, so they can be compared directly\. After filtering for refusals, answer leakage and keeping pairs intact, we report summary statistics in Table[2](https://arxiv.org/html/2610.22632#S3.T2)\. Training and validation splits contain an even balance of the four types of questions\. Table[3](https://arxiv.org/html/2610.22632#S3.T3)shows examples\.

To verify the vignettes are well\-formed, we confirm humans can interpret theimplicitandconflictedvariants in Section[3\.1](https://arxiv.org/html/2610.22632#S3.SS1)\. For theexplicitvariant, we verify the questions are more straightforward than theimplicitset by checking that even smaller models can answer them\. We find that all models 8B or larger that we test achieve≥\\geq93% multiple choice accuracy on theexplicitvariant\.

### 3\.1Human baselines

To measure human performance on the vignettes, we recruit four computer science PhD students333The annotators have no particular expertise in psychology or narrative, so we treat them as non\-expert readers; three of the four grew up or spent substantial time in non\-English\-speaking countries, and individual cultural and linguistic backgrounds may shape how people read some narratives\., who each answer 20 non\-overlapping multiple choice questions from theimplicitandconflictedvariants, balanced across the four question types \(interface in Appendix Figure[7](https://arxiv.org/html/2610.22632#A6.F7); results in Table[4](https://arxiv.org/html/2610.22632#S5.T4)and Figure[2](https://arxiv.org/html/2610.22632#S5.F2)\)\. Annotators avoid the distractor answers on more than 80% of the questions for both subsets, indicating that the vignettes are interpretable to human readers\. Perfect accuracy is not expected, since the vignettes are synthetic and distractors can be close to the correct answer by chance \(e\.g\.,suspicionandwariness\)\. For inter\-annotator agreement, an author of this paper additionally annotates 40 of these vignettes, producing substantial agreement \(Cohen’sκ=0\.73\\kappa=0\.73; 74% raw agreement\)\. The author’s ratings are excluded from human baselines in Table[4](https://arxiv.org/html/2610.22632#S5.T4)as the author has particular expertise in the task\. For the open\-ended task, more than a month after completing the multiple choice task, two of the same annotators each answer ~40 new questions \(78 responses in total\) that they had not seen in multiple choice form\.

## 4Methods

We first benchmark model performance on theimplicitvariant of ToMStories, and then explore how models are interpreting these hidden psychological aspects of characters through further experiments\.

### 4\.1Benchmarking

We test whether models can identify the correct answer from the four answer options \(multiple choice scoring\), as well as whether they can generate a reasonably close answer when not given answer options \(open\-ended scoring\), which is substantially more challenging[Röttger et al\. \(2024\)](https://arxiv.org/html/2610.22632#bib.bib17)\(full implementation details reported in Appendix[E](https://arxiv.org/html/2610.22632#A5)\)\.

#### Multiple choice scoring

For multiple choice questions, we compute accuracy based on whether a model selects the correct answer choice\.

#### Open\-ended scoring

For the open\-ended questions, model responses are free\-form text rather than constrained to a multiple choice selection\. We score these responses using two complementary methods: EmbSim and LLMScore\. We embed the model’s response and each of the four multiple choice answer options withall\-MiniLM\-L6\-v2444[https://huggingface\.co/sentence\-transformers/all\-MiniLM\-L6\-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)and compute two measures\. Open\-endedaccuracycounts a response as correct if its embedding is closest, by cosine similarity, to the correct option\.EmbSimis the cosine similarity between the response and the correct option, which captures how close a response is even when a distractor is closer\. For LLM\-based scoring \(LLMScore\), we use gpt\-4o\-mini as a judge to assign a Likert score from 0 to 4, indicating how similar the model’s responserris to the correct answer\.

#### Models

We test a variety of open\-source models from 1B to 72B parameters across the Llama, Qwen, Gemma, Phi, and DeepSeek model families \(full list shown in Table[4](https://arxiv.org/html/2610.22632#S5.T4)\)\. See Appendix[B](https://arxiv.org/html/2610.22632#A2)for hardware details\.

### 4\.2Generating indicator phrases

To study whether models have a shared understanding of how to communicate and interpret hidden psychological states, we ask each model tolist ten behavioral or physical cues a character might display if they are experiencing <psychological\-state\> but not talking about it openly\(full prompt in code repository\)\. We sample 100 states used in ToMStories \(25 per question type\) and prompt each of the 13 open\-weight models and Claude Sonnet 4\.6 to generate phrases for these states\. We refer to these phrases asindicator phrasesand show examples in Appendix Table[10](https://arxiv.org/html/2610.22632#A8.T10)\.

#### Overlapping indicator phrases\.

We want to determine whether models have a shared understanding of what indicator phrases are associated with specific psychological states, and because models can describe the same character behavior in multiple ways, we compare indicator phrases by meaning rather than wording\. After parsing and cleaning generated phrases \(full details shown in Appendix[G\.1](https://arxiv.org/html/2610.22632#A7.SS1)\), we obtain 13,798 indicator phrases, which we embed withall\-MiniLM\-L6\-v2and cluster with average\-linkage agglomerative clustering\. We select 0\.5 as the cosine distance threshold for clustering as a reasonable threshold for this type of task with good performance on manual inspection\. We expect this threshold is conservative as examples of clusters in Appendix Table[9](https://arxiv.org/html/2610.22632#A8.T9)demonstrate that phrases are coherently grouped \(e\.g\.,pauses excessivelyandpausing for longer than usualin the same cluster\) but some clusters are still separate that could likely be combined \(e\.g\., two different clusters focused onfidgeting\)\. We consider two phrases to*overlap*in meaning if they fall in the same cluster, and we demonstrate in Appendix[G\.2](https://arxiv.org/html/2610.22632#A7.SS2)that our primary conclusions arising from this definition are robust to different choices of clustering threshold\.

### 4\.3Embedding\-based interpretability

To probe model\-internal representations of character psychology, we extract hidden\-state activations from each layer of the evaluated transformer models\. For a model withLLlayers, leth\(ℓ\)​\(x\)h^\{\(\\ell\)\}\(x\)denote the hidden state at layerℓ∈\{1,…,L\}\\ell\\in\\\{1,\\ldots,L\\\}at the last token position of inputxx, where the input includes the vignette, question, and answer options\. For each narrative vignettevv, we extract\{h\(ℓ\)​\(v\)\}ℓ=1L\\\{h^\{\(\\ell\)\}\(v\)\\\}\_\{\\ell=1\}^\{L\}, yielding a sequence of representations that track the model’s encoding of the vignette\.

#### Linear probes

To identify which layers contribute most to the model’s ability to detect implicit psychological factors, we train linear probes on the per\-layer embeddings described above\. For each layerℓ\\ell, we fit a logistic regression classifierf\(ℓ\):ℝd→\{1,2,3,4\}f^\{\(\\ell\)\}:\\mathbb\{R\}^\{d\}\\rightarrow\\\{1,2,3,4\\\}that maps the hidden stateh\(ℓ\)​\(v\)h^\{\(\\ell\)\}\(v\)to a predicted answer index:

f\(ℓ\)​\(h\)=arg⁡maxk∈\{1,2,3,4\}​𝐖\(ℓ\)​h\+𝐛\(ℓ\)f^\{\(\\ell\)\}\(h\)=\\underset\{k\\in\\\{1,2,3,4\\\}\}\{\\arg\\max\}\\;\\mathbf\{W\}^\{\(\\ell\)\}h\+\\mathbf\{b\}^\{\(\\ell\)\}where𝐖\(ℓ\)∈ℝ4×d\\mathbf\{W\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{4\\times d\}and𝐛\(ℓ\)∈ℝ4\\mathbf\{b\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{4\}are learned parameters andddis the hidden state dimension\. Probes are trained using cross\-entropy loss with L2 regularization on the train split\. We report layer\-wise accuracy on the validation split\. For each model, we selectℓ∗\\ell^\{\*\}for further analysis as the layer at which the linear probe accuracy starts to plateau \(equations in Appendix[A\.1](https://arxiv.org/html/2610.22632#A1.SS1)\)\.

#### Steering vectors

To extract steering vectors, we first compute centroids for each question typeq∈\{wound,emotion,fear,motivation\}q\\in\\\{\\text\{\{wound\}, \{emotion\}, \{fear\}, \{motivation\}\}\\\}and for each variantc∈\{implicit,explicit\}c\\in\\\{\\text\{\{implicit\}, \{explicit\}\}\\\}:

μq,c\(ℓ∗\)=1\|Sq,c\|∑v∈Sq,ch\(ℓ∗\)\(v\)\\mu^\{\(\\ell\*\)\}\_\{q,c\}=\\frac\{1\}\{\|S\_\{q,c\}\|\}\\sum\_\{v\\in S\_\{q,c\}\}h^\{\(\\ell^\{\*\}\)\}\(v\)whereSq,cS\_\{q,c\}is the set of vignettes of typeqqunder variantcc\. Following the mean\-difference approach of[Sofroniew et al\. \(2026\)](https://arxiv.org/html/2610.22632#bib.bib20), we define the steering vector at layerℓ∗\\ell\*as

𝐬\(ℓ∗\)=μimplicit\(ℓ∗\)−μexplicit\(ℓ∗\)\\mathbf\{s\}^\{\(\\ell^\{\*\}\)\}=\\mu^\{\(\\ell^\{\*\}\)\}\_\{\\textit\{implicit\}\}\-\\mu^\{\(\\ell^\{\*\}\)\}\_\{\\textit\{explicit\}\}whereμimplicit\(ℓ∗\)\\mu^\{\(\\ell^\{\*\}\)\}\_\{\\textit\{implicit\}\}andμexplicit\(ℓ∗\)\\mu^\{\(\\ell^\{\*\}\)\}\_\{\\textit\{explicit\}\}are the implicit and explicit centroids, pooled across all question types\. This vector captures how the model’s representations of implicit and explicit vignettes differ\. Steering generation along it lets us interpret what that difference encodes\. To steer, we add the unit\-normalized vector𝐬^\(ℓ∗\)=𝐬\(ℓ∗\)/∥𝐬\(ℓ∗\)∥\\hat\{\\mathbf\{s\}\}^\{\(\\ell^\{\*\}\)\}=\\mathbf\{s\}^\{\(\\ell^\{\*\}\)\}/\\lVert\\mathbf\{s\}^\{\(\\ell^\{\*\}\)\}\\rVert, scaled by a coefficientα\\alpha, to the residual stream at layerℓ∗\\ell^\{\*\}at every position during generation:

h~\(ℓ∗\)​\(x\)=h\(ℓ∗\)​\(x\)\+α​𝐬^\(ℓ∗\)\\tilde\{h\}^\{\(\\ell^\{\*\}\)\}\(x\)=h^\{\(\\ell^\{\*\}\)\}\(x\)\+\\alpha\\,\\hat\{\\mathbf\{s\}\}^\{\(\\ell^\{\*\}\)\}whereα\>0\\alpha\>0steers toward more implicit expression andα<0\\alpha<0toward more explicit expression\. For each model, we tuneα\\alphausing gpt\-4o\-mini as a judge for coherence \(full details in Appendix[D](https://arxiv.org/html/2610.22632#A4)\)\.

## 5Results

We assess model performance on ToMStories, and then explore how models are interpreting implicit meaning in the vignettes\.

### 5\.1Benchmarking open models

Table 4:Performance on theimplicitToMStories\. Most Llama\-3\.2\-1B outputs do not follow the answer format\.In Table[4](https://arxiv.org/html/2610.22632#S5.T4), we benchmark open models of different sizes and families on both the multiple choice and open\-ended questions of theimplicitvignettes\. We include GPT\-4o\-mini as a reference of a closed foundation model to contextualize scores\. We see thaton the multiple choice questions, the largest models can match human\-level performance, with Llama\-3\.1\-70B achieving the best overall performance\. Most models have the strongest performance on emotion questions\. Interestingly, the DeepSeek reasoning\-native variants of Qwen\-2\.5\-7B and Llama\-3\.1\-8B have lower scores across the board than the baseline models, suggesting that reasoning tokens may not be helpful for these tasks and models learn to represent subtext during pretraining\. Given that performance plateaus for both humans and GPT\-4o\-mini, we expect the top scores represent roughly the saturation point for this data due to some answers being highly similar to distractors by chance \(e\.g\.,giving up a childandnavigating a changing family situation\)\.

Accuracy on the open\-ended questions is lower than humans for all models, with Qwen\-2\.5\-72B performing the best\. These results show thathumans are better than the models on the open\-ended task\(64% accuracy vs\. 59% for the best model\), which remains true for GPT\-4o\-mini \(53% accuracy\)\. Humans and models are similar on the open\-ended emotion and motivation vignettes, but humans perform roughly 10% better than models on wound and fear questions\.

Figure 2:Answer selections onconflictedToMStories\.We next report results on theconflictedvariant of ToMStories in Figure[2](https://arxiv.org/html/2610.22632#S5.F2)\. In this case, there is not one correct answer, but an answer implied more by the description in the vignette and an answer implied more by the dialogue between characters, as well as two randomly selected distractors\. We observe that models have strong performance at avoiding distractor answers \(combined width of the pink bars\), but thatmodels tend to choose more of the dialogue\-aligned answers than the human would\(dark pink bars\)\. The two Llama 70B models align most closely to the human breakdown, demonstrating a more even split between dialogue\-aligned and description\-aligned answers with a slight dialogue bias\.

Given that these measures of interpretive ability are conducted with synthetic data, we have to be cautious in what conclusions we draw from them and we examine possible explanations for the strong model performance onimplicitvignettes in the subsequent sections\.

### 5\.2Evidence of shared indicator phrases

Figure 3:Overlap between indicator phrases across models\.\(a\)For each state, the number of open models with at least one phrase overlapping with Claude’s\.\(b\)Per model, the percentage of states with at least one phrase in the same cluster as one of Claude’s\.\(c\)Example for the motivation statebeing accepted by others\. Claude’s highest overlapping phrase overlaps with 9 of the 13 open models \(which is a count for nine in panel A\)\.We test whether there may be a shared set of indicator phrases models use to communicate and interpret hidden psychological states by evaluating overlap between open model indicator phrases and those of the synthetic data generator, Claude\. In Figure[3](https://arxiv.org/html/2610.22632#S5.F3), we present this analysis\. In Panel C, we show an example of a Claude\-generated indicator phrase overlapping with other models’ indicator phrases on a particular psychological state\. In Panel A, we consider the most\-overlapping phrase that Claude generated for each state, and count the number of other models that also generated that phrase\. We see that Claude phrases have no overlap with other models for only 7% of the states and for 10% of states, Claude suggests a phrase that at least seven other models also suggest\. In Panel B, we show for what percent of psychological states an individual model shares an indicator phrase with Claude\. The strongest models have at least one overlapping phrase with Claude on over 40% of psychological states\. These numbers collectively present strong evidence fora shared set of indicator phrases models associate with human psychological states\. Since a narrative likely includes multiple phrases, models do not need to overlap in all of their suggested phrases to receive meaningful interpretive signal\.

Indicator phrases capture how models generally conceptualize human states but may be different from what specifically appears in the generated vignettes, so in the next section we examine the connection between these phrases and model performance on ToMStories\.

### 5\.3Sharing indicator phrases with the data generator predicts model performance

Table 5:A comparison of correlation between predictors andimplicitToMStories accuracy, a multiple regression model fit to all three predictors and a dominance analysis of the general dominance of each of these predictors in this model\. We exclude Llama\-3\.2\-1B numbers from this analysis as the model failed to answer most questions\.∗p<\.05\{\}^\{\*\}p<\.05,p∗⁣∗<\.01\{\}^\{\*\*\}p<\.01In the previous sections, we uncover several aspects of models that may contribute to their overall performance onimplicitToMStories: model size, degree of description preference in interpretation, and overlap in indicator phrases with Claude \(the vignette generator\)\. We consider the log of a model’s parameter count, the percent of description\-aligned answers chosen on theconflictedvignettes, and the overlap percents shown in Panel B of Figure[3](https://arxiv.org/html/2610.22632#S5.F3)as proxies for these factors\. In Table[5](https://arxiv.org/html/2610.22632#S5.T5), we report the Pearson’s correlation of each of these predictors withimplicitaccuracy \(and plot these relationships in Appendix Figure[6](https://arxiv.org/html/2610.22632#A3.F6)\)\. We observe significant correlation between all three factors and both multiple choice and open\-ended accuracy \(except for description preference and open\-ended accuracy\)\. Indicator phrase overlap with Claude has the strongest correlation with model performance \(r=0\.89r=0\.89,p<\.01p<\.01for multiple choice;r=0\.92r=0\.92,p<\.01p<\.01for open\-ended\)\.

Because these three predictors are themselves correlated across models \(rr= \.73–\.75\), we estimate their relative contributions jointly with a multiple regression with standardized variables and dominance analysis for general dominance \(also reported in Table[5](https://arxiv.org/html/2610.22632#S5.T5)\)\. Indicator phrase overlap is the only significant predictor when all three are included \(multiple choiceβ\\beta= 0\.60,pp= \.023; open\-endedβ\\beta= 1\.03,pp= \.001\)\. Indicator phrase overlap also accounts for the largest share of explained variance \(45% and 66% ofR2R^\{2\}, vs\.≤\\leq29% for other predictors\)\. For open\-ended questions, this finding is robust to different clustering thresholds for phrase overlap \(detailed in Appendix[G\.2](https://arxiv.org/html/2610.22632#A7.SS2)\)\. These results suggest thatnot only do models learn a shared set of indicator phrases, but the degree to which their indicator phrases overlap with the synthetic data generator is the strongest predictor of interpretive performance\. Shared indicator phrases may let models “shortcut” interpretation of synthetic data, or they may reflect general social reasoning ability\. Testing whether Claude’s indicator phrases match how humans describe behavior would distinguish these accounts\. Overall, these results help us understand how models perform interpretation, while indicating that synthetic data likely simplifies the problem\. We explore another simplification models may use next\.

### 5\.4Implicit means less emotional to models

Figure 4:Effect of implicit \(purple\) and explicit \(gold\) steering relative to unsteered generations from the same prompts\. Diamonds show the median paired effect sizedzd\_\{z\}across models, filled where significant across models \(Wilcoxon, BH\-corrected\)\. Dots show individual models, filled where significant \(q<\.05q<\.05\)\.Linear probes trained to predict the correct answer gain accuracy at similar layers for explicit and implicit vignettes, but reach higher accuracy on explicit ones \(shown in Appendix Section[A](https://arxiv.org/html/2610.22632#A1)\)\. This suggests that models process both kinds of vignette similarly, but explicit vignettes carry more decodable “signal\.” A possible explanation for this signal is presented by considering the differences in model representations for implicit vs\. explicit questions, extracted as a steering vector \(visualized in Appendix Section[C](https://arxiv.org/html/2610.22632#A3)\)\.

To characterize what this direction in activation space captures, we generate vignettes steered toward the explicit and implicit directions, as well as unsteered vignettes, using the same random sample of 100 ToMStories seeds for each\. Qualitatively, implicit steering produces longer, more ornate prose, whereas explicit steering produces shorter, more direct text \(examples in Appendix Table[7](https://arxiv.org/html/2610.22632#A3.T7)\)\. To quantify the change in emotional language, we score each vignette with the NRC Emotion Intensity Lexicon\([Mohammad, 2018](https://arxiv.org/html/2610.22632#bib.bib10)\)\(the rate of words associated with each of eight emotions\) and the NRC VAD Lexicon\([Mohammad, 2025](https://arxiv.org/html/2610.22632#bib.bib11)\)\(the mean valence, arousal, and dominance of its words\)\. For each model and measure, we compare steered and unsteered vignettes with a pairedtt\-test, and we test for a consistent effect across models with a Wilcoxon signed\-rank test on the per\-model mean differences, correcting for multiple measures with Benjamini–Hochberg\. We exclude Llama\-3\.2\-1B and the two R1 distilled models, whose steered outputs frequently degenerate\.

In Figure[4](https://arxiv.org/html/2610.22632#S5.F4), we see thatimplicit steering consistently suppresses emotional language\.Relative to unsteered text, implicit vignettes contain fewer emotion words for seven of the eight emotions \(all but sadness\) and have lower valence and arousal\. This pattern is consistent with[Sofroniew et al\. \(2026\)](https://arxiv.org/html/2610.22632#bib.bib20), which found a steering vector for masking emotion through a related process\. While muting emotion is one method of implicit communication, using emotional signal as a primary axis of modulation between implicit and explicit scenarios is a simplification of human expression\. In reality, implicit psychological expression should and can still be quite emotional\.

## 6Conclusion

We introduce ToMStories as a resource for the community and use it to show how models interpret and process character psychology in narrative\. We see strong evidence that models learn shared indicator phrases that are associated with human psychological states and play a role in interpreting subtext\. We also see that suppressing emotional language is part of how models communicate psychology implicitly, and that models rely more on dialogue than description to access character internal states\. Collectively, these results show how working in the controlled setting of synthetic data can help us formulate and test hypotheses about how models perform interpretation, while also serving as a caution against taking model performance on synthetic human psychology as representative of performance on real human data\. Future work should test whether indicator phrases causally shape model interpretation, and whether humans share the indicator phrases models use\.

## Limitations

#### Synthetic data from a single generator\.

All ToMStories vignettes are written by one model, Claude Sonnet 4\.6\. The benchmark therefore reflects one model’s conventions for how inner states surface in behavior and speech, and our results show that models whose indicator phrases resemble Claude’s tend to score higher\. As discussed and elaborated on in our results, accuracy on ToMStories may thus partly measure alignment with the generator’s conventions rather than interpretive ability in general, and it may overestimate how well models interpret the more varied and less conventional indicators found in human\-written narratives\. Vignettes from other generators, or written by humans, would be needed to separate these factors\.

#### Cultural scope and human baselines\.

The states we test come from a thesaurus written for Western, English\-language writers, so both the states and the behaviors that signal them reflect one cultural framing of psychology\. Our human baselines come from a small number of annotators from varied cultural and language backgrounds; they estimate human performance only roughly, and differences between the thesaurus’s framing and annotators’ own conventions may contribute to human performance\.

#### Representation and steering analyses\.

Our probing and steering analyses are limited to open\-weight models, so we cannot examine the generator’s internal representations\. Additionally, our layer\-selection heuristic involves fixed choices of smoothing and threshold, and steering quality is sensitive to layer selection\.

## Acknowledgments

We are grateful to the Columbia University students who contributed annotations, and to Mohit Iyyer and Katherine Thai for their helpful feedback on early versions of this work\. One of the authors, Melanie Subbiah, holds an equity interest in OpenAI\.

## References

- Belinkov \(2022\)Yonatan Belinkov\. 2022\.[Probing classifiers: Promises, shortcomings, and advances](https://doi.org/10.1162/coli_a_00422)\.*Computational Linguistics*, 48\(1\):207–219\.
- Chen et al\. \(2024\)Zhuang Chen, Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, and Minlie Huang\. 2024\.[ToMBench: Benchmarking theory of mind in large language models](https://doi.org/10.18653/v1/2024.acl-long.847)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 15959–15983, Bangkok, Thailand\. Association for Computational Linguistics\.
- Demszky et al\. \(2020\)Dorottya Demszky, Dana Movshovitz\-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi\. 2020\.[GoEmotions: A dataset of fine\-grained emotions](https://doi.org/10.18653/v1/2020.acl-main.372)\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 4040–4054, Online\. Association for Computational Linguistics\.
- Gandhi et al\. \(2023\)Kanishk Gandhi, Jan\-Philipp Fränken, Tobias Gerstenberg, and Noah D\. Goodman\. 2023\.Understanding social reasoning in language models with language models\.*37th Conference on Neural Information Processing Systems*\.
- Hewitt and Liang \(2019\)John Hewitt and Percy Liang\. 2019\.[Designing and interpreting probes with control tasks](https://doi.org/10.18653/v1/D19-1275)\.In*Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\)*, pages 2733–2743, Hong Kong, China\. Association for Computational Linguistics\.
- Iyyer et al\. \(2016\)Mohit Iyyer, Anupam Guha, Snigdha Chaturvedi, Jordan Boyd\-Graber, and Hal Daumé III\. 2016\.[Feuding families and former Friends: Unsupervised learning for dynamic fictional relationships](https://doi.org/10.18653/v1/N16-1180)\.In*Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 1534–1544, San Diego, California\. Association for Computational Linguistics\.
- Kim et al\. \(2023\)Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Bras, Gunhee Kim, Yejin Choi, and Maarten Sap\. 2023\.[FANToM: A benchmark for stress\-testing machine theory of mind in interactions](https://doi.org/10.18653/v1/2023.emnlp-main.890)\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 14397–14413, Singapore\. Association for Computational Linguistics\.
- Le et al\. \(2019\)Matthew Le, Y\-Lan Boureau, and Maximilian Nickel\. 2019\.[Revisiting the evaluation of theory of mind through question answering](https://doi.org/10.18653/v1/D19-1598)\.In*Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\)*, pages 5872–5877, Hong Kong, China\. Association for Computational Linguistics\.
- Ma et al\. \(2023\)Ziqiao Ma, Jacob Sansom, Run Peng, and Joyce Chai\. 2023\.[Towards a holistic landscape of situated theory of mind in large language models](https://doi.org/10.18653/v1/2023.findings-emnlp.72)\.In*Findings of the Association for Computational Linguistics: EMNLP 2023*, pages 1011–1031, Singapore\. Association for Computational Linguistics\.
- Mohammad \(2018\)Saif M\. Mohammad\. 2018\.Word affect intensities\.In*Proceedings of the 11th Edition of the Language Resources and Evaluation Conference \(LREC\-2018\)*, Miyazaki, Japan\.
- Mohammad \(2025\)Saif M\. Mohammad\. 2025\.[Nrc vad lexicon v2: Norms for valence, arousal, and dominance for over 55k english terms](https://arxiv.org/abs/2503.23547)\.*Preprint*, arXiv:2503\.23547\.
- Nematzadeh et al\. \(2018\)Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, and Tom Griffiths\. 2018\.[Evaluating theory of mind in question answering](https://doi.org/10.18653/v1/D18-1261)\.In*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*, pages 2392–2400, Brussels, Belgium\. Association for Computational Linguistics\.
- Premack and Woodruff \(1978\)David Premack and Guy Woodruff\. 1978\.[Does the chimpanzee have a theory of mind?](https://doi.org/10.1017/S0140525X00076512)*Behavioral and Brain Sciences*, 1\(4\):515–526\.
- Puglisi and Ackerman \(2019\)Becca Puglisi and Angela Ackerman\. 2019\.*The emotion thesaurus: A writer’s guide to character expression*, volume 1\.Jadd Publishing\.
- Rashkin et al\. \(2018\)Hannah Rashkin, Maarten Sap, Emily Allaway, Noah A\. Smith, and Yejin Choi\. 2018\.[Event2Mind: Commonsense inference on events, intents, and reactions](https://doi.org/10.18653/v1/P18-1043)\.In*Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 463–473, Melbourne, Australia\. Association for Computational Linguistics\.
- Rimsky et al\. \(2024\)Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner\. 2024\.[Steering llama 2 via contrastive activation addition](https://doi.org/10.18653/v1/2024.acl-long.828)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 15504–15522, Bangkok, Thailand\. Association for Computational Linguistics\.
- Röttger et al\. \(2024\)Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy\. 2024\.[Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models](https://doi.org/10.18653/v1/2024.acl-long.816)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 15295–15311, Bangkok, Thailand\. Association for Computational Linguistics\.
- Sap et al\. \(2019\)Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A\. Smith, and Yejin Choi\. 2019\.Atomic: An atlas of machine commonsense for if\-then reasoning\.*AAAI*\.
- Shinoda et al\. \(2025\)Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Saki Mizuno, Keita Suzuki, Ryo Masumura, Hiroaki Sugiyama, and Kuniko Saito\. 2025\.Tomato: Verbalizing the mental states of role\-playing llms for benchmarking theory of mind\.*AAAI*\.
- Sofroniew et al\. \(2026\)Nicholas Sofroniew, Isaac Kauvar, William Saunders, Runjin Chen, Tom Henighan, Sasha Hydrie, Craig Citro, Adam Pearce, Julius Tarng, Wes Gurnee, Joshua Batson, Sam Zimmerman, Kelley Rivoire, Kyle Fish, Chris Olah, and Jack Lindsey\. 2026\.[Emotion concepts and their function in a large language model](https://arxiv.org/abs/2604.07729)\.*Preprint*, arXiv:2604\.07729\.
- Subbiah et al\. \(2024a\)Melanie Subbiah, Faisal Ladhak, Akankshya Mishra, Griffin Adams, Lydia B Chilton, and Kathleen McKeown\. 2024a\.Storysumm: Evaluating faithfulness in story summarization\.*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*\.
- Subbiah et al\. \(2025\)Melanie Subbiah, Akankshya Mishra, Grace Kim, Liyan Tang, Greg Durrett, and Kathleen McKeown\. 2025\.[Is the top still spinning? evaluating subjectivity in narrative understanding](https://doi.org/10.18653/v1/2025.emnlp-main.10)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*, pages 185–203, Suzhou, China\. Association for Computational Linguistics\.
- Subbiah et al\. \(2024b\)Melanie Subbiah, Sean Zhang, Lydia B\. Chilton, and Kathleen McKeown\. 2024b\.[Reading Subtext: Evaluating Large Language Models on Short Story Summarization with Writers](https://doi.org/10.1162/tacl_a_00702)\.*Transactions of the Association for Computational Linguistics*, 12:1290–1310\.
- Tak et al\. \(2025\)Ala N\. Tak, Amin Banayeeanzade, Anahita Bolourani, Mina Kian, Robin Jia, and Jonathan Gratch\. 2025\.[Mechanistic interpretability of emotion inference in large language models](https://doi.org/10.18653/v1/2025.findings-acl.679)\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 13090–13120, Vienna, Austria\. Association for Computational Linguistics\.
- Thai and Iyyer \(2025\)Katherine Thai and Mohit Iyyer\. 2025\.[Literary evidence retrieval via long\-context language models](https://doi.org/10.18653/v1/2025.acl-short.29)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\)*, pages 369–380, Vienna, Austria\. Association for Computational Linguistics\.
- Turner et al\. \(2024\)Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J\. Vazquez, Ulisse Mini, and Monte MacDiarmid\. 2024\.[Steering language models with activation engineering](https://arxiv.org/abs/2308.10248)\.*Preprint*, arXiv:2308\.10248\.
- Wang et al\. \(2025\)Qiaosi Wang, Xuhui Zhou, Maarten Sap, Jodi Forlizzi, and Hong Shen\. 2025\.[Rethinking theory of mind benchmarks for llms: Towards a user\-centered perspective](https://arxiv.org/abs/2504.10839)\.*Preprint*, arXiv:2504\.10839\.
- Wimmer and Perner \(1983\)Heinz Wimmer and Josef Perner\. 1983\.[Beliefs about beliefs: representation and constraining function of wrong beliefs in young children’s understanding of deception](https://doi.org/10.1016/0010-0277(83)90004-5)\.*Cognition*, 13\(1\):103–128\.
- Wu et al\. \(2023\)Yufan Wu, Yinghui He, Yilin Jia, Rada Mihalcea, Yulong Chen, and Naihao Deng\. 2023\.[Hi\-ToM: A benchmark for evaluating higher\-order theory of mind reasoning in large language models](https://doi.org/10.18653/v1/2023.findings-emnlp.717)\.In*Findings of the Association for Computational Linguistics: EMNLP 2023*, pages 10691–10706, Singapore\. Association for Computational Linguistics\.
- Xu et al\. \(2024\)Hainiu Xu, Runcong Zhao, Lixing Zhu, Jinhua Du, and Yulan He\. 2024\.[OpenToM: A comprehensive benchmark for evaluating theory\-of\-mind reasoning capabilities of large language models](https://doi.org/10.18653/v1/2024.acl-long.466)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 8593–8623, Bangkok, Thailand\. Association for Computational Linguistics\.
- Zhu et al\. \(2024\)Wentao Zhu, Zhining Zhang, and Yizhou Wang\. 2024\.Language models represent beliefs of self and others\.In*Proceedings of the 41st International Conference on Machine Learning*, ICML’24\. JMLR\.org\.
- Zou et al\. \(2023\)Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann\-Kathrin Dombrowski, et al\. 2023\.Representation engineering: A top\-down approach to ai transparency\.*arXiv preprint arXiv:2310\.01405*\.

## Appendix ALayer\-wise Analysis

We show some examples of layer\-accuracy trajectories for linear probes in Figure[5](https://arxiv.org/html/2610.22632#A1.F5)\. For smaller models, there seems to be a jump in accuracy over about five layers just before the half\-way point of the model\. For Qwen models, this improvement comes after the half\-way point, indicating that earlier layers perform more preprocessing, possibly because Qwen is not an English\-base model\. For the reasoning\-native DeepSeek models, we see that the embedding\-based accuracy underperforms the model’s ultimate accuracy, so more of the performance is gained in the inference\-time reasoning and how that updates internal representations\. For larger models, like Llama\-70B, we observe an improvement in accuracy across more layers in the model, demonstrating the more complex processing necessary to achieve higher ultimate accuracy\.

Figure 5:Classification accuracy for linear probes trained to predict the correct answer from the outputs of each layer\. We report overall accuracy, as well as split across theimplicitandexplicitvariants of ToMStories with sigmoids fit to the data and vertical lines indicating the medians of the sigmoid curves\.In addition to overall accuracy, we split out classification accuracy based on theimplicitandexplicit variantsof ToMStories in Figure[5](https://arxiv.org/html/2610.22632#A1.F5)\. Interestingly, we observe thatimplicit vignettes are not classified later in the layers but almost directly track classification improvements inexplicitscenarios\(albeit with lower accuracy\)\. This observation suggests that the models are processing both implicit and explicit scenarios using similar mechanisms, but for explicit scenarios there is stronger signal in the data for the answer\.

### A\.1Layer selection

The analyses in Section[5\.4](https://arxiv.org/html/2610.22632#S5.SS4)operate at a single layer per model, which we select from the probe curves\. We select the*onset*of the plateau in probe accuracy: the shallowest layer at which the answer is already mostly linearly decodable\. We first smooth the held\-out probe accuracies across layers with a 3\-layer moving average \(truncated at the first and last layers\)\. Letamaxa\_\{\\max\}be the peak of the smoothed curve andamina\_\{\\min\}the lowest smoothed accuracy at or before that peak\. We set a threshold 90% of the way up this rise,

τ=amin\+0\.9​\(amax−amin\),\\tau=a\_\{\\min\}\+0\.9\\,\(a\_\{\\max\}\-a\_\{\\min\}\),and select the earliest layer, at or before the peak, whose smoothed accuracy reachesτ\\tauand stays at or above it for the next layer as well \(i\.e\., two consecutive layers\)\. If no layer meets this condition, we use the peak layer\. Takingamina\_\{\\min\}only over layers up to the peak keeps a drop in accuracy at late layers from lowering the threshold, and requiring two consecutive layers keeps a single\-layer spike from moving the selected layer shallower\.

## Appendix BHardware Details

We run most models on one 40GB A100 GPU\. For models larger than 25B parameters, we use two 40GB A100 GPUs and 4\-bit quantization with double quantization using either AWQ or bitsandbytes\. We use temperature 0 and allow most models to generate up to 400 tokens\. For the reasoning\-native DeepSeek models, we allow 1500 tokens\.

## Appendix CEmbedding Analysis

Figure 6:Implicit accuracy against three model properties, across 12 models \(Llama\-3\.2\-1B excluded due to many unparseable answers\):\(a\)parameter count,\(b\)the share of description\-aligned answers selected onconflictedToMStories, and\(c\)indicator phrase overlap with Claude \(panel B in Figure[3](https://arxiv.org/html/2610.22632#S5.F3)\)\. Filled markers and solid lines show multiple choice; open markers and dashed lines show open\-ended; horizontal lines mark human performance\.Table 6:High\-dimensional factorization statistics per model and layer \(ll\) for steering vectors\. All metrics are averaged across the four question types\.cos⁡\(dt,dt′\)¯\\overline\{\\cos\(d\_\{t\},d\_\{t^\{\\prime\}\}\)\}: mean pairwise cosine between all steering vectors; CV\(‖dt‖\)\(\\\|d\_\{t\}\\\|\): coefficient of variation of steering vector magnitudes\.We plot cluster centroids in Figure[8](https://arxiv.org/html/2610.22632#A8.F8)for vignette embeddings across variants and types\. We additionally compute two summary statistics directly in the high\-dimensional activation space and report them in Table[6](https://arxiv.org/html/2610.22632#A3.T6)to support the PCA plots\. Letμ⁡\(t,c\)∈ℝd\\mu\(t,c\)\\in\\mathbb\{R\}^\{d\}denote the centroid of all last\-token activations at layerℓ∗\\ell^\{\*\}for question typet∈\{wound, emotion, fear, motivation\}t\\in\\\{\\text\{wound, emotion, fear, motivation\}\\\}and variantc∈\{implicit, explicit\}c\\in\\\{\\text\{implicit, explicit\}\\\}\. Define the per\-type variant\-shift vectordt=μ⁡\(t,implicit\)−μ⁡\(t,explicit\)d\_\{t\}=\\mu\(t,\\text\{implicit\}\)\-\\mu\(t,\\text\{explicit\}\)and the mean shiftv¯=14​∑tdt\\bar\{v\}=\\frac\{1\}\{4\}\\sum\_\{t\}d\_\{t\}\. Under perfect additive factorization, alldtd\_\{t\}are identical \(and equal tov¯\\bar\{v\}\)\.CV\(‖dt‖\)\(\\\|d\_\{t\}\\\|\)is the coefficient of variation \(standard deviation divided by mean\) of the four shift magnitudes\. A value near 0 indicates that the variant transformation is consistent in magnitude across types as well as direction\.Meancos⁡\(dt,dt′\)\\cos\(d\_\{t\},d\_\{t^\{\\prime\}\}\)is the average pairwise cosine similarity between all six pairs of shift vectors\.

Table 7:Examples of unsteered and steered generations for the same prompt from three models\.
## Appendix DSteering Coefficient Tuning

Algorithm 1Steering coefficient tuningmodel, steering layer, unit steering vector

v^\\hat\{v\}, writing prompts

a coefficient per direction, and one shared across directions

s​c​a​l​e←scale\\leftarrowmedian residual\-stream norm at the steering layer

b​a​s​e​l​i​n​e←baseline\\leftarrowmean coherence of unsteered generations

f​l​o​o​r←b​a​s​e​l​i​n​e−1floor\\leftarrow baseline\-1⊳\\trianglerighttolerate a 1\-point drop

for all

d​i​r​e​c​t​i​o​n∈\{\+1,−1\}direction\\in\\\{\+1,\-1\\\}do

for all

α∈\{0\.1,0\.2,0\.3,0\.5,0\.75,1,1\.5,2\}\\alpha\\in\\\{0\.1,0\.2,0\.3,0\.5,0\.75,1,1\.5,2\\\}do

steer by

direction⋅α⋅s​c​a​l​e⋅v^\\textit\{direction\}\\cdot\\alpha\\cdot scale\\cdot\\hat\{v\}at every position

c​o​h​e​r​e​n​c​e​\[α\]←coherence\[\\alpha\]\\leftarrowmean judge score over the prompts

endfor

b​e​s​t←best\\leftarrowlargest

α\\alphawith

c​o​h​e​r​e​n​c​e​\[α\]≥coherence\[\\alpha\]\\geqfloor

repeat3 times:

test the midpoint between

b​e​s​tbestand the smallest failing

α\\alpha
update

b​e​s​tbestif the midpoint coherence clears

f​l​o​o​rfloor
c​o​e​f​f​i​c​i​e​n​t​\[d​i​r​e​c​t​i​o​n\]←b​e​s​t⋅s​c​a​l​ecoefficient\[direction\]\\leftarrow best\\cdot scale

endfor

s​h​a​r​e​d←minshared\\leftarrow\\minof

b​e​s​tbestover the two directions

verify

s​h​a​r​e​dsharedclears

f​l​o​o​rfloorin both directions, else flag the model

return

c​o​e​f​f​i​c​i​e​n​t​\[⋅\]coefficient\[\\cdot\]and

s​h​a​r​e​d⋅s​c​a​l​eshared\\cdot scale

To normalize steering strength across models whose residual streams differ in scale, we parameterize it relative to each model’s own activations\. We measureρ\\rho, the median residual\-stream norm at the steering layerℓ\\ellover the tokens of a set of probe prompts \(excluding each sequence’s first token, which is generally an outlier\) and applyρ\\rhoas a scaling factor during steering\. We consider both steering directions,\+1\+1for implicit and−1\-1for explicit\. For each direction, we search a range of alphas for the largest value that does not degrade fluency on a set of 12 generic writing prompts\. To compare steering directions, we adopt the smaller of the two per\-direction magnitudes as a sharedα±\\alpha^\{\\pm\}\. This tuning process is described in more detailed pseudocode in Algorithm[1](https://arxiv.org/html/2610.22632#alg1)\.

## Appendix EBenchmark Scoring

For multiple choice scoring, we consider an answer correct if the model either places the number of the correct answer choice between <answer\></answer\> tags as the prompt instructs, or the number of the correct answer is the last number mentioned in the last 30 characters of the response\.

For open\-ended responses, responses that do not follow the specified output format are marked as unparseable and excluded from accuracy calculations\.

For embedding\-based accuracy, letrrdenote the model’s free\-form response and let\{a1,a2,a3,a4\}\\\{a\_\{1\},a\_\{2\},a\_\{3\},a\_\{4\}\\\}denote the four multiple choice options for a given vignette\. We embed each string using a fixed encoderϕ⁡\(⋅\)\\phi\(\\cdot\), and report the cosine similarity between the model’s response and each answer option as a continuous measure of response quality:

EmbSim​\(r,a\)=ϕ​\(r\)⊤​ϕ​\(a\)‖ϕ⁡\(r\)‖​‖ϕ⁡\(a\)‖\\textbf\{EmbSim\}\(r,a\)=\\frac\{\\phi\(r\)^\{\\top\}\\phi\(a\)\}\{\\\|\\phi\(r\)\\\|\\\|\\phi\(a\)\\\|\}To obtain an accuracy figure comparable to the multiple choice variant, we assign the model the answera^\\hat\{a\}with the highest similarity to the response:

a^=arg⁡maxai​EmbSim​\(r,ai\)\\hat\{a\}=\\underset\{a\_\{i\}\}\{\\arg\\max\}\\;\\textbf\{EmbSim\}\(r,a\_\{i\}\)We use all\-MiniLM\-L6\-v2555[https://huggingface\.co/sentence\-transformers/all\-MiniLM\-L6\-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)asϕ⁡\(⋅\)\\phi\(\\cdot\)\.

For LLM\-based scoring, we use an LLM\-as\-a\-judge protocol in which a judge model𝒥\\mathcal\{J\}assigns a Likert score from 0 to 4, indicating how similar the model’s responserris to the correct answera∗a^\{\*\}given the questionqq:

LLMScore=𝒥⁡\(r,a∗,q\)\\textbf\{LLMScore\}=\\mathcal\{J\}\(r,a^\{\*\},q\)We use gpt\-4o\-mini as the judge model and the prompt is shown in our repository\.

## Appendix FHuman Baseline Interface

The interface for collecting the human baseline is shown in Figure[7](https://arxiv.org/html/2610.22632#A6.F7)\. In addition to the ToMStories question, we ask the annotators to check for inappropriate content, model refusals, and incoherent vignettes or questions that cannot be answered\. No vignettes are flagged as inappropriate or model refusals\. Two vignettes/questions are flagged as incoherent in the set of 80implicitvignettes annotated by humans and none of the 80conflictedvignettes\.

![Refer to caption](https://arxiv.org/html/2610.22632v1/interface.png)Figure 7:The two interfaces for collecting the human baselines\. Part A is for the implicit vignettes, and part B is for the conflicted vignettes\. We do not display the full vignette text here for brevity\.Table 8:Robustness of the indicator\-phrase clustering results to the clustering threshold; the shaded column \(0\.50\) is used in the main text\.
## Appendix GIndicator phrases

### G\.1Parsing and cleaning generated phrases

We request a bare JSON array of short strings from models but expect formatting failures so adopt the following parsing approach\. We first remove any <think\> reasoning trace, then look for a fenced code block containing an array or a span between square braces and parse that as JSON\. If parsing fails, we fall back to treating the response as a plain or numbered list, one cue per line, removing any numbered item start\. We split long answers that are several phrases concatenated \(truncated lists, or 6\+ comma\-separated fragments\) and strip any residual JSON and Markdown debris\. Lastly, we discard phrases that are shorter than four characters, longer than 300 characters, only punctuation, beginning with conversational preamble from the model, or duplicating another phrase in the model’s list\. Before clustering, we reduce each phrase to its core behavior by removing any leading subordinate clause, truncating at the first hard delimiter or subordinating conjunction, and dropping any leading pronoun subject\. For example, "Avoids eye contact when her mother calls" and "She avoids eye contact, shoulders tensing" are both simplified to "avoids eye contact"\. These final phrases are 7\.3 words long on average\.

### G\.2Clustering threshold robustness

Indicator phrase overlap depends on the threshold used in agglomerative clustering, so we repeat our overlap analyses with clustering thresholds from 0\.40 to 0\.60 \(results in Table[8](https://arxiv.org/html/2610.22632#A6.T8)\)\. We see that higher thresholds merge more phrases, resulting in higher overlap numbers as expected\. The substantive conclusions hold across thresholds: \(i\) same\-state overlap correlates with implicit accuracy across models \(r=\.74r=\.74–\.89\.89for multiple choice and\.86\.86–\.92\.92for open\-ended; allp<\.01p<\.01\), and \(ii\) controlling for model size and description preference, overlap remains a significant predictor of implicit accuracy at every threshold for open\-ended and most thresholds for multiple choice\.

## Appendix HAI Use

We used Claude Code to support with the implementation and visualization of experiments\. All ideas came from us and we provided Claude Code with explicit instructions for what to implement or modify\. We ran all of the experiments ourselves and verified the code\. In writing the paper, we used Claude and ChatGPT as assistants with literature review, summarizing claims, and re\-wording complex sentences\.

![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/llama-3.2-1b_layer10_centroid_pca_pc12.png)\(a\)Llama\-3\.2\-1B, layer 10
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/llama-3.2-3b_layer14_centroid_pca_pc12.png)\(b\)Llama\-3\.2\-3B, layer 14
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/llama-3.1-8b_layer13_centroid_pca_pc12.png)\(c\)Llama\-3\.1\-8B, layer 13
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/llama-3.1-70b_layer31_centroid_pca_pc12.png)\(d\)Llama\-3\.1\-70B, layer 31
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/llama-3.3-70b_layer31_centroid_pca_pc12.png)\(e\)Llama\-3\.3\-70B, layer 31
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/qwen-2.5-7b_layer18_centroid_pca_pc12.png)\(f\)Qwen\-2\.5\-7B, layer 18
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/qwen-2.5-72b_layer54_centroid_pca_pc12.png)\(g\)Qwen\-2\.5\-72B, layer 54
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/gemma-2-9b_layer22_centroid_pca_pc12.png)\(h\)Gemma\-2\-9B, layer 22
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/gemma-2-27b_layer21_centroid_pca_pc12.png)\(i\)Gemma\-2\-27B, layer 21
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/mistral-7b_layer15_centroid_pca_pc12.png)\(j\)Mistral\-7B, layer 15
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/phi-4_layer17_centroid_pca_pc12.png)\(k\)Phi\-4, layer 17
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/deepseek-r1-7b_layer20_centroid_pca_pc12.png)\(l\)R1\-Distill\-Qwen\-7B, layer 20
![Refer to caption](https://arxiv.org/html/2610.22632v1/figures/deepseek-r1-8b_layer13_centroid_pca_pc12.png)\(m\)R1\-Distill\-Llama\-8B, layer 13

Figure 8:Centroids of implicit and explicit vignettes for each question type, projected onto the first two principal components, at the layer selected by linear probe analysis\.Table 9:The ten largest indicator phrase clusters, showing the total number of phrases in the cluster and ten randomly sampled distinct phrases from the cluster \(after cleaning and parsing\)Table 10:The first indicator phrase each model generated for example states of each type \(after cleaning and parsing\)\.