Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models

# Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models
Source: [https://arxiv.org/html/2606.28524](https://arxiv.org/html/2606.28524)
Cameron Jones Stony Brook University cameron\.jones@stonybrook\.eduSean Trott Rutgers University \- Newark sean\.trott@rutgers\.edu

###### Abstract

Recent work suggests that Large Language Models \(LLMs\) are sensitive to the belief states of agents described by text, as measured by the false belief task \(FBT\), yet persistent concerns of construct validity remain\. We adopt adevelopmental perspective, tracing the pattern of mental state reasoning behavior—and likelypreconditionsfor this behavior—across multiple training stages in the Olmo22and Pythia language model suites\. We find that above\-chance FBT performance depends both on model size and sufficient training volume, emerges relatively late in pretraining, and is most improved by post\-training interventions \(SFT, DPO\) in the condition most diagnostic of mentalizing \(False Belief, Implicit\)\. However, FBT performance is fragile: consistent with past work, the use of non\-factive verbs \(e\.g\., “thinks”\) increases false belief attributions even in the True Belief condition\. To contextualize these findings, we track the emergence ofsituation modeling: the ability to report on basic factual properties of a described scene\. Situation modeling accuracy generally precedes and exceeds FBT accuracy, yet situational representations also prove surprisingly incoherent in certain respects: when asked about the knowledge states of theAntagonistagent—who always knows the item’s true location—Olmo2213bbis consistently influenced both by theTargetagent’s knowledge state and the presence of non\-factive verbs\. Together, these results suggest that larger, sufficiently trained models build partially coherent situation models in a developmentally appropriate sequence, yet display surprising fragility—highlighting the value of developmental and stress\-testing approaches for evaluating LLM capabilities\.

Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models

Pamela D\. RivièreRutgers University \- Newarkpamela\.riviereruiz@rutgers\.eduCameron JonesStony Brook Universitycameron\.jones@stonybrook\.edu

Sean TrottRutgers University \- Newarksean\.trott@rutgers\.edu

## 1Introduction

The capacity to infer and reason about the belief states of others—sometimes calledmentalizing—serves as a powerful cognitive tool\(apperly\_what\_2012\)\. It plays a crucial role in interpreting and predicting the behavior of other social agents\(apperly\_what\_2012\), and may facilitate cooperation\(tomasello\_understanding\_2005\)or even persuasion\(slaughter\_i\_2013\)\. Past research has explored whether and when this capacity develops in children\(wellman\_meta\-analysis\_2001\), non\-human animals\(krupenye\_great\_2016\), and more recently, large language models \(LLMs\)\(Kouwenhovenet al\.,[2026](https://arxiv.org/html/2606.28524#bib.bib4);trott\_large\_2023;shapira\_clever\_2024;ullman\_large\_2023;kosinski\_evaluating\_2024;xu\_opentom\_2024;gandhi\_understanding\_2023;jones\_comparing\_2024;jones\_does\_2024;street\_llms\_2026\)\.

The question of whether LLMs canmentalizehas generated considerable excitement\. It informs longstanding hypotheses about the role of language input in shaping human mentalizing capacity\(brown\_why\_1996;de\_villiers\_role\_2014;trott\_language\_2026\)\. Practically, the design and interpretation of mentalizing “benchmarks” for LLMs\(xu\_opentom\_2024;jones\_comparing\_2024;gandhi\_understanding\_2023\)could affect decisions about how to interpret or control downstream behaviors of concern, such as deception or persuasion\(jones\_lies\_2026\)\. Yet both aims are complicated by persistent concerns aboutconstruct validity\(trott\_large\_2023;hu\_re\-evaluating\_2025;shapira\_clever\_2024;ullman\_large\_2023;ivanova\_how\_2025\): do tests of mentalizing in LLMs actually assess the construct they purport to measure—and how would we know?

Indeed, construct validity concerns pervade LLM benchmarking practices\(raji\_ai\_2021;saxon\_benchmarks\_2024;bean\_measuring\_2025\), research on human cognition and behavior\(borsboom\_concept\_2004;bloom\_two\_2000\), and the history of scientific measurement more broadly\(chang\_inventing\_2004\)\. Researchers typically address validity concerns using various techniques, including analyses of convergent and predictive validity\(alexandrova\_is\_2016\)\.

In the current work, we adopt a complementary perspective that emphasizes thedevelopmental trajectoryof an LLM’s mentalizing behavior throughout pretraining\. In both human psychology and the study of LLM behavior, a developmental perspective has generated fruitful insights into not only thefinal stateof a system’s behavior but the key mechanisms and milestones that precede it\(chen\_sudden\_2023;tigges\_llm\_2024; Rivière and Trott,[2025](https://arxiv.org/html/2606.28524#bib.bib20)\); see also Section[2\.2](https://arxiv.org/html/2606.28524#S2.SS2)\.

Specifically, a developmental perspective allows us to ask several targeted research questions\. First,whenduring pretraining or post\-training does putative mentalizing behavior “emerge” \(Section[4\.1](https://arxiv.org/html/2606.28524#S4.SS1)\)? Second, how does the emergence of mentalizing behavior relate to other capacities that are arguablypreconditionsto the ability to reason about belief states, e\.g\., the ability to track basic properties of asituation\(Section[4\.2](https://arxiv.org/html/2606.28524#S4.SS2)\)? This has analogies to developmental work more broadly\(chen\_sudden\_2023\)and also serves the function of “stress\-testing” measures of LLM capabilities\(Rivière and Trott,[2025](https://arxiv.org/html/2606.28524#bib.bib20);shapira\_clever\_2024;ullman\_large\_2023\)\. Arguably, the extent to which mentalizing\-like behavior is stable, robust to stress\-testing, and preceded by behavioral preconditions should shape our intuition about whether that behavior reflects mentalizing per se or a more disparate collection of shallow, task\-dependent heuristics\(shapira\_clever\_2024\)\.

The remainder of this paper is structured as follows: first, in Section[2](https://arxiv.org/html/2606.28524#S2), we describe related work on mentalizing in humans and LLMs, construct validation, and the value of a developmental perspective\. In Section[3](https://arxiv.org/html/2606.28524#S3), we describe our methods and the rationale for our experimental design and analyses\. We then present the results of our analysis of the developmental trajectory of mentalizing beahvior across select language models \(Section[4\.1](https://arxiv.org/html/2606.28524#S4.SS1)\) and its relation to other behaviors, such assituation modeling\(Section[4\.2](https://arxiv.org/html/2606.28524#S4.SS2)\)\. Finally, we discuss the implications of this work for the study and interpretation of mentalizing behavior in LLMs, as well as work on identifying LLM “capabilities” more generally \(Section[5](https://arxiv.org/html/2606.28524#S5)\)\.

## 2Related Work

### 2\.1Mentalizing in humans and LLMs

As described in Section[1](https://arxiv.org/html/2606.28524#S1),mentalizinghas long been a topic of interest throughout Psychology, and more recently, in the study of LLM behavior or “machine cognition”\. This capacity is often assessed using a variant of thefalse belief task\(FBT\)\(wimmer1983beliefs;baron1985does;krupenye\_great\_2016\)\. In the typical design, a participant witnesses an object being moved from one location to another, with or without another person’s knowledge\. The participant is then asked about that person’s likely belief states, either explicitly \(e\.g\., “Where does Sally think the book is?”\) or implicitly \(e\.g\., by predicting where Sally might look\)\. Successful performance requiressensitivityto the other person’s implied knowledge states—responding differently depending on what that person likely knows or believes\.

The FBT is widely used as an index of sensitivity to mental states\(Bradfordet al\.,[2020](https://arxiv.org/html/2606.28524#bib.bib5); Schneideret al\.,[2014](https://arxiv.org/html/2606.28524#bib.bib18); Papafragou and Grigoroglou,[2025](https://arxiv.org/html/2606.28524#bib.bib17)\), and has been adapted to the study of mentalizing behavior in non\-human animals\(krupenye\_great\_2016\)and LLMs\(Kouwenhovenet al\.,[2026](https://arxiv.org/html/2606.28524#bib.bib4);strachan\_testing\_2024;street\_llms\_2026;kosinski\_evaluating\_2024;trott\_large\_2023\)\. That said, it has faced a number of criticisms over the years, including the concern that it indexes additional capacities beyond mentalizing\(bloom\_two\_2000\)and the fact that it displays limited convergent and predictive validity in humans\(Gernsbacher and Yergeau,[2019](https://arxiv.org/html/2606.28524#bib.bib16);hayward2017reliability\)\. This has led some to call into question the coherence of “Theory of Mind” or “mentalizing” as a construct altogether\(gough2023does\), presaging analogous debates about measures of mentalizing in LLMs\(hu\_re\-evaluating\_2025\)and prompting the creation of new tasks\(Buttelmannet al\.,[2014](https://arxiv.org/html/2606.28524#bib.bib19);dodell2013using;trott2019individual;jones\_comparing\_2024\)\.

We are not committed here to a strong view regarding the validity of the FBT as a definitive window into mentalizing capacities more broadly\. Our interest, rather, is in the FBT as a behavioral index ofmental state sensitivity\. This index can then be traced developmentally and also compared to other behavioral indices, such as the ability to accurately report on the basic features of a situation \(see Section[4\.2](https://arxiv.org/html/2606.28524#S4.SS2)\)\. Indeed, these analyses might inform questions about the validity of the FBT for LLMs, roughly analogous to the logic underlying convergent validity\(alexandrova\_is\_2016\)\.

### 2\.2A Developmental Perspective

The study of development orontogenycan reveal unique insights that are more challenging to attain from a cross\-sectional perspective\. Research on human cognition has long incorporated a developmental focus, from the early work of Piaget\(Piaget,[1954](https://arxiv.org/html/2606.28524#bib.bib15)\)to more contemporary work on child development\(Gopniket al\.,[1999](https://arxiv.org/html/2606.28524#bib.bib14); Tomasello and Brooks,[2016](https://arxiv.org/html/2606.28524#bib.bib13)\), including work on mentalizing specifically\(wellman\_meta\-analysis\_2001\)\. One key scientific benefit of this perspective is that it allows researchers to observe how a particular behavior unfolds acrosstime, addressing questions about whether this pattern consists of discrete “jumps” or continual improvement\(wellman\_meta\-analysis\_2001\), as well as what kinds of experiential input are necessary or sufficient for the emergence of a particular behavior\(Astington and Jenkins,[1999](https://arxiv.org/html/2606.28524#bib.bib8)\)\. Similarly, researchers can ask about thetemporal orderingof different behaviors, granting insight into the requisite capacities or preconditions facilitating a particular capacity of interest\(de2016sensorimotor\)\.

More recently, research on LLMs has also adopted a developmental perspective, with the goal both of learning more about human developmental, e\.g\., language learning\(Warstadtet al\.,[2023](https://arxiv.org/html/2606.28524#bib.bib12); Constantinescuet al\.,[2025](https://arxiv.org/html/2606.28524#bib.bib11)\)and also learning more about LLMs themselves\(chen\_sudden\_2023; Saphra and Lopez,[2019](https://arxiv.org/html/2606.28524#bib.bib10); Rivière and Trott,[2025](https://arxiv.org/html/2606.28524#bib.bib20); Michaelovet al\.,[2025](https://arxiv.org/html/2606.28524#bib.bib7); Zhaoet al\.,[2024](https://arxiv.org/html/2606.28524#bib.bib9)\)\. This approach carries many of the same advantages described above, e\.g\., investigating which behaviors are learned in which order\(chen\_sudden\_2023\), and it also allows researchers to connect emergent behaviors to low\-level mechanisms or “circuits”\(Olssonet al\.,[2022](https://arxiv.org/html/2606.28524#bib.bib6);tigges\_llm\_2024; Rivière and Trott,[2025](https://arxiv.org/html/2606.28524#bib.bib20);trott\_toward\_2025;aoyama\_predicting\_2025\)\.

Finally, concurrent work\(Kouwenhovenet al\.,[2026](https://arxiv.org/html/2606.28524#bib.bib4)\)conducted a case study on the developmental pattern of false belief reasoning in Olmo2\(OLMoet al\.,[2024](https://arxiv.org/html/2606.28524#bib.bib2)\)\. Two key results are relevant here: first, syntactic competence emerges much earlier during pretraining than competence on the FBT; and second, post\-training had nuanced effects on FBT performance, with small improvements observed for thefalse beliefcondition but more mixed effects on thetrue beliefcondition\. We report similar results in Section[4\.1](https://arxiv.org/html/2606.28524#S4.SS1)\(seeFigure[5](https://arxiv.org/html/2606.28524#S4.F5)\)\.

The current work extends this line of inquiry in two key ways\. First, by comparing the OLMo 2 suite with the \(relatively\) undertrained Pythia suite\(Bidermanet al\.,[2023](https://arxiv.org/html/2606.28524#bib.bib3)\), we examine thejointcontribution of model size and volume of language exposure—finding that neither alone is sufficient\. Second, and more critically, we investigate whether putative mentalizing behavior is preceded by the emergence of a more basic capacity: the ability to accurately track descriptive properties of a situation, such as where an object is located or who moved it \(Section[4\.2](https://arxiv.org/html/2606.28524#S4.SS2)\)\. Thissituation modelingcapacity represents a task\-specific precondition to false belief reasoning, complementing[Kouwenhovenet al\.](https://arxiv.org/html/2606.28524#bib.bib4)’s comparison of FBT performance with general syntactic competence\.

## 3Materials and Methods

### 3\.1Mentalizing Stimuli

To examine language model \(LM\) mentalizing ability, we used a publicly accessible False Belief Task \(FBT\) stimulus set\(trott\_large\_2023\)\. Stimuli consist of passages describing scenes in which two different named agents serially change the location of an item such that its starting and final locations differ\. Crucially, a target agent has either observed or failed to observe the item’s relocation\. Each passage ends with the implication or indication that this target agent will subsequently attempt to retrieve the item, and the LM’s task is to produce probabilities over Start and End locations, indicating where the agent is most likely to search\.

trott\_large\_2023created a total of192192unique passages from1212templates by varying properties that might influence performance on the FBT, such asKnowledge State\(True or False Belief; reflecting whether the target agent observes the item’s displacement to its final location or not\) andKnowledge Cue\(Explicit or Implicit; whether the final sentence references the agent’s beliefs \(e\.g\. “Davethinksthe book is in the…” or “Davelooks forthe book in the…”\)\. Start and End locations are each mentioned twice per passage; passages consequently vary to cover all permutations of the locations’ first and last mentions\. Sample passages can be found in Appendix[A\.1](https://arxiv.org/html/2606.28524#A1.SS1)\.

### 3\.2Situation Modeling Stimuli

To further stress\-test\(shapira\_clever\_2024; Rivière and Trott,[2025](https://arxiv.org/html/2606.28524#bib.bib20)\)LMs’ abilities to create and sustain a Situation Model—that is, to form coherent representations of agents and events—we modified the original stimulus set to obtain LM probabilities that should reflect descriptive properties of a passage which the LM need only egocentrically report on\. In principle, such a capacity is likely to serve as a crucial prerequisite to the more complex ability to attribute mental states to agents who have beliefs that diverge from an omniscient observer’s\.

![Refer to caption](https://arxiv.org/html/2606.28524v1/figures/fb-task-acc-nparams-model-fam.png)Figure 1:False belief task accuracy increases as a function of LM size for the Olmo22suite\. Grey observations correspond to a given LM’s performance at distinct pretraining checkpoints\. Observations colored according to model family correspond to an LM’s performance at its final pretraining step \(for Olmo22, this is the final step in stage11pretraining\)\. Horizontal dashed line marks chance performance\.We first tested LMs’ ability to track the belief states of the Antagonist, who always moved the item to its end location\. These192192passages were identical to the original set, except for the named agent in the final critical sentence \(example in Appendix[A](https://arxiv.org/html/2606.28524#A1)\)\. In all of these cases, if the LM is capable of constructing an accurate Situation Model, it should produce larger probabilities for End locations than for Start\.

For the second block of stress\-tests, we took each of the192192passages and added one of four new query sentences whose best completions were the item’s actual location, or the name of the agent responsible for having placed the item in either its Start or End locations \(n=768n=768passages\)\. Examples below:

- •At the start of the story, John put the shovel in the
- •At the end of the story, the shovel was in the
- •The person who put the shovel in the toolbox was
- •The person who moved the shovel to the van was

Queries were designed to match the way that information was presented in the passage\. Appendix[A](https://arxiv.org/html/2606.28524#A1)additionally documents performance on minimal\-pair versions of these queries\.

![Refer to caption](https://arxiv.org/html/2606.28524v1/figures/fb-task-acc-tokens.png)Figure 2:False belief task accuracy improvements over pretraining depend on model size and exposure to training tokens\. The0t​h0^\{th\}pretraining step has been removed and the x\-axis begins with the next earliest selected step, to more clearly visualize performance at the more dynamic end of the range\. Horizontal dashed line marks chance performance\.
### 3\.3Language Model Selection

We focused our study on the Olmo22suite of openly\-available LMs\(OLMoet al\.,[2024](https://arxiv.org/html/2606.28524#bib.bib2)\)\. In a replication of the False Belief \(FB\) task examining the final checkpoint of4141open\-weight LMs,trott\_language\_2026, Olmo2213​b13bemerges as a top performer in this task\. The Olmo22suite is published along with multiple pretraining checkpoints, rendering Olmo22LMs desirable substrates with which to examine learning dynamics\.

We additionally explored the effect of LM size and volume of language exposure by charting the developmental trajectory of performance in approximately parameter\-matched LMs from the Pythia suite\(Bidermanet al\.,[2023](https://arxiv.org/html/2606.28524#bib.bib3)\)\. Notably, these LMs were trained on an order of magnitude fewer tokens than their Olmo22counterparts \(Table[1](https://arxiv.org/html/2606.28524#A1.T1)\)\.

All LMs were loaded and run using the HuggingFacetransformerspackage\(Wolfet al\.,[2020](https://arxiv.org/html/2606.28524#bib.bib22)\)on remote Lambda On\-Demand instances111[https://docs\.lambda\.ai/public\-cloud/on\-demand/](https://docs.lambda.ai/public-cloud/on-demand/), selected according to availability at runtime \(B200, H100 SXM, GH200, A100 SXM\)\. We sampled log\-spaced pretraining checkpoint indices within each pretraining stage \(if there were multiple, as was the case for Olmo22\), along with the stage’s final checkpoint\. For Olmo22, we also analyzed behavior at subsequent post\-training stages\. See Appendix[A](https://arxiv.org/html/2606.28524#A1)for a full list of LM checkpoints we examined\.

### 3\.4Procedure

We tokenized passage stimuli with the corresponding tokenizer for a given LM, then obtained the LM probability over Start and End tokens as likely passage completions\. We used this to compute our critical measure, the log\-odds of start versus end locations:l​o​g2​\(p​\(start\)p​\(end\)\)log\_\{2\}\(\\frac\{p\(\\textsc\{start\}\)\}\{p\(\\textsc\{end\}\)\}\)\. We derived accuracy by considering a log\-odds\>0\>0for False Belief, and a log\-odds<=0<=0in True Belief conditions as correct responses\. Any other mapping was marked incorrect\. Accuracy then consists of the proportion of log\-odds marked correct\.

For Situation Modeling tasks, we obtained the the probability over the correct and distractor token completions and computed the corresponding log\-odds:l​o​g2​\(p​\(correct\)p​\(distractor\)\)log\_\{2\}\(\\frac\{p\(\\textsc\{correct\}\)\}\{p\(\\textsc\{distractor\}\)\}\)\. Consequently, a log\-odds\>0\>0was marked correct, and<=0<=0incorrect, from which we then computed accuracy\. All scripts and data are available as GitHub222[https://github\.com/seantrott/dev\_tom/tree/main](https://github.com/seantrott/dev_tom/tree/main)and Open Science Framework333[https://osf\.io/ncypd/overview](https://osf.io/ncypd/overview)repositories\.

![Refer to caption](https://arxiv.org/html/2606.28524v1/figures/lmer-coefficients-with-interactions.png)Figure 3:Coefficients of a linear mixed effects model predicting False Belief task accuracy from stimulus properties \(Knowledge State, Knowledge Cue\) and language model \(LM\) properties \(z\-scored number of tokens seen and z\-scored number of LM parameters\), as well as the corresponding interactions\. LMs were more accurate overall for passages where the target agent should know where the item is to be found\.

## 4Results

All analyses and visualizations were carried out in R, with mixed effects models built using thelme4package\(Bateset al\.,[2015](https://arxiv.org/html/2606.28524#bib.bib1)\)and visualizations withggplot\(Wickham,[2016](https://arxiv.org/html/2606.28524#bib.bib21)\)\.

![Refer to caption](https://arxiv.org/html/2606.28524v1/x1.png)Figure 4:Olmo2213​b13b’s sensitivity for False Belief stimulus properties changes over the course of pretraining\. Black arrows mark chance\-level crossings in accuracy towards the end of pretraining, with sudden drops in accuracy for True Belief, Explicitly cued passages that coincide with sudden increases in accuracy for False Belief, Implicit passages\. See also companion AppendixFigure[9](https://arxiv.org/html/2606.28524#A1.F9)\.### 4\.1Pretraining Dynamics of Mentalizing

We computed accuracy over token completions for False Belief \(FB\) Task passages\(trott\_large\_2023\), across select LMs from the Olmo22\(OLMoet al\.,[2024](https://arxiv.org/html/2606.28524#bib.bib2)\)and Pythia suites \(Bidermanet al\.\([2023](https://arxiv.org/html/2606.28524#bib.bib3)\)\)\. We first evaluated whether LM FB task accuracy was systematically related to LM properties and stimulus features\. We built a linear mixed effects model predicting binary correct/incorrect responses from \(z\-scored\) number of LM parameters as a proxy for model size, \(z\-scored\) cumulative number of tokens seen, and the interaction of these two LM properties; as well as stimulus features such as \(Knowledge State, the presence ofKnowledge Cue\), and their interaction\. We additionally included random intercepts forLanguage Modeland itemStart Location\.

Unsurprisingly, LM model size predicted improvements in performance, but only in the presence of large volumes of training data: a significant positive interaction between number of parameters and number of tokens seen \(β=0\.07,S​E=0\.008,p<0\.001\\beta=0\.07,SE=0\.008,p<0\.001\) indicated that model capacity alone, if under\-trained, is insufficient to produce behavior consistent with mentalizing \(Figure[1](https://arxiv.org/html/2606.28524#S3.F1)\)\. This pattern is even clearer when visualizing pretraining dynamics: the under\-trained Pythia LMs perform consistently at chance regardless of size; while the smallest member of the Olmo22suite is unable to leverage the large amount of training tokens it has seen\. Only Olmo227​b7band13​b13b, combining superior model size with greater language exposure, improve in accuracy toward the latter end of stage11pretraining \(Figure[2](https://arxiv.org/html/2606.28524#S3.F2)\)\.

![Refer to caption](https://arxiv.org/html/2606.28524v1/x2.png)Figure 5:Olmo2213​b13bpost\-training interventions improve performance under stimulus properties that are typically used to index mentalizing \(False Belief, Implicit\), but remains relatively uniform for all others, including low\-accuracy ones that should only reflect the ability to report on factual properties of a scene \(True Belief, Explicit\)\. Subpanels depict Olmo2213​b13bFB task accuracy across various training stages, colored according toKnowledge Statecondition, and rows indicating the presence or absence ofKnowledge Cuewithin passages\. See AppendixFigure[10](https://arxiv.org/html/2606.28524#A1.F10)for the corresponding log\-odds\.Stimulus properties, such asKnowledge StateandKnowledge Cuewere also strong predictors of performance\. We found a significant interaction between these two stimulus properties \(β=1\.31,S​E=0\.0312,p<0\.001\\beta=1\.31,SE=0\.0312,p<0\.001;Figure[3](https://arxiv.org/html/2606.28524#S3.F3);Table[3](https://arxiv.org/html/2606.28524#A1.T3)\)\. In general, LMs tested here performed best in the True BeliefKnowledge Statecondition \(Table[2](https://arxiv.org/html/2606.28524#A1.T2)\), and this was particularly the case in the passages with ImplicitKnowledge Cue\(Table[3](https://arxiv.org/html/2606.28524#A1.T3)\)\. Only Olmo2213​b13band7​b7bperformed above chance for passages within the crucial False Belief condition, but they only did so when theKnowledge Cuewas Explicit in the query and performed systematically below chance when theKnowledge Cuewas absent \(Table[3](https://arxiv.org/html/2606.28524#A1.T3)\)\. In all, it appeared that LMs were sensitive to the presence of verbs that explicitly referenced the target agent’s beliefs as to the item’s location\. Notably, this sensitivity toKnowledge Cuemanifests quite late in the first stage of Olmo2213​b13bpretraining \(Figure[4](https://arxiv.org/html/2606.28524#S4.F4)\)\.

Finally, we compared performance in the Olmo22LMs across each stage of training, including Stage 1, Stage 2, the fully Pretrained model, supervised fine\-tuning \(SFT\), direct policy optimization \(DPO\), and instruction\-tuning \(Instruct\)\. AsFigure[5](https://arxiv.org/html/2606.28524#S4.F5)makes clear, accuracy was remarkably stable in 13bbacross stages when theKnowledge Cuewas Explicit; however, we did observe marked improvement \(for 13bb\) in the False Belief condition whenKnowledge Cuewas Implicit, consistent with concurrent work\(Kouwenhovenet al\.,[2026](https://arxiv.org/html/2606.28524#bib.bib4)\)\. See AppendixFigure[10](https://arxiv.org/html/2606.28524#A1.F10)for a visualization of the changes in the corresponding log\-odds across models and training stages\.

![Refer to caption](https://arxiv.org/html/2606.28524v1/x3.png)Figure 6:Antagonist Belief task mean log odds at the final checkpoint for all LMs tested \(fromstage11for the Olmo22suite\)\. Positive values indicate larger probabilities for Start locations; negative values indicate larger probabilities assigned to End locations\. In this task, the end location is always correct; positive values index low\-accuracy performance\. Note: Antagonists cannot hold False Beliefs in this task—but Olmo2213​b13bstillincorrectly assigns larger probabilities to Start locations according to theTargetagent’sKnowledge State\. See AppendixFigure[11](https://arxiv.org/html/2606.28524#A1.F11)for developmental trajectories\.
### 4\.2Charting the Emergence of Situation Modeling

In the work that follows, we focus our interpretation on Olmo2213​b13b, the LM that exhibited performance most consistent with mentalizing \(Figure[1](https://arxiv.org/html/2606.28524#S3.F1)\)\. Notably, the passages where Olmo2213​b13bdisplayed late\-stage, sudden drops in accuracy are those that \(in principle\) should require deploying a less complex cognitive “toolkit”\. In these True Belief, Explicit passages, the Target agent is privy to all the same information as the omniscient observer \(the LM\), and as a result, mentalizing itself is not strictlyrequiredto emit correct responses\. This points to an over\-reliance on certain surface properties of the stimulus, such as the presence of non\-factive verbs like “thinks”\(trott\_large\_2023;trott\_language\_2026; Kouwenhovenet al\.,[2026](https://arxiv.org/html/2606.28524#bib.bib4)\)\)\. We conducted a series of “stress\-tests” to further contextualize this finding, and probe the model’s ability to report on factual properties of the scene by presenting it with modified queries \(as described in Section[3\.2](https://arxiv.org/html/2606.28524#S3.SS2);shapira\_clever\_2024; Rivière and Trott \([2025](https://arxiv.org/html/2606.28524#bib.bib20)\)\)\.

#### 4\.2\.1Stress\-Test: Tracking Antagonist Beliefs

The Antagonist agent in each passage always moved the item to its end location, in either the Target agent’s presence \(True Belief\) or absence \(False Belief\)\. Consequently, LMs capable of constructing and maintaining a coherent Situation Model should assign larger probabilities to the item’s End location relative to the Start when the query asks where the Antagonist is likely to search\. At the final checkpoint, we found that Olmo2213​b13bmistakenly assigned larger probabilities to Start locations in passage queries that explicitly referenced the Antagonist’s beliefs \(e\.g\. “Martathinks…”\), an error pattern that only Pythia14​m14m—orders of magnitude smaller—shared \(Figure[6](https://arxiv.org/html/2606.28524#S4.F6); AppendixFigure[11](https://arxiv.org/html/2606.28524#A1.F11)\)\. In these cases, it appeared that the presence of the Explicit cue biased the LM to predict that the Antagonist would look for the item in the incorrect location\. Moreover, even though the Antagonist could only have True Beliefs, Olmo2213​b13bassigned larger probabilities to Start locations when passages contained Target agents with False Beliefs, even though their beliefs were not the ones the LM was expected to infer\.

We built a linear mixed effects model predicting Olmo2213​b13blog\-odds fromKnowledge State,Knowledge Cue, their interaction, and the order in which locations were mentioned, which were counterbalanced across passages \(Section[3](https://arxiv.org/html/2606.28524#S3)\), and a random intercept for start location\. We found significant main effects ofKnowledge State\(β=−1\.35\\beta=\-1\.35, SE =0\.2470\.247,p<0\.001p<0\.001, reference level: True Belief\) , andKnowledge Cue\(β=−3\.19\\beta=\-3\.19, SE =0\.2470\.247,p<0\.001p<0\.001, reference level: Implicit\), further reinforcing the notion that Olmo2213​b13bsuffers particular weaknesses with respect to conditions in which the agent in question should know exactly where to find the item\.

#### 4\.2\.2Stress\-Test: Tracking Causal Structure

Given Olmo2213​b13b’s failures to infer the Antagonist’s most likely search location, we explored the LM’s general capacity for modeling the events of the situation, such as reporting on where the item was located at the Start and End, and which agent moved the item to a given location \(see Section[3\.2](https://arxiv.org/html/2606.28524#S3.SS2)for a sample list of Situation Modeling queries\)\.

We found that improvements in Olmo2213​b13b’s Situation Model \(SM\) task performance generallyprecededandexceededits improvements in the False Belief \(FB\) task \(Figure[7](https://arxiv.org/html/2606.28524#S4.F7)\)\. This was particularly true for questions about the location of an item, i\.e\., at thestartorendof a passage; it was also true for questions about the identity of the agent whomovedthe item between locations \(i\.e\. the Antagonist\), which is particularly striking given Olmo2213​b13b’s failures to accurately track the Antagonist’sbeliefswhen asked \(Figure[6](https://arxiv.org/html/2606.28524#S4.F6)\)\.

![Refer to caption](https://arxiv.org/html/2606.28524v1/figures/olmo13b-sitmod-locations.png)

![Refer to caption](https://arxiv.org/html/2606.28524v1/figures/olmo13b-sitmod-agents.png)

Figure 7:Olmo2213​b13bsituation model accuracy generally increases over the course of pretraining, its improvementsprecedeand generallyexceedFalse Belief task accuracies, but biases also emerge, particularly when the model is asked to track agents\. Performance trajectories are colored according to Task Type, with False Belief accuracies in teal, and Situation Model accuracies in grey\. Dashed horizontal line indicates chance performance\.Conversely, Olmo2213​b13bwas unable to accurately report on which agent had placed the item in its start location \(i\.e\. the Target agent\), the agent whosebeliefsit was best able to track, most notably under False Belief conditions crucial to detecting the \(in\-principle\) more complex mentalizing ability \(Figure[4](https://arxiv.org/html/2606.28524#S4.F4),[9](https://arxiv.org/html/2606.28524#A1.F9)\)\. In fact, Olmo2213​b13bdisplayed a remarkably consistent bias over the course of pretraining to emit larger probabilities for the “Antagonist” agent rather than the Target agent\.

Together, these results suggest that Olmo2213​b13b’s representation of the situation is somehowincoherent: it can report the location of objects, and the Target agent’s beliefs about those object locations \(FBT\), while struggling to report the identity of the character who originally moved the object, as well as the Antagonist’s beliefs\. In principle, performing the FBT correctly should depend on knowing which characters moved the item to each location, given that this is the crucial differentiating factor between their respective knowledge states\.

## 5Discussion

This work adopted a developmental perspective to characterize the mentalizing and situation modeling capabilities of language models \(LMs\) across two broad “families”: the Olmo2213​b13b\(OLMoet al\.,[2024](https://arxiv.org/html/2606.28524#bib.bib2)\)and Pythia suites\(Bidermanet al\.,[2023](https://arxiv.org/html/2606.28524#bib.bib3)\)\. We found that performance on the False Belief task \(FBT\) depended on both model size and training volume, and was also strongly influenced by stimulus properties—such as the use of a non\-factive verb like “thinks” in the query to the LM \(Figures[1](https://arxiv.org/html/2606.28524#S3.F1)\-[3](https://arxiv.org/html/2606.28524#S3.F3); Tables[2](https://arxiv.org/html/2606.28524#A1.T2),[3](https://arxiv.org/html/2606.28524#A1.T3)\)\. This latter finding was consistent with past work\(trott\_language\_2026; Kouwenhovenet al\.,[2026](https://arxiv.org/html/2606.28524#bib.bib4)\), and notably, the pattern remained unchanged over the course of post\-training, despite improvements in other conditions \(Figure[5](https://arxiv.org/html/2606.28524#S4.F5)\)\.

In further stress\-testing, we identified a similar effect of non\-factive verbs in a task measuring a model’s ability to accurately report theAntagonist’sbelief states\. This effect was particularly strong in cases where theTargetshould hold a False Belief, suggesting the model experienced “interference” from the primary experimental manipulation \(Figure[6](https://arxiv.org/html/2606.28524#S4.F6)\)\. In a second block of stress\-testing, we found that Olmo22’s ability to report an item’s initial and final locations exceeded and preceded its ability to report second\-order beliefs about those locations \(Figure[7](https://arxiv.org/html/2606.28524#S4.F7)\)—consistent with a relatively coherent situation model\. Yet the model also displayed a systematic bias towards the Antagonist even in questions asking about the Target \(i\.e\., the identity of the character who originally moved the item\)\. Together with the first stress\-test results, this paints a nuanced picture: situation model representations are surprisingly brittle along some dimensions, despite showing coherence in others\.

These results also inform ongoing debates about the construct validity of mentalizing tests for LLMs\(hu\_re\-evaluating\_2025\)\. In some respects, the developmental pattern is encouraging: situation modeling accuracy appears to precede improvements in mentalizing performance, consistent with the intuition that tracking the basic facts of a situation should precede—and perhaps support—reasoning about \(false\) beliefs about that situation\. Yet this developmental coherence is complicated by incoherence along other dimensions, as described above\. This could point to a collection of overlapping sensitivities rather than a unified mentalizing construct\.

Finally, our results speak to longstanding debates about whether exposure to language issufficientfor mentalizing\-like behavior to emerge\(brown\_why\_1996;de\_villiers\_role\_2014\)\. The results do support the conclusion that linguistic input can producesensitivityto belief states: FBT performance improved throughout training, and was strongest for large models with extensive linguistic exposure\. At the same time, the first emergence of above\-chance performance coincided with orders of magnitude more linguistic exposure than humans encounter in a lifetime—and this performance still displayed considerable fragility in the face of minor perturbations, consistent with other recent work\(shapira\_clever\_2024\)\. While we cannot generalize about LMs not observed in our current sample, these results do suggest that linguistic input is insufficient to produce robust, generalizable mentalizing behavior intheseLMs\.

More broadly, the approach adopted here—tracing a behavior’s developmental trajectory alongside likely preconditions and stress\-tests—offers a general methodological template for evaluating LM capabilities\. It joins a growing body of work assessing the “ontogeny” of LM behaviors and mechanisms\(chen\_sudden\_2023;tigges\_llm\_2024; Rivière and Trott,[2025](https://arxiv.org/html/2606.28524#bib.bib20)\), as well as assessing the robustness of a particular behavior under various perturbations\(shapira\_clever\_2024\)\. This paradigm could complement existing benchmarking approaches and even serve as a method for construct validation\.

## Limitations

The primary limitations of the present study stem from the stimulus set and the limited selection of LMs\. We limited our stimuli to only those passages publicly accessible fromtrott\_large\_2023, although many variants of the False Belief \(FB\) task exist to test mentalizing capabilities\. It is possible that a distinct pattern of results would emerge for different variants of the FB task\. Moreover, we constrained the analysis to LM suites that release pretraining checkpoints publicly\.

## Acknowledgments

We are grateful for insightful conversations with Tom Kouwenhoven, Michiel T\. van der Meer, and Max van Duijn, and to Stephen J\. Hanson for generously providing access to an NVIDIA DGX\-H200\.

## References

- J\. W\. Astington and J\. M\. Jenkins \(1999\)A longitudinal study of the relation between language and theory\-of\-mind development\.\.Developmental psychology35\(5\),pp\. 1311\.Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p1.1)\.
- D\. Bates, M\. Mächler, B\. Bolker, and S\. Walker \(2015\)Fitting linear mixed\-effects models using lme4\.Journal of statistical software67,pp\. 1–48\.Cited by:[§4](https://arxiv.org/html/2606.28524#S4.p1.1)\.
- S\. Biderman, H\. Schoelkopf, Q\. Anthony, H\. Bradley, K\. O’Brien, E\. Hallahan, M\. A\. Khan, S\. Purohit, U\. S\. Prashanth, E\. Raff,et al\.\(2023\)Pythia: a suite for analyzing large language models across training and scaling\.InProceedings of the 40th International Conference on Machine Learning,pp\. 2397–2430\.Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p4.1),[§3\.3](https://arxiv.org/html/2606.28524#S3.SS3.p2.1),[§4\.1](https://arxiv.org/html/2606.28524#S4.SS1.p1.1),[§5](https://arxiv.org/html/2606.28524#S5.p1.2)\.
- E\. E\. Bradford, V\. E\. Brunsdon, and H\. J\. Ferguson \(2020\)The neural basis of belief\-attribution across the lifespan: false\-belief reasoning and the n400 effect\.cortex126,pp\. 265–280\.Cited by:[§2\.1](https://arxiv.org/html/2606.28524#S2.SS1.p2.1)\.
- D\. Buttelmann, H\. Over, M\. Carpenter, and M\. Tomasello \(2014\)Eighteen\-month\-olds understand false beliefs in an unexpected\-contents task\.Journal of experimental child psychology119,pp\. 120–126\.Cited by:[§2\.1](https://arxiv.org/html/2606.28524#S2.SS1.p2.1)\.
- I\. Constantinescu, T\. Pimentel, R\. Cotterell, and A\. Warstadt \(2025\)Investigating critical period effects in language acquisition through neural language models\.Transactions of the Association for Computational Linguistics13,pp\. 96–120\.Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p2.1)\.
- M\. A\. Gernsbacher and M\. Yergeau \(2019\)Empirical failures of the claim that autistic people lack a theory of mind\.\.Archives of scientific psychology7\(1\),pp\. 102\.Cited by:[§2\.1](https://arxiv.org/html/2606.28524#S2.SS1.p2.1)\.
- A\. Gopnik, A\. N\. Meltzoff, and P\. K\. Kuhl \(1999\)The scientist in the crib: minds, brains, and how children learn\.\.William Morrow & Co\.Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p1.1)\.
- T\. Kouwenhoven, M\. van der Meer, and M\. van Duijn \(2026\)Traces of social competence in large language models\.External Links:2603\.04161,[Link](https://arxiv.org/abs/2603.04161)Cited by:[§1](https://arxiv.org/html/2606.28524#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.28524#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p3.1),[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p4.1),[§4\.1](https://arxiv.org/html/2606.28524#S4.SS1.p4.3),[§4\.2](https://arxiv.org/html/2606.28524#S4.SS2.p1.4),[§5](https://arxiv.org/html/2606.28524#S5.p1.2)\.
- J\. A\. Michaelov, R\. P\. Levy, and B\. K\. Bergen \(2025\)Language model behavioral phases are consistent across architecture, training data, and scale\.arXiv preprint arXiv:2510\.24963\.Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p2.1)\.
- T\. OLMo, P\. Walsh, L\. Soldaini, D\. Groeneveld, K\. Lo, S\. Arora, A\. Bhagia, Y\. Gu, S\. Huang, M\. Jordan,et al\.\(2024\)2 OLMo 2 Furious\.arXiv preprint arXiv:2501\.00656\.Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p3.1),[§3\.3](https://arxiv.org/html/2606.28524#S3.SS3.p1.6),[§4\.1](https://arxiv.org/html/2606.28524#S4.SS1.p1.1),[§5](https://arxiv.org/html/2606.28524#S5.p1.2)\.
- C\. Olsson, N\. Elhage, N\. Nanda, N\. Joseph, N\. DasSarma, T\. Henighan, B\. Mann, A\. Askell, Y\. Bai, A\. Chen,et al\.\(2022\)In\-context learning and induction heads\.arXiv preprint arXiv:2209\.11895\.Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p2.1)\.
- A\. Papafragou and M\. Grigoroglou \(2025\)Pragmatic communication and theory of mind\.Philosophical Transactions B380\(1932\),pp\. 20230502\.Cited by:[§2\.1](https://arxiv.org/html/2606.28524#S2.SS1.p2.1)\.
- J\. Piaget \(1954\)The construction of reality in the child\.Basic Books/Hachette Book Group\.External Links:[Document](https://dx.doi.org/10.1037/11168-000)Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p1.1)\.
- P\. D\. Rivière and S\. Trott \(2025\)Start making sense \(s\): a developmental probe of attention specialization using lexical ambiguity\.arXiv preprint arXiv:2511\.21974\.Cited by:[§1](https://arxiv.org/html/2606.28524#S1.p4.1),[§1](https://arxiv.org/html/2606.28524#S1.p5.1),[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p2.1),[§3\.2](https://arxiv.org/html/2606.28524#S3.SS2.p1.1),[§4\.2](https://arxiv.org/html/2606.28524#S4.SS2.p1.4),[§5](https://arxiv.org/html/2606.28524#S5.p5.1)\.
- N\. Saphra and A\. Lopez \(2019\)Understanding learning dynamics of language models with SVCCA\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 3257–3267\.External Links:[Link](https://aclanthology.org/N19-1329/),[Document](https://dx.doi.org/10.18653/v1/N19-1329)Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p2.1)\.
- D\. Schneider, V\. P\. Slaughter, S\. I\. Becker, and P\. E\. Dux \(2014\)Implicit false\-belief processing in the human brain\.NeuroImage101,pp\. 268–275\.Cited by:[§2\.1](https://arxiv.org/html/2606.28524#S2.SS1.p2.1)\.
- M\. Tomasello and P\. J\. Brooks \(2016\)Early syntactic development: a construction grammar approach\.InThe development of language,pp\. 161–190\.Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p1.1)\.
- A\. Warstadt, A\. Mueller, L\. Choshen, E\. Wilcox, C\. Zhuang, J\. Ciro, R\. Mosquera, B\. Paranjabe, A\. Williams, T\. Linzen, and R\. Cotterell \(2023\)Findings of the BabyLM challenge: sample\-efficient pretraining on developmentally plausible corpora\.InProceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning,A\. Warstadt, A\. Mueller, L\. Choshen, E\. Wilcox, C\. Zhuang, J\. Ciro, R\. Mosquera, B\. Paranjabe, A\. Williams, T\. Linzen, and R\. Cotterell \(Eds\.\),Singapore,pp\. 1–34\.External Links:[Link](https://aclanthology.org/2023.conll-babylm.1/),[Document](https://dx.doi.org/10.18653/v1/2023.conll-babylm.1)Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p2.1)\.
- H\. Wickham \(2016\)Ggplot2: elegant graphics for data analysis\.Springer\-Verlag New York\.External Links:ISBN 978\-3\-319\-24277\-4,[Link](https://ggplot2.tidyverse.org/)Cited by:[§4](https://arxiv.org/html/2606.28524#S4.p1.1)\.
- T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz, J\. Davison, S\. Shleifer, P\. von Platen, C\. Ma, Y\. Jernite, J\. Plu, C\. Xu, T\. Le Scao, S\. Gugger, M\. Drame, Q\. Lhoest, and A\. Rush \(2020\)Transformers: state\-of\-the\-art natural language processing\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,Q\. Liu and D\. Schlangen \(Eds\.\),Online,pp\. 38–45\.External Links:[Link](https://aclanthology.org/2020.emnlp-demos.6/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-demos.6)Cited by:[§3\.3](https://arxiv.org/html/2606.28524#S3.SS3.p3.2)\.
- R\. Zhao, N\. Saphra, and S\. M\. Kakade \(2024\)Distributional scaling laws for emergent capabilities\.InNeurIPS 2024 Workshop on Scientific Methods for Understanding Deep Learning,Cited by:[§2\.2](https://arxiv.org/html/2606.28524#S2.SS2.p2.1)\.

## Appendix AAppendix

### A\.1False Belief Task: Sample Stimuli

Knowledge State: False Belief Knowledge Cue: Explicit Item Start Loc Appears First Item End Loc Appears Last David and Marta go out to get some wine for the party\. When they get home, David stores the wine in thegarageand grabs a drink from the fridge\. Then, David goes out to get some snacks\.While David is gone, Marta decides the wine would be best cooled, so she moves the wine out of the garage and into thefridge\. David returns home and wants to put out the wine\. Davidthinksthe wine is in the

Knowledge State: False Belief Knowledge Cue: Implicit Item Start Loc Appears First Item End Loc Appears Last David and Marta go out to get some wine for the party\. When they get home, David stores the wine in thegarageand grabs a drink from the fridge\. Then, David goes out to get some snacks\.While David is gone, Marta decides the wine would be best cooled, so she moves the wine out of the garage and into thefridge\. David returns home and wants to put out the wine\. Davidgoesto get the wine from the

Knowledge State: True Belief Knowledge Cue: Explicit Item Start Loc Appears First Item End Loc Appears Last David and Marta go out to get some wine for the party\. When they get home, David stores the wine in thegarageand grabs a drink from the fridge\. However, Marta decides the wine would be best cooled\.David watchesMarta move the wine out of the garage and into thefridge\. Then, David goes out to get some snacks\. When he returns home, he wants to put out the wine\. Davidthinksthe wine is in the

Knowledge State: True Belief Knowledge Cue: Implicit Item Start Loc Appears First Item End Loc Appears Last David and Marta go out to get some wine for the party\. When they get home, David stores the wine in thegarageand grabs a drink from the fridge\. However, Marta decides the wine would be best cooled\.David watchesMarta move the wine out of the garage and into thefridge\. Then, David goes out to get some snacks\. When he returns home, he wants to put out the wine\. Davidgoesto get the wine from the

Variants of these four passages then cover the remaining permutations for the sequence of Start and End location mentions\. We provide one example variant below, matched to the immediately preceding example box above in all but the sequence of start and end location mentions in the text:

Knowledge State: True Belief Knowledge Cue: Implicit Item End Loc Appears First Item Start Loc Appears Last David and Marta go out to get some wine for the party\. When they get home, David grabs a drink from thefridgeand stores the wine in the garage\. However, Marta decides the wine would be best cooled\.David watchesMarta move the wine into the fridge from thegarage\. Then, David goes out to get some snacks\. When he returns home, he wants to put out the wine\. Davidgoesto get the wine from the

### A\.2Situation Modeling: Minimal Pair Queries

The original queries described in Section[3\.2](https://arxiv.org/html/2606.28524#S3.SS2)were designed to match the way that information was presented in the original passage stimuli fromtrott\_large\_2023\. However, given Olmo2213​b13b’s sensitivity to surface properties of stimuli, it was possible that the results reported inFigure[7](https://arxiv.org/html/2606.28524#S4.F7)may have been driven by the form of the queries, which varied in whether or not the agent was named, and in which verb was used \(“put”, for Target; “moved” for Antagonist\)\. To rule out this possibility, we presented modified Situation Modeling task queries that were minimally different from each other:

- •At the start of the story, the book was in
- •At the end of the story, the book was in
- •The person who put the book in the box was
- •The person who put the book in the basket was

We extracted probabilities over correct and distractor locations and agents from Olmo2213​b13b’s final checkpoint, and we found the results consistent with those obtained for the original Situation Modeling query set\. Namely, the LM displayed a bias in favor of the Antagonist agent, even for queries for which the Target agent was the correct answer\.

### A\.3Additional Performance Visualizations

We additionally include here LM performance visualized as the log of the ratios of probabilities assigned to the Start versus End locations, which offers a transparent view of the LM’s “responses”\. All of the figures and data tables below are referenced in the Main Text, along with the figures and results they supplement\.

![Refer to caption](https://arxiv.org/html/2606.28524v1/x4.png)Figure 8:Companion to Main TextFigure[7](https://arxiv.org/html/2606.28524#S4.F7)\. Depicts Olmo2213​b13b’s final checkpoint accuracies for minimal pair versions of the original Situation Modeling set \(subpanel titles illustrate the form of the new queries\)\. Situation Modeling results are in grey; False Belief task performance for reference, in teal\. These results match the overall trends observed inFigure[7](https://arxiv.org/html/2606.28524#S4.F7)\.Table 1:Language models \(LMs\) examined in the present study, and the cumulative maximum number of tokens they observed by the end of pretraining\. Olmo22tokens are reported for “stage 1” of pretraining\.Table 2:Mean FB task accuracy by Knowledge State\. Bolded entries correspond to the larger accuracy in each row\. All LMs tested, regardless of family or size, performed best in the False Belief Task when the target agent had the same information as the observer \(e\.g\. LMs perform egocentrically, on average\)\. Note that Pythia12​b12bapproximates Olmo2213​b13b’s accuracy on the True Belief, but not False Belief, condition\.Table 3:Mean FB Task Accuracy Over Pretraining, by Knowledge State and Knowledge Cue\. The crossover interaction \(significant, according to the mixed effects model described in Results Section[4\.1](https://arxiv.org/html/2606.28524#S4.SS1)betweenKnowledge StateandKnowledge Cueis appreciable here\. Note that these are mean accuracies over pretraining, not the final checkpoint\. For Olmo2213​b13b, False Belief, Implicit mean accuracy is lower than the final checkpoint’s accuracy, which achieves slightly greater\-than\-chance performance\.![Refer to caption](https://arxiv.org/html/2606.28524v1/x5.png)Figure 9:Companion toFigure[4](https://arxiv.org/html/2606.28524#S4.F4)\. Log\-odds of the ratio of probabilities assigned to Start versus End locations by the number of tokens observed during pretraining, and split byKnowledge Stateand color\-coded byKnowledge Cue\. We added an opacity layer over each subpanel to better highlight the differences in log\-odds between the majority of stage11pretraining and the sudden shifts that emerge towards the end\.![Refer to caption](https://arxiv.org/html/2606.28524v1/x6.png)Figure 10:Companion to Main TextFigure[5](https://arxiv.org/html/2606.28524#S4.F5)\. Subpanels depict mean log\-odds across various training stages, colored according toKnowledge State, and with rows indicating whether theKnowledge Cuewas Explicit or Implicit\.![Refer to caption](https://arxiv.org/html/2606.28524v1/figures/all-lms-control-task-acc-by-tokensseen.png)

![Refer to caption](https://arxiv.org/html/2606.28524v1/figures/all-lms-control-task-lo-by-tokensseen.png)

Figure 11:Control task \(Antagonist Belief Tracking\) accuracies for all LMs tested, for the Olmo22suite LMs’ final checkpoint of stage11; and over the course of pretraining for Pythia suite LMs and Olmo2213​b13b\. Observations are color\-coded byKnowledge Stateand the marker shape corresponds to the presence or absence ofKnowledge Cuein the passages\. Companion to Main TextFigure[6](https://arxiv.org/html/2606.28524#S4.F6)\.