语言模型保留已放弃的解析：花园路径句后的残留与适应

# Language Models Keep Abandoned Analyses:Residue and Adaptation after Garden-Path Sentences
Source: [https://arxiv.org/html/2610.36797](https://arxiv.org/html/2610.36797)
###### Abstract

ReadingWhile the man hunted the deer ran into the woodsrequires giving up the analysis in whichthe deeris the object ofhunted\. People who have just read such a sentence still produce more transitive structures, so the abandoned analysis primes later language\. We ask whether language models keep using an analysis they have abandoned, and what in the model carries it\. In 24 open models from 124M to 70B parameters, a garden\-path sentence leaves two traces in the context\. A residue of the abandoned analysis raises later object attachment, by about three quarters of the effect of a sentence in which the noun phrase really is an object, and it appears in sampled text, on held\-out and human\-normed items and for a second type of garden path\. An adaptation toward the resolved structure scales with the sentence’s disambiguation error and acts mainly on comprehension\-like measures\. Activation patching separates the two\. Later text reads the residue from the role information in the noun phrase’s states, which were computed under the object analysis before the disambiguation, and replacing these states removes almost all of the residue in the 32B and 70B models\. The adaptation is read from the states that processed the disambiguation\. Overridden statements show the same persistence \(a corrected value keeps pulling later predictions in 8 of 9 models\), and in most models the pull is carried by the positions after the statement\. Finally, the human paradigm transfers only with controls for punctuation, because the models also write garden\-path sentences themselves, which reverses the paradigm’s raw statistic\.

While the man huntedthe deerran into the woods\.computed under the object analysiscomputed after the disambiguationresidueof the abandoned analysisadaptationtoward the resolved structurenext sentence:*While the nurse washed …*object attachment in production: raised by the residuesurprise at a later garden path: lowered by the adaptationFigure 1:Two traces of a garden\-path sentence in a causal language model\. Later text reads a residue of the abandoned object analysis from the states ofthe deer, which were computed before the disambiguation \(§[6](https://arxiv.org/html/2610.36797#S6)\)\. The main\-clause states carry an adaptation toward the resolved structure\. The two traces push later processing in opposite directions, and their balance differs between production\-like and comprehension\-like measures \(§[5](https://arxiv.org/html/2610.36797#S5)\)\.## 1Introduction

Garden\-path sentences make a reader build an analysis and then give it up\. InWhile the man hunted the deer ran into the woods, the noun phrasethe deeris first attached as the object ofhunted, and the verbranreveals that it is the subject of the main clause\([Frazier and Rayner, 1982](https://arxiv.org/html/2610.36797#bib.bib12)\)\. In language models \(LMs\), which have become a standard test bed for such sentences, surprisal at the disambiguating word predicts that garden\-path effects occur but underestimates their size\([Futrell et al\., 2019](https://arxiv.org/html/2610.36797#bib.bib14);[van Schijndel and Linzen, 2021](https://arxiv.org/html/2610.36797#bib.bib49);[Arehalli et al\., 2022](https://arxiv.org/html/2610.36797#bib.bib3);[Huang et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib20)\)\. AskedDid the man hunt the deer?, they often answer yes, like human readers who keep part of the initial interpretation\([Christianson et al\., 2001](https://arxiv.org/html/2610.36797#bib.bib8);[Li et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib26);[Amouyal et al\., 2025](https://arxiv.org/html/2610.36797#bib.bib1)\)\. These studies examine the garden\-path sentence itself, either the difficulty of its disambiguation or the accuracy of its final interpretation\.

The abandoned analysis can also shape what comes after the sentence\.[van Gompel et al\. \(2006\)](https://arxiv.org/html/2610.36797#bib.bib47)had participants read a garden\-path sentence and then complete a fragment such asWhile the nurse washed\. Completions were more often transitive after the garden path than after a version disambiguated by a comma \(33\.6% against 29\.1% in their first experiment\), so the object analysis that had been ruled out still primed later production\. For LMs this question has a specific architectural form\. A causal transformer computes the representation of each token when the token is read, and later tokens retrieve these representations through attention\. Whenranarrives, the states ofthe deerhave already been computed under the object analysis, and they stay in the context unchanged\. The states computed after the disambiguation, in contrast, can encode the corrected analysis\. Whether later text is shaped by the abandoned analysis therefore depends on which of these states it reads, and this cannot be decided from the architecture alone\.

Studies of structural priming in LMs have so far not been able to address this question\. LMs reuse the structure of a preceding sentence\([Sinclair et al\., 2022](https://arxiv.org/html/2610.36797#bib.bib39);[Prasad et al\., 2019](https://arxiv.org/html/2610.36797#bib.bib33);[Michaelov et al\., 2023](https://arxiv.org/html/2610.36797#bib.bib29);[Pucci et al\., 2026](https://arxiv.org/html/2610.36797#bib.bib34)\), and they do so more after less expected primes, an inverse\-frequency effect that[Zhou et al\. \(2025\)](https://arxiv.org/html/2610.36797#bib.bib56)take as evidence that in\-context learning is error\-driven\([Jumelet et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib23), see also\)\. In human research the same effect separates two accounts of priming\. Residual activation holds that a processed structure stays active and is reused because it is active\([Pickering and Branigan, 1998](https://arxiv.org/html/2610.36797#bib.bib30)\)\. Error\-driven learning holds that a prime adjusts expectations in proportion to its prediction error\([Chang et al\., 2006](https://arxiv.org/html/2610.36797#bib.bib7);[Jaeger and Snider, 2013](https://arxiv.org/html/2610.36797#bib.bib21)\)\. With unambiguous primes, both accounts predict priming of the prime’s structure, and only the scaling with prediction error tells them apart\.

Garden\-path primes add a second contrast\. For the abandoned object analysis, residual activation predicts reuse that grows with how strongly the analysis was built\. Error\-driven learning predicts the opposite tendency, because the error occurs when the object analysis fails, and learning from it should favour the resolved structure in whichthe deerstarts a new clause\. Human data show only the net result\. LMs let us measure the effect of a garden\-path prime on later text, relate it to the strength of the abandoned analysis and to the disambiguation error, and locate in the context what carries each part\.

We pair garden\-path primes from five published item sets with target fragments and measure how a prime changes a production\-like quantity, the preference for attaching the target’s noun phrase as an object, and a comprehension\-like quantity, the surprisal of the disambiguating word in a garden\-path target\. Across 24 models we compare primes matched for punctuation and verb, fix the predictions of each confirmatory experiment before collecting its data, and use activation patching to locate the effects inside the prime\.

A garden\-path prime raises later object attachment by about three quarters of the effect of a prime whose noun phrase really is an object, and most of this residue requires a noun phrase that could have been an object\. The same prime produces an adaptation toward the resolved structure that grows with its disambiguation error and acts mainly on the comprehension\-like measure, and patching shows that later text reads the two from different positions of the prime\. To see how far the persistence extends, we also test statements that a later sentence corrects\. The overridden value keeps influencing later predictions, but through a different route than the garden path\.

Our contributions are as follows\. \(1\) We show that LMs carry an abandoned syntactic analysis into later production, across 24 models, three item sets and two types of garden path, with controls that rule out punctuation, verb bias and local string overlap as its source\. \(2\) We show that the same prime also produces an adaptation toward the resolved analysis, that the two effects scale with different properties of the prime \(the strength of the abandoned analysis and the disambiguation error\), and that their balance differs between production\-like and comprehension\-like measures\. \(3\) We show by activation patching, probing and a role\-direction intervention that later text reads the abandoned analysis from the role information in noun\-phrase states computed before the disambiguation, even though the states after it encode the reanalysis\. \(4\) We show that overridden statements also persist, in most models through later positions, and that the human paradigm’s statistic reverses in LMs unless punctuation is controlled\.

## 2Related Work

### Garden paths in language models\.

Garden\-path sentences were among the first constructions used to ask whether neural LMs track syntactic state\([Futrell et al\., 2019](https://arxiv.org/html/2610.36797#bib.bib14)\)\. Surprisal from such models predicts that the effects exist but underestimates their size and misorders constructions\([van Schijndel and Linzen, 2021](https://arxiv.org/html/2610.36797#bib.bib49);[Arehalli et al\., 2022](https://arxiv.org/html/2610.36797#bib.bib3);[Huang et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib20)\)\. Recent work relates this gap to the number of parses a model keeps alive\([Timkey et al\., 2026](https://arxiv.org/html/2610.36797#bib.bib45)\)or closes it with suitably trained models\([Yoshida et al\., 2026](https://arxiv.org/html/2610.36797#bib.bib54)\)\. Comprehension studies ask whether the final interpretation is correct, and find that LMs often accept the initial misreading, with plausibility and verb type modulating this as in humans\([Li et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib26);[Amouyal et al\., 2025](https://arxiv.org/html/2610.36797#bib.bib1)\), and difficulty with such structures persists across model scale\([Amouyal et al\., 2026](https://arxiv.org/html/2610.36797#bib.bib2)\)\.[Hanna and Mueller \(2025\)](https://arxiv.org/html/2610.36797#bib.bib17)found features for both readings of a garden\-path prefix in small models and showed that the circuit that answers comprehension questions hardly reuses them\. None of this work asks what the abandoned analysis does to later text\.

### Structural priming and adaptation\.

Speakers reuse structures they have recently processed\([Bock, 1986](https://arxiv.org/html/2610.36797#bib.bib5);[Pickering and Branigan, 1998](https://arxiv.org/html/2610.36797#bib.bib30)\)\. Residual activation and error\-driven learning\([Chang et al\., 2006](https://arxiv.org/html/2610.36797#bib.bib7)\)are the two main accounts, and computational models combine short\-lived activation with longer\-lived learning\([Reitter et al\., 2011](https://arxiv.org/html/2610.36797#bib.bib37)\), in line with a lexical boost that decays while abstract priming persists\([Hartsuiker et al\., 2008](https://arxiv.org/html/2610.36797#bib.bib19)\)\. Less expected primes prime more\([Jaeger and Snider, 2013](https://arxiv.org/html/2610.36797#bib.bib21)\), and readers adapt to the main\-verb/reduced\-relative garden path with exposure\([Fine et al\., 2013](https://arxiv.org/html/2610.36797#bib.bib11)\), although such rapid adaptation is small and hard to replicate\([Harrington Stack et al\., 2018](https://arxiv.org/html/2610.36797#bib.bib18);[Prasad and Linzen, 2021](https://arxiv.org/html/2610.36797#bib.bib32)\)\.[van Gompel et al\. \(2006\)](https://arxiv.org/html/2610.36797#bib.bib47)found priming of the abandoned analysis of NP/Z garden paths, and[Fujita and Cunnings \(2020\)](https://arxiv.org/html/2610.36797#bib.bib13)found that unambiguous primes ease the reading of later garden paths without removing lingering misinterpretation\. In LMs, conditioning on a prime raises the probability of its structure in a target\([Prasad et al\., 2019](https://arxiv.org/html/2610.36797#bib.bib33);[Sinclair et al\., 2022](https://arxiv.org/html/2610.36797#bib.bib39)\), across languages\([Michaelov et al\., 2023](https://arxiv.org/html/2610.36797#bib.bib29)\)and in generation\([Pucci et al\., 2026](https://arxiv.org/html/2610.36797#bib.bib34)\)\. The effect grows for less expected primes and with lexical overlap\([Jumelet et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib23);[Zhou et al\., 2025](https://arxiv.org/html/2610.36797#bib.bib56)\), and gradient updates on garden\-path sentences improve the fit of RNN surprisal to human adaptation\([van Schijndel and Linzen, 2018](https://arxiv.org/html/2610.36797#bib.bib48)\)\. Apart from gradient\-based adaptation to garden paths\([van Schijndel and Linzen, 2018](https://arxiv.org/html/2610.36797#bib.bib48)\), these studies use unambiguous primes\.

### Overridden information in context\.

Information that later context overrides can continue to affect LMs\. Transformer models sometimes refer to discourse entities that a sentence did not introduce\([Schuster and Linzen, 2022](https://arxiv.org/html/2610.36797#bib.bib38)\), and models rarely retract their own wrong answers\([Yang and Jia, 2026](https://arxiv.org/html/2610.36797#bib.bib53)\)or correct themselves without external feedback\([Kamoi et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib24)\)\. Later tokens retrieve earlier bindings by attending back to the positions where they were encoded\([Prakash et al\., 2026](https://arxiv.org/html/2610.36797#bib.bib31)\), and activation patching, which replaces internal states with states from a counterfactual input, is the standard tool for locating such information\([Vig et al\., 2020](https://arxiv.org/html/2610.36797#bib.bib50);[Meng et al\., 2023](https://arxiv.org/html/2610.36797#bib.bib28);[Zhang and Nanda, 2024](https://arxiv.org/html/2610.36797#bib.bib55)\)\. In human sentence processing, lingering misinterpretations have been attributed to competing representations of the two analyses\([Slattery et al\., 2013](https://arxiv.org/html/2610.36797#bib.bib40)\)and to shallow, good\-enough interpretation\([Ferreira, 2003](https://arxiv.org/html/2610.36797#bib.bib10)\)\. Garden paths offer a case of overriding in which the overridden analysis, its strength and the point at which it is overridden are all known\.

## 3Experimental Setup

### Materials\.

An item family is one garden\-path sentence with its controls\. The discovery set contains 93 NP/Z families, 69 from[Amouyal et al\. \(2025\)](https://arxiv.org/html/2610.36797#bib.bib1)and 24 from the SAP benchmark\([Huang et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib20)\)\. A held\-out set of 48 families combines 24 items from[Ceháková and Chromý \(2025\)](https://arxiv.org/html/2610.36797#bib.bib6)with 24 from SyntaxGym\([Gauthier et al\., 2020](https://arxiv.org/html/2610.36797#bib.bib15)\), and was used only after the analyses had been fixed on the discovery set\. A third set contains the 42 optionally transitive items of[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8), each with a short and a long ambiguous region and a plausible and an implausible object\. Each family provides the primes in Table[1](https://arxiv.org/html/2610.36797#S3.T1)\.gpis the garden\-path sentence andcueits version with a comma after the verb\.tbreplaces the main\-clause subject with a pronoun, so that the noun phrase remains the object\.nbandncreplace the ambiguous verb with a strictly intransitive one, without and with the comma\.

PrimeExamplegpWhile the man hunted the deer ran into the woods\.cueWhile the man hunted, the deer ran into the woods\.tbWhile the man hunted the deer it ran into the woods\.nb/ncWhile the man slept\(,\) the deer ran into the woods\.nomWhile the man hunted they ran into the woods\.passWhile the man was hunted the deer ran into the woods\.TargetQuantityLLlog⁡P⁡\(…washed the baby,\)−log⁡P⁡\(…washed, the baby\)\\log P\(\\text\{\{\\ldots washed the baby,\}\}\)\-\\log P\(\\text\{\{\\ldots washed, the baby\}\}\)SS−log⁡P⁡\(cried∣While the nurse washed the baby\)\-\\log P\(\\text\{\{cried\}\}\\mid\\text\{\{While the nurse washed the baby\}\}\)GGsampled continuation ofWhile the nurse washedTable 1:Prime conditions and target quantities, illustrated with one prime and one target family\.nomandpasstest punctuation and local coherence \(§[4](https://arxiv.org/html/2610.36797#S4)\)\.
### Targets and measures\.

Each prime is followed by a target from every other family that shares no content word with it, which removes lexical repetition and gives each discovery prime 78 to 92 targets\. For a target family with subjectss, verbvvand noun phrasenn, the production\-like measure is

L=log⁡P⁡\(While​s​v​n​,\)−log⁡P⁡\(While​s​v​,​n\),L=\\log P\(\\text\{\{While\}\}\\ s\\ v\\ n\\text\{,\}\)\-\\log P\(\\text\{\{While\}\}\\ s\\ v\\text\{,\}\\ n\),\(1\)where both probabilities are conditioned on the prime and the two strings contain the same words\.LLis the log\-odds of attachingnnas the object ofvvrather than starting a new clause with it\. The comprehension\-like measureSSis the surprisal of the first word of the target’s main clause after the comma\-less fragmentWhilessvvnn, a quantity that contains no punctuation choice\. A highSSmeans that the model expectednnto be an object\. For sampled production, the model continuesWhilessvv, and a rule system over part\-of\-speech tags codes whether the continuation attaches a noun phrase as an object \(Appendix[H](https://arxiv.org/html/2610.36797#A8)\)\.

### The residue contrast\.

Comparinggpwithcueconfounds the abandoned analysis with the comma, and comparinggpwithnbconfounds it with the verb\. For any measureXXwe use

AX=\(Xgp−Xnb\)−\(Xcue−Xnc\),A\_\{X\}=\\big\(X\_\{\\textsc\{gp\}\}\-X\_\{\\textsc\{nb\}\}\\big\)\-\\big\(X\_\{\\textsc\{cue\}\}\-X\_\{\\textsc\{nc\}\}\\big\),\(2\)whereXcX\_\{c\}is the mean ofXXover prime families and targets after primes of conditioncc\. Within each bracket the two primes have the same punctuation, and across brackets the same verbs\.AXA\_\{X\}is therefore the part of the prime’s effect that requires both a missing comma and a verb that can take an object, the condition under which the object analysis is built and then abandoned\. The same contrast withtbin place ofgp,AXtbA\_\{X\}^\{\\textsc\{tb\}\}, measures the effect of a noun phrase that really is an object, and we reportr=AX/AXtbr=A\_\{X\}/A\_\{X\}^\{\\textsc\{tb\}\}as the size of the residue relative to it\.

### Models and inference\.

We use 24 open\-weight causal LMs from the GPT\-2, Pythia, Llama, Mistral, Gemma and Qwen families, base and instruction\-tuned, between 124M and 70B parameters and including one mixture\-of\-experts model \(Appendix[A](https://arxiv.org/html/2610.36797#A1)\)\. 22 of them enter every priming experiment, and two more enter the[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8)set\. Scoring uses exact log\-probabilities without chat templates, and sampling uses temperature 1 without truncation\. Intervals come from a two\-way bootstrap that resamples prime families and target families independently\. For averages across models, each bootstrap draw applies the same resampled families to every model, which keeps the dependence that arises because all models see the same items\. For every confirmatory experiment we fixed predictions and analysis in a dated record before collecting its data, and Appendix[D](https://arxiv.org/html/2610.36797#A4)lists every outcome\.

Figure 2:The residue of the abandoned object analysis\. Left:ALA\_\{L\}\(Eq\.[2](https://arxiv.org/html/2610.36797#S3.E2)\) per model on the discovery, held\-out and[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8)item sets\. Right: the same contrast on sampled completions, as a difference in the proportion of transitive continuations\. Error bars are 95% intervals from a two\-way bootstrap over prime and target families\. Models are ordered by size\.

## 4The Abandoned Analysis Persists

### The residue in production\.

After a garden\-path prime, models prefer object attachment in the target more than the controls predict \(Figure[2](https://arxiv.org/html/2610.36797#S3.F2)\)\. On the discovery setALA\_\{L\}is positive in all 22 models and excludes zero in 21, with Pythia\-160M as the exception\. Averaged over models it is 0\.72 nats, which is 76% ofALtbA\_\{L\}^\{\\textsc\{tb\}\}, the effect of a prime whose noun phrase really is the object\. The residue replicates on the held\-out families, where it is positive in all 22 models \(20 with intervals above zero\) with a median of 0\.36 nats, and on the items of[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8), where it is positive in all 24 models with a median of 1\.05 nats\.

### Punctuation habits\.

A comma\-less prime is unusual in written text, and a model could learn from it that this writer omits commas after verbs that can take objects\. Such a habit would raiseLLwithout any object analysis\. Thenomprime keeps the verb and the missing comma ofgpbut replaces the noun phrase with the nominative pronounthey, which cannot be an object\. If the residue were a punctuation habit,gpandnomwould prime alike\. Instead, the contrast of Equation[2](https://arxiv.org/html/2610.36797#S3.E2)withnomand its intransitive counterpart in place ofcueandncis positive in all 22 models \(21 significant, 0\.47 nats on average\), and across families it grows with the model’s expectation of an object after the ambiguous verb in all 22 models\. The case contrast is two thirds of the comma contrast \(0\.47 against 0\.72 nats\), so most of the residue requires a noun phrase that could be an object, and the rest follows from the object expected after the verb\.

### Local string overlap\.

The stringhunted the deermight primewashed the babyby local coherence, as it disrupts human reading even where the grammar excludes an object reading\([Tabor et al\., 2004](https://arxiv.org/html/2610.36797#bib.bib42)\)\. Thepassprime contains the same string inside a passive clause, wherethe deercannot be an object\. Its residue has a median of 12% of thegpresidue across 22 models, reliably negative in seven, mostly small, models and reliably positive in seven larger ones\. The ratio is higher in the strongest models, 0\.45 in Llama\-3\.1\-70B and above 0\.5 in the two Qwen2\.5\-7B models\. In these models a locally computed object reading of the string also leaves a trace\.

### Sampled production\.

We sampled two continuations of every target fragment after every prime in 12 models, about 57,000 prompts per model\. The residue appears in the proportion of transitive continuations in 11 of 12 models with the pre\-specified coding and in all 12 with a refined coding, by 5\.0 percentage points on average \(Figure[2](https://arxiv.org/html/2610.36797#S3.F2), right\)\. Its size across models followedALA\_\{L\}in the eight models available when we specified this prediction \(Spearmanρ=0\.81\\rho=0\.81\), and the relation is weaker across all twelve \(ρ=0\.48\\rho=0\.48\)\. The human raw contrast between garden\-path and comma primes was 4\.5 points\([van Gompel et al\., 2006](https://arxiv.org/html/2610.36797#bib.bib47)\), a statistic that cannot be compared directly with LMs \(§[8](https://arxiv.org/html/2610.36797#S8)\)\.

### A second garden path\.

In an NP/S garden path \(The student forgot the solution was in the back of the book\) the noun phrase is also first analysed as an object\. We built the same contrast for 64 NP/S families, withthatas the disambiguating cue and a verb that takes only a clausal complement \(hoped\) as the control, and measured it on the NP/Z targets\. The residue is positive in 19 of 22 models \(sign testp<0\.001p<0\.001\) with a pooled value of 0\.03 nats\. It is smaller than for NP/Z primes because the prime and target structures differ, and even a real object in an NP/S prime moves the NP/Z targets only by 0\.03 to 0\.20 nats\.

## 5Two Effects with Opposite Signs

On the comprehension\-like measure the residue is small on average \(0\.05 nats over 22 models\) and varies in sign, significantly positive in five models and significantly negative in four\. A negative value means that after a garden\-path prime the model expects an object less than the controls predict\. This is what an adaptation toward the resolved structure would produce, and it suggests that a second effect of the prime offsets the residue\.

### Commitment and error\.

The two accounts of priming tie the effect of a prime to different properties\. Residual activation depends on how strongly the abandoned analysis was built, which we measure as the model’s expectation of an object after the ambiguous verb,log⁡P⁡\(the\)−log⁡P⁡\(,\)\\log P\(\\text\{\{the\}\}\)\-\\log P\(\\text\{\{,\}\}\)\. Error\-driven learning depends on the error at the disambiguation, which we measure as the garden\-path strength of the prime, the surprisal of its disambiguating word without the comma minus that with the comma\. The two predictors are only weakly correlated \(medianr=0\.22r=0\.22\)\. In a regression of the family\-level residue on both \(Figure[5](https://arxiv.org/html/2610.36797#A3.F5)in Appendix[C](https://arxiv.org/html/2610.36797#A3)\), object expectation raises the residue on both measures in all 22 models\. Garden\-path strength lowers the residue onSSin 21 of 22 models, in all 22 when the family’s adaptation to the missing comma enters as a covariate \(19 significant\), whereas onLLit raises the residue in most models\. The error effect is specific to garden\-path resolutions: on the comma version of the same target, where the disambiguating word is expected, garden\-path strength has little effect \(mediant=−0\.6t=\-0\.6, against−2\.8\-2\.8onSS\), so the adaptation is not a general loss of confidence after a surprising prime\.

### Causal tests\.

Garden\-path strength varies between families together with other properties\. To manipulate it within a family, we kept each prime up to the noun phrase fixed, which fixes the object analysis, and varied only its continuation, which changes the surprisal at the disambiguation\. More surprising disambiguations reduce the residue onSS, withttvalues between−4\.1\-4\.1and−6\.4\-6\.4in the three models whose between\-family analysis had shown an error effect, and near zero in the two models in which it had not\. Surprisal summed over the whole continuation predicts little, so the relevant error is the one at the disambiguation point\. On the items of[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8), where plausibility changes only the main\-clause continuation, the same within\-family relation is negative in all ten models that pass the comprehension controls of §[8](https://arxiv.org/html/2610.36797#S8)\. The residue is tested causally by the patching experiments of §[6](https://arxiv.org/html/2610.36797#S6)\(see Appendix[F](https://arxiv.org/html/2610.36797#A6)for a behavioural manipulation\)\.

### Production and comprehension\.

The residue dominates the production\-like measure, and the adaptation shows mainly in the comprehension\-like one\. We tested this asymmetry prospectively on the items of[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8)with the 16 models available at that time\. The residue onLLwas positive in all 16, and the error relation was more negative onSSthan onLLin 14\. Human studies examine these two effects in separate paradigms\([van Gompel et al\., 2006](https://arxiv.org/html/2610.36797#bib.bib47);[Fine et al\., 2013](https://arxiv.org/html/2610.36797#bib.bib11)\), whereas in LMs they arise from the same prime\.

### Other garden paths\.

We tested the two item\-level relations onSSfor two further constructions with targets of the same construction\. For NP/S garden paths neither relation appeared\. For main\-verb/reduced\-relative garden paths the error relation appeared in six of nine models and the commitment relation in none\. NP/S reanalysis keeps the noun phrase inside the verb’s complement, whereas NP/Z and main\-verb/reduced\-relative reanalysis remove a committed attachment\([Sturt and Crocker, 1996](https://arxiv.org/html/2610.36797#bib.bib41)\), which may explain why only the latter produce an adaptation, while the NP/S residue of §[4](https://arxiv.org/html/2610.36797#S4)shows that the abandoned object analysis persists in both\.

## 6Where the Residue Lives

In the garden\-path prime, the states ofthe deerare identical to those in the transitive primetb, since the two share every preceding token\. Later tokens can read these states directly, or the main\-clause states, which were computed after the disambiguation\. If the residue is read from the noun\-phrase states, replacing them with states computed under the subject analysis should remove it, while replacing the main\-clause states should not\.

### Patching design\.

At every layer, the residual\-stream states of the noun\-phrase tokens ingpare replaced with the states of the same tokens incue, where the comma has already made the noun phrase a subject, and the rest of the prime is recomputed\. The target then attends to a mixture of the original and this counterfactual prime: counterfactual keys and values at the noun phrase \(NP\), at the main clauseran into the woods\.\(MC\), or at both, and original ones elsewhere\. The same procedure applied tonbwithncas donor estimates what the replacement does by importing information about the comma, and Equation[2](https://arxiv.org/html/2610.36797#S3.E2)subtracts it\. We report the retained residueρ=ALpatched/AL\\rho=A\_\{L\}^\{\\text\{patched\}\}/A\_\{L\}and the changeΔ=ASpatched−AS\\Delta=A\_\{S\}^\{\\text\{patched\}\}\-A\_\{S\}for nine models between 1\.4B and 70B parameters\.

Figure 3:Patching the garden\-path prime\. Left: residue onLLretained when the target reads counterfactual subject\-analysis states at the noun phrase \(blue\) or at the main clause \(orange\)\. Right: noun\-phrase states replaced only from a given fraction of the network’s depth onward\. Values for both measures, with intervals, are in Appendix[E](https://arxiv.org/html/2610.36797#A5)\.
### The residue is read from the noun phrase\.

Replacing the noun\-phrase states leaves 0\.01 to 0\.71 of the residue \(pooled 0\.37\), and nothing beyond the interval bounds of 0\.21 and 0\.22 in Qwen2\.5\-32B and Llama\-3\.1\-70B \(Figure[3](https://arxiv.org/html/2610.36797#S6.F3), left\)\. Replacing the main\-clause states leaves it largely intact \(pooled 1\.01\), and in every model the noun phrase carries more of the residue than the main clause\. Withnbas donor, wherethe deerfollows an intransitive verb without a comma, the retained fraction is still 0\.04 to 0\.47 in the seven models tested, so the effect is not imported information about the comma\. Replacing the noun\-phrase states only in the upper half of the network has no effect in Llama\-3\.1\-8B\-Instruct and Mistral\-7B, and only in the upper quarter none in Qwen2\.5\-7B \(Figure[3](https://arxiv.org/html/2610.36797#S6.F3), right\)\. Later text thus reads the abandoned analysis from the noun phrase’s keys and values in the lower and middle layers\.

### What the noun\-phrase states carry\.

Replacing the noun\-phrase states with states of the same tokens computed under the object analysis in another context \(It was a quiet morning\.before the prime\) leaves 70% to 111% of the residue in seven models up to 70B, so the effect of the replacement lies in the analysis it removes\. The states encode that analysis as a role\. A linear probe trained only on noun phrases without a comma, object \(tb\) against subject \(nb\), classifies object and subject noun phrases that follow a comma \(tc,cue\) with 96% to 99% accuracy in all seven models\. Moving the states along the probe direction toward the subject class removes more of the residue than random directions in three of seven models and less than replacing the whole states in six \(Appendix[E](https://arxiv.org/html/2610.36797#A5)\), so the abandoned analysis is spread over more than one direction of the noun phrase’s representation\.

## 7Beyond Syntax: Overridden Statements

Does content that a context explicitly overrides also keep influencing an LM? Unlike conflicts between a context and a model’s parametric knowledge\([Longpre et al\., 2021](https://arxiv.org/html/2610.36797#bib.bib27);[Xie et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib51)\), both the overridden and the correct value here come from the context\. We built 200 items from ten frames \(for exampleThe meeting is on Thursday\. Correction: The meeting is on Monday\.\) and measured, after the probeTo confirm, the meeting is on, how much more likely the overridden value is than the correct one\. A control replaces the overridden statement with the same statement about another entity \(The dinner is on Thursday\.\), which matches exposure to the overridden value\. Relative to this control, the overridden value pulls later predictions toward itself by 0\.6 to 2\.3 nats in eight of nine models up to 70B, in nine or ten of the ten frames each\. Qwen2\.5\-7B\-Instruct is the exception and moves away from a corrected value\.

Attention knockout shows where the answer finds the overridden value\. Blocking the answer’s attention to the correcting sentence raises the pull in all nine models, so the knockout works as intended\. In six models, blocking the answer’s attention to the overridden statement leaves the pull intact, and blocking the access of all later positions to it removes the pull: the value reaches the answer through the positions after the statement, including the correction itself\. In Qwen2\.5\-7B, Qwen2\.5\-32B and Llama\-3\.1\-70B, blocking the answer’s own access already halves or removes the pull, so the answer reads the overridden statement directly, as later text reads the abandoned noun phrase in a garden path \(Appendix[G](https://arxiv.org/html/2610.36797#A7)\)\. Overridden content thus persists in both settings, and whether it is read from where it was stated or relayed by later positions depends on the model\.

## 8Transferring the Human Paradigm

Figure 4:Sampled production after garden\-path primes\. The raw contrast with comma primes used in the human study \(orange\) is negative in most Llama and Qwen models\. Without the completions that are themselves comma\-less garden\-path sentences it is positive in every model \(gray\), and so is the controlled residueAg​e​nA\_\{gen\}\(blue\)\.### The raw statistic reverses\.

The human study compared transitive completions aftergpandcueprimes\. In LMs this contrast is reliably positive in only 4 of 12 models and significantly negative in 7 of the 8 Llama and Qwen models, most strongly in Llama\-3\.1\-70B, where garden\-path primes lower the proportion of transitive completions by 17\.6 points \(Figure[4](https://arxiv.org/html/2610.36797#S8.F4)\)\. After a garden\-path prime, up to 29% of completions are themselves comma\-less garden\-path sentences, against 0\.2% to 2\.4% after a comma prime\. These completions produce the resolved structure of the prime, a new clause after an object\-less verb, in the prime’s comma\-less style\. They are the adaptation of §[5](https://arxiv.org/html/2610.36797#S5)expressed in production, and the raw contrast counts them as evidence against the residue\. Across models, the more such completions a model writes, the more negative its raw contrast \(ρ=−0\.82\\rho=\-0\.82\)\. Without them, the raw contrast is positive in all 12 models \(median 10 points\)\. Human participants completed the fragments aloud, without punctuation\. Matching primes for punctuation, as Equation[2](https://arxiv.org/html/2610.36797#S3.E2)does, is necessary to transfer the paradigm\.

### Lingering comprehension is a different measure\.

LMs answerDid the man hunt the deer?with yes after the garden\-path sentence much more often than after its comma version, and more so when the ambiguous region is longer, as human readers do\([Christianson et al\., 2001](https://arxiv.org/html/2610.36797#bib.bib8)\)\. Across the items of[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8), however, this lingering is not positively related to the priming residue \(rrbetween−0\.30\-0\.30and0\.150\.15in the ten models that answer the unambiguous control questions correctly\), although both measures have split\-half reliabilities between 0\.77 and 0\.98\. Question answering and priming measure different properties of the model, so studies of lingering in LMs\([Li et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib26);[Amouyal et al\., 2025](https://arxiv.org/html/2610.36797#bib.bib1)\)and the residue measured here are not interchangeable\.

## 9Discussion and Conclusion

A garden\-path sentence leaves both analyses in an LM’s context\. The abandoned one is read from the noun phrase, whose states were computed before the disambiguation, and its effect on later production grows with how strongly it was built\. The resolved one is read from the states after the disambiguation, and its effect grows with the disambiguation error\. Residual activation describes the first effect and error\-driven learning the second, with in\-context adaptation in the role of learning\([Zhou et al\., 2025](https://arxiv.org/html/2610.36797#bib.bib56)\), and the two representations of the competing\-representations view of lingering\([Slattery et al\., 2013](https://arxiv.org/html/2610.36797#bib.bib40)\)occupy different positions of the context\.

The patching result was not a foregone conclusion, since later text could have relied on the main\-clause states, which are more recent and encode the corrected analysis, and in the 32B and 70B models it reads almost the whole residue from the noun phrase instead\. A disambiguating word or a correcting sentence changes what later positions compute without removing the earlier representation, so appending a correction is a weaker intervention than removing the corrected text, which bears on why models refer to entities a sentence never introduced\([Schuster and Linzen, 2022](https://arxiv.org/html/2610.36797#bib.bib38)\)and rarely retract wrong answers\([Yang and Jia, 2026](https://arxiv.org/html/2610.36797#bib.bib53)\)\. Corrected statements persist in the same way, since eight of nine models still prefer a value after it has been corrected \(§[7](https://arxiv.org/html/2610.36797#S7)\), so applications that update what a model knows by appending a correction to its context leave the overridden version as an input to later predictions\.

For human research, varying only the continuation of a fixed garden\-path prefix would separate the two accounts: residual activation predicts no change in later comprehension of garden paths because the abandoned analysis is built identically, whereas error\-driven learning predicts more adaptation after more surprising disambiguations, as in LMs\.

## Limitations

We study English NP/Z garden paths in depth and NP/S and main\-verb/reduced\-relative garden paths in single experiments, so other languages and ambiguity types may behave differently\. Our models are open\-weight models up to 70B parameters, because the analyses require exact log\-probabilities and access to internal states, and closed models were not tested\. The behavioural manipulation of the abandoned analysis \(Appendix[F](https://arxiv.org/html/2610.36797#A6)\) did not pass its pre\-specified check, so the causal evidence for the residue comes from patching and the role\-direction intervention, which were run on six to nine of the 24 models\. The contrast of Equation[2](https://arxiv.org/html/2610.36797#S3.E2)was defined after inspecting the first two models of the discovery set, and its confirmation rests on held\-out items, a prospective test and pre\-specified predictions, some of which failed \(Appendix[D](https://arxiv.org/html/2610.36797#A4)\)\. The overridden statements are templated, and one pre\-specified prediction about their mechanism failed, which led to the different\-carrier interpretation of §[7](https://arxiv.org/html/2610.36797#S7)\. The coding of sampled completions is rule\-based and was checked against gold sentences and by one annotator on a sample of generations\. We derive a prediction for human readers but do not test it\.

## Ethical Considerations

This work involves no human participants and no personal data\. All models are public checkpoints used for scoring and sampling on local hardware\. The materials come from published studies and benchmarks and are used for research with citation\.

## References

- Amouyal et al\. \(2025\)Samuel Joseph Amouyal, Aya Meltzer\-Asscher, and Jonathan Berant\. 2025\.[When the LM misunderstood the human chuckled: Analyzing garden path effects in humans and language models](https://doi.org/10.18653/v1/2025.acl-long.403)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 8235–8253, Vienna, Austria\. Association for Computational Linguistics\.
- Amouyal et al\. \(2026\)Samuel Joseph Amouyal, Aya Meltzer\-Asscher, and Jonathan Berant\. 2026\.[Comparing human and language models sentence processing difficulties on complex structures](https://doi.org/10.18653/v1/2026.acl-long.1040)\.In*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 22704–22722, San Diego, California, United States\. Association for Computational Linguistics\.
- Arehalli et al\. \(2022\)Suhas Arehalli, Brian Dillon, and Tal Linzen\. 2022\.[Syntactic surprisal from neural models predicts, but underestimates, human processing difficulty from syntactic ambiguities](https://doi.org/10.18653/v1/2022.conll-1.20)\.In*Proceedings of the 26th Conference on Computational Natural Language Learning \(CoNLL\)*, pages 301–313, Abu Dhabi, United Arab Emirates \(Hybrid\)\. Association for Computational Linguistics\.
- Biderman et al\. \(2023\)Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal\. 2023\.[Pythia: A suite for analyzing large language models across training and scaling](https://arxiv.org/abs/2304.01373)\.*Preprint*, arXiv:2304\.01373\.
- Bock \(1986\)J\.Kathryn Bock\. 1986\.[Syntactic persistence in language production](https://doi.org/10.1016/0010-0285(86)90004-6)\.*Cognitive Psychology*, 18\(3\):355–387\.
- Ceháková and Chromý \(2025\)Markéta Ceháková and Jan Chromý\. 2025\.[Beyond initial misanalysis: the disruptive nature of garden\-path sentences processing](https://doi.org/10.1080/0163853x.2025.2513210)\.*Discourse Processes*, 62\(10\):723–743\.
- Chang et al\. \(2006\)Franklin Chang, Gary S\. Dell, and Kathryn Bock\. 2006\.[Becoming syntactic\.](https://doi.org/10.1037/0033-295x.113.2.234)*Psychological Review*, 113\(2\):234–272\.
- Christianson et al\. \(2001\)Kiel Christianson, Andrew Hollingworth, John F\. Halliwell, and Fernanda Ferreira\. 2001\.[Thematic roles assigned along the garden path linger](https://doi.org/10.1006/cogp.2001.0752)\.*Cognitive Psychology*, 42\(4\):368–407\.
- DerSimonian and Laird \(1986\)Rebecca DerSimonian and Nan Laird\. 1986\.[Meta\-analysis in clinical trials](https://doi.org/10.1016/0197-2456(86)90046-2)\.*Controlled Clinical Trials*, 7\(3\):177–188\.
- Ferreira \(2003\)F Ferreira\. 2003\.[The misinterpretation of noncanonical sentences](https://doi.org/10.1016/s0010-0285(03)00005-7)\.*Cognitive Psychology*, 47\(2\):164–203\.
- Fine et al\. \(2013\)Alex B\. Fine, T\. Florian Jaeger, Thomas A\. Farmer, and Ting Qian\. 2013\.[Rapid expectation adaptation during syntactic comprehension](https://doi.org/10.1371/journal.pone.0077661)\.*PLoS ONE*, 8\(10\):e77661\.
- Frazier and Rayner \(1982\)Lyn Frazier and Keith Rayner\. 1982\.[Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences](https://doi.org/10.1016/0010-0285(82)90008-1)\.*Cognitive Psychology*, 14\(2\):178–210\.
- Fujita and Cunnings \(2020\)Hiroki Fujita and Ian Cunnings\. 2020\.[Lingering misinterpretation in native and nonnative sentence processing: Evidence from structural priming](https://doi.org/10.1017/s0142716420000351)\.*Applied Psycholinguistics*, 42\(2\):475–504\.
- Futrell et al\. \(2019\)Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy\. 2019\.[Neural language models as psycholinguistic subjects: Representations of syntactic state](https://doi.org/10.18653/v1/N19-1004)\.In*Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\)*, pages 32–42, Minneapolis, Minnesota\. Association for Computational Linguistics\.
- Gauthier et al\. \(2020\)Jon Gauthier, Jennifer Hu, Ethan Wilcox, Peng Qian, and Roger Levy\. 2020\.[SyntaxGym: An online platform for targeted evaluation of language models](https://doi.org/10.18653/v1/2020.acl-demos.10)\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations*, pages 70–76, Online\. Association for Computational Linguistics\.
- Grattafiori et al\. \(2024\)Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others\. 2024\.[The Llama 3 herd of models](https://arxiv.org/abs/2407.21783)\.*Preprint*, arXiv:2407\.21783\.
- Hanna and Mueller \(2025\)Michael Hanna and Aaron Mueller\. 2025\.[Incremental sentence processing mechanisms in autoregressive Transformer language models](https://doi.org/10.18653/v1/2025.naacl-long.164)\.In*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pages 3181–3203, Albuquerque, New Mexico\. Association for Computational Linguistics\.
- Harrington Stack et al\. \(2018\)Caoimhe M\. Harrington Stack, Ariel N\. James, and Duane G\. Watson\. 2018\.[A failure to replicate rapid syntactic adaptation in comprehension](https://doi.org/10.3758/s13421-018-0808-6)\.*Memory & Cognition*, 46\(6\):864–877\.
- Hartsuiker et al\. \(2008\)Robert J\. Hartsuiker, Sarah Bernolet, Sofie Schoonbaert, Sara Speybroeck, and Dieter Vanderelst\. 2008\.[Syntactic priming persists while the lexical boost decays: Evidence from written and spoken dialogue](https://doi.org/10.1016/j.jml.2007.07.003)\.*Journal of Memory and Language*, 58\(2\):214–238\.
- Huang et al\. \(2024\)Kuan\-Jung Huang, Suhas Arehalli, Mari Kugemoto, Christian Muxica, Grusha Prasad, Brian Dillon, and Tal Linzen\. 2024\.[Large\-scale benchmark yields no evidence that language model surprisal explains syntactic disambiguation difficulty](https://doi.org/10.1016/j.jml.2024.104510)\.*Journal of Memory and Language*, 137:104510\.
- Jaeger and Snider \(2013\)T\. Florian Jaeger and Neal E\. Snider\. 2013\.[Alignment as a consequence of expectation adaptation: Syntactic priming is affected by the prime’s prediction error given both prior and recent experience](https://doi.org/10.1016/j.cognition.2012.10.013)\.*Cognition*, 127\(1\):57–83\.
- Jiang et al\. \(2023\)Albert Q\. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie\-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed\. 2023\.[Mistral 7B](https://arxiv.org/abs/2310.06825)\.*Preprint*, arXiv:2310\.06825\.
- Jumelet et al\. \(2024\)Jaap Jumelet, Willem Zuidema, and Arabella Sinclair\. 2024\.[Do language models exhibit human\-like structural priming effects?](https://doi.org/10.18653/v1/2024.findings-acl.877)In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 14727–14742, Bangkok, Thailand\. Association for Computational Linguistics\.
- Kamoi et al\. \(2024\)Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang\. 2024\.[When can LLMs actually correct their own mistakes? a critical survey of self\-correction of LLMs](https://doi.org/10.1162/tacl_a_00713)\.*Transactions of the Association for Computational Linguistics*, 12:1417–1440\.
- Kwon et al\. \(2023\)Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica\. 2023\.[Efficient memory management for large language model serving with PagedAttention](https://doi.org/10.1145/3600006.3613165)\.In*Proceedings of the 29th Symposium on Operating Systems Principles*, SOSP ’23, page 611–626\. ACM\.
- Li et al\. \(2024\)Andrew Li, Xianle Feng, Siddhant Narang, Austin Peng, Tianle Cai, Raj Sanjay Shah, and Sashank Varma\. 2024\.[Incremental comprehension of garden\-path sentences by large language models: Semantic interpretation, syntactic re\-analysis, and attention](https://arxiv.org/abs/2405.16042)\.*Preprint*, arXiv:2405\.16042\.
- Longpre et al\. \(2021\)Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh\. 2021\.[Entity\-based knowledge conflicts in question answering](https://doi.org/10.18653/v1/2021.emnlp-main.565)\.In*Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pages 7052–7063, Online and Punta Cana, Dominican Republic\. Association for Computational Linguistics\.
- Meng et al\. \(2023\)Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov\. 2023\.[Locating and editing factual associations in GPT](https://arxiv.org/abs/2202.05262)\.*Preprint*, arXiv:2202\.05262\.
- Michaelov et al\. \(2023\)James A\. Michaelov, Catherine Arnett, Tyler A\. Chang, and Benjamin K\. Bergen\. 2023\.[Structural priming demonstrates abstract grammatical representations in multilingual language models](https://doi.org/10.18653/v1/2023.emnlp-main.227)\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 3703–3720, Singapore\. Association for Computational Linguistics\.
- Pickering and Branigan \(1998\)Martin J\. Pickering and Holly P\. Branigan\. 1998\.[The representation of verbs: Evidence from syntactic priming in language production](https://doi.org/10.1006/jmla.1998.2592)\.*Journal of Memory and Language*, 39\(4\):633–651\.
- Prakash et al\. \(2026\)Nikhil Prakash, Natalie Shapira, Arnab Sen Sharma, Christoph Riedl, Yonatan Belinkov, Tamar Rott Shaham, David Bau, and Atticus Geiger\. 2026\.[Language models use lookbacks to track beliefs](https://arxiv.org/abs/2505.14685)\.*Preprint*, arXiv:2505\.14685\.
- Prasad and Linzen \(2021\)Grusha Prasad and Tal Linzen\. 2021\.[Rapid syntactic adaptation in self\-paced reading: Detectable, but only with many participants\.](https://doi.org/10.1037/xlm0001046)*Journal of Experimental Psychology: Learning, Memory, and Cognition*, 47\(7\):1156–1172\.
- Prasad et al\. \(2019\)Grusha Prasad, Marten van Schijndel, and Tal Linzen\. 2019\.[Using priming to uncover the organization of syntactic representations in neural language models](https://doi.org/10.18653/v1/K19-1007)\.In*Proceedings of the 23rd Conference on Computational Natural Language Learning \(CoNLL\)*, pages 66–76, Hong Kong, China\. Association for Computational Linguistics\.
- Pucci et al\. \(2026\)Giulia Pucci, Ruizhe Li, and Arabella Sinclair\. 2026\.[Patterns of priming in production: Lexical, semantic and structural alignment in language model generation](https://arxiv.org/abs/2609.04484)\.*Preprint*, arXiv:2609\.04484\.
- Qwen et al\. \(2025\)Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others\. 2025\.[Qwen2\.5 technical report](https://arxiv.org/abs/2412.15115)\.*Preprint*, arXiv:2412\.15115\.
- Radford et al\. \(2019\)Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever\. 2019\.Language models are unsupervised multitask learners\.*OpenAI blog*\.
- Reitter et al\. \(2011\)David Reitter, Frank Keller, and Johanna D\. Moore\. 2011\.[A computational cognitive model of syntactic priming](https://doi.org/10.1111/j.1551-6709.2010.01165.x)\.*Cognitive Science*, 35\(4\):587–637\.
- Schuster and Linzen \(2022\)Sebastian Schuster and Tal Linzen\. 2022\.[When a sentence does not introduce a discourse entity, transformer\-based models still sometimes refer to it](https://doi.org/10.18653/v1/2022.naacl-main.71)\.In*Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 969–982, Seattle, United States\. Association for Computational Linguistics\.
- Sinclair et al\. \(2022\)Arabella Sinclair, Jaap Jumelet, Willem Zuidema, and Raquel Fernández\. 2022\.[Structural persistence in language models: Priming as a window into abstract language representations](https://doi.org/10.1162/tacl_a_00504)\.*Transactions of the Association for Computational Linguistics*, 10:1031–1050\.
- Slattery et al\. \(2013\)Timothy J\. Slattery, Patrick Sturt, Kiel Christianson, Masaya Yoshida, and Fernanda Ferreira\. 2013\.[Lingering misinterpretations of garden path sentences arise from competing syntactic representations](https://doi.org/10.1016/j.jml.2013.04.001)\.*Journal of Memory and Language*, 69\(2\):104–120\.
- Sturt and Crocker \(1996\)Patrick Sturt and Matthew W\. Crocker\. 1996\.[Monotonic syntactic processing: A cross\-linguistic study of attachment and reanalysis](https://doi.org/10.1080/016909696387123)\.*Language and Cognitive Processes*, 11\(5\):449–494\.
- Tabor et al\. \(2004\)Whitney Tabor, Bruno Galantucci, and Daniel Richardson\. 2004\.[Effects of merely local syntactic coherence on sentence processing](https://doi.org/10.1016/j.jml.2004.01.001)\.*Journal of Memory and Language*, 50\(4\):355–370\.
- Team et al\. \(2025\)Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, and 197 others\. 2025\.[Gemma 3 technical report](https://arxiv.org/abs/2503.19786)\.*Preprint*, arXiv:2503\.19786\.
- Team et al\. \(2024\)Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179 others\. 2024\.[Gemma 2: Improving open language models at a practical size](https://arxiv.org/abs/2408.00118)\.*Preprint*, arXiv:2408\.00118\.
- Timkey et al\. \(2026\)William Timkey, Brian Dillon, and Tal Linzen\. 2026\.[Why are language models less surprised than humans? testing the parse multiplicity mismatch hypothesis](https://arxiv.org/abs/2605.15440)\.*Preprint*, arXiv:2605\.15440\.
- Touvron et al\. \(2023\)Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others\. 2023\.[Llama 2: Open foundation and fine\-tuned chat models](https://arxiv.org/abs/2307.09288)\.*Preprint*, arXiv:2307\.09288\.
- van Gompel et al\. \(2006\)Roger P\.G\. van Gompel, Martin J\. Pickering, Jamie Pearson, and Gunnar Jacob\. 2006\.[The activation of inappropriate analyses in garden\-path sentences: Evidence from structural priming](https://doi.org/10.1016/j.jml.2006.06.004)\.*Journal of Memory and Language*, 55\(3\):335–362\.
- van Schijndel and Linzen \(2018\)Marten van Schijndel and Tal Linzen\. 2018\.[A neural model of adaptation in reading](https://doi.org/10.18653/v1/D18-1499)\.In*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*, pages 4704–4710, Brussels, Belgium\. Association for Computational Linguistics\.
- van Schijndel and Linzen \(2021\)Marten van Schijndel and Tal Linzen\. 2021\.[Single\-stage prediction models do not explain the magnitude of syntactic disambiguation difficulty](https://doi.org/10.1111/cogs.12988)\.*Cognitive Science*, 45\(6\)\.
- Vig et al\. \(2020\)Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sakenis, Jason Huang, Yaron Singer, and Stuart Shieber\. 2020\.[Causal mediation analysis for interpreting neural NLP: The case of gender bias](https://arxiv.org/abs/2004.12265)\.*Preprint*, arXiv:2004\.12265\.
- Xie et al\. \(2024\)Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su\. 2024\.[Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts](https://arxiv.org/abs/2305.13300)\.*Preprint*, arXiv:2305\.13300\.
- Yang et al\. \(2025\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others\. 2025\.[Qwen3 technical report](https://arxiv.org/abs/2505.09388)\.*Preprint*, arXiv:2505\.09388\.
- Yang and Jia \(2026\)Yuqing Yang and Robin Jia\. 2026\.[When do LLMs admit their mistakes? understanding the role of model belief in retraction](https://arxiv.org/abs/2505.16170)\.*Preprint*, arXiv:2505\.16170\.
- Yoshida et al\. \(2026\)Ryo Yoshida, Shinnosuke Isono, Taiga Someya, Yohei Oseki, and Tatsuki Kuribayashi\. 2026\.[An existence proof for neural language models that can explain garden\-path effects via surprisal](https://doi.org/10.18653/v1/2026.acl-long.1694)\.In*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 36556–36569, San Diego, California, United States\. Association for Computational Linguistics\.
- Zhang and Nanda \(2024\)Fred Zhang and Neel Nanda\. 2024\.[Towards best practices of activation patching in language models: Metrics and methods](https://arxiv.org/abs/2309.16042)\.*Preprint*, arXiv:2309\.16042\.
- Zhou et al\. \(2025\)Zhenghao Zhou, Robert Frank, and R\. Thomas McCoy\. 2025\.[Is in\-context learning a type of error\-driven learning? evidence from the inverse frequency effect in structural priming](https://doi.org/10.18653/v1/2025.naacl-long.586)\.In*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pages 11712–11725, Albuquerque, New Mexico\. Association for Computational Linguistics\.

## Appendix AModels

Table[2](https://arxiv.org/html/2610.36797#A1.T2)lists the 24 models\. All weights are public Hugging Face checkpoints: GPT\-2\([Radford et al\., 2019](https://arxiv.org/html/2610.36797#bib.bib36)\), Pythia\([Biderman et al\., 2023](https://arxiv.org/html/2610.36797#bib.bib4)\), Llama 2 and Llama 3\([Touvron et al\., 2023](https://arxiv.org/html/2610.36797#bib.bib46);[Grattafiori et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib16)\), Mistral\([Jiang et al\., 2023](https://arxiv.org/html/2610.36797#bib.bib22)\), Gemma 2 and Gemma 3\([Team et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib44);[Team et al\., 2025](https://arxiv.org/html/2610.36797#bib.bib43)\), Qwen2\.5\([Qwen et al\., 2025](https://arxiv.org/html/2610.36797#bib.bib35)\)and Qwen3\([Yang et al\., 2025](https://arxiv.org/html/2610.36797#bib.bib52)\)\. Pythia\-410M is the seed\-1 checkpoint of the Pythia suite\. Log\-probabilities are computed on the jointly tokenized prime and target, with the prime preceded by the beginning\-of\-sequence token and separated from the target by a space\. A target token counts toward the continuation if its character span starts at or after the continuation’s first character\. Priming scores were computed in bfloat16\. Patching runs used float32 for models up to 8B parameters, where the cached and the full\-sequence computations agree to10−410^\{\-4\}nats, and bfloat16 for Qwen2\.5\-32B and Llama\-3\.1\-70B, which ran on two GPUs\. Sampled completions were generated with vLLM\([Kwon et al\., 2023](https://arxiv.org/html/2610.36797#bib.bib25)\)using temperature 1, top\-pp1, at most 24 new tokens and a stop at the first line break\.

ModelSizeTypeDisc\.Held\-outnompassChristiansonSampledPatchingContextGPT\-20\.12Bbase✓✓✓✓✓Pythia\-160M0\.16Bbase✓✓✓✓✓Pythia\-410M0\.41Bbase✓✓✓✓✓Pythia\-1\.4B1\.4Bbase✓✓✓✓✓✓✓✓Pythia\-2\.8B2\.8Bbase✓✓✓✓✓✓✓Qwen2\.5\-0\.5B0\.5Bbase✓✓✓✓✓Qwen3\-1\.7B\-Base1\.7Bbase✓✓✓✓✓Gemma\-2\-2B2\.6Bbase✓✓✓✓✓✓✓Qwen2\.5\-3B3Bbase✓✓✓✓✓Llama\-3\.2\-3B3\.2Bbase✓✓✓✓✓✓✓✓Llama\-2\-7B7Bbase✓✓✓✓✓Mistral\-7B\-v0\.37Bbase✓✓✓✓✓✓✓✓Qwen2\.5\-7B7\.6Bbase✓✓✓✓✓✓✓✓Qwen2\.5\-7B\-Instruct7\.6Binstruct✓✓✓✓✓✓✓Llama\-3\.1\-8B\-Instruct8Binstruct✓✓✓✓✓✓✓✓Qwen3\-8B\-Base8\.2Bbase✓✓✓✓✓✓✓✓Qwen3\-8B8\.2Bpost\-trained✓Gemma\-2\-9B9\.2Bbase✓✓✓✓✓✓✓Gemma\-3\-12B\-it12Binstruct✓✓✓✓✓Qwen2\.5\-14B\-Instruct14\.7Binstruct✓Qwen3\-30B\-A3B30B \(3B active\)post\-trained MoE✓✓✓✓✓Qwen2\.5\-32B32Bbase✓✓✓✓✓✓✓Qwen3\-32B32Bpost\-trained✓✓✓✓✓✓Llama\-3\.1\-70B70Bbase✓✓✓✓✓✓✓✓

Table 2:The 24 models and the experiments they enter\. Disc\.: discovery set \(§[4](https://arxiv.org/html/2610.36797#S4), §[5](https://arxiv.org/html/2610.36797#S5)\)\. Held\-out: 48 held\-out families\.nom: case disambiguation\.pass: passive primes\. Christianson: items of[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8), used for the prospective test of §[5](https://arxiv.org/html/2610.36797#S5)and for comprehension questions \(§[8](https://arxiv.org/html/2610.36797#S8)\)\. Sampled: sampled completions\. Patching: §[6](https://arxiv.org/html/2610.36797#S6)\. Context: Appendix[F](https://arxiv.org/html/2610.36797#A6)\.
## Appendix BMaterials

The discovery families come from the released items of[Amouyal et al\. \(2025\)](https://arxiv.org/html/2610.36797#bib.bib1)and from the NP/Z subset of the SAP benchmark\([Huang et al\., 2024](https://arxiv.org/html/2610.36797#bib.bib20)\)\. For each family we wrote the comma version \(cue\), the transitive versions with a resumptive pronoun \(tb,tc\), and the two intransitive versions \(nb,nc\)\. The intransitive verbs \(slept, yawned, sneezed, laughed, sighed, snored, napped, shivered, dozed, frowned, giggled, rested\) cannot take an object, and each family uses one of them with the family’s own subject\. The held\-out families combine 24 items with reflexive\-absolute verbs from[Ceháková and Chromý \(2025\)](https://arxiv.org/html/2610.36797#bib.bib6)with 24 NP/Z items from SyntaxGym\([Gauthier et al\., 2020](https://arxiv.org/html/2610.36797#bib.bib15)\)\. The two sources differ in the object expectation of their verbs \(about−0\.3\-0\.3and1\.51\.5nats on average\), which is why we report between\-family slopes on the held\-out set with source fixed effects\. The items of[Christianson et al\. \(2001\)](https://arxiv.org/html/2610.36797#bib.bib8)were transcribed from the published appendix\. They comprise 42 optionally transitive and 12 reflexive\-absolute verbs, each with a short and a long ambiguous region \(the deeragainstthe deer that was brown and graceful\) and, for the optionally transitive verbs, a plausible and an implausible object\. Primes from this set are followed by a fixed set of 48 discovery families as targets, excluding any target that shares a content word with the prime\.

A target never shares a content word with its prime, which removes lexical repetition as a source of priming\. In the discovery set each prime is paired with 78 to 92 targets\. TheLLmeasure compares two strings that contain the same words in the same order and differ only in the position of the comma\. TheSSmeasure scores only the first word of the main clause, so its value does not depend on the length of the continuation\.

## Appendix CStatistics

For a model and a contrast, we first average the measure over targets for every prime family and condition, then average over prime families\. Intervals resample prime families and target families independently with replacement \(1,000 draws\) and recompute the contrast, which treats both as random samples of items\. Item\-level regressions \(Figure[5](https://arxiv.org/html/2610.36797#A3.F5)\) regress the family\-level contrast on the two standardized predictors with heteroskedasticity\-consistent standard errors\. Across models we pool per\-model estimates with the random\-effects estimator of[DerSimonian and Laird \(1986\)](https://arxiv.org/html/2610.36797#bib.bib9), taking the standard error of each model’s estimate from its bootstrap interval\. Models are not a random sample of language models, so we read the pooled value as the typical model of our panel and always report how many models show the effect\. Table[3](https://arxiv.org/html/2610.36797#A3.T3)gives the residue of every model on the three item sets, and Figure[5](https://arxiv.org/html/2610.36797#A3.F5)the item\-level regressions of §[5](https://arxiv.org/html/2610.36797#S5)\.

Figure 5:Item\-level predictors of the residue on the discovery set\. Each point is thettvalue of one predictor in a regression of the family\-level contrastAAon the model’s object expectation after the ambiguous verb \(blue\) and on the garden\-path strength of the prime, the surprisal of its disambiguating word without versus with the comma \(orange\)\. Dotted lines mark\|t\|=1\.96\|t\|=1\.96\. Object expectation raisesAAon both measures\. Garden\-path strength lowers it on the comprehension\-like measureSSin 21 of 22 models\.ModelDiscoveryHeld\-outChristiansonGPT\-20\.16 \(0\.08, 0\.24\)0\.16 \(0\.08, 0\.25\)0\.14 \(0\.05, 0\.22\)Pythia\-160M0\.01 \(−\-0\.18, 0\.20\)0\.10 \(−\-0\.13, 0\.33\)0\.27 \(0\.04, 0\.48\)Pythia\-410M0\.39 \(0\.28, 0\.52\)0\.25 \(0\.15, 0\.36\)0\.50 \(0\.38, 0\.64\)Pythia\-1\.4B0\.45 \(0\.32, 0\.59\)0\.32 \(0\.22, 0\.43\)0\.54 \(0\.41, 0\.68\)Pythia\-2\.8B0\.48 \(0\.32, 0\.63\)0\.45 \(0\.33, 0\.59\)0\.61 \(0\.47, 0\.76\)Qwen2\.5\-0\.5B0\.86 \(0\.68, 1\.04\)0\.53 \(0\.39, 0\.70\)1\.00 \(0\.81, 1\.19\)Qwen3\-1\.7B\-Base1\.16 \(0\.91, 1\.41\)0\.68 \(0\.49, 0\.88\)1\.31 \(1\.05, 1\.58\)Gemma\-2\-2B0\.65 \(0\.47, 0\.84\)0\.34 \(0\.19, 0\.50\)1\.03 \(0\.80, 1\.26\)Qwen2\.5\-3B0\.27 \(0\.15, 0\.40\)0\.27 \(0\.12, 0\.43\)0\.30 \(0\.19, 0\.42\)Llama\-3\.2\-3B0\.82 \(0\.62, 1\.03\)0\.26 \(0\.11, 0\.43\)1\.13 \(0\.87, 1\.37\)Llama\-2\-7B0\.84 \(0\.65, 1\.04\)0\.33 \(0\.21, 0\.45\)1\.07 \(0\.86, 1\.27\)Mistral\-7B0\.86 \(0\.64, 1\.10\)0\.39 \(0\.25, 0\.55\)1\.19 \(0\.96, 1\.42\)Qwen2\.5\-7B0\.90 \(0\.67, 1\.16\)0\.52 \(0\.33, 0\.73\)1\.30 \(0\.99, 1\.59\)Qwen2\.5\-7B\-Instruct1\.11 \(0\.77, 1\.44\)0\.38 \(0\.07, 0\.71\)1\.50 \(1\.08, 1\.92\)Llama\-3\.1\-8B\-Instruct1\.09 \(0\.87, 1\.36\)0\.63 \(0\.43, 0\.84\)1\.64 \(1\.32, 1\.96\)Qwen3\-8B\-Base0\.98 \(0\.70, 1\.26\)0\.28 \(−\-0\.08, 0\.61\)1\.04 \(0\.76, 1\.31\)Qwen3\-8B––1\.68 \(1\.35, 2\.01\)Gemma\-2\-9B0\.55 \(0\.37, 0\.76\)0\.18 \(0\.06, 0\.30\)0\.92 \(0\.71, 1\.11\)Gemma\-3\-12B\-it1\.08 \(0\.75, 1\.44\)0\.50 \(0\.22, 0\.82\)1\.84 \(1\.44, 2\.24\)Qwen2\.5\-14B\-Instruct––1\.29 \(0\.93, 1\.66\)Qwen3\-30B\-A3B1\.13 \(0\.84, 1\.45\)0\.49 \(0\.27, 0\.79\)1\.60 \(1\.22, 1\.96\)Qwen2\.5\-32B0\.61 \(0\.40, 0\.83\)0\.52 \(0\.31, 0\.75\)1\.03 \(0\.78, 1\.25\)Qwen3\-32B0\.78 \(0\.53, 1\.07\)0\.33 \(0\.12, 0\.58\)1\.37 \(1\.03, 1\.69\)Llama\-3\.1\-70B0\.52 \(0\.34, 0\.71\)0\.37 \(0\.23, 0\.52\)0\.92 \(0\.71, 1\.12\)

Table 3:ResidueALA\_\{L\}\(nats\) with 95% two\-way bootstrap intervals on the three item sets\. Dashes mark models not run on a set\.
## Appendix DPre\-specified Predictions and Outcomes

Each confirmatory experiment was preceded by a dated record of its design, predictions and analysis, committed to version control before any of its data were collected\. Table[4](https://arxiv.org/html/2610.36797#A4.T4)lists every prediction and its outcome\. Three experiments did not confirm what we expected\. On the held\-out families, the between\-family relation between object expectation and the residue was measured onSSand was confirmed in only 6 of 15 models, whereas the error relation was confirmed in 9 of 15\. Figure[5](https://arxiv.org/html/2610.36797#A3.F5)shows that object expectation predicts the residue on both measures within the discovery set, and the held\-out result indicates that the relation is weaker onSSwhen the families come from sources that differ in verb class\. For NP/S garden paths neither force appeared, which led to the scope statement of §[5](https://arxiv.org/html/2610.36797#S5)\. The context manipulation of Appendix[F](https://arxiv.org/html/2610.36797#A6)failed its manipulation check\.

The within\-family manipulation of the error used continuations generated to follow each fixed prefix and checked for grammaticality\. Each family had between four and eight continuations whose disambiguating words differed in surprisal\. The effect of the error remained when only human\-written continuations were used, when a naturalness score of the continuation was added as a covariate, and when continuations beginning with an auxiliary were removed \(ttbetween−2\.8\-2\.8and−6\.2\-6\.2in the three models with an error effect\)\.

ExperimentPre\-specified predictionOutcomeHeld\-out families\(i\) object expectation predicts the residue onSS\(ii\) error predicts it with the sign found in discovery \(iii\) sign of the net residue\(i\) 7/17*\(not confirmed\)*\(ii\) 11/17, same sign in 16/17 \(sign testp=\.0001p=\.0001\) \(iii\) 5/9Within\-family errormore surprising disambiguation lowers the residue in the models with a discovery error effectconfirmed in 3/3, and null in the 2 models predicted nullNP/Sthe two forces generalize to NP/S garden paths*not confirmed*\(1/5 and 0/4\)MV/RRerror and commitment relations in main\-verb/reduced\-relative garden paths\(i\) error 6/9 \(ii\) commitment 0/8Case \(nom\)\(i\) object expectation predicts the residue without punctuation \(ii\) error relation negative \(iii\) comma part of the predictor vanishes\(i\) 22/22 \(ii\) 20/22 \(iii\)*not confirmed*\(reinterpreted as part of object expectation\)Measures\(i\) residue onLLpositive \(ii\) error relation stronger onSSthan onLL\(i\) 16/16 \(ii\) 14/16Passive \(pass\)passive residue small relative togpmedian ratio 0\.12 \(confirmed\)Sampled text\(i\) residue positive \(ii\) rawgp−\-cuecontrast positive \(iii\) style alignment \(iv\) size followsALA\_\{L\}\(i\) 11/12 \(ii\)*4/12*\(iii\) 12/12 \(iv\)ρ=0\.81\\rho=0\.81Patching\(i\) NP retains≤50%\\leq 50\\%and less than MC onLL\(ii\) MC changesSSmore than NP\(i\) 7/9 \(ii\) 8/9Contextmanipulation check on object expectation at the verb*not met*\(0/11\)Table 4:Pre\-specified predictions and outcomes\. Counts are models in which the prediction held\. Outcomes that did not confirm the prediction are in italics\.
## Appendix EPatching

For a prime with noun\-phrase tokensNNand main\-clause tokensMM, we run the prime twice\. The original run keeps all states\. The counterfactual run overwrites, at the output of every layer, the residual\-stream vectors atNNwith the vectors of the same tokens from the donor prime, and recomputes all positions afterNNwith these vectors in place\. Since the models use rotary position embeddings, the residual stream carries no absolute position, and the one\-token offset introduced by the donor’s comma does not need correction\. The target is then scored against a key–value cache that takes the counterfactual run’s keys and values at the positions of the chosen span and the original run’s elsewhere\. The prediction of the target’s first token comes from the run that supplies the prime’s last token\. Table[5](https://arxiv.org/html/2610.36797#A5.T5)gives all estimates with intervals\. In the depth analysis the overwrite starts at layer⌊f​L⌋\\lfloor fL\\rfloorforf∈\{0\.25,0\.5,0\.75\}f\\in\\\{0\.25,0\.5,0\.75\\\}, so that the noun phrase’s keys and values below that layer keep the object analysis\. The retained fractions when the overwrite starts at a quarter, half and three quarters of the depth are 0\.51 \(0\.41, 0\.61\), 1\.07 \(1\.01, 1\.12\) and 1\.01 \(1\.00, 1\.01\) for Llama\-3\.1\-8B\-Instruct, 0\.56 \(0\.45, 0\.65\), 1\.06 \(1\.02, 1\.12\) and 1\.02 \(1\.01, 1\.04\) for Mistral\-7B, and 0\.22 \(0\.02, 0\.35\), 0\.44 \(0\.28, 0\.55\) and 1\.04 \(1\.01, 1\.08\) for Qwen2\.5\-7B\. Replacing both the noun phrase and the main clause retains 0\.32 to 0\.67 of the residue, more than replacing the noun phrase alone, because it also removes the adaptation\. For the role direction we take, at each layer, the difference between the mean noun\-phrase states of object primes without a comma \(tb\) and subject primes without a comma \(nb\), fit without the prime’s own family\. In the middle half of the layers the projection of the noun\-phrase states on this direction is set to the subject mean, and the same shift along three norm\-matched random directions serves as control\. The residue retained is 0\.50 to 0\.92 with the role direction and 0\.42 to 0\.83 with random directions, and the role direction retains less in Llama\-3\.2\-3B, Llama\-3\.1\-8B\-Instruct and Qwen2\.5\-32B\. A direction fit on object primes against subject primes with a comma retains 0\.31 to 0\.74, less than random directions in all seven models, but it also separates primes with and without a comma\.

ModelALA\_\{L\}ρ\\rho, NPρ\\rho, MCΔ​AS\\Delta A\_\{S\}, NPΔ​AS\\Delta A\_\{S\}, MCPythia\-1\.4B0\.46 \(0\.33, 0\.59\)0\.65 \(0\.56, 0\.73\)0\.85 \(0\.76, 0\.92\)−\-0\.26 \(−\-0\.30,−\-0\.22\)0\.22 \(0\.11, 0\.34\)Pythia\-2\.8B0\.49 \(0\.34, 0\.65\)0\.50 \(0\.41, 0\.59\)0\.74 \(0\.64, 0\.83\)−\-0\.35 \(−\-0\.42,−\-0\.29\)0\.40 \(0\.25, 0\.55\)Llama\-3\.2\-3B0\.82 \(0\.62, 1\.03\)0\.39 \(0\.30, 0\.48\)0\.83 \(0\.72, 0\.95\)−\-0\.51 \(−\-0\.60,−\-0\.42\)1\.12 \(0\.92, 1\.33\)Mistral\-7B0\.86 \(0\.64, 1\.09\)0\.40 \(0\.28, 0\.49\)0\.89 \(0\.74, 1\.10\)−\-0\.55 \(−\-0\.66,−\-0\.46\)0\.93 \(0\.74, 1\.11\)Qwen2\.5\-7B0\.90 \(0\.66, 1\.16\)0\.14 \(−\-0\.05, 0\.27\)1\.34 \(1\.12, 1\.71\)−\-0\.84 \(−\-0\.99,−\-0\.71\)1\.13 \(0\.93, 1\.33\)Llama\-3\.1\-8B\-Inst\.1\.09 \(0\.87, 1\.36\)0\.28 \(0\.17, 0\.38\)1\.03 \(0\.91, 1\.17\)−\-0\.83 \(−\-0\.96,−\-0\.71\)1\.33 \(1\.11, 1\.55\)Qwen3\-8B\-Base0\.95 \(0\.68, 1\.22\)0\.71 \(0\.55, 0\.92\)0\.96 \(0\.80, 1\.16\)−\-1\.11 \(−\-1\.32,−\-0\.95\)2\.00 \(1\.67, 2\.34\)Qwen2\.5\-32B0\.61 \(0\.40, 0\.83\)0\.01 \(−\-0\.35, 0\.21\)1\.94 \(1\.55, 2\.63\)−\-0\.72 \(−\-0\.83,−\-0\.60\)1\.55 \(1\.32, 1\.79\)Llama\-3\.1\-70B0\.52 \(0\.34, 0\.71\)0\.01 \(−\-0\.36, 0\.22\)1\.89 \(1\.47, 2\.63\)−\-0\.79 \(−\-0\.93,−\-0\.66\)1\.46 \(1\.21, 1\.69\)Table 5:Patching results with 95% two\-way bootstrap intervals\.ρ\\rho: retained fraction of the residue onLL\.Δ​AS\\Delta A\_\{S\}: change of the residue onSSin nats\. NP and MC: the target reads counterfactual states at the noun phrase or at the main clause\.
## Appendix FContext Manipulation

Each of 87 discovery families received five context sentences, written by Qwen3\-32B from a fixed instruction and checked automatically for length, the absence of commas, and the absence of the family’s verbs and main\-clause words\. Two contexts present the noun phrase as the subject’s goal \(The man had been tracking the deer since dawn\.\), two present the subject and the noun phrase as doing unrelated things \(The man studied the map while the deer drifted downstream\.\), and one only places them together\. Six families for which no context passed the checks were dropped as specified\. Each context precedes all four primes of Equation[2](https://arxiv.org/html/2610.36797#S3.E2), so the context’s direct effect on the target cancels\.

The specified manipulation check required object expectation after the ambiguous verb to be higher after goal contexts in more than 70% of families\. It rose on average in every model but in only 48% to 64% of families, so the confirmatory test is uninformative by its own criterion\. The contexts did change the garden path itself: the surprisal of the disambiguating word without versus with the comma rose by 0\.9 to 1\.7 nats \(Table[6](https://arxiv.org/html/2610.36797#A6.T6)\)\. This analysis was added after the first two models had been scored\. Under it, the residue onSSis larger after goal contexts in 10 of 11 models, which runs against an account in which the prime acts only through its disambiguation error, since that error also grows\.

ModelΔ\\Deltaexp\.fam\.Δ\\DeltaGPΔ​AL\\Delta A\_\{L\}Δ​AS\\Delta A\_\{S\}Pythia\-1\.4B0\.5164%0\.88−\-0\.21 \(0\.03\)−\-0\.03 \(0\.03\)Pythia\-2\.8B0\.3963%1\.14−\-0\.10 \(0\.02\)0\.06 \(0\.03\)Gemma\-2\-2B0\.1653%1\.29−\-0\.06 \(0\.04\)0\.06 \(0\.05\)Llama\-3\.2\-3B0\.1748%1\.06−\-0\.06 \(0\.04\)0\.12 \(0\.04\)Mistral\-7B0\.3057%1\.350\.13 \(0\.06\)0\.28 \(0\.05\)Qwen2\.5\-7B0\.2652%1\.100\.29 \(0\.07\)0\.27 \(0\.06\)Qwen2\.5\-7B\-Instruct0\.2951%1\.410\.46 \(0\.11\)0\.51 \(0\.10\)Llama\-3\.1\-8B\-Instruct0\.2859%1\.05−\-0\.02 \(0\.05\)0\.36 \(0\.06\)Qwen3\-8B\-Base0\.5859%1\.730\.13 \(0\.10\)0\.08 \(0\.08\)Gemma\-2\-9B0\.2953%1\.72−\-0\.01 \(0\.05\)0\.16 \(0\.07\)Llama\-3\.1\-70B0\.3757%1\.400\.26 \(0\.05\)0\.32 \(0\.06\)

Table 6:Context manipulation, goal minus unrelated contexts\.Δ\\Deltaexp\.: object expectation at the verb \(pre\-specified check\), and the share of families in which it rose\.Δ\\DeltaGP: garden\-path strength\.Δ​AL\\Delta A\_\{L\},Δ​AS\\Delta A\_\{S\}: change of the residue, with standard errors over families\.
## Appendix GOverridden Statements

The 200 items combine ten frames \(meeting day, location of objects, city of residence, start time, favourite colour, due month, profession, material, instrument, language\) with five entities and two pairs of values in both orders\. Every context starts with a short label \(Note:,Mistake:\), so that the overridden statement is tokenized identically in all conditions\. The measure isM=log⁡P⁡\(A\)−log⁡P⁡\(B\)M=\\log P\(A\)\-\\log P\(B\)after the probe, andR=MCORR−MCTRLR=M\_\{\\text\{CORR\}\}\-M\_\{\\text\{CTRL\}\}\. In the re\-encoding test, the residual\-stream states of the overridden statement’s tokens in CORR are replaced at every layer by those of the same tokens in PRE, whereMistake:precedes the statement, and the same replacement in CTRL is subtracted\. This leaves the pull largely intact \(Table[7](https://arxiv.org/html/2610.36797#A7.T7)\), as the pre\-specified prediction did not expect\. The knockout analysis uses full sequences with a four\-dimensional attention mask that blocks the listed query–key pairs in every layer and head\. Retained fractions of the pull, for blocking the answer’s access to the statement, the answer’s access to the value token only, and all later positions’ access to the statement, are 1\.10, 0\.94, 0\.12 \(Llama\-3\.2\-3B\); 1\.64, 1\.53, 0\.21 \(Llama\-3\.1\-8B\-Instruct\); 1\.34, 1\.10,−\-0\.06 \(Mistral\-7B\); 2\.08, 1\.70,−\-0\.03 \(Qwen3\-8B\-Base\); 1\.12, 1\.11, 0\.16 \(Gemma\-2\-9B\); 0\.16,−\-0\.04, 1\.13 \(Qwen2\.5\-7B\); 0\.48, 0\.66,−\-0\.20 \(Llama\-3\.1\-70B\); and−\-3\.02,−\-3\.39, 2\.90 \(Qwen2\.5\-32B, whose small pull of 0\.56 nats reverses\)\.

ModelRRCORR−\-PREρ\\rhoCLAIMρ\\rhoFULLLlama\-3\.2\-3B1\.65 \(1\.53, 1\.77\)0\.531\.000\.72Llama\-3\.1\-8B\-Instruct1\.79 \(1\.65, 1\.95\)2\.040\.72−\-0\.16Mistral\-7B2\.17 \(1\.97, 2\.37\)0\.140\.810\.55Qwen2\.5\-7B1\.35 \(1\.23, 1\.47\)0\.410\.980\.57Qwen2\.5\-7B\-Instruct−\-2\.23 \(−\-2\.57,−\-1\.90\)−\-1\.171\.131\.86Qwen3\-8B\-Base2\.26 \(2\.06, 2\.46\)0\.551\.001\.13Gemma\-2\-9B1\.57 \(1\.46, 1\.68\)−\-0\.010\.860\.33Qwen2\.5\-32B0\.56 \(0\.44, 0\.68\)−\-0\.421\.080\.13Llama\-3\.1\-70B1\.88 \(1\.77, 1\.99\)0\.430\.800\.49

Table 7:Overridden statements \(nats\)\.RR: pull of the overridden value beyond a matched mention\.ρ\\rho: retained fraction ofRRafter re\-encoding the overridden statement \(CLAIM\) or the whole context \(FULL\)\.
## Appendix HCoding of Sampled Completions

Sampled continuations ofWhilessvvare coded from part\-of\-speech tags, because a dependency parser is itself misled by comma\-less input of exactly this kind\. A continuation that begins with punctuation is intransitive\. One that begins with an object pronoun, or with a noun phrase that is followed by punctuation, a second noun phrase or the end of the clause, is transitive\. A noun phrase followed by a finite verb is intransitive, and if no comma precedes it, the continuation is itself a garden\-path sentence\. A second version of the rules also resolves coordination after the noun phrase\. Continuations with a relative clause or an unresolved coordination after the noun phrase are coded as ambiguous and excluded \(1\.5% to 13% per model\)\. On 1,066 sentences from our materials with known structure the rules are correct on 1,063\. On 180 generated continuations sampled across classes, one annotator judged 27 to 29 of 30 transitive codes and 29 of 30 intransitive codes correct\. The results of §[4](https://arxiv.org/html/2610.36797#S4)and §[8](https://arxiv.org/html/2610.36797#S8)use the first version, and the second gives the same conclusions with the residue positive in 12 of 12 models\.