Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis

# Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis
Source: [https://arxiv.org/html/2605.15440](https://arxiv.org/html/2605.15440)
11footnotetext:Department of Linguistics, New York University, 10 Washington Pl, New York, NY 10003, USA22footnotetext:Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA33footnotetext:Department of Linguistics, University of Massachusetts Amherst, 650 N Pleasant St, Amherst, MA 01002, USA11footnotetext:Correspondence: wpt2011@nyu\.edu\(May 2026\)

## Abstract

Surprisal theory posits that the processing difficulty of a word is determined by its predictability in context, offering a potential link between human sentence processing and next\-word predictions from language models\. While language model \(LM\) surprisals successfully predict reading times in naturalistic text, they systematically underpredict the magnitude of difficulty observed in controlled studies of syntactic ambiguity, particularly in garden path sentences\. This mismatch might arise from differences in the computational constraints between humans and LMs\. Here we test one such hypothesis, specifically, that LMs may be able to simultaneously consider a greater number of distinct sentence interpretations at once, compared to humans\. Using Recurrent Neural Network Grammars \(RNNGs\) with word\-synchronous beam search, we systematically vary the number of simultaneous parses used to compute word surprisal, and then use these surprisals to predict human reading times\. Reducing the number of simultaneous active parses indeed increases the magnitude of predicted garden path effects, but not nearly enough to capture the full magnitude of the effects in humans\. This suggests that differences in the number of simultaneous parses available to LMs and humans cannot reconcile LM\-based surprisal with human sentence processing\.

## Acknowledgments

Many thanks to Qingyang Zhu and members of the NYU Computation and Psycholinguistics lab for their insightful feedback and discussion\. This project is supported by the National Science Foundation \(NSF\) under grants BCS\-2020914, BCS\-2020945, IIS\-2504953 and IIS\-2504954\. This work was supported in part through the NYU IT High Performance Computing resources, services, and staff expertise\. B\.D\. is additionally supported by the Samuel F\. Conti Fellowship from the University of Massachusetts, Amherst\.

## 1 Introduction

Prediction is widely regarded to be a central organizing principle of human cognition in general and language in particular\. Suggestive of this possibility are the recent successes of large language models, which have demonstrated remarkable language understanding capabilities via the simple objective of predicting the next word in a sentence\. The advent of such models has led many psycholinguists to consider whether such models could serve as cognitive models of the word\-by\-word difficulty of processing a sentence in humans\(Franket al\.,[2019](https://arxiv.org/html/2605.15440#bib.bib15); Linzen and Baroni,[2021](https://arxiv.org/html/2605.15440#bib.bib412); Engelmannet al\.,[2009](https://arxiv.org/html/2605.15440#bib.bib43); Arehalli and Linzen,[2024](https://arxiv.org/html/2605.15440#bib.bib280); Schrimpfet al\.,[2021](https://arxiv.org/html/2605.15440#bib.bib67); Caucheteux and King,[2022](https://arxiv.org/html/2605.15440#bib.bib8); Goldsteinet al\.,[2022](https://arxiv.org/html/2605.15440#bib.bib7)\)\.

Evaluating this possibility requires a linking hypothesis between some measurement derived from the language model \(LM\) on the one hand and a behavioral measure of word\-by\-word processing difficulty in humans on the other hand\. One such linking hypothesis is provided by surprisal theory\(Hale,[2001](https://arxiv.org/html/2605.15440#bib.bib364); Levy,[2008](https://arxiv.org/html/2605.15440#bib.bib365)\), which argues that the difficulty of processing each word in a sentence–measured, for example, as the time a reader spends fixating on the word–is proportional to thesurprisalof that word in context \(i\.e\. its negative log probability\)\. The strong version of this hypothesis holds that all sources of processing difficulty—temporary ambiguity, syntactic complexity, word frequency—can be reduced to word surprisal\. This hypothesis can be fruitfully connected to LMs: If we can identify an LM whose surprisal estimates can consistently explain reading difficulty, this would suggest both that surprisal theory provides a good characterization of sentence comprehension at Marr’s computational level\(Marr,[1982](https://arxiv.org/html/2605.15440#bib.bib1)\), and beyond this, that the representations and constraints underlying the LM’s predictions may parallel those underlying the same process in humans\(e\.g\. Haleet al\.,[2018](https://arxiv.org/html/2605.15440#bib.bib370); Ryu and Lewis,[2025](https://arxiv.org/html/2605.15440#bib.bib40)\)\.

Empirically, LM surprisal has had mixed success predicting human reading behavior\. There is a robust correlation between LM surprisals and human reading times on naturalistic texts\(Smith and Levy,[2013](https://arxiv.org/html/2605.15440#bib.bib430); Wilcoxet al\.,[2020](https://arxiv.org/html/2605.15440#bib.bib343); Shainet al\.,[2024](https://arxiv.org/html/2605.15440#bib.bib285)\), and across languages\(Wilcoxet al\.,[2023](https://arxiv.org/html/2605.15440#bib.bib294)\)\. But correlational studies alone are insufficient to establish the validity of surprisal theory as a general theory of processing difficulty, and LM surprisal has fared less well at predicting human reading behavior in controlled experiments designed to adjudicate between sentence processing theories\(slaats2026more\)\. Misalignment between LMs and humans is particularly drastic ingarden path sentences–temporarily ambiguous sentences that initially favor one interpretation, but are ultimately disambiguated in favor of a dispreferred interpretation \(Figure[1](https://arxiv.org/html/2605.15440#S1.F1), top\)\(van Schijndel and Linzen,[2021](https://arxiv.org/html/2605.15440#bib.bib348); Wilcoxet al\.,[2021](https://arxiv.org/html/2605.15440#bib.bib492); Arehalliet al\.,[2022](https://arxiv.org/html/2605.15440#bib.bib491);kobzeva2024grammar; Huanget al\.,[2024](https://arxiv.org/html/2605.15440#bib.bib245)\)\. Garden path sentences have long served as a crucial testbed for understanding how comprehenders deal with the pervasive problem of linguistic ambiguity during online processing\. For example, consider an utterance beginning with “The girl fed the lamb…”\. This utterance is consistent with at least two interpretations: A preferred interpretation, in which a girl is feeding the lamb; and a dispreferred interpretation, in which the girl is being fed the lamb\. Readers experience significant processing difficulty in cases when the sentence is disambiguated in favor of this dispreferred interpretation \(e\.g\. if it is continued with “…was upset because she asked for beef\.”, which is an ungrammatical continuation under the initially preferred reading, but perfectly grammatical under the initially dispreferred reading\)\. The disambiguation cost can be isolated by measuring reading times starting at the disambiguating word “was” in the temporarily ambiguous sentence[\(1\)](https://arxiv.org/html/2605.15440#S1.I1.i1), and comparing this to reading times on the same word in an unambiguous control sentence[\(2\)](https://arxiv.org/html/2605.15440#S1.I1.i2)\. The difference in reading times between the two conditions is referred to as agarden path effect:

1. \(1\)The girlfed the lambwasupset because she asked for beef\. \(temporarily ambiguous\)
2. \(2\)The girlwho wasfed the lambwasupset because she asked for beef\. \(unambiguous\)

In a self\-paced reading experiment,Huanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)shows that the entire disambiguating region in this construction \(the disambiguating word and two following words—the difficulty incurred by a given word often manifests at the following words as well\) takes more than 300ms longer to read in the ambiguous condition compared to the unambiguous condition\. LMs have been shown to predict the direction of these garden path effects: For example, the GPT\-2 language model\(radford2019language\)estimates surprisal of the disambiguating region to be about 4 bits higher on average in the ambiguous condition than the unambiguous condition for this construction\(Huanget al\.,[2024](https://arxiv.org/html/2605.15440#bib.bib245)\)\. However, this surprisal difference fails to explain the fullmagnitudeof the effect\. Simplifying the modeling details,Huanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)estimate that reading times increase by roughly two milliseconds for every one bit increase in GPT\-2 surprisal\. If this relationship is assumed to also hold in garden path sentences, as argued by surprisal theory, then GPT\-2 surprisal only predicts a garden path effect of approximately 8ms \(4 bits \* 2ms/bit\), nearly 40 times smaller than the empirically observed effect\(Huanget al\.,[2024](https://arxiv.org/html/2605.15440#bib.bib245)\)\. This means garden path effects cannot be reduced to a difference in surprisal from any the language models tested\. In a separate study,Timkeyet al\.\([2025](https://arxiv.org/html/2605.15440#bib.bib73)\)show this pattern holds across a diverse range LM architectures, training objectives and training corpora\.

There are at least two logically possible reasons for these misalignments between LMs and humans\. It may be the case that surprisal theory is wrong, and the processing difficulty of a word cannot be fully reduced to its predictability\. Another possibility—the focus of the present work—is that surprisal theory is generally correct, but LMs and humans systematically diverge in their expectations about upcoming words because humans are more computationally constrained than LMs\. If this is the case, the alignment between human behavior and LM next\-word predictions could be improved by imposing more humanlike resource constraints on LMs\(Timkey and Linzen,[2023](https://arxiv.org/html/2605.15440#bib.bib341); Clarket al\.,[2025](https://arxiv.org/html/2605.15440#bib.bib66); Oh and Linzen,[2025](https://arxiv.org/html/2605.15440#bib.bib82)\)\.

In this work, we investigate whether the gap between humans and LM predictions in garden path sentences could be explained by differences in theirparse multiplicity: The number of distinctinterpretations\(for our purposes, distinct syntactic parses\) of an utterance that can be maintained simultaneously in memory\.†††We use the term “multiplicity” over another commonly used term “parallelism”, to avoid confusion with another unrelated sense of the latter, which referring to the number of parallel syntactic computations a parser can execute, rather than the number of pareses that can be simultaneously maintainedLewis \([2000](https://arxiv.org/html/2605.15440#bib.bib232)\)\.We refer to this hypothesis as the*parse multiplicity mismatch hypothesis*\. Concretely, we expect the magnitude of garden path effects to increase as fewer interpretations can be considered\. For example, if a comprehender must commit to only a single interpretation of the utterance “the girl fed the lamb”, they will likely commit to the more frequent interpretation of the girl giving food to the lamb\. Under this interpretation, the disambiguating continuation of “was” is not only unexpected, but potentially ungrammatical \(cf\. “The girl gave the lamb was upset…”\)\. By contrast, when more interpretations are considered simultaneously when generating expectations about the next word, the chances increase that at least one of these interpretations is the globally correct one\. Under the globally correct interpretation, in which the girl is being fed the lamb, “was” is a grammatical, predictable continuation \(cf\. “The girl given the lamb was upset…”\)\. This increases the overall predictability of the word “hates”, decreasing the magnitude of the garden path effect \(Figure[1](https://arxiv.org/html/2605.15440#S1.F1), bottom\)\.

The degree of parse multiplicity of both humans and LMs remains poorly understood\. In humans, this has been a major topic of contention for decades\(boland1996interaction; Gibson and Pearlmutter,[2000](https://arxiv.org/html/2605.15440#bib.bib233); Lewis,[2000](https://arxiv.org/html/2605.15440#bib.bib232);clifton2008parallelism\)\. Some incremental parsing models assume a fully serial architecture, in which only a single parse is maintained at once\(e\.g\. Frazier and Fodor,[1978](https://arxiv.org/html/2605.15440#bib.bib108);marcus1978theory; Frazier,[1987](https://arxiv.org/html/2605.15440#bib.bib6);gorrell1995syntax;pritchett1988garden; Lewis,[1998](https://arxiv.org/html/2605.15440#bib.bib191),[1999](https://arxiv.org/html/2605.15440#bib.bib3);van2005evidence; Lewis and Vasishth,[2005](https://arxiv.org/html/2605.15440#bib.bib421)\), while others have argued that multiple distinct syntactic hypotheses can be maintained in parallel\(e\.g\. Gibson,[1991](https://arxiv.org/html/2605.15440#bib.bib359); Hale,[2001](https://arxiv.org/html/2605.15440#bib.bib364); Levy,[2008](https://arxiv.org/html/2605.15440#bib.bib365); Trueswell and Tanenhaus,[1994](https://arxiv.org/html/2605.15440#bib.bib4)\)\.Jurafsky \([1996](https://arxiv.org/html/2605.15440#bib.bib362)\)explicitly argues that garden path effects arise from a fixed limit on parse multiplicity in humans, and treats this limit as a free parameter of the parsing model to be estimated from reading data\(see also Bostonet al\.,[2011](https://arxiv.org/html/2605.15440#bib.bib409)\)\. By contrast, evidence suggests that LMs, which are typically not designed with explicit syntactic representations, can nevertheless implicitly represent and maintain multiple syntactic parses of a sentence in parallel\(Aina and Linzen,[2021](https://arxiv.org/html/2605.15440#bib.bib28); Eisapeet al\.,[2022](https://arxiv.org/html/2605.15440#bib.bib353); Hanna and Mueller,[2024](https://arxiv.org/html/2605.15440#bib.bib186)\)\. Mathematically, an LM’s ability to accurately approximate the probability distribution over next word of a sentence should strictly improve with greater parse multiplicity: The most accurate estimate of next\-word probability results from marginalizing, implicitly or explicitly, over all possible parses of the sentence—that is, taking a weighted average of the next word’s probability under each possible parse, where the weights are determined by the probability of the parse\.If LMs do in fact have greater parse multiplicity than humans, then reducing their parse multiplicity should lead to more human\-like predictions in garden path sentences\.

One challenge in evaluating the parse multiplicity mismatch hypothesis is that parse multiplicity cannot be directly manipulated in the most commonly used LMs such as GPT\-2, as they are trained to predict the next word conditional only on the previous words\. Any syntactic representations they have are emergent and implicit, making it difficult both to understand which interpretations an LM is considering, and to intervene on these implicit representations\(though see Hanna and Mueller,[2024](https://arxiv.org/html/2605.15440#bib.bib186)\)\. Here, we address this issue by using Recurrent Neural Network Grammars \(RNNGs;Dyeret al\.[2016](https://arxiv.org/html/2605.15440#bib.bib354)\), models trained to predict both the words and structure of a sentence\. These models condition their next word predictions on an explicit syntactic parse of the context\. From these models, we can infer thekkmost likely parses of the previous words using a variant of the beam search algorithm \(to be described below\), and then approximate next\-word surprisals by marginalizing over thesekkparses \(Figure[2](https://arxiv.org/html/2605.15440#S1.F2)\)\. In this setup, we can systematically vary the parse multiplicity parameterkk\(commonly referred as the beam width\), which determines the number of distinct parses maintained in parallel\. A fully serial parser corresponds tok=1k=1, and a fully parallel parser is approximated askkapproaches infinity\.

We test whether reducing parse multiplicity in RNNG\-based LMs leads to larger predicted garden path effects, and whether such increases are sufficient to explain the full magnitude of garden path effects in humans\. This hypothesis is only partially supported: We find that models with less parallelism do predict larger garden path effects, but even the largest predicted effects are still orders of magnitude smaller than those observed in humans\. This pattern of results holds even when any parses consistent with the globally correct interpretation are manually excluded from the parser’s set of hypotheses \(in other words, when the models are forced “down the garden path”\)\. Our results suggest that human syntactic disambiguation difficulty cannot be reduced to word surprisal from an RNNG\-based LM, regardless of the number of interpretations that can be entertained at once\. The finding that surprisals derived from a serial parser forced down the garden path cannot capture the magnitude of syntactic diambiguation in reading suggests that explaining the full magnitude of garden path effects will require positing additional mechanisms, such as syntactic reanalysis\.‡‡‡Data and analysis code are provided at[https://osf\.io/7mprn/](https://osf.io/7mprn/)

![Refer to caption](https://arxiv.org/html/2605.15440v1/x1.png)Figure 1:\(Top\) An example of a garden path sentence with its initially preferred and globally correct interpretations\. Humans experience processing difficulty when the ambiguity is resolved\. \(Bottom\) The hypothesized relationship between parse multiplicity and the surprisal of the disambiguating word\. \(Illustrations courtesy of[Irasutoya](https://www.irasutoya.com/)\)![Refer to caption](https://arxiv.org/html/2605.15440v1/x2.png)Figure 2:Comparing the computation of next word probabilities in standard LMs, and in RNNGs\. This example demonstrates the top\-down parsing strategy\.
## 2 Materials

Materials for all experiments come from the garden path subset of the SAP Benchmark\(Huanget al\.,[2024](https://arxiv.org/html/2605.15440#bib.bib245)\)\. This subset of the dataset contains self\-paced reading data from 2000 participants on three different types of garden path constructions \(in the self\-paced reading paradigm, participants are presented a sentence one word at a time, pressing the space bar to reveal each new word, after which the previous words are replaced with dashes; The time between keypresses is recorded as the word’s reading time\)\. There are two versions of each sentence, one with a temporary ambiguity, and one where extra material \(here indicated in parentheses\) has been added to eliminate this temporary ambiguity\.

MV/RR: The girl \(who was\) fed the lambremainedcalm despite having asked for beef\. NP/S: The girl found \(that\) the lambremainedcalm despite the absence of its mother\. NP/Z: When the girl attacked\(,\) the lambremainedcalm despite the sudden assault\.

In the MV/RR construction, the verb ‘fed’ is initially ambiguous between being the main verb \(MV; initially preferred\), or an embedded verb in a reduced relative clause modifying ‘the girl’ \(RR; globally correct\)\. The actual main verb, ‘remained’, disambiguates the sentence toward the RR interpretation\. When the words ‘who was’ are added, ‘fed’ is unambiguously part of a relative clause\.

The NP/S ambiguity is caused by the fact that the verb ‘found’ can either take a noun phrase \(NP; initially preferred\), or a sentential \(S; globally correct\) complement\. The relativized verb ‘remained’ disambiguates the sentence in favor of the sentential complement interpretation\. With the addition of the complementizer ‘that’, the following noun phrase, ‘the lamb’, is unambiguously the subject of a sentential complement, rather than the direct object of ‘found’\.

In the NP/Z ambiguity, the verb ‘attacked’ can optionally take a noun phrase as complement \(NP; initially preferred\), or no complement \(Z; globally correct\)\. The relativized verb ‘remained’ disambiguates the sentence in favor of the zero complement \(Z\) interpretation\. When a comma is added after ‘attacked’, this signifies the end of the clause, meaning the verb unambiguously takes zero complement, and that ‘the lamb’ is part of a new clause\.

Huanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)compare reading times between the two versions of each sentence in the disambiguating word, “remained”, and the two words that immediately follow, “calm” and “despite”, which are referred to as the first spillover and second spillover words respectively\. Here, we focus on the first spillover word, which displays the largest empirical effectsHuanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)\. Results from the other two words, which did not differ qualitatively from the results at the first spillover word, are reported in Appendix[D](https://arxiv.org/html/2605.15440#A4)\.

## 3 Methods

### 3\.1 Recurrent Neural Network Grammars

Standard language models are trained to predict the next word in a sequence given the previous words as input,P​\(wi\+1\|w1:i\)P\(w\_\{i\+1\}\|w\_\{1:i\}\)\. By contrast, RNNGs jointly predict the next word and next parsing step given a particular syntactic parse of the previous words\. The probability of a parse is estimated through the sequence of actions a transition\-based constituency parser would take to produce the parse\. Simplifying somewhat, these actions can be either structural, where words are composed into phrases \(i\.e\.NT\(X\), which predicts a nonterminal, andREDUCE\(X\)\); or lexical, where the next word is predicted and then focus shifts to that word \(theSHIFTaction\)\. Given the sequence of alliiprevious words,w1:iw\_\{1:i\}, and alljjprevious parsing actionss1:js\_\{1:j\}that form the partial derivation of the parse tree up until just beforewi\+1w\_\{i\+1\}, the RNNG jointly predicts the next parsing actionP​\(sj\+1\|s1:j,w1:i\)P\(s\_\{j\+1\}\|s\_\{1:j\},w\_\{1:i\}\), and the next word,P​\(wi\+1\|s1:j​w1:i\)P\(w\_\{i\+1\}\|s\_\{1:j\}w\_\{1:i\}\)\. The next word distribution is only defined when the next action isSHIFT, as words can only be added to the parse in tandem with aSHIFTaction\.

### 3\.2 Varying parse multiplicity in RNNGs using word\-synchronous beam search

The ultimate quantity of interest for the present study is a word’s surprisal given all previous words,P​\(wi\+1\|w1:i\)P\(w\_\{i\+1\}\|w\_\{1:i\}\)\. Unlike standard language models, which estimate this quantity directly, RNNGs condition next word predictions on a parse of the previous words,P​\(wi\+1\|s,w1:i\)P\(w\_\{i\+1\}\|s,w\_\{1:i\}\)\. To produce exact next\-word probabilities we would need to marginalize over the set of all possible parses of the previous words,SS—that is, average the parse\-conditional next\-word distributions, weighted by each parse’s probability:

P​\(wi\+1\|w1:i\)=∑s∈SP​\(wi\+1\|s,w1:i\)​P​\(s\|w1:i\)P\(w\_\{i\+1\}\|w\_\{1:i\}\)=\\sum\_\{s\\in S\}P\(w\_\{i\+1\}\|s,w\_\{1:i\}\)P\(s\|w\_\{1:i\}\)\(1\)
The marginalization required to computeP​\(wi\+1\|w1:i\)P\(w\_\{i\+1\}\|w\_\{1:i\}\)from an RNNG cannot be performed exactly in practice, asSSquickly becomes far too large: Even short sentences in simple grammars are generally compatible with many different partial parses\. But this quantity can be well approximated by first using an inference algorithm to search for the topkkmost likely parses ofw1:iw\_\{1:i\}, and then marginalizing the next word distribution over only thosekkparses \(Figure[2](https://arxiv.org/html/2605.15440#S1.F2)\)\. In this work, we perform inference on our RNNG models using a beam search algorithm\(Lowerre,[1976](https://arxiv.org/html/2605.15440#bib.bib2)\), specifically theNoji and Oseki \([2021](https://arxiv.org/html/2605.15440#bib.bib352)\)implementation ofword\-synchronous beam search\(Sternet al\.,[2017](https://arxiv.org/html/2605.15440#bib.bib358)\)\. At a high level, this procedure incrementally constructs a set of candidate parses for the input by iteratively attempting sequences of parsing actions, and retaining only a specified number of the most probable parses at each word\. The key parameter governing this process is theword beam width,kwk\_\{w\}, which specifies how many distinct parses are retained as each word is observed\. The term “beam width” comes from the metaphor of searching in the dark with a flashlight: The width of the light beam represents the small set of possibilities that are visible at a given time\. With a large beam width, the model can maintain many competing interpretations of the input, more closely approximating full marginalization over possible parses\. With a small beam width, the model’s next word predictions can only reflect a small number of analyses\. In the present work, we compute surprisals usingkw=\{1,2,3,4,5,10,25,50,100,250,500,1000\}k\_\{w\}=\\\{1,2,3,4,5,10,25,50,100,250,500,1000\\\}\.

Word\-synchronous beam search also includes a second hyperparameter, theaction beam widthkak\_\{a\}\(typically denoted simply askk\), which we fix to a large value ofkak\_\{a\}= 1000 for allkwk\_\{w\}\. We discuss the motivations for this decision in Appendix[A](https://arxiv.org/html/2605.15440#A1)\.

### 3\.3 Upper and lower bounds through counterfactual beam manipulations

The parse multiplicity mismatch hypothesis—in the terms of the previous section, the hypothesis that narrower beams will lead to larger and therefore more humanlike garden path effects—relies on two premises\. The first premise is that by decreasing the beam size, we increase the chances that parses consistent with the globally correct interpretation are pruned from the beam\. The second is that the surprisal of the disambiguating word under the initial interpretation, which remains on the beam, is at least large enough to capture the full magnitude of the human garden path effect magnitude; the logic here is that if the globally correct interpretation, which is consistent with the disambiguating word, were still on the beam, the total surprisal of the word \(marginalized over the two interpretations\) would, if anything, be even lower, which in turn might account for LMs’ inability to model the magnitude of garden path disambiguation difficulty\. This entails that the disambiguating word’s surprisal conditioned on the initial, incorrect interpretation alone serves as an upper bound on the predicted garden path effect size that we can expect to derive from RNNGs surprisal\.

This upper bound is important for evaluating the parse multiplicity hypothesies, but note that it does not necessarily correspond to the disambiguating word’s surprisal whenkw=1k\_\{w\}=1\. Multiple parses of the ambiguous region are consistent with the initially preferred interpretation\. For example, the incremental parsess1s\_\{1\}ands2s\_\{2\}in Figure[2](https://arxiv.org/html/2605.15440#S1.F2)differ only in whether they posit prepositional modifier following “the lamb”, but they do not differ with respect to the ambiguity of interest; both treat “fed” as a main verb \(MV\) instead of a reduced relative \(RR\) verb as ins3s\_\{3\}\. Thus, if we want to calculate the surprisal of the disambiguating word given a particular interpretation, we must marginalize over all parses that are consistent with that interpretation\. To this end, in addition to surprisals from the various fixedkwk\_\{w\}values, we also compute these upper\-bound surprisals in what we refer to as theForced Garden Pathcondition: We select the largest beam widthkw=1000k\_\{w\}=1000, manually sort the parses into bins according to the ambiguity of interest \(i\.e\. whether “fed” is a main verb or part of a reduced relative clause\), and then compute surprisal by marginalizing over only the parses in the bin that correspond to the main verb interpretation, instead of over all 1000 parses\. This is done only for the ambiguous sentences; for the unambiguous versions of the sentences, we simply use the unmodified surprisal estimates atkw=1000k\_\{w\}=1000\.

At the other extreme, it is still possible—and in fact quite common—that even atkw=1000k\_\{w\}=1000all parses are only consistent with the initial interpretation, and no parses are consistent with the globally correct one\. To address this issue, we introduce a lower\-bound “fully\-parallel” condition designed to more explicitly approximate a fully parallel parser\. This condition guarantees that at least one parse consistent with the globally correct interpretation is always on the beam before and after disambiguation\. Our method for manually constructing the beam and computing surprisals in this condition is described in Appendix[B](https://arxiv.org/html/2605.15440#A2)\.

### 3\.4 Top\-down and Left\-corner parsing strategies

We evaluate RNNGs trained with two different parsing strategies, top\-down and left\-corner parsing\. Since the parse multiplicity mismatch hypothesis does not favor one of these strategies over the other, we experiment with both\.

Parsing strategies differ from one another in how eager they are to make predictions about a sentence’s structure\. Bottom\-up parsing is minimally predictive: It waits until every word of a constituent is observed before positing the constituent in the parse \(we do not experiment with this strategy\)\. Top\-down parsing is maximally predictive: It eagerly posits every node dominating the next word in the tree as soon as the current word is observed\. The left\-corner strategy falls in the middle of these two strategies\. A left\-corner parser waits until a word is observed, then eagerly predicts only its parent and sister node\. For example, given the noun phrase \(NP\) “The girl”, a bottom up strategy predicts the NP node after observing “girl”, a left corner strategy does so after observing “the” and a top\-down strategy does so before observing “the”\.

Left\-corner parsers have been argued to be more psychologically plausible than top\-down and bottom\-up parsers\(Resnik,[1992](https://arxiv.org/html/2605.15440#bib.bib271);nelson2017neurophysiological\)\. Left\-corner parsers are also more memory efficient than top\-down parsers, requiring a much smaller beam size to achieve similar performance to top\-down parsers\(Kuncoroet al\.,[2018](https://arxiv.org/html/2605.15440#bib.bib243)\)\. We conduct our experiments using both top\-down and left\-corner RNNG models, but we do not intend to adjudicate between the two in the present work\. We do not test models with the minimally predictive bottom\-up strategy as their incremental parses do not distinguish between the two interpretations prior to disambiguation in both the MV/RR and NP/S constructions\.

### 3\.5 Model training

We use theNoji and Oseki \([2021](https://arxiv.org/html/2605.15440#bib.bib352)\)RNNG implementation in pytorch\. RNNGs need to be trained on syntactic trees; this implementation preprocesses each tree into a sequence of left\-corner or top\-down parsing actions\. These trees can come from a hand\-annotated treebank, such as the Penn Treebank\(Marcuset al\.,[1993](https://arxiv.org/html/2605.15440#bib.bib269)\), or a machine\-parsed corpus\. We opt for latter to avoid sparsity issues caused by the limited size of hand\-annotated treebanks\. We use two corpora for model training: A 42M word subset of the BLLIP corpus, commonly used to train RNNG models in prior works\(Huet al\.,[2020](https://arxiv.org/html/2605.15440#bib.bib106)\)and the 100M word BabyLM corpus\(Warstadtet al\.,[2023](https://arxiv.org/html/2605.15440#bib.bib293)\), a corpus of child directed speech, television subtitles, stories, and Wikipedia articles, designed to approximate the quality and quantity of linguistic input a child receives before adulthood\. Both corpora were parsed using the Berkeley parser\(Kitaev and Klein,[2018](https://arxiv.org/html/2605.15440#bib.bib70)\)\. For all four combinations of datasets and parsing strategies, we train five random seeds of the RNNG models, giving us 20 models in total\.

To ensure that our trained models provide high quality estimates of parse probabilities when combined with beam search, we evaluated their parsing performance on the WSJ subset of the Penn Treebank\(Marcuset al\.,[1993](https://arxiv.org/html/2605.15440#bib.bib269)\)\. Their performance was surprisingly competitive with state\-of\-the\-art constituency parsers \(see Appendix[E](https://arxiv.org/html/2605.15440#A5)\)\. In what follows, we focus on models trained on the BabyLM corpus, as these models consistently attained better parsing performance on the Penn Treebank than the models trained on the BLLIP corpus, but results were remarkably similar across both training datasets \(results from the BLLIP models are summarized in Appendix[D](https://arxiv.org/html/2605.15440#A4)\)\.

### 3\.6 Predicting reading times from surprisal

To predict reading times from RNNG surprisals, we follow the methodology established byvan Schijndel and Linzen \([2021](https://arxiv.org/html/2605.15440#bib.bib348)\), and used in a number of studies since then\(e\.g\. Wilcoxet al\.,[2021](https://arxiv.org/html/2605.15440#bib.bib492); Arehalliet al\.,[2022](https://arxiv.org/html/2605.15440#bib.bib491); Huanget al\.,[2024](https://arxiv.org/html/2605.15440#bib.bib245);kobzeva2024grammar\)\. In theHuanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)study, the garden path sentences read by participants were interleaved with naturalistic filler sentences\. We use reading data collected on these filler sentences to estimate the relationship between word surprisal and reading times, as follows\. We fit linear mixed\-effect regression models to predict reading times from model surprisal estimates and several nuisance variables \(word length, frequency, position\) on the filler sentences\. Then, the fitted model is used to predict reading times in the garden path stimuli\. When converting surprisals to reading times in theForced Garden PathandFull\-Parallelconditions, we use filler models fit to thekw=1000k\_\{w\}=1000surprisal estimates\. Estimating the surprisal\-to\-reading time relationship using only filler sentences tests a key prediction of surprisal theory: If surprisal can fully explain processing difficulty in complex and simple sentences alike, then the relationship between surprisal and processing difficulty in the filler sentences should generalize to all sentences, including garden paths\(van Schijndel and Linzen,[2021](https://arxiv.org/html/2605.15440#bib.bib348)\)\. We then estimate the difference in predicted reading times between the ambiguous and unambiguous conditions at the disambiguating word and the two following words \(to account for spillover effects\) using Bayesian linear mixed effects regression models\. This process is repeated for all 12 values ofkwk\_\{w\}for each of the 20 RNNG models\. Our statistical methods are described in more detail in Appendix[F](https://arxiv.org/html/2605.15440#A6)\.

![Refer to caption](https://arxiv.org/html/2605.15440v1/x3.png)Figure 3:Goodness of fit \(increase in log\-likelihood compared to baseline model; higher is better\) to filler sentences improves when surprisal is estimated using larger beam widths\. Results from the language models investigated inHuanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)—GPT\-2 small\(radford2019language\)and an LSTM language model\(Gulordavaet al\.,[2018](https://arxiv.org/html/2605.15440#bib.bib71)\)—are provided for reference\. Error bars are standard errors over each condition’s 5 model seeds\.![Refer to caption](https://arxiv.org/html/2605.15440v1/x4.png)Figure 4:Predicted garden path effects summed across the disambiguating word and two spillover words, measured in surprisal, by construction type, parsing strategy and beam width, including the two counterfactual beam conditions—theForced Garden Pathcondition, which serves as an upper bound on the effect, and theFull\-Parallellower bound—and two parsing strategies\. Error bars represent 95% credible intervals, estimated from Bayesian regression models, described in Appendix[F](https://arxiv.org/html/2605.15440#A6)\.![Refer to caption](https://arxiv.org/html/2605.15440v1/x5.png)Figure 5:Predicted garden path effects at the first spillover region, measured in milliseconds of reading time, across the three construction types, 12 beam sizes \(1\-1000\) and the two counterfactual beam conditionsForced Garden PathandFull\-Parallel, and two parsing strategies\. Empirical effect sizes are represented as horizontal bands\. Error bars represent 95% credible intervals, estimated from Bayesian regression models, described in Appendix[F](https://arxiv.org/html/2605.15440#A6)\.

## 4 Surprisal better predicts reading times in naturalistic sentences as syntactic parallelism increases

Before turning to the garden path sentences, we first examine the general relationship between parse multiplicity and the ability of surprisal to predict reading times in the naturalistic filler sentences\. We quantify gooddness of fit as the increase in log\-likelihood on the filler data of a regression model including surprisal predictors over a model with only the baseline predictors of word length, frequency and position \(see Appendix[F](https://arxiv.org/html/2605.15440#A6)for details\)\. Each regression model contains surprisal estimates at only a singlekwk\_\{w\}\.

We find that surprisal provides a better fit to filler sentence reading times at larger beam widths, as shown in Figure[3](https://arxiv.org/html/2605.15440#S3.F3)\. RNNGs with a left\-corner strategy show a consistently better fit than the top\-down models across all beam widths, but especially at smaller beam sizes, consistent with prior results indicating that left\-corner parsers require smaller beam sizes to reach comparable parsing performance to top\-down parsers\(Noji and Oseki,[2021](https://arxiv.org/html/2605.15440#bib.bib352)\)\. Improvements in goodness\-of\-fit quickly diminish in both strategies aroundkw=25k\_\{w\}=25\. These results are consistent with a range of past work predicting reading times in naturalistic sentences from surprisal across various beam widths, which have shown that larger beam sizes result in consistently better fits to human reading times\(Bostonet al\.,[2011](https://arxiv.org/html/2605.15440#bib.bib409); Haleet al\.,[2018](https://arxiv.org/html/2605.15440#bib.bib370); Noji and Oseki,[2021](https://arxiv.org/html/2605.15440#bib.bib352)\)\.

## 5 Constraining parallelism leads to larger \(but not large enough\) garden path effects

### 5\.1 Setup

In this section, we assess the relationship between parse multiplicity and the size of predicted garden path effects\. First, we report the magnitude of garden path effects in bits of surprisal across various beam sizes\. Then, we use the methods outlined in Section[3\.6](https://arxiv.org/html/2605.15440#S3.SS6)to predict garden path effect sizes in reading times using the surprisal\-to\-reading time conversion factors estimated from the filler sentences\.

### 5\.2 Limited parse multiplicity increases surprisal

Across all constructions and parsing strategies, predicted garden path effects were largest at smaller\-to\-intermediate beam widths, consistent with the parse multiplicity mismatch hypothesis \(Figure[4](https://arxiv.org/html/2605.15440#S3.F4)\)\. The effect always decreased in magnitude before reaching an asymptote at the largest beam widths\. Interestingly, garden path effects were consistently smallest atkw=1k\_\{w\}=1, which seems to contradict our hypothesis that garden path effect magnitudes will increases askwk\_\{w\}decreases\. A manual inspection of the beam revealed that parses were generally of very poor quality atkw=1k\_\{w\}=1in both the ambiguous and unambiguous conditions, making the disambiguating word an equally poor continuation with similarly high surprisals in both conditions\. There appears to be a “sweet spot” of2≤kw≤102\\leq k\_\{w\}\\leq 10where the beam width is large enough to produce parses of reasonable quality in both conditions, but small enough that the globally correct parse is pruned from the beam in the ambiguous condition before the disambiguating word is observed\. The results from theForced Garden PathandFull\-Parallelconditions are also consistent with our hypothesis: Garden path effects were consistently larger when models were explicitly forced to consider only parses consistent with the initial interpretation than when they are forced to always consider both interpretations\.

This result appears to be inconsistent with our observation that larger beam widths provided better fits to human reading times in the filler sentences, if the parse multiplicity mismatch hypothesis is expected to explain the gap between reading times predicted from LM surprisals and empirical reading times\. If the number of parses one can maintain in parallel is a fixed cognitive constraint that does not depend on the particular properties of the sentence—as assumed in prior models such asJurafsky \([1996](https://arxiv.org/html/2605.15440#bib.bib362)\)—then the beam width that provides the best fit to reading times in filler sentences should also predict the largest \(i\.e\. most humanlike\) garden path effects\.

To explicitly evaluate whether decreased beam size is associated with an increased probability of the globally correct interpretation being pruned from the beam, we manually inspected the relative probability of the initially preferred and globally correct interpretations at the three words preceding and following disambiguation\. Detailed results can be found in Appendix[G](https://arxiv.org/html/2605.15440#A7)\. We indeed find that the relative probability of the globally correct interpretation is lower at smaller beam widths than larger beam widths prior to disambiguation\.

### 5\.3 Limited parse multiplicity does not close the gap with reading times

As shown in Figure[5](https://arxiv.org/html/2605.15440#S3.F5), surprisals at all beam widths drastically underestimate the magnitude of the effects observed in humans in both parsing strategies and all three constructions\. This suggests that the misalignment of language models and human behavior in garden path sentences cannot be alleviated through limits on parse multiplicity alone\. The qualitative similarity of results in the two extreme conditionsForced Garden PathandFull\-Parallel, shows that the underprediction of garden path effects cannot be attributed to simple differences between models and humans in the parses under consideration prior to disambiguation: Effects are underpredicted by orders of magnitude both when no globally correct parse is under consideration, and when globally correct parses are guaranteed to be under consideration\.

## 6 Discussion

The present work evaluates the parse multiplicity mismatch hypothesis, according to which the misalignment between language model predictions and human reading behavior in garden path sentences is caused by a misalignment in the computational constraints of neural language models and humans, and in particular, a misalignment in the number of simultaneous syntactic analyses of an utterance that can be maintained in memory\. This hypothesis would be a way to reconcile the strong version of the surprisal hypothesis with the large empirical effects observed in human studies: If word surprisal estimates from a memory\-limited language model can predict the full magnitude of garden path effects, then this would suggest both that sentence processing difficulty can indeed be reduced to word predictability, and that the mechanisms underlying this process in humans are constrained in a similar way\(Oh and Linzen,[2025](https://arxiv.org/html/2605.15440#bib.bib82)\)\.

We have operationalized this memory limitation using Recurrent Neural Network Grammars combined with the word\-synchronous beam search inference algorithm, which together can provide word predictability estimates given any specified numberkwk\_\{w\}of parallel candidate syntactic parses\. While we found robust evidence that RNNGs predict larger garden path effects when we reduce their syntactic parallelism \(lowerkwk\_\{w\}\), the predicted effects were dwarfed by the empirical ones for all values ofkwk\_\{w\}, suggesting that this hypothesized memory constraint cannot close the gap between LM surprisals and human reading behavior\.

Our results are unlikely to depend on our choice to use the word synchronous beam search inference algorithm over alternatives such as particle filters\(Levyet al\.,[2008](https://arxiv.org/html/2605.15440#bib.bib369); Maina\-Kilaas and Levy,[2026](https://arxiv.org/html/2605.15440#bib.bib18)\), or beam search with different pruning criteria, such as the relative\-width beam criterion proposed byJurafsky \([1996](https://arxiv.org/html/2605.15440#bib.bib362)\)\. Even in theForced Garden Pathcondition, where surprisals were computed using only parses consistent with the initially preferred interpretation, predicted effects remained far too small\. This suggests that the misalignment is not driven by the subset of parses considered by the model, but by the too\-low surprisal assigned to the disambiguating wordgiven the initially preferred interpretation\(an interpretation that renders the word ungrammatical\)\. This problem would persist if we used another algorithm to perform inference over parses, as this choice can only influence which parses are under consideration, not the surprisal of the next word given a particular parse\.

### 6\.1 Reanalysis

One family of parsing models assumes that parsing proceeds in two stages\(Frazier and Fodor,[1978](https://arxiv.org/html/2605.15440#bib.bib108); Lewis,[1998](https://arxiv.org/html/2605.15440#bib.bib191)\)\. In the first stage, readers incrementally construct a limited set of possible candidate parses for the utterance\. This is followed by a second stage, an error\-driven repair process occurring when the next word of a sentence cannot be integrated with any of the candidate parses\. In the present work, we implicitly assumed a single\-stage account of syntactic disambiguation difficulty, in which there are no separate reanalysis mechanisms, and all of the costs associated with processing any word—including the disambiguating word of a garden path sentence—should be reflected in the word’s surprisal\.

Our results suggest that a single\-stage account based on language model surprisal is not sufficient, regardless of memory constraints, but it may still be possible for limited parallel language models to provide a partial account of disambiguation difficulty in a two\-stage architecture\. A two\-stage account of processing has at least three requirements: A means of computing “stage one” parsing costs, a means of detecting parsing failures in order to activate the second stage, and a means of computing “stage two” reanalysis costs\. Predictability estimates from RNNGs with beam search may be able to fulfill the first two of these three requirements\. For example, surprisal from a limited parallel model might provide a generally sufficient characterization of the costs of processing a word when there is no integration failure, and a sudden spike in a word’s surprisal \(perhaps relative to its unigram surprisal\) could be used as a signal of parsing failures\. Inviting this possibility,Timkeyet al\.\([2025](https://arxiv.org/html/2605.15440#bib.bib73)\), shows in an eyetracking\-while\-reading study that LM surprisal can account for garden path effects in trials where the disambiguating word does not cause rereading, but drastically underpredicts the rate at which the disambiguating word triggers rereading of previous words\. It is possible that difficulty metrics from a limited\-beam RNNG could be better predictors of regressive eye movements than standard LMs\. While our results suggest that LM surprisal alone cannot fully account for syntactic disambiguation difficulty, then, it remains possible that LMs can provide a full account of processing difficulty in naturalistic and garden path sentences alike, albeit with a more complex linking function that is assumed by surprisal theory\.

## 7 Conclusion

This work evaluated whether differences between humans and language models in parse multiplicity—the number of parses that can be considered in parallel—can explain the consistent failure of neural language models to capture the magnitude of garden path effects\. Using RNNGs with word synchronous beam search as a next word prediction language model, we showed that limiting the number of simultaneously maintained parses leads to garden path effects that are larger in magnitude, but these memory\-limited models still fall far short of matching the full human magnitude, even under extreme conditions that enforce a single, incorrect interpretation\. These findings suggest that the gap between model predictions and human behavior cannot be resolved by adjusting parse multiplicity alone, and instead points to a more fundamental limitation of single\-stage, surprisal\-based accounts\. Accurately modeling human sentence processing will likely require incorporating additional mechanisms that go beyond word predictability, such as explicit syntactic reanalysis\.

## References

- L\. Aina and T\. Linzen \(2021\)The Language Model Understood the Prompt was Ambiguous: Probing Syntactic Uncertainty Through Generation\.InProceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP,J\. Bastings, Y\. Belinkov, E\. Dupoux, M\. Giulianelli, D\. Hupkes, Y\. Pinter, and H\. Sajjad \(Eds\.\),Punta Cana, Dominican Republic,pp\. 42–57\.External Links:[Link](https://aclanthology.org/2021.blackboxnlp-1.4/),[Document](https://dx.doi.org/10.18653/v1/2021.blackboxnlp-1.4)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- S\. Arehalli, B\. Dillon, and T\. Linzen \(2022\)Syntactic Surprisal From Neural Models Predicts, But Underestimates, Human Processing Difficulty From Syntactic Ambiguities\.InProceedings of the 26th Conference on Computational Natural Language Learning \(CoNLL\),Abu Dhabi, United Arab Emirates \(Hybrid\),pp\. 301–313\.External Links:[Link](https://aclanthology.org/2022.conll-1.20)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p3.1),[§3\.6](https://arxiv.org/html/2605.15440#S3.SS6.p1.2)\.
- S\. Arehalli and T\. Linzen \(2024\)Neural Networks as Cognitive Models of the Processing of Syntactic Constraints\.Open Mind8,pp\. 558–614\.External Links:ISSN 2470\-2986,[Link](https://doi.org/10.1162/opmi_a_00137),[Document](https://dx.doi.org/10.1162/opmi%5Fa%5F00137)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p1.1)\.
- M\. F\. Boston, J\. T\. Hale, S\. Vasishth, and R\. Kliegl \(2011\)Parallel processing and sentence comprehension difficulty\.Language and Cognitive Processes26\(3\),pp\. 301–349\.Note:\_eprint: https://doi\.org/10\.1080/01690965\.2010\.492228External Links:ISSN 0169\-0965,[Link](https://doi.org/10.1080/01690965.2010.492228),[Document](https://dx.doi.org/10.1080/01690965.2010.492228)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1),[§4](https://arxiv.org/html/2605.15440#S4.p2.1)\.
- C\. Caucheteux and J\. King \(2022\)Brains and algorithms partially converge in natural language processing\.Communications Biology5\(1\),pp\. 134\(en\)\.External Links:ISSN 2399\-3642,[Link](https://www.nature.com/articles/s42003-022-03036-1),[Document](https://dx.doi.org/10.1038/s42003-022-03036-1)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p1.1)\.
- C\. Clark, B\. Oh, and W\. Schuler \(2025\)Linear Recency Bias During Training Improves Transformers’ Fit to Reading Times\.InProceedings of the 31st International Conference on Computational Linguistics,O\. Rambow, L\. Wanner, M\. Apidianaki, H\. Al\-Khalifa, B\. D\. Eugenio, and S\. Schockaert \(Eds\.\),Abu Dhabi, UAE,pp\. 7735–7747\.External Links:[Link](https://aclanthology.org/2025.coling-main.517/)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p6.1)\.
- C\. Dyer, A\. Kuncoro, M\. Ballesteros, and N\. A\. Smith \(2016\)Recurrent Neural Network Grammars\.InProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,San Diego, California,pp\. 199–209\.External Links:[Link](https://aclanthology.org/N16-1024),[Document](https://dx.doi.org/10.18653/v1/N16-1024)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p9.5)\.
- T\. Eisape, V\. Gangireddy, R\. Levy, and Y\. Kim \(2022\)Probing for Incremental Parse States in Autoregressive Language Models\.InFindings of the Association for Computational Linguistics: EMNLP 2022,Abu Dhabi, United Arab Emirates,pp\. 2801–2813\.External Links:[Link](https://aclanthology.org/2022.findings-emnlp.203),[Document](https://dx.doi.org/10.18653/v1/2022.findings-emnlp.203)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- F\. Engelmann, S\. Vasishth, A\. Howes, D\. Peebles, and R\. P\. Cooper \(2009\)Processing grammatical and ungrammatical center embeddings in English and German: A computational model\.In\{Proceedings of 9th International Conference on Cognitive Modeling, ICCM\},\(English\)\.External Links:[Link](https://research.manchester.ac.uk/en/publications/processing-grammatical-and-ungrammatical-center-embeddings-in-eng/)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p1.1)\.
- S\. L\. Frank, P\. Monaghan, and C\. Tsoukala \(2019\)Neural Network Models of Language Acquisition and Processing\.InHuman Language,P\. Hagoort \(Ed\.\),pp\. 277–292\(en\)\.External Links:ISBN 978\-0\-262\-35386\-1,[Link](https://direct.mit.edu/books/book/5443/chapter/3959978/Neural-Network-Models-of-Language-Acquisition-and),[Document](https://dx.doi.org/10.7551/mitpress/10841.003.0026)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p1.1)\.
- L\. Frazier and J\. D\. Fodor \(1978\)The sausage machine: A new two\-stage parsing model\.Cognition6\(4\),pp\. 291–325\.External Links:ISSN 0010\-0277,[Link](https://www.sciencedirect.com/science/article/pii/0010027778900021),[Document](https://dx.doi.org/10.1016/0010-0277%2878%2990002-1)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1),[§6\.1](https://arxiv.org/html/2605.15440#S6.SS1.p1.1)\.
- L\. Frazier \(1987\)Sentence processing: A tutorial review\.InAttention and performance 12: The psychology of reading,pp\. 559–586\.External Links:ISBN 978\-0\-86377\-083\-8 978\-0\-86377\-084\-5Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- E\. Gibson \(1991\)A computational theory of human linguistic processing: memory limitations and processing breakdown\.External Links:[Link](https://www.semanticscholar.org/paper/A-computational-theory-of-human-linguistic-memory-Gibson/e391766e52473f70e0b7b3ef76e6810eca98ce4d)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- E\. Gibson and N\. J\. Pearlmutter \(2000\)Distinguishing Serial and Parallel Parsing\.Journal of Psycholinguistic Research29\(2\),pp\. 231–240\(en\)\.External Links:ISSN 1573\-6555,[Link](https://doi.org/10.1023/A:1005153330168),[Document](https://dx.doi.org/10.1023/A%3A1005153330168)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- A\. Goldstein, Z\. Zada, E\. Buchnik, M\. Schain, A\. Price, B\. Aubrey, S\. A\. Nastase, A\. Feder, D\. Emanuel, A\. Cohen, A\. Jansen, H\. Gazula, G\. Choe, A\. Rao, C\. Kim, C\. Casto, L\. Fanda, W\. Doyle, D\. Friedman, P\. Dugan, L\. Melloni, R\. Reichart, S\. Devore, A\. Flinker, L\. Hasenfratz, O\. Levy, A\. Hassidim, M\. Brenner, Y\. Matias, K\. A\. Norman, O\. Devinsky, and U\. Hasson \(2022\)Shared computational principles for language processing in humans and deep language models\.Nature Neuroscience25\(3\),pp\. 369–380\(en\)\.External Links:ISSN 1546\-1726,[Link](https://www.nature.com/articles/s41593-022-01026-4),[Document](https://dx.doi.org/10.1038/s41593-022-01026-4)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p1.1)\.
- K\. Gulordava, P\. Bojanowski, E\. Grave, T\. Linzen, and M\. Baroni \(2018\)Colorless Green Recurrent Networks Dream Hierarchically\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long Papers\),M\. Walker, H\. Ji, and A\. Stent \(Eds\.\),New Orleans, Louisiana,pp\. 1195–1205\.External Links:[Link](https://aclanthology.org/N18-1108/),[Document](https://dx.doi.org/10.18653/v1/N18-1108)Cited by:[Figure D\.1](https://arxiv.org/html/2605.15440#A4.F1),[Figure 3](https://arxiv.org/html/2605.15440#S3.F3)\.
- J\. Hale, C\. Dyer, A\. Kuncoro, and J\. Brennan \(2018\)Finding syntax in human encephalography with beam search\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Melbourne, Australia,pp\. 2727–2736\.External Links:[Link](https://aclanthology.org/P18-1254),[Document](https://dx.doi.org/10.18653/v1/P18-1254)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p2.1),[§4](https://arxiv.org/html/2605.15440#S4.p2.1)\.
- J\. T\. Hale \(2014\)Automaton Theories of Human Sentence Comprehension\.Studies in Computational Linguistics,Center for the Study of Language and Information\(en\)\.External Links:ISBN 978\-1\-57586\-747\-2,[Link](https://press.uchicago.edu/ucp/books/book/distributed/A/bo20712531.html)Cited by:[Appendix A](https://arxiv.org/html/2605.15440#A1.p1.5)\.
- J\. Hale \(2001\)A Probabilistic Earley Parser as a Psycholinguistic Model\.InSecond Meeting of the North American Chapter of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/N01-1021)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p2.1),[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- M\. Hanna and A\. Mueller \(2024\)Incremental Sentence Processing Mechanisms in Autoregressive Transformer Language Models\.arXiv\.Note:arXiv:2412\.05353 \[cs\]External Links:[Link](http://arxiv.org/abs/2412.05353),[Document](https://dx.doi.org/10.48550/arXiv.2412.05353)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1),[§1](https://arxiv.org/html/2605.15440#S1.p9.5)\.
- J\. Hu, J\. Gauthier, P\. Qian, E\. Wilcox, and R\. Levy \(2020\)A Systematic Assessment of Syntactic Generalization in Neural Language Models\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 1725–1744\.External Links:[Link](https://aclanthology.org/2020.acl-main.158/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.158)Cited by:[§3\.5](https://arxiv.org/html/2605.15440#S3.SS5.p1.1)\.
- K\. Huang, S\. Arehalli, M\. Kugemoto, C\. Muxica, G\. Prasad, B\. Dillon, and T\. Linzen \(2024\)Large\-scale benchmark yields no evidence that language model surprisal explains syntactic disambiguation difficulty\.Journal of Memory and Language137,pp\. 104510\.External Links:ISSN 0749\-596X,[Link](https://www.sciencedirect.com/science/article/pii/S0749596X24000135),[Document](https://dx.doi.org/10.1016/j.jml.2024.104510)Cited by:[Figure D\.1](https://arxiv.org/html/2605.15440#A4.F1),[§F\.1\.1](https://arxiv.org/html/2605.15440#A6.SS1.SSS1.p1.6),[§F\.2\.1](https://arxiv.org/html/2605.15440#A6.SS2.SSS1.p3.1),[§F\.2](https://arxiv.org/html/2605.15440#A6.SS2.p1.1),[§1](https://arxiv.org/html/2605.15440#S1.p3.1),[§1](https://arxiv.org/html/2605.15440#S1.p5.1),[§2](https://arxiv.org/html/2605.15440#S2.p1.1),[§2](https://arxiv.org/html/2605.15440#S2.p5.1),[Figure 3](https://arxiv.org/html/2605.15440#S3.F3),[§3\.6](https://arxiv.org/html/2605.15440#S3.SS6.p1.2)\.
- D\. Jurafsky \(1996\)A Probabilistic Model of Lexical and Syntactic Access and Disambiguation\.Cognitive Science20\(2\),pp\. 137–194\(en\)\.Note:\_eprint: https://onlinelibrary\.wiley\.com/doi/pdf/10\.1207/s15516709cog2002\_1External Links:ISSN 1551\-6709,[Link](https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog2002_1),[Document](https://dx.doi.org/10.1207/s15516709cog2002%5F1)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1),[§5\.2](https://arxiv.org/html/2605.15440#S5.SS2.p2.1),[§6](https://arxiv.org/html/2605.15440#S6.p3.1)\.
- N\. Kitaev and D\. Klein \(2018\)Constituency Parsing with a Self\-Attentive Encoder\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),I\. Gurevych and Y\. Miyao \(Eds\.\),Melbourne, Australia,pp\. 2676–2686\.External Links:[Link](https://aclanthology.org/P18-1249/),[Document](https://dx.doi.org/10.18653/v1/P18-1249)Cited by:[Appendix E](https://arxiv.org/html/2605.15440#A5.p2.1),[§3\.5](https://arxiv.org/html/2605.15440#S3.SS5.p1.1)\.
- A\. Kuncoro, C\. Dyer, J\. Hale, D\. Yogatama, S\. Clark, and P\. Blunsom \(2018\)LSTMs Can Learn Syntax\-Sensitive Dependencies Well, But Modeling Structure Makes Them Better\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),I\. Gurevych and Y\. Miyao \(Eds\.\),Melbourne, Australia,pp\. 1426–1436\.External Links:[Link](https://aclanthology.org/P18-1132),[Document](https://dx.doi.org/10.18653/v1/P18-1132)Cited by:[§3\.4](https://arxiv.org/html/2605.15440#S3.SS4.p3.1)\.
- R\. Levy, F\. Reali, and T\. Griffiths \(2008\)Modeling the effects of memory on human online sentence processing with particle filters\.InAdvances in Neural Information Processing Systems,Vol\.21\.External Links:[Link](https://proceedings.neurips.cc/paper/2008/hash/a02ffd91ece5e7efeb46db8f10a74059-Abstract.html)Cited by:[§6](https://arxiv.org/html/2605.15440#S6.p3.1)\.
- R\. Levy \(2008\)Expectation\-based syntactic comprehension\.Cognition106\(3\),pp\. 1126–1177\(en\)\.External Links:ISSN 0010\-0277,[Link](https://www.sciencedirect.com/science/article/pii/S0010027707001436),[Document](https://dx.doi.org/10.1016/j.cognition.2007.05.006)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p2.1),[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- R\. L\. Lewis and S\. Vasishth \(2005\)An Activation\-Based Model of Sentence Processing as Skilled Memory Retrieval\.Cognitive Science29\(3\),pp\. 375–419\(en\)\.External Links:ISSN 1551\-6709,[Link](https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog0000_25),[Document](https://dx.doi.org/10.1207/s15516709cog0000%5F25)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- R\. L\. Lewis \(1998\)Reanalysis and Limited Repair Parsing: Leaping off the Garden Path\.InReanalysis in Sentence Processing,J\. D\. Fodor and F\. Ferreira \(Eds\.\),pp\. 247–285\(en\)\.External Links:ISBN 978\-94\-015\-9070\-9,[Link](https://doi.org/10.1007/978-94-015-9070-9_8),[Document](https://dx.doi.org/10.1007/978-94-015-9070-9%5F8)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1),[§6\.1](https://arxiv.org/html/2605.15440#S6.SS1.p1.1)\.
- R\. L\. Lewis \(1999\)Specifying Architectures for Language Processing: Process, Control, and Memory in Parsing and Interpretation\.InArchitectures and Mechanisms for Language Processing,C\. Clifton, M\. Pickering, and M\. W\. Crocker \(Eds\.\),pp\. 56–89\.External Links:ISBN 978\-0\-521\-02750\-2,[Link](https://www.cambridge.org/core/books/architectures-and-mechanisms-for-language-processing/specifying-architectures-for-language-processing-process-control-and-memory-in-parsing-and-interpretation/6F5C9054DCAC1C27489C3EE42233E166),[Document](https://dx.doi.org/10.1017/CBO9780511527210.004)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- R\. L\. Lewis \(2000\)Falsifying Serial and Parallel Parsing Models: Empirical Conundrums and An Overlooked Paradigm\.Journal of Psycholinguistic Research29\(2\),pp\. 241–248\(en\)\.External Links:ISSN 1573\-6555,[Link](https://doi.org/10.1023/A:1005105414238),[Document](https://dx.doi.org/10.1023/A%3A1005105414238)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1),[footnote †](https://arxiv.org/html/2605.15440#footnote2)\.
- T\. Linzen and M\. Baroni \(2021\)Syntactic Structure from Deep Learning\.Annual Review of Linguistics7\(1\),pp\. 195–212\.Note:\_eprint: https://doi\.org/10\.1146/annurev\-linguistics\-032020\-051035External Links:[Link](https://doi.org/10.1146/annurev-linguistics-032020-051035),[Document](https://dx.doi.org/10.1146/annurev-linguistics-032020-051035)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p1.1)\.
- B\. T\. Lowerre \(1976\)The Harpy speech recognition system\.Ph\.D\. thesis\.Note:ADS Bibcode: 1976PhDT……\.\.81LExternal Links:[Link](https://ui.adsabs.harvard.edu/abs/1976PhDT........81L)Cited by:[§3\.2](https://arxiv.org/html/2605.15440#S3.SS2.p2.7)\.
- A\. Maina\-Kilaas and R\. Levy \(2026\)Algorithmic Consequences of Particle Filters for Sentence Processing: Amplified Garden\-Paths and Digging\-In Effects\.arXiv\.Note:arXiv:2603\.11412 \[cs\]External Links:[Link](http://arxiv.org/abs/2603.11412),[Document](https://dx.doi.org/10.48550/arXiv.2603.11412)Cited by:[§6](https://arxiv.org/html/2605.15440#S6.p3.1)\.
- M\. P\. Marcus, B\. Santorini, and M\. A\. Marcinkiewicz \(1993\)Building a Large Annotated Corpus of English: The Penn Treebank\.Computational Linguistics19\(2\),pp\. 313–330\.Note:Place: Cambridge, MAExternal Links:[Link](https://aclanthology.org/J93-2004)Cited by:[§3\.5](https://arxiv.org/html/2605.15440#S3.SS5.p1.1),[§3\.5](https://arxiv.org/html/2605.15440#S3.SS5.p2.1)\.
- D\. Marr \(1982\)Vision: A Computational Investigation into the Human Representation and Processing of Visual Information\.Henry Holt and Co\., Inc\.,USA\.External Links:ISBN 978\-0\-7167\-1567\-2Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p2.1)\.
- H\. Noji and Y\. Oseki \(2021\)Effective Batching for Recurrent Neural Network Grammars\.InFindings of the Association for Computational Linguistics: ACL\-IJCNLP 2021,Online,pp\. 4340–4352\.External Links:[Link](https://aclanthology.org/2021.findings-acl.380),[Document](https://dx.doi.org/10.18653/v1/2021.findings-acl.380)Cited by:[§3\.2](https://arxiv.org/html/2605.15440#S3.SS2.p2.7),[§3\.5](https://arxiv.org/html/2605.15440#S3.SS5.p1.1),[§4](https://arxiv.org/html/2605.15440#S4.p2.1)\.
- B\. Oh and T\. Linzen \(2025\)To model human linguistic prediction, make LLMs less superhuman\.arXiv\.Note:arXiv:2510\.05141 \[cs\]External Links:[Link](http://arxiv.org/abs/2510.05141),[Document](https://dx.doi.org/10.48550/arXiv.2510.05141)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p6.1),[§6](https://arxiv.org/html/2605.15440#S6.p1.1)\.
- P\. Resnik \(1992\)Left\-Corner Parsing and Psychological Plausibility\.InCOLING 1992 Volume 1: The 14th International Conference on Computational Linguistics,External Links:[Link](https://aclanthology.org/C92-1032)Cited by:[§3\.4](https://arxiv.org/html/2605.15440#S3.SS4.p3.1)\.
- S\. H\. Ryu and R\. L\. Lewis \(2025\)Memory for prediction: A Transformer\-based theory of sentence processing\.Journal of Memory and Language145,pp\. 104670\.External Links:ISSN 0749\-596X,[Link](https://www.sciencedirect.com/science/article/pii/S0749596X25000634),[Document](https://dx.doi.org/10.1016/j.jml.2025.104670)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p2.1)\.
- M\. Schrimpf, I\. A\. Blank, G\. Tuckute, C\. Kauf, E\. A\. Hosseini, N\. Kanwisher, J\. B\. Tenenbaum, and E\. Fedorenko \(2021\)The neural architecture of language: Integrative modeling converges on predictive processing\.Proceedings of the National Academy of Sciences118\(45\),pp\. e2105646118\.External Links:[Link](https://www.pnas.org/doi/abs/10.1073/pnas.2105646118),[Document](https://dx.doi.org/10.1073/pnas.2105646118)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p1.1)\.
- C\. Shain, C\. Meister, T\. Pimentel, R\. Cotterell, and R\. Levy \(2024\)Large\-scale evidence for logarithmic effects of word predictability on reading time\.Proceedings of the National Academy of Sciences121\(10\),pp\. e2307876121\.External Links:[Link](https://www.pnas.org/doi/abs/10.1073/pnas.2307876121),[Document](https://dx.doi.org/10.1073/pnas.2307876121)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p3.1)\.
- N\. J\. Smith and R\. Levy \(2013\)The effect of word predictability on reading time is logarithmic\.Cognition128\(3\),pp\. 302–319\(en\)\.External Links:ISSN 0010\-0277,[Link](https://www.sciencedirect.com/science/article/pii/S0010027713000413),[Document](https://dx.doi.org/10.1016/j.cognition.2013.02.013)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p3.1)\.
- M\. Stern, D\. Fried, and D\. Klein \(2017\)Effective Inference for Generative Neural Parsing\.InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,Copenhagen, Denmark,pp\. 1695–1700\.External Links:[Link](https://aclanthology.org/D17-1178),[Document](https://dx.doi.org/10.18653/v1/D17-1178)Cited by:[§3\.2](https://arxiv.org/html/2605.15440#S3.SS2.p2.7)\.
- W\. Timkey, K\. Huang, B\. Oh, G\. Prasad, S\. Arehalli, T\. Linzen, and B\. Dillon \(2025\)Eye movements reveal a dissociation between prediction and structural processing in language comprehension\.\(en\-US\)\.External Links:[Link](https://osf.io/eq2ra_v1)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p5.1),[§6\.1](https://arxiv.org/html/2605.15440#S6.SS1.p2.1)\.
- W\. Timkey and T\. Linzen \(2023\)A Language Model with Limited Memory Capacity Captures Interference in Human Sentence Processing\.arXiv\.Note:arXiv:2310\.16142 nullExternal Links:[Link](http://arxiv.org/abs/2310.16142)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p6.1)\.
- J\. C\. Trueswell and M\. K\. Tanenhaus \(1994\)Toward a lexicalist framework of constraint\-based syntactic ambiguity resolution\.InPerspectives on sentence processing,pp\. 155–179\.External Links:ISBN 978\-0\-8058\-1581\-8 978\-0\-8058\-1582\-5Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p8.1)\.
- M\. van Schijndel and T\. Linzen \(2021\)Single\-Stage Prediction Models Do Not Explain the Magnitude of Syntactic Disambiguation Difficulty\.Cognitive Science45\(6\),pp\. e12988\(en\)\.External Links:ISSN 1551\-6709,[Link](https://onlinelibrary.wiley.com/doi/abs/10.1111/cogs.12988),[Document](https://dx.doi.org/10.1111/cogs.12988)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p3.1),[§3\.6](https://arxiv.org/html/2605.15440#S3.SS6.p1.2)\.
- A\. Warstadt, A\. Mueller, L\. Choshen, E\. Wilcox, C\. Zhuang, J\. Ciro, R\. Mosquera, B\. Paranjabe, A\. Williams, T\. Linzen, and R\. Cotterell \(2023\)Findings of the BabyLM Challenge: Sample\-Efficient Pretraining on Developmentally Plausible Corpora\.InProceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning,A\. Warstadt, A\. Mueller, L\. Choshen, E\. Wilcox, C\. Zhuang, J\. Ciro, R\. Mosquera, B\. Paranjabe, A\. Williams, T\. Linzen, and R\. Cotterell \(Eds\.\),Singapore,pp\. 1–34\.External Links:[Link](https://aclanthology.org/2023.conll-babylm.1),[Document](https://dx.doi.org/10.18653/v1/2023.conll-babylm.1)Cited by:[§3\.5](https://arxiv.org/html/2605.15440#S3.SS5.p1.1)\.
- E\. G\. Wilcox, T\. Pimentel, C\. Meister, R\. Cotterell, and R\. P\. Levy \(2023\)Testing the Predictions of Surprisal Theory in 11 Languages\.Transactions of the Association for Computational Linguistics11,pp\. 1451–1470\.External Links:ISSN 2307\-387X,[Link](https://doi.org/10.1162/tacl_a_00612),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00612)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p3.1)\.
- E\. G\. Wilcox, J\. Gauthier, J\. Hu, P\. Qian, and R\. Levy \(2020\)On the Predictive Power of Neural Language Models for Human Real\-Time Comprehension Behavior\.InProceedings of the 42th Annual Meeting of the Cognitive Science Society \- Developing a Mind: Learning in Humans, Animals, and Machines, CogSci 2020, virtual, July 29 \- August 1, 2020,pp\. 1707–1713\.Note:arXiv:2006\.01912 \[cs\]External Links:[Link](https://www.cognitivesciencesociety.org/cogsci20/papers/0375/)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p3.1)\.
- E\. Wilcox, P\. Vani, and R\. Levy \(2021\)A Targeted Assessment of Incremental Processing in Neural Language Models and Humans\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),Online,pp\. 939–952\.External Links:[Link](https://aclanthology.org/2021.acl-long.76),[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.76)Cited by:[§1](https://arxiv.org/html/2605.15440#S1.p3.1),[§3\.6](https://arxiv.org/html/2605.15440#S3.SS6.p1.2)\.

## Appendix AThe action beam width parameter,kak\_\{a\}

Word\-synchronous beam search includes a second hyperparameter, theaction beam widthkak\_\{a\}\(often denoted simply askk\), which limits how many intermediate parsing states are considered as the model builds structure between successive words\. While this parameter can have a large effect on the search procedure, our hypothesis concerns constraints on the number of competing parses that are actively maintained as each word is observed, rather than constraints on the step\-by\-step construction of structurebetweentwo words\. Indeed, it is not theoretically necessary to assume that syntactic structure is built via a sequence of discrete intermediate actions at all\. One could instead imagine that the relevant structure between two words is computed in a single step\(e\.g\.berwick1984grammatical; Hale,[2014](https://arxiv.org/html/2605.15440#bib.bib19), Ch\. 8\), in which case constraints onkak\_\{a\}would have no clear cognitive interpretation\. Accordingly, we fixkak\_\{a\}to a large value \(ka=1000k\_\{a\}=1000\) in all analyses, so as not to artificially restrict the space of intermediate derivations between two words\.

## Appendix BApproximating a fully\-parallel parser

TheFull\-Parallelcondition serves as a baseline to calculate surprisals from the RNNG using a beam that is guaranteed to contain the globally correct parse before and after disambiguation\. We first obtained the output of beam search on the original ambiguous garden path sentences atkw=1000k\_\{w\}=1000\. We refer to the set of parses obtained by beam search on the original sentences asbb\. We then constructed new versions of the garden path sentences where the first verb in the ambiguous condition is replaced with a verb that eliminates the temporary ambiguity\. The globally correct parse of the modified sentences is structurally identical to the globally correct parse of the original ambiguous garden path version:

MV/RR: The suspectsentgiventhe filedeservedfurther investigation\. NP/S: The suspectshowedsaidthe filedeservedfurther investigation\. NP/Z: Because the suspectchangeddiedthe filedeservedfurther investigation\.

We then carry out beam search withkw=1000k\_\{w\}=1000on these modified versions of the sentences, which produces the set of parsesb′b^\{\\prime\}\. This step is only done to obtain the parses themselves, but their associated probabilities are ignored\. We then manually confirmed for each item that at least one of the 1000 parses was consistent with the globally correct interpretation \(i\.e\. the RR, S, and Z interpretations of the MV/RR, NP/S and NP/Z sentences respectively\) at the end of the sentence\. For each parse inb′b^\{\\prime\}, we then replace the unambiguous verb with the ambiguous verb from the original sentence\. This gives us a set of parses,b′′b^\{\\prime\\prime\}that contain thewordsof the original ambiguous sentences, but with the set ofstructureswe would expect to see if there was never any temporary ambiguity in the first place\. The parse probabilities and next word surprisals can then be computed directly from the RNNG for allb′′b^\{\\prime\\prime\}\. Finally, we compute marginal next\-word probabilities by marginalizing over the union of the original parses,bb, and the counterfactual parses,b′′b^\{\\prime\\prime\}\(we ensure that duplicate parses frombbandb′′b^\{\\prime\\prime\}are counted only once\)\. This gives us next word surprisals that are guaranteed to include the globally correct interpretation of the sentence on the beam prior to disambiguation\. As with theForced Garden Pathcondition, we use thek=1000k=1000filler models when converting surprisals from this condition into predicted reading times\.

## Appendix CFull Results from RNNGs trained on BabyLM corpus

![Refer to caption](https://arxiv.org/html/2605.15440v1/x6.png)Figure C\.1:Predicted garden path effects from the BabyLM RNNGs at the critical word, measured in milliseconds of reading time\. Empirical effect sizes are represented as horizontal bands\. Error bars represent 95% credible intervals, with adjustments for repeated measures across RNNG model seeds![Refer to caption](https://arxiv.org/html/2605.15440v1/x7.png)Figure C\.2:Predicted garden path effects from the BabyLM RNNGs at the first spillover word, measured in milliseconds of reading time\. Empirical effect sizes are represented as horizontal bands\. Error bars represent 95% credible intervals, with adjustments for repeated measures across RNNG model seeds![Refer to caption](https://arxiv.org/html/2605.15440v1/x8.png)Figure C\.3:Predicted garden path effects from the BabyLM RNNGs at the second spillover word, measured in milliseconds of reading time\. Empirical effect sizes are represented as horizontal bands\. Error bars represent 95% credible intervals, with adjustments for repeated measures across RNNG model seedsGarden path effects measured as a difference in predicted reading times, are summarized for the disambiguating word in Figure[C\.1](https://arxiv.org/html/2605.15440#A3.F1), the first spillover word in Figure[C\.2](https://arxiv.org/html/2605.15440#A3.F2), and the second spillover word in Figure[C\.3](https://arxiv.org/html/2605.15440#A3.F3)\.

## Appendix DResults from RNNGs trained on BLLIP corpus

In the RNNGs trained on the BLLIP corpus, the goodness\-of\-fit to reading times in the filler sentences showed a remarkably similar pattern to the models trained on the BabyLM corpus\. Both provided a better fit to reading times as beam size increased, with the left\-corner models consistently outperforming the top\-down models, particularly at smaller beam sizes \(Figure[D\.1](https://arxiv.org/html/2605.15440#A4.F1)\)\.

Surprisals from the BLLIP models exhibited a very similar pattern to those from the BabyLM models\. Effects were consistently larger in theForced Garden Pathcondition than theFull\-Parallelcondition, and the beam width that predicted the largest effect magnitude was always between two and ten \(Figure[D\.2](https://arxiv.org/html/2605.15440#A4.F2)\)\. One possible exception to the general pattern, and a qualitative divergence from the BabyLM models, was the MV/RR construction in the top\-down models, where the largest predicted effect atkw=10k\_\{w\}=10was only marginally larger than the effect at the largest beam sizes\. The qualitative pattern of predicted reading times was identical to that of the raw surprisal effects, but predicted effect sizes were once again orders of magnitude smaller than the empirical effects at the disambiguating word \(Figure[D\.3](https://arxiv.org/html/2605.15440#A4.F3)\), the first spillover word \(Figure[D\.3](https://arxiv.org/html/2605.15440#A4.F3)\), and the second spillover word \(Figure[D\.3](https://arxiv.org/html/2605.15440#A4.F3)\)\.

![Refer to caption](https://arxiv.org/html/2605.15440v1/x9.png)Figure D\.1:Goodness of fit \(Delta log\-likelihood over baseline model; higher is better\) to filler sentences improves when surprisal is estimated using larger beam widths\. Results from the language models investigated inHuanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\), GPT\-2 small\(radford2019language\)and an LSTM language model\(Gulordavaet al\.,[2018](https://arxiv.org/html/2605.15440#bib.bib71)\)are provided for reference\. Error bars are standard errors over each condition’s 5 model seeds\.![Refer to caption](https://arxiv.org/html/2605.15440v1/x10.png)Figure D\.2:Predicted garden path effects from the RNNGs trained on the BLLIP corpus\. Effects are measured in surprisal, summed across the disambiguating word and two spillover words\. Error bars represent 95% credible intervals, estimated from Bayesian regression models, described in Appendix[F](https://arxiv.org/html/2605.15440#A6)\.![Refer to caption](https://arxiv.org/html/2605.15440v1/x11.png)Figure D\.3:Predicted garden path effects from the BLLIP RNNGs at the critical word, measured in milliseconds of reading time\. Empirical effect sizes are represented as horizontal bands\. Error bars represent 95% credible intervals, with adjustments for repeated measures across RNNG model seeds![Refer to caption](https://arxiv.org/html/2605.15440v1/x12.png)Figure D\.4:Predicted garden path effects from the BLLIP RNNGs at the first spillover word, measured in milliseconds of reading time\. Empirical effect sizes are represented as horizontal bands\. Error bars represent 95% credible intervals, with adjustments for repeated measures across RNNG model seeds![Refer to caption](https://arxiv.org/html/2605.15440v1/x13.png)Figure D\.5:Predicted garden path effects from the BLLIP RNNGs at the second spillover word, measured in milliseconds of reading time\. Empirical effect sizes are represented as horizontal bands\. Error bars represent 95% credible intervals, with adjustments for repeated measures across RNNG model seeds
## Appendix ERNNG Parsing performance on Penn Treebank

For each of our 20 RNNGs, we obtain the most probable parse of each sentence in the Penn Treebank test set \(partition 23\) using word synchronous beam search withka=2000k\_\{a\}=2000andkw=200k\_\{w\}=200, and compute the labeled F1 score against the gold parse, averaging this score across 5 random seeds from each combination of training corpus and parsing strategy\.

The left\-corner and top\-down models trained on the BabyLM corpus obtain average F1 scores of0\.923and0\.890respectively\. The left\-corner and top\-down and models trained on the BLLIP corpus obtain average F1 scores of0\.887and0\.850respectively\. These scores are surprisingly competitive with state\-of\-the\-art constituency parsers \(e\.g\.Kitaev and Klein \([2018](https://arxiv.org/html/2605.15440#bib.bib70)\), which we used to parse our training corpora, obtain an F1 score of 0\.95\)\. Unlike our RNNG models, these state\-of\-the\-art parsers are typically trained directly on in\-distribution, gold\-annotated parses from the training partition of the Penn Treebank\. Overall, these results give us reasonable confidence that our trained models can provide high quality estimates of parse probabilities\.

## Appendix FStatistical modeling

### F\.1 Garden path effect estimation

#### F\.1\.1 Empirical garden path effects

Empirical garden path effects are estimated from reading time data using Bayesian mixed effects linear regression from thebrmspackage in R\. We do not fit new empirical models for our analyses; we use those provided byHuanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)\. These models were fit for each region of interest: The disambiguating word, and each of the two immediately following words\. Each model includes all three constructions\. The three constructions were coded using treatment coding, where the first variableNPScontrasted the NP/S construction from the others \(MV/RR:0, NP/Z:0, NP/S:11\), and the second variableNPZcontrasted the NP/Z construction from the others \(MV/RR:0, NP/S:0, NP/Z:11\)\. The regression models included the main effects of ambiguity,NPS, andNPZ, as well as the two\-way interactionsAmbiguity:NPSandAmbiguity:NPZ\. The models also included by\-participant and by\-item random slopes for all main and interaction effects, as well as by\-participant and by\-item random intercepts:

M​e​a​s​u​r​e​∼\\displaystyle Measure\\text\{ \}\\sim\\text\{ \}a​m​b​i​g​u​i​t​y⋅\(N​P​S​\+​N​P​Z\)​\+\\displaystyle ambiguity\\boldsymbol\{\\cdot\}\(NPS\\text\{ \+ \}NPZ\)\\text\{ \+ \}\(1​\+​a​m​b​i​g​u​i​t​y⋅\(N​P​S​\+​N​P​Z\)∥s​u​b​j​e​c​t\)​\+\\displaystyle\\bigl\(1\\text\{ \+ \}ambiguity\\boldsymbol\{\\cdot\}\(NPS\\text\{ \+ \}NPZ\)\\parallel subject\\bigr\)\\text\{ \+ \}\(1​\+​a​m​b​i​g​u​i​t​y⋅\(N​P​S​\+​N​P​Z\)∥i​t​e​m\)\\displaystyle\\bigl\(1\\text\{ \+ \}ambiguity\\boldsymbol\{\\cdot\}\(NPS\\text\{ \+ \}NPZ\)\\parallel item\\bigr\)
Per\-construction ambiguity effect estimates and 95% credible intervals were estimated for each construction using the posterior samples from the trainedbrmsmodel\.

### F\.2 Filler models

Our method for fitting filler models was identical to that described inHuanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)\. All models are fit with linear mixed\-effects regression models from thelme4package in R to reading time data from the filler sentences in the SAP Benchmark self\-paced reading dataset\(Huanget al\.,[2024](https://arxiv.org/html/2605.15440#bib.bib245)\)\. We first fit a baseline filler model that predicts reading times on the filler sentences from a set of baseline factors including word position, word length, word frequency, and the interaction of frequency and length\. The model also includes random intercepts for participants and items\. All predictors are scaled to mean zero and standard deviation of one\. The baseline filler model regression formula is described below:

R​T​\(wi\)​∼\\displaystyle RT\(w\_\{i\}\)\\text\{ \}\\sim\\text\{ \}L​o​g​f​r​e​q​\(wi\)⋅L​e​n​g​t​h​\(wi\)​\+\\displaystyle Logfreq\(w\_\{i\}\)\\boldsymbol\{\\cdot\}Length\(w\_\{i\}\)\\text\{ \+ \}L​o​g​f​r​e​q​\(wi−1\)⋅L​e​n​g​t​h​\(wi−1\)​\+\\displaystyle Logfreq\(w\_\{i\-1\}\)\\boldsymbol\{\\cdot\}Length\(w\_\{i\-1\}\)\\text\{ \+ \}L​o​g​f​r​e​q​\(wi−2\)⋅L​e​n​g​t​h​\(wi−2\)​\+\\displaystyle Logfreq\(w\_\{i\-2\}\)\\boldsymbol\{\\cdot\}Length\(w\_\{i\-2\}\)\\text\{ \+ \}P​o​s​i​t​i​o​n​\(wi\)​\+\\displaystyle Position\(w\_\{i\}\)\\text\{ \+ \}\(1∣p​a​r​t​i​c​p​a​n​t\)​\+\\displaystyle\(1\\mid particpant\)\\text\{ \+ \}\(1∣i​t​e​m\)\\displaystyle\(1\\mid item\)
The full filler models additionally include surprisals from the current word \(wiw\_\{i\}\) and the previous two words, as well as random surprisal slopes for participants\. One filler model is fit for each combination of RNNG seed, parsing strategy, training corpus, and beam width:

R​T​\(wi\)​∼\\displaystyle RT\(w\_\{i\}\)\\text\{ \}\\sim\\text\{ \}S​u​r​p​\(wi\)​\+​S​u​r​p​\(wi−1\)​\+​S​u​r​p​\(wi−2\)​\+\\displaystyle Surp\(w\_\{i\}\)\\text\{ \+ \}Surp\(w\_\{i\-1\}\)\\text\{ \+ \}Surp\(w\_\{i\-2\}\)\\text\{ \+ \}L​o​g​f​r​e​q​\(wi\)⋅L​e​n​g​t​h​\(wi\)​\+\\displaystyle Logfreq\(w\_\{i\}\)\\boldsymbol\{\\cdot\}Length\(w\_\{i\}\)\\text\{ \+ \}L​o​g​f​r​e​q​\(wi−1\)⋅L​e​n​g​t​h​\(wi−1\)​\+\\displaystyle Logfreq\(w\_\{i\-1\}\)\\boldsymbol\{\\cdot\}Length\(w\_\{i\-1\}\)\\text\{ \+ \}L​o​g​f​r​e​q​\(wi−2\)⋅L​e​n​g​t​h​\(wi−2\)​\+\\displaystyle Logfreq\(w\_\{i\-2\}\)\\boldsymbol\{\\cdot\}Length\(w\_\{i\-2\}\)\\text\{ \+ \}P​o​s​i​t​i​o​n​\(wi\)​\+\\displaystyle Position\(w\_\{i\}\)\\text\{ \+ \}\(1\+S​u​r​p​\(wi\)​\+​S​u​r​p​\(wi−1\)\+S​u​r​p​\(wi\)​\+​S​u​r​p​\(wi−2\)∣p​a​r​t​i​c​p​a​n​t\)​\+\\displaystyle\(1\+Surp\(w\_\{i\}\)\\text\{ \+ \}Surp\(w\_\{i\-1\}\)\+Surp\(w\_\{i\}\)\\text\{ \+ \}Surp\(w\_\{i\-2\}\)\\mid particpant\)\\text\{ \+ \}\(1∣i​t​e​m\)\\displaystyle\(1\\mid item\)
Each full filler model includes surprisals from a single RNNG model at a single beam width\. Goodness of fit to the training data calculated by subtracting the log likelihood of the filler data given the baseline model from the log likelihood of the filler data given a full filler model\.

#### F\.2\.1 Surprisal\-predicted garden path effects

When estimating garden path effects in raw surprisal, we fit a single Bayesian regression model for each unique combination of beam width and parsing strategy\. This model predicts the summed surprisal of the disambiguating word, first spillover and second spillover words from the same ambiguity and construction predictors as in the empirical model\. The random effects structure for items is the same as in the empirical model, and we also include a random intercept for model seed:

S​u​r​p​r​i​s​a​l​∼\\displaystyle Surprisal\\text\{ \}\\sim\\text\{ \}a​m​b​i​g​u​i​t​y⋅\(N​P​S​\+​N​P​Z\)​\+\\displaystyle ambiguity\\boldsymbol\{\\cdot\}\(NPS\\text\{ \+ \}NPZ\)\\text\{ \+ \}\(1∣s​e​e​d\)​\+\\displaystyle\\bigl\(1\\mid seed\\bigr\)\\text\{ \+ \}\(1​\+​a​m​b​i​g​u​i​t​y⋅\(N​P​S​\+​N​P​Z\)∥i​t​e​m\)\\displaystyle\\bigl\(1\\text\{ \+ \}ambiguity\\boldsymbol\{\\cdot\}\(NPS\\text\{ \+ \}NPZ\)\\parallel item\\bigr\)
When estimating garden path effects as differences in predicted reading times, we followHuanget al\.\([2024](https://arxiv.org/html/2605.15440#bib.bib245)\)in first using the trained filler models to predict reading times in the garden path stimuli from surprisals\. For the unambiguous stimuli in theForced Garden PathandFull\-Parallelconditions, we simply use surprisals estimated atkw=1000k\_\{w\}=1000, rather than manually pruning the beam to have only the ultimately correct interpretation \(as in practice, nearly all of the parses on the beam in the unambiguous sentences are consistent with the globally correct interpretation before and after the disambiguating word, see Figure[G\.3](https://arxiv.org/html/2605.15440#A7.F3)of Appendix[G](https://arxiv.org/html/2605.15440#A7)\)\. We then estimate garden path effect sizes from predicted reading times using Bayesian regression models with the same formula as the empirical models, with an additional random intercept for model seeds\. We fit one model for each unique combination of training corpus, parsing strategy and beam size, with the following formula:

R​T​\(wi\)​∼\\displaystyle RT\(w\_\{i\}\)\\text\{ \}\\sim\\text\{ \}a​m​b​i​g​u​i​t​y⋅\(N​P​S​\+​N​P​Z\)​\+\\displaystyle ambiguity\\boldsymbol\{\\cdot\}\(NPS\\text\{ \+ \}NPZ\)\\text\{ \+ \}\(1​\+​a​m​b​i​g​u​i​t​y⋅\(N​P​S​\+​N​P​Z\)∥s​u​b​j​e​c​t\)​\+\\displaystyle\\bigl\(1\\text\{ \+ \}ambiguity\\boldsymbol\{\\cdot\}\(NPS\\text\{ \+ \}NPZ\)\\parallel subject\\bigr\)\\text\{ \+ \}\(1​\+​a​m​b​i​g​u​i​t​y⋅\(N​P​S​\+​N​P​Z\)∥i​t​e​m\)​\+\\displaystyle\\bigl\(1\\text\{ \+ \}ambiguity\\boldsymbol\{\\cdot\}\(NPS\\text\{ \+ \}NPZ\)\\parallel item\\bigr\)\\text\{ \+ \}\(1∣s​e​e​d\)\\displaystyle\\bigl\(1\\mid seed\\bigr\)
95% credible intervals are estimated using the same methodology as with the empirical effect estimates\.

## Appendix GInterpretation probabilities by beam width

In this analysis, we manually inspect the word\-by\-word probabilities assigned by the RNNGs to each interpretation of the garden path sentences \(the initially preferred or ultimately correct interpretation\) as a function of the word beam width parameterkwk\_\{w\}\. As each word is observed, the word synchronous beam search algorithm outputs a set of thekwk\_\{w\}highest probability parses of those words\. We manually bin the parses at each word by whether they are consistent with the initially preferred interpretation \(the NP interpretation in the NP/Z and NP/S constructions and the MV interpretation in the MV/RR sentences\), or the globally correct interpretation\. We sum the probability of all the parses in each bin, and normalize by the total probability of all parses\. These probabilities are averaged across RNNG seeds and items\. Results from ambiguous sentences are provided in Figure[G\.1](https://arxiv.org/html/2605.15440#A7.F1)for the top\-down BabyLM models, and Figure[G\.2](https://arxiv.org/html/2605.15440#A7.F2)for the left\-corner BabyLM models\. Results from unambiguous sentences are provided in Figure[G\.3](https://arxiv.org/html/2605.15440#A7.F3)for the top\-down BabyLM models, and Figure[G\.4](https://arxiv.org/html/2605.15440#A7.F4)for the left\-corner BabyLM models\.

![Refer to caption](https://arxiv.org/html/2605.15440v1/x14.png)Figure G\.1:Relative probabilities of each interpretation in the ambiguous condition from the top\-down RNNGs trained on the BabyLM dataset, split by beam width \(kwk\_\{w\}\), and construction\. Within each plot, the horizontal bar denotes the disambiguating word, and the x\-axis is labeled based on the word’s index aligned to the disambiguating word \(i\.e\. “1” corresponds to the first spillover word, and “\-1” corresponds to the word immediately preceding the disambiguating word\)\.![Refer to caption](https://arxiv.org/html/2605.15440v1/x15.png)Figure G\.2:Relative probabilities of each interpretation in the ambiguous condition from the left\-corner RNNGs trained on the BabyLM dataset, split by beam width \(kwk\_\{w\}\), and construction\. Within each plot, the horizontal bar denotes the disambiguating word, and the x\-axis is labeled based on the word’s index aligned to the disambiguating word \(i\.e\. “1” corresponds to the first spillover word, and “\-1” corresponds to the word immediately preceding the disambiguating word\)\.![Refer to caption](https://arxiv.org/html/2605.15440v1/x16.png)Figure G\.3:Relative probabilities of each interpretation in the unambiguous condition from the top\-down RNNGs trained on the BabyLM dataset, split by beam width \(kwk\_\{w\}\), and construction\. Within each plot, the horizontal bar denotes the disambiguating word, and the x\-axis is labeled based on the word’s index aligned to the disambiguating word \(i\.e\. “1” corresponds to the first spillover word, and “\-1” corresponds to the word immediately preceding the disambiguating word\)\.![Refer to caption](https://arxiv.org/html/2605.15440v1/x17.png)Figure G\.4:Relative probabilities of each interpretation in the unambiguous condition from the left\-corner RNNGs trained on the BabyLM dataset, split by beam width \(kwk\_\{w\}\), and construction\. Within each plot, the horizontal bar denotes the disambiguating word, and the x\-axis is labeled based on the word’s index aligned to the disambiguating word \(i\.e\. “1” corresponds to the first spillover word, and “\-1” corresponds to the word immediately preceding the disambiguating word\)\.