A Data-free Universal Prior over Syntactic Structures

# A Data-free Universal Prior over Syntactic Structures
Source: [https://arxiv.org/html/2609.16854](https://arxiv.org/html/2609.16854)
###### Abstract

Probability is fundamental to theories of language comprehension, production, acquisition, and evolution, as well as to large language models\.\[[27](https://arxiv.org/html/2609.16854#biba.bib1),[25](https://arxiv.org/html/2609.16854#biba.bib8),[7](https://arxiv.org/html/2609.16854#biba.bib2),[10](https://arxiv.org/html/2609.16854#biba.bib9),[16](https://arxiv.org/html/2609.16854#biba.bib3),[8](https://arxiv.org/html/2609.16854#biba.bib11),[30](https://arxiv.org/html/2609.16854#biba.bib4),[3](https://arxiv.org/html/2609.16854#biba.bib5)\]Existing theories estimate the probability of syntactic structures from language\-specific data\.\[[9](https://arxiv.org/html/2609.16854#biba.bib12),[11](https://arxiv.org/html/2609.16854#biba.bib7)\]Whether part of this probability structure can arise independently of language\-specific experience remains unknown\. Here I show that a universal prior over syntactic structures emerges from a cognitively motivated model of incremental language production,\[[21](https://arxiv.org/html/2609.16854#biba.bib6)\]in which words are progressively integrated into syntactic structure through network growth\.\[[14](https://arxiv.org/html/2609.16854#biba.bib10)\]The resulting prior assigns probabilities to syntactic structures –represented as dependency trees– without fitting parameters to linguistic data, and assigns higher probabilities to attested than to random trees in all 138 typologically diverse languages examined\. These prior probabilities correlate positively with probabilities estimated from corpora in 33 of 34 languages\. The results indicate that part of the probability structure of syntax can arise independently of language\-specific statistical learning\. Linguistic experience may therefore refine probabilities that are already structured by the process of language production, rather than create them from an initially uniform space\. This identifies a possible cognitive origin for part of the probability distribution over syntactic structures, linking language production and statistical learning while providing a data\-independent structural bias for probabilistic models of language\.

\{affiliations\}

Department of Computer Science and Technology, University of Cambridge, United Kingdom

Language is fundamentally probabilistic\. Humans continuously estimate the probabilities of forthcoming words and syntactic structures during language comprehension and production, and acquire these expectations from finite linguistic experience\.\[[27](https://arxiv.org/html/2609.16854#biba.bib1),[7](https://arxiv.org/html/2609.16854#biba.bib2),[16](https://arxiv.org/html/2609.16854#biba.bib3),[8](https://arxiv.org/html/2609.16854#biba.bib11)\]Likewise, modern large language models represent language as probability distributions learned from vast corpora, enabling them to predict and generate remarkably fluent text\.\[[30](https://arxiv.org/html/2609.16854#biba.bib4),[3](https://arxiv.org/html/2609.16854#biba.bib5)\]Understanding the origins of these probabilities is therefore central to both cognitive science and artificial intelligence\.

Probabilistic theories of syntax estimate the probabilities of syntactic structures from language\-specific observations\.\[[9](https://arxiv.org/html/2609.16854#biba.bib12),[11](https://arxiv.org/html/2609.16854#biba.bib7)\]A fundamental question is whether, across languages, there exists a universal prior over syntactic structures, or whether all structures are equally plausible before linguistic evidence is observed\. Universal hierarchical priors have proved invaluable in Bayesian nonparametrics\.\[[22](https://arxiv.org/html/2609.16854#biba.bib14),[28](https://arxiv.org/html/2609.16854#biba.bib13),[12](https://arxiv.org/html/2609.16854#biba.bib17)\]A cognitively grounded universal prior would provide a language\-independent starting point from which language\-specific probabilities could be learned, separating universal structural preferences from those acquired through experience\. It could also provide a principled inductive bias for probabilistic models of language, including large language models\.

In this paper, I show that a model of incremental human language production\[[21](https://arxiv.org/html/2609.16854#biba.bib6)\]defines a universal data\-free prior distribution over syntactic structures\. This prior predicts both which structures occur across languages and their probabilities estimated from language\-specific data\.

Figure 1:Incremental language production induces a universal prior over dependency trees\.A dependency tree can be constructed incrementally as words become available during language production\. Different orders of word availability can generate the same final dependency tree\. Summing across all possible production orders assigns a probability to each dependency\-tree shape, thereby defining a prior distribution over trees of a fixed number of nodes\. The numerical probabilities shown are just for illustration\.## A universal prior over syntactic structures

Language production is inherently incremental, with syntactic structure emerging progressively during the planning and production of an utterance rather than being assembled all at once\.\[[15](https://arxiv.org/html/2609.16854#biba.bib15),[6](https://arxiv.org/html/2609.16854#biba.bib16)\]More generally, constraints imposed by language production have been argued to shape the structure of language itself\.\[[17](https://arxiv.org/html/2609.16854#biba.bib28),[18](https://arxiv.org/html/2609.16854#biba.bib29)\]At every stage of production, only a partial dependency tree has been constructed, and each newly available word must be integrated into this evolving structure by establishing a syntactic dependency with one of the words already present\. This process can be naturally formulated as a growing network, in which words correspond to nodes, syntactic dependencies to edges, and the partial dependency tree grows through the successive addition of new nodes\.\[[21](https://arxiv.org/html/2609.16854#biba.bib6)\]Fig\.[1](https://arxiv.org/html/2609.16854#S0.F1)illustrates this generative process\.

The choice of where to attach each newly available word depends on multiple syntactic, semantic, pragmatic, and contextual factors that are difficult to model explicitly\. The current partial dependency tree nevertheless provides observable information about this latent attachment propensity: The number of direct dependents already attracted by a word provides a natural observable proxy for its latent propensity to attract further ones\.\[[21](https://arxiv.org/html/2609.16854#biba.bib6)\]This naturally gives rise to preferential attachment, in which attachment probability increases with the number of existing connections\.\[[1](https://arxiv.org/html/2609.16854#biba.bib18)\]However, language production is only approximately incremental\. Speakers often plan multiple words or constituents simultaneously, and previously activated syntactic structures facilitate the production of subsequent ones through structural priming\.\[[2](https://arxiv.org/html/2609.16854#biba.bib19),[23](https://arxiv.org/html/2609.16854#biba.bib20),[19](https://arxiv.org/html/2609.16854#biba.bib21)\]These interactions weaken the assumption that each attachment decision is made independently, attenuating the cumulative advantage characteristic of linear preferential attachment\. The attachment process is therefore expected to follow sublinear preferential attachment \(sPA\),\[[14](https://arxiv.org/html/2609.16854#biba.bib10)\]in which the probability that a newly available word attaches to nodeiiat timettis

Pt​\(i\)∝\(kt​\(i\)\+1\)α,0<α<1,P\_\{t\}\(i\)\\propto\(k\_\{t\}\(i\)\+1\)^\{\\alpha\},\\qquad 0<\\alpha<1,\(1\)wherekt​\(i\)k\_\{t\}\(i\)denotes the number of direct dependents \(i\.e\., the out\-degree\) of nodeiiin the partially constructed dependency tree\.

Different node insertion orders may lead to the same final structure\. Among all node orderings, only those in which every node appears after all of its ancestors are compatible with incremental construction of a specific observed tree\. Denoting the set of compatible orderings for a treeTTbyV⁡\(T\)V\(T\), the probability of a dependency tree is obtained by summing over all compatible orderings,

P⁡\(T\)=∑o∈V⁡\(T\)P⁡\(o\)​P​\(T∣o\)\.P\(T\)=\\sum\_\{o\\in V\(T\)\}P\(o\)\\,P\(T\\mid o\)\.\(2\)Assuming no prior knowledge beyond the sentence length, alln\!n\!node orderings should –a priori– be considered equally likely\. The tree probability therefore decomposes into

P⁡\(T\)=\|V⁡\(T\)\|n\!​⟨P⁡\(T∣o\)⟩o∈V⁡\(T\),P\(T\)=\\frac\{\|V\(T\)\|\}\{n\!\}\\left<P\(T\\mid o\)\\right\>\_\{o\\in V\(T\)\},\(3\)where\|V⁡\(T\)\|\|V\(T\)\|is the number of compatible node orderings and⟨P⁡\(T∣o\)⟩o∈V⁡\(T\)\\left<P\(T\\mid o\)\\right\>\_\{o\\in V\(T\)\}is the average probability with which the tree is generated across those node orderings\. The compatible node orderings are precisely the*increasing labellings*\(equivalently, the linear extensions\) of the tree, a classical combinatorial object with known cardinality\.\[[13](https://arxiv.org/html/2609.16854#biba.bib22)\]

The sublinear preferential attachment process induces a prior probability over dependency trees\. Rather than fixing the attachment parameterα\\alphaof Eq\.[1](https://arxiv.org/html/2609.16854#Sx1.E1), I integrate over its full sublinear range,0<α<10<\\alpha<1, using the symmetric priorα∼Beta⁡\(2,2\)\\alpha\\sim\\mathrm\{Beta\}\(2,2\)\.aaendnote:aThe results are insensitive to the marginalisation overα\\alpha: repeating the analyses with the previously estimated fixed valueα=\.42\\alpha=\.42produces qualitatively equivalent results\.This prior is broad across the sublinear range while placing moderately greater weight on intermediate values; its mode atα=\.5\\alpha=\.5is also close to the value previously found to best describe human dependency trees\.\[[21](https://arxiv.org/html/2609.16854#biba.bib6)\]The resulting tree probability is therefore

P⁡\(T\)=\|V⁡\(T\)\|n\!​∫01⟨P⁡\(T∣o,α\)⟩o∈V⁡\(T\)​p​\(α\)​𝑑α,P\(T\)=\\frac\{\|V\(T\)\|\}\{n\!\}\\int\_\{0\}^\{1\}\\left\\langle P\(T\\mid o,\\alpha\)\\right\\rangle\_\{o\\in V\(T\)\}p\(\\alpha\)\\,d\\alpha,\(4\)wherep⁡\(α\)p\(\\alpha\)is theBeta⁡\(2,2\)\\mathrm\{Beta\}\(2,2\)density\. In this way,α\\alphais marginalised as a latent variable rather than fitted to linguistic data\. This results in a data\-independent prior distribution over dependency trees\. The computation of the number of compatible orderings and the numerical estimation of this integral is described in the Methods\.

## The universal prior distinguishes language from randomness

I evaluated the universal sPA prior on up to 500 dependency trees randomly sampled from each of 138 languages in the Universal Dependencies v2\.11 corpora,\[[5](https://arxiv.org/html/2609.16854#biba.bib23)\]retaining all suitable trees for languages with fewer than 500\. I considered sentences containing 50 words or fewer \(see Methods for corpus preprocessing and sampling, and Extended Data Table 1 for the list of languages\)\.

For each attested tree ofnnwords, I compared its probability under the sPA prior with that under a uniform prior over rooted labelled trees\. By Cayley’s formula,\[[4](https://arxiv.org/html/2609.16854#biba.bib24)\]there arenn−2n^\{n\-2\}labelled trees andnnpossible choices of root, yieldingnn−1n^\{n\-1\}rooted labelled trees and hence a uniform probability ofn1−nn^\{1\-n\}for each tree\. As sanity checks, I used two controls to verify that this comparison discriminates between the two generative processes: For each attested tree, I generated a tree of the same size from a true sPA model and another uniformly from the space of rooted labelled trees, and evaluated both under the same two priors\.

The sPA prior assigns systematically higher probabilities to attested dependency trees than the uniform prior \(Fig\.[2](https://arxiv.org/html/2609.16854#Sx2.F2)a\)\. As expected, the same preference is observed for trees generated by the sPA process, whereas uniformly generated trees show the opposite pattern\. For attested trees, the effect is remarkably consistent across every sentence length examined \(Fig\.[2](https://arxiv.org/html/2609.16854#Sx2.F2)b\) and for every language studied \(Fig\.[2](https://arxiv.org/html/2609.16854#Sx2.F2)c\)\. Together, these results show that attested dependency\-tree topologies are systematically and substantially closer to those favoured by the sPA prior than expected under a uniform distribution over trees, and that this pattern generalises across typologically diverse languages\.

ab![Refer to caption](https://arxiv.org/html/2609.16854v1/panel2a.png)![Refer to caption](https://arxiv.org/html/2609.16854v1/panel2b.png)c![Refer to caption](https://arxiv.org/html/2609.16854v1/panel2c.png)Figure 2:Probability advantage of the sPA model over the uniform model\.a, Distribution oflog2⁡PsPA−log2⁡Puniform\\log\_\{2\}P\_\{\\rm sPA\}\-\\log\_\{2\}P\_\{\\rm uniform\}for attested dependency trees, trees generated by the sPA model, and uniformly generated random trees\. Violins show the distributions; internal boxes indicate the median and interquartile range\.b, Mean log\-probability difference for each attested sentence length, shown for the same three sets of trees\. Points indicate means and error bars 95% confidence intervals\.c, Mean log\-probability difference for each language, shown for the same three sets of trees\. Points indicate means and error bars 95% confidence intervals\. Languages are ordered by the mean difference for attested sentences\. In all three panels, positive values indicate higher probability under sPA\. Conversely, negative values reflect that the probability is higher under the uniform model than under the sPA one\.
## The universal prior predicts language\-specific probabilities

The preceding analysis establishes that attested dependency\-tree shapes are systematically favoured by the sPA prior, but this pattern could in principle partly reflect conventions used to construct dependency\-tree annotations rather than the probabilistic structure of language itself\. A stronger test is whether sPA probabilities predict the probabilities of syntactic structures estimated independently from the statistics of individual languages\. To approximate these language\-specific probabilities, I used probabilistic context\-free grammars \(PCFGs\)\.\[[9](https://arxiv.org/html/2609.16854#biba.bib12)\]For each language in the Universal Dependencies represented by at least 10,000 non\-empty trees \(3434languages\), I induced a PCFG \(see Methods for the detailed procedure\)\. From each of the languages, I randomly sampled 500 sentences containing between 4 and 30 words\. I computed the universal sPA probability of each dependency tree and the probability of the corresponding context\-free derivation tree under the language\-specific PCFG\.

ab![Refer to caption](https://arxiv.org/html/2609.16854v1/panel3a.png)![Refer to caption](https://arxiv.org/html/2609.16854v1/panel3b.png)c![Refer to caption](https://arxiv.org/html/2609.16854v1/panel3c.png)Figure 3:Correlation between sPA and language\-specific PCFG probabilities\.a, Example of the relationship between the two probabilities for sentences of length 20 across all 34 languages\. Each point represents an individual sentence; the line shows the linear regression fit, with shading indicating its 95% confidence interval\.b, Mean within\-language Pearson correlation as a function of sentence length, with correlations combined across languages using Fisher’szztransformation\. Shading indicates 95% confidence intervals\.c, Mean within\-language Pearson correlation for each language, with correlations computed separately by sentence length and combined across lengths using Fisher’szztransformation\. Error bars indicate 95% confidence intervals, and languages are ordered by decreasing correlation\. Inbandc, correlations are weighted byn−3n\-3, the inverse of the approximate sampling variance of the Fisher\-transformed correlation, wherennis the number of sentences contributing to each correlation\.The sPA and language\-specific PCFG probabilities were positively correlated at every sentence length examined \(Fig\.[3](https://arxiv.org/html/2609.16854#Sx3.F3)a,b\), with all correlations being significantly greater than zero, and of a similar magnitude across all lengths, with no clear evidence of a systematic relation between sentence length and correlation strength\. The pattern was remarkably consistent across languages \(Fig\.[3](https://arxiv.org/html/2609.16854#Sx3.F3)c\): The correlations were positive for 33 of the 34 languages studied, and significant for 28 of those\. Japanese was the sole exception, with a negative \(but not significant\) average correlation\. This exception may reflect limitations of the PCFG approximation rather than a failure of the universal prior: in the preceding analysis, Japanese dependency trees, like those of every other language, are systematically better predicted by the sPA prior than by the uniform prior \(Fig\.[2](https://arxiv.org/html/2609.16854#Sx2.F2)c\)\. These correlations are moderate in magnitude –roughlyR≈\.3R\\approx\.3across sentence lengths– but substantial given that the sPA prior contains no language\-specific information\. The relationship was further confirmed by a linear mixed\-effects analysis of sentence\-level probabilities, accounting for variation across languages\. The length\-standardised sPA probability was a strong positive predictor of language\-specific length\-standardised PCFG probability \(β=\.146\\beta=\.146, 95% CI\[\.118,\.175\]\[\.118,\.175\],P<\.0001P<\.0001; see Methods\)\.

## The origins of syntactic structure probabilities

These results suggest that the probabilities of syntactic structures do not arise solely from statistical regularities acquired through language\-specific experience\. Incremental language production already induces a non\-uniform distribution over possible structures, before any language\-specific statistics are introduced\. Linguistic experience can therefore be understood as updating a structured universal prior rather than inducing the probabilities of syntactic structures from scratch\. The correlation between the data\-free prior and probabilities estimated independently from individual languages is consistent with this account: a universal component arising from language production remains detectable within the distributions shaped by language\-specific experience\.

This distinction connects two processes that are usually treated separately\. Language production constrains how syntactic structures can be incrementally constructed, whereas statistical learning determines how experience modifies their probabilities\. The present results suggest that the former constrains the initial distribution on which the latter operates\. This echoes a broader principle introduced by D’Arcy Thompson in the study of biological form: understanding why particular forms arise requires considering the generative processes and constraints that produce them, rather than only the forces acting on the forms once produced\.\[[29](https://arxiv.org/html/2609.16854#biba.bib30)\]Applied to language, this perspective suggests that the distribution of syntactic structures may be shaped not only by linguistic experience, but also by the very process through which those structures are gradually built up\.

More generally, the present results show how a universal probabilistic bias emerges from a cognitive process without requiring either language\-specific statistical estimation or an initially uniform distribution over syntactic structures\. The same principle could provide a data\-independent structural bias for computational models, including large language models, whose probability distributions are predominantly learned from linguistic data\. The probabilistic structure of language may therefore reflect not only what speakers have experienced, but also how language must be produced\.

## References

## References

- \[1\]A\. Barabási and R\. Albert\(1999\)Emergence of scaling in random networks\.Science286\(5439\),pp\. 509–512\.External Links:[Document](https://dx.doi.org/10.1126/science.286.5439.509)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1)\.
- \[2\]J\. K\. Bock\(1986\)Syntactic persistence in language production\.Cogn\. Psychol\.18\(3\),pp\. 355–387\.External Links:[Document](https://dx.doi.org/10.1016/0010-0285%2886%2990004-6)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1)\.
- \[3\]T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei\(2020\)Language models are few\-shot learners\.InAdv\. Neural Inf\. Process\. Syst\. \(NeurIPS\),H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 1877–1901\.Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[4\]A\. Cayley\(1889\)A theorem on trees\.Q\. J\. Pure Appl\. Math\.23,pp\. 376–378\.Cited by:[The universal prior distinguishes language from randomness](https://arxiv.org/html/2609.16854#Sx2.p2.1)\.
- \[5\]M\. de Marneffe, C\. D\. Manning, J\. Nivre, and D\. Zeman\(2021\)Universal Dependencies\.Comput\. Linguist\.47\(2\),pp\. 255–308\.External Links:[Document](https://dx.doi.org/10.1162/coli%5Fa%5F00402)Cited by:[The universal prior distinguishes language from randomness](https://arxiv.org/html/2609.16854#Sx2.p1.1),[§0\.1](https://arxiv.org/html/2609.16854#Sx5.SS1.p1.1)\.
- \[6\]F\. Ferreira, L\. R\. Slevc, and E\. S\. Rogers\(2007\)How do speakers avoid ambiguous linguistic expressions?\.Cognition104\(2\),pp\. 358–374\.External Links:[Document](https://dx.doi.org/10.1016/j.cognition.2006.07.002)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1)\.
- \[7\]J\. Hale\(2001\)A probabilistic Earley parser as a psycholinguistic model\.InProc\. 2nd Conf\. North Am\. Chapter Assoc\. Comput\. Linguist\. Hum\. Lang\. Technol\. \(NAACL\-HLT\),Pittsburgh, PA,pp\. 1–8\.External Links:[Document](https://dx.doi.org/10.3115/1073336.1073357)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[8\]T\. F\. Jaeger\(2010\)Redundancy and reduction: speakers manage information density\.Psychol\. Rev\.117\(1\),pp\. 23–56\.External Links:[Document](https://dx.doi.org/10.1037/a0017907)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[9\]M\. Johnson\(1998\)PCFG models of linguistic tree representations\.Comput\. Linguist\.24\(4\),pp\. 613–632\.Cited by:[The universal prior predicts language\-specific probabilities](https://arxiv.org/html/2609.16854#Sx3.p1.1),[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[10\]S\. Kirby, H\. Cornish, and K\. Smith\(2008\)Cumulative cultural evolution in the laboratory: an experimental approach to the origins of structure in human language\.Proc\. Natl\. Acad\. Sci\. U\.S\.A\.105\(31\),pp\. 10681–10686\.External Links:[Document](https://dx.doi.org/10.1073/pnas.0707835105)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1)\.
- \[11\]D\. Klein and C\. D\. Manning\(2003\)Accurate unlexicalized parsing\.InProc\. 41st Annu\. Meet\. Assoc\. Comput\. Linguist\. \(ACL\),Sapporo, Japan,pp\. 423–430\.External Links:[Document](https://dx.doi.org/10.3115/1075096.1075150)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[12\]D\. A\. Knowles and Z\. Ghahramani\(2011\)Pitman\-Yor diffusion trees\.InProc\. 27th Conf\. Uncertain\. Artif\. Intell\. \(UAI\),pp\. 410–418\.Cited by:[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[13\]D\. E\. Knuth\(1998\)The art of computer programming, volume 3: sorting and searching\.2 edition,Addison–Wesley,Reading, MA\.Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p3.3),[§0\.2](https://arxiv.org/html/2609.16854#Sx5.SS2.p2.1)\.
- \[14\]P\. L\. Krapivsky, S\. Redner, and F\. Leyvraz\(2000\)Connectivity of growing random networks\.Phys\. Rev\. Lett\.85\(21\),pp\. 4629–4632\.External Links:[Document](https://dx.doi.org/10.1103/PhysRevLett.85.4629)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1),[Abstract](https://arxiv.org/html/2609.16854#abstract1.1)\.
- \[15\]W\. J\. M\. Levelt\(1989\)Speaking: from intention to articulation\.MIT Press,Cambridge, MA\.Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1)\.
- \[16\]R\. Levy\(2008\)Expectation\-based syntactic comprehension\.Cognition106\(3\),pp\. 1126–1177\.External Links:[Document](https://dx.doi.org/10.1016/j.cognition.2007.05.006)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[17\]M\. C\. MacDonald\(1999\)Distributional information in language comprehension, production, and acquisition: three puzzles and a moral\.InThe Emergence of Language,B\. MacWhinney \(Ed\.\),pp\. 177–196\.Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1)\.
- \[18\]M\. C\. MacDonald\(2013\)How language production shapes language form and comprehension\.Front\. Psychol\.4,pp\. 226\.External Links:[Document](https://dx.doi.org/10.3389/fpsyg.2013.00226)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1)\.
- \[19\]K\. Mahowald, A\. James, R\. Futrell, and E\. Gibson\(2016\)A meta\-analysis of syntactic priming in language production\.J\. Mem\. Lang\.91,pp\. 5–27\.External Links:[Document](https://dx.doi.org/10.1016/j.jml.2016.03.009)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1)\.
- \[20\]F\. Moscoso del Prado Martín\(2025\)Measuring grammatical diversity from small corpora: Derivational entropy rates, mean length of utterances, and annotation invariance\.Comput\. Linguist\.51\(4\),pp\. 1191–1233\.External Links:[Document](https://dx.doi.org/10.1162/coli.a.15)Cited by:[§0\.4](https://arxiv.org/html/2609.16854#Sx5.SS4.p1.1)\.
- \[21\]F\. Moscoso del Prado Martín\(2026\)Universal topological regularity of syntactic structures\.arXiv preprint\.External Links:arXiv:2302\.00129v2Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1),[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1),[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p4.1),[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p5.1)\.
- \[22\]R\. M\. Neal\(2003\)Density modeling and clustering using Dirichlet diffusion trees\.InBayesian Statistics 7,J\. M\. Bernardoet al\.\(Eds\.\),pp\. 619–629\.Cited by:[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[23\]M\. J\. Pickering and V\. S\. Ferreira\(2008\)Structural priming: a critical review\.Psychol\. Bull\.134\(3\),pp\. 427–459\.External Links:[Document](https://dx.doi.org/10.1037/0033-2909.134.3.427)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1)\.
- \[24\]H\. Prüfer\(1918\)Neuer Beweis eines Satzes über Permutationen\.Arch\. Math\. Phys\.27,pp\. 142–144\.Cited by:[§0\.3](https://arxiv.org/html/2609.16854#Sx5.SS3.p1.1)\.
- \[25\]J\. R\. Saffran, R\. N\. Aslin, and E\. L\. Newport\(1996\)Statistical learning by 8\-month\-old infants\.Science274\(5294\),pp\. 1926–1928\.External Links:[Document](https://dx.doi.org/10.1126/science.274.5294.1926)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1)\.
- \[26\]S\. Seabold and J\. Perktold\(2010\)Statsmodels: econometric and statistical modeling with Python\.InProc\. 9th Python Sci\. Conf\.,pp\. 92–96\.External Links:[Document](https://dx.doi.org/10.25080/Majora-92bf1922-011)Cited by:[§0\.5](https://arxiv.org/html/2609.16854#Sx5.SS5.p4.1)\.
- \[27\]C\. E\. Shannon\(1951\)Prediction and entropy of printed English\.Bell Syst\. Tech\. J\.30\(1\),pp\. 50–64\.External Links:[Document](https://dx.doi.org/10.1002/j.1538-7305.1951.tb01366.x)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[28\]Y\. W\. Teh, M\. I\. Jordan, M\. J\. Beal, and D\. M\. Blei\(2006\)Hierarchical Dirichlet processes\.J\. Am\. Stat\. Assoc\.101\(476\),pp\. 1566–1581\.External Links:[Document](https://dx.doi.org/10.1198/016214506000000302)Cited by:[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[29\]D\. W\. Thompson\(1917\)On growth and form\.Cambridge University Press,Cambridge\.Cited by:[The origins of syntactic structure probabilities](https://arxiv.org/html/2609.16854#Sx4.p2.1)\.
- \[30\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdv\. Neural Inf\. Process\. Syst\. \(NeurIPS\),I\. Guyon, U\. V\. Luxburg, S\. Bengio, H\. Wallach, R\. Fergus, S\. Vishwanathan, and R\. Garnett \(Eds\.\),Vol\.30,pp\. 5998–6008\.Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.

\{methods\}

### 0\.1Corpora and preprocessing

I analysed all languages represented in the Universal Dependencies v2\.11 treebanks \(UD\)\.\[[5](https://arxiv.org/html/2609.16854#biba.bib23)\]For languages represented by multiple treebanks, I concatenated all treebanks listed for that language\. I retained only lexical vertices, removing punctuation and range vertices \(e\.g\., “3–4” in the CoNLL format used in UD\), deleted relation labels, and replaced vertex labels with consecutive integers\. After these modifications, I retained only graphs that remained actual trees \(see Extended Data Table[1](https://arxiv.org/html/2609.16854#Sx5.T1)for the count of valid trees available in each language\)\.

For the comparison between the sPA and uniform priors, I randomly sampled up to 500 trees \(taking all trees whenever fewer than 500 were available\) with a maximum length of 50 words\. For the analysis of the correlation between sPA and PCFG probabilities, I instead used a sample of 500 trees with a maximum length of 30 words, as longer trees were increasingly rare\.

Because trees withN≤3N\\leq 3do not exhibit any shape variability \(all trees with three or fewer vertices are simultaneously line and star graphs\), in both samples I considered only dependency graphs with at least four vertices\.

### 0\.2Computing the universal prior

Computing the universal prior in Eq\.[4](https://arxiv.org/html/2609.16854#Sx1.E4)requires two quantities: the number of node orderings compatible with a dependency tree, and the average probability with which the tree is generated across those orderings\. The former can be computed exactly, whereas the latter is estimated by Monte Carlo sampling\.

Compatible node orderings\.In the network growth model, only orderings in which every node appears after all of its ancestors are compatible with a given dependency tree\. These compatible orderings are precisely the increasing labellings of the tree, whose number is given exactly by the hook length formula,\[[13](https://arxiv.org/html/2609.16854#biba.bib22)\]

\|V⁡\(T\)\|=n\!∏v∈Thv,\|V\(T\)\|=\\frac\{n\!\}\{\\prod\_\{v\\in T\}h\_\{v\}\},\(5\)where the hook length \(hvh\_\{v\}\) of vertexvvis the number of vertices in the subtree rooted atvv, includingvvitself\. Notice that the factorial in the numerator of Eq\.[5](https://arxiv.org/html/2609.16854#Sx5.E5)cancels out with the factorial in Eq\.[4](https://arxiv.org/html/2609.16854#Sx1.E4), simplifying its calculation\.

Probability of a tree given a node ordering\.Given a compatible node ordering, the probability of generating the tree is computed by replaying the sublinear preferential attachment process\. Letvtv\_\{t\}denote the node introduced at steptt, and letVt−1V\_\{t\-1\}denote the set of nodes already present\. If nodeu∈Vt−1u\\in V\_\{t\-1\}currently hasku​\(t\)k\_\{u\}\(t\)direct dependents, its attachment weight is

w⁡\(u,α\)=\(ku​\(t\)\+1\)α\.w\(u,\\alpha\)=\(k\_\{u\}\(t\)\+1\)^\{\\alpha\}\.\(6\)The probability that the newly introduced node attaches to its observed parentptp\_\{t\}is therefore

P⁡\(vt→pt∣Vt−1,α\)=w⁡\(pt,α\)∑u∈Vt−1w⁡\(u,α\)\.P\(v\_\{t\}\\rightarrow p\_\{t\}\\mid V\_\{t\-1\},\\alpha\)=\\frac\{w\(p\_\{t\},\\alpha\)\}\{\\sum\_\{u\\in V\_\{t\-1\}\}w\(u,\\alpha\)\}\.\(7\)The probability of the complete production history is obtained by multiplying these attachment probabilities over all construction steps,

P⁡\(T∣o,α\)=∏t=2nP⁡\(vt→pt∣Vt−1,α\),P\(T\\mid o,\\alpha\)=\\prod\_\{t=2\}^\{n\}P\(v\_\{t\}\\rightarrow p\_\{t\}\\mid V\_\{t\-1\},\\alpha\),\(8\)while updating direct dependent counts after each attachment\.

Monte Carlo estimation\.The number of node orderings compatible with a given tree \(\|V⁡\(T\)\|\|V\(T\)\|\) can become extremely large as tree size increases\. Exhaustive evaluation of Eq\.[4](https://arxiv.org/html/2609.16854#Sx1.E4)is therefore computationally infeasible even for moderately sized dependency trees\. Instead, I estimate it using a Monte Carlo method\.

The probability in Eq\.[4](https://arxiv.org/html/2609.16854#Sx1.E4)requires averaging jointly over compatible node orderings and over the attachment exponentα\\alpha\. For this exponent, I use the prior

α∼Beta⁡\(2,2\),\\alpha\\sim\\mathrm\{Beta\}\(2,2\),\(9\)which has support over the complete sublinear range0<α<10<\\alpha<1and is symmetric aroundα=\.5\\alpha=\.5\. The prior remains sufficiently broad to assign substantial probability throughout this interval\. For each Monte Carlo sampleii, I independently draw a compatible node orderingoio\_\{i\}uniformly fromV⁡\(T\)V\(T\)and an attachment exponentαi∼Beta⁡\(2,2\)\\alpha\_\{i\}\\sim\\mathrm\{Beta\}\(2,2\), and evaluateP⁡\(T∣oi,αi\)P\(T\\mid o\_\{i\},\\alpha\_\{i\}\)by replaying the growth process\. ForNNsamples, the integrated mean conditional probability is then estimated as

𝔼o,α​\[P⁡\(T∣o,α\)\]≈1N​∑i=1NP⁡\(T∣oi,αi\)\.\\mathbb\{E\}\_\{o,\\alpha\}\[P\(T\\mid o,\\alpha\)\]\\approx\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}P\(T\\mid o\_\{i\},\\alpha\_\{i\}\)\.\(10\)Substituting this estimate into Eq\.[4](https://arxiv.org/html/2609.16854#Sx1.E4)gives

P^​\(T\)=\|V⁡\(T\)\|n\!​1N​∑i=1NP⁡\(T∣oi,αi\)\.\\widehat\{P\}\(T\)=\\frac\{\|V\(T\)\|\}\{n\!\}\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}P\(T\\mid o\_\{i\},\\alpha\_\{i\}\)\.\(11\)This Monte Carlo estimator jointly marginalises over uncertainty in production order and in the sublinear attachment exponent\. This estimator is unbiased and converges to the exact expectation asNNincreases \(see Extended Data Fig\.[1](https://arxiv.org/html/2609.16854#Sx5.F1)\)\. Unless otherwise stated, all analyses useN=1000N=1000joint samples per tree\.

### 0\.3Generation of random tree controls

Uniform random\-tree baselines for each number of nodes were generated by uniformly sampling Prüfer sequences\[[24](https://arxiv.org/html/2609.16854#biba.bib26)\]and then uniformly selecting a root node to determine the orientation of the edges\.

Random sPA tree baselines were generated sequentially, starting from a single root node and adding one node at each step\. Each new node was attached to an existing nodeiiwith probability proportional to\(ki\+1\)α\(k\_\{i\}\+1\)^\{\\alpha\}, wherekik\_\{i\}is the current number of direct dependents of nodeii\. Trees were grown until they matched the number of nodes of the corresponding attested dependency tree\. I used a fixed value ofα=\.42\\alpha=\.42, which was previously found to be optimal for human language sentences\.11

### 0\.4Language\-specific probability estimation\.

I converted the dependency trees into phrase\-structure trees using a common procedure\.\[[20](https://arxiv.org/html/2609.16854#biba.bib27)\]These trees were built to have parts\-of\-speech as their leaves \(i\.e\., they were preterminalised\)\. From the resulting non\-empty phrase\-structure trees \(without any minimum or maximum number of words\), I induced, by maximum likelihood, a separate PCFG for each language\. I restricted these analyses to languages with at least 10,000 non\-empty dependency trees to ensure that the PCFG estimates were based on sufficiently large samples, resulting in 34 languages\. The language\-specific log probability of each selected sentence was approximated by the log probability assigned by the corresponding PCFG to its phrase\-structure tree, computed as the sum of the log probabilities of the grammar rules used in its derivation\.

### 0\.5Statistical Analyses

I tested the relationship between the universal sPA and language\-specific PCFG probabilities using a linear mixed\-effects model with log PCFG probability as the dependent variable and log sPA probability, sentence length, and their interaction as fixed effects\. PCFG and sPA log probabilities were independently standardised to zero mean and unit variance within each sentence length, and the sentence lengths were centred to have a mean of zero\.

I compared two random\-effects structures fitting the models by maximum likelihood: a model containing only a random intercept for language, and a model additionally containing a language\-specific random slope for sPA probability\. Adding the random slope substantially improved model fit \(likelihood\-ratio test:χ2​\(3\)=397\.65\\chi^\{2\}\(3\)=397\.65,P<\.0001P<\.0001\), and produced lower AIC \(42,192\.61 versus 41,800\.97\) and BIC \(42,231\.32 versus 41,862\.90\)\. I therefore retained the random intercept and random slope and refitted the final model using restricted maximum likelihood\.

The final model showed a strong positive relationship between length\-standardised sPA and PCFG probabilities \(β=\.146\\beta=\.146, 95% CI\[\.118,\.175\]\[\.118,\.175\],z=10\.056z=10\.056,P<\.0001P<\.0001\)\. Sentence length also had a significant main effect \(β=\.080\\beta=\.080, 95% CI\[\.058,\.102\]\[\.058,\.102\],z=7\.156z=7\.156,P<\.0001P<\.0001\), whereas the interaction between sPA probability and sentence length did not reach significance \(β=−\.002\\beta=\-\.002, 95% CI\[−\.004,\.000\]\[\-\.004,\.000\],z=−1\.726z=\-1\.726,P=\.084P=\.084\)\.

Models were fitted in Python using thestatsmodelspackage\.\[[26](https://arxiv.org/html/2609.16854#biba.bib25)\]

## Methods\-Only References

- \[1\]A\. Barabási and R\. Albert\(1999\)Emergence of scaling in random networks\.Science286\(5439\),pp\. 509–512\.External Links:[Document](https://dx.doi.org/10.1126/science.286.5439.509)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1)\.
- \[2\]J\. K\. Bock\(1986\)Syntactic persistence in language production\.Cogn\. Psychol\.18\(3\),pp\. 355–387\.External Links:[Document](https://dx.doi.org/10.1016/0010-0285%2886%2990004-6)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1)\.
- \[3\]T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei\(2020\)Language models are few\-shot learners\.InAdv\. Neural Inf\. Process\. Syst\. \(NeurIPS\),H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 1877–1901\.Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[4\]A\. Cayley\(1889\)A theorem on trees\.Q\. J\. Pure Appl\. Math\.23,pp\. 376–378\.Cited by:[The universal prior distinguishes language from randomness](https://arxiv.org/html/2609.16854#Sx2.p2.1)\.
- \[5\]M\. de Marneffe, C\. D\. Manning, J\. Nivre, and D\. Zeman\(2021\)Universal Dependencies\.Comput\. Linguist\.47\(2\),pp\. 255–308\.External Links:[Document](https://dx.doi.org/10.1162/coli%5Fa%5F00402)Cited by:[The universal prior distinguishes language from randomness](https://arxiv.org/html/2609.16854#Sx2.p1.1),[§0\.1](https://arxiv.org/html/2609.16854#Sx5.SS1.p1.1)\.
- \[6\]F\. Ferreira, L\. R\. Slevc, and E\. S\. Rogers\(2007\)How do speakers avoid ambiguous linguistic expressions?\.Cognition104\(2\),pp\. 358–374\.External Links:[Document](https://dx.doi.org/10.1016/j.cognition.2006.07.002)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1)\.
- \[7\]J\. Hale\(2001\)A probabilistic Earley parser as a psycholinguistic model\.InProc\. 2nd Conf\. North Am\. Chapter Assoc\. Comput\. Linguist\. Hum\. Lang\. Technol\. \(NAACL\-HLT\),Pittsburgh, PA,pp\. 1–8\.External Links:[Document](https://dx.doi.org/10.3115/1073336.1073357)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[8\]T\. F\. Jaeger\(2010\)Redundancy and reduction: speakers manage information density\.Psychol\. Rev\.117\(1\),pp\. 23–56\.External Links:[Document](https://dx.doi.org/10.1037/a0017907)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[9\]M\. Johnson\(1998\)PCFG models of linguistic tree representations\.Comput\. Linguist\.24\(4\),pp\. 613–632\.Cited by:[The universal prior predicts language\-specific probabilities](https://arxiv.org/html/2609.16854#Sx3.p1.1),[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[10\]S\. Kirby, H\. Cornish, and K\. Smith\(2008\)Cumulative cultural evolution in the laboratory: an experimental approach to the origins of structure in human language\.Proc\. Natl\. Acad\. Sci\. U\.S\.A\.105\(31\),pp\. 10681–10686\.External Links:[Document](https://dx.doi.org/10.1073/pnas.0707835105)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1)\.
- \[11\]D\. Klein and C\. D\. Manning\(2003\)Accurate unlexicalized parsing\.InProc\. 41st Annu\. Meet\. Assoc\. Comput\. Linguist\. \(ACL\),Sapporo, Japan,pp\. 423–430\.External Links:[Document](https://dx.doi.org/10.3115/1075096.1075150)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[12\]D\. A\. Knowles and Z\. Ghahramani\(2011\)Pitman\-Yor diffusion trees\.InProc\. 27th Conf\. Uncertain\. Artif\. Intell\. \(UAI\),pp\. 410–418\.Cited by:[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[13\]D\. E\. Knuth\(1998\)The art of computer programming, volume 3: sorting and searching\.2 edition,Addison–Wesley,Reading, MA\.Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p3.3),[§0\.2](https://arxiv.org/html/2609.16854#Sx5.SS2.p2.1)\.
- \[14\]P\. L\. Krapivsky, S\. Redner, and F\. Leyvraz\(2000\)Connectivity of growing random networks\.Phys\. Rev\. Lett\.85\(21\),pp\. 4629–4632\.External Links:[Document](https://dx.doi.org/10.1103/PhysRevLett.85.4629)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1),[Abstract](https://arxiv.org/html/2609.16854#abstract1.1)\.
- \[15\]W\. J\. M\. Levelt\(1989\)Speaking: from intention to articulation\.MIT Press,Cambridge, MA\.Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1)\.
- \[16\]R\. Levy\(2008\)Expectation\-based syntactic comprehension\.Cognition106\(3\),pp\. 1126–1177\.External Links:[Document](https://dx.doi.org/10.1016/j.cognition.2007.05.006)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[17\]M\. C\. MacDonald\(1999\)Distributional information in language comprehension, production, and acquisition: three puzzles and a moral\.InThe Emergence of Language,B\. MacWhinney \(Ed\.\),pp\. 177–196\.Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1)\.
- \[18\]M\. C\. MacDonald\(2013\)How language production shapes language form and comprehension\.Front\. Psychol\.4,pp\. 226\.External Links:[Document](https://dx.doi.org/10.3389/fpsyg.2013.00226)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1)\.
- \[19\]K\. Mahowald, A\. James, R\. Futrell, and E\. Gibson\(2016\)A meta\-analysis of syntactic priming in language production\.J\. Mem\. Lang\.91,pp\. 5–27\.External Links:[Document](https://dx.doi.org/10.1016/j.jml.2016.03.009)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1)\.
- \[20\]F\. Moscoso del Prado Martín\(2025\)Measuring grammatical diversity from small corpora: Derivational entropy rates, mean length of utterances, and annotation invariance\.Comput\. Linguist\.51\(4\),pp\. 1191–1233\.External Links:[Document](https://dx.doi.org/10.1162/coli.a.15)Cited by:[§0\.4](https://arxiv.org/html/2609.16854#Sx5.SS4.p1.1)\.
- \[21\]F\. Moscoso del Prado Martín\(2026\)Universal topological regularity of syntactic structures\.arXiv preprint\.External Links:arXiv:2302\.00129v2Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p1.1),[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1),[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p4.1),[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p5.1)\.
- \[22\]R\. M\. Neal\(2003\)Density modeling and clustering using Dirichlet diffusion trees\.InBayesian Statistics 7,J\. M\. Bernardoet al\.\(Eds\.\),pp\. 619–629\.Cited by:[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[23\]M\. J\. Pickering and V\. S\. Ferreira\(2008\)Structural priming: a critical review\.Psychol\. Bull\.134\(3\),pp\. 427–459\.External Links:[Document](https://dx.doi.org/10.1037/0033-2909.134.3.427)Cited by:[A universal prior over syntactic structures](https://arxiv.org/html/2609.16854#Sx1.p2.1)\.
- \[24\]H\. Prüfer\(1918\)Neuer Beweis eines Satzes über Permutationen\.Arch\. Math\. Phys\.27,pp\. 142–144\.Cited by:[§0\.3](https://arxiv.org/html/2609.16854#Sx5.SS3.p1.1)\.
- \[25\]J\. R\. Saffran, R\. N\. Aslin, and E\. L\. Newport\(1996\)Statistical learning by 8\-month\-old infants\.Science274\(5294\),pp\. 1926–1928\.External Links:[Document](https://dx.doi.org/10.1126/science.274.5294.1926)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1)\.
- \[26\]S\. Seabold and J\. Perktold\(2010\)Statsmodels: econometric and statistical modeling with Python\.InProc\. 9th Python Sci\. Conf\.,pp\. 92–96\.External Links:[Document](https://dx.doi.org/10.25080/Majora-92bf1922-011)Cited by:[§0\.5](https://arxiv.org/html/2609.16854#Sx5.SS5.p4.1)\.
- \[27\]C\. E\. Shannon\(1951\)Prediction and entropy of printed English\.Bell Syst\. Tech\. J\.30\(1\),pp\. 50–64\.External Links:[Document](https://dx.doi.org/10.1002/j.1538-7305.1951.tb01366.x)Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.
- \[28\]Y\. W\. Teh, M\. I\. Jordan, M\. J\. Beal, and D\. M\. Blei\(2006\)Hierarchical Dirichlet processes\.J\. Am\. Stat\. Assoc\.101\(476\),pp\. 1566–1581\.External Links:[Document](https://dx.doi.org/10.1198/016214506000000302)Cited by:[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p4.1)\.
- \[29\]D\. W\. Thompson\(1917\)On growth and form\.Cambridge University Press,Cambridge\.Cited by:[The origins of syntactic structure probabilities](https://arxiv.org/html/2609.16854#Sx4.p2.1)\.
- \[30\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdv\. Neural Inf\. Process\. Syst\. \(NeurIPS\),I\. Guyon, U\. V\. Luxburg, S\. Bengio, H\. Wallach, R\. Fergus, S\. Vishwanathan, and R\. Garnett \(Eds\.\),Vol\.30,pp\. 5998–6008\.Cited by:[Abstract](https://arxiv.org/html/2609.16854#abstract1.1),[A Data\-free Universal Prior over Syntactic Structures](https://arxiv.org/html/2609.16854#p3.1)\.

###### Notes

1. [a](https://arxiv.org/html/2609.16854#endnote1)a a endnote a The results are insensitive to the marginalisation over α : repeating the analyses with the previously estimated fixed value = α \.42 produces qualitatively equivalent results\.

\{addendum\}

I am indebted to Prof\. Enrique Amigó and Prof\. Michael Anderson for helpful suggestions on this manuscript\.

Python functions for computing probabilities of dependency trees under the sPA prior, sampling compatible node orderings, and sampling trees from the sPA and uniform models are available athttps://github\.com/fermosc24/ProbTreesPA\.

The author declares that he has no competing financial interests\.

Correspondence and requests for materials should be addressed to F\. MdP\. \(email: fm611@cst\.cam\.ac\.uk\)\.

![Refer to caption](https://arxiv.org/html/2609.16854v1/convergence.png)Extended Data Figure 1:Convergence of the Monte Carlo mean conditional probability estimate\.The figure plots the mean estimated surprisal−⟨l​o​g​P​\(T\|o\)⟩o∈V⁡\(T\)\-\\left<logP\(T\|o\)\\right\>\_\{o\\in V\(T\)\}as a function of the number of sampled histories for a 30\-node tree randomly generated by sublinear preferential attachment withα=0\.42\\alpha=0\.42\. The shaded region denotes the 95% confidence interval of the estimate\. The horizontal axis is logarithmic\.Extended Data Table 1:Languages included in the analysis, ordered by decreasing number of valid available trees\.LanguageFamilyGroupNumber of Valid TreesGermanIndo\-EuropeanWest Germanic198,830JapaneseJaponicJapanese120,218CzechIndo\-EuropeanWest Slavic115,497RussianIndo\-EuropeanEast Slavic101,260PortugueseIndo\-EuropeanRomance67,709TurkishTurkicOghuz63,453Latin†Indo\-EuropeanItalic \(Latino\-Faliscan\)53,303IcelandicIndo\-EuropeanNorth Germanic52,395FrenchIndo\-EuropeanRomance46,378Classical Chinese†Sino\-TibetanSinitic45,893EnglishIndo\-EuropeanWest Germanic41,123RomanianIndo\-EuropeanRomance39,802NorwegianIndo\-EuropeanNorth Germanic37,679ItalianIndo\-EuropeanRomance36,676PolishIndo\-EuropeanWest Slavic35,839PersianIndo\-EuropeanIranian34,685SpanishIndo\-EuropeanRomance34,134EstonianUralicFinnic34,072KoreanKoreanicKorean33,191FinnishUralicFinnic31,574Ancient Greek†Indo\-EuropeanHellenic28,975ArabicAfro\-AsiaticSemitic27,500BelarusianIndo\-EuropeanEast Slavic21,060Old East Slavic†Indo\-EuropeanEast Slavic19,303DutchIndo\-EuropeanWest Germanic18,354HindiIndo\-EuropeanIndo\-Aryan17,611CatalanIndo\-EuropeanRomance16,500Old French†Indo\-EuropeanRomance15,927LatvianIndo\-EuropeanBaltic15,632SlovenianIndo\-EuropeanSouth Slavic14,932ChineseSino\-TibetanSinitic12,300SwedishIndo\-EuropeanNorth Germanic11,594HebrewAfro\-AsiaticSemitic11,015BulgarianIndo\-EuropeanSouth Slavic10,451CroatianIndo\-EuropeanSouth Slavic8,921Basqueisolate–8,691SlovakIndo\-EuropeanWest Slavic8,678NaijacreoleEnglish\-lexifier8,619IndonesianAustronesianMalayic7,555UkrainianIndo\-EuropeanEast Slavic6,403Western ArmenianIndo\-EuropeanArmenian6,205PomakIndo\-EuropeanSouth Slavic5,791IrishIndo\-EuropeanCeltic5,773Old Church Slavonic†Indo\-EuropeanSouth Slavic5,449UrduIndo\-EuropeanIndo\-Aryan5,128DanishIndo\-EuropeanNorth Germanic5,087GalicianIndo\-EuropeanRomance4,899Gothic†Indo\-EuropeanEast Germanic4,742ArmenianIndo\-EuropeanArmenian4,448SerbianIndo\-EuropeanSouth Slavic4,363Scottish GaelicIndo\-EuropeanCeltic4,275LithuanianIndo\-EuropeanBaltic3,641Sanskrit†Indo\-EuropeanIndo\-Aryan3,380UyghurTurkicKarluk3,221VietnameseAustroasiaticVietic2,886North SamiUralicSami2,728FaroeseIndo\-EuropeanNorth Germanic2,720WelshIndo\-EuropeanCeltic2,445GreekIndo\-EuropeanHellenic2,423Manx†Indo\-EuropeanCeltic2,207Coptic†Afro\-AsiaticEgyptian2,146Turkish Germancode\-switchingOghuz / West Germanic2,140WolofAtlantic\-CongoAtlantic2,076AfrikaansIndo\-EuropeanWest Germanic1,929MalteseAfro\-AsiaticSemitic1,899Hindi Englishcode\-switchingIndo\-Aryan / West Germanic1,894Akkadian†Afro\-AsiaticSemitic1,844HungarianUralicUgric1,780ErzyaUralicMordvinic1,695Ancient Hebrew†Afro\-AsiaticSemitic1,579KicheMayanK’ichean1,121Mbya GuaraniTupianTupi\-Guarani1,086GuajajaraTupianTupi\-Guarani1,017AmharicAfro\-AsiaticSemitic1,013ThaiKra\-DaiTai999KazakhTurkicKipchak958GhegIndo\-EuropeanAlbanian956BambaraMandeWestern Mande930TamilDravidianSouth Dravidian878BuryatMongolicCentral Mongolic844CantoneseSino\-TibetanSinitic838BretonIndo\-EuropeanCeltic807Western Sierra Puebla NahuatlUto\-AztecanNahuan794Komi ZyrianUralicPermic787XibeTungusicJurchenic780KurmanjiIndo\-EuropeanIranian743TeluguDravidianSouth\-Central Dravidian665ChukchiChukotko\-KamchatkanChukotkan653Upper SorbianIndo\-EuropeanWest Slavic638ZaarAfro\-AsiaticChadic581Tupinamba†TupianTupi\-Guarani429MarathiIndo\-EuropeanIndo\-Aryan393Frisian DutchIndo\-EuropeanWest Germanic385MokshaUralicMordvinic350BhojpuriIndo\-EuropeanIndo\-Aryan349YorubaAtlantic\-CongoVolta\-Niger315KaroTupianRamarama297LigurianIndo\-EuropeanRomance297YupikEskimo\-AleutYupik284KangriIndo\-EuropeanIndo\-Aryan281YakutTurkicSiberian Turkic237KarelianUralicFinnic220Skolt SamiUralicSami216NheengatuTupianTupi\-Guarani178TagalogAustronesianPhilippine178Swedish Sign Languagesign language–169CebuanoAustronesianPhilippine150TatarTurkicKipchak146TekoTupianTupi\-Guarani132Hittite†Indo\-EuropeanAnatolian130JavaneseAustronesianMalayo\-Polynesian123LivviUralicFinnic119MundurukuTupianMunduruku113Swiss GermanIndo\-EuropeanWest Germanic100SinhalaIndo\-EuropeanIndo\-Aryan100ApurinaArawakanPurus98Low SaxonIndo\-EuropeanWest Germanic93AkuntsuTupianTupari93AbazaNorthwest CaucasianAbkhaz\-Abaza86Komi PermyakUralicPermic86South Levantine ArabicAfro\-AsiaticSemitic84Umbrian†Indo\-EuropeanItalic \(Sabellic\)80AlbanianIndo\-EuropeanAlbanian60BejaAfro\-AsiaticCushitic52KaaporTupianTupi\-Guarani48WarlpiriPama\-NyunganNgumpin\-Yapa47AssyrianAfro\-AsiaticSemitic \(Aramaic\)45BengaliIndo\-EuropeanIndo\-Aryan36MakurapTupianTupari24MalayalamDravidianSouth Dravidian23GuaraniTupianTupi\-Guarani22Old Turkish†TurkicCommon Turkic16XavanteMacro\-JeJe16MadiArawanMadi11NayiniIndo\-EuropeanIranian10KhunsariIndo\-EuropeanIranian9SoiIndo\-EuropeanIranian7NeapolitanIndo\-EuropeanRomance1†Extinct language\.