A Modular Architecture for Typologically Controlled Lexicon Generation

# A Modular Architecture for Typologically Controlled Lexicon Generation
Source: [https://arxiv.org/html/2605.28824](https://arxiv.org/html/2605.28824)
Sankalp Tattwadarshi Swain, Dhruv Kumar Birla Institute of Technology and Science, Pilani Pilani, Rajasthan 333031, India \{f20230769, dhruv\.kumar\}@pilani\.bits\-pilani\.ac\.in

###### Abstract

Constructing artificial lexicons that are pronounceable, typologically plausible, and semantically structured remains an open challenge in computational linguistics\. Existing conlang generators either lack formal phonotactic guarantees or delegate generation to opaque, non\-reproducible LLM\-based pipelines\. We propose a modular framework that samples phoneme inventories from PHOIBLE, generates word forms under interchangeable phonological grammars \(deterministic, OT, and MaxEnt\), and assigns meanings via a Swadesh–Leipzig–Jakarta ontology with explicit form–meaning alignment\. Evaluation on characternn\-gram perplexity, log\-likelihood, and KL divergence against PHOIBLE across lexicon sizes of 100\-5,000 forms shows that probabilistic grammars consistently outperform deterministic and random baselines on both phonotactic coherence and typological realism\.

## 1Introduction

Modelling phonological and lexical structure remains central to understanding how language systems can be formally represented, learned, and generated\(Prince & Smolensky,[2004](https://arxiv.org/html/2605.28824#bib.bib14); Hayes & Wilson,[2008](https://arxiv.org/html/2605.28824#bib.bib9)\)\. A practically important instance of this problem is*typologically grounded lexicon generation*: constructing artificial lexicons that map pronounceable, phonotactically well\-formed word forms to meanings while remaining aligned with cross\-linguistic phonological patterns\. Existing approaches are either rule\-based tools that offer designer control but lack formal phonotactic guarantees and typological grounding\(Heyer,[2021](https://arxiv.org/html/2605.28824#bib.bib10); Cai & Martens,[2023](https://arxiv.org/html/2605.28824#bib.bib3)\), or LLM\-based pipelines that produce plausible output but surrender interpretability and reproducibility entirely\(Alper et al\.,[2025](https://arxiv.org/html/2605.28824#bib.bib1); Taguchi & Sproat,[2025](https://arxiv.org/html/2605.28824#bib.bib16)\)\. Critically, no existing system enables a researcher to hold the phoneme inventory and constraint set fixed while varying the phonological grammar — the minimum requirement for isolating the contribution of any single formalisation to lexical structure\.

We introduce a modular, fully parameterized lexicon generation framework that resolves this gap by decomposing the pipeline into three independently controllable components\. First, a phonological inventory sampler draws phoneme inventories from PHOIBLE frequency distributions\(Moran & McCloy,[2019](https://arxiv.org/html/2605.28824#bib.bib12)\), enforcing empirically validated implicational universals through a statistically grounded repair stage\. Second, a word generation module evaluates candidate forms against a shared constraint set under four interchangeable grammatical formalisms — deterministic, Optimality Theory \(OT\), Harmonic Grammar \(HG\), and Maximum Entropy \(MaxEnt\) — that differ only in how constraint violations are resolved, ensuring that every generated word form is pronounceable by construction\. Third, a semantic assignment module maps generated forms to meanings drawn from a merged Leipzig–Jakarta and Swadesh ontology, optimising a Spearman rank correlation between phonological and semantic distance to produce a lexicon in which form–meaning alignment is an explicit objective rather than an incidental property\.

We evaluate the framework quantitatively along two complementary axes across lexicon sizes from 100 to 5,000 word forms\. Results show that probabilistic grammars — OT and MaxEnt — consistently outperform the deterministic and random baselines on both axes, and that the deterministic grammar pays a measurable typological cost through categorical constraint enforcement that systematically skews the phoneme distribution\.This work is relevant to the language modelling community as both a controlled testbed for probing phonotactic knowledge in LLMs and a pipeline for generating pronounceable, typologically grounded synthetic corpora for low\-resource speech and language modelling\.

##### Contributions\.

Our main contributions are as follows:

- •Modular, parameterized lexicon generation\.We propose a unified framework in which deterministic, OT, HG, and MaxEnt phonological grammars operate over a shared inventory and shared constraint set\. The novelty lies not in the grammars themselves but in their integration with typologically grounded inventory sampling, enabling confound\-free empirical comparison across formalisms that existing systems do not support\.
- •Typologically grounded inventory sampling\.We integrate PHOIBLE frequency distributions and statistically validated implicational universals as structured generative priors for phoneme inventory construction, grounding generated lexicons in cross\-linguistic typological reality\.
- •Pronounceability as a formal guarantee\.By enforcing phonotactic constraints — including the Sonority Sequencing Principle, onset/coda structure, and nasal–stop homorganicity — at generation time, the framework guarantees that every output word form is well\-formed, making the lexicon a principled foundation for spoken constructed languages\.
- •Form–meaning alignment via semantic optimisation\.We introduce a hill\-climbing semantic assignment procedure that maximises Spearman rank correlation between phonological and ontological semantic distance, operationalising lexical iconicity as an explicit generation objective\.
- •Quantitative evaluation framework\.We introduce a joint evaluation protocol combining character\-level language model perplexity and average log\-likelihood with KL divergence against a typological reference distribution, providing a quantitative benchmark for comparing phonotactic well\-formedness and typological realism of*generated*lexicons\.
- •Cross\-grammar compatibility analysis\.We empirically demonstrate that OT and MaxEnt induce distributionally equivalent phonotactic spaces, and that deterministic grammars generate a strict subset of probabilistic phonotactic patterns\.

Table 1:Comparison of lexicon generation systems across five dimensions: inventory source, word form generation, phonotactics, semantics, and typological grounding\. Our system is the only one that unifies all five within a single pipeline\.

## 2Background

##### Formal phonological grammars\.

Optimality Theory formalises phonotactic well\-formedness through a ranked constraint hierarchy under strict domination\(Prince & Smolensky,[2004](https://arxiv.org/html/2605.28824#bib.bib14)\), while Harmonic Grammar relaxes this into a numeric weighting scheme\(Smolensky & Legendre,[2006](https://arxiv.org/html/2605.28824#bib.bib15)\), and Maximum Entropy grammar extends it further into a fully probabilistic model over weighted constraint violations\(Goldwater & Johnson,[2003](https://arxiv.org/html/2605.28824#bib.bib8); Hayes & Wilson,[2008](https://arxiv.org/html/2605.28824#bib.bib9)\)\. These frameworks have each been studied and applied to individual natural languages in isolation\. However, a unified architecture in which deterministic, OT, HG, and MaxEnt models operate over a shared inventory and shared constraint set — enabling direct, confound\-free empirical comparison — has not been previously proposed\.

##### Typological databases and phoneme inventory modelling\.

Cross\-linguistic resources such as PHOIBLE\(Moran & McCloy,[2019](https://arxiv.org/html/2605.28824#bib.bib12)\)aggregate phoneme inventories from thousands of languages, encoding implicational universals and quantitative markedness distributions that demonstrate phoneme inventories are structured objects shaped by universal typological pressures\. Despite the richness of this data, the integration of typological databases as structured generative priors for phoneme inventory sampling within a language generation pipeline has not been addressed\.

##### Rule\-based and procedural conlang generation\.

Heyer \([2021](https://arxiv.org/html/2605.28824#bib.bib10)\)andCai & Martens \([2023](https://arxiv.org/html/2605.28824#bib.bib3)\)demonstrate that automated conlang construction is feasible through user\-defined phoneme templates and parameterisable syllable pipelines\. However, neither system grounds its inventory in cross\-linguistic data, employs a formal phonological grammar, or provides any quantitative measure of the phonotactic well\-formedness or typological plausibility of the generated output — a gap that prevents rigorous evaluation or reproducible comparison across parameter settings\.

##### Neural and probabilistic phonotactic models\.

Futrell et al\. \([2017](https://arxiv.org/html/2605.28824#bib.bib7)\)introduce a Bayesian generative model that captures sub\-lexical phonotactic structure across typologically diverse languages, andPimentel et al\. \([2020](https://arxiv.org/html/2605.28824#bib.bib13)\)establish a neural bits\-per\-phoneme measure of phonotactic complexity across 106 languages using LSTM language models\. While these contributions demonstrate that probabilistic and neural methods can characterise the phonotactic structure of natural lexicons with considerable precision, the application of such models to the*generation*of new lexicons under controlled typological conditions and with structured semantic assignment has not been addressed\.

##### Large language model pipelines for conlang generation\.

Alper et al\. \([2025](https://arxiv.org/html/2605.28824#bib.bib1)\)introduce ConlangCrafter, a multi\-hop LLM pipeline evaluated through macro\-level WALS\-feature diversity metrics, andTaguchi & Sproat \([2025](https://arxiv.org/html/2605.28824#bib.bib16)\)present IASC, an agentic system generating phonotactic grammars as executable code\. Both delegate phonotactic decisions to an LLM, producing output that is opaque, non\-reproducible, and without formal well\-formedness guarantees\. Neither evaluates phonotactic quality at the word level nor supports controlled ablation over grammatical formalisms\.

## 3Methodology

### 3\.1Overview

Our lexicon generation framework comprises two principal components: \(1\) a statistically grounded phonological inventory sampler, and \(2\) a modular word generation system that operates over the sampled inventory\. The pipeline first constructs a typologically plausible phoneme inventory by drawing on cross\-linguistic frequency distributions from PHOIBLE\(Moran & McCloy,[2019](https://arxiv.org/html/2605.28824#bib.bib12)\)\. This inventory then serves as the fixed, shared foundation for four interchangeable phonotactic grammars—deterministic, Optimality Theory \(OT\), Harmonic Grammar \(HG\), and Maximum Entropy \(MaxEnt\)—which share the same constraint representations but differ fundamentally in how violations are evaluated and resolved\. This design deliberately separates typological realism, modelled at the inventory level, from phonotactic variation, modelled at the grammatical level, enabling controlled comparison across categorical and probabilistic phonological formalisms within a single unified architecture\.

### 3\.2Phonological Inventory Sampling

The phonological sampler models sound systems as samples drawn from an empirically structured typological distribution\. Each phoneme is assigned a base sampling probability proportional to its cross\-linguistic frequency across PHOIBLE inventories, thereby operationalising markedness in probabilistic terms: typologically common segments are preferentially selected over rare ones, without categorical exclusion of any attested segment type\.

Inventory sizes are constrained by cross\-linguistic distributions\. Consonant and vowel counts are sampled such that the vowel\-to\-consonant ratio falls within the range\[0\.15,0\.40\]\[0\.15,\\,0\.40\], reflecting attested typological ranges\(Moran & McCloy,[2019](https://arxiv.org/html/2605.28824#bib.bib12)\)\. Optional archetype modes restrict sampling to particular structural subspaces—for instance, small CV\-dominant systems or consonant\-rich systems—enabling controlled typological variation while preserving empirical plausibility\.

To capture implicational universals and co\-occurrence tendencies, we conduct feature\-level statistical analysis over PHOIBLE inventories\. To validate the statistical basis for these typological constraints, we computed Pearson correlation coefficients and chi\-square statistics over all PHOIBLE inventories \(Table[2](https://arxiv.org/html/2605.28824#S3.T2)\)\. Correlation analysis reveals that inventory size is positively associated with the presence of marked segment classes: consonant count correlates significantly with ejective presence \(r=0\.324r=0\.324,p<0\.001p<0\.001\) and click presence \(r=0\.188r=0\.188,p<0\.001p<0\.001\), while vowel count correlates with vowel length contrast \(r=0\.412r=0\.412,p<0\.001p<0\.001\), justifying the conditioning of marked segment sampling probabilities on inventory size\. For several feature pairs, chi\-square tests yieldedχ2=0\\chi^\{2\}=0withp=1\.0p=1\.0, a result that initially suggests independence but upon inspection reflects near\-deterministic implicational structure with empty or near\-empty contingency cells\. Because chi\-square assumes symmetric variation, it is an inappropriate test for directional universals; we therefore turn to conditional probabilitiesP​\(Y∣X\)P\(Y\\mid X\)as a direct measure of implicational strength \(Table[3](https://arxiv.org/html/2605.28824#S3.T3)\)\. This analysis reveals that four of the five incorporated implicational universals are fully categorical \(P=1\.0P=1\.0\), while the remaining one is a near\-universal \(P=0\.997P=0\.997, failure rate=0\.003=0\.003\)\. Collectively, these results confirm that the constraints encoded in the sampler reflect statistically validated typological regularities rather than heuristic assumptions, providing an empirical grounding for both the deterministic enforcement of true universals and the probabilistic enforcement of near\-universals\.

RelationshipStatisticValuep\-valueConsonant count vs\. ejective presencerr0\.3240\.324<0\.001<0\.001Consonant count vs\. click presencerr0\.1880\.188<0\.001<0\.001Inventory size vs\. click presencerr0\.1880\.188<0\.001<0\.001Vowel count vs\. vowel length contrastrr0\.4120\.412<0\.001<0\.001Ejective↔\\leftrightarrowuvularχ2\\chi^\{2\}0\.0000\.0001\.0001\.000Pharyngeal↔\\leftrightarrowuvularχ2\\chi^\{2\}0\.0000\.0001\.0001\.000Nasal vowel↔\\leftrightarroworal vowelχ2\\chi^\{2\}0\.0000\.0001\.0001\.000Voiced↔\\leftrightarrowvoiceless obstruentχ2\\chi^\{2\}0\.6560\.6560\.4180\.418Fricative↔\\leftrightarrowstopχ2\\chi^\{2\}0\.0000\.0001\.0001\.000Front rounded↔\\leftrightarrowfront unroundedχ2\\chi^\{2\}0\.0000\.0001\.0001\.000Table 2:Pearson correlation coefficients \(rr\) and chi\-square statistics \(χ2\\chi^\{2\}\) computed over PHOIBLE inventories\. Correlation values measure symmetric associations between inventory size and marked segment presence; chi\-square values reflect feature co\-occurrence tests\. All correlation results are significant atp<0\.001p<0\.001\.Table 3:Conditional probability results for the five implicational universals incorporated into the sampler\.*TotalXX*denotes the number of inventories in which the antecedent feature is present;*Violations*counts cases where the consequent feature is absent;P​\(Y∣X\)P\(Y\\mid X\)is the empirical conditional probability; and*Failure Rate*is the complementary proportion\. Four of the five implications are fully categorical \(P=1\.0P=1\.0\); the voiced–voiceless obstruent implication is a near\-universal \(P=0\.997P=0\.997\)\.
### 3\.3Word Generation Framework

All word generators operate over the same sampled phoneme inventory and share a unified constraint set encoding widely attested phonotactic generalisations:Onset,NoCoda,\*Complex, the Sonority Sequencing Principle \(SSP\), and nasal\-stop homorganicity\. Candidate word forms are produced by sampling syllable structures from a parameterised template and filling syllable positions with segments drawn from the inventory\.

![Refer to caption](https://arxiv.org/html/2605.28824v1/pipeline.png)Figure 1:Overview of the lexicon generation pipeline\. PHOIBLE\-sampled inventories feed four interchangeable phonological grammars; the generated word space is then mapped to meanings via the semantic assignment module\.#### 3\.3\.1Deterministic Generator

The deterministic phonotactic generator implements a rule\-based grammar in which constraints function as hard filters\. Candidate forms that violate structural limits \(e\.g\., maximum onset complexity\), the SSP as formalised under\(Clements,[1990](https://arxiv.org/html/2605.28824#bib.bib5)\)sonority hierarchy, or the homorganicity requirement for nasal–stop sequences are rejected outright\. This corresponds to the classical generative view in which phonotactic well\-formedness is binary and constraint violations are categorically illicit\(Chomsky & Halle,[1968](https://arxiv.org/html/2605.28824#bib.bib4)\)\.

#### 3\.3\.2Optimality Theory Generator

The Optimality\-Theoretic \(OT\) generator replaces categorical rejection with evaluation over a fully generated candidate set\. Three variants are implemented within a shared constraint architecture:Strict OT, in which constraints are hierarchically ranked under strict domination such that any higher\-ranked violation is fatal\(Prince & Smolensky,[2004](https://arxiv.org/html/2605.28824#bib.bib14)\);Stochastic OT, which preserves this ranking but adds Gaussian noise at evaluation time to yield probabilistic variation\(Boersma,[1997](https://arxiv.org/html/2605.28824#bib.bib2)\); andHarmonic Grammar \(HG\), which replaces strict ranking with numeric weights, computing harmony as a weighted sum of violations and permitting multiple lower\-weighted violations to collectively outweigh a single higher\-weighted one\(Smolensky & Legendre,[2006](https://arxiv.org/html/2605.28824#bib.bib15); Legendre et al\.,[1990](https://arxiv.org/html/2605.28824#bib.bib11)\)\. Although all three variants share the same constraint definitions, they differ fundamentally in how constraint interaction is formalised\.

#### 3\.3\.3Maximum Entropy Generator

The Maximum Entropy \(MaxEnt\) grammar extends Harmonic Grammar into a fully probabilistic model\. Harmony is computed as a weighted sum of violations, and candidate probabilities are derived via the softmax transformationP​\(w\)∝e−H​\(w\)P\(w\)\\propto e^\{\-H\(w\)\}\(Goldwater & Johnson,[2003](https://arxiv.org/html/2605.28824#bib.bib8); Hayes & Wilson,[2008](https://arxiv.org/html/2605.28824#bib.bib9)\), yielding a proper probability distribution over surface forms\. Unlike winner\-take\-all OT, MaxEnt assigns gradient acceptability to candidates and supports intrinsic variation, making it particularly well\-suited for modelling probabilistic phonotactics and lexical frequency distributions\.

### 3\.4Semantic Ontology and Meaning Space

We construct a hierarchical ontology graph in which each node carries a canonical identifier and a human\-readable gloss, with parent–child edges encoding hyponymic relations\. The meaning space is derived by merging the Leipzig–Jakarta and Swadesh lists, producing a typologically motivated core vocabulary\. Lexicalization operates exclusively at leaf nodes, preventing semantic collapsing\. Leaf concepts are sampled to match the target lexicon size, weighted by pre\-leaf parent category to ensure breadth across the ontology\.

### 3\.5Semantic Assignment

GivenNNselected meaningsℳ=\{m1,…,mN\}\\mathcal\{M\}=\\\{m\_\{1\},\\ldots,m\_\{N\}\\\}andNNgenerated word forms𝒲=\{w1,…,wN\}\\mathcal\{W\}=\\\{w\_\{1\},\\ldots,w\_\{N\}\\\}, the module seeks a bijectionσ:ℳ→𝒲\\sigma:\\mathcal\{M\}\\rightarrow\\mathcal\{W\}such that phonological similarity tracks semantic similarity — a form–meaning alignment objective grounded in lexical iconicity\(Dingemanse et al\.,[2015](https://arxiv.org/html/2605.28824#bib.bib6)\)\.

##### Semantic distance\.

Distance between meanings is computed as tree path distance over the ontology:

dsem​\(mi,mj\)=depth​\(mi\)\+depth​\(mj\)−2⋅depth​\(lca​\(mi,mj\)\),d\_\{\\mathrm\{sem\}\}\(m\_\{i\},m\_\{j\}\)=\\mathrm\{depth\}\(m\_\{i\}\)\+\\mathrm\{depth\}\(m\_\{j\}\)\-2\\cdot\\mathrm\{depth\}\\\!\\left\(\\mathrm\{lca\}\(m\_\{i\},m\_\{j\}\)\\right\),\(1\)wherelca​\(mi,mj\)\\mathrm\{lca\}\(m\_\{i\},m\_\{j\}\)denotes their lowest common ancestor\.

##### Form distance\.

Phonological distance is measured using Levenshtein edit distancedform​\(wi,wj\)d\_\{\\mathrm\{form\}\}\(w\_\{i\},w\_\{j\}\), a language\-agnostic and computationally tractable proxy for phonological similarity\.

##### Alignment objective and optimisation\.

Mapping quality is assessed by the Spearman rank correlation between semantic and form distance vectors over all meaning pairs:

𝒮​\(σ\)=ρs​\(\[dsem​\(mi,mj\)\]i<j,\[dform​\(σ​\(mi\),σ​\(mj\)\)\]i<j\)\.\\mathcal\{S\}\(\\sigma\)=\\rho\_\{s\}\\\!\\left\(\\bigl\[d\_\{\\mathrm\{sem\}\}\(m\_\{i\},m\_\{j\}\)\\bigr\]\_\{i<j\},\\;\\bigl\[d\_\{\\mathrm\{form\}\}\(\\sigma\(m\_\{i\}\),\\sigma\(m\_\{j\}\)\)\\bigr\]\_\{i<j\}\\right\)\.\(2\)Since exhaustive search over allN\!N\!bijections is intractable, we employ hill\-climbing with random restarts: each restart proposes random pairwise swapsσ​\(mi\)↔σ​\(mj\)\\sigma\(m\_\{i\}\)\\leftrightarrow\\sigma\(m\_\{j\}\), retaining only improvements to𝒮\\mathcal\{S\}\. The dominant cost isO​\(N2\)O\(N^\{2\}\)per iteration, tractable for the lexicon sizes considered\. The final lexicon is stored as a JSON mapping from canonical semantic identifiers to surface word forms\.

## 4Evaluation

### 4\.1Phonotactic Well\-Formedness

Phonotactic well\-formedness is evaluated using a smoothed character\-levelnn\-gram language model trained on each generated lexicon\. For a lexicon of sizeNN, a model is trained on the word forms produced by the corresponding generator and evaluated on a held\-out sample\. We report two metrics:*perplexity*\(lower is better\) and*average log\-likelihood*\(higher, i\.e\., less negative, is better\)\. Both metrics are computed as a function of lexicon size, ranging from100100to50005000word forms, to assess how structural coherence scales with vocabulary size\. In addition, we report an improvement ratio for each constrained model relative to the random baseline, providing a normalised measure of the benefit conferred by grammatical constraint enforcement\.

### 4\.2Typological Realism

Typological realism is evaluated by measuring the divergence between the phoneme frequency distribution of a generated lexicon and the global phoneme frequency distribution derived from PHOIBLE\(Moran & McCloy,[2019](https://arxiv.org/html/2605.28824#bib.bib12)\)\. Concretely, each generated lexicon is tokenised into its constituent phonemes, and the resulting unigram distributionPPis compared against the PHOIBLE reference distributionQQusing*Kullback–Leibler \(KL\) divergence*DKL​\(P∥Q\)D\_\{\\mathrm\{KL\}\}\(P\\,\\\|\\,Q\)\(lower is better\)\. KL divergence is computed across lexicon sizes from100100to50005000, providing a trajectory of typological alignment as vocabulary size grows\.

### 4\.3Cross\-Grammar Compatibility

To assess the distributional relationships between phonological formalisms, we construct a cross\-grammar evaluation matrix\. For each grammarg∈\{deterministic, OT, MaxEnt, random\}g\\in\\\{\\text\{deterministic, OT, MaxEnt, random\}\\\}, a character\-level neural language model is trained on a lexicon generated byggand evaluated on lexicons generated by every other grammar\. This yields a4×44\\times 4matrix of average log\-likelihood and perplexity values, where entry\(gtrain,gtest\)\(g\_\{\\text\{train\}\},g\_\{\\text\{test\}\}\)reflects how well the phonotactic distribution learned fromgtraing\_\{\\text\{train\}\}explains word forms generated bygtestg\_\{\\text\{test\}\}\. Diagonal entries capture within\-grammar fit; off\-diagonal entries reveal cross\-grammar transferability\. All models are trained and evaluated at a fixed lexicon size ofN=5000N=5000\.

## 5Results

### 5\.1Phonotactic Well\-Formedness

![Refer to caption](https://arxiv.org/html/2605.28824v1/perplexity_vs_lexicon_size.png)\(a\)Character\-level perplexity \(lower is better\)\.
![Refer to caption](https://arxiv.org/html/2605.28824v1/avg_log_likelihood_vs_lexicon_size.png)\(b\)Average log\-likelihood \(higher is better\)\.

Figure 2:Phonotactic well\-formedness as a function of lexicon size\. OT and MaxEnt consistently outperform the deterministic and random baselines across all lexicon sizes on both metrics\.Figure[2](https://arxiv.org/html/2605.28824#S5.F2)reveals that probabilistic constraint evaluation produces measurably more learnable phonotactic structure than categorical filtering\. OT and MaxEnt converge to substantially lower perplexity \(≈5\.2\{\\approx\}5\.2and5\.45\.4respectively atN=5000N=5000\) than the deterministic generator \(≈7\.4\{\\approx\}7\.4\), which in turn outperforms the random baseline \(≈8\.6\{\\approx\}8\.6\)\. The consistent ranking across both metrics — and its stability over the full100100–50005000size range — indicates that this is a property of the grammars themselves rather than a sample\-size artifact\. The close alignment of OT and MaxEnt suggests that probabilistic constraint weighting, whether winner\-take\-all or distributional, produces similarly coherent phonotactic regularities; the deterministic generator’s gap reflects the cost of hard\-filtering, which eliminates consistent structural patterns rather than penalising them probabilistically\.

### 5\.2Typological Realism

![Refer to caption](https://arxiv.org/html/2605.28824v1/typological_kl_divergence_vs_lexicon_size.png)Figure 3:KL divergenceDKL​\(P∥Q\)D\_\{\\mathrm\{KL\}\}\(P\\,\\\|\\,Q\)between the generated phoneme distribution and the PHOIBLE global reference, as a function of lexicon size\. Lower values indicate greater typological realism\.Figure[3](https://arxiv.org/html/2605.28824#S5.F3)shows an unexpected dissociation from the phonotactic results: the deterministic generator, which performed second\-best on well\-formedness, exhibits the*highest*KL divergence at convergence \(≈1\.915\{\\approx\}1\.915\), while MaxEnt converges alongside the random baseline \(≈1\.852\{\\approx\}1\.852–1\.8551\.855\)\. This dissociation arises from a fundamental difference between the two evaluation axes\. Phonotactic well\-formedness rewards internal consistency — the degree to which word forms obey the same structural patterns — whereas typological realism rewards distributional breadth\. Hard\-filtering in the deterministic generator enforces consistency by suppressing entire syllable structures, but this categorical exclusion systematically under\-samples certain phoneme types, skewing the generated distribution away from the cross\-linguistic reference\. MaxEnt, by contrast, samples probabilistically over the full candidate space, allowing rare phoneme contexts to surface occasionally and incidentally recovering a distribution close to PHOIBLE’s\. That the random baseline matches MaxEnt here is not a failure of MaxEnt but a ceiling effect: once constraint enforcement is sufficiently gradient, typological alignment is largely determined by the inventory composition rather than the grammar\.

### 5\.3Cross\-Grammar Compatibility

![Refer to caption](https://arxiv.org/html/2605.28824v1/neural_phonotactic_loglik_heatmap.png)\(a\)Average log\-likelihood \(higher is better\)\.
![Refer to caption](https://arxiv.org/html/2605.28824v1/neural_phonotactic_perplexity_heatmap.png)\(b\)Perplexity \(lower is better\)\.

Figure 4:Cross\-grammar evaluation matrices\. Rows denote the training grammar; columns denote the test grammar\. Diagonal entries reflect within\-grammar fit; off\-diagonal entries reveal distributional transferability across formalisms\.##### OT and MaxEnt are distributionally equivalent\.

The OT–MaxEnt and MaxEnt–OT off\-diagonal entries \(−1\.75\-1\.75and−1\.79\-1\.79log\-likelihood;5\.785\.78and5\.965\.96perplexity\) are nearly indistinguishable from the respective diagonal entries \(−1\.74\-1\.74and−1\.74\-1\.74;5\.695\.69and5\.705\.70\)\. A model trained on OT lexicons explains MaxEnt lexicons almost as well as it explains its own — and vice versa\. This near\-symmetry indicates that OT and MaxEnt induce effectively the same phonotactic distribution, consistent with the theoretical relationship between the two: MaxEnt is a probabilistic relaxation of the same weighted constraint evaluation that underlies OT\.

##### Deterministic is a subset of the probabilistic space\.

The deterministic row shows moderate cross\-grammar log\-likelihoods on OT and MaxEnt test sets \(−1\.98\-1\.98and−1\.91\-1\.91\), indicating that a model trained on deterministic lexicons partially transfers to probabilistic ones\. The converse is markedly asymmetric: OT and MaxEnt models assign substantially lower likelihood to deterministic test lexicons \(−2\.31\-2\.31and−2\.23\-2\.23\)\. This directional asymmetry implies that the deterministic grammar generates a restricted subspace of the phonotactic patterns covered by probabilistic grammars — its word forms are partially explained by probabilistic models, but probabilistic word forms fall outside what the deterministic model has learned\.

##### Random lexicons are the universal outlier\.

All trained models assign their worst likelihoods to random test lexicons, and the random\-trained model transfers poorly to all grammars\. Crucially, the random model’s diagonal entry \(−2\.05\-2\.05perplexity7\.747\.74\) is substantially worse than even the deterministic diagonal, confirming that constraint enforcement — even categorical — produces genuinely learnable structure absent from unconstrained generation\.

## Acknowledgments

The authors acknowledge the use of AI tools such as ChatGPT, Claude, and Gemini for improving the presentation and grammar of this paper\. All the results, analysis, and proposed techniques remain a concrete representation of the author’s contributions\. The authors take full responsibility for the contents in this paper\.

## References

- Alper et al\. \(2025\)Morris Alper, Moran Yanuka, Raja Giryes, and Gašper Beguš\.ConlangCrafter: Constructing languages with a multi\-hop LLM pipeline\.*arXiv preprint arXiv:2508\.06094*, 2025\.
- Boersma \(1997\)Paul Boersma\.How we learn variation, optionality, and probability\.*Proceedings of the Institute of Phonetic Sciences of the University of Amsterdam*, 21:43–58, 1997\.
- Cai & Martens \(2023\)Angela Cai and Chris Martens\.Towards procedural generation of constructed languages for games\.In*Proceedings of the Workshop on Experimental AI in Games \(EXAG\)*, 2023\.
- Chomsky & Halle \(1968\)Noam Chomsky and Morris Halle\.*The Sound Pattern of English*\.Harper & Row, New York, 1968\.
- Clements \(1990\)George N\. Clements\.The role of the sonority cycle in core syllabification\.In John Kingston and Mary E\. Beckman \(eds\.\),*Papers in Laboratory Phonology I: Between the Grammar and Physics of Speech*, pp\. 283–333\. Cambridge University Press, Cambridge, 1990\.
- Dingemanse et al\. \(2015\)Mark Dingemanse, Damián E\. Blasi, Gary Lupyan, Morten H\. Christiansen, and Padraic Monaghan\.Arbitrariness, iconicity, and systematicity in language\.*Trends in Cognitive Sciences*, 19\(10\):603–615, 2015\.
- Futrell et al\. \(2017\)Richard Futrell, Adam Albright, Peter Graff, and Timothy J\. O’Donnell\.A generative model of phonotactics\.*Transactions of the Association for Computational Linguistics*, 5:73–86, 2017\.
- Goldwater & Johnson \(2003\)Sharon Goldwater and Mark Johnson\.Learning OT constraint rankings using a maximum entropy model\.In*Proceedings of the Stockholm Workshop on Variation within Optimality Theory*, pp\. 111–120, 2003\.
- Hayes & Wilson \(2008\)Bruce Hayes and Colin Wilson\.A maximum entropy model of phonotactics and phonotactic learning\.*Linguistic Inquiry*, 39\(3\):379–440, 2008\.
- Heyer \(2021\)Fabian Heyer\.Generating immersive conlangs\.Master’s thesis, University of Tübingen, 2021\.
- Legendre et al\. \(1990\)Géraldine Legendre, Yoshiro Miyata, and Paul Smolensky\.Harmonic grammar: A formal multi\-level connectionist theory of linguistic well\-formedness — theoretical foundations\.In*Proceedings of the 12th Annual Conference of the Cognitive Science Society*, pp\. 388–395\. Lawrence Erlbaum, Hillsdale, NJ, 1990\.
- Moran & McCloy \(2019\)Steven Moran and Daniel McCloy\.*PHOIBLE 2\.0*\.Oxford University Press, Oxford, 2019\.
- Pimentel et al\. \(2020\)Tiago Pimentel, Brian Roark, and Ryan Cotterell\.Phonotactic complexity and its trade\-offs\.In*Transactions of the Association for Computational Linguistics*, volume 8, pp\. 1–18, 2020\.
- Prince & Smolensky \(2004\)Alan Prince and Paul Smolensky\.*Optimality Theory: Constraint Interaction in Generative Grammar*\.Blackwell, Malden, MA, 2004\.Originally circulated as 1993 Rutgers University Center for Cognitive Science Technical Report 2\.
- Smolensky & Legendre \(2006\)Paul Smolensky and Géraldine Legendre\.*The Harmonic Mind: From Neural Computation to Optimality\-Theoretic Grammar*\.MIT Press, Cambridge, MA, 2006\.
- Taguchi & Sproat \(2025\)Chihiro Taguchi and Richard Sproat\.IASC: Interactive agentic system for ConLangs\.*arXiv preprint arXiv:2510\.07591*, 2025\.

## Appendix ATypological Universals Evaluated in the Phoneme Sampler

The phoneme inventory sampler incorporates three categories of typological tendencies, each validated over PHOIBLE inventories\(Moran & McCloy,[2019](https://arxiv.org/html/2605.28824#bib.bib12)\)prior to integration\. Correlational tendencies inform the probability of marked segment inclusion as a function of inventory size; co\-occurrence tendencies constrain which feature combinations are permitted; and implicational universals enforce directional dependencies between features, either categorically \(P=1\.0P=1\.0\) or probabilistically where near\-universal violations are attested\.

##### Correlational tendencies \(Pearsonrr\)\.

Symmetric associations between inventory size and marked segment presence, measured using Pearson correlation\.

- •Inventory size↔\\leftrightarrowpresence of clicks
- •Consonant count↔\\leftrightarrowpresence of ejectives
- •Consonant count↔\\leftrightarrowpresence of clicks
- •Vowel count↔\\leftrightarrowpresence of vowel length contrast

##### Co\-occurrence tendencies \(chi\-squareχ2\\chi^\{2\}\)\.

Symmetric associations between pairs of binary phonological features, measured using chi\-square tests of independence\. Values ofχ2=0\\chi^\{2\}=0reflect near\-deterministic contingency structure rather than independence\.

- •Ejective consonants↔\\leftrightarrowuvular consonants
- •Pharyngeal consonants↔\\leftrightarrowuvular consonants
- •Nasal vowels↔\\leftrightarroworal vowels
- •Voiced obstruents↔\\leftrightarrowvoiceless obstruents
- •Fricatives↔\\leftrightarrowstops
- •Front rounded vowels↔\\leftrightarrowfront unrounded vowels

##### Implicational universals \(conditional probabilityP​\(Y∣X\)P\(Y\\mid X\)\)\.

Directional dependencies of the formX⇒YX\\Rightarrow Y, measured as conditional probabilities over inventories containingXX\. Universals withP=1\.0P=1\.0are enforced deterministically in the sampler; the one near\-universal \(P=0\.997P=0\.997\) is enforced probabilistically\.

- •Pharyngeal consonants⇒\\Rightarrowuvular consonants
- •Nasal vowels⇒\\Rightarroworal vowels
- •Voiced obstruents⇒\\Rightarrowvoiceless obstruents \(P=0\.997P=0\.997, near\-universal\)
- •Fricatives⇒\\Rightarrowstops
- •Front rounded vowels⇒\\Rightarrowfront unrounded vowels

## Appendix BCode and Reproducibility

All code, evaluation scripts, generated lexicons, and experimental results reported in this paper are publicly available at:

[https://anonymous\.4open\.science/r/lexicon\-generation/](https://anonymous.4open.science/r/lexicon-generation/)

The repository includes the phoneme inventory sampler, all four phonological grammar implementations \(deterministic, OT, HG, and MaxEnt\), the semantic assignment module, and the phonotactic and typological evaluation pipelines described in Sections[3](https://arxiv.org/html/2605.28824#S3)and[4](https://arxiv.org/html/2605.28824#S4)\.