Does Morphological Correctness Matter for Tokenization? A Scrambled Control Study in Tagalog

# Does Morphological Correctness Matter for Tokenization? A Scrambled Control Study in Tagalog
Source: [https://arxiv.org/html/2610.20859](https://arxiv.org/html/2610.20859)
Abdul Rehman AntallBaidya Nath SahaAffiliation:Concordia University of Edmonton, Alberta, Canadal230605@lhr\.nu\.edu\.pkbaidya\.saha@concordia\.ab\.ca

###### Abstract

Typology\-aware tokenizers are motivated by the claim that segmentation which respects morpheme boundaries preserves information that frequency\-driven segmentation destroys\. Studies routinely establish that it helps while assuming the boundaries must be linguistically correct\. We separate them\. Tagalog marks voice with infixes \(s\-um\-ulat‘wrote’,s\-in\-ulat‘was written’\), so the root is not contiguous and no sequence of BPE merges can recover it\. We build MorphAustro\-Tok, a tokenizer that emits the root as a contiguous unit followed by a positional infix token, and compare it against a corpus\-matched BPE baseline and, at two of three fertility levels, against a scrambled\-morpheme control that preserves morph count, invertibility and per\-type consistency while randomising the pseudo\-infix\. Across 33 runs, MorphAustro\-Tok improves Tagalog machine translation over matched BPE by 0\.84, 1\.49 and 3\.40 mean sentence\-level chrF\+\+ points\. Token count does not explain the gain, since it grows as segmentation coarsens, and neither does capacity: at the coarsest level the scrambled arm carries more parameters and scores lower\. Scrambling the analysis moves the gain by less than between\-seed dispersion at both controlled levels, so we detect no contribution from morphological correctness at this scale\.

## 1Introduction

The case for typology\-aware tokenization rests on an irreversibility argument\. A tokenizer fixes the alphabet in which a model perceives its input, and a contrast destroyed by segmentation cannot be recovered by any amount of downstream capacity\. Where a language marks grammatical function by means that frequency\-driven merges systematically fragment, the argument predicts a cost that no encoder can repay\([Bostrom and Durrett, 2020](https://arxiv.org/html/2610.20859#bib.bib4);[Park et al\., 2021](https://arxiv.org/html/2610.20859#bib.bib8)\)\.

Tagalog is close to the strongest available test of that argument in a Latin\-script language\. Voice is marked by the infixes\-um\-\(actor voice\) and\-in\-\(patient voice\), inserted after the root\-initial onset, so thatsulat‘write’ becomess\-um\-ulat‘wrote’ ands\-in\-ulat‘was written’\([Schachter and Otanes, 1972](https://arxiv.org/html/2610.20859#bib.bib26);[Kroeger, 1993](https://arxiv.org/html/2610.20859#bib.bib27)\)\. The root is discontinuous in the surface string\. A BPE merge joins adjacent symbols\([Sennrich et al\., 2016](https://arxiv.org/html/2610.20859#bib.bib1)\), so no merge sequence can representsulatas a unit insidesumulat, and the paradigm fragments into unrelated whole\-word types\. This is the same class of problem that root\-and\-pattern morphology poses for Semitic, arriving inside an otherwise ordinary Latin\-script word\.

If the irreversibility argument is right, a tokenizer that recovers the root should help\. Our results say that it does, and they also say that the recovery does not have to be correct\.

That distinction is the contribution of this paper\. A tokenizer intervention has at least two separable properties: it changeswherethe cuts fall, and it changeshow consistentlya given word type is cut\. A design that varies both at once, as ours does and as the interventions in this literature generally do, cannot attribute an observed gain to either property\. A control that holds granularity and consistency fixed while randomising linguistic content can at least remove one of them from the explanation, and running that control is what separates a positive result from an attributable one\.111Code, per\-run records and the gate log are available at[https://github\.com/AbdulRehmanAntall/morphaustro\-tok](https://github.com/AbdulRehmanAntall/morphaustro-tok)\.

#### Contributions\.

We state up front what level of evidence supports each claim, since the claims differ substantially in strength\.

1. 1\.MorphAustro\-Tok, a two\-stage tokenizer in which an infix\-aware pre\-segmentation emits the Tagalog root as a contiguous unit followed by a positional infix token, and a constrained BPE learns merges that cannot cross a morph boundary \(Algorithm[1](https://arxiv.org/html/2610.20859#alg1)\)\.
2. 2\.Ascrambled\-morpheme controlthat fires on exactly the word types the real analyzer fires on and preserves morph count, invertibility and per\-type consistency while destroying the intended morphological alignment \(Algorithm[2](https://arxiv.org/html/2610.20859#alg2)\)\.
3. 3\.Evidence that the machine\-translation gain over matched BPE isnot explained by token count or by vocabulary\-induced capacity: it grows monotonically as segmentation coarsens, and at the coarsest level the arm with more parameters scores lower \(§[5\.4](https://arxiv.org/html/2610.20859#S5.SS4)\)\.
4. 4\.No detectable contribution from morphological correctnessat this scale\. Against the scrambled control the difference is−0\.055\-0\.055chrF\+\+ at the central fertility level and\+0\.661\+0\.661at the coarsest, opposite in sign and both far below between\-seed dispersion \(§[5\.3](https://arxiv.org/html/2610.20859#S5.SS3)\)\.
5. 5\.One asymmetric result that survives both uncertainty floors: on named entity recognition, scramblingcosts0\.023 F1 relative to the real analysis, while the real analysis does not beat BPE \(§[5\.6](https://arxiv.org/html/2610.20859#S5.SS6)\)\.

We do not claim that morphological correctness is inert, only that at this scale, with these tasks, we could not detect a contribution from it at the resolution our example\-level and seed\-level variation allows\.

## 2Related work

#### The tokenizer stage\.

Subword segmentation determines what a model can represent, and tokenizer quality tracks downstream quality\([Rust et al\., 2021](https://arxiv.org/html/2610.20859#bib.bib5);[Gowda and May, 2020](https://arxiv.org/html/2610.20859#bib.bib9)\)\. Work on morphologically rich languages has reported that morphology\-aware segmentation improves downstream results and has attributed the improvement to morphological validity\([Bostrom and Durrett, 2020](https://arxiv.org/html/2610.20859#bib.bib4);[Hofmann et al\., 2022](https://arxiv.org/html/2610.20859#bib.bib6);[Park et al\., 2021](https://arxiv.org/html/2610.20859#bib.bib8)\), rarely testing that attribution against a granularity\-matched control\. A tokenizer that changes segmentationconsistencyalongsidecorrectnessleaves the two confounded, and[Schmidt et al\. \(2024\)](https://arxiv.org/html/2610.20859#bib.bib7)make the related point that compression alone does not explain what tokenizers contribute\. We take that argument further by removing correctness while holding consistency and granularity fixed\.

#### Infixation and symmetrical voice in Philippine languages\.

Tagalog exhibits a symmetrical voice system in which the semantic role of the pivot argument is determined by verbal morphology\([Schachter and Otanes, 1972](https://arxiv.org/html/2610.20859#bib.bib26);[Kroeger, 1993](https://arxiv.org/html/2610.20859#bib.bib27);[Himmelmann, 2005](https://arxiv.org/html/2610.20859#bib.bib28)\)\. Two of the productive markers are infixes\. Because an infix is inserted inside the root rather than concatenated to it, the root is not a contiguous substring, and the standard subword machinery, which operates by merging adjacent units, is structurally unable to isolate it\. Unsupervised segmenters such as Morfessor\([Smit et al\., 2014](https://arxiv.org/html/2610.20859#bib.bib10)\)inherit the restriction, since they too model words as concatenations of contiguous morphs\.

#### Numeral classifiers in Malayic\.

Indonesian requires a classifier between a numeral and a counted noun \(tiga ekor kucing‘three cats’\), and its most frequent realisation fuses the numeral ‘one’ with the classifier into one orthographic word \(seekor kucing‘one cat’\)\([Sneddon et al\., 2010](https://arxiv.org/html/2610.20859#bib.bib29)\)\.

#### Evaluation resources\.

FLORES\+ provides parallel evaluation across Austronesian varieties\([Goyal et al\., 2022](https://arxiv.org/html/2610.20859#bib.bib14);[NLLB Team, 2024](https://arxiv.org/html/2610.20859#bib.bib15)\), and WikiANN provides named\-entity annotation\([Pan et al\., 2017](https://arxiv.org/html/2610.20859#bib.bib16);[Rahimi et al\., 2019](https://arxiv.org/html/2610.20859#bib.bib17)\)\. Training bitext for Tagalog is scarce\([Baliber et al\., 2020](https://arxiv.org/html/2610.20859#bib.bib23);[Lovenia et al\., 2024](https://arxiv.org/html/2610.20859#bib.bib22)\): OPUS\-100\([Zhang et al\., 2020](https://arxiv.org/html/2610.20859#bib.bib13)\)contains no Tagalog configuration, and the mined NLLB corpus is distributed in a form no longer loadable by current tooling\. We therefore fetch ParaCrawl v9\([Bañón et al\., 2020](https://arxiv.org/html/2610.20859#bib.bib11)\)directly from OPUS\([Tiedemann, 2012](https://arxiv.org/html/2610.20859#bib.bib12)\), which keeps the two arms on a common source\.

## 3Methodology

### 3\.1Hypotheses and the decisive criterion

We test one hypothesis and one anti\-hypothesis, both fixed before the runs were executed\. H1 states that a morpheme\-boundary constraint on BPE improves downstream quality for Tagalog relative to a corpus\-matched BPE baseline\. The anti\-hypothesis states that any observed gain reduces to compression and disappears once tokens per character are held equal\. A decisive criterion was fixed alongside them: if performance survives a scramble of the morphological analysis, correctness of the analysis is not what produced the gain\. The Indonesian classifier arm is an exploratory feasibility check rather than a hypothesis test \(§[4\.4](https://arxiv.org/html/2610.20859#S4.SS4)\)\. The study was not preregistered, and we use “fixed before the runs” in that narrower sense\.

### 3\.2MorphAustro\-Tok

Algorithm 1Stage\-1 segmentation of one word1:word

ww; root\-evidence set

EE
2:

w←nfc​\(w\)w\\leftarrow\\textsc\{nfc\}\(w\)⊳\\trianglerightbefore any offset is computed

3:for

o←1o\\leftarrow 1to

22do⊳\\trianglerightshortest onset wins

4:if

w\[0:o\]w\[0\{:\}o\]contains a vowelthen

5:continue⊳\\trianglerightum\-prefixes vowel\-initial roots

6:endif

7:for

i∈\{um,in\}i\\in\\\{\\texttt\{um\},\\texttt\{in\}\\\}do

8:if

w\[o:o\+2\]=iw\[o\{:\}o\{\+\}2\]=iand

\|w\|≥o\+\|i\|\+3\|w\|\\geq o\+\|i\|\+3then

9:

ρ←w\[0:o\]⋅w\[o\+\|i\|:\]\\rho\\leftarrow w\[0\{:\}o\]\\cdot w\[o\{\+\}\|i\|\{:\}\]
10:if

ρ∈E\\rho\\in Ethen

11:return

\[ρ,<INF:i@o\>\]\[\\,\\rho,\\ \\texttt\{<INF:\}i\\texttt\{@\}o\\texttt\{\>\}\\,\]
12:endif

13:endif

14:endfor

15:endfor

16:returnconcatenative strip, or

\[w\]\[w\]if none applies

The tokenizer has two stages, of which the first is the intervention\.

#### Stage 1: morphological pre\-segmentation\.

A surface word maps to a sequence of morphs by Algorithm[1](https://arxiv.org/html/2610.20859#alg1)\. For an infixed form the root is emitted as acontiguousunit followed by a positional infix token, so thatsumulatbecomes\["sulat", "<INF:um@1\>"\]\. The root is now contiguous in the token stream although discontinuous in the string, which is precisely what no sequence of BPE merges achieves\. Root\-then\-infix ordering keeps the word\-boundary marker on the root and makes voice a discrete decision the decoder takes after the root\. For Indonesian, aNUM CLF Nor fusedseCLF Nsequence binds classifier and noun into a single span morph\.

#### Stage 2: constrained BPE over morphs\.

Merges are learned by frequencywithinmorphs and can never cross a morph boundary, because the trainer sees morphs as pre\-tokenized units\. Granularity therefore remains learned and corpus\-driven; only the positions at which cuts may fall are constrained\.

#### Invertibility\.

Exact reassembly is a hard requirement, since decoding is meaningless without it, and the positional offset in the infix token is what makes it exact\. Section[4\.3](https://arxiv.org/html/2610.20859#S4.SS3)reports a defect this requirement caught\.

### 3\.3The matched baseline

The baseline is BPE trained on thesamecorpus lines, with thesameword\-boundary convention and normalisation and its vocabulary size chosen to hit the same fertility target, implemented over the same tokenizer back end\([Kudo and Richardson, 2018](https://arxiv.org/html/2610.20859#bib.bib3)\)\. It is not an off\-the\-shelf tokenizer: whether a merge respects a morpheme boundary is decided by corpus frequency alone, which is the failure mode under test\.

### 3\.4The scrambled\-morpheme control

Algorithm 2The scrambled control, for a word the analyzer accepted1:

ww; true analysis

\(i⋆,o⋆\)\(i^\{\\star\},o^\{\\star\}\); per\-type cache

κ\\kappa
2:if

w∈κw\\in\\kappathen

3:return

κ⁡\[w\]\\kappa\[w\]⊳\\trianglerightone assignment per type, corpus\-wide

4:endif

5:

𝒜←\{\(o,k\):o,k∈\{1,2,3\},\|w\|≥o\+k\+3,\(o,k\)≠\(o⋆,\|i⋆\|\)\}\\mathcal\{A\}\\leftarrow\\\{\(o,k\):o,k\\in\\\{1,2,3\\\},\\ \|w\|\\geq o\+k\+3,\\ \(o,k\)\\neq\(o^\{\\star\},\|i^\{\\star\}\|\)\\\}
6:if

𝒜=∅\\mathcal\{A\}=\\emptysetthen

7:

κ⁡\[w\]←\(i⋆,o⋆\)\\kappa\[w\]\\leftarrow\(i^\{\\star\},o^\{\\star\}\)⊳\\trianglerighttoo short to derange

8:else

9:

\(o,k\)∼Uniform⁡\(𝒜\)\(o,k\)\\sim\\mathrm\{Uniform\}\(\\mathcal\{A\}\)
10:

κ\[w\]←\(w\[o:o\+k\],o\)\\kappa\[w\]\\leftarrow\(w\[o\{:\}o\{\+\}k\],\\ o\)
11:endif

12:return

\[w\[0:o\]⋅w\[o\+k:\],<INF:w\[o:o\+k\]@o\>\]\[\\,w\[0\{:\}o\]\\cdot w\[o\{\+\}k\{:\}\],\\ \\texttt\{<INF:\}w\[o\{:\}o\{\+\}k\]\\texttt\{@\}o\\texttt\{\>\}\\,\]

The control fires onexactlythe word types the real analyzer fires on and always emits exactly two morphs, but it excises a pseudo\-infix at an \(offset, length\) pair guaranteed to differ from the true analysis \(Algorithm[2](https://arxiv.org/html/2610.20859#alg2)\)\. The assignment is cached per type, so it is deterministic given the seed and stable across the corpus\. Both arms use the same positional marker schema,<INF:string@offset\>, so the control introduces no marker category of its own, although the string a given marker carries differs between the arms\.

#### Worked example\.

Forsumulat,𝒜\\mathcal\{A\}contains\(1,1\),\(1,3\),\(2,1\),\(2,2\)\(1,1\),\(1,3\),\(2,1\),\(2,2\)and\(3,1\)\(3,1\), giving the residuessmulat,slat,suulat,sulatandsumlat\. Forsinulatthe same offsets givesnulat,slat,siulat,silatandsinlat\. The two residue sets meet only atslat, so of the 25 joint assignments exactly one lets the paradigm share a unit, and that unit is not the root\. Forkumain/kinainthe residues are disjoint, so no assignment collapses the paradigm\.

The residue is therefore not a real root, and paradigm members stop collapsing onto a shared unit\. What the control preserves is granularity \(morph counts are identical line by line, verified in Appendix[D](https://arxiv.org/html/2610.20859#A4)\), invertibility, and per\-type consistency of the decomposition\. What it destroys is the intended morphological alignment: the substring excised, its length and offset, and the residue all change, so the control is not a correctness\-only manipulation\.

### 3\.5What is tokenized, and where vocabulary parameters enter

A single tokenizer is trained over both sides of the bitext and shared by the encoder and the decoder; the embedding matrix is shared and the output projection tied to it, so vocabulary parameters enter the count exactly once\. Stage\-1 segmentation therefore applies to the English source as well as the target, but the root evidence that licenses an infix analysis is induced from target\-side text alone, so an English word is decomposed only if its residue is an attested Tagalog type\. Vocabulary matching refers to this single joint vocabulary\.

### 3\.6Fertility matching, and what it costs

A typology\-aware tokenizer changes how fine it cuts and where it cuts at the same time, and only a fertility\-matched comparison separates the two\. We match on tokens per character rather than tokens per word, because classifier\-span binding merges two whitespace words into one unit and would drive tokens per word below 1\.0 as a pure artifact\.

Target levels are derived from the measured intersection of each condition’s achievable envelope rather than fixed a priori, since a target reachable for one arm can be unreachable for another\. The derived levels are 0\.319, 0\.379 and 0\.438 tokens per character, and no cell deviates from its target by more than 0\.005, the tolerance the matching gate asserts\. Appendix[C](https://arxiv.org/html/2610.20859#A3)gives the achieved value and vocabulary size for every cell\.

Matching fertility forces different vocabulary sizes\. MorphAustro\-Tok requires 33\.2%, 19\.8% and 16\.3% more vocabulary entries than BPE to reach the same granularity at the three levels, since morph\-constrained merges buy less compression per entry\. We report this as a result in its own right \(§[5\.5](https://arxiv.org/html/2610.20859#S5.SS5)\) and treat the consequent parameter difference as a confound to quantify rather than a defect to forbid\.

### 3\.7Two sources of uncertainty, and the correction families

Model\-level uncertaintyis variation across seeds, reported as per\-condition means, standard deviations andtt\-intervals\. Atn=3n=3\(tcrit=4\.30t\_\{\\text\{crit\}\}=4\.30\) these intervals are wide, and we treat them as descriptive\.

Test\-example\-level uncertaintyis variation attributable to the evaluation sample with trained models held fixed, captured by a two\-sided paired bootstrap over matched test examples with 10,000 resamples\([Koehn, 2004](https://arxiv.org/html/2610.20859#bib.bib18);[Efron and Tibshirani, 1993](https://arxiv.org/html/2610.20859#bib.bib25)\)\. Per\-example scores are averaged across seeds before resampling, so the bootstrap measures example\-level variance and does not re\-count seed variance\.

Holm correction\([Holm, 1979](https://arxiv.org/html/2610.20859#bib.bib24)\)is applied within two families fixed in advance: the 22 contrasts of each non\-baseline condition against matched BPE, pooled across both languages, all tasks and all fertility levels; and, separately, the nine MorphAustro\-Tok versus scrambled contrasts\. Pooling the first family across tasks is conservative, since it corrects the MT contrasts for the NER and probe comparisons as well\.

A bootstrappp\-value here is a tail proportion over the 10,000 resamples, so the procedure resolves nothing below10−410^\{\-4\}before correction\. Contrasts whose uncorrected tail proportion is zero are reported as reaching that floor rather than with a more precise figure\.

Neither source of uncertainty subsumes the other and we never combine them\. A bootstrap interval excluding zero is not evidence that an effect would survive retraining\. We therefore apply a prespecified stability criterion, under which a claim about mechanism requires both example\-level significance and a seed\-level difference exceeding between\-seed dispersion\. One standard deviation over three seeds is a conservative heuristic rather than an inferential test\. Section[5\.2](https://arxiv.org/html/2610.20859#S5.SS2)reports a case of divergence between the two, and the divergence is itself the finding\.

## 4Implementation

### 4\.1Corpora and provenance

Training bitext is OPUS ParaCrawl v9 \(en\-tl,en\-id\), fetched from pinned versioned URLs with SHA\-256 recorded over the downloaded bytes; a failed fetch raises rather than substituting\. Filtering \(deduplication, length ratio, language identification\) retained 90\.0% of Tagalog and 70\.8% of Indonesian pairs and is applied identically to every condition\. Evaluation uses FLORES\+ \(fil\_Latn,ind\_Latn,eng\_Latn\) and WikiANN\. Translation quality is measured with chrF\+\+\([Popović, 2015](https://arxiv.org/html/2610.20859#bib.bib19);[Popović, 2017](https://arxiv.org/html/2610.20859#bib.bib20)\)\. Appendix[A](https://arxiv.org/html/2610.20859#A1)gives the training configuration and Appendix[B](https://arxiv.org/html/2610.20859#A2)the corpus sizes and the sources we rejected\.

#### Contamination\.

FLORES\+ test text was checked against training bitext by normalised exact match, character 5\-gram Jaccard, and containment\. Containment matters independently: a training sentence that is aprefixof a longer test sentence has low Jaccard while leaking entirely\. Measured contamination was 0\.0% on both languages by all three criteria\.

#### A register gap we could not close\.

FLORES\+ labels Tagalog as Filipino \(fil\_Latn\), the standardised national register, while ParaCrawl is labelled Tagalog\. Infixation is identical across both, so the mechanism is unaffected, but the train/test gap is large: type\-Jaccard is 0\.179, with 60\.9% of test types unseen in training\. It applies identically to all conditions and cannot manufacture a between\-condition difference, although it holds absolute scores at floor level\.

### 4\.2The mechanism is verifiably active

On held\-out infixed forms whose roots the corpus attests, segmentation recovers root and infix in 11 of 11 cases, and on a negative fixture of words that merely look infixed \(bumbilya‘light bulb’,lumpia‘spring roll’\) precision is 1\.000, so the analyzer does not fire on everything\. The mechanism fires on 33\.75% of Tagalog lines and 1\.63% of whitespace words, and MorphAustro\-Tok separates from matched BPE by 6\.94% in tokens per character\. Section[4\.4](https://arxiv.org/html/2610.20859#S4.SS4)gives the much lower Indonesian figures and explains why that arm is exploratory\.

### 4\.3The harness, and the defects it caught

The credibility of a control\-based null depends on the reader believing that the null was not produced by a defect\. Every validation criterion is runnable code that halts on failure, and Appendix[D](https://arxiv.org/html/2610.20859#A4)lists the gates and the defects they caught\. One defect shaped the design\. Stage 1 originally computed the infix offset on the raw surface string while the tokenizer normalised to NFC afterwards, so where a code point before the offset was decomposed, reassembly inserted the infix at the wrong position and produced plausible\-looking wrong words rather than errors\. The roundtrip gate caught it at a rate of 1 line in 2,000, and a fixture\-based check would not have\. Normalisation now precedes any offset computation, which is why it is the first line of Algorithm[1](https://arxiv.org/html/2610.20859#alg1)\.

### 4\.4Capability gating, and one arm we treat as exploratory

A component whose requirement is unmet is recorded asSKIPPEDwith a reason rather than run against a substitute\. Malay and Indonesian cannot be pooled, so Indonesian runs alone \(Appendix[E](https://arxiv.org/html/2610.20859#A5)\)\. Tagalog UD tagging is unusable, since UD\_Tagalog\-TRG contains 55 sentences and Ugnayan 94\([Nivre et al\., 2020](https://arxiv.org/html/2610.20859#bib.bib21)\); WikiANNtlis substituted, and the substitution is logged rather than silent\. The top rung of the supervision ladder does not exist either, because no maintained Philippine morphological analyzer is available off the shelf\. Rungs 1 \(corpus\-derived root evidence\) and 2 \(lexicon\) are implemented, and rung 1 alone attested 9 of 10 held\-out fixture roots\.

#### Indonesian is a feasibility arm\.

The classifier mechanism reaches 3\.95% of lines and 0\.27% of words, giving a fertility separation of 0\.167% against 6\.94% for Tagalog, which is too rare to move corpus\-level scores at any seed count this budget affords\. We therefore cut Indonesian MT and NER on measured evidence and kept only the classifier\-agreement probe, which is scored on classifier\-bearing sentences and so is unaffected by corpus\-level rarity\. Appendix[E](https://arxiv.org/html/2610.20859#A5)gives the detail\.

### 4\.5A circular probe, measured rather than assumed

The Tagalog voice probe is solvable by reading the<INF:…\>token alone: a trivial baseline that inspects only which infix token appears scores 1\.000 under MorphAustro\-Tok and 0\.000 under matched BPE\. Attempts to break the circularity all failed for one structural reason, since any corruption of the infix also changes whether the analyzer fires, which is itself visible in the token stream\. We therefore report the probe as diagnostic rather than evidential; it carries no evidential weight\. The Indonesian probe is not circular in the same way \(Appendix[E](https://arxiv.org/html/2610.20859#A5)\)\.

## 5Evaluation

### 5\.1Setup

Table 1:chrF\+\+ on FLORES\+ devtest, mean±\\pmsd over three seeds, with the paired\-bootstrap difference and its 95% interval\. All three reach the bootstrap resolution floor of §[3\.7](https://arxiv.org/html/2610.20859#S3.SS7)\.Three conditions \(MorphAustro\-Tok, matched BPE, scrambled control\) were run at three fertility levels with three seeds each\. The scrambled control is absent at the finest level, a budget\-driven omission\. The 33 runs are 24 Tagalog and 9 Indonesian\. A Tagalog run trains three separate models on one tokenizer build and one seed, an encoder–decoder for MT, an encoder for NER and an encoder for the voice probe, and an Indonesian run trains the classifier probe alone\. Tasks within a run share a configuration fingerprint, which is what makes the task\-transfer correlation well posed\.

Model capacity is restricted deliberately rather than by necessity\. At between 1\.12M and 1\.92M inference\-path parameters there is little room for a model to memorise its way around a segmentation that fragments a paradigm, so a difference between tokenizers surfaces in the score instead of being absorbed by capacity\. We therefore treat the setup as a glass\-box diagnostic rather than a competitive benchmark, and read the absolute scores as a floor\.

The in\-domain pool is a seed\-fixed random sample of the filtered bitext, drawn before the training split so the two are disjoint\. The scored subset is the firstNNsentences in corpus order \(500 of 1,012 FLORES\+ devtest; 300 of 2,000 in\-domain\), identical under every condition and seed, so the bootstrap pairing is exact\.

NER uses WikiANNtl\(10,000 train and 1,000 test sentences, seven IOB2 labels over PER, ORG and LOC\), scored with entity\-level precision, recall and F1 fromseqeval\. The tagger is a two\-layer Transformer encoder of the same geometry as the MT encoder, trained for 800 steps with a linear classification head\. A word’s representation is theunweighted meanof every subword piece that covers it, obtained by multiplying the encoder states by a row\-stochastic piece\-to\-word matrix; a word whose pieces are all truncated away has an all\-zero row and is masked out of the loss rather than scored\. Labels are never assigned to the first subtoken, which is not comparable across conditions\.

### 5\.2Machine translation against matched BPE

Table[1](https://arxiv.org/html/2610.20859#S5.T1)reports the comparison against the matched baseline\. MorphAustro\-Tok improves chrF\+\+ at all three levels, by 0\.844, 1\.488 and 3\.396 mean sentence\-level points, each at the bootstrap resolution floor, and the in\-domain ParaCrawl split reproduces the pattern with larger effects \(\+1\.168, \+1\.631, \+3\.933\)\. Two estimands appear in that table and they do not subtract\. The condition columns are corpus\-level scores averaged over seeds, whereasΔ\\Deltais the mean per\-sentence difference over the matched evaluation set, since a corpus aggregate cannot be resampled over examples\. The corpus\-level gaps are \+0\.913, \+1\.191 and \+3\.446\.

Seed dispersion qualifies two of the three\. At F0 and F1 the effect \(0\.844, 1\.488\) lies below the largest between\-seed standard deviation in the contrast \(2\.265, 2\.527\), and therefore inside retraining noise\. Only at F2 does the effect \(3\.396\) exceed the floor \(1\.577\)\. By the stability criterion of §[3\.7](https://arxiv.org/html/2610.20859#S3.SS7), F0 and F1 are example\-level results that we cannot claim would survive retraining\.

### 5\.3No detectable contribution from correct boundaries

Table 2:Mean per\-sentence chrF\+\+ differences over the matched evaluation set\. The last column is the pooled between\-seed standard deviation for the Morph versus Scrambled contrast\.Table[2](https://arxiv.org/html/2610.20859#S5.T2)reports the study’s principal finding\. At the central fertility level the scrambled control matches or exceeds the real analysis on both evaluation sets, and at the coarsest level the real analysis leads by 0\.661 chrF\+\+, which clears the example\-level floor but amounts to one third of between\-seed dispersion\. In the table, “at floor” marks a bootstrap tail proportion at the resolution limit of §[3\.7](https://arxiv.org/html/2610.20859#S3.SS7)\.

Two levels, opposite signs, both far below the retraining floor\. By the stability criterion of §[3\.7](https://arxiv.org/html/2610.20859#S3.SS7), a contribution from linguistic correctness is not established at either level\. That is an absence of detection rather than equivalence: we specified no margin within which the arms would count as equal, and a difference below our floor is not the same as no difference\.

### 5\.4Token count and capacity do not explain the gain

LevelCond\.VocabParamschrF\+\+F0BPE5,8421,673,98410\.44Morph7,7801,922,04811\.35F1BPE2,4511,239,9369\.29Morph2,9361,302,01610\.48Scr\.3,5671,382,78410\.62F2BPE1,4831,116,0327\.48Morph1,7251,147,00810\.93Scr\.1,8721,165,82410\.24Table 3:Vocabulary size, inference\-path parameters and chrF\+\+ for every Tagalog cell of the sweep\.#### Not token count alone\.

The effect grows as segmentation coarsens, from \+0\.84 to \+1\.49 to \+3\.40 across F0, F1 and F2\. Holding tokens per character equal does not remove the effect; it increases it\. The anti\-hypothesis, that any gain reduces to fewer tokens per character, is not supported\.

#### Not the capacity that fertility matching buys\.

Matched fertility forces MorphAustro\-Tok to a larger vocabulary and therefore to more embedding parameters, so the confound is real and we measured it \(Table[3](https://arxiv.org/html/2610.20859#S5.T3)\)\. Every count there is926,208\+128​\|V\|926\{,\}208\+128\\,\|V\|, because the non\-embedding block is identical across conditions and the embedding is counted once: source and target share one vocabulary and the output projection is tied to it\. The cleanest evidence is a single within\-level comparison rather than a trend across levels\. At F2 the scrambled arm carries 18,816 more parameters than MorphAustro\-Tok, exactly the 147 pseudo\-infix rows the scramble discovers, and scores 0\.69 chrF\+\+ lower, which is the sign a capacity account forbids\. Across levels the parameter advantage is largest where the gain is smallest, which points the same way\.

### 5\.5A vocabulary cost

MorphAustro\-Tok requires 33\.2%, 19\.8% and 16\.3% more vocabulary entries than matched BPE to reach the same tokens per character, because constrained merges cannot cross morph boundaries and each entry therefore buys less compression\. Typology\-aware segmentation is not free at matched granularity, and the cost is largest when compression is hardest\.

### 5\.6Named entity recognition

Table 4:WikiANN Tagalog entity\-level precision, recall and F1 \(seqeval\), mean±\\pmsd over three seeds\.Table[4](https://arxiv.org/html/2610.20859#S5.T4)gives the raw scores\. The scrambled arm is the only condition that falls below the baseline, and it does so on both precision and recall\. The paired\-bootstrap differences, which are again sentence\-level means and do not subtract from the corpus\-level scores, tell a more specific story\. At F1, MorphAustro\-Tok does not beat the baseline \(\+0\.013,pHolm=0\.541p\_\{\\text\{Holm\}\}=0\.541\) and the scrambled control falls below it \(−0\.010\-0\.010,pHolm=0\.919p\_\{\\text\{Holm\}\}=0\.919\), while the contrast between the two is \+0\.023 atpHolm=0\.0045p\_\{\\text\{Holm\}\}=0\.0045against a between\-seed pooled sd of 0\.011, the only non\-circular result in the study that clears both floors\.

The defensible statement is therefore that linguistically wrong morpheme boundaries damage sequence labelling, not that morphology\-aware boundaries improve it\. The effect does not replicate at F2 \(\+0\.003,pHolm=0\.850p\_\{\\text\{Holm\}\}=0\.850\) and is absent at F0, so we report it at one fertility level rather than as a general effect\.

### 5\.7The Indonesian exploratory arm

On the classifier\-agreement probe, MorphAustro\-Tok scores0\.903±0\.0030\.903\\pm 0\.003, matched BPE0\.900±0\.0060\.900\\pm 0\.006and the scrambled control0\.896±0\.0020\.896\\pm 0\.002\. The ordering is correct and the differences are negligible; the largest, \+0\.007 against the scrambled control, is not significant after correction \(pHolm=0\.168p\_\{\\text\{Holm\}\}=0\.168\)\. We draw no inference from the arm and report it as a feasibility check only \(Appendix[E](https://arxiv.org/html/2610.20859#A5)\)\.

### 5\.8No task transfer

Correlating per\-configuration MT and NER gains over the nine shared Tagalog configurations, joined on run fingerprint so that both tasks provably used the same tokenizer build and seed, gives Pearsonr=−0\.366r=\-0\.366\(p=0\.333p=0\.333\) and Spearmanρ=−0\.450\\rho=\-0\.450\(p=0\.224p=0\.224\)\. An MT gain does not predict an NER gain, which is consistent with reports that tokenizer quality transfers unevenly across tasks\([Rust et al\., 2021](https://arxiv.org/html/2610.20859#bib.bib5)\)\.

## 6Discussion

The natural reading of our results is that a morpheme\-boundary constraint on BPE improves machine translation for Tagalog without requiring the boundaries to becorrect\.

Both MorphAustro\-Tok and its scramble impose the same structural change on the same set of word types\. A word that would otherwise be segmented by whatever the merge table favours is instead always decomposed the same way, everywhere it occurs\. Frequency\-driven BPE has no such guarantee, since a rare inflected form may be cut differently from a frequent one\. Both conditions remove that inconsistency\.

Consistency is the property the two arms share that the baseline lacks, and it remains the leading candidate, but the design does not isolate it from what the arms also share: the same triggering types, morph count and root\-plus\-token shape\.

If the reading is right, it has a practical implication that runs against the motivation for this class of intervention: the expensive part may not be the part that pays\. Morphological analyzers are costly and scarce for exactly the languages that most need them, and a linguistically arbitrary decomposition captured the same MT gain here\.

#### Threats to validity\.

The scrambled control is not linguistically random in every respect: it fires on the same word types the real analyzer selects, so it inherits the decision aboutwhichwords to decompose while randomisinghow\. The selection of infixed types is itself linguistic information and may carry the effect, in which case the conclusion narrows further to “correct placement does not matter given correct selection”\. That is the most important open question our design leaves\. Scale is a second threat: an effect visible only once a model can exploit paradigm structure would be invisible at the size we trained\.

#### The sequence\-labelling asymmetry\.

The NER result sits awkwardly with the MT result and we do not have a confident account of it\. Sequence labelling appears sensitive to wrong boundaries in a way that generation is not: on NER, scrambling costs 0\.023 F1 relative to real morphology, while on MT it costs nothing we can detect\. One possibility is that word\-level label pooling makes token\-internal structure matter more than it does for generation, whose decoder can compensate\. The result replicates at only one of two levels, and we flag it as a lead rather than a finding\.

## 7Conclusion

Across 33 runs, a morpheme\-boundary constraint on BPE improved Tagalog machine translation over a matched baseline at all three fertility levels, by 0\.84, 1\.49 and 3\.40 mean sentence\-level chrF\+\+ points\. Token count does not explain the gain, since it grows as segmentation coarsens, and neither does the capacity that fertility matching buys: at the coarsest level the arm with more parameters scores lower\.

A scrambled control preserving granularity, invertibility and per\-type consistency but not morphological validity matched or exceeded the real analysis at the central fertility level and trailed it by one third of a standard deviation at the coarsest\. Sequence labelling differs: scrambling costs 0\.023 F1 at one level, although MorphAustro\-Tok still does not beat the baseline\.

In conclusion, the claim we defend is narrow: for this language, at this scale, we detect no contribution from segmenting correctly, and the result is consistent with a role for segmenting consistently that a further control would have to confirm\. Three follow\-ups are implied: a stronger scramble that randomises type selection, a consistency\-only baseline, and more seeds\.

## Limitations

First, scale\. Small models are a design choice rather than a constraint we worked around \(§[5\.1](https://arxiv.org/html/2610.20859#S5.SS1)\), but the choice has a cost\. The fertility\-matched models carry between 1\.12M and 1\.92M inference\-path parameters and are trained from scratch for 1,500 steps, roughly 1\.6 epochs, so every model is under\-trained\. The constraint applies identically across conditions and the contrasts therefore hold, but absolute scores are at floor level \(chrF\+\+ 6–11\) and we cannot claim that these effects persist at contemporary scale or under transfer from a pretrained checkpoint\.

Second, three seeds\. Withtcrit=4\.30t\_\{\\text\{crit\}\}=4\.30, seed\-level intervals are wide and strictly descriptive\. Any effect smaller than between\-seed dispersion is undetectable here, and two of our three MT effects fall in that range\.

Third, there is no scrambled control at the finest fertility level\. F0 was run with two conditions only, for budget reasons, so the F0 MT effect is uncontrolled and we make no attribution for it\.

Fourth, the scrambled control inherits type selection\. It randomises how a word is decomposed, not whether it is decomposed at all, so the design narrows the explanation without isolating consistency\.

Fifth, the Indonesian arm is underpowered by construction rather than by accident of sample size, since the mechanism reached 0\.27% of words\. It is reported as an exploratory feasibility check and supports no inference either way\.

Sixth, the voice probe is circular and is reported as diagnostic only \(§[4\.5](https://arxiv.org/html/2610.20859#S4.SS5)\)\. The evidential weight rests on MT and NER\.

Seventh, the register gap\. ParaCrawl Tagalog against FLORES\+ Filipino leaves 60\.9% of test types unseen in training\. The gap is symmetric across conditions, but it holds absolute quality at floor level and may interact with the effects we measure in ways we cannot separate\.

Eighth, decoding is greedy and evaluation is capped at 500 FLORES\+ and 300 in\-domain sentences, both for budget\. These settings are applied identically to every condition; the bootstrap runs on a smaller matched set and its intervals are correspondingly wider\.

Ninth, language identification in bitext filtering is weak\. It is a function\-word heuristic rather than a trained classifier, and it is unreliable on short sentences\.

Tenth, one line in 2,000 failed exact roundtrip under the morphological conditions after the normalisation fix\. The line is a training sentence rather than an evaluation example, so no paired comparison is affected, and the exclusion is applied to the union over conditions, so no condition is trained on material the others lack\.

Eleventh, the study covers two languages from one family and one tokenizer comparison\. A unigram\-LM tokenizer\([Kudo, 2018](https://arxiv.org/html/2610.20859#bib.bib2)\)at matched vocabulary is the obvious missing baseline\.

## Ethical Considerations

This work uses publicly released corpora under their stated licences\. FLORES\+ is distributed under CC\-BY\-SA\-4\.0 and is gated, so access requires accepting its conditions; ParaCrawl is released under CC0; and WikiANN is released under ODC\-BY\-1\.0\. We ask that work building on this paper cite the originating datasets and their contributors\.

ParaCrawl is web\-mined and is known to contain noise and material its authors did not intend for redistribution, and our filtering removes neither personal information nor offensive content in any targeted way\. We did not audit the Tagalog or Indonesian portions for either, and we release no model weights trained on them\.

Two of the languages studied here are under\-served by language technology, and work of this kind carries a risk of overclaiming on their behalf\. Our central result is a negative one, and we have tried to report it in terms that do not invite the opposite reading: a scrambled control performing as well as a real morphological analysis is evidence about our experimental scale, not evidence that Tagalog morphology is unimportant or that linguistic expertise is dispensable in building resources for Philippine languages\.

The compute cost of the reported experiments is 0\.76 GPU\-hours on a single T4\. A further 6\.3 GPU\-hours were spent on runs discarded after a throughput rescale and on tokenizer construction\. We report the two figures separately rather than as an aggregate, since folding discarded compute into the reported cost misrepresents what reproducing this study requires\.

## References

- Baliberet al\.\(2020\)R\. I\. Baliber, C\. Cheng, K\. M\. Adlaon, and V\. MamonongBridging Philippine languages with multilingual neural machine translation\.InProceedings of the 3rd Workshop on Technologies for MT of Low Resource Languages,Suzhou, China,pp\. 14–22\.External Links:[Link](https://aclanthology.org/2020.loresmt-1.2/),[Document](https://dx.doi.org/10.18653/v1/2020.loresmt-1.2)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.
- Bañónet al\.\(2020\)M\. Bañón, P\. Chen, B\. Haddow, K\. Heafield, H\. Hoang, M\. Esplà\-Gomis, M\. L\. Forcada, A\. Kamran, F\. Kirefu, P\. Koehn, S\. Ortiz Rojas, L\. Pla Sempere, G\. Ramírez\-Sánchez, E\. Sarrías, M\. Strelec, B\. Thompson, W\. Waites, D\. Wiggins, and J\. ZaragozaParaCrawl: web\-scale acquisition of parallel corpora\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,Online,pp\. 4555–4567\.External Links:[Link](https://aclanthology.org/2020.acl-main.417/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.417)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.
- Bostrom and Durrett \(2020\)K\. Bostrom and G\. DurrettByte pair encoding is suboptimal for language model pretraining\.InFindings of the Association for Computational Linguistics: EMNLP 2020,Online,pp\. 4617–4624\.External Links:[Link](https://aclanthology.org/2020.findings-emnlp.414/),[Document](https://dx.doi.org/10.18653/v1/2020.findings-emnlp.414)Cited by:[§1](https://arxiv.org/html/2610.20859#S1.p1.1),[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px1.p1.1)\.
- Efron and Tibshirani \(1993\)B\. Efron and R\. J\. TibshiraniAn introduction to the bootstrap\.Chapman and Hall,New York\.Cited by:[§3\.7](https://arxiv.org/html/2610.20859#S3.SS7.p2.1)\.
- Gowda and May \(2020\)T\. Gowda and J\. MayFinding the optimal vocabulary size for neural machine translation\.InFindings of the Association for Computational Linguistics: EMNLP 2020,Online,pp\. 3955–3964\.External Links:[Link](https://aclanthology.org/2020.findings-emnlp.352/),[Document](https://dx.doi.org/10.18653/v1/2020.findings-emnlp.352)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px1.p1.1)\.
- Goyalet al\.\(2022\)N\. Goyal, C\. Gao, V\. Chaudhary, P\. Chen, G\. Wenzek, D\. Ju, S\. Krishnan, M\. Ranzato, F\. Guzmán, and A\. FanThe Flores\-101 evaluation benchmark for low\-resource and multilingual machine translation\.Transactions of the Association for Computational Linguistics10,pp\. 522–538\.External Links:[Link](https://aclanthology.org/2022.tacl-1.30/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00474)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.
- Himmelmann \(2005\)N\. P\. HimmelmannThe Austronesian languages of Asia and Madagascar: typological characteristics\.InThe Austronesian Languages of Asia and Madagascar,A\. Adelaar and N\. P\. Himmelmann \(Eds\.\),pp\. 110–181\.Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px2.p1.1)\.
- Hofmannet al\.\(2022\)V\. Hofmann, H\. Schuetze, and J\. PierrehumbertAn embarrassingly simple method to mitigate undesirable properties of pretrained language model tokenizers\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),Dublin, Ireland,pp\. 385–393\.External Links:[Link](https://aclanthology.org/2022.acl-short.43/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-short.43)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px1.p1.1)\.
- Holm \(1979\)S\. HolmA simple sequentially rejective multiple test procedure\.Scandinavian Journal of Statistics6\(2\),pp\. 65–70\.Cited by:[§3\.7](https://arxiv.org/html/2610.20859#S3.SS7.p3.1)\.
- Koehn \(2004\)P\. KoehnStatistical significance tests for machine translation evaluation\.InProceedings of the 2004 Conference on Empirical Methods in Natural Language Processing,Barcelona, Spain,pp\. 388–395\.External Links:[Link](https://aclanthology.org/W04-3250/)Cited by:[§3\.7](https://arxiv.org/html/2610.20859#S3.SS7.p2.1)\.
- Kroeger \(1993\)P\. KroegerPhrase structure and grammatical relations in tagalog\.CSLI Publications,Stanford, CA\.Cited by:[§1](https://arxiv.org/html/2610.20859#S1.p2.1),[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px2.p1.1)\.
- Kudo and Richardson \(2018\)T\. Kudo and J\. RichardsonSentencePiece: a simple and language independent subword tokenizer and detokenizer for neural text processing\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,Brussels, Belgium,pp\. 66–71\.External Links:[Link](https://aclanthology.org/D18-2012/),[Document](https://dx.doi.org/10.18653/v1/D18-2012)Cited by:[§3\.3](https://arxiv.org/html/2610.20859#S3.SS3.p1.1)\.
- Kudo \(2018\)T\. KudoSubword regularization: improving neural network translation models with multiple subword candidates\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Melbourne, Australia,pp\. 66–75\.External Links:[Link](https://aclanthology.org/P18-1007/),[Document](https://dx.doi.org/10.18653/v1/P18-1007)Cited by:[Limitations](https://arxiv.org/html/2610.20859#Sx1.p11.1)\.
- Loveniaet al\.\(2024\)H\. Lovenia, R\. Mahendra, S\. M\. Akbar, L\. J\. V\. Miranda, J\. Santoso, E\. Aco, A\. Fadhilah, J\. Mansurov, J\. M\. Imperial, O\. P\. Kampman, J\. R\. A\. Moniz, M\. R\. S\. Habibi, F\. Hudi, R\. Montalan, R\. Ignatius, J\. A\. Lopo, W\. Nixon, B\. F\. Karlsson, J\. Jaya, R\. Diandaru, Y\. Gao, P\. Amadeus, B\. Wang, J\. C\. B\. Cruz, C\. Whitehouse, I\. H\. Parmonangan, M\. Khelli, W\. Zhang, L\. Susanto, R\. A\. Ryanda, S\. L\. Hermawan, D\. J\. Velasco, M\. D\. A\. Kautsar, W\. F\. Hendria, Y\. Moslem, N\. Flynn, M\. F\. Adilazuarda, H\. Li, J\. Lee, R\. Damanhuri, S\. Sun, M\. R\. Qorib, A\. Djanibekov, W\. Q\. Leong, Q\. V\. Do, N\. Muennighoff, T\. Pansuwan, I\. F\. Putra, Y\. Xu, T\. N\. Chia, A\. Purwarianti, S\. Ruder, W\. Tjhi, P\. Limkonchotiwat, A\. F\. Aji, S\. Keh, G\. I\. Winata, R\. Zhang, F\. Koto, Z\. Yong, and S\. CahyawijayaSEACrowd: a multilingual multimodal data hub and benchmark suite for Southeast Asian languages\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 5155–5203\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.296/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.296)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.
- Nivreet al\.\(2020\)J\. Nivre, M\. de Marneffe, F\. Ginter, J\. Hajič, C\. D\. Manning, S\. Pyysalo, S\. Schuster, F\. Tyers, and D\. ZemanUniversal Dependencies v2: an evergrowing multilingual treebank collection\.InProceedings of the Twelfth Language Resources and Evaluation Conference,Marseille, France,pp\. 4034–4043\.External Links:[Link](https://aclanthology.org/2020.lrec-1.497/)Cited by:[§4\.4](https://arxiv.org/html/2610.20859#S4.SS4.p1.1)\.
- NLLB Team \(2024\)NLLB TeamScaling neural machine translation to 200 languages\.Nature630\(8018\),pp\. 841–846\.External Links:[Document](https://dx.doi.org/10.1038/s41586-024-07335-x)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.
- Panet al\.\(2017\)X\. Pan, B\. Zhang, J\. May, J\. Nothman, K\. Knight, and H\. JiCross\-lingual name tagging and linking for 282 languages\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vancouver, Canada,pp\. 1946–1958\.External Links:[Link](https://aclanthology.org/P17-1178/),[Document](https://dx.doi.org/10.18653/v1/P17-1178)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.
- Parket al\.\(2021\)H\. H\. Park, K\. J\. Zhang, C\. Haley, K\. Steimel, H\. Liu, and L\. SchwartzMorphology matters: a multilingual language modeling analysis\.Transactions of the Association for Computational Linguistics9,pp\. 261–276\.External Links:[Link](https://aclanthology.org/2021.tacl-1.16/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00365)Cited by:[§1](https://arxiv.org/html/2610.20859#S1.p1.1),[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px1.p1.1)\.
- Popović \(2015\)M\. PopovićChrF: character n\-gram F\-score for automatic MT evaluation\.InProceedings of the Tenth Workshop on Statistical Machine Translation,Lisbon, Portugal,pp\. 392–395\.External Links:[Link](https://aclanthology.org/W15-3049/),[Document](https://dx.doi.org/10.18653/v1/W15-3049)Cited by:[§4\.1](https://arxiv.org/html/2610.20859#S4.SS1.p1.1)\.
- Popović \(2017\)M\. PopovićChrF\+\+: words helping character n\-grams\.InProceedings of the Second Conference on Machine Translation,Copenhagen, Denmark,pp\. 612–618\.External Links:[Link](https://aclanthology.org/W17-4770/),[Document](https://dx.doi.org/10.18653/v1/W17-4770)Cited by:[§4\.1](https://arxiv.org/html/2610.20859#S4.SS1.p1.1)\.
- Rahimiet al\.\(2019\)A\. Rahimi, Y\. Li, and T\. CohnMassively multilingual transfer for NER\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,Florence, Italy,pp\. 151–164\.External Links:[Link](https://aclanthology.org/P19-1015/),[Document](https://dx.doi.org/10.18653/v1/P19-1015)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.
- Rustet al\.\(2021\)P\. Rust, J\. Pfeiffer, I\. Vulić, S\. Ruder, and I\. GurevychHow good is your tokenizer? on the monolingual performance of multilingual language models\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),Online,pp\. 3118–3135\.External Links:[Link](https://aclanthology.org/2021.acl-long.243/),[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.243)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px1.p1.1),[§5\.8](https://arxiv.org/html/2610.20859#S5.SS8.p1.1)\.
- Schachter and Otanes \(1972\)P\. Schachter and F\. T\. OtanesTagalog reference grammar\.University of California Press,Berkeley, CA\.Cited by:[§1](https://arxiv.org/html/2610.20859#S1.p2.1),[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px2.p1.1)\.
- Schmidtet al\.\(2024\)C\. W\. Schmidt, V\. Reddy, H\. Zhang, A\. Alameddine, O\. Uzan, Y\. Pinter, and C\. TannerTokenization is more than compression\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 678–702\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.40/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.40)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px1.p1.1)\.
- Sennrichet al\.\(2016\)R\. Sennrich, B\. Haddow, and A\. BirchNeural machine translation of rare words with subword units\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Berlin, Germany,pp\. 1715–1725\.External Links:[Link](https://aclanthology.org/P16-1162/),[Document](https://dx.doi.org/10.18653/v1/P16-1162)Cited by:[§1](https://arxiv.org/html/2610.20859#S1.p2.1)\.
- Smitet al\.\(2014\)P\. Smit, S\. Virpioja, S\. Grönroos, and M\. KurimoMorfessor 2\.0: toolkit for statistical morphological segmentation\.InProceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics,Gothenburg, Sweden,pp\. 21–24\.External Links:[Link](https://aclanthology.org/E14-2006/),[Document](https://dx.doi.org/10.3115/v1/E14-2006)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px2.p1.1)\.
- Sneddonet al\.\(2010\)J\. N\. Sneddon, A\. Adelaar, D\. N\. Djenar, and M\. EwingIndonesian reference grammar\.2 edition,Allen and Unwin,Sydney\.Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px3.p1.1)\.
- Tiedemann \(2012\)J\. TiedemannParallel data, tools and interfaces in OPUS\.InProceedings of the Eighth International Conference on Language Resources and Evaluation \(LREC’12\),Istanbul, Turkey,pp\. 2214–2218\.External Links:[Link](https://aclanthology.org/L12-1246/)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.
- Zhanget al\.\(2020\)B\. Zhang, P\. Williams, I\. Titov, and R\. SennrichImproving massively multilingual neural machine translation and zero\-shot translation\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,Online,pp\. 1628–1639\.External Links:[Link](https://aclanthology.org/2020.acl-main.148/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.148)Cited by:[§2](https://arxiv.org/html/2610.20859#S2.SS0.SSS0.Px4.p1.1)\.

## Appendix ATraining configuration

Table[5](https://arxiv.org/html/2610.20859#A1.T5)lists every setting\. All of them are held fixed across the three tokenizer conditions, and only the vocabulary size implied by fertility matching differs\. Parameter parity is enforced at matched vocabulary, where all three conditions carry 1,950,208 inference\-path parameters; under fertility matching the counts range from 1\.12M to 1\.92M and are given in Table[3](https://arxiv.org/html/2610.20859#S5.T3)\.

Table 5:Training configuration\.
## Appendix BData sizes and rejected sources

Table 6:Corpus sizes\. WikiANN rows are the Tagalog splits\.The Indonesian raw count in Table[6](https://arxiv.org/html/2610.20859#A2.T6)is a parse cap rather than the size of ParaCrawlen\-id\. The tokenizer corpus is both sides of the MT training bitext plus the NER training sentences\. Root evidence is target\-side text only: the retained bitext plus the 10,000 NER training sentences, less the in\-domain held\-out pool when part of that pool is scored, which is why the Tagalog count is 2,000 lower than the Indonesian construction would suggest\.

The Indonesian raw count in Table[6](https://arxiv.org/html/2610.20859#A2.T6)is a parse cap rather than the size of ParaCrawlen\-id, and the Indonesian WikiANN splits \(12,000 / 10,000 / 10,000\) were never used, since Indonesian NER was cut\. The tokenizer corpus is both sides of the MT training bitext plus the NER training sentences\. Root evidence is target\-side text only: the retained bitext plus the 10,000 NER training sentences, less the in\-domain held\-out pool when part of that pool is scored, which is why the Tagalog count is 2,000 lower than the Indonesian construction would suggest\.

Candidate sources were rejected on recorded grounds\.facebook/floresandallenai/nllbare loading\-script datasets and are unloadable under current tooling\. OPUS\-100 contains no Tagalog configuration\. The wikimedia, WikiMatrix and translatewiki corpora are Wikipedia\-derived and would contaminate a Wikipedia\-sourced test set by construction\. XLEnt supplies entity alignments rather than sentence bitext, and OpenSubtitles has unclear redistribution terms\.

## Appendix CThe fertility sweep

Table[7](https://arxiv.org/html/2610.20859#A3.T7)gives the achieved value for every cell\. The largest deviation from target is 0\.0048, within the 0\.005 tolerance the matching gate asserts\. Vocabulary is the size returned by bisection; the scrambled arm additionally registers 147 discovered pseudo\-infix types after training, which is why its counts in Table[3](https://arxiv.org/html/2610.20859#S5.T3)are higher by that amount\.

Table 7:Achieved tokens per character and the vocabulary size each condition required to reach it\.Across the three fertility levels the parameter advantage of MorphAustro\-Tok over matched BPE and its chrF\+\+ gain are ordered oppositely \(Pearsonr=−0\.810r=\-0\.810\)\. With three points this is descriptive only, and the argument in §[5\.4](https://arxiv.org/html/2610.20859#S5.SS4)rests on the within\-level comparison at F2 instead\.

The Tagalog condition matrix is three seeds for each of MorphAustro\-Tok and matched BPE at all three levels, plus three seeds for the scrambled control at F1 and F2, giving 24 runs\. The Indonesian arm adds three conditions at one level, giving nine\. Figure[1](https://arxiv.org/html/2610.20859#A3.F1)plots the two MT contrasts across the sweep\.

Figure 1:Gain over matched BPE against fertility, with the shaded region marking differences smaller than the between\-seed standard deviation\.
## Appendix DGates and further defects

Beyond the normalisation defect described in §[4\.3](https://arxiv.org/html/2610.20859#S4.SS3), the gates cover granularity parity between the real and scrambled segmenters \(identical morph counts on 2,000 of 2,000 lines\), derangement rate of the scramble \(1\.000 over 84 infixed types\), marker\-codepoint disjointness from corpus text, achieved\-fertility tolerance, vocabulary\-count parity, decoder causality \(maximum logit difference 0\.00 at positions preceding the cut, asserted on every run\), exact roundtrip, and a pooling audit asserting that each row covers exactly its word’s pieces\.

Three further defects are worth recording\. The BPE trainer was seeing the literal characters of tokens already registered as atomic and learning merges from inside them, so that fragments occupied real vocabulary slots and were emittable during decoding\. The Indonesian classifier binder handledNUM CLF Nbut not the fusedseCLF Nform that carries most Indonesian classifier constructions, which left activity at 0\.02% of words and a fertility separation of 0\.07%, amounting to three labels on one tokenizer\. Finally, a single wall\-clock ceiling per run allowed machine translation to consume the entire allowance, leaving NER and the probe at step 1 of their schedules, and the first live run took 182 minutes against a 12\.6\-minute estimate, of which roughly 160 minutes was unbudgeted batch\-of\-one beam search\. Ceilings are now per task, decoding is batched and greedy, and the affected runs were excluded after the matrix was rescaled\.

## Appendix EThe Indonesian feasibility arm

Malay and Indonesian cannot be pooled, which FLORES\+ makes directly measurable rather than assumed: over the 997 sentence pairs present in bothzsm\_Latnandind\_Latn, 0\.0% are identical, mean normalised edit distance is 0\.425 and type\-Jaccard is 0\.391\. Indonesian therefore runs alone\.

The classifier binder originally handledNUM CLF Nbut not the fusedseCLF Nform that carries most Indonesian classifier constructions, which left activity at 0\.02% of words and a fertility separation of 0\.07%, amounting to three labels on one tokenizer\. After the fix the mechanism reaches 3\.95% of lines and 0\.274% of words, with a fertility separation of 0\.167%\. That is two orders of magnitude below the Tagalog separation of 6\.94%, so Indonesian MT and NER were cut on measured evidence rather than attempted and reported as null\.

What remains is the classifier\-agreement probe, a corruption\-judgment task whose label is whether a classifier and its noun were swapped\. No tokenizer exposes that label, so the probe is not circular in the way the Tagalog voice probe is, and the trivial token\-inspection baseline scores 0\.000 on it\. Against matched BPE MorphAustro\-Tok gains \+0\.003 \(pHolm=0\.919p\_\{\\text\{Holm\}\}=0\.919\), and against the scrambled control \+0\.007 \(pHolm=0\.168p\_\{\\text\{Holm\}\}=0\.168\)\. We read the arm as evidence that the mechanism can be built and exercised in a second Austronesian language, not as evidence about classifier\-aware segmentation\.

## Appendix FReproducibility

Corpora are fetched at run time from pinned versioned URLs and hub identifiers, with SHA\-256 recorded over materialised content, and with licences and citations recorded in the run manifest; a failed fetch raises rather than substituting\. We release at[https://github\.com/AbdulRehmanAntall/morphaustro\-tok](https://github.com/AbdulRehmanAntall/morphaustro-tok)the notebook that implements the pipeline end to end, per\-run JSON records with metrics and per\-example scores, the full gate log, the dispersion and comparison tables in CSV, and the limitations log\.

Seeds are fixed and recorded, and string hashing uses SHA\-256 rather than Python’s salted hash\. Runs are keyed by architecture and configuration fingerprints, so a stale artefact is invalidated rather than silently reused\. Dependency constraints are satisfiable version floors verified after import, with resolved versions recorded; exact pins were unsatisfiable on the runtime available and would have been recorded as applied while silently failing\.

#### Determinism\.

Deterministic algorithms are enabled in PyTorch in warn\-only mode\. On the available hardware, fp16 reductions under automatic mixed precision have no deterministic implementation\. We therefore claim seed\-controlled reproducibility of the experimental protocol, not bitwise reproducibility of a forward pass\.