Not All Irregularity Is Equal: Causally Isolating a Rare Failure Mode in Japanese Morphological Inflection

# Not All Irregularity Is Equal: Causally Isolating a Rare Failure Mode in Japanese Morphological Inflection
Source: [https://arxiv.org/html/2609.21179](https://arxiv.org/html/2609.21179)
\[ Path = fonts/, Extension = \.otf, UprightFont = \*\-regular, BoldFont = \*\-bold, ItalicFont = \*\-italic, BoldItalicFont = \*\-bolditalic \] \[ Path = fonts/, Extension = \.otf, ItalicFont = NotoSansCJKjp\-Regular, ItalicFeatures = FakeSlant=0\.2 \]

###### Abstract

Neural morphological generation systems often achieve high aggregate accuracy on benchmark datasets, yet such performance can conceal systematic errors clustered in rare morphological subclasses\. We present an orthography\-aware diagnosis of Japanese past\-tense verb inflection, treatinghiragananot merely as a transcriptional medium but as a representational system that encodes morphophonological structure\. Using two character\-level Transformer architectures evaluated across five random seeds, we show that although both systems exceed 97% aggregate accuracy, a single structurally specific irregular subtype, verbs whose stems end in /e/ and require gemination before the past\-tense suffix and make up fewer than 1% of the data, accounts for a disproportionate 30–43% share of residual errors and contributes roughly 34–48×\\timesits prevalence to total errors\. We then move from diagnosis to causal isolation: controlled ablation experiments show that removing this subtype alone produces larger accuracy gains than removing all irregular verbs combined\. These findings indicate that error concentration in neural morphological learning is not driven by irregularityper se, but by the interaction between extreme low\-frequency morphological patterns and specific orthographic processes\. We argue that morphological evaluation should incorporate fine\-grained subclass analysis, and discuss implications for data\-efficient, developmentally plausible language model pretraining\.111Code and data available at[https://github\.com/wenzhang0222/not\-all\-irregularity](https://github.com/wenzhang0222/not-all-irregularity)\.

## 1Introduction

Japanese verbs are written in a hybrid system ofkanji,hiragana, andkatakana\. While verb stems may appear in eitherkanjiorhiragana, inflectional suffixes are consistently marked inhiragana\. For example, the verb\\jpfont書く/\\jpfontかくkakubecomes\\jpfont書いた/\\jpfontかいたkaitain the past tense, where\\jpfontたtamarks the past\-tense suffix\.Hiraganais a moraic script, representing phonological timing units and marking processes such as gemination \(the small\\jpfontっtsu\) and vowel lengthening that are central to Japanese morphophonology\([Kubozono et al\., 2009](https://arxiv.org/html/2609.21179#bib.bib18);[Labrune, 2012](https://arxiv.org/html/2609.21179#bib.bib19)\)\. Following the view that writing systems are structured representational systems rather than transparent encodings of speech\([Sproat, 2000](https://arxiv.org/html/2609.21179#bib.bib20);[Daniels and Bright, 1996](https://arxiv.org/html/2609.21179#bib.bib21)\), we treathiraganaas a linguistically meaningful layer that may shape neural model generalization, and use it to eliminate confounds fromkanjihomography by restricting all forms tohiragana\.

Most evaluations of Japanese morphological generation report aggregate exact\-match accuracy\([Cotterell et al\., 2016](https://arxiv.org/html/2609.21179#bib.bib3);[Vylomova et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib1);[Goldman et al\., 2023](https://arxiv.org/html/2609.21179#bib.bib2)\), often exceeding 95% for high\-resource languages such as Japanese\. However, high aggregate accuracy does not necessarily indicate robust generalization, and can obscure systematic weaknesses concentrated in rare morphological subclasses\([Goldman et al\., 2022](https://arxiv.org/html/2609.21179#bib.bib24)\)\. This is a manifestation of the “long tail” challenge in morphological learning\([Cotterell et al\., 2018](https://arxiv.org/html/2609.21179#bib.bib5)\), where models favor high\-frequency regularities over infrequent but productive rules\.

In this paper we combine two complementary methodologies to study this phenomenon in Japanese past\-tense inflection\. First, we conduct an orthography\-awarediagnosis, using a fine\-grained error taxonomy and subgroup\-level accuracy analysis, aggregated across five random seeds and two architectures, to characterizewheremodels fail\. Second, we conduct acausal isolationstudy, using controlled ablation experiments that selectively remove irregular verb subtypes, to testwhetherthe identified subtype is actually responsible for destabilizing generalization, and not irregularity in general\. Combining them lets us both locate the failure and test its cause\.

Our contributions are:

- •A structural subgroup evaluation framework for Japanese morphological inflection, combining a six\-category orthography\-aware error taxonomy with a Disparity Ratio metric for quantifying subgroup\-level error concentration\.
- •A multi\-seed, two\-architecture empirical analysis showing that a single low\-frequency irregular subtype \(<1% of the data\) accounts for 30–43% of residual errors\.
- •Controlled ablation experiments showing that selective removal of this subtype alone produces larger accuracy gains than removing all irregular verbs combined, suggesting that structural position, not frequency alone, is the primary driver of the effect\.

### 1\.1Japanese Past\-Tense Morphology

Japanese verbs are traditionally classified by inflectional behavior\. Past\-tense formation involves the suffix\\jpfontた\-taand morphophonological alternations such as consonant mutation, gemination, and vowel changes, all of which are systematically reflected inhiraganaorthography\.Godan\(u\-\) verbsform the past tense through suffix\-conditioned stem alternations, often involving consonant changes and gemination, e\.g\.,\\jpfontかくkaku‘to write’→\\to\\jpfontかいたkaita‘wrote’\.Ichidan\(ru\-\) verbsexhibit stable stems, with the past tense formed via direct suffix attachment, e\.g\.,\\jpfontたべるtaberu‘to eat’→\\to\\jpfontたべたtabeta‘ate’\.Canonical irregular verbsare a small closed class including\\jpfontするsuru‘to do’→\\to\\jpfontしたshita‘did’ and\\jpfontくるkuru‘to come’→\\to\\jpfontきたkita‘came’\. Because these alternations are encoded transparently inhiragana, past\-tense inflection is a dense testbed for studying morphophonological learning at the character level, without requiring lexical segmentation\.

## 2Related Work

#### Morphological Learning and Generalization\.

Neural morphological inflection has been driven largely by Transformer\-based encoder–decoder architectures\([Vaswani et al\., 2017](https://arxiv.org/html/2609.21179#bib.bib10)\), achieving high aggregate accuracy on SIGMORPHON benchmarks\([Vylomova et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib1);[Goldman et al\., 2023](https://arxiv.org/html/2609.21179#bib.bib2)\)\. There is growing recognition that aggregate metrics obscure performance disparities across morphological classes\([Goldman et al\., 2022](https://arxiv.org/html/2609.21179#bib.bib24)\)\. We contribute to this line of work by showing how a single structurally idiosyncratic subclass can act as a stress test for neural generalization, and by testing this causally\. This work synthesizes and extends our prior analyses of Japanese morphological inflection\([Zhang, 2026a](https://arxiv.org/html/2609.21179#bib.bib28);[Zhang, 2026b](https://arxiv.org/html/2609.21179#bib.bib27)\), combining orthography\-aware error diagnosis with causal ablation methodology\.

#### Subgroup Analysis and Fairness in NLP\.

The challenge of modeling rare data is not unique to morphology; it mirrors broader concerns about subgroup disparity in neural models, where models often prioritize majority\-group performance at the expense of underrepresented strata\([Sagawa et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib13);[Buolamwini and Gebru, 2018](https://arxiv.org/html/2609.21179#bib.bib14)\)\. We adopt diagnostic tools from this literature, in particular a Disparity Ratio, to quantify error concentration across morphological subtypes, while noting that our “subgroups” are defined structurally and not socially\([Blodgett et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib15)\)\.

#### Inductive Bias and Irregularity\.

The learnability of irregular morphology has long been central to debates between rule\-based and connectionist accounts of language processing\([Rumelhart and McClelland, 1986](https://arxiv.org/html/2609.21179#bib.bib16);[Pinker, 1994](https://arxiv.org/html/2609.21179#bib.bib17)\)\. Our findings align with evidence that neural models can capture systematic patterns of morphological irregularity\([Wu et al\., 2019](https://arxiv.org/html/2609.21179#bib.bib7);[Kakolu Ramarao et al\., 2025](https://arxiv.org/html/2609.21179#bib.bib9)\), but extend this literature by showing that not all irregular forms are equally destabilizing\. Orthographic transparency and structural frequency, not irregular status alone, predict where models fail\.

#### Evaluation Metrics and Orthographic Bias\.

Recent work shows that normalized crosslingual evaluation metrics introduce systematic biases rooted in tokenization, encoding, and orthographic differences\([Yang et al\., 2026](https://arxiv.org/html/2609.21179#bib.bib26)\)\. Our findings offer a parallel concern at the subtype level within a single language: aggregate accuracy within Japanese morphological evaluation similarly obscures systematic failures clustered in structurally specific subclasses, suggesting that subgroup\-aware diagnostics are necessary at both the crosslingual and intra\-paradigm levels\.

## 3Data

We use a Japanese verb inflection dataset in standard morphological transduction format, following the setup of[Vylomova et al\. \(2020\)](https://arxiv.org/html/2609.21179#bib.bib1)and[Goldman et al\. \(2023\)](https://arxiv.org/html/2609.21179#bib.bib2)\. All forms are converted tohiraganato maintain orthographic consistency and eliminate confounds introduced bykanjihomography\([Zhang, 2023](https://arxiv.org/html/2609.21179#bib.bib25)\)\. Each instance consists of three tab\-separated fields: lemma, target form, and a placeholder tag \(no explicit morphosyntactic features are provided, so the model must learn the lemma\-to\-form mapping directly\), e\.g\.\\jpfontねがえる\\jpfontねがえった\_\.

### 3\.1Verb Classification

Verbs are classified according to traditional Japanese conjugation classes, refined to capture orthography\-sensitive variation\. Canonical irregular verbs \(\\jpfontする,\\jpfontくる; Type 3\) and polysemous lemmas with multiple inflected forms are excluded to maintain a one\-to\-one lemma–form mapping\. The remaining verbs fall into three types:

- •Type 1 \(Godan\): regularuu\-verbs, e\.g\.\\jpfontかくkaku→\\to\\jpfontかいたkaita\. Count: 2,503\.
- •Type 2 \(Ichidan\): regularr​uru\-verbs, e\.g\.\\jpfontたべるtaberu→\\to\\jpfontたべたtabeta\. Count: 1,298\.
- •Type 4\(irregular\): further subdivided into three subtypes: - –Type 4\-1: stem\-final /i/ \+ gemination, e\.g\.\\jpfontまじるmajiru→\\to\\jpfontまじったmajitta\. Count: 119\. - –Type 4\-2: stem\-final /e/ \+ gemination, e\.g\.\\jpfontあきれかえるakirekaeru→\\to\\jpfontあきれかえったakirekaetta\. Count: 37\. - –Type 4\-3: localized idiosyncratic deviations, e\.g\.\\jpfontいくiku→\\to\\jpfontいったitta\. Count: 1\.

Table[1](https://arxiv.org/html/2609.21179#S3.T1)summarizes dataset composition\. Type 4\-2 is the smallest structurally coherent subclass, comprising only 0\.9% of the data; the full verb list appears in Appendix[A](https://arxiv.org/html/2609.21179#A1)\.

Table 1:Dataset statistics by verb type\. Canonical irregulars \(\\jpfontする,\\jpfontくる; Type 3\) are excluded, as described in §3\.1\.

## 4Models

We evaluate two character\-level Transformer encoder–decoder architectures for Japanese past\-tense inflection, both operating overhiraganastrings and generating inflected forms autoregressively\. The first is aPointer\-Generator Transformer\(henceforthPGT\), incorporating a pointer\-generator copying mechanism\([See et al\., 2017](https://arxiv.org/html/2609.21179#bib.bib23)\)with attention over the input lemma, trained and evaluated on the morphological inflection data\([Vylomova et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib1)\)\. The second follows the architecture of[Goldman et al\. \(2023\)](https://arxiv.org/html/2609.21179#bib.bib2), aLemma\-Split Transformer\(henceforthLST\) that additionally uses a lemma\-split training and evaluation regime, preventing a lemma from appearing in both training and test data and improving generalization to unseen lemmas\.

## 5Experimental Setup

### 5\.1Training Regime

Both models are trained with cross\-entropy loss and teacher forcing, using the default hyperparameter configurations from their respective shared\-task baselines\([Vylomova et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib1);[Goldman et al\., 2023](https://arxiv.org/html/2609.21179#bib.bib2)\), optimized with Adam\([Kingma and Ba, 2015](https://arxiv.org/html/2609.21179#bib.bib11)\)and standard Transformer learning\-rate scheduling\([Vaswani et al\., 2017](https://arxiv.org/html/2609.21179#bib.bib10)\)\.

### 5\.2Multi\-Seed Baseline Evaluation

Neural model performance is sensitive to random initialization, data shuffling, and optimization dynamics\([Reimers and Gurevych, 2017](https://arxiv.org/html/2609.21179#bib.bib12)\)\. To reduce selective\-reporting bias, we train each model on the full dataset five times with different random seeds, using an 80/10/10 train/development/test split\. The error taxonomy and quantitative error\-distribution analyses \(§6\.2–6\.4\) are aggregated across all five seeds\. The controlled ablation experiments \(§6\.7\) involve eight training conditions per model and are conducted on the primary seed\.

### 5\.3Controlled Ablation Conditions

To causally test the contribution of individual irregular subtypes, we train both architectures under eight controlled data conditions: the full dataset; regular verbs only \(Types 1–2\); the full dataset minus one irregular subtype \(removing 4\-1, 4\-2, or 4\-3 individually\); and the full dataset minus each pairwise combination of subtypes\. For every condition, the same verb types are removed from both training and test data\.

### 5\.4Evaluation Metrics

We reportexact\-match accuracyat the lemma level\([Cotterell et al\., 2017](https://arxiv.org/html/2609.21179#bib.bib4);[Goldman et al\., 2023](https://arxiv.org/html/2609.21179#bib.bib2)\);subgroup accuracy, computed separately per verb type\([Kann and Schütze, 2016](https://arxiv.org/html/2609.21179#bib.bib6);[Makarov and Clematide, 2018](https://arxiv.org/html/2609.21179#bib.bib8);[Vylomova et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib1)\); and theDisparity Ratiofor a subgroupgg,

DRg=Error SharegData Shareg,\\textit\{DR\}\_\{g\}=\\frac\{\\text\{Error Share\}\_\{g\}\}\{\\text\{Data Share\}\_\{g\}\},where a value greater than 1 indicates a disproportionately high error burden relative to prevalence, following diagnostic tools from subgroup fairness analysis\([Sagawa et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib13);[Buolamwini and Gebru, 2018](https://arxiv.org/html/2609.21179#bib.bib14);[Blodgett et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib15)\)\.

## 6Results

### 6\.1Baseline Performance

On the primary run, both systems achieve high aggregate accuracy on the full dataset: 97\.98% \(PGT\) and 97\.73% \(LST\)\. Averaged across all five seeds, accuracy is 97\.17%±\\pm0\.65 \(PGT\) and 96\.97%±\\pm0\.82 \(LST\) \(mean±\\pmsample standard deviation\), confirming that high aggregate accuracy is a stable property of both architectures\. Despite this overall stability, errors cluster in specific low\-frequency subclasses, as we show next\.

### 6\.2Error Taxonomy

We defined a taxonomy of six orthography\- and morphophonology\-sensitive failure modes \(Table[2](https://arxiv.org/html/2609.21179#S6.T2)\): gemination omission, gemination insertion, phonological substitution, morpheme boundary errors, character recognition \(UNK\) errors, and compound verb structural errors\. We used this taxonomy to manually classify all residual errors \(Table[3](https://arxiv.org/html/2609.21179#S6.T3)\)\. This taxonomy highlights that residual failures are systematic, not random noise\. For example, failures to insert, or spuriously insert, the small\\jpfontっreflect specific difficulty with consonant doubling\.

Table 2:Error taxonomy forhiragana\-based inflection\.
### 6\.3Quantitative Error Distribution

Aggregated across five seeds, we observe 104 total errors, 53 from the PGT system and 51 from the LST system\. Table[3](https://arxiv.org/html/2609.21179#S6.T3)reports the six observed error types\. Gemination\-related errors \(omission and insertion combined\) dominate residual failures, accounting for 75\.5% \(PGT\) and 80\.3% \(LST\) of all errors, with omissions far more frequent than insertions in both systems\.

Table 3:Quantitative error distribution by error type\.Figure 1:Per\-type error rate \(errors / total items of that type\) for PGT and LST systems\. Type 4\-2 shows an error rate of 43% \(PGT\) and 60% \(LST\), more than 70 times the rate of Type 1 verbs\.
### 6\.4Verb\-Class Asymmetry and Disparity Ratios

Table[4](https://arxiv.org/html/2609.21179#S6.T4)breaks the same errors down by verb type\. Type 4\-2 verbs are massively overrepresented, accounting for 30\.2% \(PGT\) and 43\.1% \(LST\) of all errors while comprising just 0\.9% of the dataset\. Type 4\-1 is also somewhat overrepresented, while Type 1 and Type 2 show error rates below their dataset share\. Figure[1](https://arxiv.org/html/2609.21179#S6.F1)shows the per\-type error rate directly, making the disproportionate difficulty of Type 4\-2 visually apparent\.

Table 4:Error distribution by verb type against dataset share\.Table[5](https://arxiv.org/html/2609.21179#S6.T5)converts this into Disparity Ratios\. Type 4\-2 contributes 34–48×\\timesits proportional representation to total errors, as visualized in Figure[2](https://arxiv.org/html/2609.21179#S6.F2), the largest disparity of any subclass in either system\. This pattern is consistent across both architectures\.

To confirm this over\-representation is not an artifact of small counts, we tested it directly\. Under the null hypothesis that a verb’s probability of erring is independent of its type \(i\.e\., errors are distributed in proportion to each type’s data share\), we would expect Type 4\-2 to account for only 0\.9% of errors\. A one\-sided exact binomial test rejects this null decisively for both systems \(PGT: 16/53 errors,p=3\.6×10−20p=3\.6\\times 10^\{\-20\}; LST: 22/51 errors,p=2\.7×10−31p=2\.7\\times 10^\{\-31\}\)\.

Table 5:Disparity Ratio \(error share / data share\) by verb type\. Values\>\>1 indicate error over\-representation relative to prevalence\.Figure 2:Disparity Ratio by verb type for PGT and LST systems\. Type 4\-2 shows a dramatically higher disparity ratio than all other subtypes, contributing 34–48×\\timesits proportional share to total errors\.
### 6\.5Cross\-Model Consistency

Despite architectural differences, error patterns are broadly consistent across the PGT and LST systems\. Gemination\-related failures dominate in both \(75\.5%/80\.3%\), Type 4\-2 is the most overrepresented subclass in both \(Table 4\), and an identical single UNK error appears in both systems’ outputs \(\\jpfontつっぷした→\\to\\jpfontっ<UNK\>\\jpfontした\)\. This consistency, together with the seed\-level stability reported in §6\.1, suggests the failures reflect systematic properties of orthographic representation rather than idiosyncrasies of a single model\.

### 6\.6Qualitative Error Patterns

The dominant Type 4\-2 failure is omission of required gemination before a stem\-final /e/, e\.g\.\\jpfontあきれかえったakirekaetta→\\to\\jpfontあきれかえたakirekaeta\. Spurious gemination insertion occurs mainly in Type 2 verbs, e\.g\.\\jpfontおきたokita→\\to\\jpfontおきったokitta\. Morpheme boundary errors arise when compound boundaries are unmarked inhiragana, e\.g\.\\jpfontほめたたえたhometataeta→\\to\\jpfontほめたえたhometaeta\.

### 6\.7Causal Ablation: Isolating Type 4\-2

The analyses above are descriptive: they showwhereerrors concentrate, but not whether removing the implicated subtype actually improves generalization more than removing irregularity broadly\. Table[6](https://arxiv.org/html/2609.21179#S6.T6)and Figure[3](https://arxiv.org/html/2609.21179#S6.F3)summarize the accuracy gain over the full\-data baseline for each condition\. Table[7](https://arxiv.org/html/2609.21179#S6.T7)reports test accuracy under all eight ablation conditions\. Because the same verb types are removed from both training and test across all conditions, accuracy differences between conditions reflect differences in which subtypes destabilize learning of the remaining vocabulary, not differences in test set composition\.

Table 6:Accuracy gain over full\-data baseline for each ablation condition\. Boldface marks the single\-subtype condition producing the largest improvement in each system\.Figure 3:Accuracy gains from Table[6](https://arxiv.org/html/2609.21179#S6.T6)visualized\. The−\-4\-2 condition stands out clearly in both systems\.Table 7:Test\-set exact\-match accuracy \(%\) under each ablation condition\. Bold marks the best condition per system\.For theLST system, removing only Type 4\-2 yields the largest observed gain: 97\.73%→\\to99\.75% \(\+2\.02\), corresponding to an approximately 89% reduction in error rate\. The gain is substantially larger than removing all irregular verbs \(\+1\.22\) and matches or exceeds every other single\- or paired\-subtype removal\. For thePGT system, removing Type 4\-2 alone is still the single best condition \(97\.98%→\\to98\.98%, \+1\.00\)\.

In both systems, removing the full irregular set \(Type 4\) never yields the maximal accuracy\. Some irregular subtypes \(4\-1, 4\-3\) can be retained without cost: in the LST system, removing only Type 4\-2 while retaining 4\-1 and 4\-3 \(condition−\-4\-2\) achieves 99\.75%, and additionally removing 4\-1 as well \(condition−\-4\-1,4\-2\) performs nearly as well at 99\.74%\. This indicates that the relevant factor is not irregular statusper se, but the specific structural configuration instantiated by Type 4\-2\.

## 7Discussion

Taken together, the diagnostic and causal results support a consistent picture\. Irregularity is not uniformly detrimental to neural morphological learning\. A single low\-frequency subtype, verbs requiring gemination after a stem\-final /e/, is structurally distinct from other irregulars\. It accounts for a disproportionate share of errors \(34–48×\\timesits data share\) and, for LST, is causally responsible for most of the achievable accuracy gain from irregular\-verb ablation\.

The picture is more nuanced for the PGT architecture\. Type 4\-2 removal is directionally consistent with the LST result, remaining the best single condition, but the margin over alternative ablations is small\. We view this as a genuine and informative asymmetry and not a weakness to be smoothed over: the diagnostic finding, that Type 4\-2 is where errors concentrate, is robust across architectures and seeds, while the strength of the causal evidence for removing Type 4\-2 specifically varies between them\.

Several factors likely contribute to Type 4\-2’s structural distinctiveness\. First, consonant gemination \(small\\jpfontっ\) introduces an additional mora explicitly represented inhiragana, altering a word’s rhythmic structure in a way character\-level models must generate in the correct context\([Pimentel et al\., 2020](https://arxiv.org/html/2609.21179#bib.bib22)\)\. Second, orthographic form alone does not encode all conditioning factors \(e\.g\., lexical frequency\), so some verbs remain more error\-prone than surface similarity would suggest\. Third, at under 1% of training data, Type 4\-2 instances are too sparse for models to reliably learn the conditioning environment, while still being frequent enough, and structurally regular enough, to be systematically attempted rather than simply memorized or ignored\. This contrasts with Type 4\-3, whose single instance produces zero errors in both systems, likely because it is either trivially memorized or entirely absent from test predictions in a way that does not surface as a systematic failure\.

This has a direct implication for evaluation practice\. Aggregate accuracy near 98% in both our systems obscures an effect visible only under subtype\-level analysis; we suggest the same is plausibly true of aggregate scores on existing benchmarks\. Per\-item predictions can be regrouped by structural subclass and scored with the Disparity Ratio metric introduced in §5\.4\. Evaluation protocols for data\-efficient pretraining may therefore benefit from subgroup\-aware diagnostics of the kind proposed here, and not aggregate accuracy alone\.

#### Future Directions\.

The structural confusability effect identified here raises two natural extensions\. First, whether the same effect appears when models are evaluated zero\-shot at scale, testing whether frontier language models show the same Type 4\-2 sensitivity or whether scale eliminates the inductive bias\. Second, whether the Disparity Ratio metric generalizes beyond Japanese to other morphologically rich languages, testing whether structural confusability constitutes a domain\-general constraint on morphological learning\.

## 8Conclusion

We presented a combined diagnostic and causal analysis of Japanese past\-tense inflection\. Aggregating over five seeds and two architectures, we showed that a single irregular subtype comprising under 1% of the data accounts for 30–43% of residual errors \(a 34–48×\\timesdisparity ratio\), concentrated in gemination\-related failures\. Controlled ablation experiments show that removing only this subtype improves generalization more than removing all irregular verbs\. These results demonstrate that high aggregate accuracy can mask structurally localized failures, that not all irregularity is equally destabilizing to neural generalization, and that fine\-grained, subgroup\-aware evaluation is necessary to reveal and causally test such effects\. We argue this evaluation practice is particularly relevant to sample\-efficient, developmentally plausible language modeling\. Future work will test whether the same structural confusability effects appear in large language models, asking whether scale alone is sufficient to overcome the inductive biases identified here\.

## Limitations

Several limitations qualify our findings\. First, our ablation experiments were each run once per condition; replication across multiple training runs would strengthen the causal claim\. Second, our study focuses on a single language, a single morphological paradigm, and two Transformer\-based architectures\. Cross\-linguistic validation is necessary to determine whether comparable low\-frequency structural subtypes produce similar disparities in other morphologically rich languages, and whether larger pretrained models exhibit different sensitivity to the patterns we identify\. Third, our analysis is correlational with respect to the specific mechanism \(frequency vs\. structural complexity vs\. their interaction\) underlying Type 4\-2’s distinctiveness; the ablation results support a causal role for this subtype’s presence in training data, but do not isolate frequency from structural complexity as independent factors\. We leave controlled frequency\-matched comparisons, and evaluation under alternative input representations \(e\.g\., IPA or romanization\), to future work\.

## Ethics Statement

This work involves no sensitive data, human subjects, or personally identifiable information\. Data and models are derived from publicly available databases and SIGMORPHON shared\-task resources\.

## Acknowledgments

We thank the BabyLM 2026 workshop for their thoughtful feedback, which strengthened this work\.

## References

- Blodgettet al\.\(2020\)S\. L\. Blodgett, S\. Barocas, H\. Daumé III, and H\. WallachLanguage \(technology\) is power: a critical survey of “bias” in NLP\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 5454–5476\.External Links:[Link](https://aclanthology.org/2020.acl-main.485/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.485)Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px2.p1.1),[§5\.4](https://arxiv.org/html/2609.21179#S5.SS4.p1.2)\.
- Buolamwini and Gebru \(2018\)J\. Buolamwini and T\. GebruGender shades: intersectional accuracy disparities in commercial gender classification\.InProceedings of the 1st Conference on Fairness, Accountability and Transparency,S\. A\. Friedler and C\. Wilson \(Eds\.\),Proceedings of Machine Learning Research, Vol\.81,pp\. 77–91\.External Links:[Link](https://proceedings.mlr.press/v81/buolamwini18a.html)Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px2.p1.1),[§5\.4](https://arxiv.org/html/2609.21179#S5.SS4.p1.2)\.
- Cotterellet al\.\(2018\)R\. Cotterell, C\. Kirov, J\. Sylak\-Glassman, G\. Walther, E\. Vylomova, A\. D\. McCarthy, K\. Kann, S\. J\. Mielke, G\. Nicolai, M\. Silfverberg, D\. Yarowsky, J\. Eisner, and M\. HuldenThe CoNLL–SIGMORPHON 2018 shared task: universal morphological reinflection\.InProceedings of the CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection,M\. Hulden and R\. Cotterell \(Eds\.\),Brussels,pp\. 1–27\.External Links:[Link](https://aclanthology.org/K18-3001/),[Document](https://dx.doi.org/10.18653/v1/K18-3001)Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p2.1)\.
- Cotterellet al\.\(2017\)R\. Cotterell, C\. Kirov, J\. Sylak\-Glassman, G\. Walther, E\. Vylomova, P\. Xia, M\. Faruqui, S\. Kübler, D\. Yarowsky, J\. Eisner, and M\. HuldenCoNLL\-SIGMORPHON 2017 shared task: universal morphological reinflection in 52 languages\.InProceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection,M\. Hulden \(Ed\.\),Vancouver,pp\. 1–30\.External Links:[Link](https://aclanthology.org/K17-2001/),[Document](https://dx.doi.org/10.18653/v1/K17-2001)Cited by:[§5\.4](https://arxiv.org/html/2609.21179#S5.SS4.p1.1)\.
- Cotterellet al\.\(2016\)R\. Cotterell, C\. Kirov, J\. Sylak\-Glassman, D\. Yarowsky, J\. Eisner, and M\. HuldenThe SIGMORPHON 2016 shared Task—Morphological reinflection\.InProceedings of the 14th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology,M\. Elsner and S\. Kuebler \(Eds\.\),Berlin, Germany,pp\. 10–22\.External Links:[Link](https://aclanthology.org/W16-2002/),[Document](https://dx.doi.org/10.18653/v1/W16-2002)Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p2.1)\.
- P\. T\. Daniels and W\. Bright \(Eds\.\) \(1996\)P\. T\. Daniels and W\. Bright \(Eds\.\)The world’s writing systems\.Oxford University Press,Oxford\.Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p1.1)\.
- Goldmanet al\.\(2023\)O\. Goldman, K\. Batsuren, S\. Khalifa, A\. Arora, G\. Nicolai, R\. Tsarfaty, and E\. VylomovaSIGMORPHON–UniMorph 2023 shared task 0: typologically diverse morphological inflection\.InProceedings of the 20th SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology,G\. Nicolai, E\. Chodroff, F\. Mailhot, and Ç\. Çöltekin \(Eds\.\),Toronto, Canada,pp\. 117–125\.External Links:[Link](https://aclanthology.org/2023.sigmorphon-1.13/),[Document](https://dx.doi.org/10.18653/v1/2023.sigmorphon-1.13)Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p2.1),[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.21179#S3.p1.1),[§4](https://arxiv.org/html/2609.21179#S4.p1.1),[§5\.1](https://arxiv.org/html/2609.21179#S5.SS1.p1.1),[§5\.4](https://arxiv.org/html/2609.21179#S5.SS4.p1.1)\.
- Goldmanet al\.\(2022\)O\. Goldman, D\. Guriel, and R\. Tsarfaty\(Un\)solving morphological inflection: lemma overlap artificially inflates models’ performance\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 864–870\.External Links:[Link](https://aclanthology.org/2022.acl-short.96/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-short.96)Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p2.1),[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px1.p1.1)\.
- Kakolu Ramaraoet al\.\(2025\)A\. Kakolu Ramarao, K\. Tang, and D\. Baer\-HenneyFrequency matters: modeling irregular morphological patterns in Spanish with transformers\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 4474–4489\.External Links:[Link](https://aclanthology.org/2025.findings-acl.230/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.230),ISBN 979\-8\-89176\-256\-5Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px3.p1.1)\.
- Kann and Schütze \(2016\)K\. Kann and H\. SchützeSingle\-model encoder\-decoder with explicit morphological representation for reinflection\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),K\. Erk and N\. A\. Smith \(Eds\.\),Berlin, Germany,pp\. 555–560\.External Links:[Link](https://aclanthology.org/P16-2090/),[Document](https://dx.doi.org/10.18653/v1/P16-2090)Cited by:[§5\.4](https://arxiv.org/html/2609.21179#S5.SS4.p1.1)\.
- Kingma and Ba \(2015\)D\. P\. Kingma and J\. BaAdam: a method for stochastic optimization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5\.1](https://arxiv.org/html/2609.21179#S5.SS1.p1.1)\.
- Kubozonoet al\.\(2009\)H\. Kubozono, J\. Ito, and A\. MesterConsonant gemination in japanese loanword phonology\.InCurrent Issues in Unity and Diversity of Languages: Collection of Papers Selected from the 18th International Congress of Linguists \[CIL 18\],T\. L\. S\. of Korea \(Ed\.\),pp\. 953–973\.Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p1.1)\.
- Labrune \(2012\)L\. LabruneThe phonology of Japanese\.Oxford University Press\.Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p1.1)\.
- Makarov and Clematide \(2018\)P\. Makarov and S\. ClematideImitation learning for neural morphological string transduction\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,E\. Riloff, D\. Chiang, J\. Hockenmaier, and J\. Tsujii \(Eds\.\),Brussels, Belgium,pp\. 2877–2882\.External Links:[Link](https://aclanthology.org/D18-1314/),[Document](https://dx.doi.org/10.18653/v1/D18-1314)Cited by:[§5\.4](https://arxiv.org/html/2609.21179#S5.SS4.p1.1)\.
- Pimentelet al\.\(2020\)T\. Pimentel, J\. Valvoda, R\. H\. Maudslay, R\. Zmigrod, A\. Williams, and R\. CotterellInformation\-theoretic probing for linguistic structure\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 4609–4622\.External Links:[Link](https://aclanthology.org/2020.acl-main.420/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.420)Cited by:[§7](https://arxiv.org/html/2609.21179#S7.p3.1)\.
- Pinker \(1994\)S\. PinkerThe language instinct\.William Morrow and Company\.Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px3.p1.1)\.
- Reimers and Gurevych \(2017\)N\. Reimers and I\. GurevychReporting score distributions makes a difference: performance study of LSTM\-networks for sequence tagging\.InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,M\. Palmer, R\. Hwa, and S\. Riedel \(Eds\.\),Copenhagen, Denmark,pp\. 338–348\.External Links:[Link](https://aclanthology.org/D17-1035/),[Document](https://dx.doi.org/10.18653/v1/D17-1035)Cited by:[§5\.2](https://arxiv.org/html/2609.21179#S5.SS2.p1.1)\.
- Rumelhart and McClelland \(1986\)D\. E\. Rumelhart and J\. L\. McClellandOn learning the past tenses of english verbs\.InParallel Distributed Processing: Explorations in the Microstructure of Cognition,J\. L\. McClelland, D\. E\. Rumelhart, and the PDP Research Group \(Eds\.\),Vol\.2,pp\. 216–271\.Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px3.p1.1)\.
- Sagawaet al\.\(2020\)S\. Sagawa, P\. W\. Koh, T\. B\. Hashimoto, and P\. LiangDistributionally robust neural networks for group shifts: on the importance of regularization for worst\-case generalization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px2.p1.1),[§5\.4](https://arxiv.org/html/2609.21179#S5.SS4.p1.2)\.
- Seeet al\.\(2017\)A\. See, P\. J\. Liu, and C\. D\. ManningGet to the point: summarization with pointer\-generator networks\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),R\. Barzilay and M\. Kan \(Eds\.\),Vancouver, Canada,pp\. 1073–1083\.External Links:[Link](https://aclanthology.org/P17-1099/),[Document](https://dx.doi.org/10.18653/v1/P17-1099)Cited by:[§4](https://arxiv.org/html/2609.21179#S4.p1.1)\.
- Sproat \(2000\)R\. SproatA computational theory of writing systems\.Cambridge University Press\.Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p1.1)\.
- Vaswaniet al\.\(2017\)A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. PolosukhinAttention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30,pp\. 5998–6008\.Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2609.21179#S5.SS1.p1.1)\.
- Vylomovaet al\.\(2020\)E\. Vylomova, J\. White, E\. Salesky, S\. J\. Mielke, S\. Wu, E\. M\. Ponti, R\. H\. Maudslay, R\. Zmigrod, J\. Valvoda, S\. Toldova, F\. Tyers, E\. Klyachko, I\. Yegorov, N\. Krizhanovsky, P\. Czarnowska, I\. Nikkarinen, A\. Krizhanovsky, T\. Pimentel, L\. Torroba Hennigen, C\. Kirov, G\. Nicolai, A\. Williams, A\. Anastasopoulos, H\. Cruz, E\. Chodroff, R\. Cotterell, M\. Silfverberg, and M\. HuldenSIGMORPHON 2020 shared task 0: typologically diverse morphological inflection\.InProceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology,G\. Nicolai, K\. Gorman, and R\. Cotterell \(Eds\.\),Online,pp\. 1–39\.External Links:[Link](https://aclanthology.org/2020.sigmorphon-1.1/),[Document](https://dx.doi.org/10.18653/v1/2020.sigmorphon-1.1)Cited by:[§1](https://arxiv.org/html/2609.21179#S1.p2.1),[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.21179#S3.p1.1),[§4](https://arxiv.org/html/2609.21179#S4.p1.1),[§5\.1](https://arxiv.org/html/2609.21179#S5.SS1.p1.1),[§5\.4](https://arxiv.org/html/2609.21179#S5.SS4.p1.1)\.
- Wuet al\.\(2019\)S\. Wu, R\. Cotterell, and T\. O’DonnellMorphological irregularity correlates with frequency\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,A\. Korhonen, D\. Traum, and L\. Màrquez \(Eds\.\),Florence, Italy,pp\. 5117–5126\.External Links:[Link](https://aclanthology.org/P19-1505/),[Document](https://dx.doi.org/10.18653/v1/P19-1505)Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px3.p1.1)\.
- Yanget al\.\(2026\)X\. Yang, E\. G\. Wilcox, and C\. ArnettApples to apples? Towards comparable crosslingual language model evaluation\.arXiv preprint arXiv:2608\.25089\.External Links:[Link](https://arxiv.org/abs/2608.25089)Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px4.p1.1)\.
- Zhang \(2023\)W\. ZhangPronunciation ambiguities in Japanese kanji\.InProceedings of the Workshop on Computation and Written Language \(CAWL 2023\),K\. Gorman, R\. Sproat, and B\. Roark \(Eds\.\),Toronto, Canada,pp\. 50–60\.External Links:[Link](https://aclanthology.org/2023.cawl-1.7/),[Document](https://dx.doi.org/10.18653/v1/2023.cawl-1.7)Cited by:[§3](https://arxiv.org/html/2609.21179#S3.p1.1)\.
- Zhang \(2026a\)W\. ZhangMind your moras: orthography\-aware error analysis of neural Japanese morphological generation\.arXiv preprint arXiv:2605\.20043\.External Links:[Link](https://arxiv.org/abs/2605.20043)Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px1.p1.1)\.
- Zhang \(2026b\)W\. ZhangWhen irregularity helps: A subclass analysis of inductive bias in neural morphology\.arXiv preprint arXiv:2605\.20558\.External Links:[Link](https://arxiv.org/abs/2605.20558)Cited by:[§2](https://arxiv.org/html/2609.21179#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix AType 4\-2 Verb Inventory

The following table lists all 37 Type 4\-2 verbs in our dataset with their lemma and past\-tense forms inhiragana\.

Table A1: All 37 Type 4\-2 verbs in our dataset, with their lemma and past\-tense forms inhiragana\.