Contextual Tamil Spelling and Grammar Correction Using Progressively Fine-Tuned Sequence-to-Sequence Transformers

# Contextual Tamil Spelling and Grammar Correction UsingProgressively Fine-Tuned Sequence-to-Sequence Transformers
Source: [https://arxiv.org/html/2609.03273](https://arxiv.org/html/2609.03273)
\[ Extension = \.otf, UprightFont = \*\-regular, BoldFont = \*\-bold, ItalicFont = \*\-italic, BoldItalicFont = \*\-bolditalic\] \[ Extension = \.otf, UprightFont = \*\-regular, BoldFont = \*\-bold, ItalicFont = \*\-italic, BoldItalicFont = \*\-bolditalic\] \[ Extension = \.otf, UprightFont = \*\-regular, BoldFont = \*\-bold, ItalicFont = \*\-italic, BoldItalicFont = \*\-bolditalic\]

Karthikeyan AAffiliation:National Institute of Technology, TiruchirappalliAffiliation:Email:[vsga0026@gmail\.com](mailto:[email protected])Jaya Nirmala SAffiliation:National Institute of Technology, TiruchirappalliAffiliation:Email:[vsga0026@gmail\.com](mailto:[email protected])Sangeetha SivanesanAffiliation:National Institute of Technology, TiruchirappalliAffiliation:Email:[vsga0026@gmail\.com](mailto:[email protected])Pranav KumarAffiliation:National Institute of Technology, TiruchirappalliAffiliation:Email:[vsga0026@gmail\.com](mailto:[email protected])Bharat Jude JohnsonAffiliation:National Institute of Technology, TiruchirappalliAffiliation:Email:[vsga0026@gmail\.com](mailto:[email protected])Vishnu RamAffiliation:National Institute of Technology, TiruchirappalliAffiliation:Email:[vsga0026@gmail\.com](mailto:[email protected])

###### Abstract

Tamil spell and grammar correction is challenging because Tamil is an agglutinative low\-resource language with rich verbal morphology, complex sandhi \(phonetic transformation\) rules at word boundaries, and a script of 247 distinct letters\. Prior work targets word\-level surface errors with rule\-based methods, statisticalnn\-gram models, Minimum Edit Distance, or hybrid pipelines with a transformer re\-ranker; such methods cannot reliably handle contextual errors — subject–verb agreement, tense consistency, or cross\-word sandhi — which require sentence\-level understanding\. We propose an end\-to\-end sequence\-to\-sequence formulation and fine\-tune mT5\-small and mBART\-50 on a synthetic corpus of up to 657,720 noisy–clean Tamil sentence pairs spanning ten error categories\. Both backbones follow the same four\-stage progressive schedule, each stage targeting one weakness: surface noise \(v2\), contextual grammar \(v3\), single\-site sandhi \(v4\), and multi\-site cross\-word sandhi \(v5\)\. On a 1,000\-sentence balanced diagnostic set verified disjoint from all training data, our best model, mBART\-50 v5, reaches 69\.3% top\-1 exact\-match accuracy, with 87\.5% on sandhi and 43\.5% on subject–verb agreement\. The schedule is what produces these gains: subject–verb accuracy rises from 1\.0% to 52\.5% once contextual pairs are introduced, and sandhi from 0% to 87\.5% once multi\-site sandhi pairs are\. We additionally quantify a precision–recall trade\-off this literature has not reported: sandhi recall is paid for monotonically in identity accuracy\. Finally, Tamil\-LLaMA\-7B\-Instruct reaches 19\.0% zero\-shot and 24\.7% with three demonstrations against a 20\.0% copy baseline, showing that a Tamil\-adapted instruction model does not transfer to specialised sentence\-level correction without task\-specific supervision\.

## 1Introduction

Tamil is one of the oldest classical languages in the world, spoken by over 75 million people, with a literary tradition spanning more than 2,000 years\. It uses 12 vowels \(*uyir ezhuthukal*\), 18 consonants \(*mei ezhuthukal*\), and 216 compound characters \(*uyirmei ezhuthukal*\) — a total of 247 distinct letters\. Its agglutinative morphology packs tense, person, gender, number and honorific information into single verb endings, which means that spelling and grammatical correctness are deeply intertwined and context\-dependent\.

While spell correction has been studied extensively for English and Hindi, Tamil has received comparatively limited attention\. In English, errors are largely surface\-level and can be corrected by edit\-distance models trained on large datasets\. In Tamil, errors extend far beyond character\-level mistakes\. The words\\tamilfontபழம் \(“fruit”\) and\\tamilfontபலம் \(“strength”\) differ by one letter but mean entirely different things\. Tamil also follows strict sandhi rules: the phrase\\tamilfontஅதை கொண்டு வா \(“bring that”\) should be written\\tamilfontஅதைக் கொண்டு வா, with the consonant\\tamilfontக் inserted because of the*vallinam*\-triggered transformation\. The agglutinative nature of Tamil and the low\-resource setting together make Tamil spell correction substantially harder than for high\-resource languages\.

Existing Tamil spell\-correction approaches struggle along three dimensions\. First, they typically address only a narrow set of error types \(usually surface\-level phonetic substitution\) and ignore contextual errors\. Second, they often produce a ranked list of candidates rather than a single correction, which is unsuitable for real\-time applications\. Third, recent hybrid work that combines transformer re\-ranking with statistical retrieval treats the transformer only as a scoring function, leaving its generative capacity unused\. None of these approaches reliably handles subject–verb agreement, tense consistency or cross\-word sandhi at the sentence level\.

To address these challenges, this work:

1. 1\.an end\-to\-end sequence\-to\-sequence formulation that fine\-tunes mT5\-small and mBART\-50 to map noisy Tamil sentences directly to clean ones, rather than using transformers as candidate re\-rankers;
2. 2\.the same four\-stage progressive schedule \(v2→\\rightarrowv5\) applied to both backbones, separating the contribution of curriculum from that of architecture;
3. 3\.a noise\-generation pipeline producing 657,720 training pairs over ten error categories, including corpus\-mined agreement errors and multi\-site cross\-word sandhi violations, with a 70/20/10 split per stage;
4. 4\.evaluation on a 1,000\-sentence balanced diagnostic set verified disjoint from training data, reporting top\-1 exact\-match accuracy rather than the top\-kkaccuracy used in prior context\-blind work; and
5. 5\.a controlled comparison against Tamil\-LLaMA\([Balachandran, 2023](https://arxiv.org/html/2609.03273#bib.bib1)\)under prompt selection, and against a trivial copy baseline that establishes the floor\.

Figure[1](https://arxiv.org/html/2609.03273#S1.F1)gives the overall picture: dataset construction, progressive fine\-tuning, and single\-pass inference\.

Tamil Wikipedia\(≈\\approx17M words\)Cleaning, dedup\.,length filter \(3–50\)1\.2M cleanTamil sentencesContextual\-error generatorSurface\-noise generatorSandhi\-violation generator500,000 pairs,5 surface types575,000 pairs, 10 context\. types30k single\-site,40k multi\-site\(a\)Dataset constructionPre\-trained backbonemT5\-small / mBART\-50Stage 1 \(v2\)surface noiseStage 2 \(v3\)contextual grammarStage 3 \(v4\)single\-site sandhiStage 4 \(v5\)multi\-site sandhiFinal checkpointmBART\-50 v5500k surface575k context\+\+82\.7k aug\.30k sandhi\+\+30k replay40k multi\-site\+\+20k replayresume from v3\(b\)Progressive fine\-tuningNoisy TamilsentenceUnicode normalisation\(NFC, punctuation\)SentencePiece tokenizerta\_INlanguage tagEncoder12 layersDecoderbeam=4=4, no\-repeat 3CorrectedTamil sentence\(c\)Single\-pass inference
Figure 1:Overall system architecture\. \(a\) Clean Tamil Wikipedia text passes through three generators injecting surface, contextual and sandhi errors\. \(b\) One backbone is fine\-tuned in four stages, each resuming from the previous checkpoint and adding a new error class plus a replay sample against catastrophic forgetting; v4 and v5 both resume from v3\. \(c\) At inference a sentence is corrected in one forward pass\.
## 2Previous Work

#### Rule\-based and statistical approaches\.

[Parthasarathi et al\. \(2003\)](https://arxiv.org/html/2609.03273#bib.bib10)proposed a morphological\-analysis\-based spell checker using linguistic rules; the work reports no quantitative evaluation, and rule\-based systems in general struggle to scale to diverse error patterns or unseen words\. Solthiruthi\([Elanjelian et al\., 2004](https://arxiv.org/html/2609.03273#bib.bib4)\)and Vaani\([Rajaraman, 2014](https://arxiv.org/html/2609.03273#bib.bib12)\)are publicly available tools in the same category\.[Segar and Sarveswaran \(2015\)](https://arxiv.org/html/2609.03273#bib.bib16)proposed a bigram\-based contextual spell checker achieving 89\.13% accuracy; its two\-word context window is the main limitation, because Tamil grammatical dependencies frequently span more than two words\.[Sakuntharaj and Mahesan \(2016\)](https://arxiv.org/html/2609.03273#bib.bib13)introduced a hybrid tree\-basednn\-gram approach reporting 91% accuracy on non\-word errors only, and[Sakuntharaj and Mahesan \(2018\)](https://arxiv.org/html/2609.03273#bib.bib14)reported 98% on real\-word errors under the strong assumption that every word is individually valid\.[Kumar et al\. \(2020\)](https://arxiv.org/html/2609.03273#bib.bib7)used Minimum Edit Distance for Tamil correction\.

#### Deep\-learning and hybrid approaches\.

[Etoori et al\. \(2018\)](https://arxiv.org/html/2609.03273#bib.bib5)introduced a sequence\-to\-sequence deep\-learning model for Hindi and Telugu spell correction, generating synthetic training pairs to overcome low\-resource constraints; they did not extend the approach to Tamil and did not model Tamil\-specific phenomena such as sandhi\.[Sampath and Shanmugavel \(2023\)](https://arxiv.org/html/2609.03273#bib.bib15)combined edit distance, Soundex matching, rule\-based correction and an LSTM scoring component to achieve 95\.67% accuracy\. More recent hybrid approaches integrate MED,nn\-gram probabilities, FastText\([Bojanowski et al\., 2017](https://arxiv.org/html/2609.03273#bib.bib2)\)embeddings and pre\-trained transformer re\-rankers; while effective for ranking surface\-level candidates, such approaches do not generate corrections and cannot fix grammatical errors that the candidate generator did not propose\.[Sharma and Bhattacharyya \(2025\)](https://arxiv.org/html/2609.03273#bib.bib17)explore the closely related problem of Hindi grammatical error correction in a low\-resource setting, comparing direct\-noise injection, round\-trip translation and neural error generation — evidence that synthetic\-data strategies are the standard answer to the absence of annotated Indic GEC corpora\. Tamil\-specific neural work includes DDSpell\([Uthayamoorthy et al\., 2019](https://arxiv.org/html/2609.03273#bib.bib21)\), a context\-aware Sinhala–Tamil correction environment\([Sithamparanathan and Uthayasanker, 2019](https://arxiv.org/html/2609.03273#bib.bib19)\), and a RoBERTa spell checker\([Rajalakshmi et al\., 2023](https://arxiv.org/html/2609.03273#bib.bib11)\), all of which judge pre\-enumerated candidates rather than generate a corrected sentence\.

#### Multilingual transformers for seq2seq correction\.

[Xue et al\. \(2021\)](https://arxiv.org/html/2609.03273#bib.bib22)introduced mT5, a multilingual text\-to\-text transformer pre\-trained on the mC4 corpus across 101 languages including Tamil\.[Liu et al\. \(2020\)](https://arxiv.org/html/2609.03273#bib.bib8)and[Tang et al\. \(2020\)](https://arxiv.org/html/2609.03273#bib.bib20)developed mBART and its 50\-language extension, both pre\-trained with a denoising sequence\-to\-sequence objective conceptually close to spell correction\.[Elango and Pati \(2023\)](https://arxiv.org/html/2609.03273#bib.bib3)fine\-tune a multilingual T5 for Tamil error correction, and[Yazhmozhi VM et al\. \(2026\)](https://arxiv.org/html/2609.03273#bib.bib24)compare mBART, mT5 and NLLB\([NLLB Team et al\., 2024](https://arxiv.org/html/2609.03273#bib.bib9)\)on one error class; to our knowledge none applies these backbones to a taxonomy spanning surface, agreement and cross\-word sandhi errors under a single sentence\-level model\.

#### Mayangoli\-specific correction\.

Closest to our setting is TamilMayangoliSpell\([Yazhmozhi VM et al\., 2026](https://arxiv.org/html/2609.03273#bib.bib24)\), which also fine\-tunes multilingual seq2seq models on synthetic Tamil pairs and reports 93\.50% exact match with mT5\. It targets a single class — Mayangoli confusions among phonetically similar graphemes \(\\tamilfontல/\\tamilfontள/\\tamilfontழ,\\tamilfontர/\\tamilfontற,\\tamilfontந/\\tamilfontன/\\tamilfontண\), corresponding to our phonetic category alone — with substitutions constrained to remain dictionary\-valid, and explicitly excludes sandhi, Kuril–Nedil and non\-word errors\. It evaluates on a 10% split of the same induced distribution used for training, with one error per sentence and no no\-edit items\. Its pipeline, data and models are publicly released, which our work does not yet match\.

#### Tamil\-adapted large language models\.

Tamil\-LLaMA\([Balachandran, 2023](https://arxiv.org/html/2609.03273#bib.bib1)\)extends LLaMA\-2\-7B by adding 16K Tamil tokens, continually pre\-training on Tamil text and applying instruction tuning\. Its performance on focused spell\-correction tasks has not previously been measured\.

Compared with prior Tamil\-specific work, our approach \(i\) uses end\-to\-end seq2seq generation rather than retrieval\-plus\-ranking, \(ii\) covers ten error categories including agreement and cross\-word sandhi rather than a single confusion class, \(iii\) reports top\-1 accuracy on a balanced set verified disjoint from training data, with a 200\-item identity slice and a copy baseline fixing the floor, and \(iv\) compares directly against a Tamil\-adapted LLM\.

## 3Dataset Creation

The shortage of large\-scale annotated Tamil error corpora is the central obstacle for deep\-learning approaches to Tamil spell correction\. Existing resources do not capture the diversity of errors users actually make, particularly contextual phenomena\. We therefore construct a synthetic dataset by injecting controlled noise into clean Tamil text\. All training and evaluation pairs in this work are synthetic in this sense; no corpus of authentic annotated Tamil spelling errors is publicly available\.

### 3\.1Corpus

We use the Tamil Wikipedia dataset\([Gaurav and Wikipedia Contributors, 2019](https://arxiv.org/html/2609.03273#bib.bib6)\)as our source of clean Tamil text, comprising approximately 17 million words\. The pre\-processing pipeline removes non\-Tamil characters, English words, XML tags, parenthetical content and punctuation\. After deduplication and length filtering \(3 to 50 words per sentence\) we obtain approximately 1\.2 million clean Tamil sentences, which form the basis for noise generation\. Encyclopaedic prose differs in register, sentence length and vocabulary from the messaging, search and student\-writing contexts in which a spell checker is most used, so both the training corpus and the diagnostic set inherit a Wikipedia domain bias\. The Tamil Wikipedia content is used under the CC BY\-SA 4\.0 licence, consistent with its intended use for research purposes\.

Table 1:Surface\-level error categories in the base 500,000\-pair surface\-noise dataset\.Table 2:Contextual error categories in the 75,000\-pair contextual augmentation set\.
### 3\.2Error Categories

Our error taxonomy comprises ten categories designed to cover the range of mistakes Tamil writers commonly produce\. The categories fall into two groups: surface\-level errors that can be detected without sentence context, and contextual errors that require understanding of the surrounding words\. Table[1](https://arxiv.org/html/2609.03273#S3.T1)lists the surface\-level types used in the base 500,000\-pair dataset; Table[2](https://arxiv.org/html/2609.03273#S3.T2)lists the contextual categories introduced in the 75,000\-pair augmentation set\. Sandhi violations are structurally distinct because they operate across word boundaries, and are addressed by dedicated datasets generated separately for v4 and v5 \(Section[3\.4](https://arxiv.org/html/2609.03273#S3.SS4)\)\.

### 3\.3Noise Generation

The dataset is built in three phases\. The first produces 500,000 surface\-noise pairs, injecting noise at the word level using the weighted distribution in Table[1](https://arxiv.org/html/2609.03273#S3.T1), with single errors in 32–35% of words and double errors in 3–5% \(the per\-word maximum\)\. The second generates 75,000 contextual pairs \(Table[2](https://arxiv.org/html/2609.03273#S3.T2)\), 45% by corpus mining — substituting one verb ending in a real Wikipedia sentence with an incorrect alternative of the same tense — and the rest from templates covering subject–verb combinations the corpus underrepresents\.

### 3\.4Sandhi\-Specific Augmentation

Sandhi errors require special treatment because they cannot be modelled as word\-internal perturbations\. We construct two sandhi datasets directly from clean Wikipedia text\. The first \(30,000 pairs, used for v4\) identifies sentences containing one*vallinam*sandhi site — a word ending in\\tamilfontக்,\\tamilfontச்,\\tamilfontத் or\\tamilfontப் followed by a word beginning with the same consonant — and removes exactly one suffix to produce the noisy version\. The second \(40,000 pairs, used for v5\) extends this by allowing up to three sandhi sites to be removed per sentence: 32,425 pairs with one site removed, 6,608 with two sites and 967 with three\.

### 3\.5Dataset Composition per Model

Each stage in our progression is trained on a different combination of the above components; Table[3](https://arxiv.org/html/2609.03273#S4.T3)lists the composition\. For v3 the augmentation block adds 2,720 subject–verb template pairs, 30,000 random phonetic perturbations, 20,000 within\-word sandhi noise pairs and 30,000 identity pairs on top of the 575,000\-pair main file, yielding 657,720 effective training pairs\.

### 3\.6Train, Validation and Test Splits

Each stage corpus is split 70/20/10 into train, validation and test portions\. The split is hashed on the clean side of each pair, so a sentence and all of its noisy variants always fall in the same partition and no near\-duplicate can straddle the boundary\. The validation portion is used for checkpoint selection by token\-F1 during training; the 10% test portion is never seen during model selection\. For the v3 stage, for example, this yields 460,527 train, 131,112 validation and 66,081 test pairs\.

### 3\.7The Balanced Diagnostic Test Set

Aggregate accuracy on an in\-distribution test slice hides which linguistic phenomena a model has actually learned, so for final reporting we additionally construct a balanced diagnostic set of 1,000 sentences: 200 per category, covering phonetic confusion \(\\tamilfontழ/\\tamilfontள/\\tamilfontல,\\tamilfontற/\\tamilfontர,\\tamilfontண/\\tamilfontன\), subject–verb agreement \(gender, person, number\), pulli omission \(missing dot\\tamilfont◌்\), cross\-word*vallinam*sandhi, and identity \(correct sentences that must not be changed\)\.

Crucially, the set is not assumed to be unseen — it is verified to be\. Every candidate sentence is checked against all four training files by exact\-match hash and by 5\-gram Jaccard overlap, and any candidate above the overlap threshold is rejected and logged\.

Because 200 of the 1,000 items are identity sentences that require no edit, a trivial system that returns its input unchanged scores exactly 20\.0% by construction\. We report this copy baseline alongside all trained and prompted systems, since it is the floor any correction system must clear\.

## 4Proposed Work

We propose a progressive fine\-tuning pipeline applied to two multilingual transformer architectures: mT5\-small and mBART\-50\. The two models differ in scale, pre\-training objective and the way they signal the target language\. Applying the same four\-stage schedule to both allows the effect of the training curriculum to be separated from the effect of the architecture\.

### 4\.1Backbones

#### mT5\-small\.

mT5\([Xue et al\., 2021](https://arxiv.org/html/2609.03273#bib.bib22)\)is a multilingual text\-to\-text transformer pre\-trained with a span\-corruption objective on mC4 across 101 languages\. We use themt5\-smallcheckpoint \(300M parameters\)\. Every input is prefixed with the task instruction string"correct tamil: "followed by the noisy Tamil sentence, and the model is trained to generate the corrected sentence\.

mT5’s span\-corruption pre\-training introduced an implementation issue: the model was trained to reconstruct masked spans using sentinel tokens \(<extra\_id\_0\>, …\), and without intervention these tokens leak into generated output\. We overridedecoder\_start\_token\_idto the pad token and removeforced\_bos\_token\_id, so the decoder emits clean Tamil rather than sentinel\-prefixed sequences\. We use Adafactor\([Shazeer and Stern, 2018](https://arxiv.org/html/2609.03273#bib.bib18)\)rather than AdamW for its lower memory footprint on T5\-family models and its stability under mixed precision, where we observed NaN losses with AdamW\.

#### mBART\-50\.

mBART\([Liu et al\., 2020](https://arxiv.org/html/2609.03273#bib.bib8)\)and its 50\-language extension\([Tang et al\., 2020](https://arxiv.org/html/2609.03273#bib.bib20)\)are multilingual seq2seq models pre\-trained with a denoising objective that is conceptually very close to spell correction: given a corrupted input, reconstruct the original\. mBART\-50 uses explicit language\-ID tokens \(ta\_INfor Tamil\) prepended to both source and target sequences, which makes it particularly suitable for monolingual transformations within a single language\. We use thefacebook/mbart\-large\-50\-many\-to\-many\-mmtcheckpoint \(610M parameters\)\.

### 4\.2The Progressive Fine\-Tuning Schedule

Rather than training a single model end\-to\-end on the union of all data, we adopt a four\-stage pipeline in which each stage begins from the best checkpoint of the previous stage and targets a specific category of errors\. Progression decisions were made after qualitative inspection of error patterns on the validation set and on the diagnostic set\.

#### v2 — surface noise\.

Fine\-tunes the backbone on the 500,000\-pair surface\-noise dataset, establishing baseline ability to handle insertion, deletion, substitution and transposition errors\.

#### v3 — contextual augmentation\.

Building on v2, v3 incorporates the 75,000\-pair contextual augmentation \(Table[2](https://arxiv.org/html/2609.03273#S3.T2)\) plus 82,720 additional augmented pairs, for a total of 657,720 training pairs\. Training resumes from v2 at a lower learning rate to prevent catastrophic forgetting of v2’s surface abilities, with a cosine schedule, 3% warmup and early stopping \(patience 3\) on token\-F1\.

#### v4 — initial sandhi exposure\.

Error analysis of v3 showed that cross\-word*vallinam*sandhi was not learned from the augmented in\-word sandhi pairs alone\. We generated a 30,000\-pair dataset by removing one*vallinam*sandhi suffix from real corpus sentences and continued training from v3 for one epoch, with 30,000 replay samples from the v3 corpus\.

#### v5 — multi\-site sandhi\.

To extend v4’s single\-site exposure to sentences containing multiple sandhi sites, we constructed a 40,000\-pair dataset removing 1, 2 or 3 sites per sentence \(32,425 / 6,608 / 967 respectively\)\. Training resumed from v3 for 2 epochs with 20,000 replay samples\. Table[4](https://arxiv.org/html/2609.03273#S4.T4)summarises the hyperparameters per stage\.

Table 3:Training\-data composition by stage\. v4 and v5 both resume from v3; entries marked†are anti\-forgetting replay pairs drawn from the v3 corpus\.

### 4\.3Inference

At inference time both backbones generate corrections using beam search with 4 beams\. We useno\_repeat\_ngram\_size = 3to prevent repetitive output, length penalty 1\.0, early stopping, and a maximum generation length of 128 tokens\. For mBART\-50 we additionally setforced\_bos\_token\_idto theta\_INlanguage code to ensure the decoder generates Tamil rather than another of the 50 languages in the multilingual vocabulary\. Each sentence requires a single forward pass\.

### 4\.4Locality of Corrections

Generating the corrected sentence in a single forward pass removes that structural ceiling and yields one deterministic output per input, which suits real\-time use\. The cost, as Section[5](https://arxiv.org/html/2609.03273#S5)shows, is that corrections are no longer localised: the model can also change words that did not need changing\.

Table 4:Training hyperparameters per stage\. All runs used an NVIDIA A100\.Table 5:Per\-category top\-1 exact\-match accuracy on the 1,000\-sentence balanced diagnostic set \(200 per category\)\. The copy baseline returns its input unchanged, scoring 20\.0% by construction\. Tamil\-LLaMA is prompted, not fine\-tuned \(Section[5\.5](https://arxiv.org/html/2609.03273#S5.SS5)\); its zero\-shot result is not statistically distinguishable from that baseline \(p=0\.59p=0\.59\)\.

## 5Results and Discussion

We evaluate using top\-1 exact\-match accuracy at the sentence level on the 1,000\-sentence balanced diagnostic set described in Section[3\.7](https://arxiv.org/html/2609.03273#S3.SS7), and additionally track token\-F1 and character error rate\. Table[5](https://arxiv.org/html/2609.03273#S4.T5)reports per\-category results for all five checkpoints, the copy baseline and the two Tamil\-LLaMA conditions\.

### 5\.1Headline Comparison

mBART\-50 v5 achieves the highest overall accuracy at 69\.3%, ahead of the corresponding mT5\-small stage \(v5, 63\.1%\) by 6\.2 percentage points\. The gap is consistent with the architectural argument: mBART’s denoising pre\-training objective — reconstruct the original from a corrupted input — is essentially the task itself, whereas mT5’s span\-corruption objective is a less direct match, and mBART’s explicit language\-ID conditioning is better suited to a monolingual transformation than mT5’s prefix\-based task signalling\. mBART is also roughly twice the size, so the two effects are not fully separable here\.

### 5\.2What Each Stage Contributes

#### Contextual data is what teaches agreement\.

Subject–verb accuracy at v2 is 1\.0% — effectively zero\. The v2 model has seen half a million surface\-noise pairs and has learned to fix characters, but it has no notion that\\tamilfontஅவள் constrains the verb ending\. Introducing the contextual augmentation at v3 lifts this to 52\.5% \(2/200 to 105/200\) without any change of architecture\. v2 saw no agreement supervision at all, so the jump reflects the introduction of the category rather than an unusually large gain per training pair\.

#### Cross\-word sandhi requires cross\-word supervision, and multi\-site supervision at that\.

Sandhi accuracy is 0\.0% at both v2 and v3, despite v3’s 20,000 within\-word sandhi pairs\. Only when genuinely cross\-word pairs are introduced does the capability appear: 63\.0% at v4 \(single\-site\) and 83\.5% at v5 \(multi\-site\)\. The 20\.5\-point v4–v5 gain comes purely from allowing 1, 2 or 3 sites per training sentence, teaching the model that a correction at one position does not preclude another later on\. mBART\-50 v5 reaches 87\.5% here, the strongest per\-category result in the paper\.

### 5\.3The Sandhi–Identity Trade\-off

The most consistent pattern in Table[5](https://arxiv.org/html/2609.03273#S4.T5)is one that prior Tamil spell\-correction work does not report, because prior work does not attempt sandhi: identity accuracy falls monotonically as sandhi accuracy rises\. Across the mT5 stages, identity moves87\.0→82\.0→76\.5→72\.587\.0\\rightarrow 82\.0\\rightarrow 76\.5\\rightarrow 72\.5while sandhi moves0→0→63\.0→83\.50\\rightarrow 0\\rightarrow 63\.0\\rightarrow 83\.5\. mBART\-50 v5 shows the same relationship at a better operating point \(87\.5% sandhi at 74\.0% identity\)\.

Inspection of the failures makes the mechanism clear: almost all identity losses are the model applying sandhi to an already\-acceptable sentence, for example rewriting\\tamilfontமாணவர்கள் அமைதியாக தேர்வு எழுதினார்கள் as\\tamilfontமாணவர்கள் அமைதியாகத் தேர்வு எழுதினார்கள், or\\tamilfontஇந்த புத்தகம் as\\tamilfontஇந்தப் புத்தகம்\. These are not random corruptions: the suffixed variant is prescriptively preferred while the unsuffixed one is widely used and was labelled correct in our gold set\. What the metric records as an identity failure is therefore partly a disagreement about whether optional sandhi is obligatory — a contested question in Tamil prescriptive grammar\. Since our gold standard resolves it in one direction throughout, the identity and sandhi columns measure conformity to a single convention, not adjudicated ground truth\.

For deployment the sandhi stages are therefore a tunable rather than a strict improvement: a writing assistant that flags suggestions is well served by v5’s high sandhi recall, while a silent auto\-correct is better served by v3, which never touches a correct sentence for sandhi reasons\.

### 5\.4Per\-Category Analysis

#### Phonetic confusion\.

mBART\-50 v5 reaches 69\.5%, ahead of every mT5 stage \(50\.5–56\.5%\)\. Common confusions \(\\tamilfontழ↔\\leftrightarrow\\tamilfontல,\\tamilfontள↔\\leftrightarrow\\tamilfontல\) are handled reliably by both backbones; residual failures concentrate on rarer pairs and word\-initial positions, where less context is available\.

[Yazhmozhi VM et al\. \(2026\)](https://arxiv.org/html/2609.03273#bib.bib24)report 93\.50% exact match on the same confusion groups, against our 69\.5%\. The figures are not on the same scale: their models are trained and tested on that one class alone, with a single dictionary\-constrained substitution per sentence drawn from the same induced distribution as training, whereas ours must select among ten categories on a separately constructed set and leave 200 sentences untouched\. Their cross\-genre scores match in\-domain validation exactly, which they attribute to controlled induction flattening genre differences — a caveat our Wikipedia bias shares\.

#### Subject–verb agreement\.

This is the one category where the smaller model wins: mT5 v3 achieves 52\.5% against mBART\-50 v5’s 43\.5%\. The mT5 trajectory also declines after v3 \(52\.5→44\.5→40\.052\.5\\rightarrow 44\.5\\rightarrow 40\.0\), which indicates that the sandhi\-focused stages induce partial forgetting of agreement despite the replay sample\. This is a concrete, actionable finding: the replay fraction for v4 and v5 is currently drawn uniformly from the v3 corpus, and weighting it toward agreement pairs is the obvious next experiment\.

#### Pulli omission\.

Performance is stable in the 63–72% band, best at 72\.0% \(mBART\-50 v5\)\. Pulli restoration is a local decision, and neither the contextual nor the sandhi stages change it much\.

### 5\.5Comparison with Tamil\-LLaMA

We evaluate Tamil\-LLaMA\-7B\-Instruct\([Balachandran, 2023](https://arxiv.org/html/2609.03273#bib.bib1)\)on the same 1,000\-sentence diagnostic set in 4\-bit NF4 quantization, using the model’s native Alpaca format with English and Tamil instructions, zero\-shot and with three demonstrations drawn from the training pool and therefore disjoint from the diagnostic set by construction\. The best\-scoring prompt of each kind is selected on a stratified 100\-sentence probe; scores spanned 20\.0–25\.0%, within noise atn=100n=100, and excluding probe items moves the reported figures by 0\.2 points or less\.

Zero\-shot Tamil\-LLaMA reaches 19\.0% \(95% Wilson CI 16\.7–21\.5\), which an exact McNemar test cannot distinguish from the 20\.0% copy baseline \(135 won, 145 lost,p=0\.59p=0\.59\)\. Three\-shot prompting reaches 24\.7% \(95% CI 22\.1–27\.5\), a significant but small gain over both the copy baseline \(p=0\.006p=0\.006\) and zero\-shot \(p=1\.3×10−7p=1\.3\\times 10^\{\-7\}\), against 69\.3% for mBART\-50 v5\.

Few\-shot prompting buys restraint rather than skill: demonstrations raise identity accuracy from 27\.5% to 42\.0% and cut the rate at which the model edits an already\-correct sentence from 144/200 to 115/200, while errors left uncorrected rise from 134/800 to 258/800\. The model is competitive only on subject–verb agreement \(47\.5%\), where its pre\-training prior over Tamil verb morphology applies directly\. On pulli and sandhi it produces the required edit in 21\.0% and 17\.0% of cases but reaches exact match on only 9\.5% and 12\.5%, because it simultaneously rewrites unrelated parts of the sentence\. Prompting alone therefore does not close the gap\.

### 5\.6Key Findings

1. 1\.Curriculum matters more than any single dataset\.Each category became learnable only when supervision of exactly that kind was introduced — agreement at v3, cross\-word sandhi at v4, multi\-site sandhi at v5\. Neither more surface noise nor a larger backbone substituted for the right data\.
2. 2\.Gains in one category are not free\.Sandhi recall is bought with identity precision, monotonically — an aggregate number would have concealed this, which is the argument for category\-balanced evaluation\.
3. 3\.Prompting does not substitute for task supervision\.Tamil\-LLaMA\-7B scores 19\.0% zero\-shot and 24\.7% few\-shot against a 20\.0% copy baseline, the zero\-shot condition statistically inseparable from it\. Its accuracy concentrates in the one category where a language\-model prior transfers directly, and its dominant failure mode is rewriting text that needed no rewriting — a characterisation of prompted transfer to a narrow orthographic task, not of the generation the model was built for\.

## 6Conclusion

We present a context\-aware Tamil spell and grammar correction system based on end\-to\-end fine\-tuning of pre\-trained multilingual seq2seq transformers under a four\-stage progressive schedule\. Our best model, mBART\-50 v5 \(610M\), reaches 69\.3% top\-1 exact\-match accuracy on a 1,000\-sentence balanced diagnostic set verified disjoint from training data, with 87\.5% on cross\-word sandhi — a category no prior Tamil spell checker attempts\. The staged ablation shows that subject–verb agreement becomes learnable only with contextual supervision \(1\.0%→\\rightarrow52\.5%\) and cross\-word sandhi only with multi\-site cross\-word supervision \(0%→\\rightarrow83\.5%\), and it exposes a systematic trade\-off in which sandhi recall is paid for in identity precision\. Tamil\-LLaMA\-7B\-Instruct, prompted on the same test set, trails mBART\-50 v5 by 44\.6 points, indicating that task\-specific supervision, not model scale, is what this problem currently requires\.

#### Future scope\.

The most direct extensions are weighting the v4/v5 replay sample toward agreement pairs, and extending the multi\-site sandhi dataset to the contexts v5 still misses\. Beyond that, LoRA fine\-tuning of Tamil\-LLaMA on the same data would separate task supervision from architecture; training on a genre\-balanced corpus such as TamilCorp\([Yazhmozhi VM and Waller, 2025](https://arxiv.org/html/2609.03273#bib.bib23)\)would address the Wikipedia bias directly, and adding NLLB would test the curriculum beyond the mT5/mBART pair; a confidence threshold on optional sandhi would make the sandhi–identity trade\-off a deployment parameter; adjudication by native Tamil speakers would settle the optional\-sandhi convention; and authentic errors from social media and student writing would test generalisation beyond Wikipedia noise\.

## Limitations

#### Partial sandhi coverage\.

Sandhi failures cluster in specific syntactic patterns, notably dative\-marked nouns followed by certain verbs, which the multi\-site generator undersamples relative to their difficulty\. Rarer phenomena such as nasal assimilation are absent entirely, so the reported 87\.5% covers*vallinam*sites alone\.

## References

- Balachandran \(2023\)Abhinand Balachandran\. 2023\.Tamil\-llama: A new tamil language model based on LLaMA 2\.*arXiv preprint arXiv:2311\.05845*\.
- Bojanowski et al\. \(2017\)Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov\. 2017\.Enriching word vectors with subword information\.*Transactions of the Association for Computational Linguistics*, 5:135–146\.
- Elango and Pati \(2023\)Vaan Amuthu Elango and Peeta Basa Pati\. 2023\.Tamil text error correction with multi\-lingual T5 model\.In*2023 2nd International Conference on Vision Towards Emerging Trends in Communication and Networking Technologies \(ViTECoN\)*, pages 1–6\.
- Elanjelian et al\. \(2004\)V\. Elanjelian et al\. 2004\.Thamizha\! solthiruthi \(tamil spellchecker\)\.Firefox add\-on\.
- Etoori et al\. \(2018\)Pravallika Etoori, Manoj Chinnakotla, and Radhika Mamidi\. 2018\.Automatic spelling correction for resource\-scarce languages using deep learning\.In*Proceedings of ACL 2018, Student Research Workshop*, pages 146–152\.
- Gaurav and Wikipedia Contributors \(2019\)Gaurav and Wikipedia Contributors\. 2019\.Tamil wikipedia articles dataset\.Kaggle dataset\. Original content from Tamil Wikipedia \(CC BY\-SA 4\.0\)\.
- Kumar et al\. \(2020\)P\. Kumar, A\. Kannan, and N\. Goel\. 2020\.Design and implementation of NLP\-based spell checker for the Tamil language\.In*1st International Electronic Conference on Applied Sciences*, volume 10, page 30\.
- Liu et al\. \(2020\)Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer\. 2020\.Multilingual denoising pre\-training for neural machine translation\.*Transactions of the Association for Computational Linguistics*, 8:726–742\.
- NLLB Team et al\. \(2024\)NLLB Team, Marta R\. Costa\-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, et al\. 2024\.Scaling neural machine translation to 200 languages\.*Nature*, 630:841–846\.
- Parthasarathi et al\. \(2003\)R\. Parthasarathi, T\. V\. Geetha, and T\. Dhanabalan\. 2003\.Tamil spell checker\.In*Sixth Tamil Internet Conference*\.
- Rajalakshmi et al\. \(2023\)Ratnavel Rajalakshmi, Varsha Sharma, and Anand Kumar M\. 2023\.Context sensitive tamil language spellchecker using RoBERTa\.In*Speech and Language Technologies for Low\-Resource Languages*, pages 51–61\. Springer International Publishing\.
- Rajaraman \(2014\)Rajaraman\. 2014\.Vaani: A tamil spell checker\.Web tool\.
- Sakuntharaj and Mahesan \(2016\)R\. Sakuntharaj and S\. Mahesan\. 2016\.A novel hybrid approach to detect and correct spelling in Tamil text\.In*International Conference on Information and Automation for Sustainability \(ICIAfS\)*, pages 1–6\. IEEE\.
- Sakuntharaj and Mahesan \(2018\)R\. Sakuntharaj and S\. Mahesan\. 2018\.Detecting and correcting real\-word errors in Tamil sentences\.*Ruhuna Journal of Science*, 9\(2\)\.
- Sampath and Shanmugavel \(2023\)A\. Sampath and V\. Shanmugavel\. 2023\.Hybrid Tamil spell checker with combined character splitting\.*Concurrency and Computation: Practice and Experience*, 35\.
- Segar and Sarveswaran \(2015\)J\. Segar and K\. Sarveswaran\. 2015\.Contextual spell checking for Tamil language\.In*14th Tamil Internet Conference*, pages 1–5\.
- Sharma and Bhattacharyya \(2025\)Ujjwal Sharma and Pushpak Bhattacharyya\. 2025\.Hi\-GEC: Hindi grammar error correction in low resource scenario\.In*Proceedings of COLING 2025*, pages 6063–6075\.
- Shazeer and Stern \(2018\)Noam Shazeer and Mitchell Stern\. 2018\.Adafactor: Adaptive learning rates with sublinear memory cost\.In*Proceedings of ICML*, pages 4596–4604\.
- Sithamparanathan and Uthayasanker \(2019\)Lakshikka Sithamparanathan and Thayasivam Uthayasanker\. 2019\.A Sinhala and Tamil extension to generic environment for context\-aware correction\.In*2019 National Information Technology Conference \(NITC\)*, pages 102–106\. IEEE\.
- Tang et al\. \(2020\)Yuqing Tang, Chau Tran, Xian Li, Peng\-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan\. 2020\.Multilingual translation with extensible multilingual pretraining and fine\-tuning\.*arXiv preprint arXiv:2008\.00401*\.
- Uthayamoorthy et al\. \(2019\)Keerthana Uthayamoorthy, Kirshika Kanthasamy, Thavarasa Senthaalan, Kengatharaiyer Sarveswaran, and Gihan Dias\. 2019\.DDSpell — a data driven spell checker and suggestion generator for the Tamil language\.In*2019 19th International Conference on Advances in ICT for Emerging Regions \(ICTer\)*, pages 1–6\. IEEE\.
- Xue et al\. \(2021\)Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al\-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel\. 2021\.mT5: A massively multilingual pre\-trained text\-to\-text transformer\.In*Proceedings of NAACL\-HLT*, pages 483–498\.
- Yazhmozhi VM and Waller \(2025\)Yazhmozhi VM and Annalu Waller\. 2025\.Building a balanced Tamil corpus: EDA and lexical diversity comparison with English\.In*2025 International Conference on Data Science, Agents & Artificial Intelligence \(ICDSAAI\)*, pages 1–6\.
- Yazhmozhi VM et al\. \(2026\)Yazhmozhi VM, Annalu Waller, and Jacky Visser\. 2026\.TamilMayangoliSpell: An open\-source neural framework for context\-sensitive mayangoli error correction in Tamil\.In*Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages*, pages 42–51\. Association for Computational Linguistics\.