Loanword or Switch? The Annotation Boundary, Not the Model, Drives Kazakh-Russian Code-Switching Identification

# Loanword or Switch? The Annotation Boundary, Not the Model, Drives Kazakh–Russian Code-Switching Identification
Source: [https://arxiv.org/html/2608.00581](https://arxiv.org/html/2608.00581)
Bogdan Savelyev Independent Researcher Kazakhstan \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=Englishbogsav999@gmail\.com

\[ Extension = \.otf, UprightFont = \*\-regular, BoldFont = \*\-bold, ItalicFont = \*\-italic, BoldItalicFont = \*\-bolditalic \] \[ Extension = \.ttf, UprightFont = \*, BoldFont = \*\-Bold, ItalicFont = \*\-Oblique, BoldItalicFont = \*\-BoldOblique, Scale = 0\.9 \] \[ Extension = \.otf, UprightFont = \*, BoldFont = \*Bold, ItalicFont = \*Italic, BoldItalicFont = \*BoldItalic, Script = Cyrillic \] \[ Extension = \.otf, UprightFont = \*, BoldFont = \*Bold, ItalicFont = \*Italic, BoldItalicFont = \*BoldItalic, Script = Cyrillic \] \[ Extension = \.ttf, UprightFont = \*, BoldFont = \*\-Bold, ItalicFont = \*\-Oblique, BoldItalicFont = \*\-BoldOblique, Scale = 0\.9, Script = Cyrillic \]\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English

Loanword or Switch? The Annotation Boundary, Not the Model, Drives Kazakh–Russian Code\-Switching Identification

Bogdan SavelyevIndependent ResearcherKazakhstan\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=Englishbogsav999@gmail\.com

## \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English1Introduction

Most people in Kazakhstan speak both Kazakh and Russian, and online writing often mixes the two in one message\. We set out to build sentiment analysis for that code\-switched slice, but no open labeled set existed for the task, so we collected a corpus of over 420,000 Telegram, Kaspi, and 2GIS texts\. The scrape was unlabeled: to train sentiment only on genuine*mixed*messages \(Kazakh and Russian used in one document\), we first needed automatic language identification \(LID\) at a scale that rules out hand\-filtering every row\.

A first, letter\-based heuristic tagged a message as*mixed*whenever it contained both Kazakh\- and Russian\-specific letters\. Kazakh and Russian both use Cyrillic and share most of their alphabet, and everyday Kazakh carries many identical Russian loanwords, so almost any Kazakh sentence looked “mixed” under that rule\. Off\-the\-shelf LID \(FastText, Lingua\) makes the same mistake: it cannot tell an integrated borrowing from a real switch\.

The distinction is easy to miss on a letter\-level rule, but clear once the annotation line is spelled out\. Consider two short messages:

> Mixed:\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=RussianКурьер молодец, уақытында әкелді\. \(Russian clause \+ Kazakh clause: a genuine switch\.\) Not mixed \(loanword\):\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=RussianКачествосы жақсы, арзан\. \(Kazakh morphosyntax with an integrated Russian loan,\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianкачество \+ Kazakh suffix\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russian\-сы; no switch\.\)

Without an explicit loanword\-vs\-switch rule, both look “mixed” to a naive letter heuristic and to many off\-the\-shelf LID systems, because both contain Russian and Kazakh material written in the same Cyrillic script\.

The false\-*mixed*problem is not only weak models; it is a failure of the label definition\. We show this on a shared gold held\-out test \(n=461n\{=\}461\), comparing neural LID with simple non\-neural baselines\.

Our contributions are:

1. \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English1\.A document\-level gold LID set of 3,076 messages \(ru / kz / mixed\) with an explicit loanword\-vs\-switch annotation rule\.
2. \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English2\.A comparison ladder on that test: FastText, Lingua, HeLI \(raw / loanword\-neutral / overlapping windows\), character\-trigram NB, and fine\-tuned XLM\-R; Char\-3gram is near ceiling on monolingual classes \(ru146/150146/150, kz147/150147/150\) but recovers only114/161114/161true mixed, so the bottleneck is the label boundary rather than missing character signal\.
3. \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English3\.Corpus\-scale application: on 331,468 documents the filter keeps4\.9%4\.9\\%as mixed, against a keyword heuristic with≈1\.66%\{\\approx\}1\.66\\%precision on its Telegram mixed pool\.
4. \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English4\.A filter\-first sentiment head \(pos/neg\) trained on a mixed\-only gold pool and run only after LID; reported as downstream utility, not as the main claim\.

## \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English2Related Work

The theoretical line behind our label rule is the long\-standing distinction between borrowing and code\-switching\(Poplack,[1980](https://arxiv.org/html/2608.00581#bib.bib6)\): integrated loanwords belong to the recipient language, while productive switches do not\. Computational work often collapses that line\. Shared or ambiguous lexicon is treated as a special tag in some Arabic CS corpora\(Wintner et al\.,[2023](https://arxiv.org/html/2608.00581#bib.bib7)\); we keep a document\-level three\-way label \(*ru*/*kz*/*mixed*\) and push the same ambiguity into the annotation guideline instead\.

Language identification is surveyed extensively byJauhiainen et al\. \([2019](https://arxiv.org/html/2608.00581#bib.bib4)\)\. For short, noisy, or multilingual spans, word\-list systems such as HeLI and language\-set identification over overlapping windows\(Jauhiainen et al\.,[2015](https://arxiv.org/html/2608.00581#bib.bib3)\)remain strong, interpretable baselines\. Large\-coverage neural or n\-gram LID tools \(OpenLID, GlotLID\) work well on clean monolingual text but are known to overestimate accuracy on web data\(Burchell et al\.,[2023](https://arxiv.org/html/2608.00581#bib.bib2); Kargaran et al\.,[2023](https://arxiv.org/html/2608.00581#bib.bib5)\)\. Our setting adds a harder failure mode: Kazakh and Russian share Cyrillic, so loanword\-bearing Kazakh looks mixed to letter heuristics and to many off\-the\-shelf detectors\.

Character n\-grams have a long history for Turkic and web\-scale LID\(Baisa and Suchomel,[2012](https://arxiv.org/html/2608.00581#bib.bib1)\); we include a smoothed character\-trigram Naive Bayes baseline for that reason\. Closely related contact pairs with the same boundary problem appear in Kyrgyz–Russian work and in Kyrgyz web corpora contaminated by neighboring Turkic text; we treat those as parallel motivation rather than as datasets we reuse\.

Code\-switching workshops \(CALCS\) and variation shared tasks \(VarDial\) are natural homes for underrepresented pairs and for similar\-language confusability\. We do not frame kk–ru as dialect identification: the two are distinct languages in contact\. The scientific claim is narrower—that an explicit loanword\-vs\-switch annotation rule changes what automatic LID can honestly measure on a shared\-script pair\.

## \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English3The Gold Set

Table[\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English1](https://arxiv.org/html/2608.00581#S3.T1)summarizes the two labeled resources\. LID is document\-level ru/kz/mixed; SA is pos/neg on mixed reviews only\. Both were annotated solo \(no IAA\)\.

\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=EnglishTable 1:Gold resources\. LID classes: mixed 1077 / ru 1000 / kz 999 \(split 2691/462/461\)\. SA∼\{\\sim\}50/50 pos/neg \(split 3334/526/525; 882 synthetic\)\. Solo annotation; no IAA\.
## \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English4Experimental Setup

All LID systems are evaluated on the same held\-out gold test \(n=461n\{=\}461; train2,6912\{,\}691/ val462462/ test461461\)\. We report accuracy, macro\-F1, and mixed\-class precision/recall\. Supervised models \(FastText, Char\-3gram NB, XLM\-R LID\) train on the gold training split only\.

### HeLI ladder\.

As an interpretable non\-neural baseline we use HeLI through*heliport*\.\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English1\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English1\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English1Rust/Python port of HeLI\-OTS:[\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=Englishhttps://github\.com/ZJaume/heliport](https://github.com/ZJaume/heliport)The ladder below follows advice from Tommi Jauhiainen in email correspondence, adapted here to short kk–ru social texts and our document\-level*mixed*label\.

The first step is aloanword\-neutral re\-identificationpass: build a list of 100%\-identical Russian borrowings used in Kazakh, remove \(or score neutral\) those tokens, and re\-run HeLI \(HeLI\+neutral\)\. That cleans shared vocabulary but does not by itself detect genuine code\-switching\. On our test that prediction held: HeLI raw reaches0\.6970\.697macro\-F1, while HeLI\+neutral is slightly worse \(0\.6830\.683\), leaving8080gold\-*mixed*documents still tagged*rus*after stripping\.

To recover those switches, we then adapt HeLI’s language\-set idea\(Jauhiainen et al\.,[2015](https://arxiv.org/html/2608.00581#bib.bib3)\)down from long documents to short UGC lines: after the loanword strip, split each text into overlapping two\-, three\-, or longer word windows, run heliport on each window, and output*mixed*if more than one language appears \(optionally with a higher count threshold\)\. A smallgrid searchover window sizes andmin​\_​count\\mathrm\{min\\\_count\}tracks the typical length of switches in this data\. We implement that asHeLI\+windows; the best setting on our grid is sizes\(2,3\)\(2,3\)withmin​\_​count=1\\mathrm\{min\\\_count\}\{=\}1, which raises macro\-F1 from0\.6970\.697\(raw\) to0\.8690\.869and flips69/8069/80of the residual mixed\-as\-*rus*bucket to*mixed*\.

### Other systems\.

The character baseline is Multinomial Naive Bayes over raw\-text character trigrams \(\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=EnglishCountVectorizer\(analyzer=’char’, ngram\_range=\(3,3\)\), Laplaceα=1\\alpha\{=\}1\)\. Off\-the\-shelf comparators are FastText and Lingua \(v1/v2\)\. The neural LID model is XLM\-RoBERTa fine\-tuned on Gold LID \(XLM\-R LID v1/v2; v2 is the main neural result\)\.

Sentiment analysis \(SA\) is a separate binary head \(pos/neg\) on the filtered*mixed*review slice only \(filter\-first cascade\), evaluated on its own hold\-out \(n=525n\{=\}525\), not on the LID test\.

## \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English5Results: The Ladder

Table[\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English2](https://arxiv.org/html/2608.00581#S5.T2)reports the full comparison\.

\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=EnglishTable 2:Full model ladder\. Top: document\-level LID\. Bottom: SA on the filtered code\-switched slice only \(filter\-first cascade\)\. For SA v1, negative\-class P/R=0\.970/0\.977\{=\}0\.970/0\.977; for SA v2 both=0\.962\{=\}0\.962\. Most SA labels are LLM\-drafted\.Several patterns stand out from Table[\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English2](https://arxiv.org/html/2608.00581#S5.T2)\. First, stripping shared loanwords alone does not help HeLI: HeLI\+neutral \(macro\-F10\.6830\.683\) is slightly worse than HeLI raw \(0\.6970\.697\)\. Lexical neutralization without structure is not enough for document\-level switch detection\. Second, overlapping windows carry most of the non\-neural gain\. HeLI\+windows \(sizes2\+32\{\+\}3,min​\_​count=1\\mathrm\{min\\\_count\}\{=\}1\) jumps to0\.8690\.869macro\-F1 and mixed precision0\.9250\.925; of the8080gold\-mixed documents that stayed*rus*after the strip,6969flip to*mixed*\. Third, the character\-trigram NB is the strongest simple orthographic baseline \(0\.8800\.880macro\-F1\)\. Monolingual recall is near ceiling \(ru146/150146/150, kz147/150147/150\), yet mixed recall is only0\.7080\.708\(4747misses:20→ru20\{\\to\}\\mathrm\{ru\},27→kz27\{\\to\}\\mathrm\{kz\}\)\. The bottleneck is the loanword\-vs\-switch boundary, not missing character signal\. Fourth, off\-the\-shelf LID shows a different trade\-off: Lingua v2 reaches mixed recall0\.9880\.988but precision only0\.7680\.768, so it over\-tags mixed and is a weak corpus filter\. FastText v2 improves on v1 but stays near HeLI raw on macro\-F1 \(∼0\.71\{\\sim\}0\.71\)\. Fifth, XLM\-R LID v2 is the only system with balanced mixed P/R near0\.950\.95\(macro\-F10\.9660\.966\)\. The gap over Char\-3gram is contextual modeling of the whole message, not merely “neural vs non\-neural\.” Sixth, sentiment analysis is a downstream head on the filtered mixed slice\. XLM\-R SA v1 is best \(accuracy / macro\-F10\.9730\.973onn=525n\{=\}525\); v2 is close but lower \(0\.9620\.962\)\. These numbers measure cascade utility after LID, not a standalone sentiment benchmark, and most tone labels are LLM\-drafted\.

## \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English6Error Analysis

Of the 80 gold\-*mixed*documents that HeLI still tagged*rus*after loanword stripping, HeLI\+windows \(sizes2\+32\{\+\}3,min​\_​count=1\\mathrm\{min\\\_count\}\{=\}1\) flips 69 to*mixed*\. We inspected the remaining 11 by hand: tokens after strip, every window code from heliport, and isolated probes of the Kazakh spans\.

All 11 fail the same way\. After stripping, no 2\- or 3\-word window ever receives code*kaz*, so the language\-set vote never sees a Kazakh side and cannot emit*mixed*\. None of the cases fail because the document is too short, and none fail because a*kaz*window existed but lost the vote\. The limit sits in heliport on short windows where Kazakh is glued to Russian neighbors or written with informal spelling\.

### Single\-word Kazakh insertions \(5/11\)\.

One Kazakh content token sits inside an otherwise Russian matrix \(\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianөсек,\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianқоқыс,\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianсәлем,\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianүйренеді,\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianбұйырса\)\. With minimum window size 2, that token always shares a window with one or two Russian neighbors, and heliport returns*rus*for spans such as “\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianздесь үйренеді” or “\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianэто өсек”\. A pure Kazakh window never forms\. Probed alone at size 1, the same five tokens all map to*kaz*: the material is recognizable; windowing at≥2\{\\geq\}2is what buries it\.

### Short Kazakh spans misread as other languages \(4/11\)\.

Two\-word Kazakh chunks get a related Turkic or Slavic code, which our vote discards because we only keep*kaz*/*rus*:\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianтерең ой→\{\\to\}*kir*,\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianбіткенше күлдім→\{\\to\}*ukr*\(standard spelling; the 200\-language identifier confuses short Kazakh with Kyrgyz or Ukrainian\), and\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianқудай сактады/\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianкудай сақтасын→\{\\to\}*kir*, where informal spelling drops\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianқ/\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianұ and removes the main Kazakh cue\. Rewritten as\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianқұдай сақтасын or\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianқұдай сақтады, heliport returns*kaz*\.

### One token heliport never tags as Kazakh \(1/11\)\.

\\fontspec\_if\_script:nTF

cyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianшапшаң alone scores as*mhr*\(Meadow Mari\)\. Size\-1 windows would not recover it; the miss is in the identifier, not in the window schedule\.

### Likely label noise \(1/11\)\.

\\fontspec\_if\_script:nTF

cyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russian«Порог растет, а зп нет» has no Kazakh letters and no Kazakh words \(k​z​\_​s​i​g​n​a​l=Falsekz\\\_signal\{=\}\\mathrm\{False\}\), yet gold marks it*mixed*\. We treat it as a candidate annotation error pending a second look at the source thread\.

### Size\-1 windows as a diagnostic\.

Adding size\-1 windows \(sizes\(1,2,3\)\(1,2,3\)\) raises macro\-F1 from0\.8690\.869to0\.8850\.885and mixed recall from0\.6890\.689to0\.7760\.776, but mixed precision falls from0\.9250\.925to0\.8800\.880\(k​z→m​i​x​e​dkz\{\\to\}mixederrors8→148\{\\to\}14,r​u→m​i​x​e​dru\{\\to\}mixed1→31\{\\to\}3\)\. Single\-token votes catch insertional switches and also punish clean Kazakh that happens to contain one Russian\-looking token\. The advice that motivated the ladder was “two word, three word or longer”; size 1 is outside that definition and closer to per\-token LID\. We therefore keep\(2,3\)\(2,3\)as the HeLI\+windows rung on the ladder and report size 1 only here, as an explicit trade\-off rather than as a replacement\.

### Takeaway\.

Windowed HeLI recovers clause\-level switches; by construction it misses one\-word insertions when the minimum window is 2\. At least four of the eleven residuals are not about windows at all: heliport confuses short Kazakh with Kyrgyz \(or Mari\), and informal spelling without\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianқ/\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianұ makes that worse\. Char\-3gram fails on a related boundary: of 161 true mixed documents it reads 47 as monolingual \(20→ru20\{\\to\}\\mathrm\{ru\},27→kz27\{\\to\}\\mathrm\{kz\}\), again short insertions and loanword\-bearing spans\. Both non\-neural baselines stop at the same loanword\-vs\-switch line; XLM\-R LID crosses it with full\-message context\.

## \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English7Corpus\-Scale Application

A gold test ofn=461n\{=\}461only shows that the filter works on held\-out labels\. The practical question is what happens when the same XLM\-R LID v2 model scores a full scrape\.

We build a pooled corpus of Telegram comments, Kaspi reviews, and 2GIS reviews across Kazakhstani cities, then deduplicate and clean it into\\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=Englishmain\.csv:331,468331\{,\}468documents\. XLM\-R LID v2 assigns281,409281\{,\}409*ru*,33,69533\{,\}695*kz*, and16,36416\{,\}364*mixed*\. That is a mixed rate of4\.9%4\.9\\%\.

The contrast with the early keyword heuristic is the whole point of the resource\. On the Telegram slice alone \(241,576241\{,\}576messages\), a narrow “Kazakh\-letter \+ Russian\-letter” rule flagged27,62827\{,\}628rows as*mixed*; hand\-checking that pool kept only460460genuine switches \(precision≈1\.66%\{\\approx\}1\.66\\%\)\. A later FastText\-v2 pass over a larger scrape still sat near2%2\\%heuristic precision on its mixed predictions\. Once the gold\-trained filter replaces those rules, the corpus\-level mixed share collapses from “almost everything bilingual” to roughly one document in twenty\.

Two caveats keep the claim honest\. First, the16,36416\{,\}364mixed labels are model predictions, not a second full hand audit; gold mixed precision/recall near0\.950\.95onn=461n\{=\}461does not automatically transfer to every corpus row\. Second, the27,628/46027\{,\}628/460audit covers the Telegram heuristic pool, not the entire331331k file\. Even with those limits, the order of magnitude is stable enough to change what one can claim about code\-switching prevalence in this domain: loanword\-heavy Kazakh was inflating the mixed count, and a document\-level loanword\-vs\-switch rule cuts that inflation down\.

## \\fontspec\_if\_language:nTFENG\\addfontfeatureLanguage=English8Conclusion

False*mixed*on Kazakh–Russian social text is mostly a labeling problem, not a model problem\. Once the guideline treats integrated Russian borrowings as Kazakh and reserves*mixed*for clause\-level switches, the same held\-out gold test \(n=461n\{=\}461\) separates the systems cleanly: FastText and raw HeLI sit near0\.700\.70macro\-F1, HeLI\+windows \(language\-set adaptation with overlapping sizes2\+32\{\+\}3\) reaches0\.8690\.869, Char\-3gram NB0\.8800\.880, and XLM\-R LID v20\.9660\.966\. Char\-trigram almost never confuses monolingual ru/kz, yet recovers only70\.8%70\.8\\%of true mixed; the bottleneck is the loanword\-vs\-switch line, which full\-message context closes\.

At corpus scale the same filter changes the story you can tell about the data\. On331,468331\{,\}468documents, XLM\-R LID v2 keeps16,36416\{,\}364\(4\.9%4\.9\\%\) as*mixed*, against a Telegram keyword heuristic that flagged27,62827\{,\}628rows with only460460hand\-confirmed switches \(≈1\.66%\{\\approx\}1\.66\\%precision\)\.

We release the gold LID and SA pools, the HeLI ladder code, and the trained weights\. Next steps are a second annotator for IAA, a morphology cue \(Apertium\) for bare Russian nouns inside Kazakh, and polarity models that run only on the filtered mixed slice rather than on loanword\-inflated “mixed” noise\.

## Limitations

Annotation is solo; we report no inter\-annotator agreement\. That is the main weakness of the resource, and several external readers already flagged it\.

The16,36416\{,\}364corpus mixed labels are XLM\-R predictions, not a full hand audit\. Gold mixed P/R near0\.950\.95onn=461n\{=\}461should not be read as a guarantee on every corpus row\.

Sentiment labels are mostly LLM\-drafted then lightly audited\. We therefore treat SA metrics as cascade utility after LID, not as a primary claim about human sentiment annotation\.

Labels are document\-level\. That flattens token\-level gray zones \(bare Russian nouns with no Kazakh morphology; short insertional switches that windowed HeLI misses by construction\)\. Informal spelling without\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianқ/\\fontspec\_if\_script:nTFcyrl\\addfontfeatureScript=Cyrillic\\fontspec\_if\_language:nTFRUS\\addfontfeatureLanguage=Russianұ also pushes short Kazakh spans into neighboring Turkic codes in heliport\.

## Acknowledgments

This work began as an individual capstone project at Samsung Innovation Campus\. I thank Tommi Jauhiainen for detailed advice on the HeLI ladder \(loanword\-neutral re\-identification, overlapping windows after the strip, and grid search over window size andmin​\_​count\\mathrm\{min\\\_count\}\), and for pointing me to heliport and the language\-set identification line of work\. I also thank Anton Alekseev, Jonathan Washington, Jonathan Dunn, Maite Heredia, and Nikola Ljubešić for framing advice and for pushing the character\-trigram baseline\.

## References

- Baisa and Suchomel \(2012\)Vít Baisa and Vít Suchomel\. 2012\.Large corpora for Turkic languages and unsupervised morphological analysis\.In*Proceedings of the Eighth International Conference on Language Resources and Evaluation \(LREC’12\)*, pages 28–32\.Workshop on Language Resources for Turkic Languages; character\-trigram LID used in corpus construction\.
- Burchell et al\. \(2023\)Laurie Burchell, Alexandra Birch, Nikolay Bogoychev, and Kenneth Heafield\. 2023\.[An open dataset and model for language identification](https://doi.org/10.18653/v1/2023.acl-short.75)\.In*Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\)*, pages 865–879, Toronto, Canada\. Association for Computational Linguistics\.
- Jauhiainen et al\. \(2015\)Tommi Jauhiainen, Krister Lindén, and Heidi Jauhiainen\. 2015\.[Language set identification in noisy synthetic multilingual documents](https://doi.org/10.1007/978-3-319-18111-0_48)\.In*Computational Linguistics and Intelligent Text Processing: 16th International Conference, CICLing 2015, Proceedings, Part I*, volume 9041 of*Lecture Notes in Computer Science*, pages 633–643\. Springer\.
- Jauhiainen et al\. \(2019\)Tommi Jauhiainen, Marco Lui, Marcos Zampieri, Timothy Baldwin, and Krister Lindén\. 2019\.[Automatic language identification in texts: A survey](https://doi.org/10.1613/jair.1.11675)\.*Journal of Artificial Intelligence Research*, 65:675–782\.
- Kargaran et al\. \(2023\)Amir Hossein Kargaran, Ayyoob Imani, François Yvon, and Hinrich Schuetze\. 2023\.[GlotLID: Language identification for low\-resource languages](https://doi.org/10.18653/v1/2023.findings-emnlp.410)\.In*Findings of the Association for Computational Linguistics: EMNLP 2023*, pages 6155–6218, Singapore\. Association for Computational Linguistics\.
- Poplack \(1980\)Shana Poplack\. 1980\.[Sometimes I’ll start a sentence in Spanish y termino en español: Toward a typology of code\-switching](https://doi.org/10.1515/ling.1980.18.7-8.581)\.*Linguistics*, 18\(7–8\):581–618\.
- Wintner et al\. \(2023\)Shuly Wintner, Safaa Shehadi, Yuli Zeira, Doreen Osmelak, and Yuval Nov\. 2023\.[Shared lexical items as triggers of code switching](https://doi.org/10.1162/tacl_a_00613)\.*Transactions of the Association for Computational Linguistics*, 11:1471–1484\.