BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models

# BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models
Source: [https://arxiv.org/html/2609.09554](https://arxiv.org/html/2609.09554)
###### Abstract

We introduceBuzzASR, a collection of language\-specialized fine\-tuned Whisper models adapted for automatic speech recognition \(ASR\) in 102 languages\. Large end\-to\-end Transformer\-based ASR models such as Whisper\([Radford et al\., 2023](https://arxiv.org/html/2609.09554#bib.bib44)\)have revolutionized ASR, but most prominent models are highly multilingual\. As a result, these models often perform poorly on languages less well\-represented in their training set\. While it has long been known that effective language adaptation can be achieved through simple fine\-tuning on monolingual data, this strategy has only been applied to a small number of languages\. We massively scale up this simple approach to 102 languages covered in the FLEURS dataset, while also implementing a more complex language adaptation strategy that integrates monolingual tokenizer replacement and data augmentation using text\-only fine\-tuning\.BuzzASRmodels outperformWhisper\-large\-v3on 77 out of 102 languages, reducing character error rates \(CER\) by a factor of over 2\.8 on average\. Our models achieve state\-of\-the\-art CER among open\-source systems on 27 of 102 languages on the combined FLEURS and Common Voice test set\. Our tokenizer replacement strategy yields an average 3\.3×\\timesimprovement in compression rate \(characters per token\) over Whisper’s multilingual BPE, with gains of up to 21\.7×\\times\. We release all models, code, and detailed results: [https://lemn\-lab\.github\.io/buzz\-asr](https://lemn-lab.github.io/buzz-asr)\.

## 1Introduction

Figure 1:ComparisonBuzzASRand Whisper on automatic speech recognition across 102 languages\. For each language, we report only the bestBuzzASRmodel\.*Some outliers are excluded for presentation\.*Figure 2:Overview of our two language adaptation approaches\.Simple fine\-tuningis a minimal approach in which we fine\-tuneWhisper\-large\-v3on monolingual speech recognition data\.Full fine\-tuningis a 3\-stage pipeline involvingtokenizer replacement,multitask fine\-tuningontext\-onlyand speech recognition, and ending with speech recognition fine\-tuning\.Language technologies for the top 200 most widely spoken languages would serve approximately 90% of the global population\([Ethnologue, 2025](https://arxiv.org/html/2609.09554#bib.bib17)\)\. Currently, automatic speech recognition \(ASR\) models are highly inaccurate for all but a small fraction of those 200 languages\. This is not just due to a lack of data\. ASR datasets of≥\\geq10 hours of speech recordings with text transcriptions exist for at least 1000 languages\([Keren et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib25)\)\. While this may not seem to be very much data, it can be surprisingly effective when applying domain adaptation to large pretrained multilingual ASR models\.

Previous work was able to cut word error rates on low\-resource languages by half compared to \(at the time\) state\-of\-the\-art models, using less than 10 hours of monolingual data \([Jimerson et al\., 2018](https://arxiv.org/html/2609.09554#bib.bib22);[Liu et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib29);[Özyilmaz et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib62);[Imam et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib21),inter alia\)\. However, these contributions primarily applied this approach to a small number of domains or languages\. Thus, the reason why ASR models underperform for most widely spoken languages is not that we lack the ability to train such models; it is simply that the resources that exist have primarily been used to train either massively multilingual models or one\-off language\-specialized models\. Recently, in the text domain, it has been shown that small monolingual text generation models may outperform larger, massively multilingual models on linguistic tasks at the scale of hundreds of languages\([Chang et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib8)\)\. We take a similar approach to ASR and ask whether relatively small models dedicated to a single language can offer better performance, especially for lower\-resource languages\.

In this paper, we present a suite of 102 language\-specific ASR models, which are fine\-tuned models based on Whisper\([Radford et al\., 2023](https://arxiv.org/html/2609.09554#bib.bib44)\)\. For 77 of the 102 languages, the best of our fine\-tuned models outperforms the original Whisper\-Large\-v3 on normalized CER, with the median language seeing CER cut by 2\.18×\\timesand the mean by 2\.84×\\times\. The median absolute CER drops from 15\.5% \(Whisper zero\-shot\) to 7\.6% \(best fine\-tuned\)\. Among open\-source ASR systems, our models achieve state\-of\-the\-art CER on 27 of 102 languages on a combined evaluation drawn from FLEURS and Common Voice\. The gains are greatest where Whisper is weakest: in Whisper’s 51 worst languages by CER, the median reduction is 3\.45×\\times, and for ten languages the best fine\-tuned model cuts CER by more than 6×\\times, led by Amharic, Maltese, and Sorani Kurdish\.

While simple fine\-tuning is effective, we also explore two improvements to the naïve approach: First, we modify the Whisper tokenizer for each language\. The original tokenizer was trained on an imbalanced multilingual dataset, leading to overly long sequence lengths—and therefore worse performance and inference latencies—in poorly represented languages or languages using non\-Latin scripts\. Second, we fine\-tune the Whisper text decoder on monolingual text data, in addition to fine\-tuning both the audio encoder and text decoder on ASR data\. Many languages are poorly represented in Whisper’s pretraining data, and the limited ASR data is insufficient to learn the target language’s lexicon and grammar\. In many such cases, however, sufficient text data is available, allowing us to instill this language\-specific knowledge without requiring large amounts of audio data\.

The results of this work demonstrate that language\-specific models can perform better and more efficiently, making use of relatively small resources\. We release a suite of models for 102 languages, 27 of which are, to our knowledge, the state of the art for open\-source models\.

## 2Related Work

### 2\.1Low\-Resource & Multilingual ASR

Earlier work in low\-resource ASR primarily focused on developing monolingual or limited multilingual systems for individual low\-resource languages \([Naing et al\., 2015](https://arxiv.org/html/2609.09554#bib.bib38);[Popović et al\., 2017](https://arxiv.org/html/2609.09554#bib.bib40);[Kipyatkova and Karpov, 2016](https://arxiv.org/html/2609.09554#bib.bib26);[Bali et al\., 2013](https://arxiv.org/html/2609.09554#bib.bib5);[Upadhyaya et al\., 2017](https://arxiv.org/html/2609.09554#bib.bib53);[Ali et al\., 2014](https://arxiv.org/html/2609.09554#bib.bib1);[Luong and Vu, 2016](https://arxiv.org/html/2609.09554#bib.bib31);[Deka et al\., 2018](https://arxiv.org/html/2609.09554#bib.bib13),inter alia\)\. Later, multilingual pretraining with little to no extra supervision was found to be effective\([Watanabe et al\., 2017](https://arxiv.org/html/2609.09554#bib.bib56);[Toshniwal et al\., 2018](https://arxiv.org/html/2609.09554#bib.bib52)\)\. Motivated by these findings,[Srivastava et al\. \(2018\)](https://arxiv.org/html/2609.09554#bib.bib50)and[Diwan et al\. \(2021\)](https://arxiv.org/html/2609.09554#bib.bib15)released high\-quality speech datasets and conducted shared tasks encouraging research in multilingual ASR\.[Klejch et al\. \(2021\)](https://arxiv.org/html/2609.09554#bib.bib27)and[Mirishkar et al\. \(2021\)](https://arxiv.org/html/2609.09554#bib.bib37)achieved state\-of\-the\-art results on the released shared task datasets, applying various established data\-cleaning, augmentation, and multilingual techniques\.

In addition to large\-scale data efforts, architectural innovations[Vaswani et al\. \(2017\)](https://arxiv.org/html/2609.09554#bib.bib54)and advancements in self\-supervised learning have made it possible to build powerful scaled\-up ASR models like Whisper that achieve state\-of\-the\-art results on several benchmarks[Radford et al\. \(2023\)](https://arxiv.org/html/2609.09554#bib.bib44)\. Notably, wav2vec[Schneider et al\. \(2019\)](https://arxiv.org/html/2609.09554#bib.bib46)shows speech representations can be learned from unlabeled audio, leading to reduced dependency on labeled data\. Many have scaled this paradigm up in terms of language coverage, number of training hours, and model size[Baevski et al\. \(2020\)](https://arxiv.org/html/2609.09554#bib.bib4);[Babu et al\. \(2022\)](https://arxiv.org/html/2609.09554#bib.bib3);[Zhang et al\. \(2023\)](https://arxiv.org/html/2609.09554#bib.bib60);[Pratap et al\. \(2024\)](https://arxiv.org/html/2609.09554#bib.bib41)\.

At the time of release, Whisper achieved state\-of\-the\-art performance across languages and diverse conditions, leading to widespread adoption\([Radford et al\., 2023](https://arxiv.org/html/2609.09554#bib.bib44);[Olatunji et al\., 2023](https://arxiv.org/html/2609.09554#bib.bib39);[Bhogale et al\., 2023b](https://arxiv.org/html/2609.09554#bib.bib7);[Talafha et al\., 2023](https://arxiv.org/html/2609.09554#bib.bib51)\)\. Whisper also motivated a range of adaptation strategies for new languages, domains, and tasks including fine\-tuning\([Bhogale et al\., 2023b](https://arxiv.org/html/2609.09554#bib.bib7);[Yadavalli et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib58)\), parameter\-efficient fine\-tuning\([Kang et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib24);[Song et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib49)\), prompt\-tuning\([Ma et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib32);[Yang et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib59)\), multilingual or multitask fine\-tuning\([Bhogale et al\., 2023b](https://arxiv.org/html/2609.09554#bib.bib7);[Olatunji et al\., 2023](https://arxiv.org/html/2609.09554#bib.bib39)\), and other methods\([Zhao et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib61);[Juvekar et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib23)\)\. Given the broad adoption and the growing body of work on adapting Whisper, we experiment with Whisper models in this study\.

### 2\.2ASR Datasets

Recent efforts in creating and publicly releasing large multilingual ASR corpora have enabled the scaling of multilingual ASR models[Pratap et al\. \(2020\)](https://arxiv.org/html/2609.09554#bib.bib42);[Wang et al\. \(2021\)](https://arxiv.org/html/2609.09554#bib.bib55);[Li et al\. \(2024\)](https://arxiv.org/html/2609.09554#bib.bib28)\. However, many of these datasets primarily focus on high\-resource European languages\. To address this limitation, substantial efforts have been made to collect speech corpora for other languages and regions\. One such early effort is BABEL[Gales et al\. \(2014\)](https://arxiv.org/html/2609.09554#bib.bib18), a multilingual conversational telephone speech corpus covering 17 low\-resource languages\. Common Voice[Ardila et al\. \(2020\)](https://arxiv.org/html/2609.09554#bib.bib2)is a continuously evolving large\-scale crowdsourced effort maintained by Mozilla to collect speech data across a large number of languages\. FLEURS[Conneau et al\. \(2023\)](https://arxiv.org/html/2609.09554#bib.bib12)is annn\-way parallel speech dataset covering 102 languages and is adapted from the FLORES\-101[Goyal et al\. \(2022\)](https://arxiv.org/html/2609.09554#bib.bib20)machine translation benchmark\.[Chen et al\. \(2024\)](https://arxiv.org/html/2609.09554#bib.bib10)release an un\-transcribed multilingual speech dataset with coverage spanning more than 4000 languages\. Most recently, Omnilingual released transcribed data for 348 traditionally under\-resourced languages\([Keren et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib25)\)\.

Several region\-specific initiatives have also focused on improving speech resources for underrepresented languages\. Project Vaani[Pulikodan et al\. \(2026\)](https://arxiv.org/html/2609.09554#bib.bib43)released more than 30,000 hours of culturally relevant speech data spanning over 100 languages in the Indian subcontinent while ensuring regional diversity\. Shrutilipi[Bhogale et al\. \(2023a\)](https://arxiv.org/html/2609.09554#bib.bib6)released more than 6,000 hours of transcribed Indic speech data mined from in\-the\-wild sources\. Similarly, African Next Voices[Marivate et al\. \(2026\)](https://arxiv.org/html/2609.09554#bib.bib34), NaijaVoices[Emezue et al\. \(2025\)](https://arxiv.org/html/2609.09554#bib.bib16), and WAXAL\([Diack et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib14)\)provide high\-quality culturally rich speech datasets for several African languages\. In this work, we use Common Voice[Ardila et al\. \(2020\)](https://arxiv.org/html/2609.09554#bib.bib2)and FLEURS[Conneau et al\. \(2023\)](https://arxiv.org/html/2609.09554#bib.bib12)because they provide high\-quality multilingual speech data with broad language coverage, making them well\-suited for our work\.

### 2\.3Curse of Multilinguality

While joint multilingual training can lead to impressive results due to crosslingual transfer for ASR\([Pratap et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib41);[Keren et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib25)\), multilingual pretraining may also be limited by the “curse of multilinguality”\([Conneau et al\., 2020](https://arxiv.org/html/2609.09554#bib.bib11);[Mei et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib35);[Zhao et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib61)\)\. This refers to the phenomenon where training on a very large number of language leads to degraded performance for many of the languages\. This has been studied for text generation models, and has been found to be driven by limited model capacity and negative interference from unrelated languages\([Chang et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib9)\)\.

## 3Adaptation strategies

In this paper, we hope to challenge the reliance on the one\-model\-to\-rule\-them\-all paradigm and advocate for language\- and context\-specific training to maximize performance for each language\. We consider two approaches to adapting pretrained Whisper to a target language:Simple fine\-tuningentails fine\-tuning the entire Whisper model on aligned monolingual speech–text data using the ASR objective\.Full fine\-tuningis a more complex pipeline in which we replace Whisper’s multilingual tokenizer with a language\-specialized one and fine\-tune on a mixture of monolingual text\-only and speech–text data\.

Referring to[Figure2](https://arxiv.org/html/2609.09554#S1.F2), we introduce the basic building blocks of these pipelines below:tokenizer replacement,ASR fine\-tuning, andtext fine\-tuning\. Additionally,multitask fine\-tuningcombinestext fine\-tuningandASR fine\-tuning\.

### 3\.1Tokenizer Replacement

Whisper uses a byte\-level byte\-pair encoding \(BPE\) tokenizer\([Sennrich et al\., 2016](https://arxiv.org/html/2609.09554#bib.bib47);[Radford et al\., 2019](https://arxiv.org/html/2609.09554#bib.bib45)\)with a vocabulary of 51,865 tokens: 50,257 BPE merges and 1,608 special tokens, including 99 language tokens\. The Whisper tokenizer was trained on a corpus heavily skewed towards English\([Radford et al\., 2023](https://arxiv.org/html/2609.09554#bib.bib44)\)\. As a result, the tokenizer is well\-adapted only to the most highly represented languages, and is especially poorly suited to languages with non\-Latin and unique scripts\. Replacing the vocabulary allows the models to maximally and efficiently benefit from the language\-specific adaptation \([Minixhofer et al\., 2022](https://arxiv.org/html/2609.09554#bib.bib36),inter alia\)\.

To performtokenizer replacementfor a target language, we begin by training a new BPE tokenizer on a monolingual text corpus\. While we could simply randomly initialize a new embedding matrix for the new vocabulary, this approach fails to leverage existing structure in the pretrained model\. Instead, we adopt awarm\-start initializationof the embedding matrix, resembling[Gee et al\. \(2022\)](https://arxiv.org/html/2609.09554#bib.bib19)\. First, embeddings for any entries appearing in both tokenizers’ vocabularies are initialized using the original Whisper embedding\. Second, for any entries appearing only in the new tokenizer’s vocabulary, we tokenize that entry according to the original tokenizer, and initialize the new embedding as the average of the original tokenizer’s embeddings\.

For example, the German compound*Geschwindigkeit*\(“speed”\) receives its own dedicated entry in our per\-language German BPE\. Whisper’s original tokenizer, lacking a single merge for this compound, decomposes it into four subword tokens:G\+esch\+wind\+igkeit\. The new*Geschwindigkeit*embedding is initialized as the mean of those four subword embeddings, preserving whatever lexical information Whisper had already encoded at the level of static embeddings\.

We apply the identical warm\-start procedure to the output \(unembedding\) projection, initializing each row from \(the average of\) the corresponding Whisper output embeddings, so that both the input and output representations of a new token inherit Whisper’s encoded information\. Throughout, we keep the token\-to\-index mapping consistent between the embedding and unembedding matrices\.

### 3\.2ASR Fine\-Tuning

DuringASR fine\-tuning, the entire Whisper encoder–decoder model is fine\-tuned on aligned speech–text data using an ASR objective\. Oursimple fine\-tuningpipeline consists only of this step\.

### 3\.3Text Fine\-Tuning

Duringtext fine\-tuning, the Whisper text decoder is fine\-tuned on text\-only data using the autoregressive language modeling objective \(next\-token prediction\)\. As Whisper’s decoder incorporates cross\-attention from the encoder embeddings, we pass dummy inputs to the encoder in the form of random embeddings and freeze the encoder\.

Text fine\-tuning is primarily useful insofar as text\-only data is more abundant than speech–text data for the target language\. Generally, there are several orders of magnitude difference in dataset size\. For some languages, the only ASR data available is from FLEURS, comprising only about 40,000 English words, while we have at least 1M tokens of text data for each language—and in many cases much more data\. While in most cases we would preferASR fine\-tuninggiven sufficient data, fortunately, there are two areas in which we speculate that training the decoder alone on text data is sufficient:

First, the syntactic priors of the model can conceivably be improved throughtext fine\-tuning\. The decoder alone is responsible for outputting grammatical text, while the encoder may specialize in acoustic processing and phonological representations\.[Chang et al\. \(2026\)](https://arxiv.org/html/2609.09554#bib.bib8)showed that even with less than 1GB of text data, small models can learn syntactic information as well as much larger models trained on orders of magnitude more data\. Not only is the syntax largely dependent on the text alone, but syntax differs substantially across languages\. Text\-generation models exhibit worse cross\-lingual transfer when they have less similar syntax\([Chang et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib9)\)\. If syntax is more language\-specific and harder to learn, we reason that the decoder stands to benefit from relatively abundant text\-only input\.

Second, if we are performing tokenizer replacement, the model must adapt to the new vocabulary\. As this is likely to require extensive training, and the tokenizer is used only by the decoder, text\-only inputs are particularly valuable at this stage to prevent tokens in the new vocabulary from being under\-trained\.

### 3\.4Multitask Learning

Multitask fine\-tuningis a single fine\-tuning stage in which each minibatch contains a mixture of ASR examples \(aligned speech–text pairs\) and text\-only examples, with the fraction of ASR examples controlled by a hyperparameterα∈\(0,1\]\\alpha\\in\(0,1\], the*ASR proportion*\. Each example in the batch contributes its own loss term: the ASR loss for aligned pairs, computed against the full encoder–decoder output; and the language\-modeling loss for text\-only examples, computed by passing random embeddings as encoder input and freezing the encoder, as described in[Section3\.3](https://arxiv.org/html/2609.09554#S3.SS3)\. Settingα=1\.0\\alpha=1\.0recovers pureASR fine\-tuning; settingα=0\\alpha=0recovers puretext fine\-tuning\.

## 4Experiments

We study and compare two fine\-tuning strategies \(simple fine\-tuningandfull fine\-tuning\) to adapt a multilingual model to 102 target languages\. All our experiments are done using the 1\.55B parameterWhisper\-large\-v3model\. For each fine\-tuning strategy and language, we conduct three fine\-tuning runs with different hyperparameter configurations, resulting in 612 fine\-tuned models\. We report our hyperparameters in[AppendixB](https://arxiv.org/html/2609.09554#A2)\.

### 4\.1Data and Languages

#### Text–AudioData

We draw alignedtext–audiodata from two sources:FLEURS\([Conneau et al\., 2023](https://arxiv.org/html/2609.09554#bib.bib12)\)andCommonVoicev25\([Ardila et al\., 2020](https://arxiv.org/html/2609.09554#bib.bib2)\)\. We limit all our experiments and models to the 102 languages represented inFLEURS\. Due to compute limitations, we restrict speech–text data to 100 hours \(in English, this is approximately 1 million words\) for higher\-resource languages\.

FLEURSis annn\-way parallel corpus consisting of English Wikipedia data expert\-translated into 102 languages with audio recordings by 3 native speakers\. The training set contains approximately 10 hours per language, and the development and test splits contain on the order of 200–1000 samples per language depending on language coverage\.

CommonVoiceis a crowdsourced dataset with variable coverage of almost 300 languages\. We use only data from the 77 languages also inFLEURS\. Due to vast differences in data quantities per language and computational constraints, we cap theCommonVoicetraining data at 90 hours per language\. To equalize compute across languages with very large test sets \(e\.g\., English, Russian, Mandarin\), we cap each language’s test data at 2,000 utterances\. This cap is reached for3636of the7777languages that are included inCommonVoice\.111Languages whose CV\-test split is below 2,000 are evaluated on all available samples\.The remaining2525out of 102 languages did not have anyCommonVoicedata, so we use onlyFLEURSdata\.

FLEURSprovides normalized transcripts, which we use as\-is\. ForCommonVoice, we apply only lightweight text normalization: lowercasing, punctuation removal with a shared regular expression, and whitespace normalization\. We keep the preprocessing minimal so as to avoid altering transcript content beyond surface formatting\.

For all datasets, audio is resampled to 16 kHz, and model inputs are limited to at most 30 seconds of audio\. We use the provided test splits for both datasets for evaluation\.

#### Text Data

We sample 500k lines from thefish\-foodcorpus\([Chang et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib8)\)\. Only oneFLEURSlanguage \(Kamba\) had no data infish\-food, so we use theFLEURStraining transcripts astext\-onlydata\. Text\-only training corpora are NFC\-normalized,222This is a standard unicode normalization that combines separate diacritics and base characters into a single, precomposed character\.lowercased, punctuation\-stripped, filtered to require≥70%\\geq 70\\%characters in the language’s primary script, and deduplicated based on exact string match\.

#### 17\-Language Subset

For some supplementary experiments, we study a subset of the 102 FLEURS languages, selected heuristically to represent a wide variety of language families, scripts, and resource levels\. These languages, along with key metadata, are listed in[Table2](https://arxiv.org/html/2609.09554#A1.T2)\.

### 4\.2Tokenizer Replacement

For each language, we conducttokenizer replacementas described in[Section3\.1](https://arxiv.org/html/2609.09554#S3.SS1)\. We train a 51,865\-token byte\-level BPE tokenizer per language, matching Whisper’s original vocabulary size\. This allows us to apply warm\-start initialization without resizing the special\-token block or the decoder’s unembedding head\. We use the same pre\-tokenizer as Whisper\. Each tokenizer is trained on the same per\-languagefish\-foodcorpus used fortext fine\-tuning, so the tokenizer’s vocabulary matches the distribution the decoder will be fine\-tuned on\.

### 4\.3Multitask Learning

Formultitask fine\-tuningexperiments, we interleave text\-only and ASR samples within a single fine\-tuning stage\. At each optimization step, we sample either an aligned speech\-text sample with probabilityα\\alphaor a text\-only sample\. In a supplementary experiment, we tested different mixing ratios of text\-only and ASR samples \(10%, 20%, and 50% text data;[AppendixC](https://arxiv.org/html/2609.09554#A3)\) with a subset of languages, and found that 50% text data was optimal\. Therefore, this is the mix that we used\.

### 4\.4Experimental Configurations

For each of the 102 target languages and each fine\-tuning strategy, we train a small set of learning\-rate configurations and select, per language, the one with the lowest word error rate on the FLEURS development split\. These configurations \(cfg A,cfg B, andcfg C\) set absolute learning rates forsimple fine\-tuning\. Forfull fine\-tuning, the configurations set the embedding\- and decoder\-learning\-rate multipliers on a fixed base rate\. We determined this set of configurations heuristically following a hyperparameter search with a subset of 17 languages \([AppendixB](https://arxiv.org/html/2609.09554#A2)\)\.

#### Optimization\.

All runs use AdamW[Loshchilov and Hutter \(2019\)](https://arxiv.org/html/2609.09554#bib.bib30)with a per\-device batch size of 2 and patience\-based early stopping on FLEURS\-validation WER, evaluated every 100 steps \(patience 3 forsimple fine\-tuning, 5 forfull fine\-tuning\), for at most 6 epochs\.Simple fine\-tuninguses a 150\-step warmup and weight decay≈4×10−5\{\\approx\}4\\times 10^\{\-5\}\.Full fine\-tuninguses a 200\-step warmup and weight decay0\.010\.01\(per\-configuration values in[Table3](https://arxiv.org/html/2609.09554#A2.T3)\)\. We compute cross\-entropy loss over the label tokens, masking padding positions with−100\-100\. For text\-only batches infull fine\-tuning, the encoder receives random embeddings and is frozen during the backward pass, so gradients flow only through the decoder\.

### 4\.5Evaluation

Our primary evaluation metric is normalized character error rate \(CER\)\. We report both CER and WER in[AppendixJ](https://arxiv.org/html/2609.09554#A10), but focus our discussion on CER since word length varies widely across languages and makes WER less comparable\. All reported numbers are on the combined FLEURS \+ Common Voice test split; for the 25 languages not covered by Common Voice, we report FLEURS results only\. CER and WER are computed withjiwer\.333[https://github\.com/jitsi/jiwer](https://github.com/jitsi/jiwer)

We also report Whisper zero\-shot as a baseline\. Of the 102 target languages, 83 are supported by Whisper, which we decode using the corresponding Whisper language token\. The remaining 19 languages are not supported by Whisper; for these we prompt the model with the most closely related supported language \(e\.g\., Asturian with Spanish, Kyrgyz with Kazakh, Oriya with Bengali\)\. The full mapping is given in[AppendixA](https://arxiv.org/html/2609.09554#A1)\. All systems including the Whisper zero\-shot baseline are decoded identically with greedy search \(num\_beams=1,no\_repeat\_ngram\_size=3,repetition\_penalty=1\.2\) and no external language\-model fusion\.

### 4\.6Baselines

We compare our methods to the originalWhisper\-large\-v3models and several recently released multilingual ASR models: Omnilingual 1B and 7B\([Keren et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib25)\); Massively Multilingual Speech \(MMS,[Pratap et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib41)\); Qwen3\-ASR 1\.7B\([Shi et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib48)\); and Cohere Transcribe 2B\([Mack et al\., 2026](https://arxiv.org/html/2609.09554#bib.bib33)\)\. We report raw CER/WER for Qwen3\-ASR and Cohere Transcribe, which do not provide normalized scores\. We evaluate on all 102 languages, although not all languages are supported by every model\.

## 5Results

Figure 3:CERs for 102 languages for Whisper,BuzzASRmodels, Omnilingual models, and all other baseline models \(only the best per language is shown\)\.Table 1:Win rates against external baselines on the combined FLEURS\+CV25 test set\. Each cell is the number of languages \(of 102\) where the givenBuzzASRstrategy achieves lower CER/WER than the baseline\. Best is the better of Simple\-FT and Full\-FT per language\.In this section, we compare the performance ofBuzzASRto Whisper and other pretrained ASR models, as well as reporting the efficacy of tokenizer adaptation\. The complete CER and WER rates forBuzzASRand other pretrained models on all 102 languages are reported in[Table7](https://arxiv.org/html/2609.09554#A10.T7)and[Table8](https://arxiv.org/html/2609.09554#A10.T8), respectively\. Complete tokenizer performance metrics for the adapted tokenizers and the original Whisper tokenizer are reported in[Table6](https://arxiv.org/html/2609.09554#A9.T6)\.

### 5\.1ASR Results

As theBuzzASRmodels are fine\-tuned fromWhisper\-large\-v3, we first compare the performance between the two\.[Figure1](https://arxiv.org/html/2609.09554#S1.F1)compares CER on the combined test split forWhisper\-large\-v3vs\.BuzzASR\. We report the best performance among our two fine\-tuning strategies\. We find that theBuzzASRmodels outperform Whisper on 77 languages, reducing CER by a factor of 2\.8 on average\. The scatter plot \([Figure1](https://arxiv.org/html/2609.09554#S1.F1); top\) shows that not only do most points fall below the diagonal, indicating thatBuzzASRgenerally improves onWhisper\-large\-v3, but most points are well below the diagonal, showing substantial improvement\. Improvements are especially pronounced for languages where Whisper CER is high\. The majority of languages where we see little or no improvement are languages where Whisper CER is below 10, including English, Russian, French, Dutch, and Swedish\. This suggests that fine\-tuning is most valuable for languages under\-served by Whisper’s original training distribution, while for high\-resource languages, Whisper can remain competitive\. For a small set of languages, such as Finnish and Marathi, we find that fine\-tuning leads to substantially*worse*performance\. We infer that fine\-tuning can sometimes be unstable, and leave open the possibility that additional fine\-tuning runs could still benefit these languages\.

To determine the magnitude of these improvements, we calculate the CER reduction factor for each language, which we define as the ratio between Whisper zero\-shot CER and the lower CER achieved by eithersimple fine\-tuningorfull fine\-tuning\. For example, a value of 2 indicates thatBuzzASRachieves half the CER ofWhisper\-large\-v3, and any value greater than 1 indicates an improvement\. The distribution of CER reduction factors is visualized in[Figure1](https://arxiv.org/html/2609.09554#S1.F1)\(bottom\)\. Across 102 languages, the median reduction factor is 2\.2, meaning that the best fine\-tuned system more than halves the CER for the median language\. On the other hand, the mean reduction factor is much higher at 2\.8, reflecting large gains for several languages with very poor zero\-shot Whisper performance\. In the strongest cases, such as Amharic, Armenian, Maltese, Assamese, Uzbek, and Kabuverdianu, fine\-tuning reduces CER by more than 6×\\timesrelative to Whisper zero\-shot\.

In[Table1](https://arxiv.org/html/2609.09554#S5.T1), we show the win rates forBuzzASRmodels vs\. all external baselines\. We find thatsimple fine\-tuningimproves the CER for 65 languages, whilefull fine\-tuningis better for 69 languages\. This suggests that neither fine\-tuning method is optimal for all languages, but insteadfull fine\-tuningis more suitable for languages where the base model has a high error rate andsimple fine\-tuningis more appropriate for languages the model already makes fewer errors on\.

Next, we compare to other open\-weight models\. Importantly,BuzzASRbeats all baselines on CER for 27 languages reaching SOTA performance\.444BuzzASRoutperforms all baselines on 39 languages when using WER as an evaluation metric\.Omni\-7B is the strongest baseline, with our best model outperforming it on only 42 languages, using CER\. However, against the more comparably sized Omni\-1B,BuzzASRperforms better for 60 languages\. The bestBuzzASRmodel per language outperforms Qwen3\-ASR and Cohere Transcribe on nearly every language \(94 and 99, respectively\)\. The full distribution of CERs for most models are visualized in[Figure3](https://arxiv.org/html/2609.09554#S5.F3)\.

### 5\.2Tokenizer\-quality metrics

Next, we analyze the impact of tokenizer adaptation\.[Figure4](https://arxiv.org/html/2609.09554#S5.F4)\(top\) compares the compression rate, measured as characters per token, of our tokenizer and Whisper’s tokenizer across all the evaluated languages\. The density plot shows that our models have higher compression rates for the vast majority of languages\. The median compression rate increases from 2\.38 characters per token for Whisper to 5\.04 for our tokenizers, resulting in a median gain of 2\.18×\\times\. The effect is especially pronounced for non\-Latin scripts, where Whisper’s tokenization is highly fragmented the median compression gain is 3\.80×\\timesfor non\-Latin scripts versus 1\.93×\\timesfor Latin scripts\. This reduction in fragmentation shortens sequence lengths, thus making multilingual speech recognition more efficient\. We refer the readers to[Table5](https://arxiv.org/html/2609.09554#A8.T5), which shows the script\-wise improvement in ASR latency due to our improved tokenizers\. Furthermore, the distribution of compression rates across languages appears roughly normal, which suggests that the compression rates are more consistent cross\-linguistically\.

We also examine how the improvements in tokenization efficiency are associated with downstream ASR gains using[Figure4](https://arxiv.org/html/2609.09554#S5.F4)\(bottom\)\. Thexx\-axis shows the compression rate ratio between our tokenizer and Whisper’s tokenizer, while theyy\-axis shows the CER reduction factor obtained by theBuzzASRmodel withfull fine\-tuningrelative to Whisper zero\-shot\. The regression line shows a general positive relationship\. This indicates that languages for which our tokenizer provides larger compression gains also tend to show larger CER reductions after fine\-tuning\. However, this trend is moderate rather than deterministic, indicating that tokenization is only one of the many factors affecting the ASR performance\. The effect is especially pronounced for non\-Latin script languages—they have a higher median compression ratio than Latin\-script languages and also tend to obtain larger CER reductions\. We report additional tokenizer metrics, such as UTF\-8 coverage, vocab utilization, and boundary crossing, in[AppendixI](https://arxiv.org/html/2609.09554#A9)\.

Figure 4:Top: Distribution of tokenizer compression rates for all languages with original and adapted tokenizers\.Bottom:Relationship between improvement in CER between Whisper andfull fine\-tuningby writing system and change in compression rate after tokenizer adaptation\.
### 5\.3Supplementary Results

In the Appendix, we report several supplementary experiments\. We briefly summarize those findings below\.

#### Formultitask fine\-tuning, higher proportions of ASR training are better\.

In[AppendixC](https://arxiv.org/html/2609.09554#A3), we explore variants of themultitask fine\-tuningtraining hyperparameters\. Among the settings we compare, 50% ASR data is optimal, and increasing the amount of text\-only data yields modest improvements\.

#### Warm\-start initialization improves tokenizer adaptation\.

In[AppendixD](https://arxiv.org/html/2609.09554#A4), we compare alternative strategies for tokenizer adaptation, including a tokenizer replacement strategy where we randomly initialize new language\-specific tokens and a tokenizer adaptation strategy where we append new tokens to the end of the original tokenizer merge list\. The warm\-start tokenizer replacement strategy used in our mainfull fine\-tuningexperiments performs best\.

#### Out\-of\-domain evaluation hurts performance slightly\.

In[AppendixE](https://arxiv.org/html/2609.09554#A5), we test out\-of\-domain generalization of models trained on FLEURS alone\. We find that out\-of\-domain evaluation hurts performance slightly\. This suggests that some \(but probably not all\) of the performance gains we observe over baseline models are due to better domain alignment of our models’ train and test data, rather than language\-specific fine\-tuning\.

#### Multilingual fine\-tuning harms performance slightly\.

In[AppendixF](https://arxiv.org/html/2609.09554#A6), we compare bilingual fine\-tuning runs to monolingual runs\. For nearly all language pairs \(including related and unrelated language pairs\), we find that monolingual fine\-tuning is more effective\.

#### Multitask fine\-tuningis not helpful without tokenizer replacement\.

In[AppendixG](https://arxiv.org/html/2609.09554#A7), we disentangle the effects of tokenizer replacement andmultitask fine\-tuning, which are conflated in ourfull fine\-tuningexperiments\. We find thatfull fine\-tuningis more effective thanmultitask fine\-tuningwithout tokenizer replacement\.

## 6Discussion and Conclusion

BuzzASRmodels provide SOTA or near\-SOTA automatic speech recognition for dozens of languages\. Given their Whisper\-based architecture,BuzzASRmodels can be easily deployed in any pipeline already implemented for Whisper where strong monolingual performance is needed on one of our covered languages\. WhileBuzzASRmodels are outperformed by Omni\-7B on the majority of languages, we note thatBuzzASRgenerally outperforms Omni\-1B, which has a more comparable number of parameters\. Thus, in cases where Omni\-7B is too large to deploy,BuzzASRoffers competitive performance for the majority of languages we cover\.

In addition to providing a new open\-source model suite, our experiments demonstrate the efficacy of language\-specialized fine\-tuning at scale\. Given the benefits of language\-specialized models to the global community, our work motivates similar approaches with other ASR model architectures, as well as with text models and multimodal models\.

Finally, our tokenizer replacement approach reveals the extent to which multilingual tokenizers under\-deliver, particularly for non\-Latin scripts\. We show that language\-adapted tokenizers not only improve ASR performance disproportionately in cases where the original tokenizer was ill\-suited, they also decrease inference latency and compute by reducing sequence lengths\.

#### Future Work

There are significant opportunities for future improvements toBuzzASR\. First, we believe through further hyperparameter\-tuning for vocabulary adaptation and multi\-task learning we could achieve better performance\. We focused on a limited set of design choices to optimize, and most significantly improved performance\. By further tuning the multitask learning protocol, we could better make use of abundant text\-only data\.

Second, there is a wide array of established optimizations for ASR that we do not yet exploit\. These include data augmentation through the addition of noise or text\-to\-speech, interpolation with monolingual text\-only LMs, and training simultaneously on closely related languages\.

Finally, we focus on a relatively small set of languages\. As we note above, there is sufficient fine\-tuning data for roughly 1000 languages\. We hope to expand our suite ofBuzzASRmodels, especially to include under\-represented and under\-studied languages\. As ASR can be used to increase text data for training language models, creating better tools for these languages can lead to improvements in other language technologies\.

## Limitations

In this paper, we only conduct fine\-tuning experiments on Whisper\. However, Omni\-7B still shows state\-of\-the\-art performance for most languages\. We anticipate that fine\-tuning Omnilingual would result in even better performance\. We focused on Whisper because it is very widely used and integrated into many existing pipelines\. Replacing Whisper with one of our models would require minimal adjustments for users\. Furthermore,Whisper\-large\-v3is significantly smaller than Omni\-7B, with only 1\.6B parameters\. Focusing on smaller models results in more efficient and economical models, and we note thatBuzzASRmodels outperform the similarly sized Omni\-1B in the vast majority of cases\. While we aimed to produce performant models, the main goal of the paper was to demonstrate the efficacy of our fine\-tuning approach for models optimized for a single language\. Future work could explore using other models as starting points\.

We used only a small fraction of the existing ASR data\. We limited the data we use to 100 hours of data, primarily due to computational resource limitations\. For the languages we fine\-tuned models for in this paper, we could achieve even higher performance by using more of the available data\. There are also many languages which we did not train models for, but for which there is a significant amount of data\. We hope this approach can be applied to a wider set of languages\.

We use the test splits from our training datasets \(FLEURS and Common Voice\) for evaluation\. Due to data limitations, we do not evaluate performance across different domains or dialects, as such datasets are not available for many of the languages we consider in this paper\. A supplementary analysis[AppendixE](https://arxiv.org/html/2609.09554#A5)reveals that out\-of\-domain generalization may be a challenge for adapted ASR models resembling theBuzzASRreleases\.

We also note that monolingual models may provide better performance in controlled evaluation, but might not be appropriate for all applications\. Monolingual models are best when one already knows the language that is being used, which may limit their utility\. If coverage of multiple languages is required, deploying multiple monolingual models will require substantially more compute resources than deploying a single multilingual model\. This is especially true given we perform full fine\-tuning, rather than parameter\-efficient fine\-tuning, which improves memory efficiency in exchange for some losses in performance on ASR\([Liu et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib29)\)\. Additionally, multilingual models may better be able to handle code switching or multi\-dialectal speech, though this may require additional specialized training or fine\-tuning\.

## Acknowledgments

This work used resources available through the National Research Platform \(NRP\) at the University of California, San Diego\([Weitzel et al\., 2025](https://arxiv.org/html/2609.09554#bib.bib57)\)\. NRP has been developed, and is supported in part, by funding from National Science Foundation, from awards 1730158, 1540112, 1541349, 1826967, 2112167, 2100237, and 2120019, as well as additional funding from community partners\. We also thank CoreWeave for providing compute for this project\.

## References

- Ali et al\. \(2014\)Ahmed Ali, Yifan Zhang, Patrick Cardinal, Najim Dahak, Stephan Vogel, and James Glass\. 2014\.[A complete KALDI recipe for building Arabic speech recognition systems](https://doi.org/10.1109/SLT.2014.7078629)\.In*2014 IEEE Spoken Language Technology Workshop \(SLT\)*, pages 525–529\.
- Ardila et al\. \(2020\)Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber\. 2020\.[Common Voice: A massively\-multilingual speech corpus](https://aclanthology.org/2020.lrec-1.520/)\.In*Proceedings of the Twelfth Language Resources and Evaluation Conference*, pages 4218–4222, Marseille, France\. European Language Resources Association\.
- Babu et al\. \(2022\)Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, Alexei Baevski, Alexis Conneau, and Michael Auli\. 2022\.[XLS\-R: Self\-supervised Cross\-lingual Speech Representation Learning at Scale](https://doi.org/10.21437/Interspeech.2022-143)\.In*Interspeech 2022*, pages 2278–2282\.
- Baevski et al\. \(2020\)Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli\. 2020\.[wav2vec 2\.0: A framework for self\-supervised learning of speech representations](https://proceedings.neurips.cc/paper_files/paper/2020/file/92d1e1eb1cd6f9fba3227870bb6d7f07-Paper.pdf)\.In*Advances in Neural Information Processing Systems*, volume 33, pages 12449–12460\. Curran Associates, Inc\.
- Bali et al\. \(2013\)Kalika Bali, Sunayana Sitaram, Sebastien Cuendet, and Indrani Medhi\. 2013\.[A Hindi speech recognizer for an agricultural video search application](https://doi.org/10.1145/2442882.2442889)\.In*Proceedings of the 3rd ACM Symposium on Computing for Development*, ACM DEV ’13, New York, NY, USA\. Association for Computing Machinery\.
- Bhogale et al\. \(2023a\)Kaushal Bhogale, Abhigyan Raman, Tahir Javed, Sumanth Doddapaneni, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh M\. Khapra\. 2023a\.[Effectiveness of mining audio and text pairs from public data for improving ASR systems for low\-resource languages](https://doi.org/10.1109/ICASSP49357.2023.10096933)\.In*ICASSP 2023 \- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 1–5\.
- Bhogale et al\. \(2023b\)Kaushal Bhogale, Sai Sundaresan, Abhigyan Raman, Tahir Javed, Mitesh M\. Khapra, and Pratyush Kumar\. 2023b\.[Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR](https://doi.org/10.21437/Interspeech.2023-2589)\.In*Interspeech 2023*, pages 4384–4388\.
- Chang et al\. \(2026\)Tyler A\. Chang, Catherine Arnett, Zhuowen Tu, and Benjamin Bergen\. 2026\.[Goldfish: Monolingual language models for 350 languages](https://doi.org/10.63317/5ceec3hhv4d5)\.*Proceedings of the Fifteenth Language Resources and Evaluation Conference*, pages 3750–3781\.
- Chang et al\. \(2024\)Tyler A\. Chang, Catherine Arnett, Zhuowen Tu, and Benjamin K\. Bergen\. 2024\.[When is multilinguality a curse? Language modeling for 250 high\- and low\-resource languages](https://doi.org/10.18653/v1/2024.emnlp-main.236)\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 4074–4096, Miami, Florida, USA\. Association for Computational Linguistics\.
- Chen et al\. \(2024\)William Chen, Wangyou Zhang, Yifan Peng, Xinjian Li, Jinchuan Tian, Jiatong Shi, Xuankai Chang, Soumi Maiti, Karen Livescu, and Shinji Watanabe\. 2024\.[Towards robust speech representation learning for thousands of languages](https://doi.org/10.18653/v1/2024.emnlp-main.570)\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 10205–10224, Miami, Florida, USA\. Association for Computational Linguistics\.
- Conneau et al\. \(2020\)Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov\. 2020\.[Unsupervised cross\-lingual representation learning at scale](https://doi.org/10.18653/v1/2020.acl-main.747)\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 8440–8451, Online\. Association for Computational Linguistics\.
- Conneau et al\. \(2023\)Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna\. 2023\.[FLEURS: Few\-shot learning evaluation of universal representations of speech](https://doi.org/10.1109/SLT54892.2023.10023141)\.In*2022 IEEE Spoken Language Technology Workshop \(SLT\)*, pages 798–805\.
- Deka et al\. \(2018\)Barsha Deka, S R Nirmala, and Samudravijaya K\. 2018\.[Development of Assamese Continuous Speech Recognition System](https://doi.org/10.21437/SLTU.2018-46)\.In*6th Workshop on Spoken Language Technologies for Under\-Resourced Languages \(SLTU 2018\)*, pages 220–224\.
- Diack et al\. \(2026\)Abdoulaye Diack, Perry Nelson, Kwaku Agbesi, Angela Nakalembe, MohamedElfatih MohamedKhair, Vusumuzi Dube, Tavonga Siyavora, Subhashini Venugopalan, Jason Hickey, Uche Okonkwo, Abhishek Bapna, Isaac Wiafe, Raynard Dodzi Helegah, Elikem Doe Atsakpo, Charles Nutrokpor, Fiifi Baffoe Payin Winful, Kafui Kwashie Solaga, Jamal\-Deen Abdulai, Akon Obu Ekpezu, and 24 others\. 2026\.[WAXAL: A large\-scale multilingual African language speech corpus](https://arxiv.org/abs/2602.02734)\.
- Diwan et al\. \(2021\)Anuj Diwan, Rakesh Vaideeswaran, Sanket Shah, Ankita Singh, Srinivasa Raghavan, Shreya Khare, Vinit Unni, Saurabh Vyas, Akash Rajpuria, Chiranjeevi Yarra, Ashish Mittal, Prasanta Kumar Ghosh, Preethi Jyothi, Kalika Bali, Vivek Seshadri, Sunayana Sitaram, Samarth Bharadwaj, Jai Nanavati, Raoul Nanavati, and Karthik Sankaranarayanan\. 2021\.[MUCS 2021: Multilingual and Code\-Switching ASR Challenges for Low Resource Indian Languages](https://doi.org/10.21437/Interspeech.2021-1339)\.In*Interspeech 2021*, pages 2446–2450\.
- Emezue et al\. \(2025\)Chris Emezue, The NaijaVoices Community, Busayo Awobade, Abraham Toluwase Owodunni, Handel Emezue, Gloria Monica Tobechukwu Emezue, Nefertiti Nneoma Emezue, Sewade Ogun, Bunmi Akinremi, David Ifeoluwa Adelani, and Chris Pal\. 2025\.[The NaijaVoices Dataset: Cultivating Large\-Scale, High\-Quality, Culturally\-Rich Speech Data for African Languages](https://doi.org/10.21437/Interspeech.2025-1104)\.In*Interspeech 2025*, pages 1338–1342\.
- Ethnologue \(2025\)Ethnologue\. 2025\.What are the top 200 most spoken languages?[https://www\.ethnologue\.com/insights/ethnologue200/](https://www.ethnologue.com/insights/ethnologue200/)\.Ethnologue Insights\.
- Gales et al\. \(2014\)Mark J\. F\. Gales, Kate M\. Knill, Anton Ragni, and Shakti P\. Rath\. 2014\.[Speech recognition and keyword spotting for low\-resource languages: Babel project research at CUED](https://www.isca-archive.org/sltu_2014/gales14_sltu.html)\.In*4th Workshop on Spoken Language Technologies for Under\-Resourced Languages \(SLTU 2014\)*, pages 16–23\.
- Gee et al\. \(2022\)Leonidas Gee, Andrea Zugarini, Leonardo Rigutini, and Paolo Torroni\. 2022\.[Fast vocabulary transfer for language model compression](https://doi.org/10.18653/v1/2022.emnlp-industry.41)\.In*Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track*, pages 409–416, Abu Dhabi, UAE\. Association for Computational Linguistics\.
- Goyal et al\. \(2022\)Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng\-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan\. 2022\.[The Flores\-101 evaluation benchmark for low\-resource and multilingual machine translation](https://doi.org/10.1162/tacl_a_00474)\.*Transactions of the Association for Computational Linguistics*, 10:522–538\.
- Imam et al\. \(2025\)Sukairaj Hafiz Imam, Babangida Sani, Dawit Ketema Gete, Bedru Yimam Ahmed, Ibrahim Said Ahmad, Idris Abdulmumin, Seid Muhie Yimam, Muhammad Yahuza Bello, and Shamsuddeen Hassan Muhammad\. 2025\.[Automatic speech recognition for African low\-resource languages: Challenges and future directions](https://doi.org/10.18653/v1/2025.africanlp-1.13)\.In*Proceedings of the Sixth Workshop on African Natural Language Processing \(AfricaNLP 2025\)*, pages 89–94, Vienna, Austria\. Association for Computational Linguistics\.
- Jimerson et al\. \(2018\)Robert Jimerson, Karthik Simha, Raymond Ptucha, and Emily Prudhommeaux\. 2018\.[Improving ASR output for endangered language documentation](https://doi.org/10.21437/SLTU.2018-39)\.In*Proceedings of the 6th Workshop on Spoken Language Technologies for Under\-Resourced Languages \(SLTU 2018\)*, pages 187–191\. International Speech Communication Association\.
- Juvekar et al\. \(2026\)Kush Juvekar, Kavya Manohar, Aditya Srinivas Menon, Arghya Bhattacharya, and Kumarmanas Nethil\. 2026\.[Vividh\-ASR: A complexity\-tiered benchmark and optimization dynamics for robust Indic speech recognition](https://arxiv.org/abs/2605.13087)\.*Preprint*, arXiv:2605\.13087\.
- Kang et al\. \(2024\)Ji\-Hun Kang, Jae\-Hong Lee, Mun\-Hak Lee, and Joon\-Hyuk Chang\. 2024\.[Whisper Multilingual Downstream Task Tuning Using Task Vectors](https://doi.org/10.21437/Interspeech.2024-513)\.In*Interspeech 2024*, pages 2385–2389\.
- Keren et al\. \(2025\)Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang, Ife Adebara, Michael Auli, Can Balioglu, Kevin Chan, Chierh Cheng, Joe Chuang, Caley Droof, Mark Duppenthaler, Paul\-Ambroise Duquenne, Alexander Erben, Cynthia Gao, Gabriel Mejia Gonzalez, Kehan Lyu, and 13 others\. 2025\.[Omnilingual ASR: Open\-source multilingual speech recognition for 1600\+ languages](https://arxiv.org/abs/2511.09690)\.*Preprint*, arXiv:2511\.09690\.
- Kipyatkova and Karpov \(2016\)Irina Kipyatkova and Alexey Karpov\. 2016\.[DNN\-based acoustic modeling for Russian speech recognition using Kaldi](https://doi.org/10.1007/978-3-319-43958-7_29)\.In*Speech and Computer*, pages 246–253, Cham\. Springer International Publishing\.
- Klejch et al\. \(2021\)Ondřej Klejch, Electra Wallington, and Peter Bell\. 2021\.[The CSTR System for Multilingual and Code\-Switching ASR Challenges for Low Resource Indian Languages](https://doi.org/10.21437/Interspeech.2021-1035)\.In*Interspeech 2021*, pages 2881–2885\.
- Li et al\. \(2024\)Song Li, Yongbin You, Xuezhi Wang, Zhengkun Tian, Ke Ding, and Guanglu Wan\. 2024\.[MSR\-86K: An Evolving, Multilingual Corpus with 86,300 Hours of Transcribed Audio for Speech Recognition Research](https://doi.org/10.21437/Interspeech.2024-890)\.In*Interspeech 2024*, pages 1245–1249\.
- Liu et al\. \(2024\)Yunpeng Liu, Xukui Yang, and Dan Qu\. 2024\.[Exploration of Whisper fine\-tuning strategies for low\-resource ASR](https://doi.org/10.1186/s13636-024-00349-3)\.*EURASIP Journal on Audio, Speech, and Music Processing*, 2024\(1\):31\.
- Loshchilov and Hutter \(2019\)Ilya Loshchilov and Frank Hutter\. 2019\.[Decoupled weight decay regularization](https://openreview.net/forum?id=Bkg6RiCqY7)\.In*International Conference on Learning Representations*\.
- Luong and Vu \(2016\)Hieu\-Thi Luong and Hai\-Quan Vu\. 2016\.[A non\-expert Kaldi recipe for Vietnamese speech recognition system](https://aclanthology.org/W16-5207/)\.In*Proceedings of the Third International Workshop on Worldwide Language Service Infrastructure and Second Workshop on Open Infrastructures and Analysis Frameworks for Human Language Technologies \(WLSI/OIAF4HLT2016\)*, pages 51–55, Osaka, Japan\. The COLING 2016 Organizing Committee\.
- Ma et al\. \(2024\)Hao Ma, Zhiyuan Peng, Mingjie Shao, Jing Li, and Ju Liu\. 2024\.[Extending Whisper with prompt tuning to target\-speaker ASR](https://doi.org/10.1109/ICASSP48485.2024.10447492)\.In*ICASSP 2024 \- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 12516–12520\.
- Mack et al\. \(2026\)Julian Mack, Ekagra Ranjan, Walter Beller\-Morales, Bharat Venkitesh, and Pierre Richemond\. 2026\.[cohere\-transcribe\-03\-2026 \(revision d96e814\)](https://doi.org/10.57967/hf/8653)\.
- Marivate et al\. \(2026\)Vukosi Marivate, Kayode Olaleye, Sitwala Mundia, Andinda Bakainga, Unarine Netshifhefhe, Mahmooda Milanzie, Tsholofelo Hope Mogale, Thapelo Sindane, Zainab Abdulrasaq, Kesego Mokgosi, Chijioke Okorie, Nia Zion Van Wyk, Graham Morrissey, Dale Dunbar, Francois Smit, Tsosheletso Chidi, Rooweither Mabuya, Andiswa Bukula, Respect Mlambo, and 4 others\. 2026\.[Swivuriso: The South African next voices multilingual speech dataset](https://arxiv.org/abs/2512.02201)\.*Preprint*, arXiv:2512\.02201\.
- Mei et al\. \(2026\)Yuxiang Mei, Delai Qiu, Shengping Liu, Jiaen Liang, and Yanhua Long\. 2026\.[Zipper\-LoRA: Dynamic parameter decoupling for speech\-LLM based multilingual speech recognition](https://arxiv.org/abs/2603.17558)\.*Preprint*, arXiv:2603\.17558\.
- Minixhofer et al\. \(2022\)Benjamin Minixhofer, Fabian Paischer, and Navid Rekabsaz\. 2022\.[WECHSEL: Effective initialization of subword embeddings for cross\-lingual transfer of monolingual language models](https://doi.org/10.18653/v1/2022.naacl-main.293)\.In*Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 3992–4006, Seattle, United States\. Association for Computational Linguistics\.
- Mirishkar et al\. \(2021\)Ganesh Mirishkar, Aditya Yadavalli, and Anil Kumar Vuppala\. 2021\.[An investigation of hybrid architectures for low resource multilingual speech recognition system in Indian context](https://aclanthology.org/2021.icon-main.25/)\.In*Proceedings of the 18th International Conference on Natural Language Processing \(ICON\)*, pages 205–212, National Institute of Technology Silchar, Silchar, India\. NLP Association of India \(NLPAI\)\.
- Naing et al\. \(2015\)Hay Mar Soe Naing, Aye Mya Hlaing, Win Pa Pa, Xinhui Hu, Ye Kyaw Thu, Chiori Hori, and Hisashi Kawai\. 2015\.[A Myanmar large vocabulary continuous speech recognition system](https://doi.org/10.1109/APSIPA.2015.7415529)\.In*2015 Asia\-Pacific Signal and Information Processing Association Annual Summit and Conference \(APSIPA\)*, pages 320–327\.
- Olatunji et al\. \(2023\)Tobi Olatunji, Tejumade Afonja, Aditya Yadavalli, Chris Chinenye Emezue, Sahib Singh, Bonaventure F\. P\. Dossou, Joanne Osuchukwu, Salomey Osei, Atnafu Lambebo Tonja, Naome Etori, and Clinton Mbataku\. 2023\.[AfriSpeech\-200: Pan\-African accented speech dataset for clinical and general domain ASR](https://doi.org/10.1162/tacl_a_00627)\.*Transactions of the Association for Computational Linguistics*, 11:1669–1685\.
- Popović et al\. \(2017\)Branislav Popović, Edvin Pakoci, and Darko Pekar\. 2017\.[End\-to\-end large vocabulary speech recognition for the Serbian language](https://doi.org/10.1007/978-3-319-66429-3_33)\.In*Speech and Computer*, pages 343–352, Cham\. Springer International Publishing\.
- Pratap et al\. \(2024\)Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel\-Zarandi, Alexei Baevski, Yossi Adi, Xiaohui Zhang, Wei\-Ning Hsu, Alexis Conneau, and Michael Auli\. 2024\.[Scaling speech technology to 1,000\+ languages](http://jmlr.org/papers/v25/23-1318.html)\.*Journal of Machine Learning Research*, 25\(97\):1–52\.
- Pratap et al\. \(2020\)Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert\. 2020\.[MLS: A Large\-Scale Multilingual Dataset for Speech Research](https://doi.org/10.21437/Interspeech.2020-2826)\.In*Interspeech 2020*, pages 2757–2761\.
- Pulikodan et al\. \(2026\)Sujith Pulikodan, Abhayjeet Singh, Agneedh Basu, Nihar Desai, Pavan Kumar J, Pranav D Bhat, Raghu Dharmaraju, Ritika Gupta, Sathvik Udupa, Saurabh Kumar, Sumit Sharma, Vaibhav Vishwakarma, Visruth Sanka, Dinesh Tewari, Harsh Dhand, Amrita Kamat, Sukhwinder Singh, Shikhar Vashishth, Partha Talukdar, and 2 others\. 2026\.[Vaani: Capturing the language landscape for an inclusive digital india](https://arxiv.org/abs/2603.28714)\.*Preprint*, arXiv:2603\.28714\.
- Radford et al\. \(2023\)Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever\. 2023\.[Robust speech recognition via large\-scale weak supervision](https://proceedings.mlr.press/v202/radford23a.html)\.In*Proceedings of the 40th International Conference on Machine Learning*, volume 202 of*Proceedings of Machine Learning Research*, pages 28492–28518\. PMLR\.
- Radford et al\. \(2019\)Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever\. 2019\.[Language models are unsupervised multitask learners](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\.*OpenAI Technical Report*\.
- Schneider et al\. \(2019\)Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli\. 2019\.[wav2vec: Unsupervised Pre\-Training for Speech Recognition](https://doi.org/10.21437/Interspeech.2019-1873)\.In*Interspeech 2019*, pages 3465–3469\.
- Sennrich et al\. \(2016\)Rico Sennrich, Barry Haddow, and Alexandra Birch\. 2016\.[Neural machine translation of rare words with subword units](https://aclanthology.org/P16-1162)\.In*Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics*, pages 1715–1725, Berlin, Germany\. Association for Computational Linguistics\.
- Shi et al\. \(2026\)Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, Jin Xu, Jingren Zhou, and Junyang Lin\. 2026\.[Qwen3\-ASR Technical Report](https://arxiv.org/abs/2601.21337)\.*Preprint*, arXiv:2601\.21337\.
- Song et al\. \(2024\)Zheshu Song, Jianheng Zhuo, Yifan Yang, Ziyang Ma, Shixiong Zhang, and Xie Chen\. 2024\.[LoRA\-Whisper: Parameter\-Efficient and Extensible Multilingual ASR](https://doi.org/10.21437/Interspeech.2024-892)\.In*Interspeech 2024*, pages 3934–3938\.
- Srivastava et al\. \(2018\)Brij Mohan Lal Srivastava, Sunayana Sitaram, Rupesh Kumar Mehta, Krishna Doss Mohan, Pallavi Matani, Sandeepkumar Satpal, Kalika Bali, Radhakrishnan Srikanth, and Niranjan Nayak\. 2018\.[Interspeech 2018 Low Resource Automatic Speech Recognition Challenge for Indian Languages](https://doi.org/10.21437/SLTU.2018-3)\.In*6th Workshop on Spoken Language Technologies for Under\-Resourced Languages \(SLTU 2018\)*, pages 11–14\.
- Talafha et al\. \(2023\)Bashar Talafha, Abdul Waheed, and Muhammad Abdul\-Mageed\. 2023\.[N\-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition](https://doi.org/10.21437/Interspeech.2023-1044)\.In*Interspeech 2023*, pages 5092–5096\.
- Toshniwal et al\. \(2018\)Shubham Toshniwal, Tara N\. Sainath, Ron J\. Weiss, Bo Li, Pedro Moreno, Eugene Weinstein, and Kanishka Rao\. 2018\.[Multilingual speech recognition with a single end\-to\-end model](https://doi.org/10.1109/ICASSP.2018.8461972)\.In*2018 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 4904–4908\.
- Upadhyaya et al\. \(2017\)Prashant Upadhyaya, Omar Farooq, Musiur Raza Abidi, and Yash Vardhan Varshney\. 2017\.[Continuous Hindi speech recognition model based on Kaldi ASR toolkit](https://doi.org/10.1109/WiSPNET.2017.8299868)\.In*2017 International Conference on Wireless Communications, Signal Processing and Networking \(WiSPNET\)*, pages 786–789\.
- Vaswani et al\. \(2017\)Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N\. Gomez, Łukasz Kaiser, and Illia Polosukhin\. 2017\.[Attention is all you need](https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf)\.In*Proceedings of the 31st International Conference on Neural Information Processing Systems*, NIPS’17, page 6000–6010, Red Hook, NY, USA\. Curran Associates Inc\.
- Wang et al\. \(2021\)Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux\. 2021\.[VoxPopuli: A large\-scale multilingual speech corpus for representation learning, semi\-supervised learning and interpretation](https://doi.org/10.18653/v1/2021.acl-long.80)\.In*Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\)*, pages 993–1003, Online\. Association for Computational Linguistics\.
- Watanabe et al\. \(2017\)Shinji Watanabe, Takaaki Hori, and John R\. Hershey\. 2017\.[Language independent end\-to\-end architecture for joint language identification and speech recognition](https://doi.org/10.1109/ASRU.2017.8268945)\.In*2017 IEEE Automatic Speech Recognition and Understanding Workshop \(ASRU\)*, pages 265–271\.
- Weitzel et al\. \(2025\)Derek Weitzel, Ashton Graves, Sam Albin, Huijun Zhu, Frank Wuerthwein, Mahidhar Tatineni, Dmitry Mishin, Elham Khoda, Mohammad Sada, Larry Smarr, Thomas DeFanti, and John Graham\. 2025\.[The National Research Platform: Stretched, multi\-tenant, scientific kubernetes cluster](https://doi.org/10.1145/3708035.3736060)\.In*Practice and Experience in Advanced Research Computing 2025: The Power of Collaboration*, PEARC ’25, New York, NY, USA\. Association for Computing Machinery\.
- Yadavalli et al\. \(2025\)Aditya Yadavalli, Tiago Pimentel, Tamar I Regev, Ethan Wilcox, and Alex Warstadt\. 2025\.[What do prosody and text convey? Characterizing how meaningful information is distributed across multiple channels](https://arxiv.org/abs/2512.16832)\.*Preprint*, arXiv:2512\.16832\.
- Yang et al\. \(2025\)Hongli Yang, Yizhou Peng, Hao Huang, and Sheng Li\. 2025\.[Adapting Whisper for Parameter\-efficient Code\-Switching Speech Recognition via Soft Prompt Tuning](https://doi.org/10.21437/Interspeech.2025-2549)\.In*Interspeech 2025*, pages 5203–5207\.
- Zhang et al\. \(2023\)Yu Zhang, Wei Han, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Bo Li, Vera Axelrod, Gary Wang, Zhong Meng, Ke Hu, Andrew Rosenberg, Rohit Prabhavalkar, Daniel S\. Park, Parisa Haghani, Jason Riesa, Ginger Perng, Hagen Soltau, and 8 others\. 2023\.[Google USM: Scaling automatic speech recognition beyond 100 languages](https://arxiv.org/abs/2303.01037)\.*Preprint*, arXiv:2303\.01037\.
- Zhao et al\. \(2026\)Qiuming Zhao, Guangzhi Sun, and Chao Zhang\. 2026\.[Low\-rank and sparse model merging for multi\-lingual speech recognition and translation](https://doi.org/10.1109/ICASSP55912.2026.11460666)\.In*ICASSP 2026 \- 2026 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 18827–18831\.
- Özyilmaz et al\. \(2025\)Ömer Tarik Özyilmaz, Matt Coler, and Matias Valdenegro\-Toro\. 2025\.[Overcoming Data Scarcity in Multi\-Dialectal Arabic ASR via Whisper Fine\-Tuning](https://doi.org/10.21437/Interspeech.2025-2260)\.In*Interspeech 2025*, pages 1158–1162\.

## Appendix AUnsupported Whisper Languages

Out of the 102 languages we experiment with, 19 of them are not natively supported by Whisper\. For such languages, we prompt the model with the language token that is most closely related to the target language\. These languages are Asturian, Cebuano, Sorani Kurdish, Fulah, Irish, Igbo, Kamba, Kabuverdianu, Kyrgyz, Luganda, Luo, Northern Sotho, Nyanja, Oromo, Oriya, Umbundu, Wolof, Xhosa, and Zulu\.

Table 2:Overview of 17 sample languages, with ISO 639\-1 codes, writing systems, language families, primary geographic regions, and approximate number of first\-language \(L1\) speakers, based on Wikipedia data as of 2026 \(mainly citing Ethnologue\)\.- aCantonese lacks an ISO 639\-1 code, being covered by the Chinese macrolanguage codezh\. Its ISO 639\-3 code isyue\.
- bThis figure reflects the combined L1 total across all varieties under the Arabic macrolanguage as classified by Ethnologue/ISO\.
- cSwahili L1 speakers are vastly outnumbered by L2 speakers \(approx\. 92M L2 vs\. 5\.3M L1\)\.

## Appendix BHyperparameters

### B\.1Hyperparameters formultitask fine\-tuning

For each language and each fine\-tuning strategy we train models with three configurations, A, B, C \([Table3](https://arxiv.org/html/2609.09554#A2.T3)\)\. We use two random seeds \(42, 1337\) for simple fine\-tuning and three \(42, 1337, 2024\) for full fine\-tuning\.

Forsimple fine\-tuning, we use an absolute decoder learning rate and set the encoder rate to0\.3×0\.3\\timesthe decoder rate, so as to preserve Whisper’s pretrained acoustic features\.

We apply patience\-based early stopping on FLEURS development throughout\. Forfull fine\-tuning, we use a patience of 5 and evaluate every 100 steps\. Forsimple fine\-tuning, we use a patience of 3 and evaluate every 100 steps and select checkpoints by validation WER\. Forfull fine\-tuning, we select checkpoints by validation WER for high\-coverage languages but use validation loss for a small set of low\-coverage languages where the WER is noisy\.555Full fine\-tuningloss\-based selection is used for: Tamil, Latvian, Uzbek, Cantonese, Pashto, Swahili, Tajik, Georgian, and Ukrainian\. This split was determined empirically\.

We use 8×\\timesA100\-80GB for all experiments\.

Cfgdec LRenc LRembed LRgasched\.decaywarmupwdSFTA6\.48×10−66\.48\{\\times\}10^\{\-6\}0\.3×dec LR0\.3\{\\times\}\\textnormal\{dec LR\}–16cosine0\.951504\.35×10−54\.35\{\\times\}10^\{\-5\}B2\.00×10−52\.00\{\\times\}10^\{\-5\}0\.3×dec LR0\.3\{\\times\}\\textnormal\{dec LR\}–32cosine0\.951504\.00×10−54\.00\{\\times\}10^\{\-5\}C3\.24×10−63\.24\{\\times\}10^\{\-6\}0\.3×dec LR0\.3\{\\times\}\\textnormal\{dec LR\}–16cosine0\.951504\.35×10−54\.35\{\\times\}10^\{\-5\}FFTA3\.0×10−53\.0\{\\times\}10^\{\-5\}3×10−63\\times 10^\{\-6\}3\.0×10−53\.0\{\\times\}10^\{\-5\}16cosine\-delay–2000\.01B1\.0×10−51\.0\{\\times\}10^\{\-5\}3×10−63\\times 10^\{\-6\}3\.0×10−53\.0\{\\times\}10^\{\-5\}16cosine\-delay–2000\.01C2\.0×10−52\.0\{\\times\}10^\{\-5\}3×10−63\\times 10^\{\-6\}2\.0×10−52\.0\{\\times\}10^\{\-5\}16cosine\-delay–2000\.01Table 3:Fine\-tuning configurations, fully specified\.dec/embed LR: decoder/embedding learning rate;ga: gradient accumulation;sched\.: LR schedule \(cosine\-delay = cosine with a 1000\-step delay, 8000\-step period\);decay: scheduler decay rate;wd: weight decay\. All runs use AdamW and micro\-batch size 2 \(effective batch=2×=2\{\\times\}ga\)\. SFT uses a single learning rate \(no separate embedding LR\); FFT scales a fixed base rateη\\etaby per\-config embedding/decoder multipliers and differs across configs only in those two rates\. SFT and FFT letters index independent schemes\. For each language and strategy we train configurations\{\\\{A, B, C\}\\\}and keep the run with the lowest FLEURS\-dev WER, over two seeds \(42, 1337\) for SFT and three seeds \(42, 1337, 2024\) for FFT\.
### B\.2Hyperparameter Sweep Results

We conducted a set of Bayesian hyperperameter sweeps for the 17 core languages\.[Table4](https://arxiv.org/html/2609.09554#A2.T4)lists the per\-language optima for the 17 core languages sorted by FLEURS\-test WER \(lower is better\)\. These optima motivated the discretesimple fine\-tuningconfigurations used at scale and were not deployed directly\. We ran a 5\-run sweep over a six\-dimensional search space: decoder learning rate∈\[10−5,8×10−5\]\\in\[10^\{\-5\},8\\times 10^\{\-5\}\], gradient accumulation∈\{16,24,32\}\\in\\\{16,24,32\\\}, scheduler∈\{linear,cosine,exponential\}\\in\\\{\\text\{linear\},\\text\{cosine\},\\text\{exponential\}\\\}, decay rate∈\{0\.9,0\.95,0\.99\}\\in\\\{0\.9,0\.95,0\.99\\\}, warmup steps∈\{100,150\}\\in\\\{100,150\\\}, and weight decay∈\[10−5,10−3\]\\in\[10^\{\-5\},10^\{\-3\}\]\. Each run trained for up to 12 epochs with patience 3\.

Table 4:Per\-language optima from a Bayesian hyperparameter pilot over the 17 core languages, sorted by FLEURS\-test WER \(lower is better\)\. Because per\-language search does not scale to 102 languages, these optima were used to*design*the small set of discrete configurations applied in all main experiments \([Table3](https://arxiv.org/html/2609.09554#A2.T3)\); the deployed models do not use per\-language hyperparameters\.lr: decoder learning rate;ga: gradient accumulation;sched\.: LR schedule;decay: scheduler decay rate;warmup: warmup steps;wd: weight decay\. Encoder LR is held at0\.3×0\.3\\timesthe decoder LR\. All runs use AdamW, micro\-batch size 2, and patience\-based early stopping on FLEURS\-validation\.

## Appendix CMultitask Learning Mixing Ratios

Figure 5:Comparison ofmultitask fine\-tuningwith different proportions of ASR data and text dataset sizes of 500k and 2\.5M lines\. We use an ASR proportion of 0\.5 and a text size of 500k lines in our main experiments\.In our mainfull fine\-tuningexperiments, we combinetext\-onlyandASR fine\-tuningbatches in a 1:1 ratio, i\.e\., the proportion of ASR batches is 50%\. We test the effect of this mixing ratio, conducting additional experiments with ASR proportions of 10% and 20%\.666We opt for lower percentages of ASR data due to the fact that we generally have less ASR data thantext\-onlydata\.We independently manipulate the length ofmultitask fine\-tuningto either 500k lines \(as in our main experiments\) or 2\.5M lines\.

Our results, shown in[Figure5](https://arxiv.org/html/2609.09554#A3.F5), indicate that the ASR proportion used in our main experiments is optimal, at least among the settings we evaluate\. More specifically, we find that performance improves with larger ASR proportions\. Furthermore, we find little or no improvement due to increasing the text dataset size from 500k lines \(as in our main experiments\) to 2\.5M lines\. These findings leave open the possibility that higher ASR proportions—or perhaps annealing curricula that increase the ASR proportion towards the end of training—might yield stronger performance\. However, due to the wide range of possiblemultitask fine\-tuningapproaches, we leave a more thorough investigation to future work\.

## Appendix DTokenizer Replacement Strategies

Figure 6:Comparison of tokenizer replacement strategies\. We use the warm\-start approach in our main experiments\.*For presentational reasons, we limit theyy\-axis, excluding some outliers\.*We test three strategies for tokenizer replacement\. All three strategies begin by training a new BPE tokenizer on a monolingual dataset\.

1. 1\.Thewarm\-startstrategy \(described in greater detail in[Section3\.1](https://arxiv.org/html/2609.09554#S3.SS1)\) entails replacing the original tokenizer entirely with the new tokenizer, and initializing all embeddings \(and unembeddings\) using the original \(un\)embeddings or as a sum of the original \(un\)embeddings making up each new token\.
2. 2\.Therandomstrategy replaces the original tokenizer entirely with the new tokenizer, but initializes \(un\)embeddings randomly\.
3. 3\.Theextendedstrategy appends all new tokens to the end of the original tokenizer merge list initializes new embeddings as a sum of the original embeddings\.

For each tokenizer replacement strategies, we conduct three training runs for the 17 core languages\. The results are shown in[Figure6](https://arxiv.org/html/2609.09554#A4.F6)\. We find that the warm\-start strategy adopted in our main experiments is the most effective, yielding a median CER of 7\.4\. The remaining random and extended strategies lead to worse performance overall, yielding median CERs of 15\.1 and 15\.7, respectively\.

## Appendix EOut\-of\-Domain Generalization

Figure 7:Comparison of in\-domain and out\-of\-domain evaluation\. We compare models trained on FLEURS alone to models trained on both Common Voice and Fleurs as in our main experiments\. Models are evaluated on Common Voice \(out\-of\-domain\), FLEURS \(in\-domain\), and both \(mixed\)\.In our main experiments, our train and test sets for most languages are a combination of data from FLEURS and Common Voice\. However, other models we compare against using the same test data may not be trained on these domains, or are trained on a wider set of domains\. Therefore, we conduct out\-of\-domain evaluations to investigate the extent to which our performance gains are due to the similarity of the train/test domain, rather than language\-specific fine\-tuning\.

We train models on the 17 core languages usingsimple fine\-tuningon only the FLEURS portion of the training data, and we evaluate on either the Common Voice test data \(out\-of\-domain\), the FLEURS test set \(in\-domain\), or both \(mixed\)\. The results in[Figure7](https://arxiv.org/html/2609.09554#A5.F7)reveal a modest in\-domain advantage relative to models trained on both FLEURS and Common Voice\. The FLEURS\-only model tested OOD is outperformed by a model trained on both domains by a factor of 1\.15×\\times, while it outperforms the mixed model on the in\-domain evaluation by a factor of 1\.33×\\times\. On the combined test set, there is little difference, with the mixed models outperforming the FLEURS models by a factor 1\.03×\\timeson average\. We conclude that some of the performance gains over Whisper and external baselines that we observe may be due to better alignment between the fine\-tuning data and the test data domains\. However, this alone cannot explain the average performance gain of 2\.8×\\timesthat we observe for BuzzASR over Whisper\.

## Appendix FMultilingual Fine\-Tuning

Figure 8:Comparison of monolingualsimple fine\-tuningand bilingualsimple fine\-tuningfor 10 language pairs \(5 pairs of related languages and 5 pairs of unrelated languages\)\.*For presentational reasons, we limit theyy\-axis, excluding some outliers\.*The main premise of our work is that monolingual fine\-tuning of massively multilingual models can improve language\-specific ASR performance\. However, prior work on text\-based models showed that limiting multilinguality to a small number of languages can sometimes improve performance over monolingual models\([Chang et al\., 2024](https://arxiv.org/html/2609.09554#bib.bib9)\)\. We test the initial promise of that approach in our setting by conducting*bilingual*fine\-tuning runs\. We select 10 language pairs from our core set of 17 languages\. 5 pairs are unrelated \(Arabic–Tamil, Cantonese–Georgian, German–Swahili, Latvian–Uzbek, Spanish–Cantonese\), and 5 pairs are related \(Catalan–Galician, Italian–Spanish, Polish–Russian, Russian–Ukrainian, Tajik–Pashto\)\. For each pair, we conduct threesimple fine\-tuningtraining runs with different random seeds\.

The results are shown in[Figure8](https://arxiv.org/html/2609.09554#A6.F8)\. We find that the best bilingual result is nearly always worse than the best monolingualsimple fine\-tuningresult for each language in that pair\. Furthermore, we find no advantage to training on related languages as opposed to unrelated languages: The median improvements of monolingual training over bilingual training are 1\.25×\\timesand 1\.24×\\timesfor unrelated and related language pairs, respectively\. The only exceptions come from the bilingual Cantonese–Georgian models, where we observe a substantial bilingual improvement of 1\.38×\\timesfor Cantonese and a modest bilingual improvement of 1\.01×\\timesfor Georgian\.

## Appendix GAblation: Multitask Learning Without Tokenizer Replacement

Figure 9:Ablation of tokenizer replacement from thefull fine\-tuningsetting\. We comparesimple fine\-tuningandfull fine\-tuningto runs where we performmultitask fine\-tuningwithout tokenizer replacement \(SFTMTL\)\.To isolate the effects ofmultitask fine\-tuningand tokenizer replacement, we conduct additional training runs in which we performmultitask fine\-tuning*without*tokenizer adaptation\.777We do not investigate tokenizer adaptation withoutmultitask fine\-tuningbecause we consider tokenizer adaptation to be likely to fail without the additionaltext\-onlyfine\-tuning data\.For each of the 17 core languages, we conduct three additionalmultitask fine\-tuningruns of this kind\.

The results in[Figure9](https://arxiv.org/html/2609.09554#A7.F9)compare allsimple fine\-tuningandfull fine\-tuningruns against thismultitask\-onlyapproach\. In the aggregate, we find thatfull fine\-tuningis more effective than either alternative, and there is little difference in performance betweensimple fine\-tuningandmultitask\-only\.

## Appendix HDecoder Latency

We benchmark per\-utterance decoder latency across the three systems \(Whisper zero\-shot,simple fine\-tuning, andfull fine\-tuning\) to assess whether the tokenizer replacement employed infull fine\-tuningtranslates to a wall\-clock speedup\. For each \(language, system\) pair we sample 50 FLEURS\-test utterances, and, where available, 50 from Common Voice; warm up with three untimedgenerate\(\)calls; then time each subsequent call withtorch\.cuda\.synchronizebarriers andtime\.perf\_counterdeltas\. All three systems use plain greedy decoding \(num\_beams=1, no repetition penalty orno\-repeat\-ngramsuppression\) so the comparison isolates the model and tokenizer rather than decoding heuristics\. We use FP16, batch size 1,max\_new\_tokens=256, and a single A100\-80GB GPU per measurement\.

We analyze three quantities per \(lang, system\): median per\-utterance wall\-clock latency in milliseconds, median per\-token throughput in tokens per second, and median number of tokens generated\. The first captures user\-facing latency; the second isolates raw decode speed \(which we expect to be constant across systems since all share the Whisper\-large\-v3 architecture\); the third explains the gap\.

Table 5:Per\-script decoder latency on the combined FLEURS\+CV25 test set \(idle GPU\), median over languages\. Speedup is the median per\-language ratio of Whisper\-zs to system latency\. “Other” covers unique scripts: Amharic \(Ge’ez\), Armenian, Georgian, Greek, and Hebrew\.†Sino\-Japanese is roughly latency\-neutral; the residual cost is driven by Japanese, which over\-generates due to a generation failure\.Three findings follow\. First, per\-token throughput is essentially identical across the three systems they share the Whisper\-large\-v3 decoder \(median≈\\approx73 tokens/sec\) confirming that latency differences arise from sequence length rather than per\-token decode speed\. Second, FFT’s native tokenizer delivers a wall\-clock speedup on every script bucket except Sino\-Japanese, and each speedup tracks that script’s token\-count compression: 1\.59×\\timeson Latin, 1\.75×\\timeson Cyrillic, 1\.98×\\timeson Arabic, 1\.16×\\timeson Brahmic, and 3\.51×\\timeson other languages, for a global median of∼\\sim1\.6×\\times\.

## Appendix IPer\-Language Tokenizer Metrics

[Figure10](https://arxiv.org/html/2609.09554#A9.F10)summarizes per\-language tokenizer\-quality metrics for all 102 languages: characters per token, UTF\-8 coverage, vocabulary utilization, and boundary crossing of our per\-language BPE over Whisper’s multilingual BPE on a sample of the text corpus used to train the per\-language tokenizers\. Detailed results are provided in[Table6](https://arxiv.org/html/2609.09554#A9.T6)\.

Nearly across the board, the BuzzASR tokenizers outperform Whisper tokenizers\. Our tokenizers generally achieve around 5 characters per token, while the Whisper tokenizer achieves less than 2 for all non\-Latin/Cyrillic scripts\. The main exception is Sino\-Japanese scripts, which are largely logographic\. In fact, the Whisper tokenizer reaches less than 1 character per token for Brahmic and the rarer scripts \(“other”\)\. Our UTF\-8 coverage is typically at 100%, meaning that when a token sequence is decoded back to text, essentially no characters are corrupted\. Vocab utilization is generally above 20% for BuzzASR and below 10% for Whisper\. For Arabic, Brahmic, and rarer scripts, Whisper vocab utilization is generally below 3%, meaning 97% of the vocabulary is unused\. Finally, boundary crossing for BuzzASR is below 0\.1% for the vast majority of languages except those using Brahmic script\. Boundary crossing is the percentage of tokens that contain an internal whitespace character i\.e\., a single token that spans a word boundary by merging parts of adjacent words By contrast, for Whisper, boundary crossing only falls below this threshold for languages using Latin and Cyrillic scripts\.

Figure 10:Tokenizer performance metrics for Whisper and BuzzASR, separated by script\. Arrows indicate the direction associated with*better*performance\.Table 6:Per\-language tokenizer quality metrics\. “ours” is our per\-language BPE, “W” is Whisper\-large\-v3’s multilingual BPE\. Higher chars/tok and vocab utilization are better; lower tokens/word is better\. Kamba’s per\-language tokenizer is anomalous due to its tiny training corpus and is excluded from aggregate statistics in the main text\.chars/toktokens/wordvocab util\.Comp\.LanguageScriptoursWoursWoursWgainAfrikaansLatin5\.963\.100\.921\.7721\.2%7\.3%1\.93xAmharicOther3\.520\.431\.4311\.7435\.5%1\.4%8\.23xArabicArabic4\.781\.871\.223\.1028\.7%1\.8%2\.55xArmenianOther5\.110\.881\.438\.2823\.4%2\.0%5\.79xAssameseBrahmic4\.520\.511\.4012\.4229\.2%2\.0%8\.85xAsturianLatin5\.042\.891\.212\.1119\.9%10\.5%1\.74xAzerbaijaniLatin5\.252\.141\.413\.4724\.5%6\.2%2\.46xBelarusianCyrillic4\.781\.951\.463\.5726\.9%3\.8%2\.45xBengaliBrahmic4\.650\.511\.3612\.3628\.4%1\.0%9\.11xBosnianLatin4\.822\.531\.402\.6622\.6%8\.2%1\.90xBulgarianCyrillic4\.882\.471\.282\.5227\.3%4\.6%1\.98xBurmeseBrahmic2\.540\.375\.1234\.7516\.9%2\.4%6\.78xCantoneseCJK1\.860\.8728\.3460\.7333\.8%4\.8%2\.14xCatalanLatin5\.793\.451\.011\.6922\.6%9\.5%1\.68xCebuanoLatin5\.422\.801\.112\.1518\.7%8\.9%1\.94xCroatianLatin5\.352\.391\.282\.8624\.4%6\.2%2\.24xCzechLatin4\.722\.301\.322\.7128\.4%7\.1%2\.05xDanishLatin6\.073\.051\.002\.0023\.4%7\.9%1\.99xDutchLatin5\.963\.431\.041\.8121\.2%8\.8%1\.74xEnglishLatin5\.814\.830\.991\.1921\.7%15\.5%1\.20xEstonianLatin5\.612\.571\.342\.9323\.5%6\.6%2\.18xFilipinoLatin5\.733\.031\.031\.9523\.7%8\.4%1\.89xFinnishLatin7\.592\.841\.072\.8722\.5%7\.0%2\.67xFrenchLatin5\.803\.771\.041\.6022\.9%11\.2%1\.54xFulahLatin4\.122\.281\.392\.5227\.4%10\.2%1\.81xGalicianLatin5\.923\.491\.011\.7223\.3%9\.5%1\.69xGeorgianOther4\.580\.411\.7920\.0022\.2%1\.3%11\.17xGermanLatin5\.303\.701\.331\.9020\.5%10\.9%1\.43xGreekOther4\.981\.981\.303\.2727\.7%3\.0%2\.52xGujaratiBrahmic4\.330\.411\.3414\.0930\.6%2\.4%10\.53xHausaLatin5\.592\.660\.982\.0621\.2%6\.3%2\.10xHebrewOther4\.251\.821\.313\.0729\.9%2\.1%2\.33xHindiBrahmic4\.671\.021\.074\.9028\.3%2\.7%4\.59xHungarianLatin5\.352\.431\.362\.9824\.9%6\.4%2\.20xIcelandicLatin5\.292\.241\.152\.7225\.4%5\.8%2\.36xIgboLatin4\.862\.001\.032\.5122\.4%6\.8%2\.43xIndonesianLatin5\.903\.401\.172\.0423\.8%6\.9%1\.74xIrishLatin6\.122\.390\.972\.4822\.9%4\.9%2\.56xItalianLatin5\.793\.501\.101\.8223\.1%9\.2%1\.65xJapaneseCJK2\.341\.1214\.5830\.4438\.7%6\.2%2\.09xJavaneseLatin4\.562\.851\.372\.2018\.6%6\.9%1\.60xKabuverdianuLatin4\.642\.651\.121\.9625\.5%10\.0%1\.75xKambaLatin47\.452\.180\.132\.784\.1%6\.3%21\.72xKannadaBrahmic4\.180\.611\.9113\.1228\.5%1\.4%6\.86xKazakhCyrillic5\.741\.581\.344\.8923\.5%3\.5%3\.64xKhmerBrahmic4\.700\.484\.0339\.8022\.0%1\.5%9\.87xKoreanCJK2\.611\.451\.642\.9536\.5%7\.6%1\.80xKyrgyzCyrillic5\.861\.671\.324\.6123\.5%3\.4%3\.50xLaoBrahmic5\.040\.373\.0942\.5026\.8%2\.8%13\.77xLatvianLatin4\.862\.201\.463\.2325\.1%7\.7%2\.21xLingalaLatin4\.302\.151\.402\.7916\.4%8\.3%2\.00xLithuanianLatin5\.422\.391\.383\.1323\.6%6\.9%2\.27xLugandaLatin5\.982\.351\.373\.4715\.9%2\.7%2\.54xLuoLatin4\.012\.621\.312\.0124\.9%9\.7%1\.53xLuxembourgishLatin4\.882\.861\.312\.2418\.8%9\.6%1\.71xMacedonianCyrillic5\.002\.271\.242\.7226\.3%4\.1%2\.20xMalayLatin6\.863\.430\.961\.9119\.0%5\.5%2\.00xMalayalamBrahmic4\.570\.482\.0419\.5326\.7%1\.8%9\.58xMalteseLatin5\.972\.081\.303\.7321\.5%5\.7%2\.87xMandarinCJK1\.760\.8528\.0057\.8839\.4%5\.5%2\.07xMaoriLatin4\.502\.371\.152\.1911\.9%5\.4%1\.90xMarathiBrahmic4\.730\.901\.397\.3129\.3%1\.4%5\.26xMongolianCyrillic5\.491\.451\.164\.4122\.9%3\.1%3\.80xNepaliBrahmic4\.580\.911\.366\.8831\.5%1\.9%5\.05xNorwegianLatin6\.033\.041\.001\.9923\.8%8\.2%1\.99xNyanjaLatin4\.472\.621\.532\.6118\.2%9\.1%1\.71xOccitanLatin4\.922\.781\.222\.1520\.9%9\.6%1\.77xOriyaBrahmic4\.840\.381\.3316\.9329\.7%0\.9%12\.73xOromoLatin4\.502\.471\.602\.9225\.5%8\.4%1\.82xPashtoArabic5\.101\.340\.903\.4227\.1%0\.9%3\.81xPersianArabic5\.431\.570\.913\.1723\.6%1\.7%3\.47xPolishLatin5\.023\.041\.392\.2926\.8%8\.5%1\.65xPortugueseLatin5\.793\.801\.031\.5722\.8%10\.1%1\.52xPunjabiBrahmic4\.790\.541\.079\.5528\.2%2\.1%8\.95xRomanianLatin5\.292\.811\.152\.1724\.8%8\.5%1\.88xRussianCyrillic5\.343\.101\.352\.3225\.3%6\.4%1\.72xSepediLatin5\.302\.561\.122\.3112\.1%5\.8%2\.07xSerbianCyrillic4\.872\.061\.313\.0924\.9%3\.4%2\.36xShonaLatin4\.532\.711\.622\.7116\.7%8\.2%1\.67xSindhiArabic5\.161\.380\.973\.6424\.1%2\.3%3\.75xSlovakLatin5\.042\.361\.282\.7326\.2%7\.0%2\.14xSlovenianLatin5\.572\.561\.142\.4824\.6%6\.3%2\.18xSomaliLatin6\.432\.481\.002\.5922\.0%4\.8%2\.60xSorani\-kurdishArabic4\.491\.091\.445\.9225\.0%1\.5%4\.10xSpanishLatin5\.973\.861\.001\.5522\.4%10\.7%1\.55xSwahiliLatin5\.962\.561\.052\.4622\.2%6\.1%2\.33xSwedishLatin5\.583\.231\.101\.9023\.8%8\.6%1\.73xTajikCyrillic5\.441\.731\.223\.8323\.9%2\.9%3\.14xTamilBrahmic4\.991\.161\.737\.4627\.0%1\.7%4\.32xTeluguBrahmic4\.340\.561\.7513\.6630\.1%2\.1%7\.81xThaiBrahmic4\.461\.214\.2015\.5230\.0%2\.6%3\.69xTurkishLatin5\.432\.991\.372\.4923\.8%7\.4%1\.81xUkrainianCyrillic5\.202\.441\.352\.8725\.6%4\.6%2\.13xUmbunduLatin4\.582\.561\.272\.2723\.7%8\.2%1\.79xUrduArabic5\.521\.730\.832\.6625\.6%1\.4%3\.20xUzbekLatin5\.312\.481\.382\.9618\.3%4\.6%2\.15xVietnameseLatin5\.262\.600\.851\.7224\.4%4\.1%2\.02xWelshLatin5\.902\.430\.972\.3621\.5%5\.8%2\.43xWolofLatin4\.782\.431\.042\.0416\.8%6\.3%1\.96xXhosaLatin4\.652\.481\.843\.4418\.7%6\.0%1\.87xYorubaLatin4\.831\.780\.982\.6720\.8%5\.2%2\.72xZuluLatin3\.982\.572\.093\.2518\.5%9\.5%1\.55x
## Appendix JFull Evaluation Results

Table[7](https://arxiv.org/html/2609.09554#A10.T7)and[8](https://arxiv.org/html/2609.09554#A10.T8)provide the full CER and WER results for FLEURS for all languages\.

Table 7:Test\-set CER \(%\) on the test for all languages, which consists of the combined CV and FLEURS datasets when available and just FLEURS elsewhere\. Lower is better\.Boldindicates the best score within the Whisper group;underlineindicates the overall best score\. For conditions with multiple runs, the best\-performing run is reported\. Scores are normalised CER \.OursBaselinesLanguageFFTSFTSFTMTLWhisperOmni 1BOmni 7BMMSCohereQwen3Afrikaans13\.235\.357\.6310\.888\.016\.387\.2630\.9430\.56Amharic8\.5549\.3951\.04191\.3512\.427\.398\.39140\.55120\.60Arabic14\.0611\.7410\.7611\.825\.094\.317\.755\.806\.55Armenian3\.5513\.5017\.3815\.534\.923\.913\.92129\.0992\.29Assamese12\.1162\.4362\.9480\.729\.287\.009\.24125\.0496\.14Asturian7\.724\.895\.2314\.117\.355\.385\.2618\.8718\.34Azerbaijani5\.015\.225\.705\.685\.414\.115\.3993\.6134\.17Belarusian22\.302\.753\.2910\.954\.293\.113\.77101\.3738\.04Bengali7\.1653\.9167\.3334\.046\.794\.637\.44117\.3593\.75Bosnian7\.493\.593\.783\.874\.913\.073\.5158\.1842\.67Bulgarian2\.291\.211\.684\.194\.913\.534\.09100\.7022\.90Burmese32\.5884\.3882\.26123\.3211\.148\.2719\.65105\.1998\.34Cantonese25\.3913\.0112\.7758\.0258\.3654\.1460\.8871\.1052\.99Catalan12\.354\.504\.204\.323\.582\.553\.1431\.6527\.65Cebuano8\.395\.915\.8515\.425\.414\.484\.3361\.7216\.67Croatian5\.843\.093\.615\.1124\.3623\.343\.3856\.7372\.08Czech10\.283\.413\.003\.073\.892\.883\.0756\.0411\.40Danish15\.8516\.2114\.554\.747\.094\.687\.0460\.1812\.04Dutch5\.561\.491\.971\.983\.692\.723\.465\.745\.80English11\.574\.844\.604\.344\.803\.544\.767\.636\.13Estonian2\.001\.963\.516\.193\.512\.802\.5377\.4634\.44Filipino5\.575\.045\.074\.464\.053\.323\.6866\.8411\.23Finnish29\.2831\.7931\.751\.893\.092\.472\.3680\.889\.11French8\.707\.897\.297\.654\.473\.124\.245\.845\.41Fulah16\.5916\.6017\.1733\.1215\.1215\.0514\.1281\.0760\.56Galician5\.671\.482\.913\.843\.632\.582\.9118\.8018\.18Georgian4\.3972\.6577\.9619\.434\.473\.314\.91123\.5593\.70German6\.093\.202\.872\.772\.682\.002\.688\.468\.31Greek11\.065\.053\.214\.376\.273\.745\.336\.2516\.19Gujarati8\.1177\.4079\.7420\.175\.874\.866\.42113\.9292\.34Hausa11\.585\.507\.3133\.696\.986\.316\.8476\.7754\.49Hebrew11\.3010\.8912\.1312\.7313\.3910\.5017\.13111\.4647\.03Hindi15\.9516\.2314\.7611\.929\.228\.146\.40113\.856\.93Hungarian3\.912\.683\.763\.594\.743\.194\.3095\.1016\.13Icelandic7\.269\.009\.7411\.256\.144\.657\.3673\.5350\.48Igbo14\.5017\.2617\.3047\.6915\.3712\.7812\.9786\.1252\.18Indonesian5\.832\.631\.992\.363\.342\.483\.0986\.415\.63Irish17\.6515\.7018\.2887\.8827\.2521\.6126\.1184\.7371\.66Italian3\.762\.862\.532\.742\.161\.611\.674\.674\.42Japanese44\.4624\.8423\.0023\.3417\.8212\.0424\.0510\.9811\.98Javanese4\.815\.155\.1925\.135\.084\.074\.9484\.7125\.31Kabuverdianu18\.955\.035\.5333\.025\.524\.174\.2637\.5637\.78Kamba21\.5615\.9218\.0941\.5513\.2110\.9711\.7388\.2748\.72Kannada10\.2051\.6750\.5918\.725\.104\.075\.25112\.65101\.46Kazakh9\.189\.088\.949\.753\.542\.553\.40112\.9285\.29Khmer25\.6882\.9381\.65103\.1913\.299\.8718\.26134\.86103\.11Korean17\.374\.615\.165\.7210\.318\.5717\.279\.767\.49Kyrgyz20\.6211\.1613\.4427\.884\.203\.314\.10108\.1590\.25Lao20\.3281\.5581\.1561\.0922\.0116\.1722\.44128\.37100\.22Latvian3\.963\.875\.245\.904\.062\.892\.7782\.3358\.61Lingala12\.037\.428\.3918\.585\.474\.494\.4362\.7430\.39Lithuanian2\.922\.112\.736\.695\.894\.104\.0977\.3487\.37Luganda21\.6824\.9729\.2728\.479\.198\.938\.0874\.5139\.78Luo90\.4376\.9277\.2335\.856\.595\.445\.6078\.7836\.56Luxembourgish9\.6210\.7811\.1629\.8810\.626\.958\.7541\.3441\.73Macedonian1\.861\.311\.965\.822\.982\.542\.13100\.029\.05Malay4\.032\.642\.542\.324\.243\.213\.6282\.346\.75Malayalam16\.1963\.9862\.2287\.875\.374\.215\.12105\.62100\.70Maltese3\.152\.663\.6026\.357\.996\.583\.8575\.4970\.27Mandarin31\.6422\.4421\.6315\.3153\.3850\.2960\.9453\.4752\.29Maori32\.5610\.9011\.3417\.209\.747\.829\.0374\.6242\.22Marathi57\.8049\.1955\.4820\.267\.776\.387\.60118\.0229\.84Mongolian4\.879\.4211\.0938\.2312\.177\.778\.70104\.3290\.07Nepali13\.5916\.2320\.7220\.929\.097\.388\.78115\.6538\.80Northern Sotho19\.4512\.9113\.1555\.109\.486\.387\.3582\.2659\.92Norwegian6\.172\.782\.096\.374\.303\.245\.1053\.6522\.22Nyanja10\.4611\.1911\.2839\.629\.317\.037\.3095\.3047\.13Occitan18\.1311\.8510\.7324\.0712\.199\.459\.8938\.5635\.26Oriya17\.0492\.9596\.2493\.5710\.248\.219\.17117\.3197\.17Oromo16\.9520\.0119\.7745\.2518\.7816\.5217\.4181\.6675\.81Pashto13\.0311\.1015\.5435\.0115\.4114\.6216\.6181\.5863\.52Persian5\.0936\.1936\.5611\.985\.443\.204\.8287\.4611\.41Polish4\.6013\.0212\.891\.813\.512\.653\.055\.778\.21Portuguese11\.043\.652\.801\.853\.042\.532\.826\.025\.77Punjabi8\.5153\.5048\.7741\.248\.546\.219\.40116\.0187\.05Romanian4\.813\.122\.903\.404\.013\.143\.5651\.8411\.18Russian5\.592\.972\.781\.483\.422\.534\.23100\.825\.96Serbian65\.9769\.4265\.2515\.9572\.0457\.4089\.4463\.6668\.43Shona7\.048\.428\.7926\.034\.653\.724\.4288\.5540\.67Sindhi13\.3116\.1015\.9493\.979\.397\.358\.45122\.6992\.41Slovak4\.464\.602\.753\.433\.272\.512\.5145\.7723\.89Slovenian9\.632\.312\.145\.435\.444\.064\.4860\.8886\.40Somali17\.7919\.1219\.0534\.4912\.1212\.1813\.4192\.3788\.48Sorani Kurdish6\.0512\.7119\.8544\.208\.206\.438\.5572\.8847\.73Spanish10\.621\.952\.041\.772\.471\.892\.105\.144\.67Swahili14\.6117\.8719\.628\.743\.853\.403\.7074\.8523\.48Swedish4\.203\.222\.142\.485\.153\.535\.5559\.189\.73Tajik6\.045\.616\.3129\.914\.714\.374\.75100\.4187\.42Tamil77\.4167\.6478\.4212\.049\.638\.2311\.69107\.75103\.49Telugu12\.7962\.0460\.6182\.657\.926\.879\.15110\.43100\.56Thai39\.8028\.3827\.586\.56——12\.86136\.318\.29Turkish4\.974\.003\.762\.423\.372\.843\.9892\.326\.07Ukrainian8\.103\.442\.572\.993\.672\.703\.8099\.9139\.92Umbundu19\.6819\.9120\.4248\.5512\.0611\.1013\.6469\.7648\.94Urdu12\.239\.249\.468\.0860\.6083\.079\.9694\.4286\.79Uzbek4\.393\.866\.6328\.405\.905\.026\.0091\.6689\.64Vietnamese7\.205\.385\.433\.907\.285\.5412\.078\.196\.71Welsh11\.345\.275\.6712\.8810\.937\.389\.3281\.4866\.24Wolof16\.3516\.0516\.8079\.8212\.3411\.4011\.7090\.5762\.91Xhosa10\.3811\.4811\.5340\.128\.406\.626\.87102\.4950\.64Yoruba24\.3226\.9229\.2047\.9617\.3616\.7117\.9194\.8262\.62Zulu9\.9711\.8612\.1541\.786\.835\.556\.4185\.3545\.84Table 8:Test\-set WER \(%\) on the test for all languages, which consists of the combined CV and FLEURS datasets when available and just FLEURS elsewhere\. Lower is better\.Boldindicates the best score within the Whisper group;underlineindicates the overall best score\. For conditions with multiple runs, the best\-performing run is reported\. Scores are normalised WER\.OursBaselinesLanguageFFTSFTSFTMTLWhisperOmni 1BOmni 7BMMSCohereQwen3Afrikaans30\.0916\.0124\.1533\.0423\.0618\.2222\.8675\.6273\.70Amharic24\.8091\.6093\.33154\.0157\.8631\.7029\.79130\.18124\.01Arabic37\.5630\.3927\.3728\.5617\.1413\.7228\.7421\.1222\.62Armenian10\.8040\.4547\.7255\.5120\.0314\.3719\.38161\.98113\.80Assamese37\.0497\.0397\.28108\.0235\.2727\.8935\.67147\.04120\.96Asturian23\.4216\.8418\.5652\.5929\.6621\.7319\.5563\.6662\.96Azerbaijani17\.4920\.4122\.0121\.6722\.9716\.3324\.58127\.8386\.25Belarusian45\.1210\.6711\.7345\.7517\.5311\.5916\.73111\.7892\.71Bengali23\.9689\.7196\.6169\.8726\.5818\.5930\.19132\.51116\.06Bosnian25\.4613\.4613\.6813\.6514\.949\.6914\.43108\.5065\.75Bulgarian6\.334\.396\.3315\.8318\.8712\.4116\.38112\.2466\.35Burmese94\.63114\.28147\.35304\.5074\.5164\.65100\.00175\.14137\.66Cantonese43\.5547\.5345\.02100\.1799\.9499\.71100\.0099\.3999\.96Catalan25\.7510\.1610\.409\.1111\.538\.2811\.7483\.2270\.38Cebuano29\.9718\.3017\.5944\.9919\.7815\.5714\.47105\.8354\.85Croatian21\.5011\.6713\.4911\.7534\.7231\.7513\.79106\.9589\.22Czech27\.0212\.2811\.6810\.5613\.869\.0512\.76101\.6339\.91Danish16\.8417\.9212\.6013\.5321\.8913\.8523\.94102\.0836\.04Dutch17\.604\.675\.825\.2912\.678\.9813\.0023\.9024\.57English26\.929\.779\.328\.5912\.899\.5413\.3225\.6125\.91Estonian7\.898\.7615\.5727\.7417\.3312\.4713\.50146\.3191\.43Filipino14\.8613\.9513\.7312\.4014\.8512\.3013\.00105\.5736\.70Finnish22\.3925\.0324\.079\.2915\.9611\.8714\.31162\.2244\.00French25\.8817\.5417\.4415\.9914\.4611\.3814\.5922\.6522\.15Fulah50\.6952\.0053\.7090\.2753\.5851\.8249\.62114\.2297\.89Galician18\.665\.1310\.7314\.5112\.848\.1511\.3066\.3465\.18Georgian16\.7099\.97129\.0072\.5720\.4414\.1725\.71163\.77129\.15German17\.299\.078\.217\.479\.816\.6110\.7343\.7844\.04Greek33\.5415\.9110\.1713\.5820\.8511\.3519\.2624\.6340\.57Gujarati31\.98100\.4599\.5851\.6823\.2819\.5727\.24127\.22110\.56Hausa31\.2618\.4825\.5989\.2226\.3123\.7325\.48115\.8499\.83Hebrew27\.2825\.4327\.7928\.1736\.2325\.4350\.54120\.9182\.36Hindi43\.7839\.5738\.3930\.9822\.2518\.7720\.30123\.7220\.95Hungarian15\.3712\.7816\.4115\.0620\.5613\.2220\.15141\.3549\.89Icelandic23\.7132\.8734\.8638\.3324\.1218\.4630\.96110\.0696\.00Igbo39\.7649\.5648\.71100\.0154\.7747\.3744\.62118\.9698\.46Indonesian12\.608\.346\.957\.0215\.109\.5713\.75137\.8626\.52Irish32\.3931\.1333\.28131\.0964\.6654\.1361\.78114\.54100\.62Italian13\.158\.517\.837\.367\.014\.746\.3419\.9820\.95Japanese98\.4872\.0969\.8951\.14112\.60107\.1999\.9099\.95101\.18Javanese17\.5419\.8919\.6968\.8721\.9017\.8420\.76133\.3175\.63Kabuverdianu38\.1217\.7218\.7589\.9120\.3614\.3415\.7494\.8093\.76Kamba60\.7552\.0858\.6197\.1049\.3640\.6943\.48134\.25103\.32Kannada41\.08118\.47119\.2565\.6923\.6318\.4229\.63140\.65147\.57Kazakh21\.3837\.0236\.5239\.6115\.3810\.5916\.90139\.17107\.72Khmer104\.74154\.75181\.93141\.4492\.8986\.06100\.00686\.02139\.78Korean42\.7313\.7415\.2417\.2327\.3223\.1948\.7025\.2320\.67Kyrgyz45\.7943\.3650\.9086\.7216\.7112\.1618\.37134\.00106\.50Lao79\.4399\.99103\.32111\.9990\.6983\.20100\.00608\.15101\.02Latvian11\.7711\.6016\.9021\.4617\.6011\.2413\.60135\.06105\.47Lingala30\.0121\.5226\.3572\.5118\.2214\.6015\.69116\.0187\.93Lithuanian10\.848\.7511\.5428\.3624\.9315\.9718\.25128\.07111\.77Luganda43\.5644\.8059\.41103\.6048\.9149\.4740\.69136\.46112\.44Luo78\.4777\.4779\.1294\.0430\.3324\.2526\.17117\.2092\.13Luxembourgish28\.2435\.6536\.3985\.4638\.2625\.3632\.4396\.6995\.68Macedonian5\.704\.947\.1620\.7710\.407\.699\.18109\.8334\.15Malay13\.439\.499\.067\.6218\.2913\.2916\.33136\.3628\.56Malayalam49\.6099\.4199\.46119\.8630\.6724\.4732\.21137\.39147\.20Maltese11\.0511\.2814\.2879\.4663\.0457\.9418\.57110\.90106\.33Mandarin67\.1772\.7574\.9353\.6399\.9599\.81100\.0099\.2899\.98Maori53\.9224\.6525\.5040\.0528\.9522\.4225\.09107\.6283\.63Marathi71\.1659\.2972\.0761\.9631\.4525\.2432\.59136\.9188\.32Mongolian12\.7026\.4629\.8888\.2044\.9928\.4732\.76122\.87106\.58Nepali37\.9145\.8552\.4652\.7832\.3824\.7932\.35131\.45111\.95Northern Sotho51\.2037\.9440\.14111\.1735\.6322\.1727\.24116\.3099\.73Norwegian18\.389\.206\.8614\.7714\.5210\.4719\.60101\.3768\.24Nyanja39\.6347\.2447\.05112\.1041\.5031\.9734\.22155\.85115\.10Occitan42\.5434\.4130\.6170\.5342\.6534\.4734\.4387\.9385\.16Oriya39\.91112\.22107\.14112\.0839\.0131\.5637\.46135\.68128\.90Oromo57\.3171\.6072\.27100\.8474\.6270\.7664\.71104\.87109\.41Pashto30\.0331\.4041\.1889\.2840\.1538\.3444\.35108\.5896\.46Persian15\.8633\.6734\.4837\.6621\.0911\.9319\.20113\.6034\.92Polish14\.5011\.3410\.735\.1712\.358\.3311\.9327\.3933\.14Portuguese24\.159\.768\.173\.989\.317\.449\.5623\.4623\.78Punjabi25\.3078\.3080\.7678\.1927\.2120\.0530\.49126\.61100\.92Romanian14\.3411\.3510\.2110\.2414\.3710\.8712\.84106\.1835\.55Russian18\.519\.248\.744\.7414\.609\.9720\.47116\.6229\.85Serbian55\.1574\.2350\.0826\.4186\.3271\.9799\.29108\.4685\.87Shona31\.7238\.6641\.09116\.5523\.3718\.1922\.97164\.22124\.23Sindhi27\.9839\.1039\.51104\.8727\.6720\.6624\.67126\.27101\.33Slovak13\.1217\.6310\.549\.7410\.306\.839\.4697\.7571\.65Slovenian18\.707\.836\.9319\.1317\.9912\.7515\.89109\.77101\.85Somali51\.7355\.8056\.8691\.8238\.8839\.3344\.47109\.51103\.67Sorani Kurdish22\.5748\.1462\.31111\.3436\.2228\.3238\.40110\.00105\.69Spanish22\.755\.255\.414\.617\.114\.717\.2420\.4320\.19Swahili22\.7722\.3827\.2434\.3513\.8111\.3514\.98115\.1366\.58Swedish12\.979\.676\.738\.2319\.1312\.4622\.36110\.9333\.50Tajik16\.5218\.9622\.1081\.3613\.5111\.2316\.00111\.86106\.89Tamil79\.0077\.1398\.0521\.0438\.9731\.9142\.87137\.43148\.10Telugu43\.26121\.65105\.66141\.1132\.4327\.7038\.95130\.84132\.62Thai106\.79100\.92107\.008\.67——99\.90870\.5179\.71Turkish16\.6512\.4413\.078\.4414\.1210\.9518\.76127\.0930\.99Ukrainian30\.8813\.2210\.608\.1915\.4110\.3218\.83109\.6081\.02Umbundu53\.0254\.3659\.53134\.6252\.8951\.7043\.66143\.41130\.79Urdu25\.6026\.7727\.0322\.8178\.0399\.6730\.97112\.60100\.44Uzbek16\.1616\.0027\.1988\.8625\.4619\.7627\.89121\.77115\.81Vietnamese15\.0510\.6810\.957\.5117\.4412\.5730\.3320\.2617\.52Welsh24\.3615\.1515\.8934\.5938\.4823\.8033\.83118\.39103\.10Wolof51\.3446\.2547\.45132\.3741\.1439\.2039\.13120\.7198\.88Xhosa44\.4051\.7253\.26146\.0844\.2935\.4536\.71206\.51139\.63Yoruba62\.8370\.2676\.02101\.2752\.3450\.2553\.52121\.71101\.21Zulu36\.8548\.4549\.50144\.7035\.3029\.4333\.55188\.96138\.03