GigaAM Multilingual: Foundation Model for Underrepresented Languages

Hugging Face Daily Papers Papers

Summary

This paper introduces GigaAM Multilingual, a Conformer encoder pre-trained on 2M hours of audio with a HuBERT-style objective, addressing data scarcity for Central Asian languages (Kazakh, Kyrgyz, Uzbek). It employs cluster-level data balancing and domain-aware sampling to outperform strong baselines like Whisper Large v3 on target languages.

Despite recent scaling successes, multilingual ASR performance remains highly uneven, with long-tail languages suffering from severe data scarcity. This work addresses the challenge of building robust foundation models for underrepresented Central Asian languages (Kazakh, Kyrgyz, Uzbek). We present GigaAM Multilingual, a Conformer encoder pre-trained on 2M hours of audio using a HuBERT-style objective. Crucially, we introduce a cluster-level data balancing strategy during pre-training and a domain-aware sampling method during fine-tuning to mitigate head-language dominance. In controlled comparisons, our approach outperforms strong open pretrained encoders (Whisper Large v3, Omnilingual-1B) on target languages, achieving significant gains on spontaneous speech while maintaining efficiency. We release the foundation encoder and ASR model, offering a proven recipe for effective multilingual adaptation under realistic data imbalance.
Original Article
View Cached Full Text

Cached at: 07/21/26, 10:36 AM

Paper page - GigaAM Multilingual: Foundation Model for Underrepresented Languages

Source: https://huggingface.co/papers/2607.10371

Abstract

Despiterecentscalingsuccesses,multilingualASRperformanceremainshighlyuneven,withlong-taillanguagessufferingfromseveredatascarcity.ThisworkaddressesthechallengeofbuildingrobustfoundationmodelsforunderrepresentedCentralAsianlanguages(Kazakh,Kyrgyz,Uzbek).WepresentGigaAMMultilingual,aConformerencoderpre-trainedon2MhoursofaudiousingaHuBERT-styleobjective.Crucially,weintroduceacluster-leveldatabalancingstrategyduringpre-trainingandadomain-awaresamplingmethodduringfine-tuningtomitigatehead-languagedominance.Incontrolledcomparisons,ourapproachoutperformsstrongopenpretrainedencoders(WhisperLargev3,Omnilingual-1B)ontargetlanguages,achievingsignificantgainsonspontaneousspeechwhilemaintainingefficiency.WereleasethefoundationencoderandASRmodel,offeringaprovenrecipeforeffectivemultilingualadaptationunderrealisticdataimbalance.

View arXiv pageView PDFGitHub692Add to collection

Similar Articles

GigaChat Audio: Time-aware Large Audio Language Model

Hugging Face Daily Papers

This paper introduces GigaChat Audio, a time-aware large audio language model that answers questions with explicit timestamps for up to 120 minutes of audio, using interleaved periodic time markers and synthetic supervision. The model achieves strong temporal grounding accuracy on benchmarks and the authors release model weights and datasets.

Command A Plus GGUFs posted

Reddit r/LocalLLaMA

Cohere has released GGUF quantized versions of its Command A+ model (25B active / 218B total parameters, Apache 2.0) for local inference, optimized for agentic and multilingual tasks.

Adding E4B audio encoder to larger models

Reddit r/LocalLLaMA

The author proposes a method to add the E4B audio encoder to larger models by extracting the encoder, creating a linear projection layer, and fine-tuning only that layer with text-audio pairs, similar to a referenced paper but using Gemma instead of Whisper.