Tag
This study develops an Automatic Speech Recognition system for Mizo, a low-resource language, by fine-tuning Whisper and SraVaani 1.0 models, achieving a morphology-aware WER of 7.22% with Whisper-large-v3.
SuTRA is a morphology-aware tokenization algorithm that preserves akshara indivisibility for Indic languages, reducing morphological shattering and achieving improvements in machine translation metrics over standard BPE methods.
This paper introduces BHARATI, a set of SentencePiece BPE tokenizers trained on a balanced multilingual corpus for classical Indian languages. Subword fertility analysis shows significant improvements in tokenization efficiency, reducing sequence length by up to 90% compared to baseline tokenizers, thereby enhancing effective context length for downstream language models.