Efficient Adaptation of LLMs for Hate Speech Detection in Low-Resource Languages: A Comparative Study on Roman Urdu
Summary
The paper evaluates Large Language Models for hate speech detection in Roman Urdu, a low-resource language, demonstrating that Parameter-Efficient Fine-Tuning with LoRA significantly improves classification performance compared to zero-shot inference.
View Cached Full Text
Cached at: 08/20/26, 10:00 AM
# Efficient Adaptation of LLMs for Hate Speech Detection in Low-Resource Languages: A Comparative Study on Roman Urdu Source: [https://arxiv.org/abs/2608.18142](https://arxiv.org/abs/2608.18142) [View PDF](https://arxiv.org/pdf/2608.18142) > Abstract:It is challenging to detect hate speech in Low Resource Languages \(LRLs\) because of the absence of annotated data, the informality of its language structure, and the lack of standardized grammar\. A good example of such a challenge is Roman Urdu which is broadly used by South Asians on social media and has a high variation while lacking contextually consistent spellings\. The objective of this paper is to conduct a comprehensive assessment of Large Language Models \(LLMs\) for Hate Speech Detection \(HSD\) in Roman Urdu script and fine\-tune these models using the Parameter\-Efficient Fine\-Tuning \(PEFT\) method called Low\-Rank Adaptation \(LoRA\)\. To evaluate zero\-shot inference, we benchmarked it against PEFT on different transformer models, including Mistral, LLaMA, Falcon, and multilingual BERT\. Experiments are conducted on the PURUTT \(Parallel Urdu and Roman Urdu Corpus for Toxic Comments and Transliteration\) dataset with over 72,000 annotated comments\. The results suggest that zero shot models perform moderately \(F1 = 0\.56\), but updating a small fraction of the model trainable parameters improves the classification performance significantly \(F1 \> 0\.93\)\. Our results have shown that PEFT delivers outstanding performance alongside excellent computational efficiency, making it highly suitable for low\-resource language processing tasks\. ## Submission history From: Toneema Zubair \[[view email](https://arxiv.org/show-email/f0678e4b/2608.18142)\] **\[v1\]**Thu, 6 Aug 2026 11:28:19 UTC \(1,417 KB\)
Similar Articles
Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
This tutorial paper provides an overview of building multilingual and multimodal LLMs for low-resource languages, covering data creation, model alignment, fine-tuning, and evaluation, with a focus on practical recipes and hands-on resources.
Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research
This paper systematically evaluates the applications of large language models in low-resource language research, analyzing opportunities and challenges across linguistic variation, historical documentation, cultural expressions, and literary analysis. The study emphasizes interdisciplinary collaboration and customized model development to preserve linguistic and cultural heritage while addressing issues of data accessibility, model adaptability, and cultural sensitivity.
Data-Efficient Adaptation of LLMs via Attention Head Reweighting
Introduces Attention Head Reweighting (AHR), a data-efficient method for adapting LLMs to text classification tasks by learning a single scalar per attention head, drastically reducing trainable parameters while outperforming LoRA in limited data settings.
UrduMMLU: A Massive Multitask Benchmark for Urdu Language Understanding
UrduMMLU is a new benchmark of 26,431 multiple-choice questions across 26 subjects for evaluating LLMs on Urdu language understanding, sourced from native educational materials. Evaluation of 30 LLMs reveals Gemini-3.5-Flash performs best, while open-source models and region-specific subjects pose significant challenges.
@jbhuang0604: LoRA, low-rank adaptation, is arguably the most popular parameter-efficient fine-tuning method for LLMs. But how does i…
LoRA (low-rank adaptation) is the most popular parameter-efficient fine-tuning method for LLMs. This video introduces how LoRA and its variants (LoRA+, QLoRA, VeRA, DoRA) work.