Language Specificity vs. Domain Diversity: Benchmarking Transformers for Bangla Medical NER
Summary
This paper benchmarks transformer models for Bangla medical Named Entity Recognition, demonstrating that fine-tuned XLM-RoBERTa achieves state-of-the-art performance and outperforms language-specific models, emphasizing the importance of domain diversity in low-resource settings.
View Cached Full Text
Cached at: 09/25/26, 09:44 AM
# Language Specificity vs. Domain Diversity: Benchmarking Transformers for Bangla Medical NER Source: [https://arxiv.org/abs/2609.29101](https://arxiv.org/abs/2609.29101) [View PDF](https://arxiv.org/pdf/2609.29101) > Abstract:Medical Named Entity Recognition \(NER\) for low\-resource languages remains a challenging task due to high linguistic variability and a scarcity of domain\-specific annotated corpora\. This work presents a comprehensive empirical benchmark evaluating three fine\-tuned transformer encoders\-BanglaBERT, multilingual BERT \(mBERT\), and XLM\-RoBERTa\-against GPT\-4o mini under zero\-shot and few\-shot prompting configurations for Bangla medical NER\. In contrast to prior studies that evaluated large language models on limited subsets of only 50 samples, we conduct a large\-scale evaluation across the full test set of 3,179 samples, providing statistically robust and reproducible baselines\. Our fine\-tuned XLM\-RoBERTa model achieves an F1\- score of 0\.5959, establishing a new state\-of\-the\-art and surpassing the previously reported best result of 0\.5848\. Crucially, we demonstrate that the language\-specific BanglaBERT model consistently underperforms its multilingual counterparts with an F1\-score of 0\.4937, indicating that pretraining domain diversity can outweigh language specificity in highly specialized clinical settings\. Furthermore, we present a detailed per\-entity\-type analysis for this task, revealing that Medicine and Specialist categories are recognized with high reliability, achieving F1\- scores above 0\.83, while the Symptom category remains the most challenging with an F1\-score of 0\.4367 despite being the most frequent training class\. Finally, fine\-tuned transformer models outperform the optimal prompting configuration by a factor of 3\.76, confirming that prompt\-only pipelines remain inadequate for structured clinical entity extraction in low\-resource language environments\. ## Submission history From: Rakib Abdullah \[[view email](https://arxiv.org/show-email/c08e8643/2609.29101)\] **\[v1\]**Thu, 24 Sep 2026 06:32:55 UTC \(585 KB\)
Similar Articles
BanglaRhet: Benchmarking Classical and Transformer Models for Rhetorical and Persuasion Detection in Bangla Political Speech
The paper introduces BanglaRhet, a 30,289-segment annotated corpus of Bangla political speech, and benchmarks classical and transformer models (BanglaBERT, SahajBERT, XLM-RoBERTa) for rhetorical and persuasion technique detection, with BanglaBERT achieving the best macro-F1 of 66.46%.
Transformer-Based Language Models Across Domain Verticals: Architectures, Applications and Critical Assessment
A comprehensive survey of transformer-based language models covering architectures, applications across domain verticals (healthcare, finance, legal, etc.), and critical assessment of trade-offs including compute cost, alignment, and data provenance.
BERT-based Models vs. Large Language Models for Low-Resource Named Entity Recognition: A Comparative Study on Marathi
This paper compares fine-tuned MahaBERT-based models with large language models (Gemini, LLaMA-3.3-70B, Gemma) for Marathi named entity recognition, finding that the specialized BERT models significantly outperform the LLMs, achieving F1-scores of 0.88–0.91 versus 0.57–0.69.
Beyond Clean Text: Evaluating Encoder and Decoder Robustness for Bangla Event Detection in Noisy Text
This paper introduces a Bangla event detection benchmark with noisy text (ASR, orthographic corruption) and evaluates encoder-only and decoder-only LLMs, finding decoder models more robust to noise.
When a Name Is Not a Name: A Benchmark Dataset and Distilled Reasoning for Culturally Entangled Bangla Homographs in Low-Resource LLMs
This paper introduces a benchmark dataset of 1,516 expert-verified Bangla sentences for disambiguating culturally entangled homographs (words that are both names and common nouns). It shows that LLMs suffer from dominant-meaning bias and proposes contrastive chain-of-thought prompting and distillation to reduce this bias.