Polarization Detection: A Hybrid Approach with AfroXLMR-Social and DeBERTa for Low- and High-Resource Settings

# Polarization Detection: A Hybrid Approach with AfroXLMR-Social and DeBERTa for Low- and High-Resource Settings
Source: [https://arxiv.org/html/2607.10312](https://arxiv.org/html/2607.10312)
###### Abstract

The rapid proliferation of online polarization threatens social cohesion, necessitating robust automated detection systems that operate effectively across diverse linguistic contexts\. This paper presents our system description for the POLAR Shared Task 2026, focusing on the detection and characterization of polarized discourse in English and Hausa\. We propose a hybrid modeling strategy: for English binary detection, we leverage the monolingual strength ofDeBERTa, while for Hausa and all fine\-grained subtasks \(Types and Manifestations\), we utilizeAfroXLMR\-Social\. This domain\-adapted multilingual model proved critical for capturing the nuances of polarization in social media text\. To further address computational constraints and data scarcity, we implement Low\-Rank Adaptation \(LoRA\) and textual data augmentation vianlpaug\. We report competitive results across all three subtasks, demonstrating that model selection tailored to specific subtask requirements yields the best balance of performance\.[https://github\.com/mohdasaid/NLP\-LLM/blob/main/Polarization\.ipynb](https://github.com/mohdasaid/NLP-LLM/blob/main/Polarization.ipynb)

Polarization Detection: A Hybrid Approach with AfroXLMR\-Social and DeBERTa for Low\- and High\-Resource Settings

Muhammad Abdullahi SaidAfrican Institute for Mathematical Sciences \(AIMS\)mohdasaid@aims\.ac\.za

## 1Introduction

Online polarization defined as the sharp division of public opinion into opposing factions characterized by hostility and lack of empathy has become a pervasive issue in digital discourse\. The POLAR Shared Task 2026 aims to benchmark Natural Language Processing \(NLP\) systems on their ability to detect this phenomenon across a wide range of languages and cultures\(Naseem et al\.,[2025](https://arxiv.org/html/2607.10312#bib.bib5)\)\.

Detecting polarization extends beyond traditional sentiment analysis\. It requires capturing subtle rhetorical strategies such as "us vs\. them" framing, invalidation, and dehumanization\. This complexity is exacerbated in low\-resource languages like Hausa, where labeled data is scarce and standard multilingual models often lack sufficient exposure to dialectal variations\.

In this work, we propose a strategic, hybrid training pipeline\. Our key contributions are:

1. 1\.Task\-Specific Model Selection: We observe that while English benefits from specialized monolingual models for binary detection \(Subtask 1\), the fine\-grained tasks \(Subtasks 2 & 3\) in both languages require the robust social\-media understanding provided by AfroXLMR\-Social\(Belay et al\.,[2025](https://arxiv.org/html/2607.10312#bib.bib2)\)\.
2. 2\.Domain\-Adaptive Efficacy: We demonstrate that AfroXLMR\-Social, a model continued\-pre\-trained on African social media content, is indispensable for the Hausa subtasks, significantly outperforming generic multilingual baselines\.
3. 3\.Parameter Efficiency & Augmentation: We employ LoRA \(Low\-Rank Adaptation\)\(Hu et al\.,[2021](https://arxiv.org/html/2607.10312#bib.bib3)\)to fine\-tune effectively on consumer hardware and usenlpaug\(Ma,[2019](https://arxiv.org/html/2607.10312#bib.bib4)\)to generate synthetic examples, addressing severe class imbalance in the manifestation labels\.

## 2Related Work

### 2\.1Polarization Detection

Early computational approaches to polarization relied on network topology, inferring ideological stance from retweet or follower graphs\. However, content\-based approaches using Transformers have recently gained traction\. Studies have shown that models like BERT and RoBERTa can detect stance and toxicity, which are proxies for polarization\. The POLAR benchmark advances this by explicitly taxonomizing polarization into types \(political, religious\) and rhetorical manifestations\.

### 2\.2Multilingual Models for African NLP

Massively multilingual models like mBERT and XLM\-R often underperform on African languages due to the "curse of multilinguality," where capacity is diluted across too many languages\.AfroXLMR\-Social\(Belay et al\.,[2025](https://arxiv.org/html/2607.10312#bib.bib2)\)builds upon the success of the original AfroXLMR\(Alabi et al\.,[2022](https://arxiv.org/html/2607.10312#bib.bib1)\)by incorporating "Domain\-Adaptive Pre\-training" \(DAPT\) on social media corpora\. This adaptation is crucial for Hausa, as it aligns the model’s internal representations with the noisy, informal, and code\-mixed text styles found in the POLAR dataset\.

### 2\.3Efficient Fine\-Tuning

As model sizes grow, full fine\-tuning becomes prohibitively expensive\. Parameter\-Efficient Fine\-Tuning \(PEFT\) methods like Adapters and LoRA have emerged as standard solutions\. LoRA, in particular, has been shown to match or exceed full fine\-tuning performance in low\-data regimes by reducing the risk of catastrophic forgetting\. By freezing the pre\-trained backbone, we maintain the linguistic knowledge acquired during the extensive DAPT phase\.

## 3Task Description

We address three hierarchical subtasks provided by the organizers:

1. 1\.Subtask 1 \(Detection\):A binary classification task to determine if a textxxcontains polarized content \(y∈\{0,1\}y\\in\\\{0,1\\\}\.
2. 2\.Subtask 2 \(Type\):A multi\-label classification task identifying the topic of polarization\. Labels includePolitical,Religious,Racial/Ethnic,Gender/Sexual Identity, andOther\.
3. 3\.Subtask 3 \(Manifestation\):A multi\-label task identifying rhetorical devices used to express polarization\. Labels includeVilification,Stereotyping,Dehumanization,Extreme Language,Lack of Empathy, andInvalidation\.

## 4Methodology

### 4\.1Model Architecture Strategy

#### 4\.1\.1English Subtask 1: DeBERTa

For the binary detection task in English, we utilizedDeBERTa\-v3\-base\. DeBERTa improves upon BERT and RoBERTa by using disentangled attention and an enhanced mask decoder, making it highly effective for subtle classification tasks in high\-resource languages where standard grammar prevails\.

#### 4\.1\.2Hausa & Subtasks 2/3: AfroXLMR\-Social

For all Hausa tasks and the fine\-grained English tasks \(Types and Manifestations\), we employedAfroXLMR\-Social\. The complexity of Subtasks 2 and 3 requires a model that understands the social context of language slang, hashtags, and informal phrasing which is the core strength of this domain\-adapted checkpoint\.

### 4\.2Low\-Rank Adaptation \(LoRA\)

To adapt these models efficiently, we freeze the pre\-trained backbone and inject trainable low\-rank matrices \(A,BA,B\) into the Query \(QQ\) and Value \(VV\) attention projections\.

h=W0​x\+αr​B​A​xh=W\_\{0\}x\+\\frac\{\\alpha\}\{r\}BAx\(1\)We used rankr=8r=8and scaling factorα=16\\alpha=16\. This approach allowed us to fine\-tune distinct models for each subtask without exceeding GPU memory limits\.

### 4\.3Data Augmentation

We utilizednlpaug\(Ma,[2019](https://arxiv.org/html/2607.10312#bib.bib4)\)to robustly handle class imbalance\.

- •Synonym Replacement:We replaced up to 10% of words with synonyms from WordNet\.
- •Random Insertion:We inserted contextually relevant words to vary sentence structure\.

This was crucial for Subtask 3, where classes likeDehumanizationhad very few positive examples\.

### 4\.4Training Strategy

#### 4\.4\.1Loss Function

For Subtask 1, we minimize Binary Cross\-Entropy \(BCE\)\. For Subtasks 2 and 3, we useBCEWithLogitsLoss, which applies a sigmoid activation to each class logit independently:

ℒ=−1C​∑c=1C\[yc​log⁡\(σ​\(zc\)\)\+\(1−yc\)​log⁡\(1−σ​\(zc\)\)\]\\mathcal\{L\}=\-\\frac\{1\}\{C\}\\sum\_\{c=1\}^\{C\}\[y\_\{c\}\\log\(\\sigma\(z\_\{c\}\)\)\+\(1\-y\_\{c\}\)\\log\(1\-\\sigma\(z\_\{c\}\)\)\]\(2\)

#### 4\.4\.2Dynamic Learning Rate

We implemented a linear warmup for the first 10% of steps, followed by a cosine decay\. This schedule proved essential for stabilizing the training of the DeBERTa model, which can be sensitive to initialization\.

## 5Experimental Setup

### 5\.1Data and Preprocessing

The data was provided in CSV format\. We cleaned the text by removing URL artifacts but retained emojis and hashtags, as they are strong indicators of sentiment in social media\. We used the XLM\-R tokenizer with a maximum sequence length of 128\.

### 5\.2K\-Fold Cross\-Validation

We employed 5\-Fold Cross\-Validation\. For each language, the training data was split into 5 stratified folds\. We trained 5 independent models, each using 4 folds for training and 1 for validation\. The final test predictions were generated using soft voting from the ensemble of 5 models\.

### 5\.3Hyperparameters

Table[1](https://arxiv.org/html/2607.10312#S5.T1)details the configuration used\.

Table 1:Hyperparameters for both English and Hausa\.

## 6Results

### 6\.1Subtask 1: Polarization Detection

Table[2](https://arxiv.org/html/2607.10312#S6.T2)presents the detection results\. The switch to DeBERTa for English yielded a noticeable improvement in F1\-Score over our initial multilingual baselines\. For Hausa, AfroXLMR\-Social remained superior\.

Table 2:Subtask 1 Results \(Validation Average\)\.
### 6\.2Subtasks 2 & 3: Fine\-grained Tasks

Tables[3](https://arxiv.org/html/2607.10312#S6.T3)and[4](https://arxiv.org/html/2607.10312#S6.T4)summarize the multi\-label results\.AfroXLMR\-Social performed exceptionally well here for both languages\.Its pre\-training on social media data likely allowed it to better recognize the rhetorical "manifestations" \(Subtask 3\) which are often signaled by informal social cues rather than formal vocabulary\.

Table 3:Subtask 2 \(Type\) using AfroXLMR\-Social\.Table 4:Subtask 3 \(Manifestation\) using AfroXLMR\-Social\.

## 7Discussion

### 7\.1The Dominance of AfroXLMR\-Social

A key finding of our experiments is the robustness ofAfroXLMR\-Socialfor the complex tasks \(2 and 3\)\. Even for English, where monolingual models usually dominate, AfroXLMR\-Social provided competitive and stable results for determining polarization types and manifestations\. We hypothesize this is because the "social" pre\-training exposes the model to the exact kind of toxic and polarized discourse patterns that these subtasks aim to classify, regardless of the language\.

### 7\.2Challenges with Augmentation

Whilenlpaugimproved recall for minority classes, it occasionally introduced semantic drift\. For instance, replacing "regime" with "government" in a political post might subtly alter the polarized tone \(negative to neutral\)\. Future work should explore embedding\-based augmentation \(BERT\-based insertion\) to preserve semantic consistency better\.

## 8Conclusion

Our system demonstrates that combining a domain\-adapted multilingual backbone with efficient fine\-tuning \(LoRA\) is a highly effective strategy for polarization detection in low\-resource languages\.

## References

- Alabi et al\. \(2022\)Jesujoba O Alabi, David I Adelani, Marius Mosbach, and Dietrich Klakow\. 2022\.Adapting pre\-trained language models to african languages via multilingual adaptive fine\-tuning\.In*Proceedings of COLING*, pages 4336–4349\.
- Belay et al\. \(2025\)Tadesse Destaw Belay, Israel Abebe Azime, and Ibrahim Said Ahmad\. 2025\.Afroxlmr\-social: Adapting pre\-trained language models for african languages social media text\.*arXiv preprint arXiv:2503\.18247*\.
- Hu et al\. \(2021\)Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\. 2021\.Lora: Low\-rank adaptation of large language models\.*arXiv preprint arXiv:2106\.09685*\.
- Ma \(2019\)Edward Ma\. 2019\.[Nlp augmentation](https://pypi.org/project/nlpaug/)\.*PyPI*\.
- Naseem et al\. \(2025\)Usman Naseem, Juan Ren, Saba Anwar, and Sarah Kohail\. 2025\.Polar: A benchmark for multilingual, multicultural, and multi\-event online polarization\.*arXiv preprint arXiv:2505\.20624*\.

## Appendix AAppendix: Detailed Experimental Settings

### A\.1Hardware and Compute

All models were trained on a single NVIDIA T4 GPU \(16GB VRAM\) using the Google Colab environment\. The average training time per fold was approximately 20 minutes for Hausa and 15 minutes for English\.

### A\.2Augmentation Details

For thenlpaugimplementation, we used the SynonymAug and RandomWordAug classes\.

- •Synonym Augmentation:a​u​g​\_​p=0\.1aug\\\_p=0\.1\(probability of augmenting a token\)\.
- •Stop Words:We utilized the NLTK stop word list to prevent the augmentation of structurally important words\.

### A\.3Label Distribution

The POLAR dataset exhibits significant class imbalance\. In Subtask 3, categories likeDehumanizationwere significantly rarer thanVilification\. The use of weighted loss functions was considered but ultimately the augmentation strategy proved more effective in preliminary trials\.