Hate Speech Detection in Turkish and Arabic Languages: A Comprehensive Study

# Hate Speech Detection in Turkish and Arabic Languages: A Comprehensive Study
Source: [https://arxiv.org/html/2607.00143](https://arxiv.org/html/2607.00143)
\\yil\\vol\\fpage\\lpage

Elif EROL4[https://orcid.org/0009-0000-1554-0622](https://orcid.org/0009-0000-1554-0622), Arzucan ÖZGÜR3[https://orcid.org/0000-0001-8376-1056](https://orcid.org/0000-0001-8376-1056), Berrin YANIKOGLU1,2[https://orcid.org/0000-0001-7403-7592](https://orcid.org/0000-0001-7403-7592) 1Department of Computer EngineeringSabanci UniversityIstanbulTurkey 34956 2Center of Excellence in Data Analytics \(VERIM\)Sabanci UniversityIstanbulTurkey 34956 3Department of Computer EngineeringBogazici UniversityIstanbulTurkey 34342 4Hrant Dink FoundationIstanbulTurkey 34373 \\rec

###### Abstract

Online hate speech has been linked to a global rise in violence against minorities, including incidents such as mass shootings, lynchings, and ethnic cleansing\. Societies grappling with this issue, particularly when hate speech targets specific groups based on religion, race, ethnicity, culture, nationality, or migration status, face the challenge of balancing freedom of expression with the need for effective content moderation on widely used online platforms\. In response to this challenge, we introduce a comprehensive hate speech dataset covering five distinct topics in Turkish: refugees, the Israel\-Palestine conflict, anti\-Greek sentiment in Turkey, ethnic or religious communities \(Alevis, Armenians, Arabs, Jews, and Kurds\), and LGBTI\+, alongside one topic in Arabic \(refugees\)\. In addition, we develop state\-of\-the\-art BERT\-based models to address multiple dimensions of hate speech analysis, including hate category classification, hate intensity prediction, target identification, and hate speech span detection, enabling a comprehensive understanding of hateful content in online discourse\.

###### keywords:

Hate Speech Detection, Hate Intensity Prediction, Hate Speech Target Identification, Hate Speech Span Detection, NLP, BERT, ChatGPT

## Disclaimer:

This study contains examples of offensive language and hate speech for research purposes\. These examples do not reflect the authors’ views and are included solely to support the detection and prevention of harmful content targeting vulnerable communities\.

## 1Introduction

With the widespread use of social media, online platforms have increasingly become spaces where hate speech can spread rapidly\. Such content fosters hostility and intolerance and, in some cases, contributes to real\-world violence targeting religious, racial, ethnic, and gender\-based groups\. As a result, detecting and mitigating hate speech has become a critical challenge for both online platforms and policymakers\. To address this issue, researchers have increasingly turned to Natural Language Processing \(NLP\) techniques, which enable the automatic identification and analysis of hateful content in large volumes of social media text\.

Hate speech detection is a challenging task due to both the complexity of defining hate speech and the nuances of language\. In recent years, numerous studies have focused on developing automatic methods for detecting hate speech in social media\. However, there is a limited amount of research on hate speech detection in Turkish and Arabic\. Early approaches to hate speech detection focused on using lexicons of manually chosen hateful keywords\[[1](https://arxiv.org/html/2607.00143#bib.bib1),[2](https://arxiv.org/html/2607.00143#bib.bib2)\]\. However, this technique tends to be limited in effectiveness, as hate speech is not always overt\. More recent research in the field, particularly for English, has shifted toward n\-grams, TF\-IDF, and word embedding techniques \(e\.g\. Word2Vec, GLoVe\)\[[3](https://arxiv.org/html/2607.00143#bib.bib3),[4](https://arxiv.org/html/2607.00143#bib.bib4),[5](https://arxiv.org/html/2607.00143#bib.bib5)\]\.

More recent advancements leverage large language models \(e\.g\., BERT, RoBERTa, ConvBERT, mBERT, and XLM\-R\) for hate speech detection\[[6](https://arxiv.org/html/2607.00143#bib.bib6),[7](https://arxiv.org/html/2607.00143#bib.bib7),[8](https://arxiv.org/html/2607.00143#bib.bib8),[9](https://arxiv.org/html/2607.00143#bib.bib9),[10](https://arxiv.org/html/2607.00143#bib.bib10),[11](https://arxiv.org/html/2607.00143#bib.bib11),[12](https://arxiv.org/html/2607.00143#bib.bib12),[13](https://arxiv.org/html/2607.00143#bib.bib13)\]\. Additionally, Most research in this domain adopts a binary approach to hate speech classification\. However, contemporary studies have recognized the limitations of this approach, prompting a shift towards multi\-class classification to gain a better understanding of the nature and dimensions of hate speech\[[14](https://arxiv.org/html/2607.00143#bib.bib14),[15](https://arxiv.org/html/2607.00143#bib.bib15)\]\.

Furthermore, predicting the intensity of hate serves as a valuable metric to assess the degree of hate and offensive speech\[[16](https://arxiv.org/html/2607.00143#bib.bib16)\]\. Hate intensity prediction, or hate speech strength prediction, involves assessing the level \(degree\) or severity \(strength\) of hatred expressed within a given message\. This task goes beyond simply classifying a message as hateful or non\-hateful; it aims to quantify how intense or extreme the hateful sentiment is\. By determining the degree of hatred, this approach provides a more granular understanding of offensive content, enabling researchers and practitioners to differentiate between mild and severe expressions of hate\[[17](https://arxiv.org/html/2607.00143#bib.bib17),[18](https://arxiv.org/html/2607.00143#bib.bib18),[16](https://arxiv.org/html/2607.00143#bib.bib16)\]\.

Despite a significant increase in hate speech detection models, many of these studies fail to address the identification of specific targets of hate speech, particularly in the Turkish and Arabic languages\. Identification of the target group\(s\) is also essential for understanding the scope and potential impact of hate speech, as messages aimed at specific groups such as refugees, women, or LGBTI\+ communities pose distinct societal risks\. Unfortunately, the lack of comprehensive datasets that explicitly annotate these targets has hampered progress in this area\[[19](https://arxiv.org/html/2607.00143#bib.bib19),[20](https://arxiv.org/html/2607.00143#bib.bib20)\]\.

Furthermore, pinpointing the precise spans of hateful content within a text, rather than just classifying the entire message, is another promising approach that has seen limited exploration\. Span detection provides more granular identification of offensive content, offering better insights into the nature and severity of hate speech\[[21](https://arxiv.org/html/2607.00143#bib.bib21)\]\. Unlike traditional classification that labels entire texts, span detection identifies the specific segments responsible for the expression of hate speech\. This method is particularly beneficial for human moderators, who often deal with lengthy, tedious comments and prefer clear explanations over system\-generated tags that lack context\. Recent studies have also demonstrated that structured explanations, such as highlighting specific hateful spans, can improve moderator efficiency by reducing decision\-making time\[[22](https://arxiv.org/html/2607.00143#bib.bib22)\]\.

In this study, we introduce a new dataset and perform a comprehensive analysis of hate speech discourse in Turkish and Arabic\. Unlike previous works that often focus on only one or two aspects, our study encompasses hate speech classification, hate intensity prediction, target group identification, and span detection using BERT\-based models\. By addressing these interrelated tasks together, we aim to provide a deeper understanding of hate speech discourse, its intensity, and its targeted nature, offering a holistic perspective that advances the field\.

Our main contributions are as follows:

- •We introduce a novel and extensive hate speech dataset that covers five distinct topics—refugees, the Israel\-Palestine conflict, anti\-Greek sentiment in Turkey, LGBTI \+ and religion/race/ethnicity—in Turkish, along with one topic in Arabic \(refugees\)\.
- •We develop state\-of\-the\-art BERT\-based models for multiple hate speech analysis tasks, including classification, intensity prediction, target identification, and span detection\.
- •We leverage ChatGPT for hashtag segmentation and synthetic data generation to improve data quality and mitigate class imbalance\.
- •We provide the first comprehensive benchmark for multi\-dimensional hate speech analysis in Turkish and Arabic\.

The paper is organized as follows: In Section[2](https://arxiv.org/html/2607.00143#S2), we introduce our hate speech dataset, along with the data annotation process and statistical analysis of the inter\-annotator agreement\. In Section[3](https://arxiv.org/html/2607.00143#S3), we detail our proposed approach, covering the architecture of our transformer\-based models, the baseline model, dual contrastive learning for hate speech classification, a regression model for predicting hate speech intensity, a multi\-label classification model for target identification, and token classification model for span detection, hashtag segmentation and synthetic data generation using ChatGPT\. Section[4](https://arxiv.org/html/2607.00143#S4)presents our experimental results, highlighting performance gains\. Finally, Section[5](https://arxiv.org/html/2607.00143#S5)summarizes our findings and outlines future research directions\.

## 2Hate Speech Dataset Introduced in This Study

We collected data from X, formerly Twitter, on five topics in Turkish—immigrants and refugees, the Israel\-Palestine conflict, anti\-Greek sentiment in Turkey, ethnic or religious communities \(Alevis, Armenians, Arabs, Jews, and Kurds\), and LGBTI\+—as well as one topic in Arabic, specifically immigrants and refugees\. Our dataset partially overlaps with two publicly available datasets\[[7](https://arxiv.org/html/2607.00143#bib.bib7),[23](https://arxiv.org/html/2607.00143#bib.bib23)\], which cover four topics: immigrants and refugees, the Israel\-Palestine conflict, anti\-Greek sentiment, and the Istanbul Convention\. Furthermore, our dataset represents the finalized version of the one used in\[[24](https://arxiv.org/html/2607.00143#bib.bib24)\], during a big collaborative project on the topic\. The initial set of tweets collected within this project has already been shared in\[[23](https://arxiv.org/html/2607.00143#bib.bib23),[25](https://arxiv.org/html/2607.00143#bib.bib25),[10](https://arxiv.org/html/2607.00143#bib.bib10),[26](https://arxiv.org/html/2607.00143#bib.bib26),[27](https://arxiv.org/html/2607.00143#bib.bib27)\]\. The topics covered in the dataset are as follows:

Immigrants and Refugees in Turkey:In recent years, the civil wars in Syria and Afghanistan have driven countless individuals to seek refuge in Turkey\. As of 2023, around 3\.4 million Syrians111https://multeciler\.org\.tr/eng/number\-of\-syrians\-in\-turkey/and approximately 300,000 Afghans222https://www\.voanews\.com/a/afghan\-refugees\-in\-turkey\-hope\-for\-relocation\-fear\-deportation/7400549\.htmlhave settled in the country\. Initially, public sentiment toward refugees was largely supportive; however, as the number of asylum seekers grew, challenges associated with their integration and disinformation about refugees receiving privileges unavailable to Turkish citizens have contributed to increasing negative attitudes\. This trend is not unique to Turkey; similar shifts in public opinion have been observed globally, often resulting in heightened hostility and hate speech targeting refugees on social media platforms\.

Israel\-Palestine Conflict:The Israel–Palestine conflict, ongoing since the mid\-20th century, remains one of the most complex and divisive disputes in modern history\. Public discourse on the issue has long been shaped by deeply polarized opinions between pro\-Israeli and pro\-Palestinian groups\. In Turkey, the conflict has sparked significant debate and strong reactions, underscoring its broader regional and global implications\. It is important to emphasize that our dataset was collected prior to October 7, 2023, and therefore does not capture the subsequent escalation of hate speech\.

Anti\-Greek Sentiment in Turkey:Anti\-Hellenism, or anti\-Greek sentiment, refers to hostility toward Greeks, Greek culture, or Greece\. Relations between Turkey and Greece have long been shaped by disputes over issues such as Aegean sovereignty, territorial waters, airspace, and minority rights\. In 2022, Greece’s increased military presence on certain Aegean islands further escalated tensions between the two countries333https://www\.dailysabah\.com/politics/eu\-affairs/greece\-scales\-up\-crete\-naval\-base\-armament\-drive\. Such developments often contribute to the spread of anti\-Greek rhetoric and hate speech in public discourse\.

Ethnic or religious communities \(Alevis, Armenians, Arabs, Jews, and Kurds\):In addition to the main topics, hate speech frequently targets ethnic and religious communities in Turkey and the Middle East\. Groups such as Alevis, Armenians, Arabs, Jews, Kurds, and Roma are often subjected to discriminatory and hateful discourse shaped by historical, political, and social tensions\. To capture these forms of hate speech, we included tweets targeting these communities in our dataset\.

LGBTI\+:In Turkey and many Muslim\-majority countries, negative attitudes toward LGBTI\+ individuals are often influenced by cultural, religious, and social norms\. Conservative interpretations of Islam and traditional views on family structures and gender roles can contribute to societal discrimination and hostility toward LGBTI\+ communities\. In some contexts, political rhetoric further reinforces these attitudes, increasing the prevalence of anti\-LGBTI\+ hate speech\.

### 2\.1Data Annotation

We use a prescriptive annotation method to assign tweets to specific hate speech categories and a descriptive approach to rate the perceived intensity of hate speech on a scale of\[0,10\]\[0,10\]\. The annotation guidelines were refined iteratively to reduce ambiguities in categorization\. Hate speech intensity was evaluated separately from the categories, providing a secondary metric to examine its relationship with the hate speech classifications\.

The tweets were grouped into batches of 50 and assigned to three annotators via Label Studio444https://labelstud\.io/\. Each annotator was instructed to label the tweets individually, following the provided guidelines\. To maintain label quality, each tweet was independently annotated by three different individuals, with each batch being reviewed by three separate annotators across distinct ports\. Multiple selections were allowed for the “Target Group” and “Hate Speech Category” labels, as multiple groups can be targeted inside one tweet\. In such cases, we split the annotator vote among the selected groups or categories\. Our annotation guidelines are made publicly available555[English Annotation Guideline Document, HDV Publications](https://hrantdink.org/attachments/article/4413/UTILIZING%20AI%20AGAINST%20HATE%20SPEECH%20A%20guide%20to%20annotation,%20classification,%20and%20detection.pdf),666[Turkish Annotation Guideline Document, HDV Publications](https://hrantdink.org/attachments/article/4412/NEFRET%20S%C3%96YLEM%C4%B0YLE%20M%C3%9CCADELEDE%20YAPAY%20ZEK%C3%82%20Etiketleme,%20s%C4%B1n%C4%B1fland%C4%B1rma%20ve%20tespit%20k%C4%B1lavuzu.pdf)\. The following six categories were decided for a comprehensive labelling:

1. 0\.No Hate Speech:Tweet does not contain hate speech\.
2. 1\.Exclusionary/Discriminatory Discourse:These are narratives in which an identity is portrayed as inherently inferior or undeserving compared to the dominant group regarding societal benefits, rights, and freedoms\. For example, statements like “Mülteciler sağlık hizmetlerinden yararlanmasın” \(Refugees should not access healthcare\) and “Kadınlar iş dünyasında daha az yer almalı” \(Women should have a lesser role in the workforce\) exemplify discriminatory discourse\.
3. 2\.Exaggeration, Generalization, Attribution, Distortion:These are discourses that generalize individual events or situations to an entire group, manipulate facts by distorting them, or attribute isolated incidents to an entire identity based on the actions of a few\. For instance, expressions like “Yunan yine kışkırtma peşinde \(The Greeks seek provocation again\)” and “Mültecilerin aldıklariyardımlar yüzünden halkımız fakir kaldi\(Our people is poor because of the aid refugees receive\)” following specific and unrelated incidents illustrate such manipulative and generalized hate speech\.
4. 3\.Symbolization:These are discourses in which an aspect of identity is employed as a means of insult, hatred, or humiliation, often symbolizing the identity in a derogatory manner\. For instance, expressions like “Bizi Eurovision’da Yahudi mi temsil edecek?” \(Will the Jew represent us at Eurovision?\) exemplify such hateful and symbolic discourse\.
5. 4\.Swearing, Insult, Defamation, Dehumanization:These are discourses that involve direct profanity, insults, or contempt towards an identity, often dehumanizing them by describing their actions or characteristics with terms typically used for non\-human beings\. For example, “Küstah Rum’a Gözdağı” \(Intimidation to the Arrogant Greek\) and “Danimarkaliitler iftar sofralarınibasıp Kuran\-iKerim yaktılar \(Danish dogs raided iftar and burned the Quran\)” reflect such hateful and dehumanizing expressions\.
6. 5\.Threat of Enmity, War, Attack, Murder, or Harm:These are discourses that include hostile expressions towards an identity, often evoking the threat of war or violence, or expressing a desire to harm the targeted group\. For example, “Yunanistan ateşle oynuyor; bir gece ansızın gelebiliriz\! \(Greece is playing with fire; we may come unexpectedly one night\!\)” reflects such a threatening discourse\.

In the following, we provide statistics for the binary and multi\-class classification problems in Tables[1](https://arxiv.org/html/2607.00143#S2.T1),[2](https://arxiv.org/html/2607.00143#S2.T2),[3](https://arxiv.org/html/2607.00143#S2.T3), and[4](https://arxiv.org/html/2607.00143#S2.T4)\. For multi\-class categorization, we used two different settings\. One of the settings involves 6 classes \(fine\-grained\), which are the “no hate speech” class and the five categories in the Table[3](https://arxiv.org/html/2607.00143#S2.T3)\. In the other setting, we designed a 4\-class classification \(coarse\-grained\) model by merging the categories 2 and 3 as a single category, and 4 and 5 as a single category for simplifying the task based on\[[24](https://arxiv.org/html/2607.00143#bib.bib24)\], shown in Table[2](https://arxiv.org/html/2607.00143#S2.T2)\. For training hate speech classification models, we used only instances with agreements or those where majority voting is clear for hate speech category annotation\.

### 2\.2Target Annotation

Target annotations were also assigned to each tweet in the dataset as multi\-label annotations, as shown in Table[5](https://arxiv.org/html/2607.00143#S2.T5), which presents target statistics for the Turkish and Arabic datasets\. The target categories are defined as follows:

1. 0\.Target group not specified or not present:Cases where the target identity is vague or not explicitly stated\.
2. 1\.Country, Nationality, Race, Ethnicity:Instances where an identity is targeted based on their country, nationality, race or ethnicity\. This includes categories such as refugees, Greeks, Armenians, Kurds, Roma, and Arabs\.
3. 2\.Religion:Instances where an identity face discrimination due to their religious beliefs\. The relevant categories in this study include Jews and Alevis\.
4. 3\.Gender, Sexual Orientation:Discourses where an identity is targeted based on their gender or sexual orientation\. The categories in this project include LGBTI\+s and Women\.

Table 1:Statistics for 2\-class labelled in Turkish tweetsTable 2:Statistics for 4\-class labelled in Turkish tweets\.Table 3:Statistics for 6\-class labelled in Turkish tweets\.Table 4:Statistics for 2\-class, 4\-class, and 6\-class labeled in Arabic tweets onRefugeestopicTable 5:Statistics on hate directed towards different general targets in our Turkish and Arabic datasetTable 6:Distribution of tweets and spans across hate speech categories\.
### 2\.3Span Annotation

Beyond tweet\-level annotations, we also conducted span\-level annotations to identify the specific segments responsible for hate speech, following the annotation methodology described in\[[27](https://arxiv.org/html/2607.00143#bib.bib27)\]\. Initial span annotations were generated using ChatGPT and subsequently refined by human annotators to correct errors and remove hallucinated spans\. After the manual refinement process, the final span\-level dataset consists of 2,981 tweets and 4,465 hateful spans\. Detailed statistics regarding category is presented in Tables[6](https://arxiv.org/html/2607.00143#S2.T6)\.

## 3Methodology

In this section, we first introduce our baseline model \(Section[3\.1](https://arxiv.org/html/2607.00143#S3.SS1)\), then we introduce our dual contrastive learning model for hate speech classification \(Section[3\.2](https://arxiv.org/html/2607.00143#S3.SS2)\)\. Then, we describe our target identification and span detection models \(Sections[3\.4](https://arxiv.org/html/2607.00143#S3.SS4)and[3\.5](https://arxiv.org/html/2607.00143#S3.SS5)\)\. After that, we describe our zero\-shot strategy and few\-shot strategy for hashtag segmentation and synthetic data generation using ChatGPT, respectively \(Sections[3\.6](https://arxiv.org/html/2607.00143#S3.SS6)\)\.

### 3\.1Baseline Model for Hate Speech Classification Problem

We design our baseline model using transfer learning, with a single layer on top of the learned encoder \(BERT\)\[[28](https://arxiv.org/html/2607.00143#bib.bib28)\]to predict the hate speech categories using cross\-entropy loss as follows:

LC​E=−∑i=1Nyi​l​o​g​\(y^i\)L\_\{CE\}=\-\\sum\_\{i=1\}^\{N\}y\_\{i\}log\(\\hat\{y\}\_\{i\}\)\(1\)whereyiy\_\{i\}is the target value for theiith input andy^i\\hat\{y\}\_\{i\}is the prediction\.

### 3\.2Dual Contrastive BERT Model for Hate Speech Classification

We also design our dual contrastive model using transfer learning, with a single layer on top of the learned encoder \(BERT\) to predict the hate speech categories as in\[[10](https://arxiv.org/html/2607.00143#bib.bib10)\]\. As we use a contrastive design, the input of our model is a batch of tweets including both hate and non\-hate text \(binary\-class or multi\-class\)\. The model is trained using a dual contrastive loss function that is a combination of cross\-entropy loss \(LC​EL\_\{CE\}\) and supervised contrastive loss \(LS​C​LL\_\{SCL\}\):

LD​u​a​l​C​L=LC​E\+λ​LS​C​LL\_\{DualCL\}=L\_\{CE\}\+\\lambda L\_\{SCL\}\(2\)
whereλ∈\[0,1\]\\lambda\\in\\left\[0,1\\right\]is a weighting hyperparameter that controls the impact of these two loss functions\. We tried differentλ\\lambdavalues and observed that the best performance forλ=0\.5\\lambda=0\.5\. The two objectives learn classifier parameters and improve the the quality of the representations of the features, simultaneously\.

As our supervised contrastive loss \(LS​C​L\)L\_\{SCL\}\), we use the loss introduced in\[[29](https://arxiv.org/html/2607.00143#bib.bib29)\]\. They extended the self\-supervised NT\-Xent loss\[[30](https://arxiv.org/html/2607.00143#bib.bib30)\]to the fully supervised setting by adding multiple positivesfrom the same class\. The NT\-Xent loss only accepts one positive samplexi\+x\_\{i\}^\{\+\},obtained via augmentation, for an anchorxix\_\{i\}, and uses the other samples in the mini\-batch as negatives, obtaining a mini\-batch of\(xi,xi\+,x1−,…,xK−\)\.\(x\_\{i\},x\_\{i\}^\{\+\},x\_\{1\}^\{\-\},\.\.\.,x\_\{K\}^\{\-\}\)\.Supervised contrastive loss extends the number of positives in the mini\-batch toPPpositive samplesfrom the training set, obtaining a mini\-batch of\(xi,x1\+,…,xP\+,x1−,…,xK−\)\(x\_\{i\},x\_\{1\}^\{\+\},…,x\_\{P\}^\{\+\},x\_\{1\}^\{\-\},\.\.\.,x\_\{K\}^\{\-\}\)\. Our SCL is defined as:

LS​C​L=−1P​∑p=1Plog⁡e\(xi⋅xp\+/τ\)∑k=1Ke\(xi⋅xk−/τ\)L\_\{SCL\}=\-\\frac\{1\}\{P\}\\sum\_\{p=1\}^\{P\}\\log\{\\frac\{\{e^\{\(x\_\{i\}\\cdot x\_\{p\}^\{\+\}/\\tau\{\}\)\}\}\}\{\\sum\_\{k=1\}^\{K\}e^\{\(x\_\{i\}\\cdot x\_\{k\}^\{\-\}/\\tau\{\}\)\}\}\}\(3\)
wherexix\_\{i\}is the embedding vector of sentencesis\_\{i\},xp\+x\_\{p\}^\{\+\}andxk−x\_\{k\}^\{\-\}indicate positive and negative samples respectively; the symbol⋅\\cdotdenotes the inner product andτ∈R\+\\tau\\in R^\{\+\}denotes the temperature parameter set to0\.10\.1\.

### 3\.3Hate Intensity Prediction Problem

We design our regression model using transfer learning, with a single layer on top of the learned encoder \(BERT\) to predict the hate speech degree \(intensity\) using MSE loss as follows:

LM​S​E=1N​∑i=1N\(yi−y^i\)2L\_\{MSE\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\(y\_\{i\}\-\\hat\{y\}\_\{i\}\)^\{2\}\(4\)
whereyiy\_\{i\}andy^i\\hat\{y\}\_\{i\}are the desired values and predicted values, respectively\.

### 3\.4Multi\-label Target Classification Problem

We fine\-tune the pretrained BERT model by incorporating an additional output layer specifically for the labeling task\. This layer features a feedforward neural network with sigmoid activation, enabling the model to independently assign a score \(0/1\) to each label\. We use BCEWithLogitsLoss from PyTorch, which combines a Sigmoid layer and binary cross\-entropy loss \(BCE Loss\)\. BCE loss learns the probability distributionP​\(yi\|s→\)P\(y\_\{i\}\|\\overrightarrow\{s\}\), for a sentence feature vectors→\\overrightarrow\{s\}and the predicted label of theit​hi\_\{th\}class,i∈0,1i\\in\{0,1\},yiy\_\{i\}, as follows:

LB​C​E=−1N​∑i=1N\(yi⋅l​o​g​\(p​\(si\)\)\+\(1−yi\)⋅l​o​g​\(1−p​\(si\)\)\)L\_\{BCE\}=\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\(y\_\{i\}\\cdot log\(p\(s\_\{i\}\)\)\+\(1\-y\_\{i\}\)\\cdot log\(1\-p\(s\_\{i\}\)\)\\right\)\(5\)
where the probabilityp​\(si\)p\(s\_\{i\}\)is the predicted probability that instance \(sentence\)sis\_\{i\}belongs to the default class andyiy\_\{i\}is the true label, either 0 or 1, for instanceii\.

### 3\.5Span Detection Model

We model span detection as a token classification problem by fine\-tuning a pretrained BERT model\. Specifically, we formulate two token\-level tasks: \(i\) binary span detection and \(ii\) multi\-class span categorization\.

For binary detection, we adopt the BIO tagging scheme, where each token is labeled as beginning \(B\), inside \(I\), or outside \(O\) of a hateful span\. For categorization, we use an IO tagging scheme, where each token is either labeled as outside \(O\) any hateful span or assigned to one of five hate speech categories\.

We fine\-tune the BERT encoder with a token\-level classification layer and optimize the model using token\-level Cross\-Entropy \(CE\) loss\. Given the input token sequencexx, the model predicts token label probabilitiesp​\(yi\|x\)p\(y\_\{i\}\|x\)for each tokenii\. The loss function is defined as:

LT​o​k​e​n​L​e​v​e​l​C​E=−1N​∑i=1N∑c=1Cyi,c⋅log⁡\(pi,c\)L\_\{TokenLevelCE\}=\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\sum\_\{c=1\}^\{C\}y\_\{i,c\}\\cdot\\log\(p\_\{i,c\}\)\(6\)
whereNNis the number of tokens,CCis the number of classes,yi,cy\_\{i,c\}is the one\-hot encoded ground truth for tokeniiand classcc, andpi,cp\_\{i,c\}is the predicted probability of tokeniibelonging to classcc\.

### 3\.6Hashtag Segmentation and Synthetic Data Generatıon Using ChatGPT

Hashtags can be a rich source of information for sentiment analysis and hate speech detection, as they often encapsulate users’ opinions and sentiments succinctly\. People frequently use hashtags to convey their views on various topics, making them crucial for understanding the overall tone and context of social media posts\.

Since, BERT Tokenizer is not able to segment hashtags correctly, we tried other BERT based tool, namely Hashformers777https://github\.com/ruanchaves/hashformers\[[31](https://arxiv.org/html/2607.00143#bib.bib31)\], which is developed specifically for hashtag segmentation\. However, it also failed to correctly segment hashtags in Turkish, while it is very slow and requires a lot of memory to run \(about 22 GB of GPU RAM\)\.

Recently, ChatGPT has excelled in various NLP tasks, including text classification, sentiment analysis, and text generation\. Leveraging this capability, we employed ChatGPT in a zero\-shot setting for hashtag segmentation\. Since some hashtags in Turkish are written in Arabic or English \(e\.g\. \#genocideingazza, \#babykiller\), we included a step in our prompt for ChatGPT to first translate these hashtags into Turkish before segmenting them\. From our dataset covering five topics, we extracted about 3,447 unique hashtags\. Tables[7](https://arxiv.org/html/2607.00143#S3.T7)show an example of hashtag segmentation by these three mentioned methods\.

Furthermore, recent studies have shown that Large Language Models \(LLMs\) can generate synthetic data that helps address low\-resource challenges in NLP\[[32](https://arxiv.org/html/2607.00143#bib.bib32),[33](https://arxiv.org/html/2607.00143#bib.bib33)\]\. Therefore, we used ChatGPT \(gpt\-3\.5\-turbo\-1106\) with few\-shot prompting to generate additional samples for underrepresented hateful classes, including swearing, insult, threat, injury, enmity, and harm\.

For each topic in our dataset \(Refugees, Israel\-Palestine conflict, Turkey\-Greece relations, religion/ethnicity, and LGBTI\+\), we provided representative hateful examples and generated additional instances\. We also created samples targetingwomenand developed a hate lexicon based on frequently observed hate expressions\. Using these resources, ChatGPT generated 761 synthetic samples, including 547 hateful instances\.

Table 7:Hashtag segmentation example using BERTurk Tokenizer, Hashformer tool\[[31](https://arxiv.org/html/2607.00143#bib.bib31)\]and ChatGPT\.

## 4Experimental Results

In this section, we present comprehensive experiments and their results to evaluate the effectiveness of the proposed approaches in identifying and detecting hate speech across various tasks, including binary classification, multi\-class classification, target identification, and span detection, and we discuss each result in detail\.

### 4\.1Training Setting

We use a pretrained BERT transformer with a linear classification head on top of the pooled output, using the Huggingface Transformer package\. For Turkish and Arabic, we start fine\-tuning from pretrained checkpoint of BERTurk888https://huggingface\.co/dbmdz/bert\-base\-turkish\-uncasedand AraBERT999https://huggingface\.co/aubmindlab/bert\-base\-arabert, respectively\. We allocated approximately 80% of the data for training and 20% for testing\.

### 4\.2Experiment 1: Binary and Multi\-class Classification in Turkish and Arabic:

In our experiments, we evaluated two models, BERTurk\-CE and BERTurk\-DualCL, across three classification settings: binary \(2\-class\), coarse\-grained \(4\-class\), and fine\-grained \(6\-class\) hate speech categorization\. The 2\-class setting focuses on detecting the presence or absence of hate speech, while the 6\-class model distinguishes between “no hate speech” and five specific hate speech categories\. To create a 4\-class setting, we followed the category mapping strategy proposed in\[[24](https://arxiv.org/html/2607.00143#bib.bib24)\], where categories 2 and 3 were merged, as well as categories 4 and 5, to produce a more generalizable classification task\. For all experiments, We only included samples where annotators reached agreement or where the majority vote was clear, thereby ensuring label quality\. As a result of this filtering and the mapping between classification settings, the number of training and test samples varied slightly across the three experiments\.

Results and Discussion:

Table[8](https://arxiv.org/html/2607.00143#S4.T8)presents the performance of the baseline Cross\-Entropy \(CE\) and the proposed DualCL models across binary \(2\-class\), coarse\-grained \(4\-class\), and fine\-grained \(6\-class\) hate speech classification tasks\. Across all settings and both languages, DualCL consistently outperforms the CE baseline, demonstrating the benefits of combining contrastive learning with cross\-entropy loss\.

For the Turkish dataset, DualCL achieves Macro\-F1 scores of 84\.85%, 75\.35%, and 66\.66% in the 2\-, 4\-, and 6\-class settings, respectively, with improvements of up to 1\.88 points over CE\. The gains are particularly evident in the more challenging fine\-grained classification task\. For the Arabic dataset, the improvements are more substantial, especially in the 4\-class and 6\-class settings, where Macro\-F1 increases from 42\.31% to 47\.41% and from 22\.86% to 31\.38%, respectively\.

As expected, both models achieve their best performance in the binary setting, while performance decreases as the number of classes increases\. Nevertheless, DualCL maintains a consistent advantage across all scenarios\. Despite the smaller size and narrower topical focus of the Arabic dataset, the larger relative improvements suggest that DualCL is particularly effective in low\-resource and domain\-specific hate speech classification tasks\.

Table 8:Results of baseline \(CE\) and our DualCL models on Turkish and Arabic datasetsTable 9:Results of hate intensity \(degree of hate\) prediction on Turkish and Arabic datasetTable 10:Results of multi\-label target classification on Turkish and Arabic datasetTable 11:Results of span detection on Turkish dataset
### 4\.3Experiment 2: Hate Intensity Prediction in Turkish and Arabic

In this second experiment, we measured the Hate Intensity \(strength or degree of hate\) on a scale from 0 to 10\. For evaluating our Hate Intensity Prediction model \(regression model\), we use RMSE \(Root Mean Squared Error\) and Pearson’s correlation coefficient metrics\. The Pearson correlation coefficient, often symbolized asr, is a widely used metric to assess linear correlations between two variables\. It produces a value ranging from –1 to 1, indicating both the magnitude and direction of the correlation\. 1 indicates a perfect positive linear relationship, \-1 indicates a perfect negative linear relationship, and 0 indicates that there is no linear relationship between the variables\.

Results and Discussion:

Table[9](https://arxiv.org/html/2607.00143#S4.T9)presents the performance of the regression models for hate intensity prediction on Turkish and Arabic datasets\. The BERTurk model achieves an RMSE of 1\.64 and a Pearson correlation of 73\.41, while AraBERT obtains a lower RMSE of 1\.20 with a Pearson correlation of 67\.38\. The lower RMSE of AraBERT indicates fewer large prediction errors, whereas the higher Pearson correlation of BERTurk shows a stronger alignment between predicted and actual intensity values\.

The difference in performance may be influenced by the characteristics of thetest dataset\. The Arabic test set contains fewer samples \(533\) compared to the Turkish test set \(2,524\), making its evaluation more sensitive to individual predictions\. Overall, BERTurk provides more stable correlation results, while AraBERT demonstrates lower prediction error in a smaller, domain\-specific setting\.

### 4\.4Experiment 3: Multi\-label Target Classification in Turkish and Arabic

In the third experiment, we extended our analysis by incorporating target group identification along with hate speech classification and hatefulness intensity measurement\. In addition to classifying and assessing the strength of hate on a scale from 0 to 10, we identified both the general and specific target groups within hateful tweets\. Multi\-label general target classification involved determining whether a tweet targeted a group and categorizing it into four broad categories, as shown in Table[5](https://arxiv.org/html/2607.00143#S2.T5), for the Turkish and Arabic datasets\.

Results and Discussion:

Table[10](https://arxiv.org/html/2607.00143#S4.T10)presents the results of multi\-label hate target classification for the Turkish and Arabic datasets\. Overall, the model achieves stronger performance on the Turkish dataset, which contains a larger and more diverse set of samples\. For Turkish, the highest F1\-score is obtained for the“Country, Nationality, Race, Ethnicity”category \(0\.82\), followed by“Target not specified or not present”\(0\.70\), with an overall micro\-average F1\-score of 0\.73\. Lower performance is observed for underrepresented categories such as“Religion”\(0\.46\) and“Gender, Sexual Orientation”\(0\.43\)\.

For Arabic, the model performs best on“Target not specified or not present”\(0\.89\) and“Country, Nationality, Race, Ethnicity”\(0\.65\), reflecting the dataset’s focus on refugee\-related hate speech\. Performance is lower for less represented categories, including“Religion”\(0\.26\) and“Gender, Sexual Orientation”\(0\.58\)\. These results suggest that class distribution and topic coverage have a significant impact on target classification performance\.

### 4\.5Experiment 5: Span Detection in Turkish

In this experiment, we evaluate token\-level detection and categorization of hateful spans in Turkish\. We formulate two tasks: \(i\) binary span detection using the BIO tagging scheme, and \(ii\) multi\-class span categorization using an IO scheme based on five hate speech categories\.

We fine\-tune BERTurk101010[https://huggingface\.co/dbmdz/bert\-base\-turkish\-cased](https://huggingface.co/dbmdz/bert-base-turkish-cased)using the Huggingface Transformers library\. The model consists of a linear classification head applied to token\-level representations\. Training is conducted using 5\-fold cross\-validation, while a separate set of 300 tweets is held out for final evaluation\. Evaluation is performed using the SeqEval111111[https://github\.com/chakki\-works/seqeval](https://github.com/chakki-works/seqeval)library, with token\-level precision, recall, and F1 scores computed by flattening predictions and gold labels\. Results are reported as the mean and standard deviation across folds\.

To ensure robustness and avoid overfitting to a small evaluation set, we adopt a hybrid strategy that combines cross\-validation with a held\-out test set\. The test set comprises 300 tweets that reached inter\-annotator agreement in the first annotation round, ensuring high label reliability\. Separate models are trained for detection and categorization, as the tasks differ in label granularity and class distribution, requiring different learning dynamics\.

Results and Discussion:

Results are provided in Table[11](https://arxiv.org/html/2607.00143#S4.T11)\. The span detection model demonstrates moderately strong performance in identifying hateful spans, with an F1 score of 59%\. In contrast, categorizing spans into specific hate speech types proves significantly more difficult, with an F1 score of 31%\.

Additionally, the model performs best on explicit categories such as“Threat of Enmity, War, Attack, Murder, or Harm”\(F1 = 0\.44\), while struggling with more implicit and context\-dependent categories like“Exclusionary/Discriminatory Discourse”\(F1 = 0\.08\) and“Exaggeration, Generalization, Attribution, Distortion”\(F1 = 0\.17\)\. These findings suggest that while explicit threats are more easily identifiable, subtle forms of hate speech require deeper contextual understanding and remain a challenge for the current approach\.

## 5Summary and Conclusions

In this work, we introduced benchmark datasets for detecting and measuring hate speech in Turkish and Arabic and conducted extensive experiments across multiple tasks\. The datasets include 10,953 Turkish tweets covering topics such as Refugees, the Israel–Palestine conflict, Anti\-Greek sentiment in Turkey, Religion, Race/Ethnicity, and LGBTI\+ issues, as well as 2,510 Arabic tweets mainly focused on Refugees\. The data was annotated to support hate speech classification, intensity prediction, target identification, group detection, and span detection\.

To ensure annotation reliability, each tweet was evaluated by three annotators, resulting in an average Krippendorff’s Alpha score of 0\.234, indicating the challenges and subjectivity of hate speech annotation\. To address this issue, we relied on agreement\-based and majority\-voted samples to ensure reliable training data\.

Our findings demonstrate that the DualCL model consistently outperformed the cross\-entropy baseline, with larger improvements in complex multi\-class settings and the low\-resource Arabic dataset\. The models captured hate intensity variations effectively, while target and span detection models performed well but remained challenged by underrepresented categories and implicit hate expressions\. Overall, this study provides comprehensive benchmarks, datasets, and models for multi\-dimensional hate speech analysis in Turkish and Arabic, supporting future research and applications\.

## Acknowledgments

This article was produced within the scope of the project “Utilizing Digital Technology for Social Cohesion, Positive Messaging and Peace by Boosting Collaboration, Exchange and Solidarity” \(EuropeAid/170389/DD/ACT/ Multi\), implemented bythe Hrant Dink Foundation \(HDF\)in partnership withSabancı UniversityandBoğaziçi University, and supported bythe European Unionandthe Friedrich Naumann Foundation\. The implementing parties are solely responsible for the content of this publication, and the views expressed herein do not necessarily reflect those of the supporters\. Early versions of parts of this work were published in conference proceedings\[[10](https://arxiv.org/html/2607.00143#bib.bib10),[27](https://arxiv.org/html/2607.00143#bib.bib27)\]\. The present article provides a more comprehensive and integrated treatment of these earlier results\. We would like to thankNural ÖzelandHasan Kerem Şekerfor their valuable assistance in data collection, transfer, and management throughout the annotation process\. We also thank the HDF project team and the annotators for their valuable contributions\.

## Author Contributions

All authors contributed to the conception and design of the study\.Somaiyeh DehghanandGökçe Uludoğanjointly developed the methodology\.Somaiyeh Dehghanimplemented the binary and multi\-class classification models, hatefulness intensity prediction models, multi\-label target classification models, hashtag segmentation and synthetic data generation procedures, and conducted the related experiments\.Gökçe Uludoğanimplemented the span annotation process and span detection models and conducted the related experiments\.Mehmet Umut Şenprepared the dataset and derived gold labels\.Somaiyeh Dehghanwrote the main parts of the manuscript, whileGökçe Uludoğanauthored the sections related to span detection\.Elif Erolcoordinated data annotation guidelines;Berrin YanıkoğluandArzucan Özgürwere the principal investigators of the project and helped revise the manuscript\. All authors read and approved the final manuscript\.

## Declarations

Competing Interests: The authors have no relevant financial or non\-financial interests to disclose\.

## References

- \[1\]Gitari N M, Zuping Z, Damien H, and Jun Long J\.A lexicon\-based approach for hate speech detection\.International Journal of Multimedia and Ubiquitous Engineering, 10:215–230, 2015\.
- \[2\]Vargas F, Góes F R, Carvalho I, Benevenuto F, and Pardo T\.Contextual\-lexicon approach for abusive language detection\.InProceedings of the International Conference on Recent Advances in Natural Language Processing \(RANLP 2021\), pages 1438–1447, Held Online, September 2021\. INCOMA Ltd\.
- \[3\]Çöltekin Ç\.A corpus of Turkish offensive language on social media\.InProceedings of the Twelfth Language Resources and Evaluation Conference, pages 6174–6184, Marseille, France, May 2020\. European Language Resources Association\.
- \[4\]Şahi H, KılıçY, and Saǧlam R B\.Automated detection of hate speech towards woman on twitter\.In2018 3rd International Conference on Computer Science and Engineering \(UBMK\), pages 533–536, 2018\.
- \[5\]Abro S, Shaikh S, Khand Z H, Ali Z, Khan S, and Mujtaba G\.Automatic hate speech detection using machine learning: A comparative study\.International Journal of Advanced Computer Science and Applications, 11, 2020\.
- \[6\]Hüsünbeyi Z M, Akar D, and Özgür A\.Identifying hate speech using neural networks and discourse analysis techniques\.InProceedings of the First Workshop on Language Technology and Resources for a Fair, Inclusive, and Safe Society within the 13th Language Resources and Evaluation Conference, pages 32–41, Marseille, France, June 2022\. European Language Resources Association\.
- \[7\]Beyhan F, Çarık B, Arın I, Terzioğlu A, Yanikoglu B, and Yeniterzi R\.A Turkish hate speech dataset and detection system\.InProceedings of the 13th Conference on Language Resources and Evaluation \(LREC 2022\), page 4177–4185, 2022\.[https://aclanthology\.org/2022\.lrec\-1\.443/](https://aclanthology.org/2022.lrec-1.443/)\.
- \[8\]Toraman C, Şahinuç F, and Yilmaz E\.Large\-scale hate speech detection with cross\-domain transfer\.InProceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2215–2225, Marseille, France, June 2022\. European Language Resources Association\.
- \[9\]Kurt M S and Demirel E Y\.Türkçe hakaret ve nefret söylemi otomatik tespit modeli\.Veri Bilimi, 6\(1\):61–73, 2023\.
- \[10\]Dehghan S and Yanikoglu B\.Multi\-domain hate speech detection using dual contrastive learning and paralinguistic features\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\), pages 11745–11755, Torino, Italia, May 2024\. ELRA and ICCL\.
- \[11\]Alsafari S, Sadaoui S, and Mouhoub M\.Hate and offensive speech detection on arabic social media\.Online Social Networks and Media, 19:100096, 2020\.
- \[12\]Salomon P O, Kechaou Z, and Wali A\.Arabic hate speech detection system based on arabert\.In2022 IEEE 21st International Conference on Cognitive Informatics and Cognitive Computing \(ICCI\*CC\), pages 208–213, Toronto, Canada, 2022\.
- \[13\]Khezzar R, Moursi A, and Al Aghbari Z\.arhatedetector: detection of hate speech from standard and dialectal arabic tweets\.Discover Internet of Things, 3, 2023\.
- \[14\]Bäumler J, Blöcher L, Frey L\-J, Chen X, Bayer M, and Reuter C\.A survey of machine learning models and datasets for the multi\-label classification of textual hate speech in english\.ACM Trans\. Knowl\. Discov\. Data, June 2026\.Just Accepted\.
- \[15\]Santos G L, Santos V G d, and Kearns C\.All together against hate: ensemble based llms for multi class hate speech classification in the football context, 2026\.
- \[16\]Ayele A A, Jalew E A, Ali A C, Yimam S M, and Biemann C\.Exploring boundaries and intensities in offensive and hate speech: Unveiling the complex spectrum of social media discourse, 2024\.Preprint at[https://doi\.org/10\.48550/arXiv\.2404\.12042](https://doi.org/10.48550/arXiv.2404.12042)\.
- \[17\]Geleta R R, Eckelt K, Parada\-Cabaleiro E, and Schedl M\.Exploring intensities of hate speech on social media: A case study on explaining multilingual models with XAI\.InProceedings of the 4th Conference on Language, Data and Knowledge, pages 532–537, Vienna, Austria, September 2023\. NOVA CLUNL, Portugal\.
- \[18\]Riyadi S, Masyhur A M, and Andriyani A D\.Deep learning for hate speech intensity analysis: Distilbert classification algorithm\.In2024 IEEE Symposium on Industrial Electronics and Applications \(ISIEA\), pages 1–6, 2024\.
- \[19\]Büyükdemirci K, Kucukkaya I E, Ölmez E, and Toraman C\.JL\-hate: An annotated dataset for joint learning of hate speech and target detection\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\), pages 9543–9553, Torino, Italia, may 2024\. ELRA and ICCL\.
- \[20\]G Uludoğan, Yüksel A E, Tunçer U, Işık B, Korkmaz Y, Akar D, and Özgür A\.Detecting hate speech in Turkish print media: A corpus and a hybrid approach with target\-oriented linguistic knowledge\.InProceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio\-political Events from Text \(CASE 2024\), pages 205–214, St\. Julians, Malta, March 2024\. Association for Computational Linguistics\.
- \[21\]Hoang P G, Luu C D, Tran K Q, Nguyen K V, and Nguyen N L\-T\.ViHOS: Hate speech spans detection for Vietnamese\.InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 652–669, Dubrovnik, Croatia, May 2023\. Association for Computational Linguistics\.
- \[22\]Calabrese A, Neves L, Shah N, Bos M, Ross B, Lapata M, and Barbieri F\.Explainability and hate speech: Structured explanations make social media moderators faster\.In Lun\-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\), pages 398–408, Bangkok, Thailand, August 2024\. Association for Computational Linguistics\.
- \[23\]Arın I, Işık Z, Kutal S, Dehghan S, Özgür A, and Yanikoglu B\.Siu2023\-nst \- hate speech detection contest\.In2023 31st Signal Processing and Communications Applications Conference \(SIU\), pages 1–4, 2023\.
- \[24\]Dehghan S, Sen M U, and Yanikoglu B\.Dealing with annotator disagreement in hate speech classification, 2025\.Preprint at[https://doi\.org/10\.48550/arXiv\.2502\.08266](https://doi.org/10.48550/arXiv.2502.08266)\.
- \[25\]Uludoğan G, Dehghan S, Arin I, Erol E, Yanikoglu B, and Özgür A\.Overview of the hate speech detection in Turkish and Arabic tweets \(HSD\-2Lang\) shared task at CASE 2024\.InProceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio\-political Events from Text \(CASE 2024\), pages 229–233, St\. Julians, Malta, March 2024\. Association for Computational Linguistics\.
- \[26\]Dehghan S and Yanikoglu B\.Evaluating ChatGPT’s ability to detect hate speech in Turkish tweets\.InProceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio\-political Events from Text \(CASE 2024\), pages 54–59, St\. Julians, Malta, March 2024\. Association for Computational Linguistics\.
- \[27\]Şeker H K, Uludoğan G, Önal P, and Özgür A\.HATECAT\-TR: A hate speech span detection and categorization dataset for Turkish\.In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 2025, pages 25568–25579, Suzhou, China, November 2025\. Association for Computational Linguistics\.
- \[28\]Devlin J, Chang M W, Lee K, and Toutanova K\.BERT: Pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\), pages 4171–4186, Minneapolis, Minnesota, June 2019\. Association for Computational Linguistics\.
- \[29\]Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Isola P, Maschinot A, Liu C, and Krishnan D\.Supervised contrastive learning\.InProceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA, 2020\. Curran Associates Inc\.
- \[30\]Chen T, Kornblith S, Norouzi M, and Hinton G\.A simple framework for contrastive learning of visual representations\.InProceedings of the 37 th International Conference on Machine Learning, Vienna, Austria, 2020\.
- \[31\]Rodrigues R C, Inuzuka M A, Gomes J R S, Rocha A S, Calixto I, and Nascimento H A D\.Zero\-shot hashtag segmentation for multilingual sentiment analysis, 2021\.Preprint at[https://doi\.org/10\.48550/arXiv\.2112\.03213](https://doi.org/10.48550/arXiv.2112.03213)\.
- \[32\]Li Z, Zhu H, Lu Z, and Yin M\.Synthetic data generation with large language models for text classification: Potential and limitations\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10443–10461, Singapore, December 2023\. Association for Computational Linguistics\.
- \[33\]Guo X and Chen Y\.Generative ai for synthetic data generation: Methods, challenges and the future, 2024\.Preprint at[https://doi\.org/10\.48550/arXiv\.2403\.04190](https://doi.org/10.48550/arXiv.2403.04190)\.