ToxiREX: A Dataset on Toxic REasoning in ConteXt

arXiv cs.CL Papers
toxicity dataset multilingual toxic-reasoning implicit-toxicity reddit context-aware

Summary

ToxiREX is a new multilingual dataset of Reddit comments annotated for implicit toxicity using a toxic reasoning schema, covering six languages and multiple events.

arXiv:2606.27981v1 Announce Type: new Abstract: We introduce a new, contextual, multilingual dataset called ToxiREX: Toxic REasoning in ConteXt. The dataset consists of threads of Reddit comments and structured characterizations of what the comments imply, following a systematic toxic reasoning schema developed in a previous paper. Using the schema allows us to capture and explain implicit and context-dependent toxicity, while supporting mappings to existing toxicity taxonomies. The dataset includes comments in six languages (English, Arabic, Turkish, Spanish, German, and Dutch), collected from posts connected to specific major events (e.g. the 2023 Turkey earthquakes; the Russian invasion of Ukraine). We describe the context-preserving preprocessing of the threads. We create a training set of 125 thousand comments which is annotated by a commercially available LLM, and a test set of just under three thousand comments that is annotated by native speakers. We show that apparent disagreements in the test set annotations often reflect defensible alternative interpretations rather than noise. Finally, we provide baseline results by prompting and fine-tuning language models. To produce these results, we develop evaluation strategies for our hierarchical, schema-based predictions. While models perform better than random, there remains a lot of room for improvement, showing the task to be challenging. ToxiREX is the first dataset to simultaneously incorporate multiple languages, conversational context, and implicit toxicity, while using the toxic reasoning schema for rich, structured annotations. Dataset available at: https://github.com/cltl/toxirex
Original Article
View Cached Full Text

Cached at: 06/29/26, 05:25 AM 

# ToxiREX: A Dataset on Toxic REasoning in ConteXt
Source: [https://arxiv.org/abs/2606.27981](https://arxiv.org/abs/2606.27981)
[View PDF](https://arxiv.org/pdf/2606.27981)

> Abstract:We introduce a new, contextual, multilingual dataset called ToxiREX: Toxic REasoning in ConteXt\. The dataset consists of threads of Reddit comments and structured characterizations of what the comments imply, following a systematic toxic reasoning schema developed in a previous paper\. Using the schema allows us to capture and explain implicit and context\-dependent toxicity, while supporting mappings to existing toxicity taxonomies\. The dataset includes comments in six languages \(English, Arabic, Turkish, Spanish, German, and Dutch\), collected from posts connected to specific major events \(e\.g\. the 2023 Turkey earthquakes; the Russian invasion of Ukraine\)\. We describe the context\-preserving preprocessing of the threads\. We create a training set of 125 thousand comments which is annotated by a commercially available LLM, and a test set of just under three thousand comments that is annotated by native speakers\. We show that apparent disagreements in the test set annotations often reflect defensible alternative interpretations rather than noise\. Finally, we provide baseline results by prompting and fine\-tuning language models\. To produce these results, we develop evaluation strategies for our hierarchical, schema\-based predictions\. While models perform better than random, there remains a lot of room for improvement, showing the task to be challenging\. ToxiREX is the first dataset to simultaneously incorporate multiple languages, conversational context, and implicit toxicity, while using the toxic reasoning schema for rich, structured annotations\. Dataset available at:[this https URL](https://github.com/cltl/toxirex)

## Submission history

From: Stefan Schouten MSc \[[view email](https://arxiv.org/show-email/72f07455/2606.27981)\] **\[v1\]**Fri, 26 Jun 2026 11:30:42 UTC \(952 KB\)

Similar Articles