MUCnoHARM@GermEval Shared Task 2026: Retrieval-based In-Context Learning for Defamatory Offences, and Where It Falls Short

# MUCnoHARM@GermEval Shared Task 2026: Retrieval-based In-Context Learning for Defamatory Offences, and Where It Falls Short
Source: [https://arxiv.org/html/2609.09791](https://arxiv.org/html/2609.09791)
Kristin GnadtAffiliation:Central Office for Information Technology in the Security Sector \(ZITiS\), Munich, GermanyAffiliation:Department of Statistics, LMU Munich, GermanyMaximilian MeidingerAffiliation:Central Office for Information Technology in the Security Sector \(ZITiS\), Munich, GermanyMatthias AßenmacherAffiliation:Department of Statistics, LMU Munich, GermanyAffiliation:Munich Center for Machine Learning \(MCML\), GermanyCorrespondence:[kristin\.gnadt@zitis\.bund\.de](mailto:[email protected])

###### Abstract

With hate speech being ubiquitous online, automatic detection is crucial, in particular when it comes to criminally relevant social media posts\. We study a variety of retrieval\-based in\-context learning \(RetICL\) strategies for detecting defamatory offences under§§ 185–187 StGB\(the subject of GermEval 2026 Subtask 4\)\. Few\-shot prompting beats zero\-shot, but retrieval\-based approaches offer only marginal gains over random demonstrations, and even fall behind an optimised static set of demonstrations\. Providing concrete legal knowledge helps, yet model choice outweighs every other system choice\. Models over\-predict criminal relevance while still missing 26–57% of criminally relevant posts, suiting them for triage rather than autonomous moderation\.

## 1Introduction

Hate speech detection has become a socially consequential application domain for LLM\-based text classification\. Hate speech is highly prevalent online: In Germany, 34% of internet users reported encountering instances thereof in the first quarter of 2025\([Destatis, 2025](https://arxiv.org/html/2609.09791#bib.bib15)\)\. The volume of potentially harmful content has far outstripped the capacity of human moderation, making automated detection a critical concern for platforms and policymakers alike\([European Union Agency for Fundamental Rights, 2023](https://arxiv.org/html/2609.09791#bib.bib37)\)\. Large language models \(LLMs\) have demonstrated strong capabilities in addressing the core semantic challenges of automated hate speech detection\([Albladi et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib4);[Kums et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib5)\), making them a natural fit for the task\. To date, most research has focused on English\-language datasets and settings\([Albladi et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib4);[Usman et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib9);[Gandhi et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib6)\), although a growing body of work addresses hate speech detection in other languages, including German\([Goldzycher et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib12);[Glasebach et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib13);[Demus et al\., 2022](https://arxiv.org/html/2609.09791#bib.bib14)\)\.

Detecting*criminally relevant*hate speech, however, introduces a further layer of complexity: models must be sensitive to the specific thresholds and distinctions encoded in \(country\-specific\) law, a requirement that current systems only partially meet\([Schäfer, 2023](https://arxiv.org/html/2609.09791#bib.bib7);[Kums et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib5);[Ludwig et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib11)\)\. Subtask 4 \(DEF\) of the GermEval Shared Task 2026 on*Harmful Content Detection in Social Media*instantiates precisely this challenge: classifying posts according to whether they constitute defamatory offences under§§ 185–187 StGB\(German Criminal Code\)\([Felser et al\., 2026](https://arxiv.org/html/2609.09791#bib.bib42)\)\.

Figure 1:RetICL framework\. Embedded training samples make up a knowledge base, from which few\-shot demonstrations are retrieved, which are given to a model along with a task description \(*conditioning*\) and the post\.#### Contributions\.

In this work, we test different prompting strategies for the detection of defamatory offences\. Our RetICL framework consists of two main components \(Fig\.[1](https://arxiv.org/html/2609.09791#S1.F1)\):

1. 1\.The legal information on§§ 185–187 StGBthat is provided to the model \(Conditioning\); and
2. 2\.The retrieval mechanisms for few\-shot demonstrations \(EmbeddingandRetrieval\)\.

The first component builds on[Ludwig et al\. \(2025\)](https://arxiv.org/html/2609.09791#bib.bib11), who use prompts of differing abstraction levels—“conditioning” models on different degrees of legal knowledge—to detect whether a post falls under § 130 StGB \(incitement to hatred\) \(Section[2\.2](https://arxiv.org/html/2609.09791#S2.SS2)\)\. The second component tests different strategies for selecting demonstrations for few\-shot prompting, which are also compared to zero\-shot prompting\. Section[2](https://arxiv.org/html/2609.09791#S2)situates our two components in prior work on legal conditioning and RetICL\. Section[3](https://arxiv.org/html/2609.09791#S3)details the conditioning modes and retrieval strategies we compare, and Section[4](https://arxiv.org/html/2609.09791#S4)the experimental setups\. Section[5](https://arxiv.org/html/2609.09791#S5)reports that concrete legal knowledge and few\-shot prompting help, but retrieval strategy barely matters, with the full discussion of implications in Section[6](https://arxiv.org/html/2609.09791#S6)\.

## 2Background & Related Work

### 2\.1Defamatory Offences

The StGB sections central to this task are the three defamatory offences in§§ 185–187, which all protect a person’s right to honour and differ mainly in whether the disparaging statement is an opinion or a factual claim, and to whom it is addressed\([Research Services of the German Parliament, 2022](https://arxiv.org/html/2609.09791#bib.bib20)\):§ 185*Insult*\(*Beleidigung*\),§ 186*Defamation*\(*Üble Nachrede*\), and§ 187*Intentional Defamation*\(*Verleumdung*\)\.

The dataset \(GermEval 2026, Subtask 4\) was annotated with the decision scheme of[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)\. Derived directly from the statutory norms in the German deductive civil\-law tradition, the scheme operationalises the legal assessment of§§ 185–187StGB as a sequence of six binary \(yes / no\) decisions based solely on the text of a post\. These decisions cover whether the post targets a valid holder of the right to honour, whether it is disparaging, whether it is a factual claim or a value judgment, and how freedom of expression is weighed against the right to honour \(§ 193 StGB\)\. The full scheme is provided in Appendix[A](https://arxiv.org/html/2609.09791#A1)\.

Because it decomposes a complex legal judgment into independently checkable sub\-decisions, the scheme yields reliable annotations—[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)report that instructed laypeople apply it with reasonable reliability against an expert reference\. Additionally, the scheme provides a natural source of graded legal knowledge for prompting, which our*conditioning*settings utilise \(Section[3](https://arxiv.org/html/2609.09791#S3)\)\.

### 2\.2Legal Conditioning

How a task is presented to an LLM strongly affects its performance\. Even meaning\-preserving changes to a prompt can shift performance substantially\([Sclar et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib33);[Gan and Mori, 2023](https://arxiv.org/html/2609.09791#bib.bib35);[Lu et al\., 2022](https://arxiv.org/html/2609.09791#bib.bib34)\)\. Beyond such surface\-level choices, the*content*of the prompt—which task\-relevant knowledge it makes available to the model—is itself a design decision\. This is especially pertinent in specialised domains such as law, where general prompting techniques have proven helpful but require careful adaptation\([Trautmann et al\., 2022](https://arxiv.org/html/2609.09791#bib.bib31);[Parizi et al\., 2023](https://arxiv.org/html/2609.09791#bib.bib1)\), raising the question of*how much*and*what kind*of legal knowledge to place in the prompt\.

[Ludwig et al\. \(2025\)](https://arxiv.org/html/2609.09791#bib.bib11)address this question for hate speech falling under § 130 StGB \(incitement to hatred\) by*conditioning*LLMs on legal knowledge at different abstraction levels\. Their prompts range from the title of a norm—relying on whatever knowledge the model has internalised— to the verbatim or simplified statutory text, and finally to an explicit decomposition of the offence into its constituent subtasks\. The decomposition follows the subtask\-based framing that[AlKhamissi et al\. \(2022\)](https://arxiv.org/html/2609.09791#bib.bib32)found effective for few\-shot hate speech detection\. Following these approaches, we vary the legal knowledge supplied to the model, from merely stating the relevant section numbers and titles to decomposing the task into the subtasks of the annotation scheme of[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)\.

Counterintuitively,[Ludwig et al\. \(2025\)](https://arxiv.org/html/2609.09791#bib.bib11)found that models conditioned on more abstract knowledge outperformed those given more concrete legal information\. A substantial performance gap remained between LLM prompting approaches and legal experts, and the models could barely match laypeople applying the annotation scheme\([Ludwig et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib11);[Zufall et al\., 2022](https://arxiv.org/html/2609.09791#bib.bib17)\)\. Conversely,[von Cossel \(2026\)](https://arxiv.org/html/2609.09791#bib.bib41)found that subtask decomposition for detecting § 130 StGB offences, combined with a logical scaffold in a neuro\-symbolic approach, outperformed less complex prompting approaches\.

### 2\.3Retrieval\-based In\-Context Learning

LLMs can perform classification through in\-context learning \(ICL\), conditioning on a few input\-output demonstrations in the prompt without updating model parameters\. ICL, however, is highly sensitive to the choice, number, ordering, and format of those demonstrations\([Luo et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib21)\)\. A fixed, manually curated, or randomly sampled set applied uniformly to every input is therefore suboptimal, which motivates RetICL111In the literature, RetICL is also referred to as dynamic few\-shot prompting, RAG\-enhanced LLMs, \(selective\) demonstration retrieval, and active learning for ICL\., in which demonstrations are retrieved per query from a labelled pool\([Luo et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib21)\)\.

Retrieval is usually guided by a similarity objective, most commonly selecting the top\-kkexamples by using sparse term\-matching retrievers such as BM25\([Robertson and Jones, 1976](https://arxiv.org/html/2609.09791#bib.bib23);[Robertson and Zaragoza, 2009](https://arxiv.org/html/2609.09791#bib.bib19)\)or dense sentence\-embedding retrievers scored by cosine similarity\([Luo et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib21);[Fan et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib22)\)\.[Margatina et al\. \(2023\)](https://arxiv.org/html/2609.09791#bib.bib18)show that similarity\-based retrieval consistently outperforms uncertainty\-, diversity\-, and random\-based selection, while zero\-shot prompting performs worst for text classification tasks\.[Miller et al\. \(2025\)](https://arxiv.org/html/2609.09791#bib.bib2)find RetICL best across seven open\-weights LLMs on clinical note section classification, raisingF​1macroF1\_\{\\text\{macro\}\}substantially over both zero\-shot and static few\-shot; they note that static few\-shot is sometimes no better than zero\-shot, that smaller models benefit more, and that more demonstrations do not monotonically help\.

These methods are increasingly applied to abusive language and content moderation tasks adjacent to hate speech detection\. For implicit hate speech,[Kim and Lee \(2025\)](https://arxiv.org/html/2609.09791#bib.bib3)prioritise demonstrations sharing the target group before falling back to BM25 similarity, reducing the over\-sensitivity of LLMs to toxic surface terms\. Related work uses retrieval to discover emergent dog whistles\([Sasse et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib8), coded language used to evade automatic detection mechanisms;\)\. The evidence is not uniformly positive, however:[Liu and Shi \(2024\)](https://arxiv.org/html/2609.09791#bib.bib10)found dynamic exemplar selection less reliable than their prompt\-optimisation framework\.

RetICL is a strong but task\-dependent strategy, yet the relative behaviour of zero\-shot, static, and RetICL remains unexamined for*criminally relevant*hate speech, where classification depends on a precise legal threshold rather than a general notion of toxicity\. We address this gap by systematically testing retrieval mechanisms and prompting strategies for detecting defamatory offences as part of the shared task\.

## 3Materials and Methods

#### Dataset\.

The DEF task is based on a corpus of German\-language tweets, each labelled according to the annotation scheme of[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)described in Section[2\.1](https://arxiv.org/html/2609.09791#S2.SS1), with a binary label indicating whether the post falls under§§ 185–187 StGB\. The labelled portion comprises3,2633\{,\}263tweets, on which we report cross\-validated results; a further577577tweets form the unlabelled competition test set, which is used only for the shared task submission\. In the training dataset,∼13%\{\\sim\}13\\%of the samples are labelled as positive \(criminally relevant\)\. In addition, one author annotated a subset of797797training tweets at the level of each individual decision step of the scheme according to the explanations of[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)\. Whereas the organisers’ data provide only the final label, these auxiliary annotations record the outcome of every step; they are used as the demonstration pool for few\-shot retrieval when deconstructing the classification task into individual steps \(*Explicit*conditioning setting\)\.

#### Models\.

Four instruction\-tuned models spanning different families, parameter scales, and degrees of openness are evaluated:Gemma\-4 26B\(gemma\-4\-26B\-A4B\-it\),Gemma\-4 E4B\(gemma\-4\-E4B\-it\),Qwen3\.5 9B\(Qwen3\.5\-9B\), andEuroLLM 22B\(EuroLLM\-22B\-Instruct\-2512\)\. All models are loaded from the HuggingFace Hub and run with the Transformers library\([Wolf et al\., 2020](https://arxiv.org/html/2609.09791#bib.bib24)\)\.

#### Conditioning\.

The first component varies the legal information supplied to the model, following[Ludwig et al\. \(2025\)](https://arxiv.org/html/2609.09791#bib.bib11)\. Each*conditioning*mode corresponds to a different task description\. The*Title*setting states only the numbers and titles of the relevant StGB sections\. The*Description*setting additionally provides a description of what constitutes a defamatory offence under the annotation scheme \(Section[2\.1](https://arxiv.org/html/2609.09791#S2.SS1)\)\. The*Implicit*setting presents all six decision steps of the scheme inline, so that the model traverses the full decision tree within a single inference call\. The*Explicit*setting instead decomposes the task into the six steps as separate inference calls: each step is classified on its own, and the final label is derived from the sequence of step outcomes\. Each step is run as an independent prompt containing only that step’s instructions, its demonstrations, and the post\. All prompts and additional information on chat template and few\-shot integration are provided in Appendix[B](https://arxiv.org/html/2609.09791#A2)\.

#### Demonstration selection\.

The second component governs how few\-shot demonstrations are chosen\. In the*zero\-shot*configuration \(k=0k=0\), no demonstrations are included\. In the*few\-shot*configuration, we includekkdemonstrations\. The class*ratio*of demonstrations and the within\-prompt*ordering*of demonstrations are varied for the ablation study \(Section[5\.3](https://arxiv.org/html/2609.09791#S5.SS3)\)\. Demonstrations are selected either*statically*—a single fixed set, chosen once and reused for every test instance—or*dynamically*, where a separate set is retrieved per instance at inference time\. The dynamic strategies are our main object of study and are defined by the embedding and retrieval modes below\.

#### Embedding mode\.

Dynamic retrieval draws demonstrations from a knowledge base built over the training split and stored in one of three ways\. In the*dense*setting, each example is embedded withcodefuse\-ai/F2LLM\-v2\-1\.7B\([Zhang et al\., 2026](https://arxiv.org/html/2609.09791#bib.bib25)\), the highest\-ranked model in its size range on the MTEB leaderboard for German tasks\([Muennighoff et al\., 2023](https://arxiv.org/html/2609.09791#bib.bib26)\)throughout the entire duration of this project in the first half of 2026 \(see Appendix[C\.2](https://arxiv.org/html/2609.09791#A3.SS2)\)\. The embeddings are stored in a LangChainInMemoryVectorStore\([Chase, 2022](https://arxiv.org/html/2609.09791#bib.bib38)\)\. The*sparse*setting uses a LangChainBM25Retrieverwhich vectorises training samples sparsely and retrieves demonstrations based on keyword matches using the BM25 algorithm\. The*fusion*setting interleaves the results of both the dense and sparse retrievers\.

#### Retrieval mode\.

Given a knowledge base made up of an embedded training dataset, demonstrations are selected by one of four strategies\.*Similarity*retrieval returns the top\-kkexamples by cosine similarity \(dense\) or BM25 score \(sparse\)\.*Diversity*retrieval clusters the dense training embeddings withkk\-means and draws one example at random from each cluster\.*MMR*\(Maximal Marginal Relevance\) re\-ranks dense candidates to balancesimilarityagainstdiversity\.*Random*retrieval samples uniformly from the training pool and serves as the baseline retrieval strategy\. BecauseMMRanddiversityrely ondenseembeddings, they are applied only in thedenseembedding setting;sparseandfusionembeddings are used only forsimilarity\-based retrieval\.

#### Baseline\.

As a non\-neural reference point, we implement a naive11\-nearest\-neighbour classifier using TF\-IDF representations of the training split to assign each test post the label of its single closest training neighbour\. It uses no legal domain knowledge and relies entirely on surface\-level lexical overlap; conceptually, it corresponds to dynamic few\-shot retrieval atk=1k=1\.

## 4Experiments

#### General Setup\.

All reported results use stratified four\-fold cross\-validation over the labelled set: in each fold, three parts form the knowledge base and the held\-out part is used for evaluation\. The primary metric isF​1macroF1\_\{\\text\{macro\}\}\(unweighted mean of per\-classF​1F1scores\); we additionally report True\-class Precision \(PT\) and Recall \(RT\)\. Instances on which a model abstains \(no parseable label, model refusal\) are excluded from scoring, but abstentions are reported separately\. More details on experiment settings are reported in Appendix[C](https://arxiv.org/html/2609.09791#A3)\.222Code Repository:[https://github\.com/akristing22/GermEval2026\-def\-prompting](https://github.com/akristing22/GermEval2026-def-prompting)

### 4\.1Exploration

In order to control the computational cost during the main experiments, we aim to find the optimal number of demonstrations \(in terms ofF​1macroF1\_\{\\text\{macro\}\}score\) in an exploratory analysis: we test demonstration sizesk∈\{4,8,16,32\}k\\in\\\{4,8,16,32\\\}across two prompting configurations and all models\.

### 4\.2Main Experiments

Given that RetICL is the focus of this research, the main experiments investigate the different strategies across the four open\-weights instruction\-tuned modelsGemma\-4 26B,Gemma\-4 E4B,Qwen3\.5 9B, andEuroLLM 22B, and all four conditioning modes \(*Title*,*Description*,*Implicit*, and*Explicit*\)\. For RetICL, we evaluate*random*,*diversity*\-,*MMR*\-, and*similarity*\-based retrieval; the latter for the three different embedding methods*dense*,*sparse*, and*fusion*\. Additionally, zero\-shot prompting is tested\. The retrieved demonstrations are*balanced*in terms of their class labels, and their order in the prompt is*random*\.

### 4\.3Ablations

#### Proprietary Model\.

In order to understand how the performance on this complex task scales with model size far beyond the 4B–26B parameter range, we evaluate the latest OpenAI model of the GPT series \(GPT\-5\.5\)\. Evaluations are run with the*Title*zero\-shot setting and with*Explicit*mode with*dense**similarity*\-based demonstration retrieval\.

#### Fine\-Tuning\.

ICL strategies offer lower computational cost and greater adaptability than supervised fine\-tuning \(SFT\) methods\. In order to understand how the performance of our ICL strategies compares to an SFT baseline, we fine\-tune the smallerGemma\-4 E4Bmodel using the parameter\-efficient method QLoRA\([Dettmers et al\., 2023](https://arxiv.org/html/2609.09791#bib.bib27)\)\. The model is trained with the*Implicit*conditioning mode in a zero\-shot set\-up\.

#### Class Ratio and Ordering\.

The main experiments use only*balanced*\(w\.r\.t\. class label\),*randomly*ordered few\-shot demonstrations\. We investigate the effects of class ratio and ordering on classification performance by also arranging the demonstrations by class label \(*true\-first*and*true\-last*\) and by selecting examples with a class ratio*proportional*to the ratio in the training dataset\. These effects are tested for*Implicit*conditioning with*dense*,*similarity*\-based retrieval\.

#### Static Demonstrations\.

In order to compare RetICL demonstration approaches not only to zero\-shot prompts but to a static few\-shot strategy, a set of few\-shot demonstrations is tested, which is selected by searching randomly for the best\-performing set of demonstrations \(more details are reported in Appendix[C\.6](https://arxiv.org/html/2609.09791#A3.SS6)\)\. Two sets of eight demonstrations forGemma\-4 26Bare selected, using the*Implicit*and*Title*conditioning modes\.

### 4\.4Shared Task

We select five model / configuration combinations for running on the held\-out competition test set\. The best two runs are mentioned in the respective experiment sections; more details are reported in Appendix[D\.5](https://arxiv.org/html/2609.09791#A4.SS5)\.

## 5Results

### 5\.1Exploration

Figure 2:Exploration: MeanF​1macroF1\_\{\\text\{macro\}\}by demonstration size, per model\. The black dotted line shows the meanF​1macroF1\_\{\\text\{macro\}\}per demonstration size across models\.Across models, the largest average gain comes from increasing the demonstration count from44to88\(\+0\.029\+0\.029\)\. Beyondk=8k=8, larger sizes decrease performance slightly forGemma\-4 26B, whereas performance continues to improve for the other three models \(Fig\.[2](https://arxiv.org/html/2609.09791#S5.F2)\)\. Scores are reported in detail in Appendix[D\.1](https://arxiv.org/html/2609.09791#A4.SS1)\. To limit computational cost, we setk=8k=8for all subsequent experiments, even though larger sizes would likely benefit the other three models\.

### 5\.2Main Experiments

Figure 3:Each point shows theF​1macroF1\_\{\\text\{macro\}\}score of one model and prompting configuration\. Configurations are sorted left to right by increasing cross\-model meanF​1macroF1\_\{\\text\{macro\}\}\. The lower panel shows which conditioning and retrieval choices correspond to each position on the curve\. The best configuration per model is indicated by a star\. The dotted line shows the naive nearest\-neighbour classifier used as a baseline\.A full score table can be found in Appendix[D](https://arxiv.org/html/2609.09791#A4)\. Figure[3](https://arxiv.org/html/2609.09791#S5.F3)reportsF​1macroF1\_\{\\text\{macro\}\}score across all prompting configurations, for all models\. The baseline classifier achieves anF​1macroF1\_\{\\text\{macro\}\}of0\.6440\.644, which onlyGemma\-4 26Bconsistently surpasses\. The best performing configuration \(Gemma\-4 26B,Implicitconditioning,dense,similarity\-based retrieval\) achieves anF​1macroF1\_\{\\text\{macro\}\}score of0\.7330\.733on our test set and0\.720\.72on the held\-out competition set\. As shown in Table[1](https://arxiv.org/html/2609.09791#S5.T1), all models tend to over\-predict the positive class, yielding low positive\-class Precision; even so, between26%26\\%and57%57\\%of criminally relevant posts go undetected \(positive\-class Recall0\.430\.43–0\.740\.74\)\. While the11\-NN baseline is stronger than some models in terms ofF​1macroF1\_\{\\text\{macro\}\}, it falls substantially behind all models in terms of Recall, except forQwen3\.5 9B, which is only marginally better than the baseline \(\+0\.067\+0\.067\)\.

Table 1:Best configuration per model and naive baseline classifier\.Impl\./Expl\.: Implicit / Explicit conditioning\.D/S: dense / sparse embeddings\.Sim: similarity\-based retrieval\. Best scores per metric are indicated inbold\.Across models, more detailed conditioning tends to improve performance, with*Explicit*best on average\. It is the best mode forGemma\-4 E4B,Qwen3\.5 9B, andEuroLLM 22B, whileGemma\-4 26Bbenefits most from*Implicit*conditioning \(Table[2](https://arxiv.org/html/2609.09791#S5.T2)\)\. Except forGemma\-4 E4B, the least detailed*Title*conditioning performs worst\. However, the trend is not strictly monotonic: averaged across models,*Description*\(0\.5670\.567\) slightly exceeds the more concrete*Implicit*mode \(0\.5590\.559\)\. Zero\-shot prompting yields the worst scores for every model; the gap to few\-shot is most pronounced forEuroLLM 22Band smallest for Qwen3\.5 9B\.

ModelTitleDesc\.Impl\.Expl\.ZSFSGemma\-4 26B0\.6830\.6990\.7150\.6870\.6580\.702Gemma\-4 E4B0\.5790\.5930\.5450\.6390\.5320\.598Qwen3\.5 9B0\.5290\.5880\.5980\.6050\.5510\.585EuroLLM 22B0\.3450\.3860\.3800\.4370\.2400\.411*Mean**0\.534**0\.567**0\.559**0\.592**0\.495**0\.574*

Table 2:MeanF​1macroF1\_\{\\text\{macro\}\}per model and conditioning mode, and per zero\-shot \(ZS\) and few\-shot \(FS\),k=8k=8\. Best results per model are indicated inbold\.Figure[4](https://arxiv.org/html/2609.09791#S5.F4)reports the impact of each configuration axis on the performance of each model\. Embedding and retrieval choices have little impact onF​1macroF1\_\{\\text\{macro\}\}, whereas the differences across conditioning modes and between zero\- and few\-shot prompting are more pronounced\. The best model \(Gemma\-4 26B\) is least affected by prompting configuration, whereas the weakest \(EuroLLM 22B\) is most affected, across all axes\. For comparison: impact of model choice alone is0\.3090\.309\.

Figure 4:Impact of each configuration axis onF​1macroF1\_\{\\text\{macro\}\}: difference between highest and lowest meanF​1macroF1\_\{\\text\{macro\}\}across that axis\. Each cell reports this range for one model \(row\) and one configuration dimension \(column\), marginalising over all other axes\.#### Abstentions\.

All scores are computed only on replies that could be parsed into a binaryTrue/Falselabel\. Across all2828prompting configurations per model, fewer than1%1\\%of generations could not be parsed \(further details are reported in Appendix[D\.3](https://arxiv.org/html/2609.09791#A4.SS3)\)\.

### 5\.3Ablations

#### Proprietary Model\.

Under the minimal*Title*zero\-shot setting, GPT\-5\.5 achieves anF​1macroF1\_\{\\text\{macro\}\}of0\.4400\.440, beating onlyEuroLLM 22Bin the same configuration and falling substantially behind the other models\. With*Explicit*conditioning and*dense*,*similarity*\-based retrieval, GPT\-5\.5 reaches0\.7160\.716, beating the best open\-weights modelGemma\-4 26Bin the same configuration by0\.0180\.018, but falling behindGemma\-4 26B’s best run by0\.0170\.017\.

#### Fine\-Tuning\.

Fine\-tuned with*Implicit*conditioning in a zero\-shot set\-up,Gemma\-4 E4B \(FT\)achieves anF​1macroF1\_\{\\text\{macro\}\}of0\.7410\.741\. This is marginally above the best run of the main experiments \(Gemma\-4 26B, Impl\. D\-Sim;0\.7330\.733;\+0\.008\+0\.008\), is a substantial improvement over the best*Implicit*Gemma\-4 E4Brun \(0\.5820\.582;\+0\.159\+0\.159\), and is0\.0820\.082above the best overallGemma\-4 E4Brun \(0\.6590\.659\)\.

The fine\-tunedGemma\-4 E4B \(FT\)withImplicitconditioning also achieves our best results on the held\-out competition test set: anF​1macroF1\_\{\\text\{macro\}\}score of0\.740\.74\. It shows a contrasting pattern in positive\-class Precision \(0\.7020\.702\) and Recall \(0\.4310\.431\), being less sensitive to defamatory offences but more precise in its predictions\.

#### Class Ratio and Ordering\.

Figure 5:Interaction effects between demonstration*ratio*and*order*\. MeanF​1macroF1\_\{\\text\{macro\}\}scores per model and configuration are shown\.The impact of demonstration*ratio*and*ordering*\(in terms of class labels\) onF​1macroF1\_\{\\text\{macro\}\}is highly model\-dependent, as depicted in Figures[5](https://arxiv.org/html/2609.09791#S5.F5)and[6](https://arxiv.org/html/2609.09791#S5.F6)\.Gemma\-4 26Bis barely affected\.Gemma\-4 E4Bis only slightly affected by ordering, but the effect of class ratio is more pronounced\.EuroLLM 22Bis most affected by both class ratio and ordering\. The individual ratio and order axes have little effect onQwen3\.5 9B, yet it shows the strongest interaction between them; the interaction is similar forEuroLLM 22Bbut negligible for both Gemma\-4 models\. Across models, ordering matters more when the demonstrations are class\-balanced, whereas its effect is negligible under the*proportional*ratio—except forQwen3\.5 9B\.

Figure 6:Each cell shows theF​1macroF1\_\{\\text\{macro\}\}impact of one factor for one model\. Ratio and Order report the range of marginal means\. Interaction reports the range of interaction residualsδ⁡\(r,o\)=F1​\(r,o\)−F1¯r−F1¯o\+F1¯\\delta\(r,o\)=\\text\{F1\}\(r,o\)\-\\bar\{\\text\{F1\}\}\_\{r\}\-\\bar\{\\text\{F1\}\}\_\{o\}\+\\bar\{\\text\{F1\}\}\.
#### Static Demonstrations\.

The sets of demonstrations selected with the random search are reported in Appendix[B\.3](https://arxiv.org/html/2609.09791#A2.SS3)\. Under the static configuration,Gemma\-4 26Breaches anF​1macroF1\_\{\\text\{macro\}\}of0\.7230\.723for*Title*conditioning and anF​1macroF1\_\{\\text\{macro\}\}of0\.7540\.754for*Implicit*conditioning, beating the results of best*Title*\(\+0\.028\+0\.028\) and*Implicit*\(\+0\.021\+0\.021\) conditioning configuration of the main experiments\.

## 6Discussion

Although RetICL is our focus, the retrieval mechanisms contribute remarkably little to model performance: embedding mode and retrieval strategy moveF​1macroF1\_\{\\text\{macro\}\}far less than the conditioning mode, the zero\- / few\-shot distinction, or the number, class ratio, and ordering of demonstrations, and the elaborate dynamic strategies barely beat*random*retrieval\. This nuances the endorsement of similarity\-based selection by[Margatina et al\. \(2023\)](https://arxiv.org/html/2609.09791#bib.bib18)and aligns with[Liu and Shi \(2024\)](https://arxiv.org/html/2609.09791#bib.bib10): for criminally relevant hate speech, the gains demonstrations bring primarily come from few\-shot prompting itself, not from how examples are selected\. Also consistent with[Liu and Shi \(2024\)](https://arxiv.org/html/2609.09791#bib.bib10)is that an optimised, static set of demonstrations improves performance over all dynamic approaches within the same conditioning mode \(at least for the tested configurations onGemma\-4 26B\), suggesting that good demonstration selection for this task depends on metrics other than those used for dynamic selection\. The fragility of fine\-grained demonstration tuning is underlined by the shared task results: the ratio / ordering configuration that won in cross\-validation fell behind the default on the held\-out test set \(Appendix[D\.5](https://arxiv.org/html/2609.09791#A4.SS5)\)\.

Model choice matters more than prompting: its mean impact onF​1macroF1\_\{\\text\{macro\}\}\(0\.3090\.309\) exceeds that of any single prompting axis \(Fig\.[4](https://arxiv.org/html/2609.09791#S5.F4)\)\. Scale, however, is not decisive—the smallGemma\-4 E4B\(0\.6590\.659\) beats the largerEuroLLM 22B\(0\.4740\.474\), and even GPT\-5\.5 collapses to0\.4400\.440under*Title*zero\-shot prompting before reaching a result of0\.7160\.716with*Explicit*conditioning\. Weaker models are also often more configuration\-sensitive \(EuroLLM 22Bspans∼0\.5\{\\sim\}0\.5F​1macroF1\_\{\\text\{macro\}\}across all its runs, including ablations\), so a poor model should not be dismissed before its prompting is tuned; capability tracks training and conditioning rather than size\([Miller et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib2)\)\.

Contrary to[Ludwig et al\. \(2025\)](https://arxiv.org/html/2609.09791#bib.bib11), who found abstract conditioning superior for § 130 StGB, more concrete, decomposed legal knowledge helps here:*Explicit*conditioning is best on average and best for three of four models, while*Title*is usually worst\. The effect is neither strictly monotonic nor model\-independent, supporting subtask decomposition\([AlKhamissi et al\., 2022](https://arxiv.org/html/2609.09791#bib.bib32)\)while showing that the right form of legal conditioning depends on the model\. Decomposition also aids explainability, a further argument for this configuration\.

The naive TF\-IDF baseline \(F​1macro=0\.644F1\_\{\\text\{macro\}\}=0\.644\) is strikingly competitive, outperforming most smaller\-model runs\. This may reflect homogeneous data, where lexical overlap already proxies the label well—a shortcut unlikely to survive distribution shift or concept drift, where LLMs’ semantic abilities should matter more\. Reliability is a further concern: Recall on the positive class exceeds Precision, so the systems over\-predict criminal relevance yet still miss between26%26\\%and57%57\\%of criminally relevant posts, positioning these models as triage tools for human review rather than for unsupervised moderation\. Under this framing, the models clearly beat the baseline, which misses6464% of criminally relevant posts\. Over\-sensitivity of LLMs may be an unintended side\-effect of alignment:[Selvaganapathy and Nasim \(2026\)](https://arxiv.org/html/2609.09791#bib.bib39)find that minimally aligned models outperform aligned ones in hate speech classification, echoing a broader unintended trade\-off between safety and truthfulness reported by[Mahmoud et al\. \(2026\)](https://arxiv.org/html/2609.09791#bib.bib40)\.

Finally, our QLoRA fine\-tune of the smallGemma\-4 E4B\(F​1macro=0\.741F1\_\{\\text\{macro\}\}=0\.741\) exceeds the best dynamic prompting configuration \(0\.7330\.733,Gemma\-4 26B\), a lead that holds on the held\-out test set \(0\.740\.74vs\.0\.720\.72\): a cheaply fine\-tuned 4B model can match a carefully prompted 26B one, so where a modest labelled set exists, parameter\-efficient fine\-tuning is a strong alternative to ICL\. However, it is less suited as a triage tool, as it misses more than fifty percent of defamatory offences, albeit being more precise when predicting the positive class\.

## 7Conclusion

We tested prompting strategies for detecting defamatory offences under§§ 185–187 StGB, varying both the legal knowledge supplied to the model and the way few\-shot demonstrations are retrieved\. Contrary to[Ludwig et al\. \(2025\)](https://arxiv.org/html/2609.09791#bib.bib11), more concrete legal knowledge helps; and while few\-shot prompting clearly outperforms zero\-shot, more elaborate retrieval offers only a marginal advantage over randomly chosen demonstrations\. Optimised static demonstrations can outperform dynamic retrieval, at least in the settings we tested\. Model choice has more impact on task performance than any prompting configuration axis alone, and the task remains hard: most of the open\-weights models we tested do not consistently match a naive nearest\-neighbour baseline in terms ofF​1macroF1\_\{\\text\{macro\}\}\. However, models outperform this baseline consistently in terms of Recall, which we argue is the more relevant metric for a triage setting\. The interaction of demonstration class ratio and ordering, the way such positional and class biases are represented within a model, and their stability across data splits remain promising directions for future work\.

## Limitations

Our findings are confined to a single language, dataset, and annotation scheme, all drawn from the same distribution; the homogeneity that makes the nearest\-neighbour baseline strong also limits external validity\. Assessing how concept drift and more heterogeneous data affect the baseline, the RetICL strategies, and the fine\-tuning approach is a natural next step—we conjecture that nearest\-neighbour, SFT and static demonstration approaches, which lean on specific training instances, may degrade more than concept\-driven LLM prompting under such shift, though this remains to be tested\. We also did not exhaust the entire configuration space: the most favourable demonstration settings were not all combined\. Reasoning\-based legal prompting \(*legal syllogism*/ chain\-of\-thought;[Zhang et al\., 2025](https://arxiv.org/html/2609.09791#bib.bib28);[Jiang and Yang, 2023](https://arxiv.org/html/2609.09791#bib.bib29);[Deng et al\., 2024](https://arxiv.org/html/2609.09791#bib.bib30)\) remains an underexplored extension for explicitly modelling the statutory decision tree, as we only use models with their respective*thinking modes*disabled\. Finally, §§ 185–187 are usually prosecuted only on application by the victim \(§ 194 StGB\); proactive automatic detection is therefore of more limited practical relevance for these offences than for*ex officio*offences such as § 130 StGB \(incitement to hatred\), where the public prosecutor initiates proceedings on its own motion\([Beulke and Swoboda, 2025](https://arxiv.org/html/2609.09791#bib.bib36)\)\.

## References

- A\. Albladi, M\. Islam, A\. Das, M\. Bigonah, Z\. Zhang, F\. Jamshidi, M\. Rahgouy, N\. Raychawdhary, D\. Marghitu, and C\. SealsHate Speech Detection Using Large Language Models: A Comprehensive Review\.IEEE Access13\(\),pp\. 20871–20892\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2025.3532397)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1)\.
- AlKhamissiet al\.\(2022\)B\. AlKhamissi, F\. Ladhak, S\. Iyer, V\. Stoyanov, Z\. Kozareva, X\. Li, P\. Fung, L\. Mathias, A\. Celikyilmaz, and M\. DiabToKen: Task decomposition and knowledge infusion for few\-shot hate speech detection\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp\. 2109–2120\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.136/)Cited by:[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p2.1),[§6](https://arxiv.org/html/2609.09791#S6.p3.1)\.
- Beulke and Swoboda \(2025\)W\. Beulke and S\. SwobodaStrafprozessrecht\.CF Müller\.Note:ISBN: 9783811468115Cited by:[Limitations](https://arxiv.org/html/2609.09791#Sx1.p1.1)\.
- Chase \(2022\)LangChainExternal Links:[Link](https://github.com/langchain-ai/langchain)Cited by:[§3](https://arxiv.org/html/2609.09791#S3.SS0.SSS0.Px5.p1.1)\.
- Demuset al\.\(2022\)C\. Demus, J\. Pitz, M\. Schütz, N\. Probol, M\. Siegel, and D\. LabuddeDeTox: A Comprehensive Dataset for German Offensive Language and Conversation Analysis\.InProceedings of the Sixth Workshop on Online Abuse and Harms \(WOAH\),K\. Narang, A\. Mostafazadeh Davani, L\. Mathias, B\. Vidgen, and Z\. Talat \(Eds\.\),Seattle, Washington \(Hybrid\),pp\. 143–153\.External Links:[Link](https://aclanthology.org/2022.woah-1.14/),[Document](https://dx.doi.org/10.18653/v1/2022.woah-1.14)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1)\.
- Denget al\.\(2024\)C\. Deng, K\. Mao, Y\. Zhang, and Z\. DouEnabling Discriminative Reasoning in LLMs for Legal Judgment Prediction\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 784–796\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.43/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.43)Cited by:[Limitations](https://arxiv.org/html/2609.09791#Sx1.p1.1)\.
- Destatis \(2025\)DestatisEin Drittel der Internetnutzenden stößt im Netz auf Hatespeech\. Pressemitteilung Nr\. 421 vom 27\. November 2025\.Note:Last accessed 03\.08\.2026External Links:[Link](https://www.destatis.de/DE/Presse/Pressemitteilungen/2025/11/PD25_421_63.html)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1)\.
- Dettmerset al\.\(2023\)T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. ZettlemoyerQLoRA: Efficient Finetuning of Quantized LLMs\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 10088–10115\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/1feb87871436031bdc0f2beaa62a049b-Paper-Conference.pdf)Cited by:[§4\.3](https://arxiv.org/html/2609.09791#S4.SS3.SSS0.Px2.p1.1)\.
- European Union Agency for Fundamental Rights \(2023\)European Union Agency for Fundamental RightsOnline content moderation–Current challenges in detecting hate speech\.Publications Office of the European Union\.Note:Last accessed 03\.08\.2026External Links:[Link](https://fra.europa.eu/sites/default/files/fra_uploads/fra-2023-online-content-moderation_en.pdf)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1)\.
- Fanet al\.\(2024\)W\. Fan, Y\. Ding, L\. Ning, S\. Wang, H\. Li, D\. Yin, T\. Chua, and Q\. LiA Survey on RAG Meeting LLMs: Towards Retrieval\-Augmented Large Language Models\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,KDD ’24,New York, NY, USA,pp\. 6491–6501\.External Links:ISBN 9798400704901,[Link](https://doi.org/10.1145/3637528.3671470),[Document](https://dx.doi.org/10.1145/3637528.3671470)Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p2.1)\.
- Felseret al\.\(2026\)J\. Felser, T\. Holzhausen, T\. Katzwinkel, F\. Zufall, F\. Ludwig, M\. Hofer, M\. Siegel, and M\. SprangerOverview of the GermEval 2026 Shared Task on Harmful Content Detection\.InProceedings of the 22nd Conference on Natural Language Processing \(KONVENS 2026\): Workshops,Hamburg, Germany\.Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p2.1)\.
- Gan and Mori \(2023\)C\. Gan and T\. MoriSensitivity and Robustness of Large Language Models to Prompt Template in Japanese Text Classification Tasks\.External Links:2305\.08714,[Link](https://arxiv.org/abs/2305.08714)Cited by:[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p1.1)\.
- Gandhiet al\.\(2024\)A\. Gandhi, P\. Ahir, K\. Adhvaryu, P\. Shah, R\. Lohiya, E\. Cambria, S\. Poria, and A\. HussainHate speech detection: A comprehensive review of recent works\.Expert Systems41\(8\),pp\. e13562\.External Links:[Document](https://dx.doi.org/10.1007/s00530-023-01051-8)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1)\.
- Glasebachet al\.\(2024\)J\. Glasebach, M\. Keller, A\. Döschl, and P\. MandlGMHP7k: A Corpus of German Misogynistic Hatespeech Posts\.Proceedings of the International AAAI Conference on Web and Social Media18\(1\),pp\. 1946–1957\.External Links:[Link](https://ojs.aaai.org/index.php/ICWSM/article/view/31438),[Document](https://dx.doi.org/10.1609/icwsm.v18i1.31438)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1)\.
- Goldzycheret al\.\(2024\)J\. Goldzycher, P\. Röttger, and G\. SchneiderImproving adversarial data collection by supporting annotators: lessons from GAHD, a German hate speech dataset\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 4405–4424\.External Links:[Link](https://aclanthology.org/2024.naacl-long.248/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.248)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1)\.
- Jiang and Yang \(2023\)C\. Jiang and X\. YangLegal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Prediction\.InProceedings of the Nineteenth International Conference on Artificial Intelligence and Law,ICAIL ’23,New York, NY, USA,pp\. 417–421\.External Links:ISBN 9798400701979,[Link](https://doi.org/10.1145/3594536.3595170),[Document](https://dx.doi.org/10.1145/3594536.3595170)Cited by:[Limitations](https://arxiv.org/html/2609.09791#Sx1.p1.1)\.
- Kim and Lee \(2025\)Y\. Kim and H\. LeeSelective Demonstration Retrieval for Improved Implicit Hate Speech Detection\.arXiv\(en\)\.Note:arXiv:2504\.12082 \[cs\]External Links:[Link](http://arxiv.org/abs/2504.12082),[Document](https://dx.doi.org/10.48550/arXiv.2504.12082)Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p3.1)\.
- Kumset al\.\(2025\)V\. Kums, F\. Meyer, L\. Pivit, U\. Vedenina, J\. Wortmann, M\. Siegel, and D\. LabuddeA Novel Dataset for Classifying German Hate Speech Comments with Criminal Relevance\.InProceedings of the The 9th Workshop on Online Abuse and Harms \(WOAH\),A\. Calabrese, C\. de Kock, D\. Nozza, F\. M\. Plaza\-del\-Arco, Z\. Talat, and F\. Vargas \(Eds\.\),Vienna, Austria,pp\. 41–52\.External Links:[Link](https://aclanthology.org/2025.woah-1.4/),ISBN 979\-8\-89176\-105\-6Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1),[§1](https://arxiv.org/html/2609.09791#S1.p2.1)\.
- Liu and Shi \(2024\)M\. Liu and G\. ShiPoliPrompt: A High\-Performance Cost\-Effective LLM\-Based Text Classification Framework for Political Science\.Available at SSRN 4940136\.External Links:[Link](https://papers.ssrn.com/sol3/Delivery.cfm?abstractid=4940136)Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p3.1),[§6](https://arxiv.org/html/2609.09791#S6.p1.1)\.
- Luet al\.\(2022\)Y\. Lu, M\. Bartolo, A\. Moore, S\. Riedel, and P\. StenetorpFantastically Ordered Prompts and Where to Find Them: Overcoming Few\-Shot Prompt Order Sensitivity\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 8086–8098\.External Links:[Link](https://aclanthology.org/2022.acl-long.556/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.556)Cited by:[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p1.1)\.
- Ludwiget al\.\(2025\)F\. Ludwig, T\. Zesch, and F\. ZufallConditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech\.InProceedings of the 21st Conference on Natural Language Processing \(KONVENS 2025\): Long and Short Papers,C\. Wartena and U\. Heid \(Eds\.\),Hannover, Germany,pp\. 154–167\.External Links:[Link](https://aclanthology.org/2025.konvens-1.14/)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.SS0.SSS0.Px1.p3.1),[§1](https://arxiv.org/html/2609.09791#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p2.1),[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p3.1),[§3](https://arxiv.org/html/2609.09791#S3.SS0.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2609.09791#S6.p3.1),[§7](https://arxiv.org/html/2609.09791#S7.p1.1)\.
- Luoet al\.\(2024\)M\. Luo, X\. Xu, Y\. Liu, P\. Pasupat, and M\. KazemiIn\-context learning with retrieved demonstrations for language models: A survey\.arXiv preprint arXiv:2401\.11624\.External Links:[Link](https://arxiv.org/pdf/2401.11624)Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p1.1),[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p2.1)\.
- Mahmoudet al\.\(2026\)O\. Mahmoud, A\. Khalil, T\. G\. Karimpanal, B\. L\. Semage, and S\. RanaThe Unintended Trade\-off of AI Alignment: Balancing Hallucination Mitigation and Safety in LLMs\.InFindings of the Association for Computational Linguistics: EACL 2026,V\. Demberg, K\. Inui, and L\. Marquez \(Eds\.\),Rabat, Morocco,pp\. 1017–1037\.External Links:[Link](https://aclanthology.org/2026.findings-eacl.53/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-eacl.53),ISBN 979\-8\-89176\-386\-9Cited by:[§6](https://arxiv.org/html/2609.09791#S6.p4.1)\.
- Margatinaet al\.\(2023\)K\. Margatina, T\. Schick, N\. Aletras, and J\. Dwivedi\-YuActive Learning Principles for In\-Context Learning with Large Language Models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 5011–5034\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.334/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.334)Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p2.1),[§6](https://arxiv.org/html/2609.09791#S6.p1.1)\.
- Milleret al\.\(2025\)K\. Miller, S\. Bedrick, Q\. Lu, A\. Wen, W\. Hersh, K\. Roberts, and H\. LiuDynamic few\-shot prompting for clinical note section classification using lightweight, open\-source large language models\.Journal of the American Medical Informatics Association32\(7\),pp\. 1164–1173\.External Links:ISSN 1527\-974X,[Link](https://www.researchgate.net/profile/Kurt-Miller-7/publication/392368366_Dynamic_few-shot_prompting_for_clinical_note_section_classification_using_lightweight_open-source_large_language_models/links/6862a30d07b3253fd1ca9c49/Dynamic-few-shot-prompting-for-clinical-note-section-classification-using-lightweight-open-source-large-language-models.pdf),[Document](https://dx.doi.org/10.1093/jamia/ocaf084)Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p2.1),[§6](https://arxiv.org/html/2609.09791#S6.p2.1)\.
- Muennighoffet al\.\(2023\)N\. Muennighoff, N\. Tazi, L\. Magne, and N\. ReimersMTEB: Massive Text Embedding Benchmark\.InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics,A\. Vlachos and I\. Augenstein \(Eds\.\),Dubrovnik, Croatia,pp\. 2014–2037\.External Links:[Link](https://aclanthology.org/2023.eacl-main.148/),[Document](https://dx.doi.org/10.18653/v1/2023.eacl-main.148)Cited by:[§3](https://arxiv.org/html/2609.09791#S3.SS0.SSS0.Px5.p1.1)\.
- Pariziet al\.\(2023\)A\. H\. Parizi, Y\. Liu, P\. Nokku, S\. Gholamian, and D\. B\. EmersonA Comparative Study of Prompting Strategies for Legal Text Classification\.Proceedings of the Natural Legal Language Processing Workshop 2023\.External Links:[Link](https://api.semanticscholar.org/CorpusID:265608067)Cited by:[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p1.1)\.
- Research Services of the German Parliament \(2022\)Research Services of the German ParliamentFundamentals of Defamatory Offences \(§§ 185 ff\. StGB\) from a Criminal and Civil Law Perspective\.SachstandTechnical ReportWD 7 \- 3000 \- 052/22,Deutscher Bundestag\.External Links:[Link](https://www.bundestag.de/resource/blob/908298/WD-7-052-22-pdf.pdf)Cited by:[Appendix A](https://arxiv.org/html/2609.09791#A1.p1.1),[§2\.1](https://arxiv.org/html/2609.09791#S2.SS1.p1.1)\.
- Robertson and Jones \(1976\)S\. E\. Robertson and K\. S\. JonesRelevance weighting of search terms\.Journal of the American Society for Information Science27\(3\),pp\. 129–146\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.1002/asi.4630270302),[Link](https://asistdl.onlinelibrary.wiley.com/doi/abs/10.1002/asi.4630270302),https://asistdl\.onlinelibrary\.wiley\.com/doi/pdf/10\.1002/asi\.4630270302Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p2.1)\.
- Robertson and Zaragoza \(2009\)S\. Robertson and H\. ZaragozaThe Probabilistic Relevance Framework: BM25 and Beyond\.Found\. Trends Inf\. Retr\.3\(4\),pp\. 333–389\.External Links:ISSN 1554\-0669,[Link](https://doi.org/10.1561/1500000019),[Document](https://dx.doi.org/10.1561/1500000019)Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p2.1)\.
- Sasseet al\.\(2025\)K\. Sasse, C\. A\. Aguirre, I\. Cachola, S\. Levy, and M\. DredzeMaking FETCH\! Happen: Finding Emergent Dog Whistles Through Common Habitats\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 5687–5709\.External Links:[Link](https://aclanthology.org/2025.acl-long.284/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.284),ISBN 979\-8\-89176\-251\-0Cited by:[§2\.3](https://arxiv.org/html/2609.09791#S2.SS3.p3.1)\.
- Schäfer \(2023\)J\. SchäferBias Mitigation for Capturing Potentially Illegal Hate Speech\.Datenbank\-Spektrum23\(1\),pp\. 41–51\.External Links:[Document](https://dx.doi.org/10.1007/s13222-023-00439-0)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p2.1)\.
- Sclaret al\.\(2024\)M\. Sclar, Y\. Choi, Y\. Tsvetkov, and A\. SuhrQuantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=RIu5lyNXjT)Cited by:[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p1.1)\.
- Selvaganapathy and Nasim \(2026\)S\. Selvaganapathy and M\. NasimConfident, Calibrated, or Complicit: Safety Alignment and Ideological Bias in LLM Hate Speech Detection\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 34537–34552\.External Links:[Link](https://aclanthology.org/2026.acl-long.1594/),ISBN 979\-8\-89176\-390\-6Cited by:[§6](https://arxiv.org/html/2609.09791#S6.p4.1)\.
- Trautmannet al\.\(2022\)D\. Trautmann, A\. Petrova, and F\. SchilderLegal prompt engineering for multilingual legal judgement prediction\.External Links:2212\.02199,[Link](https://arxiv.org/abs/2212.02199)Cited by:[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p1.1)\.
- Usmanet al\.\(2025\)M\. Usman, M\. Ahmad, G\. Sidorov, I\. Gelbukh, and R\. Q\. TellezA Large Language Model\-Based Approach for Multilingual Hate Speech Detection on Social Media\.Computers14\(7\)\.External Links:[Link](https://www.mdpi.com/2073-431X/14/7/279),ISSN 2073\-431X,[Document](https://dx.doi.org/10.3390/computers14070279)Cited by:[§1](https://arxiv.org/html/2609.09791#S1.p1.1)\.
- von Cossel \(2026\)O\. von CosselBeyond Imperfect Alternatives with Rulemapping: A Neuro\-Symbolic Case Study on Online Hate Speech\.External Links:2605\.16280,[Link](https://arxiv.org/abs/2605.16280)Cited by:[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p3.1)\.
- Wolfet al\.\(2020\)T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz, J\. Davison, S\. Shleifer, P\. von Platen, C\. Ma, Y\. Jernite, J\. Plu, C\. Xu, T\. Le Scao, S\. Gugger, M\. Drame, Q\. Lhoest, and A\. RushTransformers: State\-of\-the\-Art Natural Language Processing\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,Q\. Liu and D\. Schlangen \(Eds\.\),Online,pp\. 38–45\.External Links:[Link](https://aclanthology.org/2020.emnlp-demos.6/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-demos.6)Cited by:[Appendix C](https://arxiv.org/html/2609.09791#A3.p1.1),[§3](https://arxiv.org/html/2609.09791#S3.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2025\)K\. Zhang, W\. Yu, Z\. Sun, and J\. XuSyLeR: A Framework for Explicit Syllogistic Legal Reasoning in Large Language Models\.InProceedings of the 34th ACM International Conference on Information and Knowledge Management,CIKM ’25,New York, NY, USA,pp\. 4117–4127\.External Links:ISBN 9798400720406,[Link](https://doi.org/10.1145/3746252.3761120),[Document](https://dx.doi.org/10.1145/3746252.3761120)Cited by:[Limitations](https://arxiv.org/html/2609.09791#Sx1.p1.1)\.
- Zhanget al\.\(2026\)Z\. Zhang, Z\. Liao, H\. Yu, P\. Di, and R\. WangF2LLM\-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World\.External Links:2603\.19223,[Link](https://arxiv.org/abs/2603.19223)Cited by:[§C\.2](https://arxiv.org/html/2609.09791#A3.SS2.p1.1),[§3](https://arxiv.org/html/2609.09791#S3.SS0.SSS0.Px5.p1.1)\.
- Zufallet al\.\(2022\)F\. Zufall, M\. Hamacher, K\. Kloppenborg, and T\. ZeschA Legal Approach to Hate Speech – Operationalizing the EU’s Legal Framework against the Expression of Hatred as an NLP Task\.InProceedings of the Natural Legal Language Processing Workshop 2022,N\. Aletras, I\. Chalkidis, L\. Barrett, C\. Goanță, and D\. Preoțiuc\-Pietro \(Eds\.\),Abu Dhabi, United Arab Emirates \(Hybrid\),pp\. 53–64\.External Links:[Link](https://aclanthology.org/2022.nllp-1.5/),[Document](https://dx.doi.org/10.18653/v1/2022.nllp-1.5)Cited by:[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p3.1)\.
- Zufallet al\.\(2019\)F\. Zufall, T\. Horsmann, and T\. ZeschFrom legal to technical concept: Towards an automated classification of German political Twitter postings as criminal offenses\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 1337–1347\.External Links:[Link](https://aclanthology.org/N19-1135/),[Document](https://dx.doi.org/10.18653/v1/N19-1135)Cited by:[Appendix A](https://arxiv.org/html/2609.09791#A1.p1.1),[Appendix A](https://arxiv.org/html/2609.09791#A1.p3.1),[Appendix A](https://arxiv.org/html/2609.09791#A1.p7.1),[Appendix A](https://arxiv.org/html/2609.09791#A1.p8.1),[Appendix A](https://arxiv.org/html/2609.09791#A1.p9.1),[§2\.1](https://arxiv.org/html/2609.09791#S2.SS1.p2.1),[§2\.1](https://arxiv.org/html/2609.09791#S2.SS1.p3.1),[§2\.2](https://arxiv.org/html/2609.09791#S2.SS2.p2.1),[§3](https://arxiv.org/html/2609.09791#S3.SS0.SSS0.Px1.p1.1)\.

## Appendix

## Appendix AAnnotation Scheme

The dataset of the shared task is labelled according to the decision scheme of[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)for§§ 185–187 StGB\([subsection 2\.1](https://arxiv.org/html/2609.09791#S2.SS1)\)\. The three offences differ mainly in the nature of the statement and its addressee\([Research Services of the German Parliament, 2022](https://arxiv.org/html/2609.09791#bib.bib20)\):

- •§ 185*Insult*\(*Beleidigung*\): expressions of contempt toward a person, typically value judgments that can be proven neither true nor false\.
- •§ 186*Defamation*\(*Üble Nachrede*\): asserting a disparaging*fact*about someone to third parties when its truth cannot be established\.
- •§ 187*Intentional Defamation*\(*Verleumdung*\): the aggravated case of knowingly spreading a false fact, carrying the highest penalty\.

The scheme of[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)operationalises punishability as six binary \(yes / no\) decisions taken solely from the text of a post: a three\-step spine \(steps 1–3\) and, for value judgments, a three\-way balancing of competing rights \(steps 4–6\)\.

1. 1\.Defamatory object\.The post must target a valid holder of the right to honour: a*living individual**or*a*specific group*distinguishable from the general public \(including collective entities such as governments or companies\)\. If neither is addressed, the statement is*not punishable*\.
2. 2\.Disparaging statement\.The statement must express contempt or allege shortcomings that could lower the victim’s social standing\. If it is not disparaging, it is*not punishable*\.
3. 3\.Value judgment vs\. factual claim\.A disparaging statement is either a factual claim \(provably true or false\) or a value judgment \(personal opinion\)\. Establishing the punishability of a*factual claim*would require the court to take evidence on its truth—information beyond the text—so the assessment halts here\.*Value judgments*proceed to the balancing step\.

Balancing of rights \(steps 4–6; value judgments only\)\.A value judgment may be protected under freedom of expression and thus be safeguarded aslegitimate interest\(§ 193 StGB\)\. The scheme encodes the established case\-law for weighing this freedom against the right to honour as three further binary decisions:

1. 4\.*Abusive insult*\(*Formalbeleidigung*\): a taboo\-breaking statement intended only to defame, excluded from free\-speech protection→\\rightarrow*punishable*\.
2. 5\.*Topic of public interest*: a contribution to public discourse, carrying a presumption in favour of free speech→\\rightarrow*usually not punishable*\.
3. 6\.*Abusive criticism*\(*Schmähkritik*\): a statement going beyond plausible criticism primarily to offend the victim→\\rightarrow*usually punishable*\.

Where these signals conflict or coincide, the outcome depends on a free judicial balancing of the individual circumstances, which[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)deliberately leave unimplemented\. The task, however, requires strictly binary True / False labels and does not allow for an “undecided” class\. We therefore resolved undecided steps to “not punishable”, following the principlein dubio pro reo\(“when in doubt, for the accused”\)\. An alternative would be to resolve undecided steps to “punishable” instead—a defensible choice if the scheme is framed as a triage tool, as we argue for in the main part of this paper, where the goal is to flag potentially problematic content for human review rather than to render a final legal judgment\.

Relatedly, while[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)operationalise thepunishabilityof a post, we frame the task more broadly in this paper as classifying posts with respect to theircriminal relevanceunder§§ 185–187 StGB\. Since we are not legal experts and frame the task as a triage tool for law enforcement or content moderation rather than a substitute for judicial assessment, we deliberately leave any definitive judgement of punishability or prosecutability to human experts downstream\.

[Zufall et al\. \(2019\)](https://arxiv.org/html/2609.09791#bib.bib16)report that suitably instructed laypeople can apply the scheme with reasonable reliability relative to an expert reference, though they tend to interpret*specific group*and*disparaging statement*more leniently\.

## Appendix BConditioning

### B\.1Content

System PromptYou are a legal expert for defamatory offences according to the German Criminal Code \(§§ 185–187 StGB\)\. Help the user decide whether given posts fall within the scope of these sections\.

#### Task

All task descriptions are followed by the prompt to “Answer only with ’True’ \(criminally relevant according to§§ 185–187 StGB\) or ’False’ \(not criminally relevant according to§§ 185–187 StGB\)\.”, or “Answer only with ’True’ or ’False’\.” for the explicit conditioning mode\.

TitleIs the following post criminally relevant with respect to§§ 185–187 StGB\(Defamatory Offence\)?

DescriptionIs the following post criminally relevant with respect to§§ 185–187 StGB\(Defamatory Offence\)? A statement is criminally relevant under these titles only if either a living individual or a specific group is an object of the respective statement, if the statement is disparaging, the statement is a value judgement and not a factual claim\. If these hold and the statement constitutes an abusive insult, it is criminally relevant\. If it is not an abusive insult, it is only criminally relevant if it contains abusive criticism and if the topic is not of public interest\.

Implicit StepsStep 1: A defamatory object can be a living individual, a group of persons that is distinguishable from the general public such that every member of that group could feel their honour is infringed or collective entities such as governments or press companies with a recognised social role and who act with a collective, single will\. Is there a defamatory object in this statement? No → STOP \(NOT criminally relevant according to§§ 185–187 StGB\) Yes → Step 2 Step 2: A disparaging statement is a statement which interferes with the potential victim’s right to honour\. It is already fulfilled by expressing contempt or disrespect through the allegation of shortcomings that could reduce the victim’s social standing\. Is there a disparaging statement directed towards the defamatory object? No → STOP \(NOT criminally relevant according to§§ 185–187 StGB\) Yes → Step 3 Step 3: A factual claim is a statement that can be proven to be true or untrue in front of a court\. A value judgment constitutes an expression of personal opinions\. Is it primarily a value judgement and NOT primarily a factual claim? No → STOP \(NOT criminally relevant according to§§ 185–187 StGB\) Yes → Step 4 Step 4: An abusive insult is a statement that constitutes breaking a taboo by itself and intends only the defamation of the victim without any substantiated contribution\. Is the statement an abusive insult? Yes → STOP \(criminally RELEVANT according to§§ 185–187 StGB\) No → Step 5 Step 5: A statement is of public interest if it contains a contribution to the public discourse with respect to a particular relevant topic of public interest\. Is the statement of public interest? Yes → STOP \(NOT criminally relevant according to§§ 185–187 StGB\) No → Step 6 Step 6: A statement is considered abusive criticism if it goes beyond plausible criticism by primarily intending to abusively offend the victim, hereby neglecting a substantiated contribution\. Is the statement abusive criticism? Yes → STOP \(criminally RELEVANT according to§§ 185–187 StGB\) No → STOP \(NOT criminally relevant according to§§ 185–187 StGB\) According to this decision scheme, decide whether the following text is criminally relevant according to§§ 185–187 StGB\(Defamatory Offence\)\.

ExplicitStep 1A defamatory object can be a living individual, a group of persons that is distinguishable from the general public such that every member of that group could feel their honour is infringed or collective entities such as governments or press companies with a recognised social role and who act with a collective, single will\.Is there a defamatory object in this statement?Step 2A disparaging statement is a statement which interferes with the potential victim’s right to honour\. It is already fulfilled by expressing contempt or disrespect through the allegation of shortcomings that could reduce the victim’s social standing\.Is there a disparaging statement directed towards the defamatory object?Step 3A factual claim is a statement that can be proven to be true or untrue in front of a court\. A value judgment constitutes an expression of personal opinions\.Is the statement primarily a value judgement?Step 4An abusive insult is a statement that constitutes breaking a taboo by itself and intends only the defamation of the victim without any substantiated contribution\.Is the statement an abusive insult?Step 5A statement is of public interest if it contains a contribution to the public discourse with respect to a particular relevant topic of public interest\.Is the statement of public interest?Step 6A statement is considered abusive criticism if it goes beyond plausible criticism by primarily intending to abusively offend the victim, hereby neglecting a substantiated contribution\.Is the statement abusive criticism?

### B\.2Chat Template

All experiments were conducted with*user*\-*assistant*turns using the models’ tokenisers’ chat templates\. The prompts are structured as follows:

\[\{role:system,content:system prompt\}, \{role:user,content:task\}, \{role:user,content:demonstration\_1\}, \{role:assistant,content:class\_1\}, \{role:user,content:demonstration\_2\}, \{role:assistant,content:class\_2\}, \.\.\. \{role:user,content:demonstration\_k\}, \{role:assistant,content:class\_k\}, \{role:user,content:post\}\]

### B\.3Static Demonstrations

Content WarningThis section contains examples of hate speech, including instances that are highly offensive and, in some cases, criminally relevant under German law \(§ 130,§§ 185–187 StGB, inter alia\)\. These examples are included solely for scientific and illustrative purposes, to enable transparent evaluation and reproducibility of our methods\. They do not reflect the views of the authors\. Reader discretion is advised\.

The sets of demonstrations optimised using a random search \(see Appendix[C\.6](https://arxiv.org/html/2609.09791#A3.SS6)\) are reported here\.

Title Conditioning"… komisches Interview… Hallo Baden\-Württemberger: was hat euch dazu bewogen, diese Spastiker zu wählen ???…\." True "Die ehrlose faschistische Ratte habe ich nie gekannt, sie ist ausgelöscht aus meinem Leben\." True "Im Sommer werden solche Journalisten gehängt ;\)" False "Linksfaschos schlagen auf Grenzer ein damit Flüchtlinge aus dem Herkunftsland der Linksfaschos ausreisen können? Ja wie bekloppt ist das denn? Das ist doch völlig absurd\!" False "Ob Journalisten oder Politiker, es sind doch alle nur noch scheinheilige Geister\. Wenn sich diese sogenannte feine Gesellschaft anmaßt uns als Nazi, Mischpoke und Pack zu betiteln, dann werden wir uns doch erlauben dürfen dieses miese etwas in ähnlicher Weise zu behandeln\." False "SOLLEN FROH SEIN ÜBERHAUPT HIER SEIN ZU DÜRFEN, DRECKSGELUMPE" False "Wer keine Ahnung von der Realität hat sollte die Fresse halten, kleines Antideutsches \[name\] Würstchen :D \[url\]" True "\[name\] du roter vollpfosten" True

Implicit Conditioning"@user, du bist geistig behindert und ein saudummer AFD\-Russentroll \!\!\!" True "Grammatik bei Yoda gelernt du hast \! Was für ein unterbelichtetes Höhlenkind\. And now: GTFO \!" True "Krankes Pack\!" False "\[name\] Du kleiner dummer Junge,hast keine Ahnung vom Leben\.Ich möchte dir so gerne eine neue saubere Windel überziehen\.Die hast du dringend nötig\!" True "Solange \[name\] keine nachprüfbare Dissertation vorlegen kann, ist er für mich der größte Scharlatan und Volksschädling aller Zeiten\. Es ist an der Zeit für ihn Gift oder Kugel zu wählen\. Der angerichtete Schaden für Wiedergutmachung ist zu groß\." True "War doch eh wieder ein Asylbewerber…vielleicht gings ja wieder um die Haushaltsführung??? Kann \[name\] und Konsorten wieder Hände schütteln gehen und alles auf die bösen Nazis schieben\. Die Armen brauchen nämlich Trost und Schutz\. Am besten nimmt die \[name\] und \[name\] und wie sie alle heißen gleich noch ein paar Nutten mit, natürlich zum Nulltarif… Die nordafrikanischen Testosteronbolzenschäumen sonst über und das wollen wir ja nicht…nicht das wir noch mit dem europäischen Gerichtshof für Menschenrechte zu tun bekommen…" False "also ich würde fahren und wenn das blut erstmal 5meter weit spritzt knochen und körper fliegen machen die anderen platz haben auf der bab nichts aber auch garnichts zu suchen \( eingriff in straßenverkehr \)" False "Über die Ossis damals gemault\! Die schwarzen willkommen heißen\! Ihr seid ja Bazis" False

## Appendix CDetailed Experiment Settings

Except for the API\-based inference calls, all models are loaded from the HuggingFace Hub and run with the Transformers library\([Wolf et al\., 2020](https://arxiv.org/html/2609.09791#bib.bib24)\)\. Inference calls are executed in batches for quicker computation\.*Thinking*mode is explicitly turned off for the models and*max\_tokens*is set to1010, because we expect only “True” or “False” labels in models’ generated answers\.

For the ablation using GPT\-5\.5,*max\_tokens*has to be changed because minimum*max\_tokens*when using the API is1616, which we set accordingly\.

Generated answers are parsed directly as True / False labels, not allowing for any additional text\. This works very well across all models; fewer than1%1\\%of answers cannot be parsed \(see Appendix[D\.3](https://arxiv.org/html/2609.09791#A4.SS3)\)\.

### C\.1Exploration Setup

When exploring demonstration sizesk∈\{4,8,16,32\}k\\in\\\{4,8,16,32\\\}, each model /kkcombination is explored twice: once with a*Title*,*dynamic*,*dense*,*similarity*prompting configuration and once with*Description*,*dynamic*,*random*configuration\.

### C\.2Knowledge Bases

For the*dense*embeddings, thecodefuse\-ai/F2LLM\-v2\-1\.7B\([Zhang et al\., 2026](https://arxiv.org/html/2609.09791#bib.bib25)\)embedding model is used, which ranked among the highest on the MTEB leaderboard for German tasks \(Fig\.[7](https://arxiv.org/html/2609.09791#A3.F7)\) during the first half of 2026\. Only larger models of the same family surpass this model, but for efficiency reasons we opt for the 1\.7\-billion model\.

![Refer to caption](https://arxiv.org/html/2609.09791v1/mteb_leaderboard.png)Figure 7:Screenshot of the MTEB Leaderboard for embedding models on German tasks, dated 28 June 2026\.When using both*similarity*\- and*MMR*\-based retrieval, these embeddings are used to build a LangChain vector store333[https://reference\.langchain\.com/python/langchain\-core/vectorstores/in\_memory/InMemoryVectorStore](https://reference.langchain.com/python/langchain-core/vectorstores/in_memory/InMemoryVectorStore), which is queried at inference time for selecting the best training samples based on the respective retrieval strategy\. LangChain’s vector store enables filtering when querying, making it easy to retrieve samples based on their class\.

For*diversity*\-based retrieval, the dense embeddings are clustered using sklearn’sK\-Meansalgorithm, whereK=\#​d​e​m​o​n​s​t​r​a​t​i​o​n​s2K=\\frac\{\\\#demonstrations\}\{2\}\. Clusters are built separately for positive and negative classes\. At inference time, a training sample is drawn at random from each cluster, resulting in a diverse set of demonstrations, balanced by class\.

The*sparsely*embedded knowledge base is built with a*BM25Retriever*module444[https://reference\.langchain\.com/python/langchain\-community/retrievers/bm25/BM25Retriever](https://reference.langchain.com/python/langchain-community/retrievers/bm25/BM25Retriever), a LangChain community build\. Because this retriever does not allow for filtered similarity search, a retriever is built for each class separately\. At inference time, the most similar training samples are retrieved for both classes\.

### C\.3Explicit Conditioning

The*Explicit*mode deconstructs the task into six subtasks\. These are prompted sequentially\. After each step, the model’s answer is parsed into aTrueorFalselabel\. If the decision at stepiiresults in a definite final label \(e\.g\. if there is no defamatory object present \(step11\), resulting in a finalFalselabel\), the decision scheme is halted and the final label is set\.

The additionally labelled dataset is created in the same fashion—resulting in single\-step annotations only for the subset of steps until a final decision is reached\. When using the*Explicit*mode in a few\-shot setting, this dataset is used for the demonstration pool\. Labels get rarer for later steps, and there is a strong class imbalance for some of the steps \(in total, there are only1010positive instances for*abusive insult*\)\. Especially for the clustering method of the*diversity*\-based retrieval, this can mean that not enough demonstrations for one label can be retrieved at inference time\. In the rare cases that this happens, demonstration*ratio*takes precedence over demonstration*size*: the demonstrations will be chosen so that the resulting set of few\-shot demonstrations is*balanced*, even if this means that fewer than88demonstrations are used\.

### C\.4Proprietary Model

The GPT\-5\.5 model is used with the dated snapshot versiongpt\-5\.5\-2026\-04\-23\.

### C\.5Fine\-Tuning

Gemma\-4 E4Bis loaded with 4\-bit NF4 quantisation and double quantisation, and LoRA adapters are trained on the attention and feed\-forward projection layers\. Training uses a maximum sequence length of 1024 tokens, three epochs, batch size 1 with 32 gradient accumulation steps, a learning rate of2×10−52\\times 10^\{\-5\}, and gradient checkpointing\. Prompt tokens are masked from the loss so that optimisation targets only the label completion\. Generation usesthinking\_mode: Falsewith a maximum budget of 10 tokens per sample\.

### C\.6Static Demonstrations

To select the demonstration set for the static few\-shot condition, we perform a random search over candidate sets drawn from the training data\. We sample3030class\-balanced candidate sets ofk=8k=8demonstrations each \(k/2k/2per class\) and evaluate every set by using it as few\-shot context for classifying a fixed evaluation subset of300300posts, sampled from the training data stratified by class label and kept disjoint from the demonstration pool\. Each candidate set is scored byF​1macroF1\_\{\\text\{macro\}\}over the model’s predictions; the best\-scoring set is retained\. The search is conducted separately for the two prompt templates*Title*and*Implicit*, so the selected demonstrations are optimised for the exact prompt they are later used with\. We only optimise sets of demonstrations for theGemma\-4 26Bmodel\.

## Appendix DDetailed Results

Full results of all runs of the main experiments are reported in Table[9](https://arxiv.org/html/2609.09791#A4.T9)\.

### D\.1Exploration Results

Table[3](https://arxiv.org/html/2609.09791#A4.T3)reports the meanF​1macroF1\_\{\\text\{macro\}\}scores per model, for all explored demonstration sizesk∈\{4,8,16,32\}k\\in\\\{4,8,16,32\\\}, along with the respective gain or loss inF​1macroF1\_\{\\text\{macro\}\}score after doubling the demonstration size\.

Table 3:MeanF​1macroF1\_\{\\text\{macro\}\}per model andkk\. Values in parentheses show the gain / loss vs\. the previouskk\. Largest gains are reported inbold\.
### D\.2RetICL Configurations

In the paper, we show that the*embedding*and*retrieval*modes have very little impact on model performance in comparison to*conditioning*mode and the difference between zero\- and few\-shot\.

The best configuration per model \(Table[1](https://arxiv.org/html/2609.09791#S5.T1)\) is*similarity*\-based for all models, based on*dense*embeddings for bothGemma\-4 26BandQwen3\.5 9Band based on*sparse*embeddings forGemma\-4 E4BandEuroLLM 22B\. However, on average \(Table[4](https://arxiv.org/html/2609.09791#A4.T4)\), a knowledge base made up of*dense*embeddings performs best for all models\. While*similarity*\-based retrieval is also best on average for both Gemma\-4 models andEuroLLM 22B,*diversity*\-based retrieval performs slightly better on average forQwen3\.5 9B\.

While*dense**similarity*\-based RetICL performs well on average, these settings should be explored for each*conditioning*mode so that the best overall configuration can be found\.

Table 4:F​1macroF1\_\{\\text\{macro\}\}by model and retrieval configuration \(k=8, mean over all prompt modes\)\.Bold: best retrieval config per model;italic: worst\.Δ\\Delta= best−\-worst\.
### D\.3Abstentions

Absolute and relative abstention rates are reported in Table[5](https://arxiv.org/html/2609.09791#A4.T5)\. Abstention is very low overall, with median00–11abstentions per run, which is less than1%1\\%of posts\. ForGemma\-4 26B,648648of the657657abstentions occur in a single configuration: under*Explicit*zero\-shot prompting, the model asks for further information to classify the first step \(*defamatory object*\) instead of replyingTrueorFalse\. Without context, many posts are ambiguous at this step, since personal pronouns—especially in the plural—cannot conclusively establish the existence of a defamatory object\. When demonstrations are supplied as user–assistant turns, the model is far more likely to commit to an answer despite the ambiguity\.

Table 5:Abstentions per model across all configurations \(total, mean and median per config and total rate\)\.
### D\.4Class Ratio and Ordering

Table 6:F​1macroF1\_\{\\text\{macro\}\}and true\-class precision \(PT\) and recall \(RT\) for every demonstration ratio×\\timesorder combination \(balance×\\timesorder ablation\)\.Bold: best value per model across all combinations\.Table 7:F​1macroF1\_\{\\text\{macro\}\}and true\-class precision \(PT\) and recall \(RT\) marginalised over each factor of the balance×\\timesorder ablation \(ratio levels averaged over order; order levels averaged over ratio\)\.Bold: best value per model within each factor block\.DetailedF​1macroF1\_\{\\text\{macro\}\}scores for all ratio–ordering combinations are reported in Table[6](https://arxiv.org/html/2609.09791#A4.T6)\. Per\-factor meanF​1macroF1\_\{\\text\{macro\}\}scores \(averaged over the other factor\) are reported in Table[7](https://arxiv.org/html/2609.09791#A4.T7)\. In terms of per\-factor meanF​1macroF1\_\{\\text\{macro\}\}, the pattern is uniform: a*proportional*class ratio outperforms*balanced*demonstrations for all four models, and*true\-first*ordering outperforms both*random*and*true\-last*for all four models, with*true\-last*consistently worst\. The best individual combination deviates from these main effects for two models—Qwen3\.5 9Bpeaks at*balanced*/*true\-first*\(0\.7040\.704\) andEuroLLM 22Bat*proportional*/*random*\(0\.6630\.663\)—reflecting the ratio–order interaction discussed in Section[5\.3](https://arxiv.org/html/2609.09791#S5.SS3)\(Fig\.[6](https://arxiv.org/html/2609.09791#S5.F6)\)\.

However, True\-class Precision and Recall in Table[7](https://arxiv.org/html/2609.09791#A4.T7)show another picture\. Recall peaks forbalancedandtrue\-lastdemonstration sets for most models, while Precision is best for*proportional*and*true\-first*demonstration sets across all models\. As we argue that Recall is the most important metric for this task \(while keeping an eye on Precision\),*balanced*,*true\-last*configuration for ratio / ordering of demonstrations comes out on top\.

### D\.5Shared Task Results

We submitted five runs to the DEF Subtask; the organisers scored them against the held\-out test labels \(Table[8](https://arxiv.org/html/2609.09791#A4.T8)\)\. Our best submission, the fine\-tunedGemma\-4 E4B \(FT\)\(MUCnoHARM2\), reached0\.740\.74, narrowly ahead of the promptedGemma\-4 26B\(MUCnoHARM1,0\.720\.72\)\. These held\-out scores closely track the cross\-validated results of Section[5](https://arxiv.org/html/2609.09791#S5)\(0\.741→0\.740\.741\\rightarrow 0\.74for the fine\-tune and0\.733→0\.720\.733\\rightarrow 0\.72forGemma\-4 26B\), indicating good transferability to the official test set\. For the sameGemma\-4 26Bconfiguration, the default*balanced*,*randomly*ordered demonstrations \(MUCnoHARM1\) slightly outperformed the*proportional*,*true\-first*variant \(MUCnoHARM5,0\.700\.70\), suggesting that the gains from tuning demonstration ratio and ordering did not transfer to the held\-out data\.

Table 8:Official shared taskF​1macroF1\_\{\\text\{macro\}\}on the held\-out DEF test set\. Demonstrations: Dense\-Similarity \(D\-Sim\.\), Random \(Rand\), Dense\-Diversity \(D\-Div\) retrieval configurations, balanced \(bal\.\) or proportional \(prop\.\) class ratio / random \(rand\.\), true\-first \(T\-first\) or true\-last \(T\-last\) ordering\. Best result inbold\.Table 9:Full results for all 112 configurations \+ GPT and fine\-tuned models, sorted byF​1macroF1\_\{\\text\{macro\}\}\(descending\)\. PT/ RT: Precision / Recall for the criminal class\. FS / ZS: few\-shot / zero\-shot\. D / S / F: dense / sparse / fusion embeddings\. Sim / Div / MMR / Rand: Similarity / Diversity / MMR / Random Retrieval\.ModelPromptRetrievalF1Acc\.PTRTGemma\-4 E4B \(FT\)ImplicitZS0\.7410\.9040\.7020\.431Gemma\-4 26BImplicitFS, D\-Sim0\.7330\.8500\.4450\.740Gemma\-4 26BImplicitFS, D\-MMR0\.7230\.8410\.4280\.742Gemma\-4 26BImplicitFS, D\-Div0\.7220\.8330\.4170\.788Gemma\-4 26BImplicitFS, F\-Sim0\.7190\.8360\.4190\.745Gemma\-4 26BImplicitFS, Rand0\.7170\.8290\.4100\.790Gemma\-4 26BDescriptionFS, D\-Sim0\.7160\.8300\.4110\.773GPT\-5\.5ExplicitFS, D\-Sim0\.7160\.8640\.4150\.747Gemma\-4 26BImplicitFS, S\-Sim0\.7140\.8300\.4090\.761Gemma\-4 26BExplicitFS, D\-MMR0\.7110\.8590\.4570\.564Gemma\-4 26BDescriptionFS, F\-Sim0\.7090\.8200\.3960\.793Gemma\-4 26BDescriptionFS, D\-MMR0\.7090\.8210\.3970\.785Gemma\-4 26BExplicitFS, F\-Sim0\.7010\.8490\.4300\.571Gemma\-4 26BExplicitFS, D\-Sim0\.6980\.8590\.4510\.506Gemma\-4 26BDescriptionFS, S\-Sim0\.6970\.8070\.3780\.800Gemma\-4 26BDescriptionFS, D\-Div0\.6960\.7990\.3730\.848Gemma\-4 26BTitleFS, D\-Div0\.6950\.8080\.3780\.783Gemma\-4 26BDescriptionFS, Rand0\.6950\.7990\.3720\.846Gemma\-4 26BTitleFS, D\-MMR0\.6930\.8110\.3770\.749Gemma\-4 26BTitleFS, Rand0\.6920\.8050\.3730\.781Gemma\-4 26BTitleFS, D\-Sim0\.6910\.8150\.3800\.713Gemma\-4 26BExplicitFS, D\-Div0\.6910\.8470\.4210\.532Gemma\-4 26BTitleFS, F\-Sim0\.6900\.8090\.3740\.747Gemma\-4 26BTitleFS, S\-Sim0\.6860\.8030\.3660\.757Gemma\-4 26BExplicitFS, Rand0\.6810\.8480\.4160\.492Gemma\-4 26BExplicitFS, S\-Sim0\.6760\.8360\.3920\.523Gemma\-4 26BImplicitZS0\.6760\.7910\.3530\.764Gemma\-4 26BDescriptionZS0\.6740\.7820\.3470\.807Gemma\-4 E4BExplicitFS, S\-Sim0\.6590\.7990\.3410\.622Gemma\-4 26BExplicitZS0\.6520\.7770\.3290\.683Gemma\-4 E4BExplicitFS, D\-Sim0\.6500\.7810\.3240\.663Gemma\-4 E4BExplicitFS, Rand0\.6460\.7980\.3280\.559Gemma\-4 E4BExplicitFS, F\-Sim0\.6440\.7770\.3170\.653Gemma\-4 E4BExplicitFS, D\-MMR0\.6440\.7620\.3130\.733Gemma\-4 E4BExplicitFS, D\-Div0\.6400\.7920\.3190\.559Qwen3\.5 9BExplicitFS, D\-Sim0\.6360\.8210\.3380\.426Gemma\-4 26BTitleZS0\.6300\.7580\.2990\.670Qwen3\.5 9BImplicitFS, Rand0\.6300\.7400\.2980\.769Qwen3\.5 9BExplicitFS, D\-MMR0\.6280\.8180\.3260\.407Qwen3\.5 9BExplicitFS, F\-Sim0\.6270\.8180\.3260\.402Qwen3\.5 9BDescriptionFS, D\-Div0\.6260\.7520\.2940\.677Qwen3\.5 9BDescriptionFS, Rand0\.6260\.7490\.2930\.689Gemma\-4 E4BDescriptionFS, D\-Div0\.6120\.7150\.2800\.788Qwen3\.5 9BImplicitFS, D\-Div0\.6100\.7220\.2770\.733Gemma\-4 E4BDescriptionFS, Rand0\.6090\.7070\.2770\.807Qwen3\.5 9BExplicitFS, S\-Sim0\.6060\.8390\.3360\.275Gemma\-4 E4BTitleFS, D\-Div0\.6000\.7060\.2670\.747Gemma\-4 E4BTitleFS, D\-Sim0\.6000\.7060\.2670\.749Qwen3\.5 9BExplicitZS0\.5980\.8110\.2860\.325Gemma\-4 E4BDescriptionFS, D\-Sim0\.5980\.6970\.2660\.788Qwen3\.5 9BImplicitFS, F\-Sim0\.5970\.7050\.2640\.737Qwen3\.5 9BImplicitFS, D\-MMR0\.5970\.7020\.2650\.754Qwen3\.5 9BImplicitFS, S\-Sim0\.5970\.7080\.2630\.723Gemma\-4 E4BDescriptionZS0\.5970\.7060\.2630\.733Qwen3\.5 9BImplicitFS, D\-Sim0\.5970\.7090\.2630\.711Gemma\-4 E4BTitleFS, F\-Sim0\.5970\.6950\.2650\.793Gemma\-4 E4BDescriptionFS, F\-Sim0\.5940\.6850\.2650\.831Qwen3\.5 9BDescriptionFS, S\-Sim0\.5930\.7150\.2580\.660Gemma\-4 E4BExplicitZS0\.5880\.7950\.2640\.340Gemma\-4 E4BTitleFS, S\-Sim0\.5820\.6860\.2500\.735Gemma\-4 E4BImplicitFS, D\-Sim0\.5820\.6740\.2530\.800Qwen3\.5 9BDescriptionFS, F\-Sim0\.5800\.6920\.2460\.692Qwen3\.5 9BTitleFS, D\-Div0\.5780\.6890\.2450\.692Gemma\-4 E4BTitleFS, D\-MMR0\.5770\.6670\.2510\.815Qwen3\.5 9BDescriptionFS, D\-Sim0\.5760\.6830\.2440\.711Gemma\-4 E4BDescriptionFS, S\-Sim0\.5750\.6670\.2480\.795Gemma\-4 E4BTitleFS, Rand0\.5740\.6800\.2420\.711Qwen3\.5 9BDescriptionFS, D\-MMR0\.5730\.6750\.2430\.733Gemma\-4 E4BImplicitFS, Rand0\.5720\.6630\.2450\.793Gemma\-4 E4BImplicitFS, D\-Div0\.5720\.6630\.2450\.793Qwen3\.5 9BExplicitFS, Rand0\.5700\.8530\.3400\.164Qwen3\.5 9BExplicitFS, D\-Div0\.5680\.8490\.3210\.166Gemma\-4 E4BDescriptionFS, D\-MMR0\.5670\.6460\.2450\.860Gemma\-4 E4BImplicitFS, S\-Sim0\.5610\.6460\.2380\.815Qwen3\.5 9BImplicitZS0\.5580\.8730\.5000\.113Gemma\-4 E4BImplicitFS, F\-Sim0\.5570\.6390\.2350\.819Qwen3\.5 9BTitleFS, Rand0\.5530\.6680\.2200\.634Gemma\-4 E4BImplicitFS, D\-MMR0\.5480\.6220\.2320\.855Qwen3\.5 9BDescriptionZS0\.5410\.6600\.2070\.593Qwen3\.5 9BTitleFS, S\-Sim0\.5380\.6330\.2140\.704Gemma\-4 E4BTitleZS0\.5250\.6060\.2100\.759Qwen3\.5 9BTitleFS, F\-Sim0\.5110\.5860\.2020\.764Qwen3\.5 9BTitleZS0\.5090\.6140\.1840\.595Qwen3\.5 9BTitleFS, D\-Sim0\.5070\.5800\.2010\.771Qwen3\.5 9BTitleFS, D\-MMR0\.5070\.5760\.2020\.790EuroLLM 22BExplicitFS, S\-Sim0\.4740\.5390\.1800\.740EuroLLM 22BExplicitFS, D\-Sim0\.4730\.5230\.1900\.843EuroLLM 22BExplicitFS, F\-Sim0\.4640\.5170\.1820\.798EuroLLM 22BImplicitFS, D\-Sim0\.4470\.4850\.1820\.872GPT\-5\.5TitleZS0\.4400\.4640\.1921\.000EuroLLM 22BExplicitFS, D\-MMR0\.4380\.4760\.1760\.851EuroLLM 22BDescriptionFS, D\-Sim0\.4360\.4740\.1750\.843EuroLLM 22BExplicitFS, Rand0\.4340\.4790\.1660\.769EuroLLM 22BImplicitFS, F\-Sim0\.4330\.4670\.1770\.875EuroLLM 22BDescriptionFS, S\-Sim0\.4280\.4650\.1700\.829EuroLLM 22BImplicitFS, S\-Sim0\.4270\.4590\.1740\.870EuroLLM 22BDescriptionFS, F\-Sim0\.4250\.4590\.1720\.848Gemma\-4 E4BImplicitZS0\.4200\.4480\.1740\.892EuroLLM 22BExplicitFS, D\-Div0\.4170\.4530\.1640\.805EuroLLM 22BImplicitFS, D\-MMR0\.4160\.4430\.1730\.896EuroLLM 22BDescriptionFS, Rand0\.4040\.4310\.1660\.863EuroLLM 22BDescriptionFS, D\-MMR0\.3990\.4220\.1660\.884EuroLLM 22BTitleFS, D\-Sim0\.3950\.4180\.1650\.879EuroLLM 22BTitleFS, S\-Sim0\.3920\.4130\.1650\.889EuroLLM 22BDescriptionFS, D\-Div0\.3890\.4130\.1580\.834EuroLLM 22BTitleFS, F\-Sim0\.3850\.4030\.1650\.908EuroLLM 22BImplicitFS, D\-Div0\.3670\.3820\.1590\.899EuroLLM 22BTitleFS, D\-Div0\.3620\.3790\.1510\.841EuroLLM 22BTitleFS, Rand0\.3580\.3760\.1490\.829EuroLLM 22BExplicitZS0\.3580\.3660\.1660\.990EuroLLM 22BImplicitFS, Rand0\.3560\.3700\.1530\.872EuroLLM 22BTitleFS, D\-MMR0\.3520\.3630\.1570\.918EuroLLM 22BDescriptionZS0\.2220\.2230\.1370\.969EuroLLM 22BImplicitZS0\.2110\.2120\.1350\.964EuroLLM 22BTitleZS0\.1690\.1740\.1330\.990