Team DArgk 在 2026 ELOQUENT 实验室评估生成语言模型质量:人性的残留——通过GRPO微调逃避AI检测

arXiv cs.CL 论文

摘要

本文介绍了SHADE,一个使用GRPO微调的强化学习框架,旨在逃避AI文本检测器,在对抗替代检测器时实现了高逃避率,但对未见过的评估分类器的迁移效果有限。

arXiv:2609.22221v1 Announce Type: new Abstract: Large language models (LLMs) can generate fluent and coherent text that is increasingly difficult to distinguish from human writing, motivating the development of automatic AI-generated text detectors. However, the robustness of such detectors under adversarial generation remains uncertain. This paper presents SHADE (Stochastic Human-like generation via Adversarial Detector Evasion), a reinforcement learning framework that formulates detector evasion as a policy optimization problem. Instead of applying post-hoc perturbations or prompting-based rewriting, SHADE fine-tunes an instruction-tuned LLaMA model with Group Relative Policy Optimization (GRPO), using feedback from a surrogate detector based on the PAN 2025 mdok system. Our experiments show that full fine-tuning with a small KL regularization penalty achieves $98.5\%$ surrogate evasion, compared to $1.5\%$ for the base model, while LoRA-based adaptation is substantially less effective under regularization. Linguistic analysis reveals that successful evasion is associated with shorter, simpler, and less lexically diverse outputs, suggesting that high detector evasion does not necessarily correspond to more human-like writing. In the official Voight-Kampff competition setting, our submissions ranked sixth and seventh, indicating that optimization against a single surrogate detector only partially transfers to unseen evaluation classifiers. These results highlight both the potential and limitations of reinforcement learning for adversarial AI-text generation and motivate more robust, multi-detector evaluation protocols for AI-generated text detection.
查看原文
查看缓存全文

缓存时间: 2026/09/22 09:09

# Notebook for the Voight-Kampff Task on Eloquent Lab 2026 at CLEF 2026
Source: [https://arxiv.org/html/2609.22221](https://arxiv.org/html/2609.22221)
\*11

\*11

###### Abstract

Large language models \(LLMs\) can generate fluent and coherent text that is increasingly difficult to distinguish from human writing, motivating the development of automatic AI\-generated text detectors\. However, the robustness of such detectors under adversarial generation remains uncertain\. This paper presentsSHADE\(Stochastic Human\-like generation via Adversarial Detector Evasion\), a reinforcement learning framework that formulates detector evasion as a policy optimization problem\. Instead of applying post\-hoc perturbations or prompting\-based rewriting,SHADEfine\-tunes an instruction\-tuned LLaMA model with Group Relative Policy Optimization \(GRPO\), using feedback from a surrogate detector based on the PAN 2025mdoksystem\. Our experiments show that full fine\-tuning with a small KL regularization penalty achieves98\.5%98\.5\\%surrogate evasion, compared to1\.5%1\.5\\%for the base model, while LoRA\-based adaptation is substantially less effective under regularization\. Linguistic analysis reveals that successful evasion is associated with shorter, simpler, and less lexically diverse outputs, suggesting that high detector evasion does not necessarily correspond to more human\-like writing\. In the official Voight\-Kampff competition setting, our submissions ranked sixth and seventh, indicating that optimization against a single surrogate detector only partially transfers to unseen evaluation classifiers\. These results highlight both the potential and limitations of reinforcement learning for adversarial AI\-text generation and motivate more robust, multi\-detector evaluation protocols for AI\-generated text detection\.

###### keywords

AI\-generated text detection ,adversarial generation ,reinforcement learning ,large language models ,surrogate detectors

††copyrightyear:2026††copyright:Copyright for this paper by its authors\. Use permitted under Creative Commons License Attribution 4\.0 International \(CC BY 4\.0\)\.††venue:CLEF 2026: Conference and Labs of the Evaluation Forum, September 21–24, 2026, Jena, Germany††email:antonela\.tommasel@isistan\.unicen\.edu\.ar, antonela\.tommasel@jku\.at††address:Johannes Kepler University, Linz, Austria††address:CONICET \- UNICEN, Tandil, Buenos Aires, Argentina††email:jmro@cs\.aau\.dk††address:Department of Computer Science, Aalborg University, Aalborg, Denmark††corresp:Corresponding author\.††These authors contributed equally\.## 1Introduction

The rapid advances in Large Language Models \(LLMs\), which are now capable of producing fluent, coherent, and human\-like text across a wide range of domains and writing styles, have made it increasingly difficult to determine whether a text is written by a human or generated by an AI system[Yang et al\. \(2025b\)](https://arxiv.org/html/2609.22221#bib.bib22)\. This distinction is becoming important in settings where authorship, originality, accountability and trust are central, including education, journalism, scientific communication, online moderation and misinformation detection[Wu et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib14)\. As LLM\-generated content becomes easier to produce, adapt and distribute at scale, automatic AI\-generated text detection has emerged as a key mechanism for identifying synthetic text and supporting decisions about content provenance\. A growing body of work has therefore proposed detection methods based on perplexity, stylometric features, watermarking signals or fine\-tuned transformers[Hans et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib17);[Zhou et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib16);[Koike et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib13);[Wu et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib14)\. However, the reliability of these detectors remains uncertain, particularly when generated texts are intentionally modified to appear less machine\-generated\. Their robustness under adversarial conditions is thus still an open question[Wang et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib15);[Koike et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib13)\.

Motivated by this challenge, the Voight\-Kampff task[Gunti et al\. \(2026\)](https://arxiv.org/html/2609.22221#bib.bib18)at the ELOQUENT 2026 lab[Karlgren et al\. \(2026\)](https://arxiv.org/html/2609.22221#bib.bib19)asks participants to generate text that evades unseen AI detectors submitted by participants of PAN 2026\. In the previous edition of the Voight\-Kampff task, the strongest approaches relied on text perturbation and translation\-based transformations, whereas persona\-based and stylistic rewriting strategies \(i\.e\., prompting\-based methods\) achieved weaker results[Creo et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib2);[Vachharajani \(2025\)](https://arxiv.org/html/2609.22221#bib.bib3)\. Data\-centric approaches have also explored fine\-tuning with LoRA on human\-written corpora to better mimic the distribution of human texts in the training data[Gunti \(2025\)](https://arxiv.org/html/2609.22221#bib.bib4)\. However, these methods largely treat evasion as either a post\-processing problem or a static alignment problem, rather than as a dynamic optimization objective guided by detection feedback\.

In this paper, we proposeSHADE\(Stochastic Human\-like generation via Adversarial Detector Evasion\), a framework that formulates detector evasion as a reinforcement learning problem\. Instead of manipulating outputs post\-hoc or curating training data to approximate human style,SHADEdirectly optimizes a language model’s generation policy to reduce the confidence of an adversarial surrogate detector\. Concretely, we fine\-tune LLaMA 3\.2 3B instruct using Group Relative Policy Optimization \(GRPO\), with rewards derived from a surrogate detector based on last year’s Voight\-Kampff task, namelymdok[Macko \(2025\)](https://arxiv.org/html/2609.22221#bib.bib6)\. This adversarial formulation encourages the model to internalize evasion\-oriented generation strategies, rather than relying exclusively on surface\-level perturbations or prompting\-based transformations\.

The contributions of this paper are threefold: \(i\) we formulate AI\-generated text detector evasion as a reinforcement learning problem driven by adversarial detector feedback; \(ii\) we introduceSHADE, a GRPO\-based fine\-tuning framework that optimizes an LLM against a surrogate from the previous Voight\-Kampff task; and \(iii\) we empirically analyze the relationship between detector evasion and textual properties, showing that high evasion rates can emerge through linguistic simplification rather than through more elaborate human\-style generation\.

Our experiments show that full fine\-tuning with a small KL regularization penalty achieves a98\.5%98\.5\\%evasion rate against the surrogate detector, compared to only1\.5%1\.5\\%for the base LLM\. To better understand this behavior, we analyze the trade\-offs between evasion and textual characteristics such as repetition, lexical richness, and readability\. This analysis reveals that the model primarily evades detection by simplifying its language, suggesting that successful detector evasion does not necessarily correspond to richer or more human\-like writing\. We further show that parameter efficient tuning with LoRA is substantially less effective in this adversarial reinforcement learning setting, motivating full model adaptation for the proposed task\.

The remainder of this paper is organized as follows\.[Section2](https://arxiv.org/html/2609.22221#S2)reviews related work on AI detection evasion\.[Section3](https://arxiv.org/html/2609.22221#S3)formalizes the problem and detailsSHADE, including its training objective, surrogate detector, and inference procedure\.[Section4](https://arxiv.org/html/2609.22221#S4)analyzes the fine\-tuning configurations and the linguistic properties of the generated texts\.[Section5](https://arxiv.org/html/2609.22221#S5)reports results on the official Voight\-Kampff task evaluation\.[Section6](https://arxiv.org/html/2609.22221#S6)concludes the paper and[Section7](https://arxiv.org/html/2609.22221#S7)links the artifacts related to this work\.

## 2Related work

Research on AI\-generated text detection and evasion lies at the intersection of two closely related questions: how such detectors can be attacked, and which evasion strategies have proven effective in shared\-task settings\. We therefore first discussAI\-generated text detection, focusing on statistical, supervised and watermarking\-based approaches for distinguishing machine\-generated from human\-written text\. We then reviewattacks on AI\-generated text detectors, distinguishing input\-level transformations from model\-level adaptations\. Finally, we discussevasion approaches in the Voight\-Kampff task 2025, which provides the most direct empirical context for our contribution\. This structure allows us to positionSHADEwith respect to both the broader literature and the specific shared\-task setting\.

#### AI\-generated Text Detection

Recent surveys characterize AI\-generated text detection as a rapidly evolving field that spans several methodological families\.[Yang et al\. \(2025b\)](https://arxiv.org/html/2609.22221#bib.bib22)organize existing approaches around three main challenges: improving classifier training, exploiting intrinsic attributes of language models and embedding information such as watermarks into generated text\. They further distinguish black\-box detectors, which rely only on input–output behavior and observable textual characteristics, from white\-box detectors, which assume access to internal model information such as architecture, parameters, logits, or generation procedures\. This distinction is particularly relevant for evasion\-oriented settings, where the attacker may only have access to detector outputs or surrogate feedback rather than to the target detector itself\. This broader taxonomy also helps explain why detector evasion remains difficult to address with a single class of defenses\. Training\-based detectors depend on the representativeness of labeled data, intrinsic\-attribute methods often rely on access to model probabilities or representations, and watermarking approaches require some form of information embedding during generation\. As a result, detection performance can vary substantially depending on model access, computational assumptions, domains, and attack conditions\.

The task of distinguishing machine\-generated text from human\-written text has been mainly approached from two broad directions: zero\-shot statistical methods and supervised classification\. Statistical detectors exploit regularities in the probability distribution of LLM\-generated text, such as the tendency of generated text to exhibit lower entropy or higher likelihood under a reference model\. Binoculars[Hans et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib17), for instance, performs classification by comparing perplexity\-based scores from two language models that share a tokenizer, enabling zero\-shot detection without labeled training data\. AdaDetectGPT[Zhou et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib16)extends perturbation\-based detection with statistical guarantees, using adaptive thresholds to control false\-positive rates across domains\. Similarly,[Seeliger et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib20)proposed a statistically grounded approach for PAN 2025[Bevendorff et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib7)\. Their method first computes world\-level correlations from a binary\-term document matrix and the associated human/AI labels, then maps each text into a sequence of correlation values interpreted as a signal\. These correlation signals, combined with hand\-crafted statistical features, provide a lightweight and interpretable alternative to detectors based on LLM backbones\.

A second line of work frames AI\-generated text detection as a supervised binary classification problem[Wu et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib14);[Koike et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib13);[Macko \(2025\)](https://arxiv.org/html/2609.22221#bib.bib6)\. DetectRL[Wu et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib14)provides a benchmark for evaluating such detectors under more realistic conditions, including paraphrase attacks and domain shifts, and shows that many classifiers degrade substantially when exposed to adversarially modified inputs\. OUTFOX[Koike et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib13)addresses this limitation by augmenting the training set with adversarially generated essays, i\.e\., texts explicitly generated to challenge the detector\. In the PAN 2025 setting, mdok[Macko \(2025\)](https://arxiv.org/html/2609.22221#bib.bib6)ranked among the top\-performing systems, combining an LLM backbone with robust fine\-tuning to achieve strong performance in both binary and multiclass detection\. The continued development of detection benchmarks at PAN 2026[Bevendorff et al\. \(2026\)](https://arxiv.org/html/2609.22221#bib.bib8)reflects the rapid evolution of the field\. However, it also highlights an unresolved problem\. As detectors become stronger, evasion methods may need to become adaptive rather than relying on fixed transformations or manually designed prompts\.

#### Attacks on AI\-generated Text Detectors

The problem of evading AI\-generated text detectors can be situated withing the broader paradigm of adversarial machine learning[Lowd and Meek \(2005\)](https://arxiv.org/html/2609.22221#bib.bib5), where an attacker modifies inputs of model behavior to induce misclassification\. In the context of text detection, a common strategy is to apply input\-level transformations to generated text\. These transformations include character\-level perturbations, synonym substitutions, paraphrasing and translation\-based rewriting, with the goal of moving the text away from the distribution recognized by the detector[Creo et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib2)\. Such methods can be effective because they alter superficial or statistical cues used by detectors\. However, they may also introduce artifacts, reduce readability or produce unnatural phrasing\. A recent survey[Yang et al\. \(2025b\)](https://arxiv.org/html/2609.22221#bib.bib22)similarly identifies adversarially generated text as an emerging challenge for AI\-generated text detection, noting that such texts can deceive existing detectors and increase false\-negative rates\. This supports the need for evaluation settings in which detectors are tested not only on standard generated text, but also against adaptive or adversarial generation strategies\.

A complementary strategy adapts the source model itself\. Instead of modifying generated outputs after generation, model\-level approaches fine\-tune the generator so that its output distribution shifts toward texts that are less likely to be classified as machine\-generated\.[Wang et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib15)propose HUMPA, a proxy\-attack strategy that fine\-tunes a small surrogate model using Direct Preference Optimization \(DPO\)[Rafailov et al\. \(2023\)](https://arxiv.org/html/2609.22221#bib.bib21)\. Their approach reduces detector performance substantially, achieving an average AUROC drop of70\.4%70\.4\\%across multiple datasets\. Importantly, this gain comes with a trade\-off\. more aggressive fine\-tuning can reduce generation quality, making the regularization parameterβ\\betacentral for balancing evasion performance and text quality[Wang et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib15)\. This finding is particular relevant to our work, asSHADEalso optimizes a generator against detector feedback, but does so through reinforcement learning rather than preference optimization\.

#### Evasion Approaches at Voight\-Kampff Task 2025

The 2025 edition of the Voight\-Kampff task[Bevendorff et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib7)produced a range of evasion strategies that illustrate both the promise and the limitations of existing approaches\.[Creo et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib2)evaluated four methods: Text Perturbation, Lost in Translation, Persona Immersion, and ADHD Writing\. Text Perturbation applies character\-level modifications to obscure the generated text\. Lost in Translation translates the text into another language and then back into the original language, for example from English to Spanish and back to English, with the goal of shifting the distribution of the generated text\. Persona Immersion and ADHD Writing, in contrast, are prompting\-based methods that ask the LLM to imitate human writing either directly or through a specific cognitive or stylistic profile\. The authors found that Lost in Translation achieved the best overall evasion performance, followed by Text Perturbation and ADHD Writing\. These results suggest that translation\-based rewriting can be more effective than direct prompt\-based human imitation, likely because translation changes lexical and syntactic patterns more substantially than style instructions alone\. At the same time, these methods remain post\-processing strategies: they transform a generated text after it has been produced, rather than modifying the generation process itself\.

[Vachharajani \(2025\)](https://arxiv.org/html/2609.22221#bib.bib3)explored a related translation\-based strategy using Hindi\-to\-Spanish re\-translation\. Although this approach is conceptually similar to Lost in Translation, the resulting texts exhibited awkward phrasing, repetition, grammatical errors, inconsistent idioms, and uneven sentence structure\. These results suggest that translation\-based evasion may succeed partly by introducing distributional shifts that detectors fail to capture, but that these shifts can come at the cost of fluency and textual quality\. Finally,[Gunti \(2025\)](https://arxiv.org/html/2609.22221#bib.bib4)proposed fine\-tuning LLaMA 3 with LoRA on cleaned human\-written corpora aiming to shift the model distribution toward human\-written text\. This data\-centric strategy can preserve output quality because the model is trained on human\-authored examples\. However, it treats evasion as a static alignment problem, the model is optimized to resemble a human\-text corpus rather than to adapt directly to detector feedback\. Moreover, as it relies on the original PAN dataset as part of the training data, its effectiveness may depend on the match between the training distribution and the target evaluation setting\.

In contrast,SHADEframes detector evasion as a dynamic optimization problem\. Like model\-level adaptation approaches, it fine\-tunes the generator rather than perturbing outputs post hoc\. However, unlike LoRA\-based alignment to human corpora[Gunti \(2025\)](https://arxiv.org/html/2609.22221#bib.bib4),SHADEdoes not require a corpus of human\-written texts\. Its objective is not to imitate existing human writing directly, but to optimize generation behavior against an adversarial surrogate detector\. Moreover, unlike DPO\-based proxy attacks[Wang et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib15), which optimize a generator offline using a fixed set of paired preferences,SHADEuses online reinforcement learning with detector confidence as a per\-sample reward signal\. Specifically, we adopt GRPO[Shao et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib1)and define the reward as the complement of the confidence score assigned by a static, pretrained adversarial detector\. This placesSHADEwithin the broader paradigm of Reinforcement Learning from AI Feedback, but in a setting where the feedback is used to optimize detector evasion\. In this sense, our contribution is to apply feedback\-driven policy optimization to a problem that has more commonly been addressed through post\-hoc transformations or static distributional alignment\.

## 3SHADE:Stochastic Human\-like generation via Adversarial Detector Evasion

The goal of the Voight\-Kampff task is to evaluate whether text generated by a language model can be made difficult to distinguish from human\-written text\. In the official evaluation setting, generated texts are assessed by unseen AI\-detection classifiers\. Since participants do not have access to these target detectors, the task can be viewed as a black\-box evasion problem in which the generator must produce prompt\-consistent texts that are unlikely to be classified as AI\-generated\.

### 3\.1Problem definition

Let𝒫\\mathcal\{P\}denote the prompt space and𝒴\\mathcal\{Y\}the space of possible text outputs\. We define a parameterized language model generatorπθ:𝒫→𝒴\\pi\_\{\\theta\}:\\mathcal\{P\}\\rightarrow\\mathcal\{Y\}, which induces a distribution over outputsy∈𝒴y\\in\\mathcal\{Y\}for a given promptp∈𝒫p\\in\\mathcal\{P\}\. LetDϕ:𝒴→\[0,1\]D\_\{\\phi\}:\\mathcal\{Y\}\\rightarrow\[0,1\]be an unknown AI\-detection classifier, whereDϕ​\(y\)D\_\{\\phi\}\(y\)denotes the probability that textyyis classified as AI\-generated\. The objective is to learn parametersθ\\thetasuch that the generated texts satisfy the semantic and stylistic constraints of the input prompt while minimizing the detector confidence\. Formally, we define the evasion objective as:

θ∗=arg⁡maxθ​𝔼p∼𝒫​\[𝔼y∼p​iθ​\(p\)​\[1−Dϕ​\(y\)\]\]\\theta^\{\*\}=\\arg\\max\_\{\\theta\}\\mathbb\{E\}\_\{p\\sim\\mathcal\{P\}\}\\left\[\\mathbb\{E\}\_\{y\\sim pi\_\{\\theta\}\(p\)\}\\left\[1\-D\_\{\\phi\}\(y\)\\right\]\\right\]\(1\)
SinceDϕD\_\{\\phi\}is not accessible during training, we approximate this objective using a surrogate detectorD^ϕ\\hat\{D\}\_\{\\phi\}\. The resulting optimization problem therefore seeks to adapt the generation policy so that its outputs receive low AI\-generated probability scores under the surrogate, while remaining close enough to the original model to preserve generation quality\.

### 3\.2Solution

We proposeSHADE,Stochastic Human\-like generation via Adversarial Detector Evasion, a framework that combines adversarial learning[Lowd and Meek \(2005\)](https://arxiv.org/html/2609.22221#bib.bib5)and reinforcement learning to adapt a base language model for detector evasion\. The central idea is to modify the weights of the base model so that evasion\-oriented generation behavior is internalized by the model, rather than applied through post\-hoc perturbations, translation, or manually designed prompts\. To train the model, we use the datasets from the Voight\-Kampff tasks of Eloquent 2024, 2025, and 2026, together with the prompt instructions described in[Section3\.3](https://arxiv.org/html/2609.22221#S3.SS3)\.

As the official AI\-detection classifiersDϕD\_\{\\phi\}are not available, we train against a surrogate detector\. In particular, we use a modified version ofmdok[Macko \(2025\)](https://arxiv.org/html/2609.22221#bib.bib6), one of the best\-performing classifiers in PAN 2025\. While the originalmdoksystem usesQwen3\-14BBase[Yang et al\. \(2025a\)](https://arxiv.org/html/2609.22221#bib.bib9), our surrogate replaces this backbone withFacebookAI/roberta\-base[Liu et al\. \(2019\)](https://arxiv.org/html/2609.22221#bib.bib10)due to GPU memory constraints\. We refer to this surrogate asmdok\-roberta\. To approximate the behavior of the original mdok detector, we trainedmdok\-robertaon the PAN 2025 dataset\. On the PAN 2025 validation set,mdok\-robertaachieved an F1 score of99%99\\%\. We therefore use it as an adversarial oracle, assuming that detectors used in PAN 2026[Bevendorff et al\. \(2026\)](https://arxiv.org/html/2609.22221#bib.bib8)will exploit similar distributional regularities\.

Given this surrogate detector, we fine\-tuneLLaMA 3\.2 3Bin its instruction\-tuned version111[https://huggingface\.co/meta\-llama/Llama\-3\.2\-3B\-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct)[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib11)\. We use Group Relative Policy Optimization \(GRPO\)[Shao et al\. \(2024\)](https://arxiv.org/html/2609.22221#bib.bib1)as the optimization algorithm\. In this setting,πθ\\pi\_\{\\theta\}denotes the model being optimized,πref\\pi\_\{\\text\{ref\}\}denotes the original base model before reinforcement learning, andD^ϕ​\(y\)\\hat\{D\}\_\{\\phi\}\(y\)denotes the surrogate detector score\. SinceD^ϕ​\(y\)\\hat\{D\}\_\{\\phi\}\(y\)estimates the probability thatyyis AI\-generated, we define the reward asr⁡\(y\)=1−D^ϕ​\(y\)r\(y\)=1\-\\hat\{D\}\_\{\\phi\}\(y\)\. Thus, outputs that receive lower AI\-generated probability scores obtain higher rewards\.

Given a training batch, letri=r⁡\(yi\)r\_\{i\}=r\(y\_\{i\}\)be the reward assigned to theii\-th generated text, and letmean​\(𝐫\)\\text\{mean\}\(\\mathbf\{r\}\)andstd​\(𝐫\)\\text\{std\}\(\\mathbf\{r\}\)denote the mean and the standard deviation of rewards across the current batch\. Following the GRPO formulation, the advantage assigned to each tokenttin theii\-th outputs is computed by normalizing the reward within the current group:

A^i,t=ri−mean​\(𝐫\)std​\(𝐫\)\\hat\{A\}\_\{i,t\}=\\frac\{r\_\{i\}\-\\text\{mean\}\(\\mathbf\{r\}\)\}\{\\text\{std\}\(\\mathbf\{r\}\)\}\(2\)
Then, the loss function is defined as:

ℒGRPO\(θ\)=−1∑i=1G\|𝒪i\|∑i=1G∑t=1\|𝒪i\|\[πθ​\(oi,t∣q,oi,<t\)\[πθ​\(oi,t∣q,oi,<t\)\]oldA^i,t−β𝔻KL\[πθ∥πref\]\]\\mathcal\{L\}\_\{\\mathrm\{GRPO\}\}\(\\theta\)=\-\\frac\{1\}\{\\sum\_\{i=1\}^\{G\}\|\\mathcal\{O\}\_\{i\}\|\}\\sum\_\{i=1\}^\{G\}\\sum\_\{t=1\}^\{\|\\mathcal\{O\}\_\{i\}\|\}\\left\[\\frac\{\\pi\_\{\\theta\}\(o\_\{i,t\}\\mid q,o\_\{i,<t\}\)\}\{\\left\[\\pi\_\{\\theta\}\(o\_\{i,t\}\\mid q,o\_\{i,<t\}\)\\right\]\_\{\\mathrm\{old\}\}\}\\hat\{A\}\_\{i,t\}\-\\beta\\mathbb\{D\}\_\{\\text\{KL\}\}\\left\[\\pi\_\{\\theta\}\\,\\\|\\,\\pi\_\{\\mathrm\{ref\}\}\\right\]\\right\]\(3\)
This formulation follows the implementation provided by theHugging Face TRLlibrary222[https://huggingface\.co/docs/trl/grpo\_trainer](https://huggingface.co/docs/trl/grpo_trainer), which slightly differs from the original GRPO objective\. For the KL\-divergence, we used the same unbiased estimator as in the original implementation333[http://joschu\.net/blog/kl\-approx\.html](http://joschu.net/blog/kl-approx.html):

𝔻KL\[πθ∥πref\]=πref​\(oi,t∣q,oi,<t\)πθ​\(oi,t∣q,oi,<t\)−log\(πref​\(oi,t∣q,oi,<t\)πθ​\(oi,t∣q,oi,<t\)\)−1\\mathbb\{D\}\_\{\\text\{KL\}\}\\left\[\\pi\_\{\\theta\}\\\|\\pi\_\{\\text\{ref\}\}\\right\]=\\frac\{\\pi\_\{\\text\{ref\}\}\(o\_\{i,t\}\\mid q,o\_\{i,<t\}\)\}\{\\pi\_\{\\theta\}\(o\_\{i,t\}\\mid q,o\_\{i,<t\}\)\}\-\\log\(\\frac\{\\pi\_\{\\text\{ref\}\}\(o\_\{i,t\}\\mid q,o\_\{i,<t\}\)\}\{\\pi\_\{\\theta\}\(o\_\{i,t\}\\mid q,o\_\{i,<t\}\)\}\)\-1\(4\)
Although the KL\-divergence term is usually omitted when training models using GRPO[Hu et al\. \(2025\)](https://arxiv.org/html/2609.22221#bib.bib12), we found that using a smallβ\\betaimproves the textual properties of the generated outputs\. We analyze this effect in[Section4](https://arxiv.org/html/2609.22221#S4)\.

### 3\.3Prompt modification

For each input instance, the Voight\-Kampff task provides a base prompt that specifies the text\-generation request\. We adapt this task\-provided prompt to the chat format expected by the instruction\-tunedLLaMA 3\.2 3Bmodel\. Specifically, we wrap the original prompt in a chat template and add system\-level instructions intended to exploit the model’s instruction following capabilities444[https://huggingface\.co/docs/transformers/en/chat\_templating](https://huggingface.co/docs/transformers/en/chat_templating):

```
[
        {"role": "system",
         "content":  "You are a helpful assistant that generates helpful
                      answers. You will avoid pleasantries and small talk,
                      focusing on the task at hand."},

        {"role": "system",
         "content": "You will avoid short paragraphs and bullet points."},

        {"role": "user",
         "content": BASE_PROMPT + "\n\nThis is the text"},

        {"role": "assistant",
         "content": ""},
    ]
```

This formatting allows the prompt to match the conversational structure expected by the instruction\-tuned model while preserving the original task instructions\.

### 3\.4Final text generation

At inference time, text generation is performed using the chat\-based template described in[Section3\.3](https://arxiv.org/html/2609.22221#S3.SS3)\. We use stochastic decoding to introduce controlled variation across candidate outputs\. Sampling is controlled using temperature and top\-ppsampling\. Temperature adjusts the randomness of the output distribution, while top\-ppsampling restricts generation to the smallest set of tokens whose cumulative probability exceeds the specified threshold\. Generation continues until either an end\-of\-sequence condition is reached or the maximum number of new tokens has been produced\. We set the temperature to0\.80\.8, the top\-ppto0\.90\.9, and the maximum number of new tokens to10001000\.

For each prompt, we generate multiple candidate texts\. Each candidate is evaluated using the surrogate detector, which assigns a score in\[0,1\]\[0,1\]estimating the probability that the text was generated by an AI system\. Lower scores indicate more human\-like writing\. The system iteratively generates candidates until either a text with an AI\-generated probability below the acceptance threshold is found or the maximum number of attempts is reached\. We set the acceptance threshold to0\.20\.2and the maximum number of attempts to1010\. If no candidate satisfies the threshold, the candidate with the lowest surrogate detector score is selected as the final output\.

## 4Model analysis

When trainingSHADE, we evaluated several fine\-tuning configurations\. First, we compared parameter\-efficient fine\-tuning with LoRA against full fine\-tuning, where all model weights are updated\. Second, we varied the KL regularization coefficientβ\\beta, which controls the penalty for deviating from the reference policy, using valuesβ∈\{0,0\.05,0\.1,0\.5,1\}\\beta\\in\\\{0,0\.05,0\.1,0\.5,1\\\}\. Configurations withβ=0\.5\\beta=0\.5andβ=1\\beta=1are not reported, as their KL\-divergence term dominated the loss and prevented meaningful optimization of the evasion reward\. We also omit the configuration withβ=0\\beta=0, since the absence of KL regularization led to degenerate generations characterized by repeated\-token outputs\.

Figure 1:Generated text classified as humanFigure[1](https://arxiv.org/html/2609.22221#S4.F1)reports the percentage of generated texts classified as human by the surrogatemdokdetector for each evaluated configuration\. The baselineLLaMA 3\.2model achieved a human classification rate of only1\.5%1\.5\\%\(1/661/66\), indicating that its outputs were almost always identified as AI\-generated\. GRPO substantially improved evasion, but its effectiveness depended strongly on the fine\-tuning strategy and the KL penalty\. Full fine\-tuning withβ=0\.05\\beta=0\.05achieved the strongest result, with98\.5%98\.5\\%of outputs classified as human \(65/6665/66\)\. Increasing the penalty toβ=0\.10\\beta=0\.10reduced the rate to80\.3%80\.3\\%\(53/6653/66\), suggesting that stronger regularization limits the policy movement required for detector evasion\.

The LoRA configurations followed a different pattern\. Without KL regularization, LoRA achieved80\.3%80\.3\\%human classification \(53/6653/66\), matching full fine\-tuning withβ=0\.10\\beta=0\.10\. However, adding regularization sharply reduced evasion, reaching only9\.1%9\.1\\%\(6/666/66\) forβ=0\.05\\beta=0\.05and0%0\\%\(0/660/66\) forβ=0\.10\\beta=0\.10\. Overall, only three configurations exceeded the50%50\\%human\-classification threshold: LoRA withβ=0\\beta=0, full fine\-tuning withβ=0\.10\\beta=0\.10, and full fine\-tuning withβ=0\.05\\beta=0\.05\. Among these, full fine\-tuning withβ=0\.05\\beta=0\.05was the only configuration approaching near\-complete surrogate evasion\.

It is important to note that these results were obtained with the retry\-based generation strategy described above\. hen generation is performed in a one\-shot setting, full fine\-tuning withβ=0\.05\\beta=0\.05achieves a human\-classification rate of59\.1%59\.1\\%\(39/6639/66\), substantially lower than with retry\-based selection\. A similar drop is observed for the LoRA configuration withβ=0\\beta=0, where only30\.3%30\.3\\%\(20/6620/66\) of the generated texts are classified as human\-written in the one\-shot setting\. These results indicate that the final evasion performance depends not only on the fine\-tuned generator, but also on the inference\-time selection strategy, which increases the probability of selecting outputs that better evade the surrogate detector\.

Figure 2:Beta vs\. Confusion tradeoff[Figure2](https://arxiv.org/html/2609.22221#S4.F2)summarizes the same results as a function of the KL coefficient, making the regularization trade\-off more explicit\. Full fine\-tuning remains effective under moderate KL regularization, whereas LoRA rapidly loses evasion capability asβ\\betaincreases\. This suggests that low\-rank adapters are less able to simultaneously preserve proximity to the reference model and optimize the adversarial reward\. The Wilson confidence intervals reflect the limited evaluation size \(n=66n=66\), but they do not alter the qualitative trend\. Taken together, these results indicate that a small, non\-zero KL penalty is beneficial in the full fine\-tuning setting because it prevents degenerate policy drift while still allowing sufficient capacity for adversarial adaptation\.

Figure 3:Training metrics full modelFigure 4:Training metrics LoRA model[Figure3](https://arxiv.org/html/2609.22221#S4.F3)and[Figure4](https://arxiv.org/html/2609.22221#S4.F4)illustrate the training dynamics for the full and LoRA fine\-tuning strategies, respectively\. In the full fine\-tuning setting, both configurations rapidly increase the reward during the first epochs, indicating that the model learns to reduce the surrogate detector’s AI\-generated probability\. The run withβ=0\.05\\beta=0\.05achieves a slightly higher and more stable reward thanβ=0\.10\\beta=0\.10, which is consistent with its stronger evasion performance in[Figure1](https://arxiv.org/html/2609.22221#S4.F1)\. The KL divergence shows a large initial spike and then quickly drops by several orders of magnitude, suggesting that most of the policy shift occurs early in training\. After this initial adaptation, the model continues optimizing within a more stable region of the parameter space\.

The entropy and completion\-length curves further reveal how full fine\-tuning adapts the generation policy\. Forβ=0\.05\\beta=0\.05, policy entropy decreases sharply after approximately five epochs, while the average completion length drops from near the maximum generation length to much shorter outputs\. This suggests that the best\-performing configuration converges toward a more specialized generation strategy that the surrogate detector tends to classify as human\-written\. In contrast, theβ=0\.10\\beta=0\.10configuration maintains higher entropy and longer completions throughout training, indicating that stronger KL regularization preserves more of the original generation behavior but also limits evasion effectiveness\.

The LoRA runs exhibit a different pattern\. The unregularized LoRA configuration \(β=0\\beta=0\) reaches a high reward early in training, but this is accompanied by rapid entropy collapse and large fluctuations in completion length, suggesting a less stable policy\. When KL regularization is introduced the LoRA configurations maintain higher entropy and more stable completion lengths, but their rewards remain substantially lower, especially forβ=0\.10\\beta=0\.10\. Moreover, the KL divergence for the regularized LoRA runs remains small throughout training, indicating that the adapters induce only limited deviations from the reference policy\. This helps explain the reduced evasion performance observed in[Figure1](https://arxiv.org/html/2609.22221#S4.F1)and[Figure2](https://arxiv.org/html/2609.22221#S4.F2)\. Under KL regularization, LoRA appears unable to modify the generation policy enough to consistently evade the surrogate detector\.

Taken together, the training curves suggest that successful evasion requires a pronounced shift in generation behavior\. Full fine\-tuning provides enough capacity for such a shift, particularly when a small KL penalty prevents degenerate policy drift while still allowing adaptation\. In contrast, LoRA offers insufficient adaptation capacity once proximity to the reference model is enforced\. This supports the interpretation that detector evasion is not simply a matter of preserving fluent generation while lowering detector confidence\. Rather, the model appears to discover specific generation regimes that exploit the surrogate detector\.

Figure 5:Repetitions within the generated text[Figure5](https://arxiv.org/html/2609.22221#S4.F5)analyzes repetition patterns in the generated texts using bigram repetition, trigram repetition, and sentence near\-duplication metrics\. The baseline model exhibits the lowest repetition levels overall, suggesting that the original instruction\-tuned model produces more locally diverse outputs\. In contrast, the full fine\-tuned GRPO models introduce substantially higher repetition rates, particularly forβ=0\.10\\beta=0\.10, which shows the highest median bigram and trigram repetition among all configurations\. This indicates that full\-policy adaptation can alter local lexical structure in ways that increase repeatednn\-gram patterns\. Interestingly, this effect is stronger forβ=0\.10\\beta=0\.10than for the best\-performingβ=0\.05\\beta=0\.05configuration, suggesting that higher repetition is not itself sufficient to explain surrogate evasion\.

The LoRA models generally preserved lower repetition levels than full fine\-tuning, especially when KL regularization is applied\. This is consistent with the earlier observation that regularized LoRA remains closer to the reference policy, but also achieves substantially lower evasion rates\. Sentence\-level near\-deduplication remains close to zero across most configurations, with only isolated outliers\. Thus, the observed degeneration mainly occurs at the local lexical level through repeated bigrams and trigrams, rather than through full sentence reuse\. The contrast between the two full fine\-tuning runs is particularly informative\. Althoughβ=0\.10\\beta=0\.10produces more repetitive text, it achieves lower evasion thanβ=0\.05\\beta=0\.05\. This suggests thatSHADE’s strongest evasion performance is not simply caused by repetition, but by a broader shift in generation behavior\.

Figure 6:Generated text readability metrics[Figure6](https://arxiv.org/html/2609.22221#S4.F6)presents readability statistics across the evaluated models\. Overall, the GRPO\-based configurations tend to produce texts with higher Flesch Reading Ease scores and lower FK Grade values than the baseline, indicating a shift toward simpler and more accessible language\. This pattern is especially visible for the high\-evasion configurations\. GRPO full fine\-tuning withβ=0\.05\\beta=0\.05increases Flesch Ease from41\.941\.9to52\.752\.7while slightly reducing FK Grade from12\.812\.8to12\.212\.2, and GRPO LoRA withβ=0\\beta=0achieves the highest Flesch Ease score and \(58\.358\.3\) and the lowest FK Grade \(10\.110\.1\)\. However, readability alone does not fully explain evasion performance\. For instance, the LoRAβ=0\\beta=0configuration is the most readable but reaches only80\.3%80\.3\\%evasion, while full fine tuning withβ=0\.05\\beta=0\.05achieves98\.5%98\.5\\%evasion with less extreme readability changes\.

[Table1](https://arxiv.org/html/2609.22221#S4.T1)provides a broader view of these shifts\. The best\-performing configuration, GRPO full fine\-tuning withβ=0\.05\\beta=0\.05, produces shorter texts than the baselines \(374±136374\\pm 136vs\.551±26551\\pm 26words\), with fewer sentences, lower type\-token ratio, and lower hapax proportion\. These changes suggest that detector evasion is associated with a reduction in lexical richness and structural complexity\. At the same time, the comparison between full fine\-tuning withβ=0\.05\\beta=0\.05andβ=0\.10\\beta=0\.10shows that excessive repetition is not sufficient for stronger evasion\. Theβ=0\.10\\beta=0\.10model has substantially higher bigram and trigram repetition rates, but lower evasion\. Taken together, the readability and lexical metrics suggest thatSHADEsucceeds by moving the generation policy toward simpler, shorter, and less lexically diverse outputs, while avoiding the strongest forms of local degeneration\.

Table 1:Comparison of human\-likeness and linguistic statistics across baseline and GRPO\-trained models\.ModelCls ErrWordsSentencesTTRHapaxFlesch EaseFK GradeBigram RepTrigram RepBaseline \(LLaMA 3\.2\)1\.5%551±26551\\pm 2622\.40\.4780\.33941\.912\.80\.13760\.0426GRPO Fullβ=0\.05\\beta=0\.0598\.5%374±136374\\pm 13613\.80\.4140\.25852\.712\.20\.22310\.1060GRPO Fullβ=0\.10\\beta=0\.1080\.3%434±159434\\pm 15918\.60\.3290\.18952\.111\.50\.36050\.2140GRPO LoRAβ=0\.00\\beta=0\.0080\.3%559±22559\\pm 2225\.00\.3770\.22758\.310\.10\.23870\.1057GRPO LoRAβ=0\.05\\beta=0\.059\.1%554±34554\\pm 3423\.50\.4540\.31147\.612\.00\.15790\.0550GRPO LoRAβ=0\.10\\beta=0\.100\.0%557±23557\\pm 2322\.20\.4730\.33644\.412\.70\.14380\.0503

## 5Voight\-Kampff Task Result

In addition to the surrogate\-detector evaluation, we also analyze the results provided by the Voight\-Kampff competition organizers\. Unlike the previous experiments, which evaluate generated texts against our mdok\-based surrogate detector, the competition setting measures how closely each submission resembles source\-like behavior under the official evaluation procedure\. Therefore, these results provide a direct estimate of how well the proposed evasion strategy transfers from the surrogate detector to the unseen evaluation setting\. Our two submitted runs,DArgk \- Full fine\-tunedβ=0\.05\\beta=0\.05andDArgk \- LoRAβ=0\\beta=0, ranked in the middle of the ranking in positions sixth and seventh, respectively\. TheDArgk \- Full fine\-tunedβ=0\.05\\beta=0\.05submission obtained a mean score of0\.3200\.320, with337337lower,432432equal, and18111811higher comparisons\. TheDArgk \- LoRAβ=0\\beta=0submission obtained a slightly worse mean score of0\.3380\.338, with269269lower,402402equal, and19091909higher comparisons\.

The gap between our surrogate results and the competition ranking highlights an important limitation of detector\-specific reinforcement learning\. WhileSHADEsubstantially reduces the confidence of the surrogate detector, this advantage does not fully transfer to the official evaluation setting\. This suggests that the model may have learned generation patterns that exploit the surrogate detector in particular, rather than detector\-invariant properties of human\-written text\. TheDArgk \- Full fine\-tunedβ=0\.05\\beta=0\.05run slightly outperformed the submittedDArgk \- LoRAβ=0\\beta=0run, which is consistent with our internal finding that full fine\-tuning provides more effective adaptation than parameter\-efficient tuning\. However, the relatively small difference between the two submitted runs in the official ranking suggests that the advantages observed against the surrogate detector are attenuated when evaluated against unseen classifiers\. Overall, the competition results indicate thatSHADEis effective as a surrogate\-targeted evasion method, but that improving cross\-detector generalization remains a central challenge\.

## 6Conclusion

This paper presentedSHADE, a reinforcement learning framework for generating texts that evade AI\-generated text detectors\. Instead of relying on post\-hoc perturbations, translation, or manually designed prompting strategies,SHADEformulates detector evasion as a policy optimization problem\. Using a surrogate detector based on the PAN 2025mdoksystem, we fine\-tuned an instruction\-tuned LLaMA model with GRPO and optimized the generator to reduce the surrogate detector’s confidence that the generated text was AI\-authored\. Overall,SHADEdemonstrates both the potential and the risks of reinforcement learning for detector evasion\. The approach shows that LLM generation policies can be optimized to exploit detector feedback, but also that high surrogate evasion does not guarantee robust transfer to unseen evaluation settings or human\-like text quality\. These findings highlight the need for more robust, adaptive, and transparent evaluation protocols for AI\-generated text detection\.

Several limitations of this study suggest directions for future work\. First, our reinforcement learning objective relies on a single surrogate detector, which may bias the generator toward detector\-specific artifacts\. Future work should reduce this risk by training against ensembles of heterogeneous surrogate detectors, including statistical, neural, and watermark\-aware models, or by periodically updating the surrogate detector in an adaptive adversarial training loop\. Second, the evaluation set used for the internal analysis is relatively small, which limits the statistical strength of some comparisons\. This limitation could be addressed by evaluating the method on larger benchmark collections and by testing whether the observed trends hold across multiple domains and datasets\. Third, the reward function focuses primarily on detector evasion and does not directly optimize for semantic fidelity, fluency, coherence, or human judgments of naturalness\. Although the KL penalty partially constrains the model, future work should incorporate multi\-objective rewards that jointly balance evasion, semantic preservation, readability, lexical diversity, and fluency, complemented by human evaluation to determine whether the generated outputs are genuinely perceived as human\-written\. Finally, our analysis is limited to a specific base model, detector family, and shared\-task setting\. Future work should therefore investigate the transferability of learned evasion strategies across different LLMs, languages, domains, and unseen detector families\.

##### Ethical considerations\.

This work addresses AI\-generated text detector evasion, a topic with clear dual\-use implications\. While the proposed method can help evaluate the robustness of current detectors, similar techniques could be misused to conceal AI authorship in contexts where originality, accountability or trust are important\. We therefore frameSHADEas a robustness evaluation approach and evaluate it within the controlled setting of the Voight\-Kampff shared task\. Our analysis reports not only evasion performance but also limitations and side effects\. These results emphasize that high detector evasion should not be interpreted as evidence of genuinely human\-like writing\. More broadly, the findings caution against relying on single\-detector decisions in high\-stakes settings such as education, scientific publishing, journalism or content moderation\. Future work should emphasize multi\-detector robustness evaluation, transparent uncertainty reporting, human oversight and safeguards against deceptive deployment\.

## 7Artifacts

The following are the required artifacts to replicate the submission:

- •
- •
- •
- •
- •

## 8Declaration on Generative AI

Generative AI tools were used solely to assist with grammar correction and language polishing\. All conceptual content, analysis, and final decisions remain the authors’ responsibility\.

##### Acknowledgments

We are grateful for the support of CLAAUDIA through AI\-Cloud for providing the computational infrastructure\. This research was funded in whole or in part by the Austrian Science Fund \(FWF\):[1255776/COE12](https://doi.org/10.55776/COE12)\.

## References

- Bevendorffet al\.\(2025\)J\. Bevendorff, D\. Dementieva, M\. Fröbe, B\. Gipp, A\. Greiner\-Petter, J\. Karlgren, M\. Mayerl, P\. Nakov, A\. Panchenko, M\. Potthast, A\. Shelmanov, E\. Stamatatos, B\. Stein, Y\. Wang, M\. Wiegmann, and E\. ZangerleOverview of PAN 2025: Voight\-Kampff Generative AI Detection, Multilingual Text Detoxification, Multi\-Author Writing Style Analysis, and Generative Plagiarism Detection\.InExperimental IR Meets Multilinguality, Multimodality, and Interaction\. Proceedings of the Sixteenth International Conference of the CLEF Association \(CLEF 2025\),J\. Carrillo\-de\-Albornoz, J\. Gonzalo, L\. Plaza, A\. G\. S\. Herrera, J\. Mothe, F\. Piroi, P\. Rosso, D\. Spina, G\. Faggioli, and N\. Ferro \(Eds\.\),Lecture Notes in Computer Science, Vol\.16089,Berlin Heidelberg New York,pp\. 388–411\.External Links:[Document](https://dx.doi.org/10.1007/978-3-032-04354-2%5F21)Cited by:[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p2.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx3.p1.1)\.
- Bevendorffet al\.\(2026\)J\. Bevendorff, M\. Fröbe, A\. Greiner\-Petter, A\. Jakoby, M\. Mayerl, P\. Nakov, H\. Plutz, M\. N\. Ta, Y\. Wang, and E\. ZangerleOverview of PAN 2026: Voight\-Kampff Generative AI Detection, Text Watermarking, Multi\-author Writing Style Analysis, Generative Plagiarism Detection, and Reasoning Trajectory Detection – Extended Abstract\.InAdvances in Information Retrieval\. 48th European Conference on IR Research \(ECIR 2026\),R\. Campos, A\. Jatowt, Y\. Lan, M\. Aliannejadi, C\. Bauer, S\. MacAvaney, Z\. Ren, S\. Verberne, N\. Bai, and M\. Mansoury \(Eds\.\),Lecture Notes in Computer Science, Vol\.16485,Cham, Switzerland,pp\. 225–232\.External Links:[Document](https://dx.doi.org/10.1007/978-3-032-21321-1%5F32)Cited by:[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p3.1),[§3\.2](https://arxiv.org/html/2609.22221#S3.SS2.p2.1)\.
- Creoet al\.\(2025\)A\. Creo, M\. Hormazábal\-Lagos, H\. Cerezo\-Costas, and P\. A\. DovalFake it ’til you make it human\.InWorking Notes of the Conference and Labs of the Evaluation Forum, CLEF 2025, Madrid, Spain, 9\-12 September 2025,G\. Faggioli, N\. Ferro, P\. Rosso, and D\. Spina \(Eds\.\),CEUR Workshop Proceedings,pp\. 1381–1390\.External Links:[Link](https://ceur-ws.org/Vol-4038/paper/_109.pdf)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p2.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx2.p1.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx3.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§3\.2](https://arxiv.org/html/2609.22221#S3.SS2.p3.1)\.
- Guntiet al\.\(2026\)R\. Gunti, B\. Bayramoğlu, J\. M\. S\. Dilinika, G\. Devadasu, D\. Galat, V\. A\. Narayana, R\. Pakala, V\. R\. Reddyvari, M\. Rizoiu, J\. M\. Rodriguez, S\. S\. Sanagala, A\. Tommasel, J\. Bevendorff, and J\. KarlgrenOverview and Joint Report of the Voight\-Kampff Task at the ELOQUENT 2026 lab for evaluating generative language model quality\.InWorking Notes of the Conference and Labs of the Evaluation Forum \(CLEF 2026\),E\. S\. Salido, A\. Barrón\-Cedeño, A\. G\. S\. de Herrera, S\. MacAvaney, and J\. M\. Struß \(Eds\.\),Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p2.1)\.
- Gunti \(2025\)R\. R\. GuntiThe distinctive data\-centric approach for the voight kampff task\.InWorking Notes of the Conference and Labs of the Evaluation Forum, CLEF 2025, Madrid, Spain, 9\-12 September 2025,G\. Faggioli, N\. Ferro, P\. Rosso, and D\. Spina \(Eds\.\),CEUR Workshop Proceedings,pp\. 1391–1400\.External Links:[Link](https://ceur-ws.org/Vol-4038/paper/_110.pdf)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p2.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx3.p2.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx3.p3.1)\.
- Hanset al\.\(2024\)A\. Hans, A\. Schwarzschild, V\. Cherepanova, H\. Kazemi, A\. Saha, M\. Goldblum, J\. Geiping, and T\. GoldsteinSpotting llms with binoculars: zero\-shot detection of machine\-generated text\.InForty\-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21\-27, 2024,R\. Salakhutdinov, Z\. Kolter, K\. A\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research,pp\. 17519–17537\.External Links:[Link](https://proceedings.mlr.press/v235/hans24a.html)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p1.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p2.1)\.
- Huet al\.\(2025\)J\. Hu, Y\. Zhang, Q\. Han, D\. Jiang, X\. Zhang, and H\. ShumOpen\-reasoner\-zero: an open source approach to scaling up reinforcement learning on the base model\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38,pp\. 162239–162262\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/ed873d79e7c268c020c4b4db13a2812a-Paper-Conference.pdf)Cited by:[§3\.2](https://arxiv.org/html/2609.22221#S3.SS2.p10.1)\.
- Karlgrenet al\.\(2026\)J\. Karlgren, M\. Barrett, O\. Bojar, M\. I\. Engels, D\. Fabre, S\. Ettejjari, L\. Goeuriot, R\. R\. Gunti, J\. Mothe, P\. Mulhem, M\. Piacentini, L\. F\. V\. Madriz, D\. Schwab, P\. Šindelář, G\. Stampoulidis, K\. Thomas, and M\. VartampetianOverview of ELOQUENT 2026: shared tasks for evaluating generative language model quality\.InExperimental IR Meets Multilinguality, Multimodality, and Interaction\. Proceedings of the Seventeenth International Conference of the CLEF Association \(CLEF 2026\),M\. Hagen, M\. Potthast, B\. Stein, P\. Schaer, E\. Zangerle, S\. MacAvaney, J\. M\. Struß, E\. S\. Salido, A\. Barrón\-Cedeño, and A\. G\. S\. de Herrera \(Eds\.\),Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p2.1)\.
- Koikeet al\.\(2024\)R\. Koike, M\. Kaneko, and N\. OkazakiOUTFOX: llm\-generated essay detection through in\-context learning with adversarially generated examples\.InThirty\-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty\-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20\-27, 2024, Vancouver, Canada,M\. J\. Wooldridge, J\. G\. Dy, and S\. Natarajan \(Eds\.\),pp\. 21258–21266\.External Links:[Link](https://doi.org/10.1609/aaai.v38i19.30120),[Document](https://dx.doi.org/10.1609/AAAI.V38I19.30120)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p1.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p3.1)\.
- Liuet al\.\(2019\)Y\. Liu, M\. Ott, N\. Goyal, J\. Du, M\. Joshi, D\. Chen, O\. Levy, M\. Lewis, L\. Zettlemoyer, and V\. StoyanovRoBERTa: A robustly optimized BERT pretraining approach\.CoRRabs/1907\.11692\.External Links:[Link](http://arxiv.org/abs/1907.11692),1907\.11692Cited by:[§3\.2](https://arxiv.org/html/2609.22221#S3.SS2.p2.1)\.
- Lowd and Meek \(2005\)D\. Lowd and C\. MeekAdversarial learning\.InProceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining,KDD ’05,New York, NY, USA,pp\. 641–647\.External Links:ISBN 159593135X,[Link](https://doi.org/10.1145/1081870.1081950),[Document](https://dx.doi.org/10.1145/1081870.1081950)Cited by:[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx2.p1.1),[§3\.2](https://arxiv.org/html/2609.22221#S3.SS2.p1.1)\.
- Macko \(2025\)D\. MackoMdok of kinit: robustly fine\-tuned LLM for binary and multiclass ai\-generated text detection\.InWorking Notes of the Conference and Labs of the Evaluation Forum, CLEF 2025, Madrid, Spain, 9\-12 September 2025,G\. Faggioli, N\. Ferro, P\. Rosso, and D\. Spina \(Eds\.\),CEUR Workshop Proceedings,pp\. 3819–3826\.External Links:[Link](https://ceur-ws.org/Vol-4038/paper/_307.pdf)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p3.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p3.1),[§3\.2](https://arxiv.org/html/2609.22221#S3.SS2.p2.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.InProceedings of the 37th International Conference on Neural Information Processing Systems,NIPS ’23,Red Hook, NY, USA\.Cited by:[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx2.p2.1)\.
- Seeligeret al\.\(2025\)M\. Seeliger, P\. Styll, M\. Staudinger, and A\. HanburyHuman or not? light\-weight and interpretable detection of ai\-generated text\.Working Notes of CLEF\.Cited by:[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p2.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx3.p3.1),[§3\.2](https://arxiv.org/html/2609.22221#S3.SS2.p3.1)\.
- Vachharajani \(2025\)P\. VachharajaniLiteral re\-translation as a method for AI text disguise and detection evasion\.InWorking Notes of the Conference and Labs of the Evaluation Forum, CLEF 2025, Madrid, Spain, 9\-12 September 2025,G\. Faggioli, N\. Ferro, P\. Rosso, and D\. Spina \(Eds\.\),CEUR Workshop Proceedings,pp\. 1443–1448\.External Links:[Link](https://ceur-ws.org/Vol-4038/paper/_116.pdf)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p2.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx3.p2.1)\.
- Wanget al\.\(2025\)T\. Wang, Y\. Chen, Z\. Liu, Z\. Chen, H\. Chen, X\. Zhang, and W\. ChengHumanizing the machine: proxy attacks to mislead llm detectors\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 68339–68367\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/ab1ee157f7804a13f980414b644a9460-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p1.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx2.p2.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx3.p3.1)\.
- Wuet al\.\(2024\)J\. Wu, R\. Zhan, D\. Wong, S\. Yang, X\. Yang, Y\. Yuan, and L\. S\. ChaoDetectRL: benchmarking llm\-generated text detection in real\-world scenarios\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 100369–100401\.External Links:[Document](https://dx.doi.org/10.52202/079017-3186),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/b61bdf7e9f64c04ec75a26e781e2ad51-Paper-Datasets_and_Benchmarks_Track.pdf)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p1.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p3.1)\.
- Yanget al\.\(2025a\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§3\.2](https://arxiv.org/html/2609.22221#S3.SS2.p2.1)\.
- Yanget al\.\(2025b\)Z\. Yang, Z\. Feng, R\. Huo, H\. Lin, H\. Zheng, R\. Nie, and H\. ChenThe imitation game revisited: a comprehensive survey on recent advances in ai\-generated text detection\.Expert Systems with Applications272,pp\. 126694\.External Links:ISSN 0957\-4174,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.eswa.2025.126694),[Link](https://www.sciencedirect.com/science/article/pii/S0957417425003161)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p1.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p1.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx2.p1.1)\.
- Zhouet al\.\(2025\)H\. Zhou, J\. Zhu, P\. Su, K\. Ye, Y\. Yang, S\. Gavioli\-Akilagun, and C\. ShiAdaDetectGPT: adaptive detection of llm\-generated text with statistical guarantees\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38,pp\. 89077–89118\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/808bc5e3d076c1125a87f81b42d5e52d-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2609.22221#S1.p1.1),[§2](https://arxiv.org/html/2609.22221#S2.SS0.SSSx1.p2.1)\.

相似文章

多模块 GRPO:组合策略梯度与提示优化的语言模型程序方法

Papers with Code Trending

本文提出 mmGRPO,一种多模块扩展的群体相对策略优化(GRPO)方法,通过优化语言模型调用和提示来提升模块化 AI 系统的准确率。实验表明,该方法在各类任务上平均带来 11% 的准确率提升,并在 DSPy 中提供了开源实现。

通过细化的安全定向嵌入利用(STEER)

arXiv cs.AI

本文介绍了STEER,一种梯度引导的攻击方法,通过将高归因词翻译成低资源语言来绕过拒绝机制,利用了大语言模型安全训练分布的漏洞,在AdvBench上实现了高达96.7%的攻击成功率,并迁移到GPT-4o-mini上达到35.5%的攻击成功率。

当合成语音是唯一资源时:GRPO更胜一筹

arXiv cs.CL

本文提出使用Group Relative Policy Optimization(GRPO)来适配基于LLM的ASR模型到监管领域,仅使用合成语音,相较于监督微调实现了40-45%的相对词错误率降低。

DACA-GRPO:扩散语言模型中强化学习的去噪感知信用分配

arXiv cs.LG

本文指出了现有扩散语言模型强化学习方法中的弱点——缺乏时间信用分配和偏差似然估计——并提出了DACA-GRPO,一种即插即用的增强方案,引入了去噪进度分数和分层掩码似然,在推理、代码生成和受约束生成等多个基准上取得了一致的改进。