本体论不稳定性与统计放大:"人性化" LLM 生成文本的悖论

arXiv cs.CL 论文

摘要

这篇 arXiv 论文表明,基于 RoBERTa 的机器生成文本检测器追踪的是统计复杂性而非机器作者身份:通过 Mistral-7B 对 LLM 输出进行"人性化"处理反而提高了词汇多样性(Verb Diversity)和检测得分;该检测器在正式人类写作上的误报率高达 76.3%,且 41% 的同形字符(homoglyph)攻击可绕过检测。基于事件的隐空间检测方法则被证明对改写和字符级扰动十分脆弱。

arXiv:2610.03110v1 Announce Type: new Abstract: Supervised AI-text detectors report high benchmark accuracy, but it is not clear what their decisions are based on. We analyze a RoBERTa-based detector under semantic, structural, and tokenizer-level perturbations, using the M4 dataset (N = 10,000) and controlled generations (N = 300). When Mistral-7B-Instruct was asked to make machine text sound more human, Verb Diversity rose from 0.77 to 0.92 and the outputs became easier to detect. Detection scores appear to track statistical complexity, which also leads to a 76.3% false-positive rate on formal human writing. As a control, we evaluate event-based Latent Space detection. Paraphrasing changed 87% of its event sequences (Jaccard = 0.067), and homoglyphs altered 70% of the extracted verbs even though extraction still ran (Jaccard = 0.30). Its best domain AUC was 0.577. RoBERTa's robustness seems specific to the features it uses, and structural abstraction did not make detection more robust.
查看原文
查看缓存全文

缓存时间: 2026/10/05 10:06

# Ontological Instability and Statistical Amplification: The Paradox of “Humanizing” LLM-Generated Text
Source: [https://arxiv.org/html/2610.03110](https://arxiv.org/html/2610.03110)
Claudiu CreangaAffiliation:Interdisciplinary School of Doctoral StudiesAffiliation:University of Bucharest, RomaniaEmail:[claudiu\.creanga@fmi\.unibuc\.ro](mailto:)Liviu DinuAffiliation:Faculty of Mathematics and InformaticsAffiliation:University of Bucharest, RomaniaEmail:[liviu\.p\.dinu@unibuc\.ro](mailto:)

###### Abstract

Supervised AI\-text detectors report high benchmark accuracy, but it is not clear what their decisions are based on\. We analyze a RoBERTa\-based detector under semantic, structural, and tokenizer\-level perturbations, using the M4 dataset \(N = 10,000\) and controlled generations \(N = 300\)\. When Mistral\-7B\-Instruct was asked to make machine text sound more human, Verb Diversity rose from 0\.77 to 0\.92 and the outputs became easier to detect\. Detection scores appear to track statistical complexity, which also leads to a 76\.3% false\-positive rate on formal human writing\. As a control, we evaluate event\-based Latent Space detection\. Paraphrasing changed 87% of its event sequences \(Jaccard = 0\.067\), and homoglyphs altered 70% of the extracted verbs even though extraction still ran \(Jaccard = 0\.30\)\. Its best domain AUC was 0\.577\. RoBERTa’s robustness seems specific to the features it uses, and structural abstraction did not make detection more robust\.

Keywords:Machine\-generated text detection; adversarial robustness; large language models; statistical amplification; natural language processing\.

## 1Introduction

Large Language Models have made machine\-text detection a practical need\. Most leading systems are supervised transformer classifiers\([Solaiman et al\., 2019](https://arxiv.org/html/2610.03110#bib.bib10);[Mitchell et al\., 2023](https://arxiv.org/html/2610.03110#bib.bib8)\), and they report more than 99% accuracy on standard benchmarks\([Gritsai et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib3)\)\. What these classifiers actually pick up is less clear: it may be something artificial about the content, or simply statistical artifacts of particular generators\.

We look at this question with RoBERTa\([Liu et al\., 2019](https://arxiv.org/html/2610.03110#bib.bib7)\)and, as a structural point of comparison, a Latent Space detector\([Tian et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib11)\)\. Both are tested with three kinds of perturbation: paraphrasing with Mistral\-7B\-Instruct\([Jiang et al\., 2023](https://arxiv.org/html/2610.03110#bib.bib5)\), changes to event sequences, and homoglyph substitution at the character level\.

Our main finding is what we call*Statistical Amplification*\. When Mistral rewrote machine text to sound more human, lexical diversity went up, and RoBERTa’s scores usually stayed high\. The detector is also poorly calibrated: at the default threshold it flagged 76\.3% of formal human texts\. Homoglyph substitution evaded it in 41% of cases, and mapping confusable characters to Latin before tokenization cut this to 2%\. In a control experiment, the Latent Space detector\([Tian et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib11)\)was affected by both paraphrasing and character\-level attacks\. We are not aware of earlier character\-level attacks on event\-based latent detectors\. Taken together, the results suggest that robustness depends on the kind of features a detector uses, not on supervision as such\.

## 2Related Work

Zero\-shot methods exploit probability differences between human and machine text\([Solaiman et al\., 2019](https://arxiv.org/html/2610.03110#bib.bib10)\)\. DetectGPT, for example, looks at the curvature of the log\-probability around a text\([Mitchell et al\., 2023](https://arxiv.org/html/2610.03110#bib.bib8)\), although changes in prompting or decoding can shift the token distributions such methods depend on\([Hans et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib4)\)\.

Supervised classifiers trained on large human–machine corpora are still the most common approach\([Wang et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib12)\), but they often degrade out of distribution\. Part of our aim is to see whether their robustness comes from statistical complexity more than from any grasp of meaning\.

Paraphrasing and synonym substitution change token distributions, and earlier work has pushed detector accuracy close to chance with them\([Krishna et al\., 2023](https://arxiv.org/html/2610.03110#bib.bib6);[Sadasivan et al\., 2023](https://arxiv.org/html/2610.03110#bib.bib9)\)\. Our Mistral experiments point the other way: rewriting often made the signal stronger, because it raised lexical diversity\.

Latent\-space methods model events or discourse transitions, which are assumed to be harder for a generator to imitate\([Tian et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib11)\)\. In our experiments this approach depended heavily on the domain and was of little use outside narrative text\.

## 3Methodology

### 3\.1Models Under Evaluation

We evaluate a RoBERTa classifier trained on M4\([Wang et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib12)\)and use the Latent Space approach\([Tian et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib11)\)as a structural control, approximating event triggers with verbs extracted by spaCy\. If event features work as intended, they should tolerate homoglyph noise but be sensitive to paraphrasing\. We also include an ensemble baseline that weights RoBERTa at 40%, Sample Space perplexity at 10%, and Latent Space at 50%\.

### 3\.2Datasets

We use 10,000 English samples from M4\([Wang et al\., 2024](https://arxiv.org/html/2610.03110#bib.bib12)\), 5,000 for validation and 5,000 for testing, balanced across WikiHow, Wikipedia, Reddit, ArXiv, and PeerRead\. Most machine text comes from Bloomz; a smaller set of template\-generated samples was used for baseline tests\.

For the Persona Stress Test, we generated 300 additional samples using Mistral\-7B\-Instruct across 100 topics and three personas: Child, Neutral, and PhD\.

### 3\.3Metrics

In addition to AUROC and accuracy, we use two metrics of our own\. Verb Diversity \(VD\) is the type\-token ratio of verbs:

VD=\|Unique Verbs\|\|Total Verbs\|\\text\{VD\}=\\frac\{\|\\text\{Unique Verbs\}\|\}\{\|\\text\{Total Verbs\}\|\}\(1\)
We use it as a rough proxy for statistical complexity; low values mean that the same actions are described repeatedly\. Event Sequence Preservation \(ESP\) is the Jaccard similarity between the verb sequences of a text before and after perturbation:

ESP=\|Voriginal∩Vperturbed\|\|Voriginal∪Vperturbed\|\\text\{ESP\}=\\frac\{\|V\_\{\\text\{original\}\}\\cap V\_\{\\text\{perturbed\}\}\|\}\{\|V\_\{\\text\{original\}\}\\cup V\_\{\\text\{perturbed\}\}\|\}\(2\)

## 4Paraphrasing and Statistical Amplification

Paraphrasing is generally expected to weaken detection, since it should remove generator\-specific artifacts\([Krishna et al\., 2023](https://arxiv.org/html/2610.03110#bib.bib6)\)\. With Mistral as the paraphraser, we observed the opposite\.

### 4\.1Iterative Paraphrasing with Mistral\-7B

We rewrote machine\-generated samples iteratively with Mistral\-7B\-Instruct, prompting it to “humanize” the text, for example:“Rewrite the following text to sound more human, less robotic, and more varied in sentence structure\.”

Table 1:Detection before and after three rounds of Mistral\-7B paraphrasing\.After three rounds of rewriting, RoBERTa still detected 95% of the samples\. Verb Diversity rose from 0\.766 to 0\.924, an increase of 20\.6% \(Table[1](https://arxiv.org/html/2610.03110#S4.T1)\), which moved the texts further from the human baseline of 0\.572\. By this measure, the rewritten texts were less like human writing than the originals\.

### 4\.2Verb Diversity by Text Type

Table[2](https://arxiv.org/html/2610.03110#S4.T2)compares these values across text types\.

Table 2:Verb Diversity, verb count, and detection score by text type\.Instruction\-tuned models seem to have difficulty reproducing the repetition that is normal in human writing\. Even when asked to “sound more human”, Mistral produced text that was more diverse than the original machine output \(Figure[1](https://arxiv.org/html/2610.03110#S4.F1)\)\.

![Refer to caption](https://arxiv.org/html/2610.03110v1/plots/diversity_invariant.png)Figure 1:Left: Verb Diversity before \(0\.766\) and after \(0\.924\) Mistral\-7B rewriting, with the human baseline \(0\.572\)\. Right: stability of statistical features \(79\.4%\) and of event sequences \(13\.1%\) under rewriting\.
### 4\.3Persona Stress Test

To look at this more directly, we generated 300 texts with Mistral\-7B under three personas that ask for different levels of complexity: Child \(“Explain to a 5\-year\-old”\), Neutral, and PhD\.

![Refer to caption](https://arxiv.org/html/2610.03110v1/plots/persona_stress_test.png)Figure 2:Left: Verb Diversity against detection score for the persona samples\. Right: mean Verb Diversity per persona; Child\-persona texts \(0\.764\) stay well above the human baseline \(0\.572\), and 98\.8% of them are detected\.Table 3:Persona Stress Test Results \(N=300N=300, 100 topics×\\times3 personas\)\.As Figure[2](https://arxiv.org/html/2610.03110#S4.F2)and Table[3](https://arxiv.org/html/2610.03110#S4.T3)show, all three personas were detected at high rates\. Child\-persona texts had a mean Verb Diversity of 0\.764 and were detected 98\.8% of the time, while PhD texts reached 0\.948 with a mean score of 1\.000\. The difference between human and Child\-persona diversity was large \(t=−11\.81t=\-11\.81,p<10−24p<10^\{\-24\}, Cohen’sd=1\.99d=1\.99\)\. The only samples that evaded detection had a diversity below 0\.50\.

### 4\.4Compression Ratio

We also checked whether the higher diversity shows up beyond verbs, using the Gzip compression ratio \(R=S​i​z​ec​o​m​p​r​e​s​s​e​d/S​i​z​eo​r​i​g​i​n​a​lR=Size\_\{compressed\}/Size\_\{original\}\)\. Higher values mean the text compresses less, i\.e\. it has higher character\-level entropy\. Human text had a ratio of0\.541±0\.030\.541\\pm 0\.03and machine text0\.587±0\.040\.587\\pm 0\.04\(Table[8](https://arxiv.org/html/2610.03110#S5.T8)\)\.

Machine text was thus 8\.5% less compressible, which agrees with the Verb Diversity results\. This seems to conflict with the usual view that machine text has low perplexity; we return to this in Section[5\.4](https://arxiv.org/html/2610.03110#S5.SS4)\.

![Refer to caption](https://arxiv.org/html/2610.03110v1/plots/entropy_paradox.png)Figure 3:Left and center: Gzip ratio \(machine 0\.587, human 0\.541\) and GPT\-2 perplexity \(machine 30\.2, human 40\.0\)\. Right: per\-sample scatter of the two measures\.One Child\-persona sample, “The Food Chain,” evaded detection completely\. It repeats the same pattern throughout: “The grass gets eaten by the bunny\. The bunny gets eaten by the fox\.” Its Verb Diversity was 0\.462 and its score 0\.000\. In our data, evasion required this kind of plain repetition\.

## 5Tokenizer\-Level Attacks

RoBERTa handled paraphrasing well, but, as[Wolff and Wolff \(2020\)](https://arxiv.org/html/2610.03110#bib.bib13)showed earlier, it remains sensitive to character\-level noise\.

### 5\.1Homoglyph Substitution

Following the methodology of[Wolff and Wolff \(2020\)](https://arxiv.org/html/2610.03110#bib.bib13), we replaced characters with visually identical Unicode homoglyphs \(e\.g\., Cyrillic ’a’ \[U\+0430\] vs\. Latin ’a’ \[U\+0061\]\) at varying substitution rates\.

Table 4:Homoglyph Attack Results \(RoBERTa\)\.At 20% substitution, the detection rate fell from 100% to 59%, so 41% of samples evaded detection \(Table[4](https://arxiv.org/html/2610.03110#S5.T4)\)\. Beyond 20%, the effect levels off\. Even at 5% \(about 14 characters per text\), 13% of samples evaded \(Figure[4](https://arxiv.org/html/2610.03110#S5.F4)\)\. This is consistent with a detector that works on sub\-word token IDs: homoglyphs turn common tokens into rare or unknown ones \(\[UNK\]\), which its learned statistics do not cover\.

![Refer to caption](https://arxiv.org/html/2610.03110v1/plots/final_experiments.png)Figure 4:Homoglyph evasion rate by substitution rate\. Evasion reaches 13% at 5% substitution and levels off at about 41% from 20% onward\.
### 5\.2Control Experiment: Latent Space Detection

As a control, we tested whether event\-based Latent Space detection behaves the other way round, resisting homoglyph noise but not paraphrasing\.[Tian et al\. \(2024\)](https://arxiv.org/html/2610.03110#bib.bib11)studied semantic perturbations; we did not find prior work applying character\-level attacks to this approach\.

Table 5:Latent Space Vulnerability to Attacks\.Table[5](https://arxiv.org/html/2610.03110#S5.T5)shows that the detector was vulnerable to both attacks\. At 30% homoglyph substitution, verb extraction usually still ran, but 70% of the extracted events changed \(Jaccard 0\.30\), and nothing in the pipeline signals the error\. Mistral rewriting did even more damage \(Jaccard 0\.067\); in one example, \[analyze, discover\] became \[examine, reveal\]\.

The detector needs both stable events and correct extraction\. Paraphrasing breaks the first, and homoglyphs break the second\.

Together with the RoBERTa results, this supports the view that robustness comes from the features a detector uses\. A more abstract representation did not, by itself, make detection more robust\.

### 5\.3Performance by Domain

We evaluated detection performance across five domains \(Table[6](https://arxiv.org/html/2610.03110#S5.T6)\)\.

Table 6:Domain\-Specific Performance \(AUC\)\.The Latent Space detector was above chance only on WikiHow \(AUC 0\.577\), and it scored below chance on ArXiv \(0\.385\) and PeerRead \(0\.393\)\.

We see three reasons for this:

- •Technical writing relies on nominalizations \(“analysis” instead of “analyze”\), so the extracted events are unreliable\.
- •Human and machine academic abstracts both contain about 50–60 verbs, with similar distributions, so there is little signal to separate them\.
- •Paraphrasing changed 87% of event sequences, so events behave more like wording choices than like stable properties of the content\.

Even on narrative text the gain is small, and on technical text the detector does worse than chance\. Table[7](https://arxiv.org/html/2610.03110#S5.T7)compares the two detectors\.

Table 7:Attack results and best\-domain AUC for the two detectors\.RoBERTa resists paraphrasing but not homoglyphs, whereas the Latent Space detector is affected by both\.

### 5\.4Lexical Entropy and Perplexity

Machine text is often described as low\-entropy\([Mitchell et al\., 2023](https://arxiv.org/html/2610.03110#bib.bib8)\)\. On 200 M4 samples, we measured two different things: lexical entropy, through the Gzip ratio, and predictability to a language model, through GPT\-2 perplexity\.

Table 8:Gzip compression ratio and GPT\-2 perplexity \(N = 200\)\.Machine text was more predictable \(P​P​Lm​a​c​h​i​n​e=30\.2PPL\_\{machine\}=30\.2vs\.P​P​Lh​u​m​a​n=40\.0PPL\_\{human\}=40\.0\) but less compressible \(R​a​t​i​om​a​c​h​i​n​e=0\.587Ratio\_\{machine\}=0\.587vs\.R​a​t​i​oh​u​m​a​n=0\.541Ratio\_\{human\}=0\.541\)\. A plausible reading is that LLMs choose safe, high\-probability continuations while drawing on a wider vocabulary than human writers do\. RoBERTa may be picking up this lexical density \(Figure[3](https://arxiv.org/html/2610.03110#S4.F3)\)\.

### 5\.5Alternative Detection Methods

We also tried two other detection methods\. The Binoculars score, a perplexity ratio, reached only 50\.3% accuracy \(AUC = 0\.217\)\. Rewrite stability \(RAIDAR\), which measures edit distance after T5 rewriting, did not separate the classes \(AUC = 0\.492\), so in our setting machine text was not more “stable” under rewriting\.

## 6Calibration and Defense

### 6\.1Threshold Calibration

At the default threshold of 0\.5, the detector labelled 76\.3% of human ArXiv abstracts as machine\-generated\. Raising the threshold toτ=0\.9999\\tau=0\.9999removed these false positives, but machine recall dropped to 1\.2% and homoglyph attack success rose to 97% \(Table[9](https://arxiv.org/html/2610.03110#S6.T9)\)\. We could not find a single threshold that worked well\.

Table 9:Detector behavior at the default and strict thresholds\.In the T\-SNE projection \(Figure[5](https://arxiv.org/html/2610.03110#S6.F5)\), formal human text falls inside the machine cluster, while casual Reddit posts stay apart\. The detector seems to respond to formality more than to anything specific to machine text\.

![Refer to caption](https://arxiv.org/html/2610.03110v1/plots/tsne_embeddings.png)Figure 5:T\-SNE projection of RoBERTa embeddings\. Formal human text \(blue triangles\) falls inside the machine cluster \(red circles\)\.
### 6\.2Confusable\-Character Sanitization

NFKC normalization does not map look\-alike characters from other scripts to Latin, so it had no effect on the attack\. We therefore added a lookup table of common confusables, such as Cyrillic ‘c’ \(U\+0441\) and ‘e’ \(U\+0435\), and replaced them with their Latin counterparts before tokenization\.

Table 10:Homoglyph attack success before and after sanitization\.With this mapping, attack success fell from 41% to 2% \(Table[10](https://arxiv.org/html/2610.03110#S6.T10); Figure[6](https://arxiv.org/html/2610.03110#S6.F6)\)\. For a deployed detector, this kind of preprocessing seems necessary\.

![Refer to caption](https://arxiv.org/html/2610.03110v1/plots/aggressive_sanitization.png)Figure 6:Homoglyph attack success before \(41%\) and after \(2%\) confusable mapping\.

## 7Other Attacks

### 7\.1Syntax Simplification

Simplification with Flan\-T5\-Large\([Chung et al\., 2022](https://arxiv.org/html/2610.03110#bib.bib1)\)led to evasion in 6% of samples, with a mean score drop of 0\.031\. One sample, however, dropped by 0\.997\. As with “The Food Chain”, the few successful cases were texts that had become much simpler\.

### 7\.2Recursive Back\-Translation

Round\-trip translation \(EN→\\rightarrowFR→\\rightarrowDE→\\rightarrowEN\) led to evasion in 2% of samples, with a mean score drop of 0\.018, so the detection signal largely survived translation \(Figure[7](https://arxiv.org/html/2610.03110#S7.F7)\)\.

![Refer to caption](https://arxiv.org/html/2610.03110v1/plots/atack_effectiveness_comparison.png)Figure 7:Evasion rates for simplification \(6%\), back\-translation \(2%\), and homoglyph substitution \(41%\)\. The dashed line marks 50%\.
### 7\.3Greedy Word\-Swap Attack

We also ran a black\-box greedy word\-swap attack, using BERT\-base\([Devlin et al\., 2019](https://arxiv.org/html/2610.03110#bib.bib2)\)to propose substitutions and keeping, at each step, the one that lowered RoBERTa’s score the most\.

On 49 of the 50 samples, no substitution lowered the score enough to be worth making\. If the detector relied on a few giveaway words, the search should have found them; the result suggests that the signal is spread across the whole text\.

## 8Discussion

### 8\.1Lexical Diversity as the Main Signal

RoBERTa’s scores are closely associated with lexical diversity, although correlated syntactic or token\-level features may also play a role\. This association accounts for the three main observations: paraphrasing does not help an attacker, homoglyphs do, and formal human prose is often misclassified\.

### 8\.2Feature Choice and Robustness

The Latent Space control suggests that robustness depends on which features a detector uses\. Statistical and structural features failed under different perturbations\.

Working at the level of events did not make the detector more robust\. It added new ways to fail, such as extraction errors, while the sensitivity to paraphrasing remained\.

For an attacker, paraphrasing appears counterproductive\. In our experiments, evasion required lowering complexity through repetition, attacking the tokenizer with homoglyphs, or bringing Verb Diversity below 0\.50\.

In practice, a deployed detector should at least sanitize confusable characters, calibrate thresholds per domain, and combine statistical with structural signals\.

## 9Conclusion and Future Work

This paper examined why a supervised AI\-text detector is robust to some attacks and not to others\. The main findings are:

1. 1\.Paraphrasing with Mistral increased lexical diversity and made texts more detectable \(Statistical Amplification\)\.
2. 2\.Homoglyph substitution evaded the detector in 41% of cases, in line with earlier work\.
3. 3\.The detector confused formality with machine generation, with a 76% false\-positive rate on academic text\.
4. 4\.Mapping confusable characters to Latin reduced homoglyph attack success to 2%\.

Overall, the detector behaves more like a filter for statistical complexity than like a system that recognizes what makes a text machine\-written\.

Future work should check whether Statistical Amplification holds for newer generators, other languages, and other architectures, and should look for calibration methods that can tell academic formality apart from machine generation\.

## 10Limitations

Our main experiments use a single RoBERTa detector and English text only, so we cannot say how the results transfer to other architectures or languages\. The Latent Space detector uses spaCy verbs as a stand\-in for events, and a full event extractor might behave differently\. Homoglyphs are also easy to detect with Unicode analysis; our claim is only that detectors without sanitization miss them\.

## 11Ethics Statement

This study reports detector vulnerabilities to support more reliable systems\. It uses the public M4 dataset and open\-weight models, collects no private data, and involves no human participants\.

#### Generative AI Declaration\.

The authors used AI tools for writing code, language polishing, and reference management\. All outputs were reviewed and edited by the authors, who take full responsibility for the content of this article\.

## References

- Chung et al\. \(2022\)Chung, H\. W\., L\. Hou, S\. Longpre, B\. Zoph, Y\. Tay, W\. Fedus, Y\. Li, et al\. 2022\. “Scaling Instruction\-Finetuned Language Models\.”arXiv preprint arXiv:2210\.11416\.[https://arxiv\.org/abs/2210\.11416](https://arxiv.org/abs/2210.11416)
- Devlin et al\. \(2019\)Devlin, J\., M\.\-W\. Chang, K\. Lee, and K\. Toutanova\. 2019\. “BERT: Pre\-training of Deep Bidirectional Transformers for Language Understanding\.” InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, 4171–4186\. Association for Computational Linguistics\.[https://aclanthology\.org/N19\-1423/](https://aclanthology.org/N19-1423/)
- Gritsai et al\. \(2024\)Gritsai, G\., A\. Voznyuk, A\. Grabovoy, and Y\. Chekhovich\. 2024\. “Are AI Detectors Good Enough? A Survey on Quality of Datasets with Machine\-Generated Texts\.”arXiv preprint arXiv:2410\.14677\.[https://arxiv\.org/abs/2410\.14677](https://arxiv.org/abs/2410.14677)
- Hans et al\. \(2024\)Hans, A\., A\. Schwarzschild, V\. Cherepanova, H\. Kazemi, A\. Saha, M\. Goldblum, J\. Geiping, and T\. Goldstein\. 2024\. “Spotting LLMs with Binoculars: Zero\-Shot Detection of Machine\-Generated Text\.” InProceedings of the 41st International Conference on Machine Learning, 17519–17537\. PMLR 235\.[https://proceedings\.mlr\.press/v235/hans24a\.html](https://proceedings.mlr.press/v235/hans24a.html)
- Jiang et al\. \(2023\)Jiang, A\. Q\., A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, et al\. 2023\. “Mistral 7B\.”arXiv preprint arXiv:2310\.06825\.[https://arxiv\.org/abs/2310\.06825](https://arxiv.org/abs/2310.06825)
- Krishna et al\. \(2023\)Krishna, K\., Y\. Song, M\. Karpinska, J\. Wieting, and M\. Iyyer\. 2023\. “Paraphrasing Evades Detectors of AI\-Generated Text, but Retrieval Is an Effective Defense\.” InAdvances in Neural Information Processing Systems36: 27469–27500\.[https://proceedings\.neurips\.cc/paper\_files/paper/2023/hash/575c450013d0e99e4b0ecf82bd1afaa4\-Abstract\-Conference\.html](https://proceedings.neurips.cc/paper_files/paper/2023/hash/575c450013d0e99e4b0ecf82bd1afaa4-Abstract-Conference.html)
- Liu et al\. \(2019\)Liu, Y\., M\. Ott, N\. Goyal, J\. Du, M\. Joshi, D\. Chen, O\. Levy, M\. Lewis, L\. Zettlemoyer, and V\. Stoyanov\. 2019\. “RoBERTa: A Robustly Optimized BERT Pretraining Approach\.”arXiv preprint arXiv:1907\.11692\.[https://arxiv\.org/abs/1907\.11692](https://arxiv.org/abs/1907.11692)
- Mitchell et al\. \(2023\)Mitchell, E\., Y\. Lee, A\. Khazatsky, C\. D\. Manning, and C\. Finn\. 2023\. “DetectGPT: Zero\-Shot Machine\-Generated Text Detection Using Probability Curvature\.” InProceedings of the 40th International Conference on Machine Learning, 24950–24962\. PMLR\.[https://proceedings\.mlr\.press/v202/mitchell23a\.html](https://proceedings.mlr.press/v202/mitchell23a.html)
- Sadasivan et al\. \(2023\)Sadasivan, V\. S\., A\. Kumar, S\. Balasubramanian, W\. Wang, and S\. Feizi\. 2023\. “Can AI\-Generated Text Be Reliably Detected?”arXiv preprint arXiv:2303\.11156\.[https://arxiv\.org/abs/2303\.11156](https://arxiv.org/abs/2303.11156)
- Solaiman et al\. \(2019\)Solaiman, I\., M\. Brundage, J\. Clark, A\. Askell, A\. Herbert\-Voss, J\. Wu, A\. Radford, D\. Krueger, J\. W\. Kim, S\. Kreps, et al\. 2019\. “Release Strategies and the Social Impacts of Language Models\.”arXiv preprint arXiv:1908\.09203\.[https://arxiv\.org/abs/1908\.09203](https://arxiv.org/abs/1908.09203)
- Tian et al\. \(2024\)Tian, Y\., Z\. Pan, and N\. Peng\. 2024\. “Detecting Machine\-Generated Long\-Form Content with Latent\-Space Variables\.” InFindings of the Association for Computational Linguistics: EMNLP 2024, 10394–10408\. Association for Computational Linguistics\.[https://aclanthology\.org/2024\.findings\-emnlp\.608/](https://aclanthology.org/2024.findings-emnlp.608/)
- Wang et al\. \(2024\)Wang, Y\., J\. Mansurov, P\. Ivanov, J\. Su, A\. Shelmanov, A\. Tsvigun, C\. Whitehouse, et al\. 2024\. “M4: Multi\-Generator, Multi\-Domain, and Multi\-Lingual Black\-Box Machine\-Generated Text Detection\.” InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\), 1369–1407\. Association for Computational Linguistics\.[https://aclanthology\.org/2024\.eacl\-long\.83/](https://aclanthology.org/2024.eacl-long.83/)
- Wolff and Wolff \(2020\)Wolff, M\., and S\. Wolff\. 2020\. “Attacking Neural Text Detectors\.”arXiv preprint arXiv:2002\.11768\.[https://arxiv\.org/abs/2002\.11768](https://arxiv.org/abs/2002.11768)

相似文章

困惑度陷阱:专利法使人写作看起来如同AI生成

arXiv cs.AI

本文研究了现有AI文本检测器在专利权利要求上的高误报率——专利权利要求在法律上要求像LLM输出一样清晰简洁——并提出了一种基于语言复杂性特征的逻辑回归分类器,在消费级硬件上优于基于困惑度的检测器。

基础模型被AI检测器视为人类

arXiv cs.CL

这篇论文揭示,GPTZero和Pangram等商用AI检测器将基础语言模型生成的文本判定为几乎完全是人类撰写,而经过指令微调的模型输出则被标记为AI生成。作者提出了HIP,一种与检测器无关的迭代改写流程,能在保持语义的同时提升文本的类人性。

对数似然、辛普森悖论与机器生成文本的检测

arXiv cs.CL

本文通过指出基于似然的机器生成文本检测器在 token 分数聚合中存在的辛普森悖论,解决了此类检测器性能下降的问题。本文提出了一种学习到的局部校准步骤,显著提升了各种模型和数据集上的检测性能。