Paraphrasing Attack Resilience of Various AI-Generated Text Detection Methods

# Paraphrasing Attack Resilience of Various AI-Generated Text Detection Methods
Source: [https://arxiv.org/html/2605.14240](https://arxiv.org/html/2605.14240)
Inessa Verbitsky Northwestern University Evanston, IL, USA inessa@u\.northwestern\.edu

###### Abstract

The recent large\-scale emergence of LLMs has left an open space for dealing with their consequences, such as plagiarism or the spread of false information on the Internet\. Coupling this with the rise of AI detector bypassing tools, reliable machine\-generated text detection is in increasingly high demand\. We investigate the paraphrasing attack resilience of various machine\-generated text detection methods, evaluating three approaches: fine\-tuned RoBERTa, Binoculars, and text feature analysis, along with their ensembles using Random Forest classifiers\. We discovered that Binoculars\-inclusive ensembles yield the strongest results, but they also suffer the most significant losses during attacks\. In this paper, we present the dichotomy of performance versus resilience in the world of AI text detection, which complicates the current perception of reliability among state\-of\-the\-art techniques\.

Paraphrasing Attack Resilience of Various AI\-Generated Text Detection Methods

Andrii ShportkoNorthwestern UniversityEvanston, IL, USAandre\.s@u\.northwestern\.eduInessa VerbitskyNorthwestern UniversityEvanston, IL, USAinessa@u\.northwestern\.edu

## 1Introduction

The widespread use of LLMs can be precarious when left unchecked, with the consequences ranging from intellectual dishonesty to the spread of fake news on social media\.Elali and Rachid \([2023](https://arxiv.org/html/2605.14240#bib.bib3)\)found that AI chatbots can easily produce both realistic\-looking academic results and a polished manuscript that may well be accepted to a conference and published\. Since scientific research, especially medical, is often falsified, the emergence of such a possibility opens up a dangerous playing field\(Phogatet al\.,[2023](https://arxiv.org/html/2605.14240#bib.bib4)\)\. It was found that 14% of scientists were aware of colleagues who falsified results, whereas 72% of scientists knew of colleagues who engaged in questionable research practices\(Fanelli,[2009](https://arxiv.org/html/2605.14240#bib.bib15)\)\. More incidents of AI being used in the case of fake news spreading on the internet can be found in the Ethics Statement\.

What is particularly concerning about this is that humans have been found to perform rather poorly on manual detection of AI\-written text\. In particular, human performance has shown to be only marginally better than random classification\(Wuet al\.,[2024](https://arxiv.org/html/2605.14240#bib.bib5)\)\. In fact, in a study involving over 130 subjects,Kumar and Mindzak \([2024](https://arxiv.org/html/2605.14240#bib.bib6)\)found that participants were only able to correctly identify AI\-generated text with an accuracy rate of 24%\. Concerning the use of AI in academia,Gaoet al\.\([2022](https://arxiv.org/html/2605.14240#bib.bib14)\)conducted an experiment where participants were to identify whether abstracts for academic papers were written by ChatGPT or a human\. They found that only 68% of the AI\-detected abstracts were correctly classified\. Such a precedent makes a strong case for the necessity of precise automated AI text detection mechanisms\.

With the emergence of freely accessible sites such as ZeroGPT, DetectGPT, and Quillbot, bypassing attacks have been developed against these technologies\. Methods which are commonly used include automated paraphrasing tools, prompt engineering, and the calculated addition of errors into AI\-generated text\(Perkinset al\.,[2024](https://arxiv.org/html/2605.14240#bib.bib12)\)\. It has been generally shown that the use of these methods decreases the efficacy of the detection tool; however, we aim to put together a more comprehensive analysis of the leading AI detection methods against bypassing attacks\. In this paper, we will focus on paraphrasing attacks\.

The leading state\-of\-the\-art detectors can be categorized into two paradigms, those using training\-based and training\-free mechanisms\(Wanget al\.,[2025](https://arxiv.org/html/2605.14240#bib.bib17)\)\. Most training\-free approaches rely on statistical feature analysis and commonly look at perplexity, log probability, and n\-grams\(Chakrabortyet al\.,[2023](https://arxiv.org/html/2605.14240#bib.bib13)\)\. Although training\-based models have been widely leading, a recently developed methodology – Binoculars – proves successful in a zero\-shot context, which stands out over multiple metrics\(Hanset al\.,[2024](https://arxiv.org/html/2605.14240#bib.bib16)\)\. This approach is developed further in the Related Work section\. Training\-based methods largely rely on transformer models, namely RoBERTa\(Liuet al\.,[2019](https://arxiv.org/html/2605.14240#bib.bib11)\), a masked\-based model, easily fine\-tunable for downstream tasks such as text classification\.

The methods we stacked to develop our own model include Binoculars, RoBERTa, and text feature analysis, which we justify due to their leading benchmarks \(detailed in Related Work\)\.

## 2Related Work

### 2\.1Binoculars

The Binoculars method relies on calculations from two closely related LLMs\. It has a significant advantage over other SOTA methods in that it uses no training from the LLM that it is being tested on\. This is significant, considering Binoculars still manages to surpass every open\-source model that detects ChatGPT\. Because other detectors rely on pretraining of the models they then test, the results fail to generalize when tested across multiple AI models\. The Binoculars method, however, achieves high performance on a variety of datasets, which gather texts from different LLM sources\. Furthermore, Binoculars addresses what they call the “Capybara Problem”, which in essence refers to the phenomenon of an LLM generating high\-perplexity text simply due to a high\-perplexity prompt being used\. Other models which focus on raw perplexity will fail in such cases\. Binoculars has an accuracy rate above 90%, and a false positive rate of 0\.01%, using datasets which includeWriting prompts, News, and Student essays\(Vermaet al\.,[2024](https://arxiv.org/html/2605.14240#bib.bib2)\)\.

### 2\.2Text Features

Muñoz\-Ortizet al\.\([2024](https://arxiv.org/html/2605.14240#bib.bib10)\)analyzed linguistic patterns in human and LLM text to determine which features would provide for the most robust detection mechanism\. Using extensive data from six different LLMs, including Llama and Falcon 7\-b, they found that human writing tends to have less uniform sentence length distribution than AI\. This conclusion is supported byDesaireet al\.\([2023](https://arxiv.org/html/2605.14240#bib.bib9)\), who found that the standard deviation of sentence length was an important identifier in text classification\. As one of our five text features used, we thus implement standard deviation of sentence length\.

### 2\.3Ensembling

Abburiet al\.\([2023](https://arxiv.org/html/2605.14240#bib.bib8)\)analyze the success in using ensemble approaches for text classification\. Their ensemble involves stacking DeBERTa, RoBERTa, and xLM\-RoBERTa, fine\-tuning each model for the appropriate tasks\. They found that this approach reached 5th place in the English task and first place in the Multilingual for the Automated Text Identification shared task\.

In fact, ensembling was highly used in Task 1 of theCOLING 2025 GenAI Text detection workshop, from which we use the dataset provided to train and evaluate our own model\(Wanget al\.,[2025](https://arxiv.org/html/2605.14240#bib.bib17)\)\.Mobin and Islam \([2025](https://arxiv.org/html/2605.14240#bib.bib7)\), whose approach scored 4th among contestants, relied on ensembling RoBERTa\-base with other pre\-trained transformer models\. Our methodology also relies on RoBERTa, however, we ensemble it with Binoculars and text feature analysis, as justified above\.

### 2\.4Bypassing

![Refer to caption](https://arxiv.org/html/2605.14240v1/naacl.png)Figure 1:Pipeline of our modelThe most prominent AI text\-detection models relying on transformer fine\-tuning have been tested against bypassing and proven to largely withstand it\.Krishnaet al\.\([2023](https://arxiv.org/html/2605.14240#bib.bib1)\)provide a critical baseline by demonstrating that controlled paraphrasing can significantly undermine the performance of AI\-generated text detectors while maintaining semantic integrity\. Their work, through the DIPPER model, shows that even minimal paraphrasing – changing wording and sentence structure – can drop detection accuracy significantly\.

Some common AI detectors saw decreases of around 17% in accuracy\(Perkinset al\.,[2024](https://arxiv.org/html/2605.14240#bib.bib12)\)when bypassing methods were employed\. However, some more recently developed models were created specifically to withstand such attacks, such as the RADAR model\(Huet al\.,[2023](https://arxiv.org/html/2605.14240#bib.bib18)\), which trains the detector on paraphrasing schemes and achieves over 31\.64% of additional accuracy compared to previous methods\. The Binoculars method, however, has not been tested against bypassing, thus its general efficacy remains unclear\. This concern is explored in our paper\.

## 3Data and Methodology

To track the progress on machine\-generated text detection, we use the materials of the competition onDetecting AI Generated Content @COLING 2025 Task 1: Binary Machine\-Generated Text Detection\(Wanget al\.,[2025](https://arxiv.org/html/2605.14240#bib.bib17)\)\. It is an aggregation of other datasets that have been studied before, such as M4GT\. The experiments in the following sections are based on the testing dataset that the final leaderboard used\. All models use the training dataset, which is described in Appendix A\.0\.1\.

First, we chose to fine\-tune RoBERTa for AI text detection because it provided a substantial improvement in the model’s ability to understand nuanced language differences\. In essence, we added a final layer of size 2 for binary classification\. It is also a well\-tested approach in machine\-generated text detection\(Liuet al\.,[2019](https://arxiv.org/html/2605.14240#bib.bib11)\)\. We performed fine\-tuning over a subset \(12k entries\) of the training set provided by the workshop\. The hyperparameters are learning rate =2​e−52e\-5, batch size =1616, epochs =44, and training size =20,00020,000, with a train/test split of0\.80\.8\.

Second, we also test the Binoculars method, which reaches high accuracy and low false\-positive rates over multiple LLM tested on, without relying on training data\. Binoculars uses two closely related LLMs – ’tiiuae/falcon\-7b’, ’tiiuae/falcon\-7b\-instruct’ – to calculate cross\-perplexity, meaning perplexity is calculated using the log perplexity of text generated by one LLM and the next\-token prediction of another\.

Third, we measured several document metrics that are related to AI detection\. We selected 5 text markers: average word length, lexical diversity, punctuation frequency\(Corizzo and Leal\-Arenas,[2023](https://arxiv.org/html/2605.14240#bib.bib19)\), standard deviation of sentence length, and stopword ratio\(Grykaet al\.,[2024](https://arxiv.org/html/2605.14240#bib.bib20)\)\. The selection of features was based on the entropy values from the Random Forests\.

We combined the features extracted from each approach into a single vector for each text sample and fed it to the Random Forests model that acts like a meta\-learner\. This vector includes the prediction probabilities from the fine\-tuned RoBERTa model as well as the predicted labels, the cross\-perplexity scores from Binoculars, and the five document metrics we selected \(Fig[1](https://arxiv.org/html/2605.14240#S2.F1)\)\. In the following sections, we will show the performance of all 7 different stackings of the models\.

Since we had limited resources, we manually chose201201random entries from the evaluation dataset, with the AI label, and fed them to the high\-performance AI text detector bypasser GPTinf\. We concatenated these paraphrased entries with201201randomly selected human\-written entries from the same evaluation dataset\. GPTinf claims to bypass all AI detectors, including Turnitin AI Detector, GPTZero, ZeroGPT, and GPTRadar\. The dataset is published now on HuggingFace at ’antebe1/paraphrased\_AI\_text’\.

Although the algorithm used by GPTInf is not publicized, their website states that it works by paraphrasing the inputted text–removing common phrasing and diversifying sentence structure by varying the wording, grammar, and ordering of words used\([9](https://arxiv.org/html/2605.14240#bib.bib21)\)\. To calculate the confidence interval \(CI\) for the F1 score on the full dataset, we used a bootstrapping approach \(90009000out of7373k\)\. To verify whether the differences between modules tested on were significant, we ran2121pair\-wise McNemar statistical tests \(Table[3](https://arxiv.org/html/2605.14240#A1.T3)\)\. The Bonferroni correction forα=0\.1\\alpha=0\.1is0\.00480\.0048\.

## 4Results

### 4\.1Binoculars

#### 4\.1\.1Observations

For rapid testing purposes, all tests on Binoculars have been run on the devtest split of the dataset\.

![Refer to caption](https://arxiv.org/html/2605.14240v1/bino.png)Figure 2:Binoculars results![Refer to caption](https://arxiv.org/html/2605.14240v1/box.png)Figure 3:Binoculars score over context window of 512 w/o quantization
#### 4\.1\.2Context Window Effect

We observed that the information gain increases as the context window increases\. However, the information gain plateaus somewhere after256−512256\-512tokens\. The Jensen\-Shannon \(JS\) divergence score \(Fig[3](https://arxiv.org/html/2605.14240#S4.F3), see Appendix A\.0\.2\), which measures the similarity between probability distributions, demonstrated significant improvements from0\.03730\.0373\(context window size =3232\) to0\.28430\.2843\(context window size =512512\)\. The JS score highlighted distinct effects between human\-authored and AI\-generated text as the context window increased\.

The Binoculars score analysis reveals a clear separation between human and AI\-generated text\. Human\-written content maintains the highest median score around1\.01\.0\(Fig[2](https://arxiv.org/html/2605.14240#S4.F2)\) as predicted by the Binoculars paper, exhibiting notable variance and outliers\. The critical threshold value of0\.9010\.901, just as reported in the original paper, serves as a discriminator between human and AI\-generated content\.

### 4\.2Module Ensemble Comparisons

Three different modules give rise to 7 different ways to assemble them \(Fig[4](https://arxiv.org/html/2605.14240#S4.F4)\)\.

![Refer to caption](https://arxiv.org/html/2605.14240v1/pre_attack.png)Figure 4:Pre\-attack F1 scoresThe ensemble incorporating all modules \(Text Features, RoBERTa, and Binoculars\) achieves the highest F1 score of 80\.61%\. The second\-best performance is observed when Text Features and Binoculars are combined\. While combining Text Features with RoBERTa or RoBERTa with Binoculars also improves performance compared to individual features, they fall short of the comprehensive ensemble\. Notably, individual feature sets such as Text Features, RoBERTa, or Binoculars alone yield lower F1 scores than any combination of them \(as seen in Table[1](https://arxiv.org/html/2605.14240#S4.T1)\)\.

### 4\.3Paraphrasing Attack

![Refer to caption](https://arxiv.org/html/2605.14240v1/post_attack.png)Figure 5:Post\-attack F1 scoresAmong individual models, RoBERTa demonstrated the highest resilience to paraphrasing attacks, showing almost no degradation \(Table[1](https://arxiv.org/html/2605.14240#S4.T1)\)\. In contrast, the Binoculars method exhibited the most vulnerability, resulting in a significant degradation of0\.1960\.196\.

Interestingly, the Text Features approach also showed almost no degradation in performance against paraphrased samples\. The ensemble combining Text Features, RoBERTa, and Binoculars achieved the highest initial F1 score of0\.80610\.8061but experienced a notable drop in performance when faced with paraphrased samples, decreasing to0\.67160\.6716\. These findings highlight the varying degrees of resilience among different approaches to machine\-generated text detection\. RoBERTa’s robustness suggests that its language understanding capabilities allow it to detect AI\-generated text even after paraphrasing\. The significant drop in Binoculars’ performance indicates that its cross\-perplexity approach may be more sensitive to changes in text structure and wording introduced by paraphrasing\.

Table 1:F1 score drop per ensemble![Refer to caption](https://arxiv.org/html/2605.14240v1/degradation_plot.png)Figure 6:F1 score comparison and degradation

## 5Discussion

### 5\.1Analysis of Results

As demonstrated by our Results, we introduced a Cohesive Testing Framework \(CTF\) for classifying text as human\- versus machine\-written\. Our system streamlines the ensembling process by feeding the document input into three detectors – Binoculars, Text Features, and RoBERTa, which are then stacked and evaluated by our meta\-learner, Random Forest \(as demonstrated in Fig[1](https://arxiv.org/html/2605.14240#S2.F1)\)\. Our method employs 7 ways to combine 3 modules and make cross\-comparisons, which allows for 1\-to\-1 comparisons between performance of modules\. Our ensemble method proved significant information gain which outperforms many SOTA detectors\. Namely, it would place us 4th on the COLING2025 Workshop leaderboard\.

Our second main result was our finding that the highest performing AI detectors had the worst results when it came to paraphrasing attacks\. In fact, any ensemble that used Binoculars saw a significant decrease in F1 score\. This is particularly interesting, as it reflects more generally “The Bitter Lesson” paradox – it seems that for every interpretable training\-free method there is a better black\-box approach\.

### 5\.2Future Work

We suggest future works to build off our model by addressing the limitations we have laid out on the following page, as well as evaluating the detectors we looked at on different bypassing attacks, not only paraphrasing\. Additionally, the methods evaluated were not tested for out\-of\-distribution prompts\. Hense, accounting for this may add to a more comprehensive review of SOTA detectors\. Sentiment analysis has been shown to be distinguishable between human and machine\-written text, thus including this as a feature may contribute to some interesting results as well\.

## 6Conclusion

We believe that the tradeoff between performance and resilience is significant enough to become a leading theme in the AI\-detection community\. For example, the reported high performance of Binoculars on flagging Machine\-generated text has suffered the most drastic loss under paraphrasing attacks\. Under our testing framework, we also reaffirmed the significant information gain provided by the stacking of multiple detectors\.

## Acknowledgments

We thank the NU CS Microgrant for the computational units provided and Prof\. Wood\-Doughty for guidance and consultation on statistical analysis\. The study resulting in this paper was assisted by a Conference Travel Grant from the Office of Undergraduate Research administered by Northwestern University’s Office of the Provost\. However, the conclusions, opinions, and other statements in this paper are the author’s and not necessarily those of the sponsoring institution\.

## Limitations

Our paraphrased dataset has200200entries, as we were unable to gain API access to the platform we used\. Thus, although statistically significant, it should be important to replicate our results with a larger dataset\. Additionally, we only tested paraphrasing generated by GPTInf, which may not capture the maximum extent of paraphrasing attack capabilities\.

## Ethics Statement

When ChatGPT was released in 2022, it was widely unheard of and thus not largely anticipated, but within a short time frame, its popularity surged\. The world had not been expecting such a capable and easily accessible system, and thus its use in academic settings by students, across the internet by scammers, and in almost every practical field by workers, skyrocketed within a very brief amount of time\. As a result, the consequences of such wide\-spread AI use have not been thoroughly accounted for, and recent studies of its very real and threatening possible repercussions have only begun to be released\. It is then instrumental to first, study the effects of large\-spread AI use, and following this, develop methods that can detect the use of AI, namely in writing\.

The use of deepfakes have become increasingly prevalent in recent years\.Trandabăț and Gifu \([2023](https://arxiv.org/html/2605.14240#bib.bib22)\)investigated and assessed the threat of AI being used to generate deepfakes on a mass scale to be then published across the internet\. Google published the DeepDream algorithm in 2015, which used a convolutional neural network, trained on millions of images, to first identify objects within images, and then using these patterns create an image corresponding to a requested object \(for instance, an animal\) from memory\(Miller,[2020](https://arxiv.org/html/2605.14240#bib.bib23)\)\. Although the images that this network could produce were far from accurate and often combined elements of different objects from its training data, the release of DeepDream instantly sparked a race to use this technology and create something more powerful, as this was the first time deep learning was used to generate images from scratch\. Soon, more models and algorithms were developed, which were more advanced, with time, shrinking the gap between human recognizability of what is evidently machine\-generated in comparison to human\-created\. In their paper,Trandabăț and Gifu \([2023](https://arxiv.org/html/2605.14240#bib.bib22)\)use this background to focus on the present\-day role of AI across the internet, notably what is commonly referred to as “fake news”\. They test a few classifiers on both human and AI\-generated fake news, including RoBERTa\. They find that the true positive rate of AI\-detector models, such as RoBERTa, on AI\-generated fake news is only 3% higher than when run on human\-generated fake news, thus making AI\-generated fake news very difficult to recognize and highly useful for publishing false information online\.

In 2024, a German magazineDie Aktuellepublished an interview with a famous Formula One driver, Michael Schumacher, which was created entirely by the AI chatbot, Character\.ai, upon which the magazine was sued by Schumacher’s family\(ESPN News Services,[2024](https://arxiv.org/html/2605.14240#bib.bib24)\)\. Schumacher has been out of the image of the public eye for almost a decade due to a brain\-injury following a sports accident\. His family has taken immense action to keep his life post\-accident in private, thus the release of this article resulted in great turmoil on the family and misled readers all around the world\.

Overall, the consequences of fake news becoming prevalent can be unimaginably dangerous\. In South Korea, AI has been widely used to generate ads containing falsified information and promote the listing of fraudulent drugs and hormonal therapies for sale to the public on the internet\(Park,[2024](https://arxiv.org/html/2605.14240#bib.bib25)\)\. Because the sale of these treatments over\-the\-counter have not been government\-approved, many of the drugs listed have not been properly studied, meaning the health consequences that may arise from them are unclear, which is critically unsafe\. We thus strongly emphasize the need for reliability in AI text detection, highlighting the absolute necessity for AI text detectors that are able tobypassbypassers, in order to combat these problems addressed above and promote transparency across the Internet, in all fields and aspects\.

## References

- H\. Abburi, M\. Suesserman, N\. Pudota, B\. Veeramani, E\. Bowen, and S\. Bhattacharya \(2023\)Generative ai text classification using ensemble llm approaches\.External Links:2309\.07755,[Link](https://arxiv.org/abs/2309.07755)Cited by:[§2\.3](https://arxiv.org/html/2605.14240#S2.SS3.p1.1)\.
- S\. Chakraborty, A\. S\. Bedi, S\. Zhu, B\. An, D\. Manocha, and F\. Huang \(2023\)On the possibilities of ai\-generated text detection\.External Links:2304\.04736,[Link](https://arxiv.org/abs/2304.04736)Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p4.1)\.
- R\. Corizzo and S\. Leal\-Arenas \(2023\)One\-class learning for ai\-generated essay detection\.Applied Sciences13\(13\)\.External Links:[Link](https://www.mdpi.com/2076-3417/13/13/7901),ISSN 2076\-3417,[Document](https://dx.doi.org/10.3390/app13137901)Cited by:[§3](https://arxiv.org/html/2605.14240#S3.p4.1)\.
- H\. Desaire, A\. E\. Chua, M\. Isom, R\. Jarosova, and D\. Hua \(2023\)Distinguishing academic science writing from humans or chatgpt with over 99% accuracy using off\-the\-shelf machine learning tools\.Cell Reports Physical Science4\(6\),pp\. 101426\.Note:Epub 2023 Jun 7External Links:[Document](https://dx.doi.org/10.1016/j.xcrp.2023.101426),[Link](https://doi.org/10.1016/j.xcrp.2023.101426),ISSN 2666\-3864Cited by:[§2\.2](https://arxiv.org/html/2605.14240#S2.SS2.p1.1)\.
- F\. R\. Elali and L\. N\. Rachid \(2023\)AI\-generated research paper fabrication and plagiarism in the scientific community\.Patterns4\(3\),pp\. 100706\.External Links:ISSN 2666\-3899,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.patter.2023.100706),[Link](https://www.sciencedirect.com/science/article/pii/S2666389923000430)Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p1.1)\.
- ESPN News Services \(2024\)Schumacher family awarded €200k in compensation over ’ai interview’\.ESPN\.External Links:[Link](https://www.espn.com/f1/story/_/id/40197073/schumacher-family-awarded-200k-compensation-ai-interview)Cited by:[Ethics Statement](https://arxiv.org/html/2605.14240#Sx3.p3.1)\.
- D\. Fanelli \(2009\)How many scientists fabricate and falsify research? a systematic review and meta\-analysis of survey data\.PLoS One4\(5\),pp\. e5738\.Note:Competing Interests: The author has declared that no competing interests exist\.External Links:[Document](https://dx.doi.org/10.1371/journal.pone.0005738),ISSN 1932\-6203,[Link](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0005738)Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p1.1)\.
- C\. A\. Gao, F\. M\. Howard, N\. S\. Markov, E\. C\. Dyer, S\. Ramesh, Y\. Luo, and A\. T\. Pearson \(2022\)Comparing scientific abstracts generated by chatgpt to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2022.12.23.521610),[Link](https://www.biorxiv.org/content/early/2022/12/27/2022.12.23.521610),https://www\.biorxiv\.org/content/early/2022/12/27/2022\.12\.23\.521610\.full\.pdfCited by:[§1](https://arxiv.org/html/2605.14240#S1.p2.1)\.
- \[9\]GPTInf: ai content detection bypass tool\.Note:[https://www\.gptinf\.com/](https://www.gptinf.com/)Accessed: 2025\-01\-31Cited by:[§3](https://arxiv.org/html/2605.14240#S3.p7.5)\.
- P\. Gryka, K\. Gradoń, M\. Kozłowski, M\. Kutyła, and A\. Janicki \(2024\)Detection of ai\-generated emails \- a case study\.InProceedings of the 19th International Conference on Availability, Reliability and Security,ARES ’24,New York, NY, USA\.External Links:ISBN 9798400717185,[Link](https://doi.org/10.1145/3664476.3670465),[Document](https://dx.doi.org/10.1145/3664476.3670465)Cited by:[§3](https://arxiv.org/html/2605.14240#S3.p4.1)\.
- A\. Hans, A\. Schwarzschild, V\. Cherepanova, H\. Kazemi, A\. Saha, M\. Goldblum, J\. Geiping, and T\. Goldstein \(2024\)Spotting llms with binoculars: zero\-shot detection of machine\-generated text\.External Links:2401\.12070,[Link](https://arxiv.org/abs/2401.12070)Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p4.1)\.
- X\. Hu, P\. Chen, and T\. Ho \(2023\)RADAR: robust ai\-text detection via adversarial learning\.External Links:2307\.03838,[Link](https://arxiv.org/abs/2307.03838)Cited by:[§2\.4](https://arxiv.org/html/2605.14240#S2.SS4.p2.1)\.
- K\. Krishna, Y\. Song, M\. Karpinska, J\. Wieting, and M\. Iyyer \(2023\)Paraphrasing evades detectors of ai\-generated text, but retrieval is an effective defense\.External Links:2303\.13408,[Link](https://arxiv.org/abs/2303.13408)Cited by:[§2\.4](https://arxiv.org/html/2605.14240#S2.SS4.p1.1)\.
- R\. Kumar and M\. Mindzak \(2024\)Who wrote this? detecting artificial intelligence–generated text from human\-written text\.Canadian Perspectives on Academic Integrity7\(1\)\.External Links:[Link](https://journalhosting.ucalgary.ca/index.php/ai/article/view/77675),[Document](https://dx.doi.org/10.55016/ojs/cpai.v7i1.77675)Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p2.1)\.
- Y\. Liu, M\. Ott, N\. Goyal, J\. Du, M\. Joshi, D\. Chen, O\. Levy, M\. Lewis, L\. Zettlemoyer, and V\. Stoyanov \(2019\)RoBERTa: A robustly optimized BERT pretraining approach\.CoRRabs/1907\.11692\.External Links:[Link](http://arxiv.org/abs/1907.11692),1907\.11692Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p4.1),[§3](https://arxiv.org/html/2605.14240#S3.p2.5)\.
- A\. I\. Miller \(2020\)DeepDream: how alexander mordvintsev excavated the computer’s hidden layers\.The MIT Press Reader\.External Links:[Link](https://thereader.mitpress.mit.edu/deepdream-how-alexander-mordvintsev-excavated-the-computers-hidden-layers/)Cited by:[Ethics Statement](https://arxiv.org/html/2605.14240#Sx3.p2.1)\.
- M\. K\. Mobin and M\. S\. Islam \(2025\)LuxVeri at genai detection task 1: inverse perplexity weighted ensemble for robust detection of ai\-generated text across english and multilingual contexts\.External Links:2501\.11914,[Link](https://arxiv.org/abs/2501.11914)Cited by:[§2\.3](https://arxiv.org/html/2605.14240#S2.SS3.p2.1)\.
- A\. Muñoz\-Ortiz, C\. Gómez\-Rodríguez, and D\. Vilares \(2024\)Contrasting linguistic patterns in human and llm\-generated news text\.Artificial Intelligence Review57\(10\),pp\. 265\.External Links:[Document](https://dx.doi.org/10.1007/s10462-024-10903-2),[Link](https://doi.org/10.1007/s10462-024-10903-2),ISSN 1573\-7462Cited by:[§2\.2](https://arxiv.org/html/2605.14240#S2.SS2.p1.1)\.
- H\. J\. Park \(2024\)The rise of generative artificial intelligence and the threat of fake news and disinformation online: perspectives from sexual medicine\.Investigative and Clinical Urology65\(3\),pp\. 199–201\.Note:EditorialExternal Links:[Document](https://dx.doi.org/10.4111/icu.20240015),[Link](https://doi.org/10.4111/icu.20240015),ISSN 2466\-054XCited by:[Ethics Statement](https://arxiv.org/html/2605.14240#Sx3.p4.1)\.
- M\. Perkins, J\. Roe, B\. H\. Vu, D\. Postma, D\. Hickerson, J\. McGaughran, and H\. Q\. Khuat \(2024\)Simple techniques to bypass genai text detectors: implications for inclusive education\.International Journal of Educational Technology in Higher Education21\(1\),pp\. 53\.External Links:[Document](https://dx.doi.org/10.1186/s41239-024-00487-w),[Link](https://doi.org/10.1186/s41239-024-00487-w),ISSN 2365\-9440Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p3.1),[§2\.4](https://arxiv.org/html/2605.14240#S2.SS4.p2.1)\.
- R\. Phogat, B\. C\. Manjunath, B\. Sabbarwal, A\. Bhatnagar, Reena, and D\. Anand \(2023\)Misconduct in biomedical research: a meta\-analysis and systematic review\.Journal of International Society of Preventive & Community Dentistry13\(3\),pp\. 185–193\.Note:Copyright © 2023 Journal of International Society of Preventive and Community DentistryExternal Links:[Document](https://dx.doi.org/10.4103/jispcd.JISPCD%5F220%5F22),ISSN 2231\-0762,[Link](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10411296/)Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p1.1)\.
- D\. Trandabăț and D\. Gifu \(2023\)Discriminating ai\-generated fake news\.Procedia Computer Science225,pp\. 3822–3831\.Note:27th International Conference on Knowledge Based and Intelligent Information and Engineering Sytems \(KES 2023\)External Links:ISSN 1877\-0509,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.procs.2023.10.378),[Link](https://www.sciencedirect.com/science/article/pii/S1877050923015363)Cited by:[Ethics Statement](https://arxiv.org/html/2605.14240#Sx3.p2.1)\.
- V\. Verma, E\. Fleisig, N\. Tomlin, and D\. Klein \(2024\)Ghostbuster: detecting text ghostwritten by large language models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 1702–1717\.External Links:[Link](https://aclanthology.org/2024.naacl-long.95/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.95)Cited by:[§2\.1](https://arxiv.org/html/2605.14240#S2.SS1.p1.1)\.
- Y\. Wang, A\. Shelmanov, J\. Mansurov, A\. Tsvigun, V\. Mikhailov, R\. Xing, Z\. Xie, J\. Geng, G\. Puccetti, E\. Artemova,et al\.\(2025\)GenAI content detection task 1: english and multilingual machine\-generated text detection: ai vs\. human\.arXiv preprint arXiv:2501\.11012\.Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p4.1),[§2\.3](https://arxiv.org/html/2605.14240#S2.SS3.p2.1),[§3](https://arxiv.org/html/2605.14240#S3.p1.1)\.
- J\. Wu, S\. Yang, R\. Zhan, Y\. Yuan, D\. F\. Wong, and L\. S\. Chao \(2024\)A survey on llm\-generated text detection: necessity, methods, and future directions\.External Links:2310\.14724,[Link](https://arxiv.org/abs/2310.14724)Cited by:[§1](https://arxiv.org/html/2605.14240#S1.p2.1)\.

## Appendix AAppendix

#### A\.0\.1Dataset

The training dataset contained a total of 610k entries from HC3, M4GT, and MAGE\. The test dataset contained a total of 74k entries from CUDRT, IELTS, NLPeer, PeerSum, and MixSet\. We replicated 3 methods as well as their different ensembles over the Random Forest classifier and evaluated their performance on the MGTD testing dataset\.

#### A\.0\.2Quantization Effect

![Refer to caption](https://arxiv.org/html/2605.14240v1/quantization_effect.png)Figure 7:![Refer to caption](https://arxiv.org/html/2605.14240v1/colorful.png)Figure 8:Quantization in machine learning is the process of reducing the precision of numerical values, typically converting floating point numbers to lower\-bit representations, to decrease the model size and improve computational efficiency\. In our experiments, we quantized HuggingFace “tiiuae/falcon\-7b” to replicate the paper\. The typical degradation effect was about 2% \(Fig[7](https://arxiv.org/html/2605.14240#A1.F7)\) and it was diminishing as context was increasing\. This is unexpected because usually degradation effects for other tasks would be stronger\. It took around 27 GB of RAM to run “tiiuae/falcon\-7b” and “tiiuae/falcon\-7b\-instruct” and 11 GB for 4\-bit quantization of those models\. We conclude that the marginal improvement of the F1 score is unimportant compared to the doubled Carbon Footprint\. While non\-quantized versions achieve better results, the marginal accuracy improvement must be weighed against the significantly higher computational requirements, particularly in resource\-constrained environments\.

![Refer to caption](https://arxiv.org/html/2605.14240v1/multi.png)Figure 9:In Fig[8](https://arxiv.org/html/2605.14240#A1.F8), while some models are more resilient to Binocular detection \(gemini1\.5\) than others \(gpt4o\), the trend is repeated for all models\. Context window size significantly impacts detection accuracy, with substantial improvements observed as the window expands from 128 to 256 tokens\. The optimal range lies between 256\-512 tokens, though performance gains diminish notably beyond 300 tokens\. The maximum accuracy peaks at approximately 0\.80 for top\-performing models at 512 tokens\. Non\-quantized models consistently demonstrate superior accuracy compared to their quantized counterparts, with approximately 2% better performance\.

\\rowcolor\[gray\]0\.9Module 1Module 2p\-valueF1 \(M1\)F1 M1F1 DiffHigher F1BinoTF \+ Bino \+ RoBERTa0\.01660\.55330\.6716\-0\.1183TF \+ Bino \+ RoBERTaTFBino0\.13100\.66820\.55330\.1149TFBinoRoBERTa0\.27440\.55330\.6594\-0\.1061RoBERTaBino \+ RoBERTaTF \+ Bino \+ RoBERTa0\.04780\.57650\.6716\-0\.0950TF \+ Bino \+ RoBERTaTFBino \+ RoBERTa0\.27830\.66820\.57650\.0917TFBinoTF \+ RoBERTa0\.23740\.55330\.6368\-0\.0835TF \+ RoBERTaRoBERTaBino \+ RoBERTa0\.43970\.65940\.57650\.0829RoBERTaBinoTF \+ Bino0\.06880\.55330\.6202\-0\.0669TF \+ BinoTF \+ RoBERTaBino \+ RoBERTa0\.39240\.63680\.57650\.0603TF \+ RoBERTaTF \+ BinoTF \+ Bino \+ RoBERTa0\.28090\.62020\.6716\-0\.0514TF \+ Bino \+ RoBERTaTFTF \+ Bino0\.54930\.66820\.62020\.0480TFTF \+ BinoBino \+ RoBERTa0\.23870\.62020\.57650\.0436TF \+ BinoRoBERTaTF \+ Bino0\.45150\.65940\.62020\.0393RoBERTaTF \+ RoBERTaTF \+ Bino \+ RoBERTa0\.06230\.63680\.6716\-0\.0348TF \+ Bino \+ RoBERTaTFTF \+ RoBERTa0\.45050\.66820\.63680\.0314TFBinoBino \+ RoBERTa0\.37510\.55330\.5765\-0\.0232Bino \+ RoBERTaRoBERTaTF \+ RoBERTa0\.53890\.65940\.63680\.0226RoBERTaTF \+ BinoTF \+ RoBERTa0\.48530\.62020\.6368\-0\.0166TF \+ RoBERTaRoBERTaTF \+ Bino \+ RoBERTa0\.10970\.65940\.6716\-0\.0121TF \+ Bino \+ RoBERTaTFRoBERTa0\.43350\.66820\.65940\.0087TFTFTF \+ Bino \+ RoBERTa0\.37190\.66820\.6716\-0\.0034TF \+ Bino \+ RoBERTaTable 2:Statistical Comparison of Pre\-Attack F1 Scores Across Different Module Combinations \(n\_bootstrap =50005000, n\_subset =7300073000\)\\rowcolor\[gray\]0\.9Module 1Module 2p\-valueF1 \(M1\)F1 M1F1 DiffHigher F1BinoTF \+ Bino \+ RoBERTa0\.01260\.55330\.6716\-0\.1183TF \+ Bino \+ RoBERTaTFBino0\.13630\.66820\.55330\.1149TFBinoRoBERTa0\.26750\.55330\.6594\-0\.1061RoBERTaBino \+ RoBERTaTF \+ Bino \+ RoBERTa0\.05150\.57650\.6716\-0\.0950TF \+ Bino \+ RoBERTaTFBino \+ RoBERTa0\.27060\.66820\.57650\.0917TFBinoTF \+ RoBERTa0\.22550\.55330\.6368\-0\.0835TF \+ RoBERTaRoBERTaBino \+ RoBERTa0\.43380\.65940\.57650\.0829RoBERTaBinoTF \+ Bino0\.06820\.55330\.6202\-0\.0669TF \+ BinoTF \+ RoBERTaBino \+ RoBERTa0\.40970\.63680\.57650\.0603TF \+ RoBERTaTF \+ BinoTF \+ Bino \+ RoBERTa0\.28450\.62020\.6716\-0\.0514TF \+ Bino \+ RoBERTaTFTF \+ Bino0\.53740\.66820\.62020\.0480TFTF \+ BinoBino \+ RoBERTa0\.23110\.62020\.57650\.0436TF \+ BinoRoBERTaTF \+ Bino0\.44250\.65940\.62020\.0393RoBERTaTF \+ RoBERTaTF \+ Bino \+ RoBERTa0\.06350\.63680\.6716\-0\.0348TF \+ Bino \+ RoBERTaTFTF \+ RoBERTa0\.45810\.66820\.63680\.0314TFBinoBino \+ RoBERTa0\.37380\.55330\.5765\-0\.0232Bino \+ RoBERTaRoBERTaTF \+ RoBERTa0\.54200\.65940\.63680\.0226RoBERTaTF \+ BinoTF \+ RoBERTa0\.49060\.62020\.6368\-0\.0166TF \+ RoBERTaRoBERTaTF \+ Bino \+ RoBERTa0\.10570\.65940\.6716\-0\.0121TF \+ Bino \+ RoBERTaTFRoBERTa0\.44610\.66820\.65940\.0087TFTFTF \+ Bino \+ RoBERTa0\.38260\.66820\.6716\-0\.0034TF \+ Bino \+ RoBERTaTable 3:Statistical Comparison of Post\-Attack F1 Scores Across Different Module Combinations \(n\_bootstrap =50005000, n\_subset =402402\)