EditLens: Quantifying the extent of AI editing in text (2025)

# EditLens: Quantifying the Extent of AI Editing in Text
Source: [https://arxiv.org/html/2510.03154](https://arxiv.org/html/2510.03154)
Katherine Thai1,2Bradley Emi1Elyas Masrour1Mohit Iyyer3 1Pangram Labs2University of Massachusetts Amherst3University of Maryland, College Park

###### Abstract

A significant proportion of queries to large language models ask them to*edit*user\-provided text, rather than generate new text from scratch\. While previous work focuses on detecting fully AI\-generated text, we demonstrate that AI\-edited text is distinguishable from human\-written and AI\-generated text\. First, we propose using lightweight similarity metrics to quantify the magnitude of AI editing present in a text given the original human\-written text and validate these metrics with human annotators\. Using these similarity metrics as intermediate supervision, we then trainEditLens, a regression model that predicts the amount of AI editing present within a text\. Our model achieves state\-of\-the\-art performance on both binary \(F1=94\.7%\) and ternary \(F1=90\.4%\) classification tasks in distinguishing human, AI, and mixed writing\. Not only do we show that AI\-edited text can be detected, but also that the degree of change made by AI to human writing can be detected, which has implications for authorship attribution, education, and policy\. Finally, as a case study, we use our model to analyze the effects of AI\-edits applied by Grammarly, a popular writing assistance tool\. To encourage further research, we commit to publicly releasing our dataset and models\.

## 1Introduction

Large language models \(LLMs\) generate text that is difficult to distinguish from human writing, enabling malicious applications such as academic plagiarism and fake review farms, thus motivating the need for accurate AI detection\. While existing detectors frame the task as binary classification \(fully human vs\. fully AI\-generated\), mainstream LLM usage increasingly involves*co\-writing*, where LLMs are used for editing and brainstorming via services like Grammarly,111[https://www\.grammarly\.com/](https://www.grammarly.com/)Sudowrite,222[https://sudowrite\.com/](https://sudowrite.com/)or Google Docs’ Gemini integration\. In fact, a recent OpenAI study of over 1M ChatGPT conversations\(Chatterji et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib6)\)shows that “about two\-thirds of all Writing messages ask ChatGPT to modify user text \(editing, critiquing, translating, etc\.\) rather than creating new text from scratch\.” Binary AI detection systems are not well\-suited to detect such mixed\-authorship texts: for example,Saha & Feizi \([2025](https://arxiv.org/html/2510.03154v1#bib.bib37)\)find that binary detectors often flag AI\-polished text as AI\-generated, limiting their utility in situations where light AI editing is acceptable but fully AI\-generated text is not\.

In this paper, we developEditLens, the first AI detector that estimates the extent of AI editing in a text as a continuous score\. Previous work on detecting mixed AI and human text has treated the task as either a boundary detection problem\(Kushnareva et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib19); Lei et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib20)\), a sentence\-wise classification task\(Wang et al\.,[2023](https://arxiv.org/html/2510.03154v1#bib.bib42)\), or a ternary classification problem between human, AI, and mixed text\(Abassy et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib1); Wang et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib43)\)\. However, modern collaborative editing involves layered revisions, suggestions, and refinements that blur traditional notions of authorship, making it challenging to definitively attribute specific segments to either human or AI authors and rendering boundary detection and sentence\-level tasks ill\-posed\. Although the ternary classification approach does not require assigning direct authorship to discrete segments, it is unable to quantify the degree or the magnitude of AI editing: Was the text lightly edited for spelling and grammar, or completely rewritten and restructured? Rather than classifying a text category, our modeldirectly regresses a score that indicates the degree of AI involvement in the production of the text as a whole\.

Our contributions are the following:

1. 1\.We introduce a comprehensive dataset spanning a full taxonomy of AI\-edits to human\-written texts\.
2. 2\.We quantify the amount of AI editing applied to each text via lightweight similarity metrics, and validate that the similarity metrics correlate with the judgments of expert human annotators trained to detect AI writing styles\.
3. 3\.We use these similarity metrics to finetune a regression head on an open\-source large language model to detect the amount of AI\-editing present given only the edited text\.
4. 4\.When converted from a regression model to a binary or ternary classification model, we show that our model,EditLens, achieves state of the art performance, outperforming the best binary classifiers by 8%, and outperforming the best ternary classifiers by 16% \(macro\-F1\)\.
5. 5\.We also show that unlike the discrete classifiers, the regression model is able to show nuance in progressively classifying more intense edits with higher scores, with case studies on APT\-Eval, Beemo, and Grammarly\.

Our findings have wide\-ranging implications for AI text detection policy\. By enabling measurement for the level of AI involvement, more flexible policies acceptable usage of generative AI models can be consistently enforced\. Furthermore, our work can help mitigate false positives, a critical limitation of existing binary AI text classifiers\. With the ability to control the amount of AI editing allowed, a much lower false positive rate can be achieved under the policy cap framework suggested byJabarian & Imas \([2025](https://arxiv.org/html/2510.03154v1#bib.bib15)\)for implementation in high\-stakes settings such as academic integrity\.

![Refer to caption](https://arxiv.org/html/2510.03154v1/figures/hero.png)Figure 1:AI edits exist on a continuous spectrum from fully human written to fully AI generated\. Here we show three versions of the same human\-written text after different edits have been applied by an LLM alongside the cosine distance between the edited text and the fully human text\. Texts have been truncated for space\. “Fix any mistakes,” the most mild edit according to cosine distance, results in a text with only spelling and grammar errors corrected, while “Make it more descriptive” closely adheres to the ideas in the human\-written text while substantially rewriting it\.
## 2Quantifying AI Edit Magnitude

![Refer to caption](https://arxiv.org/html/2510.03154v1/x2.png)Figure 2:Examples of heterogeneous and homogeneous mixed authorship texts\. In heterogeneous mixed text, authorship of each token is clearly attributable\. But in homogeneous mixed text, the human\-originated ideas are clearly present in each rewritten sentence by the model, making it impossible to assign binary labels of authorship to any word or sentence\.### 2\.1Homogeneous vs\. Heterogeneous Mixed Authorship

To better motivate our work, we first introduce the concepts ofheterogeneousandhomogeneousmixed authorship texts\.

In the heterogeneous case, authorship of each segment of text can be directly attributed to a human or AI\. An example of this is a situation where a human writes one paragraph and asks the AI to write the following paragraph\. In cases like this, there exist one or more boundaries between human and AI segments\. One can create token\-level labels for heterogeneous mixed texts: every token was authored by either human or AI\. Heterogeneous mixed text detection \(also called fine\-grained AI text detection\) has been previously studied byKushnareva et al\. \([2024](https://arxiv.org/html/2510.03154v1#bib.bib19)\),Wang et al\. \([2023](https://arxiv.org/html/2510.03154v1#bib.bib42)\), andLei et al\. \([2025](https://arxiv.org/html/2510.03154v1#bib.bib20)\)\.

In the homogeneous case, authorship is entangled by the editing process\. An example of this is a situation where a human writes a paragraph and asks an AI to paraphrase it\. Even if AI replaces every word in the paragraph with a synonym, authorship is still mixed\. As such, token\-level binary labels are insufficient measures of authorship in this case, as both parties have provided input throughout the entire document\. Despite its increasing prevalence, homogeneous mixed AI text is understudied, and we focus the rest of the paper on detecting this kind of mixed text\.

### 2\.2Task Definition: Homogeneous Mixed Text

In many practical scenarios, a human\-written documentxxis subsequently edited to yield a new documentyy, where*multiple*sequential edits may have been performed by one or more agents \(human or AI\) in an indistinguishable fashion to produceyy\. Unlike the heterogeneous mixed\-text setting, where each segment is assumed to be authored wholly by either a human or an LLM, here authorship is*latent and entangled within the editing process*\. Our objective is not to attribute authorship, but to*predict the magnitude of change*betweenxxandyyaccording to a similarity metric that agrees with expert judgments of the magnitude of AI writing style and semantics\.

We model the edited text as the image of an editing operatorℰλ\\mathcal\{E\}\_\{\\lambda\}applied toxx:

y=ℰλ​\(x;z\),z∼p​\(z\),λ∈Λ,y\\;=\\;\\mathcal\{E\}\_\{\\lambda\}\(x;z\),\\qquad z\\sim p\(z\),\\quad\\lambda\\in\\Lambda,wherezzdenotes a \(latent\) sequence of micro\-edits \(insertions, deletions, substitutions, reorderings\) possibly performed by a mixture of editor types \(humans or AIs\) andλ\\lambdasummarizes an*edit intensity*\. In the homogeneous setting, the editor identity withinzzis unobserved and not required at training or inference time\. For simplicity, in this study, we focus on the case where a human text is edited in one pass by a single AI language model, but we also present results for multiple passes, and human\-edited AI text as case studies in generalization\.

#### Similarity\-driven target\.

Letsim:𝒳×𝒳→\[0,1\]\\mathrm\{sim\}:\\mathcal\{X\}\\times\\mathcal\{X\}\\to\[0,1\]be a fixed similarity functional\. We define a change magnitude functionalΔ:𝒳×𝒳→\[0,1\]\\Delta:\\mathcal\{X\}\\times\\mathcal\{X\}\\to\[0,1\]by a monotone transformation of similarity \(or distance\):

Δ​\(x,y\)=g​\(sim​\(x,y\)\),e\.g\.,g​\(s\)=1−s\\Delta\(x,y\)\\;=\\;g\\\!\\big\(\\mathrm\{sim\}\(x,y\)\\big\),\\quad\\text\{e\.g\.,\}\\quad g\(s\)=1\-swheresim\\mathrm\{sim\}is a nonnegative distance\.Δ​\(x,y\)=0\\Delta\(x,y\)=0for identical texts \(no edits\) and increases as heavier editing is applied to formyy\. We motivate the particular choice ofsim\\mathrm\{sim\}below via agreement with expert annotators’ perception of the amount of AI pervasiveness within a text, and it is assumed known during training and evaluation\.

#### Inference with edited text only\.

In most practical settings, only the edited documentyyis available at inference time\. We therefore learn a*single\-input*predictor that mapsyydirectly to a change magnitude without reconstructing or retrieving a sourcexx:

fθssi:𝒳→\[0,1\],Δ^​\(y\)=fθssi​\(y\)\.f\_\{\\theta\}^\{\\text\{ssi\}\}:\\mathcal\{X\}\\to\[0,1\],\\qquad\\hat\{\\Delta\}\(y\)=f\_\{\\theta\}^\{\\text\{ssi\}\}\(y\)\.Training remains*supervised*using pairs\{\(x\(i\),y\(i\)\)\}i=1N\\\{\(x^\{\(i\)\},y^\{\(i\)\}\)\\\}\_\{i=1\}^\{N\}only to compute targetsΔ\(i\)=Δ​\(x\(i\),y\(i\)\)\\Delta^\{\(i\)\}=\\Delta\(x^\{\(i\)\},y^\{\(i\)\}\); the model never conditions onxxat inference\. Concretely, we optimize

minθ⁡1N​∑i=1Nℒ​\(fθssi​\(y\(i\)\),Δ​\(x\(i\),y\(i\)\)\)\.\\min\_\{\\theta\}\\;\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathcal\{L\}\\\!\\Big\(f\_\{\\theta\}^\{\\text\{ssi\}\}\\big\(y^\{\(i\)\}\\big\),\\,\\Delta\\big\(x^\{\(i\)\},y^\{\(i\)\}\\big\)\\Big\)\.The Bayes\-optimal predictor for this objective is the conditional expectation

f⋆​\(y\)=𝔼​\[Δ​\(X,y\)∣Y=y\],f^\{\\star\}\(y\)\\;=\\;\\mathbb\{E\}\\\!\\left\[\\Delta\(X,y\)\\mid Y=y\\right\],but crucially we*do not*estimate this expectation via reconstruction ofxx\. Instead,fθssif\_\{\\theta\}^\{\\text\{ssi\}\}learns discriminatively fromyyalone, absorbing the necessary inductive biases \(e\.g\., lexical volatility, style drift, fluency/consistency cues\) to approximatef⋆f^\{\\star\}from labeled examples\.

For additional discussion of the precise differences between homogeneous and heterogeneous mixed detection formulations, see the Appendix\.

## 3Training a model to detect AI edits

![Refer to caption](https://arxiv.org/html/2510.03154v1/x3.png)Figure 3:EditLensarchitecture\. We generate fully AI and AI\-edited versions of human source texts, then use lightweight similarity metrics as intermediate supervision\. We partition the texts intonnbuckets according to supervised score and experiment with training both a regression model andnn\-way classification models, then using weight\-average decoding to obtain a numerical score\.### 3\.1Creating a Homogeneous Mixed Text Dataset

Because no dataset of homogeneous mixed AI\-generated text exists*at scale*, we create a training set for this task\.

We begin by collecting a source dataset of fully\-human and fully\-AI\-generated texts\. We select human\-written texts from prior to the release of large language models in 2022 from 4 domains: reviews from Amazon\(Zhang et al\.,[2015](https://arxiv.org/html/2510.03154v1#bib.bib45)\)and Google\(Li et al\.,[2022](https://arxiv.org/html/2510.03154v1#bib.bib21)\), creative writing samples from Reddit Writing Prompts\(Fan et al\.,[2018](https://arxiv.org/html/2510.03154v1#bib.bib12)\), general educational web articles from FineWeb\-EDU\(Lozhkov et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib22)\), and news articles from XSum\(Narayan et al\.,[2018](https://arxiv.org/html/2510.03154v1#bib.bib31)\)and CNN/DailyMail\(See et al\.,[2017](https://arxiv.org/html/2510.03154v1#bib.bib38)\)\. As a holdout domain to measure out\-of\-distribution performance, we also include the Enron email dataset\(Cohen,[2015](https://arxiv.org/html/2510.03154v1#bib.bib9)\)\.

Then, we generate an AI example corresponding to each human example following the synthetic mirroring procedure introduced inEmi & Spero \([2024](https://arxiv.org/html/2510.03154v1#bib.bib11)\)\. We use GPT\-4\.1, Claude 4 Sonnet, and Gemini 2\.5 Flash\. We also include Llama\-3\.3\-70B\-Instruct\-Turbo as a holdout LLM to measure performance on out\-of\-distribution LLMs\. Our final train, test, and val splits contain 60k, 6k, and 2\.4k examples respectively\. We estimate the cost of creating this dataset to be roughly $530\. Additional dataset summary statistics can be found in[Tables10](https://arxiv.org/html/2510.03154v1#A11.T10),[11](https://arxiv.org/html/2510.03154v1#A11.T11),[12](https://arxiv.org/html/2510.03154v1#A11.T12),[13](https://arxiv.org/html/2510.03154v1#A11.T13)and[14](https://arxiv.org/html/2510.03154v1#A11.T14)\.

### 3\.2Edit Prompts

We collected a set of editing prompts by first prompting ChatGPT 4o, Claude Sonnet 4, and Gemini 2\.5 Pro, then adding a small number of prompts written by the authors\. In total, we collected 303 editing prompts\. The full list of prompts and summary statistics about the categories and contributors can be found in TablesLABEL:tab:editing\_promptsand[9](https://arxiv.org/html/2510.03154v1#A11.T9)\. While this list of prompts is not exhaustive, it encompasses a significant coverage of the different ways that people use AI to edit texts\. We split this list of prompts into train, test, and validation splits so that the model cannot overfit to a particular set of prompts\.

### 3\.3Intermediate Supervision Metrics

We experiment with two methods for labeling the “difference”Δ​\(x,y\)\\Delta\(x,y\)in a text before and after AI editing\. The first is the cosine distance \(1 \- cosine similarity\) between the Linq\-Embed\-Mistral\(Choi et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib8)\)embeddings of the source text and the AI\-edited version\. We chose this embedding due to its strong all\-around performance on the MTEB benchmark\(Muennighoff et al\.,[2023](https://arxiv.org/html/2510.03154v1#bib.bib30)\)\.

The second is a precision\-based method similar to the embedding\-based ROUGE proposed byNg & Abrecht \([2015](https://arxiv.org/html/2510.03154v1#bib.bib32)\): given a minimum \(aa\) and maximum \(bb\) sequence length, we enumerate all phrases \(including overlaps\) of betweenaaandbbwords in the source and edited texts\. We compute the pairwise cosine similarity between phrases in the source and edited texts, then count the number of phrases in the edited text with a cosine similarity above a thresholdτ\\tauforanyphrase in the source text\. This count is divided by the total number of phrases in the edited text, making it a precision\-based metric\. We refer to this metric as thesoft n\-gramsscore throughout the paper\. Soft n\-grams reduces to n\-gram overlap between source and target whenτ=1\\tau=1\. We choose soft n\-grams because it expresses similarity when the AI editor replaces a phrase or word with a semantically similar one rather than requiring exact matching\. We note that this supervision metric is shortening\-invariant, i\.e\., simply deleting text from the source still yields a soft n\-grams score of 1\.

### 3\.4Human Agreement with Intermediate Supervision Metrics

How well do these automatic metrics actually capture the extent of AI editing in a text? To support our choice of intermediate supervision metric, we conduct a study that asks human annotators to compare two AI\-edited versions of the same text after findings byRussell et al\. \([2025](https://arxiv.org/html/2510.03154v1#bib.bib35)\)that humans are effective detectors of AI\-authored text\.

#### Task setup\.

Annotators are shown 3 texts side\-by\-side: a human writtensource textalongside 2

![Refer to caption](https://arxiv.org/html/2510.03154v1/figures/ai_polish.png)Figure 4:Distributions forEditLensand Pangram on the AI Polish dataset\(Saha & Feizi,[2025](https://arxiv.org/html/2510.03154v1#bib.bib37)\)\. Pangram overwhelmingly tends to predict a score of either 0 or 1, whileEditLenscaptures the increasing levels of AI polish applied to the texts\.AI\-edited versions of the source text\. The labeling interface can be seen in Figure[7](https://arxiv.org/html/2510.03154v1#A6.F7)\. Between the two AI texts, annotators are asked to select which text contains more AI edits\. Annotators may also answer that there is a “Tie,” i\.e\. both texts contain roughly the same amount of AI edits\. Annotators have the option to leave freeform comments on each task, but were not required to do so\. We recruited 3 annotators with extensive daily exposure to both human writing and AI\-generated texts\. Each annotator completed all 100 tasks in approximately 6 hours and was compensated at a rate of $30 \(USD\) per hour\.

#### Task generation procedure\.

We randomly sample 100 human\-written texts of between 50 and 300 words from our test set\. We then generate multiple AI\-edited versions of each source text using randomly assigned prompts until we have two AI\-edited versions that are within 15 words of the source text\. We impose this length restriction on the edited texts to encourage annotators to consider the actual text, rather than simply the length when selecting the version with more AI edits\.

#### Agreement with metrics\.

We report Krippendorff’sα\\alpha\(Krippendorff,[1980](https://arxiv.org/html/2510.03154v1#bib.bib17)\)for our 3 annotators and each of our two supervision metrics by treating each metric as a fourth annotator\. When considering human annotators’ ties as abstentions and designating the higher scoring text as the metric’s selection,α\\alpha= 0\.67 ± 0\.06 for cosine andα\\alpha= 0\.66 ± 0\.05 for soft n\-grams\. We also computeα\\alphawhen considering ties in Table[3](https://arxiv.org/html/2510.03154v1#A2.T3), but we note that no value is under 0\.48, indicating moderate agreement\.

### 3\.5Modeling Details

Using QLoRA\(Dettmers et al\.,[2023](https://arxiv.org/html/2510.03154v1#bib.bib10)\), we finetune models of between 3 and 24B parameters from the Mistral and Llama families\. We use QLoRA to sweep the widest possible range of sizes of base models to use as a backbone that fit in VRAM on a single GPU\. We leave other finetuning and modeling architecture choices to future work\. We experiment with directly training a regression head using MSE loss as well as training annn\-way classification model, then decoding the output to a score, using weighted\-average decoding rather than traditional argmax decoding\. Additional modeling details can be found in the Appendix\.

## 4Results

EditLensdemonstrates significantly more nuanced AI detection than existing classifiers through both quantitative metrics and qualitative analysis\. We report results for our bestEditLensmodel according to ternary classification metrics \(see Section[4\.3](https://arxiv.org/html/2510.03154v1#S4.SS3)and Table[2](https://arxiv.org/html/2510.03154v1#S4.T2)\) trained on soft n\-grams and cosine scored data after a hyperparameter sweep\. Both models have a Mistral Small \(24B\) backbone and were trained on a44\-way classification task333Additional results for different model sizes, model families, and values ofnnfornn\-way classification can be found in Tables[15](https://arxiv.org/html/2510.03154v1#A13.T15)\-[18](https://arxiv.org/html/2510.03154v1#A13.T18)\.

We compareEditLenswith several open\- and closed\-source AI detection baselines\. On the AI Polish dataset \(APT\-Eval\),EditLensachieves substantially stronger correlations with edit magnitude metrics compared to binary detectors \(correlation 0\.606\), markedly outperforming the best binary baseline Pangram \(correlation 0\.491\)\. This quantitative superiority is complemented by clear qualitative differences: while binary classifiers like Pangram predict scores clustered near 0 or 1,EditLensproduces a nuanced distribution that appropriately tracks increasing levels of AI polish from minor to major edits\. The model’s regression\-based approach enables it to achieve state\-of\-the\-art performance across evaluation paradigms, delivering 94\.0% accuracy in binary classification \(human vs\. any AI\) and 90\.2% accuracy in ternary classification \(human vs\. AI\-edited vs\. AI\-generated\), substantially outperforming existing binary and ternary detection methods\. Additionally,EditLensgeneralizes effectively outside its training distribution: to unseen prompts, LLMs, and domains, to human\-edited AI text in the BEEMO dataset, and to AI\-edited AI text as well as multi\-edited AI text\.

### 4\.1AI Polish Dataset

We first compare the performance on the AI Polish dataset \(APT\-Eval\) ofEditLensagainst the best\-performing binary AI classifier, Pangram\. APT\-Eval contains both degree\-based AI\-edited text, with 4 discrete categories \(extreme minor, minor, slight major, and major polish levels\), as well as percentage\-based AI\-edited text, where LLMs were asked to edit a certain percentage of the text, varying from 1\-75%\.

While there are no direct or exact labels, the score should generally monotonically increase as the amount of requested polish increases\. In Figure[4](https://arxiv.org/html/2510.03154v1#S3.F4), we qualitatively assess the distribution of the model prediction scores on the degree\-based edits\. We can see a clear difference between the behavior ofEditLensversus the behavior of Pangram\. Pangram almost always predicts a score very close to 0 or 1, whileEditLensis able to quantify the increasing levels of polish applied\. We show the equivalent distributions for percentage\-based polishing in the Appendix\.

Quantitatively, we also report the correlation value between theEditLenspredicted score and the similarity metrics between source and target provided by APT\-eval in Table[4](https://arxiv.org/html/2510.03154v1#A6.T4)\. ForEditLensand all binary classification baselines, we measure the Pearson correlation coefficient \(rr\) between the prediction scores and the semantic similarity \(\-0\.606\), Levenshtein distance \(0\.799\), and Jaccard distance \(0\.781\) metrics between the pre\-AI\-polished and post\-AI\-polished documents\. Stronger correlation values mean that the model is able to faithfully track edit magnitude across examples and assign higher scores as semantic similarity decreases \(and Levenshtein/Jaccard distances increase\), and lower scores when the edited text remains close to the source\.EditLensexhibits a significant correlation between these similarity metrics and its scores, while the binary AI detectors correlate less strongly with these metrics\.

### 4\.2Performance as a Binary Classifier

In the binary classification setting, how doesEditLenstreat mixed text? Different use cases may have different standards for what they consider an acceptable amount of AI\-generated text–a professor may allow the use of AI assistance for proofreading, but disallow fully AI\-generated essays\.

To measure the flexibility of our model and the baselines to be able to adjust to different sensitivity levels, we calibrate and compute the performance of each model on two settings: fully human\-written vs\. any AI\-edited or AI\-generated text, fully human\-written and AI\-edited text vs\. AI\-generated text\. Model accuracy and F1\-scores can be found in Table[1](https://arxiv.org/html/2510.03154v1#S4.T1)\. Notably,EditLensoutperforms our three binary baselines, FastDetectGPT, Binoculars, and Pangram, on our test set consisting of fully human\-written, fully AI\-generated, and AI\-edited texts\.

\(a\) Human vs\. Any AI

\(b\) Fully AI vs\. AI\-Edited \+ Human

Table 1:Accuracy and F1\-score on two binary classification tasks: \(a\) human vs\. any AI generated or edited texts and \(b\) fully AI\-generated texts vs\. AI\-edited and human texts\. Thresholds were calibrated using the val set\. “SNG” and “Cosine” denoteEditLenstrained with soft n\-grams supervised data and cosine score supervised data, respectively\.
### 4\.3Performance as a Ternary Classifier

To compare with categorical mixed AI detection models, we evaluate each model on three classes: human, AI\-generated, or AI\-edited\. To convert each binary classifier into a ternary classifier, we find two thresholds using the calibration procedure above on a held\-out validation set, optimizing the F1 score between the human/mixed and mixed/AI classes\. The decoding procedures for GPTZero and DetectAIve are detailed in the Appendix\.

Table 2:Ternary classification performance across different models\. Thresholds were calibrated using the validation set\. “Soft N\-Grams” and “Cosine” denoteEditLenstrained with soft n\-grams supervised data and cosine score supervised data, respectively\.
### 4\.4Out\-of\-Domain Performance

During dataset creation, we hold out both a model and a domain to test the ability of our model to generalize to out\-of\-distribution texts\. We created an OOD model test set of 3k examples with Llama\-3\.3\-70B\-Instruct\-Turbo generated and edited texts as well as an OOD domain test set using

![Refer to caption](https://arxiv.org/html/2510.03154v1/figures/multieditedtext.png)Figure 5:“Trajectory” ofEditLensscores after subsequent AI edits to a single text\. We can observe that the mean score predicted byEditLensafter each edit is monotonically increasing\.the Enron email dataset\(Cohen,[2015](https://arxiv.org/html/2510.03154v1#bib.bib9)\)as source texts, and measure the degradation in macro\-F1 score of our best model,EditLenswith cosine supervision\.

On the OOD domain dataset, macro\-F1 on the ternary classification task decreases from 0\.904 to 0\.866 \(\-0\.038\)\. On the OOD LLM dataset, macro\-F1 on the ternary classification task decreases from 0\.904 to 0\.850 \(\-0\.054\)\.

### 4\.5Performance on multi\-edited AI text

We also examine the case where multiple AI\-edits have been applied to a single piece of text\. We test our model on this case by applying a series of 5 edits to a piece of human\-written text and measuring theEditLensscore after each subsequent edit\. In Figure[5](https://arxiv.org/html/2510.03154v1#S4.F5), we show that for each edit, the mean score increases\.

### 4\.6Generalization to AI\-edited AI text

To ensureEditLensestimates the extent of AI\-editing rather than the presence of edits of any kind, we evaluate our detector’s mean score difference on AI\-edited, AI\-generated text\. We take synthetic mirrors of our original human dataset, considered to be ‘AI\-generated documents’ and edit them using our held\-out prompt set\. On a dataset size of n=412, the mean score difference for a single edit pass on an originally human text is 0\.38\. The mean score difference for a single edit pass on an originally AI text is \-0\.05\.

### 4\.7Generalization to Human\-edited AI text \(BEEMO\)

While the majority of our studies focus on AI\-edited human writing, we also evaluate the performance ofEditLenson human\-edited AI text using the BEEMO\(Artemova et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib3)\)dataset, which includes human expert\-edited versions of AI model outputs\. We find that the model adequately generalizes to human\-edited AI text\. The average decrease in score from the model output to the human\-edited model output is 0\.33 ± 0\.30, with the score decreasing after human\-editing in 88\.9% of the documents\. More details are presented in the Appendix\.

### 4\.8Case Study: Grammarly Edit Dataset

![Refer to caption](https://arxiv.org/html/2510.03154v1/x4.png)Figure 6:Distribution ofEditLensscores on dataset from Grammarly by edit instruction\.Grammarly444[https://www\.grammarly\.com/](https://www.grammarly.com/)is a popular subscription\-based AI writing assistant that allows users to edit text using both pre\-filled and custom prompts within a native word processor\. We manually collect a dataset of 1768 samples using 9 of the default prompts offered by Grammarly to simulate typical user queries for AI editing by sampling 197555Occasionally, Grammarly would abstain, leaving us with fewer than 197 \* 9 samples\.human\-written source texts and applying each of the 9 edits to them in the Grammarly web interface\. In Figure[6](https://arxiv.org/html/2510.03154v1#S4.F6), we present the distributions ofEditLensscores on examples from each editing instruction sorted by the median\. Perhaps counterintuitively,EditLensconsiders “Fix any mistakes” the most minor of all edits, while “Summarize this” and “Make it more detailed” are the most invasive edits\. In Figures[11](https://arxiv.org/html/2510.03154v1#A13.F11)and[10](https://arxiv.org/html/2510.03154v1#A13.F10)we show this is also true according to both the cosine and soft n\-grams scores of the examples\.

## 5Related Work

#### Binary AI\-Generated Text Detectors\.

Several works have explored the binary setting of distinguishing fully\-human from fully\-AI\-generated text\. DetectGPT\(Mitchell et al\.,[2023](https://arxiv.org/html/2510.03154v1#bib.bib29)\), FastDetectGPT\(Bao et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib4)\), DNA\-GPT\(Yang & Cheng,[2024](https://arxiv.org/html/2510.03154v1#bib.bib44)\), and Binoculars\(Hans et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib14)\)are training\-free approaches that leverage statistical properties of AI\-generated text to perform binary detection\. Ghostbusters\(Verma et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib41)\)is an open\-weight classifier trained on simple features from the text, while closed\-source classifiers such as GPTZero\(Tian & Cui,[2023](https://arxiv.org/html/2510.03154v1#bib.bib40)\)and Pangram\(Emi & Spero,[2024](https://arxiv.org/html/2510.03154v1#bib.bib11)\)have more recently emerged as accurate AI text classifiers\. All of the above methods operate in the*post\-hoc*setting, in contrast to work on watermarking\(Kirchenbauer et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib16)\)in which the model’s decoding algorithm is modified to enable detection\.

#### Heterogeneous Mixed Text Detection\.

As described above, previous work on mixed AI and human text detection focuses on the heterogeneous case: where distinct boundaries can be drawn between fully AI\-generated and fully human\-written segments\. Examples of these works include AI Boundary Detection with RoFT\(Kushnareva et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib19)\), SeqXGPT\(Wang et al\.,[2023](https://arxiv.org/html/2510.03154v1#bib.bib42)\), HaCo\-Det\(Su et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib39)\), and PALD\(Lei et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib20)\)\.

#### Categorical Mixed Text Detection\.

Alternatively, some previous work has instead focused on mixed text as an additional category or categories in addition to human and AI\. DetectAIve\(Abassy et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib1)\), HERO\(Wang et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib43)\), and GPTZero\(Tian & Cui,[2023](https://arxiv.org/html/2510.03154v1#bib.bib40)\)are all examples where mixed categories have been added\. We find the limitation of this approach is that the amount of editing cannot be quantified: all mixed text is treated as the same\.

#### Human\-Edited AI Text\.

In addition to our problem setting of AI\-edited human written text, there are also studies and datasets focusing on human\-edited AI\-generated text\. Beemo\(Artemova et al\.,[2024](https://arxiv.org/html/2510.03154v1#bib.bib3)\)is a benchmark focusing on expert\-edited AI text\. LAMP\(Chakrabarty et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib5)\)is a corpus of LLM\-generated paragraphs that have been improved by professional writers according to a defined taxonomy\.

#### Paraphrasers and Humanizers\.

Several previous works have studied the effects of automated paraphrasers\(Krishna et al\.,[2023](https://arxiv.org/html/2510.03154v1#bib.bib18); Russell et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib35); Sadasivan et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib36); Cheng et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib7)\)and “humanizers”\(Masrour et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib23)\)on how they degrade AI\-generated text\. We explore the effect of AI rewriting of AI outputs as it relates to our model in the results\.

## 6Conclusion

In this study, we introduce the task of continuous fine\-grained AI edit prediction, and show thatEditLens, based on simple embedding\-based supervision on a finetuned language model, significantly outperforms existing AI detection approaches\. By moving beyond binary or categorical detection frameworks, our method provides a more nuanced view of mixed\-authorship text, quantifying the magnitude of AI editing rather than simply flagging the presence of AI\-generated text\. This capability enables more flexible policy decisions around the use of generative AI\. We release our dataset and models to encourage future research in this area\.

## 7Ethics Statement

Our research involved using 3 human subjects to annotate the degree of AI\-editing present in a text\. We obtained informed consent from the subjects and fairly compensated them for their labor\. We commit to maintaining their privacy\.

Inaccurate AI detection software can cause harm as false accusations of AI misconduct can result in serious consequences, including emotional trauma, reputation damage, and undue punishments for academic misconduct\. We acknowledge that our model has a non\-zero error rate and its errors may result in such harms\. We commit to continuing to engage with the academic community to educate others on appropriately contextualizing and communicating the results of AI detection software\. We also commit to releasing the model for non\-commercial use only and responsibly vetting access to researchers and educators\.

We intend for our contribution to the research on AI detection to ultimately mitigate harm by providing a more nuanced picture of AI usage than binary AI detection classifiers\. The ability to calibrate the sensitivity level of the regression model is also a step towards mitigating the false positive rate and lowering the overall number of false accusations of AI misconduct\.

## 8Reproducibility Statement

## References

- Abassy et al\. \(2024\)Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ngoc Ta, Raj Vardhan Tomar, Bimarsha Adhikari, Saad El Dine Ahmed, Yuxia Wang, Osama Mohammed Afzal, Zhuohan Xie, Jonibek Mansurov, Ekaterina Artemova, Vladislav Mikhailov, Rui Xing, Jiahui Geng, Hasan Iqbal, Zain Muhammad Mujahid, Tarek Mahmoud, Akim Tsvigun, Alham Fikri Aji, Artem Shelmanov, Nizar Habash, Iryna Gurevych, and Preslav Nakov\.LLM\-DetectAIve: A Tool for Fine\-Grained Machine\-Generated Text Detection\.In Delia Irazu Hernandez Farias, Tom Hope, and Manling Li \(eds\.\),*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations*, pp\. 336–343, Miami, Florida, USA, November 2024\. Association for Computational Linguistics\.doi:10\.18653/v1/2024\.emnlp\-demo\.35\.URL[https://aclanthology\.org/2024\.emnlp\-demo\.35](https://aclanthology.org/2024.emnlp-demo.35)\.
- Anthropic \(2025\)Anthropic\.Introducing Claude 4 \(Claude Opus 4 and Claude Sonnet 4\)\.Anthropic Blog Post, May 2025\.URL[https://www\.anthropic\.com/news/claude\-4](https://www.anthropic.com/news/claude-4)\.
- Artemova et al\. \(2024\)Ekaterina Artemova, Jason Lucas, Saranya Venkatraman, Jooyoung Lee, Sergei Tilga, Adaku Uchendu, and Vladislav Mikhailov\.Beemo: Benchmark of Expert\-edited Machine\-Generated Outputs\.*arXiv preprint arXiv:2411\.04032*, 2024\.
- Bao et al\. \(2024\)Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang\.Fast\-DetectGPT: Efficient Zero\-Shot Detection of Machine\-Generated Text via Conditional Probability Curvature\.In*The Twelfth International Conference on Learning Representations \(ICLR\)*, 2024\.URL[https://arxiv\.org/abs/2310\.05130](https://arxiv.org/abs/2310.05130)\.
- Chakrabarty et al\. \(2025\)Tuhin Chakrabarty, Philippe Laban, and Chien\-Sheng Wu\.Can AI Writing Be Salvaged? Mitigating Idiosyncrasies and Improving Human\-AI Alignment in the Writing Process through Edits, 2025\.URL[https://arxiv\.org/abs/2409\.14509](https://arxiv.org/abs/2409.14509)\.
- Chatterji et al\. \(2025\)Aaron Chatterji, Thomas Cunningham, David J\. Deming, Zoe Hitzig, Christopher Ong, Carl Yan Shan, and Kevin Wadman\.How People Use ChatGPT\.Working Paper 34255, National Bureau of Economic Research, Cambridge, MA, September 2025\.URL[http://www\.nber\.org/papers/w34255](http://www.nber.org/papers/w34255)\.NBER Working Paper No\. 34255\.
- Cheng et al\. \(2025\)Yize Cheng, Vinu Sankar Sadasivan, Mehrdad Saberi, Shoumik Saha, and Soheil Feizi\.Adversarial Paraphrasing: A Universal Attack for Humanizing AI\-Generated Text, 2025\.URL[https://arxiv\.org/abs/2506\.07001](https://arxiv.org/abs/2506.07001)\.
- Choi et al\. \(2024\)Chanyeol Choi, Junseong Kim, Seolhwa Lee, Jihoon Kwon, Sangmo Gu, Yejin Kim, Minkyung Cho, and Jy\-yong Sohn\.Linq\-Embed\-Mistral Technical Report\.*arXiv preprint arXiv:2412\.03223*, December 2024\.15 pages\.
- Cohen \(2015\)William W\. Cohen\.Enron Email Dataset\.[https://www\.cs\.cmu\.edu/~enron/](https://www.cs.cmu.edu/~enron/), 2015\.Prepared by the CALO Project \(Cognitive Assistant that Learns and Organizes\)\. Originally made public by the Federal Energy Regulatory Commission\.
- Dettmers et al\. \(2023\)Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer\.QLORA: Efficient Finetuning of Quantized LLMs\.In*Proceedings of the 37th International Conference on Neural Information Processing Systems*, NIPS ’23, 2023\.
- Emi & Spero \(2024\)Bradley Emi and Max Spero\.Technical Report on the Pangram AI\-Generated Text Classifier, 2024\.URL[https://arxiv\.org/abs/2402\.14873](https://arxiv.org/abs/2402.14873)\.
- Fan et al\. \(2018\)Angela Fan, Mike Lewis, and Yann Dauphin\.Hierarchical Neural Story Generation\.*arXiv preprint arXiv:1805\.04833*, 2018\.
- Google \(2025\)Google\.Gemini 2\.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities\.Technical report, Google DeepMind, 2025\.URL[https://storage\.googleapis\.com/deepmind\-media/gemini/gemini\_v2\_5\_report\.pdf](https://storage.googleapis.com/deepmind-media/gemini/gemini_v2_5_report.pdf)\.Technical Report\.
- Hans et al\. \(2024\)Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein\.Spotting LLMs With Binoculars: Zero\-Shot Detection of Machine\-Generated Text, 2024\.
- Jabarian & Imas \(2025\)Brian Jabarian and Alex Imas\.Artificial Writing and Automated Detection\.[https://ssrn\.com/abstract=5407424](https://ssrn.com/abstract=5407424), September 2025\.Chicago Booth Research Paper Forthcoming\.
- Kirchenbauer et al\. \(2024\)John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein\.A watermark for large language models, 2024\.URL[https://arxiv\.org/abs/2301\.10226](https://arxiv.org/abs/2301.10226)\.
- Krippendorff \(1980\)Klaus Krippendorff\.*Content Analysis: An Introduction to Its Methodology*, chapter 12\.Sage Publications, Beverly Hills, CA, 1980\.
- Krishna et al\. \(2023\)Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer\.Paraphrasing Evades Detectors of AI\-Generated Text, but Retrieval is an Effective Defense\.*arXiv preprint arXiv:2303\.13408*, 2023\.
- Kushnareva et al\. \(2024\)Laida Kushnareva, Tatiana Gaintseva, German Magai, Serguei Barannikov, Dmitry Abulkhanov, Kristian Kuznetsov, Eduard Tulchinskii, Irina Piontkovskaya, and Sergey Nikolenko\.AI\-Generated Text Boundary Detection with RoFT\.In*1st Conference on Language Modeling \(COLM\)*, Philadelphia, United States, October 2024\.URL[https://openreview\.net/forum?id=kzzwTrt04Z](https://openreview.net/forum?id=kzzwTrt04Z)\.
- Lei et al\. \(2025\)Eric Lei, Hsiang Hsu, and Chun\-Fu Chen\.PaLD: Detection of Text Partially Written by Large Language Models\.In*Proceedings of the Thirteenth International Conference on Learning Representations*, 2025\.URL[https://openreview\.net/forum?id=rWjZWHYPcz](https://openreview.net/forum?id=rWjZWHYPcz)\.
- Li et al\. \(2022\)Jiacheng Li, Jingbo Shang, and Julian McAuley\.UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining\.In*Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pp\. 51–62\. Association for Computational Linguistics, 2022\.
- Lozhkov et al\. \(2024\)Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf\.FineWeb\-Edu: The Finest Collection of Educational Content, 2024\.URL[https://huggingface\.co/datasets/HuggingFaceFW/fineweb\-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu)\.
- Masrour et al\. \(2025\)Elyas Masrour, Bradley Emi, and Max Spero\.DAMAGE: Detecting Adversarially Modified AI\-Generated Text, 2025\.URL[https://arxiv\.org/abs/2501\.03437](https://arxiv.org/abs/2501.03437)\.
- Meta \(2024a\)Meta\.Llama 3\.1 8B: Open Foundation Model with 128k context\.Meta AI Announcement / Hugging Face Model Card, 2024a\.URL[https://huggingface\.co/meta\-llama/Llama\-3\.1\-8B](https://huggingface.co/meta-llama/Llama-3.1-8B)\.
- Meta \(2024b\)Meta\.Llama 3\.2: Small & Medium Vision\-Enabled Models \(3b, 11b, 90b\)\.Meta Connect 2024 Announcement, 2024b\.URL[https://ai\.meta\.com/blog/meta\-llama\-3\-2](https://ai.meta.com/blog/meta-llama-3-2)\.
- Meta \(2024c\)Meta\.Llama 3\.3 70B Instruct \(December 2024 Release\)\.Meta AI Announcement on X \(Twitter\) and Hugging Face Model Card, 2024c\.URL[https://huggingface\.co/meta\-llama/Llama\-3\.3\-70B\-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)\.
- Mistral \(2024\)Mistral\.Mistral NeMo: Our New Best Small Model \(12b, 128k context\)\.Mistral AI Blog, July 2024\.URL[https://mistral\.ai/news/mistral\-nemo](https://mistral.ai/news/mistral-nemo)\.
- Mistral \(2025\)Mistral\.Mistral Small 3: A 24b Latency\-Optimized Model \(Apache 2\.0\)\.Mistral AI Blog, January 2025\.URL[https://mistral\.ai/news/mistral\-small\-3](https://mistral.ai/news/mistral-small-3)\.
- Mitchell et al\. \(2023\)Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D\. Manning, and Chelsea Finn\.DetectGPT: Zero\-Shot Machine\-Generated Text Detection using Probability Curvature\.In*Proceedings of the 40th International Conference on Machine Learning*, ICML ’23, 2023\.URL[https://arxiv\.org/abs/2301\.11305](https://arxiv.org/abs/2301.11305)\.
- Muennighoff et al\. \(2023\)Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers\.MTEB: Massive Text Embedding Benchmark, 2023\.URL[https://arxiv\.org/abs/2210\.07316](https://arxiv.org/abs/2210.07316)\.
- Narayan et al\. \(2018\)Shashi Narayan, Shay B\. Cohen, and Mirella Lapata\.Don’t Give Me the Details, Just the Summary\! Topic\-Aware Convolutional Neural Networks for Extreme Summarization\.*arXiv*, abs/1808\.08745, 2018\.
- Ng & Abrecht \(2015\)Jun\-Ping Ng and Viktoria Abrecht\.Better Summarization Evaluation with Word Embeddings for ROUGE\.In Lluís Màrquez, Chris Callison\-Burch, and Jian Su \(eds\.\),*Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing*, pp\. 1925–1930, Lisbon, Portugal, September 2015\. Association for Computational Linguistics\.doi:10\.18653/v1/D15\-1222\.URL[https://aclanthology\.org/D15\-1222/](https://aclanthology.org/D15-1222/)\.
- OpenAI \(2025\)OpenAI\.Introducing GPT\-4\.1 in the API\.OpenAI Product Blog, April 2025\.URL[https://openai\.com/blog/introducing\-gpt\-4\-1](https://openai.com/blog/introducing-gpt-4-1)\.
- Reimers et al\. \(2021\)Nils Reimers, Iryna Gurevych, et al\.all\-MiniLM\-L6\-v2: Sentence\-Transformers model\.Sentence\-Transformers Model Card \(Hugging Face Hub\), 2021\.URL[https://huggingface\.co/sentence\-transformers/all\-MiniLM\-L6\-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)\.
- Russell et al\. \(2025\)Jenna Russell, Marzena Karpinska, and Mohit Iyyer\.People who Frequently Use ChatGPT for Writing Tasks are Accurate and Robust Detectors of AI\-Generated Text, 2025\.URL[https://arxiv\.org/abs/2501\.15654](https://arxiv.org/abs/2501.15654)\.
- Sadasivan et al\. \(2025\)Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi\.Can AI\-Generated Text be Reliably Detected?, 2025\.URL[https://arxiv\.org/abs/2303\.11156](https://arxiv.org/abs/2303.11156)\.
- Saha & Feizi \(2025\)Shoumik Saha and Soheil Feizi\.Almost AI, Almost Human: The Challenge of Detecting AI\-Polished Writing\.*arXiv preprint arXiv:2502\.15666*, 2025\.
- See et al\. \(2017\)Abigail See, Peter J\. Liu, and Christopher D\. Manning\.Get To The Point: Summarization with Pointer\-Generator Networks\.In*Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pp\. 1073–1083, Vancouver, Canada, July 2017\. Association for Computational Linguistics\.doi:10\.18653/v1/P17\-1099\.URL[https://www\.aclweb\.org/anthology/P17\-1099](https://www.aclweb.org/anthology/P17-1099)\.
- Su et al\. \(2025\)Zhixiong Su, Yichen Wang, Herun Wan, Zhaohan Zhang, and Minnan Luo\.HACo\-Det: A Study Towards Fine\-Grained Machine\-Generated Text Detection under Human\-AI Coauthoring\.In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar \(eds\.\),*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pp\. 22015–22036, Vienna, Austria, July 2025\. Association for Computational Linguistics\.ISBN 979\-8\-89176\-251\-0\.doi:10\.18653/v1/2025\.acl\-long\.1069\.URL[https://aclanthology\.org/2025\.acl\-long\.1069/](https://aclanthology.org/2025.acl-long.1069/)\.
- Tian & Cui \(2023\)Edward Tian and Alexander Cui\.GPTZero: Towards Detection of AI\-Generated Text using Zero\-Shot and Supervised Methods, 2023\.URL[https://gptzero\.me](https://gptzero.me/)\.
- Verma et al\. \(2024\)Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein\.Ghostbuster: Detecting Text Ghostwritten by Large Language Models\.In Kevin Duh, Helena Gomez, and Steven Bethard \(eds\.\),*Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pp\. 1702–1717, Mexico City, Mexico, June 2024\. Association for Computational Linguistics\.doi:10\.18653/v1/2024\.naacl\-long\.95\.URL[https://aclanthology\.org/2024\.naacl\-long\.95/](https://aclanthology.org/2024.naacl-long.95/)\.
- Wang et al\. \(2023\)Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu\.SeqXGPT: Sentence\-Level AI\-Generated Text Detection\.In Houda Bouamor, Juan Pino, and Kalika Bali \(eds\.\),*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pp\. 1144–1156, Singapore, December 2023\. Association for Computational Linguistics\.URL[https://aclanthology\.org/2023\.emnlp\-main\.73/](https://aclanthology.org/2023.emnlp-main.73/)\.
- Wang et al\. \(2025\)Yitong Wang, Zhongping Zhang, Margherita Piana, Zheng Zhou, Peter Gerstoft, and Bryan A\. Plummer\.Real, Fake, or Manipulated? Detecting Machine\-Influenced Text, 2025\.URL[https://arxiv\.org/abs/2509\.15350](https://arxiv.org/abs/2509.15350)\.
- Yang & Cheng \(2024\)Xianjun Yang and Wei Cheng\.DNA\-GPT: Divergent N\-Gram Analysis for Training\-Free Detection of GPT\-Generated Text\.In*Proceedings of the International Conference on Learning Representations*, 2024\.URL[https://openreview\.net/forum?id=Xlayxj2fWp](https://openreview.net/forum?id=Xlayxj2fWp)\.
- Zhang et al\. \(2015\)Xiang Zhang, Junbo Zhao, and Yann LeCun\.Character\-level Convolutional Networks for Text Classification\.In*Advances in Neural Information Processing Systems*, volume 28\. Curran Associates, Inc\., 2015\.

## Appendix ADifferences with the Heterogeneous Mixed Text Detection Task

The PaLD\(Lei et al\.,[2025](https://arxiv.org/html/2510.03154v1#bib.bib20)\)formulation considers a textxxsegmented asx=x1​⋯​xnx=x\_\{1\}\\cdots x\_\{n\}, where each segmentxix\_\{i\}is assumed to originate from either a human or an LLM, i\.e\.,xi∼Phumanx\_\{i\}\\sim P\_\{\\mathrm\{human\}\}orxi∼PLLMx\_\{i\}\\sim P\_\{\\mathrm\{LLM\}\}\. The learning objective is to infer latent per\-segment authorship labelsa1:n∈\{human,LLM\}na\_\{1:n\}\\in\\\{\\mathrm\{human\},\\mathrm\{LLM\}\\\}^\{n\}\(and optionally segment boundaries\), estimatingpθ​\(a1:n∣x\)p\_\{\\theta\}\(a\_\{1:n\}\\mid x\)and predictinga^1:n=arg⁡max⁡pθ​\(a1:n∣x\)\\hat\{a\}\_\{1:n\}=\\arg\\max p\_\{\\theta\}\(a\_\{1:n\}\\mid x\)\. In contrast, our*homogeneous mixed text prediction*task dispenses with provenance as supervision and regresses an authorship\-agnostic*edit magnitude*aligned to a similarity metric\. Given pre/post pairs only to derive targets, inference relies on a*single\-input*predictorfθssi​\(y\)f\_\{\\theta\}^\{\\text\{ssi\}\}\(y\)that maps the edited textyydirectly toΔ^​\(y\)∈\[0,1\]\\hat\{\\Delta\}\(y\)\\in\[0,1\], without segment labels, boundary inference, or reconstruction of the source\. This reframing changes \(i\) theassumptions\(binary authorship mixture vs\. latent, entangled edits\), \(ii\) theoutputs\(label sequencea1:na\_\{1:n\}vs\. scalar/regional magnitudesΔ\\Delta\), \(iii\) thesupervision\(segment\-level authorship vs\. metric\-aligned change signals\), and \(iv\) theevaluation\(classification metrics such as accuracy/F1 vs\. correlation and error againstΔ\\Delta, plus calibration\)\.

## Appendix BHuman Agreement with Intermediate Supervision Metrics

We compute the score for each pair of source and AI\-edited texts, then assign each AI\-edited text to one ofnnbuckets according to the bucketing scheme described in Section[C](https://arxiv.org/html/2510.03154v1#A3.SS0.SSS0.Px1)\. Allα\\alphavalues are reported in Table[3](https://arxiv.org/html/2510.03154v1#A2.T3)\.

Table 3:Agreement \(Krippendorff’sα\\alphawith bootstrap SE\) between human annotators and proposed intermediate supervision metrics under different bucketing schemes for scores\.
## Appendix CMore Modeling Details

We use QLoRA to sweep both Llama and Mistral families of backbones between 3B and 24B parameters\. We experiment with both a direct regression head and a N\-way classification head with weighted\-average decoding\.

#### Determining thresholds for fully human and fully AI texts

Some edits are too small to be detectable, such as adding a single comma, correcting a typo, etc\. We choose a minimum threshold of 0\.03 for cosine distance threshold and 0\.06 for soft n\-grams in order to supervise it as AI\-edited, chosen through manual inspection and validation of edits we would consider small enough such that the authorship is still entirely human\.

Additionally, there are cases on the other end of the spectrum where AI was so pervasive in a text that it essentially rewrote the entire document and it became fully AI\-generated\. To measure the upper threshold where we would consider a text fully AI\-generated, we analyzed the similarity metrics between the sources and their corresponding fully AI\-generated synthetic mirrors\. We selected thresholds that best separate fully AI\-generated synthetic mirrors from the heaviest AI\-edited text, which were 0\.15 for cosine distance and 0\.72 for soft n\-grams\.

### C\.1Regression Formulation

Letssdenote the raw similarity score andτlow\\tau\_\{\\text\{low\}\}andτhigh\\tau\_\{\\text\{high\}\}be the low and high thresholds, respectively\. We define the scaled similarity score as:

s~=\{0\.0if​s≤τlow1\.0if​s≥τhighs−τlowτhigh−τlowotherwise\\tilde\{s\}=\\begin\{cases\}0\.0&\\text\{if \}s\\leq\\tau\_\{\\text\{low\}\}\\\\ 1\.0&\\text\{if \}s\\geq\\tau\_\{\\text\{high\}\}\\\\ \\frac\{s\-\\tau\_\{\\text\{low\}\}\}\{\\tau\_\{\\text\{high\}\}\-\\tau\_\{\\text\{low\}\}\}&\\text\{otherwise\}\\end\{cases\}\(1\)
The regression model directly predicts the scaled similarity scores^\\hat\{s\}using a mean squared error loss:

ℒMSE=1n​∑i=1n\(s~i−s^i\)2\\mathcal\{L\}\_\{\\text\{MSE\}\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\(\\tilde\{s\}\_\{i\}\-\\hat\{s\}\_\{i\}\)^\{2\}\(2\)
wherennis the number of training examples\.

### C\.2Classification Formulation

For the classification approach, we discretize the similarity scores intoNNbuckets, whereN∈\{4,5,6\}N\\in\\\{4,5,6\\\}\. Given minimum and maximum thresholdsτmin\\tau\_\{\\text\{min\}\}andτmax\\tau\_\{\\text\{max\}\}, we define the bucket assignment function:

b​\(s\)=min⁡\(N−1,⌊s−τminτmax−τmin⋅N⌋\)b\(s\)=\\min\\left\(N\-1,\\left\\lfloor\\frac\{s\-\\tau\_\{\\text\{min\}\}\}\{\\tau\_\{\\text\{max\}\}\-\\tau\_\{\\text\{min\}\}\}\\cdot N\\right\\rfloor\\right\)\(3\)
The midpoint of bucketjjis given by:

mj=τmin\+\(j\+0\.5\)⋅\(τmax−τmin\)Nm\_\{j\}=\\tau\_\{\\text\{min\}\}\+\\frac\{\(j\+0\.5\)\\cdot\(\\tau\_\{\\text\{max\}\}\-\\tau\_\{\\text\{min\}\}\)\}\{N\}\(4\)
We train the classification model using cross\-entropy loss:

ℒCE=−1n​∑i=1nlog⁡p​\(b​\(si\)\|xi\)\\mathcal\{L\}\_\{\\text\{CE\}\}=\-\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\log p\(b\(s\_\{i\}\)\|x\_\{i\}\)\(5\)
wherep​\(j\|xi\)p\(j\|x\_\{i\}\)is the predicted probability for bucketjjgiven inputxix\_\{i\}\.

During inference, we decode the final similarity score using a weighted average strategy:

s^=∑j=0N−1p​\(j\|x\)⋅mj\\hat\{s\}=\\sum\_\{j=0\}^\{N\-1\}p\(j\|x\)\\cdot m\_\{j\}\(6\)
wherep​\(j\|x\)p\(j\|x\)is the predicted probability for bucketjjandmjm\_\{j\}is the corresponding bucket midpoint\.

### C\.3Architecture and Optimization

We train the model for 1 epoch with AdamW using a batch size of 24 and a constant learning rate of 3e\-5\. We initialize the model with pretrained weights from the base model and target*all*linear layers: self\-attention QKV, output, and all linear layers in the MLP\. We use a LayerNorm and single linear layer as the head for both prompt classification and edit heads and supervise both jointly in a multi\-task learning routine\. On 8 A100 GPUs, this takes approximately 8 hours for the largest model\.

## Appendix DTernary Classifier Decoding

GPTZero reports probabilities of three classes: “human”, “AI”, and “mixed,” so we simply use argmax decoding to select the highest probability class\. DetectAIve reports probabilities of four classes: “human”, “AI”, “AI Polished”, and “AI humanized”\. We attempted to group “AI humanized“ predictions with both the “AI” and the “AI Polished” categories for ternary classification, and found that grouping with ”AI” produced a higher F1 score\. Therefore, we group “AI Humanized” and “AI” into a single category for purposes of comparison\.

## Appendix ETernary Classification Confusion Matrices

Analyzing the confusion matrix, we see thatEditLensexhibits much stronger performance on the AI\-edited text category than the strongest ternary classifier, GPTZero\. While bothEditLensand GPTZero are nearly perfect at distinguishing fully AI\-generated text from fully human\-written text,EditLensis the only model able to also consistently detect AI\-edited text as a distinct category from fully human and fully AI\.

![[Uncaptioned image]](https://arxiv.org/html/2510.03154v1/figures/confusion_ternary.png)

## Appendix FCorrelation BetweenEditLensPredictions and AI Polish Similarity Metrics

Table 4:Pearson correlation coefficients by model\.![Refer to caption](https://arxiv.org/html/2510.03154v1/figures/dataannotatorsetup.png)Figure 7:Data Annotator tasks were set up as above\.
## Appendix GData Generation Models

Table 5:Models used for dataset generation
## Appendix HEmbedding Models

Table 6:Embedding models used for supervision
## Appendix IBase Models

Table 7:Base models used for training
## Appendix JMore Results on Human\-Edited AI Text

We focus on the human\-edited AI versions of “Generation” and “OpenQA” categories of BEEMO, because the other categories, such as “Rewrite” and “Summarize”, are already themselves AI\-edited versions of human text, “Closed QA” the answers are so tightly constrained we would consider the answers to be human\-written, and we would not consider the model outputs fully AI\-generated\. We also measure the correlation coefficient between our similarity metrics and the model scores\. The intuition for this is that if the human edit is more invasive, we would expect the similarity metrics to increase, and the model score to decrease\. As expected, we find a moderate negative correlation between our model’s scores and the similarity, with−0\.396\-0\.396for cosine distance and−0\.501\-0\.501for soft n\-grams\.

In Figures[9](https://arxiv.org/html/2510.03154v1#A10.F9)and[9](https://arxiv.org/html/2510.03154v1#A10.F9), we present the output distribution ofEditLensfor BEEMO’s Generation and OpenQA splits, on the fully AI\-generated text \(orange\) and human\-edited version \(blue\)\.

![Refer to caption](https://arxiv.org/html/2510.03154v1/figures/beemo_generation.png)Figure 8:BEEMO Generation
![Refer to caption](https://arxiv.org/html/2510.03154v1/figures/beemo_openqa.png)Figure 9:BEEMO OpenQA

As is shown in the figures, the predicted score distribution moves significantly towards human\-generated following editing, as expected\.

## Appendix KEditing Prompts

Table 8:Full list of editing prompts by split with category and contributor\.TRAINEditing PromptContributorTone and Style AdjustmentsWrite this in a way that a business person would get itHumanEdit this to sound more politeHumanInject more personality and warmth into this textGemini 2\.5 ProAdjust the tone to be more persuasive and convincingGemini 2\.5 ProMake this sound more professional and authoritativeGemini 2\.5 ProRewrite this to be more empathetic and understandingGemini 2\.5 ProMake this sound more urgent and compellingGemini 2\.5 ProMake this sound more objective and unbiasedGemini 2\.5 ProAdopt a more academic and scholarly toneGemini 2\.5 ProMake this more direct and confrontationalClaude Sonnet 4Make this more memorable and quotableClaude Sonnet 4Make this sound more diplomatic and tactfulClaude Sonnet 4Make this more emotionally resonantClaude Sonnet 4Make this more formalClaude Sonnet 4Simplify for customers with no technical backgroundClaude Sonnet 4Make this suitable for social media sharingClaude Sonnet 4Inject enthusiasm and energy into this writingClaude Sonnet 4Translate this for a teenage audienceClaude Sonnet 4Soften the tone while maintaining the messageClaude Sonnet 4Make this more relatable to the reader’s experienceClaude Sonnet 4Make this more inspiring and motivationalClaude Sonnet 4Add gravitas and weight to this statementClaude Sonnet 4Adopt a more skeptical and questioning toneClaude Sonnet 4Make this more casualClaude Sonnet 4Adjust for a peer\-reviewed academic journalClaude Sonnet 4Convert to a more analytical and logical approachClaude Sonnet 4Make this appropriate for C\-suite executivesClaude Sonnet 4Change this so it fits what a business person would wantChatGPT 4oMake this sound more sure and strongChatGPT 4oEdit this for people who don’t know the topic wellChatGPT 4oMake this more direct and boldChatGPT 4oMake this sound fair and not take sidesChatGPT 4oMake this sound more serious and importantChatGPT 4oMake this sound more excited and energeticChatGPT 4oMake this easier for someone who doesn’t know that much about itChatGPT 4oMake this more accessible to a non\-expert readerChatGPT 4oAdapt this for readers with no prior background in the topicChatGPT 4oWrite this like you’re talking to someoneChatGPT 4oMake this sound more doubtful and questioningChatGPT 4oAdjust the voice to sound more academic\.ChatGPT 4oMake this sound more relaxed and friendlyChatGPT 4oWrite this in a way that top company leaders would likeChatGPT 4oMake this more formal and properChatGPT 4oMake this easier to remember and repeatChatGPT 4oTailor this message to suit a lay audienceChatGPT 4oMake this sound more exciting and well writtenChatGPT 4oMake this sound more serious and properChatGPT 4oUse a smart and serious tone like in official stuffChatGPT 4oEdit this to sound more urgent and importantChatGPT 4oMake this more convincing and easier to understandChatGPT 4oChange this so it’s easy for a teen to readChatGPT 4oMake this more logical and fact\-basedChatGPT 4oMake the consequences feel more importantChatGPT 4oMake this sound uplifting and encouragingChatGPT 4oRewrite this to align with a formal toneChatGPT 4oMake this more convincing and clearChatGPT 4oChange this so a 5th grader can understand itChatGPT 4oTake out hard words and explain them in a simple wayChatGPT 4oFix this to make it more interestingChatGPT 4oChange this to make it as strong as possibleChatGPT 4oRewrite this to better suit a business audienceChatGPT 4oUse simpler words so anyone can understand thisChatGPT 4oMake this sound more like school writingChatGPT 4oMake this sound nicer and more funChatGPT 4oChange this to sound more thoughtfulChatGPT 4oMake this funnier and more lightheartedChatGPT 4oUse better words to make this sound smarterChatGPT 4oAdd some personality to thisChatGPT 4oMake this sound more like professional writingChatGPT 4oAdding DetailMake this more descriptiveHumanMake this more detailedHumanPlease add more details to make my argument betterHumanAdd vivid imagery and sensory details to bring this to lifeClaude Sonnet 4Add backstory or context to enrich understandingClaude Sonnet 4Add depth and context to make this more comprehensiveClaude Sonnet 4Include specific measurements, colors, and physical characteristicsClaude Sonnet 4Flesh out these ideas with supporting informationClaude Sonnet 4Provide concrete examples to illustrate these pointsClaude Sonnet 4Add sensory details to make this more vividClaude Sonnet 4Use more precise and colorful adjectivesClaude Sonnet 4Add dialogue and quoted speech to make scenes more vividClaude Sonnet 4Elaborate on the key points with concrete detailsClaude Sonnet 4Add storytelling elements to increase engagementClaude Sonnet 4Add descriptive metaphors and similes to enhance understandingClaude Sonnet 4Expand with real\-world applicationsClaude Sonnet 4Use more evocative and powerful verbsClaude Sonnet 4Include expert opinions or research findingsClaude Sonnet 4Incorporate specific brand names, locations, and proper nounsClaude Sonnet 4Paint a clearer picture with specific visual descriptionsClaude Sonnet 4Use figurative language to make concepts more tangibleClaude Sonnet 4Include personal anecdotes or case studiesClaude Sonnet 4Give examples to help make this clearerChatGPT 4oEdit this with clear examples to help explain this betterChatGPT 4oAdd details that create a mood or feelingChatGPT 4oTell some of the story behind this to help understand itChatGPT 4oExplain more about what this means and why it mattersChatGPT 4oExplain the main points using real examplesChatGPT 4oAdd descriptions of sounds, smells, textures, and how things feelChatGPT 4oInclude exact sizes, colors, and what things look likeChatGPT 4oAdd details about the setting and backgroundChatGPT 4oAdd details that help readers see, hear, and feel what’s happeningChatGPT 4oAdd conversations and quotes to make scenes more realChatGPT 4oDevelop this text further by explaining the implicationsChatGPT 4oHelp readers picture this more clearly with specific detailsChatGPT 4oDescribe how things look, sound, or feel moreChatGPT 4oUse more interesting and specific describing wordsChatGPT 4oAdd comparisons to help explain things betterChatGPT 4oUse real\-life examples to show what you meanChatGPT 4oAdd background facts to back up these pointsChatGPT 4oAdd true stories or examples from real lifeChatGPT 4oUse specific names of places, brands, and thingsChatGPT 4oAdd more ideas or facts that support what you’re sayingChatGPT 4oAdd details about the place and situationChatGPT 4oShare what experts think or what research showsChatGPT 4oUse real examples to explain this betterChatGPT 4oUse stronger, more exciting action wordsChatGPT 4oAdd illustrative examples to clarify these pointsChatGPT 4oFluency and FlowRearrange thisHumanCan you make this sound fluent?HumanImprove the transitions between the paragraphsGemini 2\.5 ProCreate a more effective and engaging openingGemini 2\.5 ProMake this read like it was written by a native speakerClaude Sonnet 4Make this sound more conversational and engagingClaude Sonnet 4Improve the rhythm and readability of this writingClaude Sonnet 4Make the progression of ideas feel effortlessClaude Sonnet 4Create smoother connections between these ideasClaude Sonnet 4Improve the natural rhythm of this textClaude Sonnet 4Smooth out the awkward phrasing in this passageClaude Sonnet 4Eliminate any choppy or awkward sentencesClaude Sonnet 4Ensure the sentences transition smoothly from one idea to the nextChatGPT 4oHelp the ideas move from one to the next easilyChatGPT 4oMake the ideas connect betterChatGPT 4oMake the language flow more fluidly without sounding forcedChatGPT 4oCan you fix parts that sound choppy or off?ChatGPT 4oMake this writing smoother and betterChatGPT 4oMake the words fit together betterChatGPT 4oMake this easier and smoother to readChatGPT 4oHelp the ideas connect more smoothlyChatGPT 4oMake this sound like someone who speaks English well wrote itChatGPT 4oMake the sentences flow betterChatGPT 4oMake the rhythm of the sentences betterChatGPT 4oConcisionRewrite to be more concise and powerfulHumanClarify the main ideaGemini 2\.5 ProRemove any filler wordsGemini 2\.5 ProRemove any jargon or technical terms and explain them in plain languageGemini 2\.5 ProReplace complex words with simpler alternativesGemini 2\.5 ProIdentify and eliminate any redundant phrases or wordsGemini 2\.5 ProMake this more concrete and less abstractGemini 2\.5 ProSimplify this text for a 5th\-grade reading levelGemini 2\.5 ProRemove every unnecessary word and phraseClaude Sonnet 4Trim the fat without losing the muscleClaude Sonnet 4Make this more concise without losing important informationClaude Sonnet 4Eliminate wordy expressions and redundanciesClaude Sonnet 4Tighten this writing by removing unnecessary wordsClaude Sonnet 4Remove all extra words and phrasesChatGPT 4oTrim this down while keeping the tone and meaning intactChatGPT 4oRemove extra or repeated wordsChatGPT 4oUse clearer words but keep the meaningChatGPT 4oGet rid of long or confusing partsChatGPT 4oMake this shorter and more directChatGPT 4oTake out anything extra but keep the good partsChatGPT 4oSay this in fewer words but still make it strongChatGPT 4oTake out words that aren’t needed to make this betterChatGPT 4oTake out words that don’t add anythingChatGPT 4oCut this down but keep the same meaning and styleChatGPT 4oStructure and OrganizationGroup related ideas together more effectivelyGemini 2\.5 ProEnsure a clear introduction, body, and conclusionGemini 2\.5 ProArrange these points in order of importanceClaude Sonnet 4Create better section breaks and headersClaude Sonnet 4Create a more compelling narrative arcClaude Sonnet 4Build toward a stronger climax or conclusionClaude Sonnet 4Reorganize this for better logical flowClaude Sonnet 4Use parallel structure to enhance readabilityClaude Sonnet 4Break this into clearer paragraphs with smooth transitionsClaude Sonnet 4Rearrange this content for a clearer argument progressionChatGPT 4oPut this in a better orderChatGPT 4oPut similar ideas together more clearlyChatGPT 4oBuild up to a strong endingChatGPT 4oPut this in an order that makes more senseChatGPT 4oSet this up in a way that’s easier to readChatGPT 4oGroup ideas that go together and add connectionsChatGPT 4oMove things around to make the main points stand outChatGPT 4oSplit this into paragraphs that connect betterChatGPT 4oMake sentences match and sound good togetherChatGPT 4oPut the most important stuff firstChatGPT 4oOrganize this information in a more reader\-friendly formatChatGPT 4oTell the story in a more interesting wayChatGPT 4oGeneral ImprovementCan you help my essay get a better grade?HumanRewrite this so it sounds goodHumanMake my essay betterHumanMake this essay look betterHumanCan you improve this?HumanMake this an A paperHumanRevise this to make it more engagingClaude Sonnet 4Enhance the overall effectiveness of this passageClaude Sonnet 4Refine this writing to make it more professionalClaude Sonnet 4Enhance this text while maintaining the original meaningClaude Sonnet 4Optimize this text for maximum impactClaude Sonnet 4Strengthen this writing by improving word choice and structureClaude Sonnet 4Transform this into more compelling proseClaude Sonnet 4Polish this text for clarity and readabilityClaude Sonnet 4Upgrade the sophistication of this writingClaude Sonnet 4Make this work better overallChatGPT 4oMake this writing more polished and effectiveChatGPT 4oRefine this text to improve its overall impactChatGPT 4oFix this but keep the same meaningChatGPT 4oFix this but keep the main point the sameChatGPT 4oImprove this passage while preserving its original intentChatGPT 4oParaphrasingRemake all of this in a different wayHumanRewrite all of this in different wordsHumanRecast these ideas in a different styleClaude Sonnet 4Rephrase this text to avoid repetitionClaude Sonnet 4Express these ideas using alternative phrasingClaude Sonnet 4Say the same thing but in a fresh wayClaude Sonnet 4Present the same information from a fresh angleClaude Sonnet 4Reframe this argument using different terminologyClaude Sonnet 4Restate this using different vocabulary and sentence structureChatGPT 4oUse easier words that mean the same thingChatGPT 4oShare this idea from a new point of viewChatGPT 4oSay this in a new wayChatGPT 4oUse different words to say the same thingChatGPT 4oSay this using different words and sentence typesChatGPT 4oSay this using different words and ideasChatGPT 4oSay this in a new and interesting wayChatGPT 4oParaphrase this to make it simpler and easier to understandChatGPT 4oClarity and PrecisionCan you fix the problems with my argument?HumanEmphasize the key pointsHumanAdd precise measurements and timeframesClaude Sonnet 4Define any terms that might be unclearClaude Sonnet 4Eliminate any ambiguous or vague languageClaude Sonnet 4Make the cause\-and\-effect relationships clearerClaude Sonnet 4Explain any hard words so people know what they meanChatGPT 4oFix parts that sound weird or hard to readChatGPT 4oSay clearly who or what each word is talking aboutChatGPT 4oMake this clear and easy to readChatGPT 4oMake sure one idea leads to the next clearlyChatGPT 4oMake it obvious what causes whatChatGPT 4oSay this in a clear and simple wayChatGPT 4oGrammar and MechanicsMake my grammar sound betterHumanFix any grammatical mistakes in this textGemini 2\.5 ProCorrect any grammar, punctuation, or spelling errors in this textChatGPT 4oUse better words and fix how the sentences are writtenChatGPT 4oVALTone and Style AdjustmentsEdit this into a blog post I can share onlineHumanRewrite this in a more conversational and approachable styleGemini 2\.5 ProMake this sound more confident and authoritativeClaude Sonnet 4Adapt this for international readersClaude Sonnet 4Make this connect better with people’s feelingsChatGPT 4oMake this sound more like a friendly conversationChatGPT 4oMake this easier for readers to relate toChatGPT 4oMake this easier for someone new to the topic to getChatGPT 4oMake this sound more businesslike and seriousChatGPT 4oParaphrasingRewrite all of thisHumanTranslate this into more accessible languageClaude Sonnet 4Find alternative ways to express these conceptsClaude Sonnet 4Rework this using varied sentence structuresClaude Sonnet 4Change how the sentences are writtenChatGPT 4oSay this in another wayChatGPT 4oAdding DetailAdd atmospheric details to create mood and settingClaude Sonnet 4Add environmental and contextual descriptionsClaude Sonnet 4Include contextual information to support these claimsChatGPT 4oGive more background so it’s easier to understandChatGPT 4oUse creative comparisons to make ideas clearerChatGPT 4oConcisionRewrite this to be more direct and to the pointGemini 2\.5 ProMake this easier to understandChatGPT 4oUse simpler language anyone can getChatGPT 4oStructure and OrganizationRestructure this to emphasize the main pointsClaude Sonnet 4Add better breaks and section titlesChatGPT 4oFluency and FlowConnect these thoughts more seamlesslyClaude Sonnet 4Help the paragraphs connect betterChatGPT 4oGrammar and MechanicsProofread this for spelling and grammar errorsHumanGeneral ImprovementElevate the quality of this writingClaude Sonnet 4Clarity and PrecisionRemove anything confusing or unclearChatGPT 4oTESTTone and Style AdjustmentsLighten the tone and add a touch of humorGemini 2\.5 ProAdjust the tone to be more friendlyClaude Sonnet 4Increase the emotional stakesClaude Sonnet 4Make this sound more formal and school\-likeChatGPT 4oMake this nicer but keep the main pointChatGPT 4oChange this so people from other countries can get it tooChatGPT 4oAdding DetailMake this longer with more evidenceHumanExpand this text with more specific examples and detailsClaude Sonnet 4Show rather than tell by adding scene\-setting detailsClaude Sonnet 4Include sounds, smells, textures, and other sensory elementsClaude Sonnet 4Add a short story to make this more interestingChatGPT 4oShow how this works in real lifeChatGPT 4oConcisionSimplify this textHumanMake this more specificHumanReduce wordiness while amplifying impactClaude Sonnet 4Say this in fewer words without losing meaningChatGPT 4oEdit this to be punchier and more directChatGPT 4oFluency and FlowMake this text flow more naturallyClaude Sonnet 4Improve how the sentences sound togetherChatGPT 4oRefine the pacing and cadence of this paragraphChatGPT 4oWrite a better and more interesting beginningChatGPT 4oClarity and PrecisionWrite this in a way that my teacher would get itHumanCan you make my paper more persuasive?HumanMake the main idea clearerChatGPT 4oImprove this to make it stronger and clearerChatGPT 4oParaphrasingParaphrase thisHumanRewrite this in different words while keeping the same meaningClaude Sonnet 4Reword this to improve clarity while keeping the meaningChatGPT 4oStructure and OrganizationMake sure there’s a beginning, middle, and endChatGPT 4oGroup related ideas and use transitions to improve structureChatGPT 4oGrammar and MechanicsCan you fix any spelling, grammar, or punctuation issues?HumanTable 9:Distribution of prompt categories and contributors across Train, Val, and Test splits shown as percentages with raw counts in parentheses\.Table 10:Count of unique source texts across Train, Val, and Test datasets\.Table 11:Distribution of examples by label across Train, Val, and Test datasets\.Table 12:Distribution of source LLM for AI\-edited and AI\-generated examples, across Train, Val, and Test datasets\.Table 13:Composition of the AI\-edited dataset, by splitTable 14:Word count statistics across splits for Human, AI\-edited, and AI\-generated texts in the dataset\.
## Appendix LLLM Usage Statement

Large Language Models \(LLMs\) were used in the experiments for the paper as described, to assist in writing the code to run the experiments, brainstorm the formalization of the task, assist in generating the figures for the paper, assist with LaTeX formatting, and review the paper to help the authors with constructive feedback\. The authors didnotuse LLMs directly in the original writing of the manuscript, but did use LLMs to help with wording and phrasing in some sections\. The authors take full responsibility for the factuality and originality of the content in this manuscript\.

## Appendix MGrammarly Supervision Scores

In Figures[10](https://arxiv.org/html/2510.03154v1#A13.F10)and[11](https://arxiv.org/html/2510.03154v1#A13.F11), we show the distributions of scores according to different intermediate supervision metrics by Grammarly edit prompt\.

![Refer to caption](https://arxiv.org/html/2510.03154v1/x5.png)Figure 10:Distribution of cosine scores on dataset from Grammarly by edit instruction\.![Refer to caption](https://arxiv.org/html/2510.03154v1/x6.png)Figure 11:Distribution of soft n\-grams scores on dataset from Grammarly by edit instruction\.Table 15:Ternary Classification \(Soft N\-Grams\)Table 16:Ternary Classification \(Cosine Similarity\)Table 17:Binary Classification \(Soft N\-Grams\)Table 18:Binary Classification \(Cosine Similarity\)