Pangram 4 Technical Report

# Pangram 4 Technical Report
Source: [https://arxiv.org/html/2607.27183](https://arxiv.org/html/2607.27183)
###### Abstract

We presentPangram 4, the latest deep\-learning\-based AI\-text classification model from Pangram Labs\.We achieve an AUROC of 0\.9916 with a false positive rate of 0\.0041% and a false negative rate of 0\.3396%\. In addition to its increased overall accuracy compared with Pangram 3,Pangram 4exhibits superior out\-of\-distribution generalization and robustness to adversarial attacks\. Another novel contribution ofPangram 4is its improved ability to distinguish fine\-grained edits and mixed AI\-human co\-authored text\. We demonstrate improvements to both boundary detection tasks and the detection of interleaved AI assistance\. Finally, we report metrics on standard AI detection benchmarks showing thatPangram 4achieves state\-of\-the\-art performance on the AI text detection task across a wide variety of settings and domains\.

11footnotetext:Equal contribution\.## 1Introduction

The world is being inundated with an unprecedented volume of AI\-generated text\. Over a third of new internet material\(Dolezalet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib4)\), 26% of long\-form social media posts\(Pangram Labs,[2026](https://arxiv.org/html/2607.27183#bib.bib11)\), 9% of news articles\(Russellet al\.,[2026a](https://arxiv.org/html/2607.27183#bib.bib2)\), and 21% of machine\-learning conference reviews\(Pangram Labs,[2025](https://arxiv.org/html/2607.27183#bib.bib10)\)are largely or fully AI\-generated\. “AI slop,” a term for unwanted AI content, is used to describe text that is obviously AI\-generated\(Shaibet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib51)\)or that offloads cognition onto the recipient\.111[https://ampblog\.substack\.com/p/what\-makes\-slop\-slop](https://ampblog.substack.com/p/what-makes-slop-slop)AI is a tool with the power to automate many long\-horizon tasks, but easily mis and overused with the potential for societal harm\.

While it has been shown that individuals who regularly use AI can detect AI text by eye\(Russellet al\.,[2025](https://arxiv.org/html/2607.27183#bib.bib61)\), this is a laborious task that requires both time and experience to do well\. This creates an effort asymmetry between readers and writers: AI can produce well\-formed text that could plausibly have come from an expert, nearly for free, but readers still incur the burden of understanding whether the content they are ingesting is meaningful, grounded, and well\-researched\. This strains existing systems that lack a way to reliably discern human from AI text\.

Additionally, people are finding ways to incorporate AI into their writing workflows, producing documents of mixed authorship\. These uses, while legitimate, do not diminish the societal harms caused by LLM\-powered bots and undisclosed, fully AI\-generated content polluting information ecosystems\(Rajet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib9); Draysonet al\.,[2025](https://arxiv.org/html/2607.27183#bib.bib55); Yuet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib56)\)\. To effectively mitigate these harms, it is imperative to understand the degree of human and AI involvement in written documents\. This task motivates the creation of an AI\-text detection model with high accuracy and granularity on AI, human, and AI\-assisted text\.

We introducePangram 4, the most capable AI text detection model developed by Pangram Labs\. Designed to detect frontier models such as Claude Fable 5\(Anthropic,[2026a](https://arxiv.org/html/2607.27183#bib.bib12)\)and GPT\-5\.6 Sol\(OpenAI,[2026c](https://arxiv.org/html/2607.27183#bib.bib13)\),Pangram 4achieves the highest accuracy on the AI\-text detection task while maintaining a false positive rate of 0\.0041% \(roughly 1 in 24,000\)\. We build on Pangram 3, an AI detection model based on the EditLens architecture\(Thaiet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib53)\), with data and architectural improvements\.Pangram 4represents a new frontier in AI text detection technology, with state\-of\-the\-art precision and recall, performance on mixed\-authorship text, fine\-grained span predictions, and robustness to humanizer attacks\.

![Refer to caption](https://arxiv.org/html/2607.27183v1/figures/pangram4_architecture_hero.png)Figure 1:Overview of thePangram 4architecture\.Contents

## 2Data

In this section, we report on the composition of our training set\. We train on a wide distribution of domain sources, languages, and generator models\. The human\-written sources in the training dataset undergo various transformations, including synthetic mirroring, AI\-editing, and translation, to produce the AI\-generated and AI\-edited texts that are present in the dataset\.

### 2\.1Human\-written Datasets

Pangram is trained on a variety of datasets consisting of human writing from several sources, domains, and languages, similar to corpora used to train large language models\. All datasets used to train Pangram are either openly licensed for commercial use or their rights are owned by the company\.Pangram 4is not trained on user\-submitted data, customer data from API users, or data obtained by unauthorized Internet crawls\. A rough distribution of the categories of text used to train the model is presented in[Figure 2](https://arxiv.org/html/2607.27183#S2.F2)\.

![Refer to caption](https://arxiv.org/html/2607.27183v1/x1.png)Figure 2:Approximate breakdown of human source text by category\.
### 2\.2AI\-Generated Datasets & Synthetic Mirroring

In keeping with previous Pangram models, the AI examples in our dataset are generated entirely in\-house from a distribution of the most widely used LLMs\. Our synthetic mirroring algorithm is described in the original Pangram technical report\(Emi and Spero,[2024](https://arxiv.org/html/2607.27183#bib.bib52)\)\. This method is used to produce a diverse dataset of AI text and ensure topic invariance\. We want the model to learn “How was this text written?” as opposed to “What is this text about?” An example of a synthetic mirror is as follows:

##### Synthetic mirror example

The synthetic mirror is produced in two steps:

1. 1\.Prompt:“What is the topic of this article?⟨\\langleOriginal Document⟩\\rangle” LLM Response:topicXX
2. 2\.Prompt:“Write an article aboutXX” LLM Response:the*synthetic mirror*of the original document

When available, synthetic mirrors are grounded by additional metadata such as article titles or keywords\. For question\-and\-answer datasets, the question alone may suffice as a synthetic mirror prompt\. To prevent synthetic mirrors from repeating human text, a final check compares the generated text against its original source and discards examples where a significant portion of the original document is repeated verbatim\.

### 2\.3AI\-Assisted Text

Several recent works\(Zhanget al\.,[2024](https://arxiv.org/html/2607.27183#bib.bib43); Saha and Feizi,[2025](https://arxiv.org/html/2607.27183#bib.bib49); Artemovaet al\.,[2025](https://arxiv.org/html/2607.27183#bib.bib60); Sahaet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib42); Quarembaet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib50); Dyckeet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib77); Bsharatet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib76)\)benchmark AI detectors’ ability to detect AI\-polished, mixed, or otherwise co\-authored writing that does not fall neatly into either the human\-written or AI\-generated categories\. Our dataset consists of AI\-assisted text in addition to human\-written text and fully AI\-generated text\. We generate the AI\-assisted text according to the method described in the EditLens paper\(Thaiet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib53)\)\.

## 3Task Formulation

### 3\.1What is AI\-generated Text?

In order to define a tractable problem, we must first scope our definition of AI\-generated text\. We cannot define this as “any text that was produced by any large language model\.” A large language model can simply memorize and reproduce human text, either from its training set or from its context window\. Additionally, a large language model can also produce purely factual information, such as the answers to “What is the capital of France?” or “What is Martin Luther King Jr\.’s birthday?” Answers to both of these questions would be ambiguous, and thus must be considered out of scope\.

Pangram defines AI\-generated text asoriginal, substantial natural\-language prose generated by an LLM in response to an open\-ended writing task or question\. Pangram is designed to detect responses to open\-ended prompts and questions, rather than responses to questions that have a single correct answer\.

In addition to the open\-generation criterion, we also require that, for AI text, the number of output tokens is greater than the number of input tokens\. While AI summarization is a popular editing task, we must consider the output to be human if the AI does not add any original tokens of its own\. Finally, we require that text be at least 50 words long to be considered for analysis\. This is a necessary condition for the model to have enough signal to make a confident prediction\. This excludes short, conversational replies from Pangram’s scope\.

### 3\.2Heterogeneous vs\. Homogeneous Mixed Text

![Refer to caption](https://arxiv.org/html/2607.27183v1/x2.png)Figure 3:The difference between heterogeneous and homogeneous mixed text\. In heterogeneous mixed text, each token has a well\-defined author\. In homogeneous mixed text, authorship attribution is mixed and ambiguous\.To motivate our labeling and architecture choices, we first introduce the concept of heterogeneous vs\. homogeneous mixed text, which was originally introduced in the EditLens paper\(Thaiet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib53)\)\. In heterogeneous mixed text, each token has a clear authorship label, and each segment can be directly attributed to a single author, either human or AI\. An example is a situation where a human writes a paragraph and then asks AI to continue it\.

In homogeneous mixed text, authorship is entangled by the editing process\. An example of this kind of mixed text is a human who writes a paragraph and then goes back and forth with an AI to edit it\. In this situation, the paragraph no longer has a distinct author; there are contributions from both\.

Additionally, the provenance of the surface style of a text can be distinct from the provenance of its underlying ideas\(Russellet al\.,[2026b](https://arxiv.org/html/2607.27183#bib.bib59)\)\. A human can have a very well\-formed, concrete idea that is then worded and rephrased by an AI assistant: this is also an example of co\-authorship\. We use this to motivate our labeling schema, wherein we aim to differentiate fully AI\-generated text from text where a human came up with the original idea, and then the AI was responsible for the final lexical choices that appear in the writing\.

### 3\.3Task Definition

To our knowledge,Pangram 4is the first AI detection model that is able to identify both homogeneous and heterogeneous mixed writingsimultaneously in a single pass\.Pangram 4takes an input sequence ofNNtokens and produces an output of logitsZtok∈ℝN×3Z\_\{\\mathrm\{tok\}\}\\in\\mathbb\{R\}^\{N\\times 3\}over the classes \{human,ai\-assisted,ai\-generated\} for each token\.

1. 1\.humanmeans that the text was solely authored by humans, or that the amount of AI contribution is so low as to be negligible \(e\.g\., light copyediting, literal translation, spelling and grammar fixes\)\.
2. 2\.AI\-Assistedmeans that the text is co\-authoredandhomogeneously mixed\. Although this label is nebulous for a single token, a segment of consecutive tokens with theAI\-Assistedlabel indicates genuine co\-authorship between humans and AI\.
3. 3\.AI\-Generatedmeans that the text is solely authored by AI, or that the text is significantly more AI\-authored than human\-authored—in this case, we would define text predicted asAI\-Generatedto be novel text from an LLM consistent with an open\-ended prompt that allowed for original content solely from the LLM\.

Because these three labels are predicted on a per\-token basis, a prediction of heterogeneous mixed text is expressed as a combination of these three labels across the sequence\. We also note that, as a corollary of this formulation, text can be simultaneously heterogeneous and homogeneous mixed: there may be some sections that are co\-authored, and some sections that have a single author\.

### 3\.4Humanization as an Auxiliary Task

As Pangram becomes a more capable AI detector, adversarial actors have greater incentive to evade it\. We define*humanization*as the deliberate transformation or obfuscation of AI\-generated text to evade AI\-text detection\(Masrouret al\.,[2025](https://arxiv.org/html/2607.27183#bib.bib70)\)\. The auxiliary output estimates whether a document exhibits characteristics consistent with humanization\.

We formulate humanization detection as a four\-way, document\-level classification task overhuman,ai\-generated,humanized\-ai,mixed\-authorship\. This supplemental classifier is applied whenPangram 4detects evidence of AI\-generated content\. It identifies evasion techniques such as deliberate typo injection, casing changes, synonym substitution, homoglyph attacks, and transformations produced by services that advertise humanization or AI\-detector evasion\. Incidental artifacts, such as PDF\-extraction errors, copy\-and\-paste corruption, OCR errors, and encoding or Unicode\-normalization problems, are not classified as humanization\.

### 3\.5Soft N\-Grams Labeling

We do not have ground\-truth tokenwise labels for mixed texta priori\. Therefore, we compute tokenwise labels for AI\-assisted text using the method described in Algorithm[1](https://arxiv.org/html/2607.27183#alg1), which we call Soft N\-Grams Labeling\. Following the assumptions in EditLens, we synthetically generate examples of mixed human\- and AI\-authored text by starting with confirmed human documents and applying AI edits to these texts\. The input to the soft n\-gram labeler is a human documentSSand its corresponding AI\-edited documentTT, which we refer to as thesourceand thetarget, respectively\.

Algorithm 1Pangram soft n\-gram labeling1:Source text

SS, edited text

TT
2:Labels for the edited text and AI fraction

fAIf\_\{\\mathrm\{AI\}\}
3:Split

SSand

TTinto clauses

4:for allclauses

yyin

TTdo

5:Find the most similar span

xxin

SS
6:Compute lexical similarity

L​\(x,y\)L\(x,y\)using token and character n\-grams

7:Compute semantic similarity

E​\(x,y\)E\(x,y\)using text embeddings

8:if

L​\(x,y\)L\(x,y\)is highthen

9:Label

yyasHuman

10:elseif

L​\(x,y\)L\(x,y\)or

E​\(x,y\)E\(x,y\)indicates a substantial rewritethen

11:Label

yyasAI\-Assisted

12:else

13:Label

yyasAI\-Generated

14:endif

15:endfor

16:Let

CHC\_\{\\mathrm\{H\}\},

CAAC\_\{\\mathrm\{AA\}\}, and

CAGC\_\{\\mathrm\{AG\}\}be the character counts for each label

17:

D←CH\+CAA\+CAGD\\leftarrow C\_\{\\mathrm\{H\}\}\+C\_\{\\mathrm\{AA\}\}\+C\_\{\\mathrm\{AG\}\}
18:

fAI←0\.5​CAA\+CAGDf\_\{\\mathrm\{AI\}\}\\leftarrow\\dfrac\{0\.5C\_\{\\mathrm\{AA\}\}\+C\_\{\\mathrm\{AG\}\}\}\{D\}
19:returnclause labels and

fAIf\_\{\\mathrm\{AI\}\}

We diverge from EditLens, however, in theresolutionof the labeler\. While EditLens \(and Pangram 3\) use a scalar distance metric in the embedding space of an LLM to compute the similarity betweenSSandTT, inPangram 4, we instead use a clause\-level222A clause is defined as a group of words that contains a subject and a verb\.soft N\-grams approach loosely inspired by the evaluation metric proposed in ROUGE\(Ng and Abrecht,[2015](https://arxiv.org/html/2607.27183#bib.bib65)\)\.

##### Clause\-level n\-grams

The goal of the soft n\-grams labeler is to extract granular provenance of each clause in the targetTT\. We select the clause as the unit of the labeler after observing that the clause is the smallest atomic unit corresponding to a single expressible “idea” whose provenance can be traced\. We begin by splitting the targetTTinto clauses using Claude Haiku 4\.5\(Anthropic,[2025](https://arxiv.org/html/2607.27183#bib.bib14)\)\. We also tried classical linguistic approaches to clause splitting, but found Claude Haiku more accurate and less brittle on unusual edge cases\. For each clause inTT, we ask the following questions:

1. 1\.For this clauseyy, is there an exact or near\-exact lexical match to this clause inSS? If so, label this clauseHuman\.
2. 2\.If no lexical match is found, for this clauseyy, is there a clause inSSthat semantically matches the idea inyy? If so, label this clauseAI\-Assisted\.
3. 3\.Finally, if there is neither a lexical nor a semantic match to the clauseyy, then we deem the clause an original open\-generation output and label this clauseAI\-Generated\.

This gives us tokenwise labels, where each token inherits the label from its parent clause\. To convert the tokenwise labels into document labels, we define the weighted AI fractionfA​If\_\{AI\}, which is the fraction of characters labeled AI\-generated, plus half the fraction of characters labeled AI\-assisted\. We then supervise this quantity as a bucketed regression target following the methodology in EditLens\.

##### Deletion and rearrangement invariance

We note that the Soft N\-Grams Labeler is invariant to both deletions in the targetTTand rearrangements of clauses betweenSSandTT\. We view this as a feature, not a bug: because at test time, our model only has access toTT, it is ill\-posed for the model to predict deletions fromSS, because it has no access toSS\. In informal language, it’s impossible for the model to predict deletions because it can’t see what was originally there\. We also consider rearrangements to be non\-edits from the perspective of the Soft N\-Grams labeler\. If all the sentences in a human\-written document are merely reordered and not rewritten, we believe it is correct for our model to classify the text as human\-written\.

## 4Training

In this section, we describe the training procedure forPangram 4\. We first describe the model architecture, ablations run to pick the backbone, and individual task heads\. We then describe the training process, active learning, and calibration to generate confidence estimates\.

### 4\.1Architecture

Here we describe the architecture of the model used to train token\- and segment\-level predictions, how calibration was conducted, and how we address humanized text\. The model architecture is depicted in[Figure 1](https://arxiv.org/html/2607.27183#S1.F1)\.

##### Backbone model

The model architecture ofPangram 4is based on a popular open\-weight MoE model\. We attach a single dense layer as a custom classification head for each task that uses hidden states computed by the shared backbone at the final sequence position as inputs\. For an input sequence of lengthSS, the shared backbone produces a hidden representation𝐡i∈ℝD\\mathbf\{h\}\_\{i\}\\in\\mathbb\{R\}^\{D\}at every positionii\. We attach an independent linear classification head for each task\. We train on three window\-level objectives: segment\-level edits, mixed\-authorship binary classification, and humanizer detection\. These tasks use the representation at the final supervised sequence position,𝐡S\\mathbf\{h\}\_\{S\}, as input\. We use this approach \(e\.g\., instead of mean pooling\) because under causal attention,𝐡S\\mathbf\{h\}\_\{S\}has access to the complete input window\. We also train a tokenwise provenance head that instead applies its linear projection to every𝐡i\\mathbf\{h\}\_\{i\}, producing a separate prediction at each token position\.

##### Base model ablations

We ran small\-scale, iso\-compute ablations against a variety of dense and sparse open\-weight models to select the most promising candidates \(performance is shown in[Table 1](https://arxiv.org/html/2607.27183#S4.T1)\)\. For each candidate, we additionally swept attention, attention \+ dense, and attention \+ dense \+ routed expert \(for MoEs\) LoRA module targeting\. Each candidate is evaluated by its FNR at a fixed FPR on our holdout dataset\. All further experimentation is done on this base model\.

Table 1:Iso\-compute backbone ablations\. Backbone identities are anonymized\.
##### Segment predictions

Following EditLens\(Thaiet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib53)\), we model the AI authorship attribution problem as a continuous spectrum rather than a binary classification task\. Each labeled document is chunked into segments of lengthS=512S=512tokens\. Each segment is then assigned a weighted AI fraction

fA​I=0\.5​CA​A\+CA​GCH\+CA​A\+CA​G,f\_\{AI\}=\\frac\{0\.5C\_\{AA\}\+C\_\{AG\}\}\{C\_\{H\}\+C\_\{AA\}\+C\_\{AG\}\},whereCHC\_\{\\textsc\{H\}\},CAAC\_\{\\textsc\{AA\}\}, andCAGC\_\{\\textsc\{AG\}\}are character counts attributed to human, AI\-assisted, and AI\-generated text, respectively\. We discretizefA​I∈\[0,1\]f\_\{AI\}\\in\[0,1\]into 15 ordered buckets and train a segment\-level classification head on the hidden state at the final supervised sequence position\.

##### Tokenwise predictions

Pangram 4significantly increases prediction granularity\. We are able to achieve this via a tokenwise prediction head\. LetH∈ℝS×DH\\in\\mathbb\{R\}^\{S\\times D\}denote the final\-layer hidden states for a window ofSStokens with hidden dimensionDD\. A shared linear token\-classification head,

Ztok=H​Wtok⊤,Wtok∈ℝ3×D,Z\_\{\\mathrm\{tok\}\}=HW\_\{\\mathrm\{tok\}\}^\{\\top\},\\qquad W\_\{\\mathrm\{tok\}\}\\in\\mathbb\{R\}^\{3\\times D\},produces logitsZtok∈ℝS×3Z\_\{\\mathrm\{tok\}\}\\in\\mathbb\{R\}^\{S\\times 3\}over the classes \{human,ai\-assisted,ai\-generated\} for each token\.

##### Repeat2

Tokenwise prediction introduces a new challenge with a causal\-LM backbone\. A segment\-level classifier can use last\-position logits to ensure the classification head attends to the entire sequence\. Naively, this is inherently not possible for a tokenwise head—the hidden state for tokenxix\_\{i\}can ordinarily attend only to the prefixx≤ix\_\{\\leq i\}\. This limitation manifests as significantly worse prediction accuracy on tokens at the start of a sequence\. To fix this, we repeat the input sequence twice, as suggested by Repeat2\(Leviathanet al\.,[2025](https://arxiv.org/html/2607.27183#bib.bib68)\), such that

x~=\(x1,…,xS,x1,…,xS\)\.\\widetilde\{x\}=\(x\_\{1\},\\ldots,x\_\{S\},x\_\{1\},\\ldots,x\_\{S\}\)\.This Repeat2 construction gives every supervised token access to the complete document context\. We mask the token\-classification loss over the entire first copy and supervise only the corresponding tokens in the second copy:

y~=\(𝚒𝚐𝚗𝚘𝚛𝚎,…,𝚒𝚐𝚗𝚘𝚛𝚎⏟S,y1,…,yS\)\.\\widetilde\{y\}=\(\\underbrace\{\\mathtt\{ignore\},\\ldots,\\mathtt\{ignore\}\}\_\{S\},y\_\{1\},\\ldots,y\_\{S\}\)\.
During training, Repeat2 is applied to each sampled tokenwise window\. At inference time, long documents are divided into overlapping windows of at mostS=512S=512tokens with a stride of256256tokens, giving adjacent full\-length windows a50%50\\%overlap\. Repeat2 is applied to each window, and the resulting predictions are aligned using their source\-token offsets and assembled before structured decoding\. This allows us to aggregate logits for the same token across all windows in which it appears, such that the output is a document\-length sequence with a three\-class vector for each token in its original position\.

##### Mixed\-authorship head

Stage 2 additionally trains a binary, window\-level mixed\-authorship head\. Its training target is positive when more than15%15\\%of the supervised tokens in a window belong to provenance classes other than the dominant class\. We use this head to predict whether a window contains a meaningful mix of token labels, which is useful for downstream decoding as we will discuss in[Section˜4\.3](https://arxiv.org/html/2607.27183#S4.SS3)\.

##### Humanizer Head

Pangram 4introduces a new supplemental task: humanization detection\. We train a probe on the shared last\-position logits that learns to differentiate regular AI text from humanized AI text\. We train this classification head as a probe to reduce the risk of humanizer\-specific learned representations increasing our false positive rate on the main AI detection task\. In controlled ablations, letting gradients flow through the base model from the humanizer head kept top\-level FPR equivalent, but concentrated the failures in specific registers: academic writing and ESL essays\. We opt to train the humanizer head as a probe to prevent the adverse impact of false positives\.

### 4\.2Training Process

Pangram 4is trained using a two\-stage process: first, we train a checkpoint supervised on just the segment and humanizer objectives\. We then merge the LoRA adapters into the base model, initialize a fresh LoRA adapter, and continue training with the addition of the tokenwise objective\. We use this two\-stage approach as an efficiency optimization\. The Repeat2 approach used in tokenwise training results in roughly half the throughput since we process twice as many tokens\. Stage 1 training lets us bootstrap useful representations in a more compute\-efficient manner, allowing us to train on the tokenwise objective for fewer total steps\.[Table 2](https://arxiv.org/html/2607.27183#S4.T2)summarizes the two stages\.

Table 2:Pangram 4training stages\. Stop\-gradient prevents the humanizer loss from updating the shared backbone in both stages\.##### Active learning and hard negatives

After training thePangram 4model candidate, we do a round of offline active learning\. This process involves running inference on a large pool of reserved data to find hard negatives: human text that the current checkpoint labels as AI\. These samples are incorporated into our data pipeline, synthetically mirrored, and added to the training dataset\.Pangram 4is then retrained from scratch on the resulting combined dataset\.

### 4\.3Tokenwise Postprocessing

Compared to Pangram 3,Pangram 4uses a much more robust postprocessing algorithm that combines both token\-level and segment\-level signals and simultaneously handles calibration and smoothing\. The new postprocessing algorithm is therefore much more precise in boundary detection and better calibrated on long documents\.

#### 4\.3\.1Motivation

##### Long documents

The model’s context length is 512 tokens\. In Pangram 3, postprocessing handles longer documents by repeatedly scoring candidate text spans, but this procedure is imprecise and computationally inefficient\. Our new algorithm instead aggregates predictions over a single sliding\-window pass and combines all observations in a CRF decoding problem\.

##### Fine\-grained Detection

Boundary detection is particularly difficult for heterogeneous documents that interleave human\-written, AI\-assisted, and AI\-generated passages\. A tokenwise decoder increases the model’s expressiveness by assigning one of these three authorship states to every token before converting contiguous labels into character\-aligned segments\.

##### Global Calibration

In Pangram 3, we calibrated individual segments and subsequently aggregated them\. Consequently, the effect of aggregation on the false positive and false negative tradeoff was difficult to control\. The tokenwise algorithm unifies calibration and postprocessing by producing a calibrated unary score for every token and decoding all token labels jointly\.

#### 4\.3\.2Inputs and Outputs

For a document containingNNsource tokens, we divide the document into overlapping windows of at most 512 source tokens with a stride of 256; the final window is anchored to the end of the document\. In Repeat2, the source tokens in each window are concatenated twice before model inference\. Let𝒲​\(i\)\\mathcal\{W\}\(i\)denote the windows that contain source tokenii\. Each windowwwproduces three relevant observations:

1. 1\.a 15\-bucket segment\-head distribution indicating the overall estimate of the fraction of AI content in the segment:𝐩ws∈ℝ15\\mathbf\{p\}^\{\\,s\}\_\{w\}\\in\\mathbb\{R\}^\{15\};
2. 2\.a ternary token\-head logit vector𝐳w,it∈ℝ3\\mathbf\{z\}^\{\\,t\}\_\{w,i\}\\in\\mathbb\{R\}^\{3\}for each token in the window; and
3. 3\.a binary mixed\-document distribution, whose positive\-class probability is denotedqwq\_\{w\}\. We supervise this as an auxiliary task in which the model is trained to produceTrueif the text has any mix of AI and human content, andFalseif the text has homogeneous authorship\.

For a document, the post\-processor returns a sequence of segments, each of which has a hard prediction of eitherHuman,AI\-Assisted, orAI\-Generated\.

#### 4\.3\.3Algorithm

##### Sliding\-window inference

We run a forward pass of the model for every overlapping window\. This creates multiple model observations for tokens that appear in more than one window\. We then collapse these observations into one ternary unary potential per source token and one transition score per adjacent token pair\.

##### Segment\-head projection

The 15\-bucket head estimates the weighted AI fraction as defined in[subsection 3\.5](https://arxiv.org/html/2607.27183#S3.SS5)\. Each bucket is projected onto three class anchors using triangular basis functions:

ϕhuman​\(f\)\\displaystyle\\phi\_\{\\mathrm\{human\}\}\(f\)=max⁡\(0,1−2​f\),\\displaystyle=\\max\(0,1\-2f\),\(1\)ϕassisted​\(f\)\\displaystyle\\phi\_\{\\mathrm\{assisted\}\}\(f\)=max⁡\(0,1−2​\|f−0\.5\|\),\\displaystyle=\\max\\\!\\left\(0,1\-2\\lvert f\-0\.5\\rvert\\right\),\(2\)ϕAI​\(f\)\\displaystyle\\phi\_\{\\mathrm\{AI\}\}\(f\)=max⁡\(0,2​f−1\)\.\\displaystyle=\\max\(0,2f\-1\)\.\(3\)The ternary document prior for windowwwis a weighted average of the basis functions weighted by the predicted bucket probability vector\.

πw​\(c\)=∑b=014pws​\(b\)​ϕc​\(fb\)∑c′∑b=014pws​\(b\)​ϕc′​\(fb\)\.\\pi\_\{w\}\(c\)=\\frac\{\\sum\_\{b=0\}^\{14\}p^\{\\,s\}\_\{w\}\(b\)\\phi\_\{c\}\(f\_\{b\}\)\}\{\\sum\_\{c^\{\\prime\}\}\\sum\_\{b=0\}^\{14\}p^\{\\,s\}\_\{w\}\(b\)\\phi\_\{c^\{\\prime\}\}\(f\_\{b\}\)\}\.This prior peaks at human for bucket 0, AI\-assisted near the middle buckets, and AI\-generated near bucket 14\.

##### Calibrated unary potentials

Raw token and segment logits are averaged across all windows containing tokenii, reducing the multiple observations of each token from sliding\-window inference into a single 6\-vector\.

For each token, the calibration feature vector is

𝐱i=\[ℓit​\(human\)ℓit​\(assisted\)ℓit​\(AI\)ℓis​\(human\)ℓis​\(assisted\)ℓis​\(AI\)\]⊤\.\\mathbf\{x\}\_\{i\}=\\begin\{bmatrix\}\\ell^\{\\,t\}\_\{i\}\(\\mathrm\{human\}\)&\\ell^\{\\,t\}\_\{i\}\(\\mathrm\{assisted\}\)&\\ell^\{\\,t\}\_\{i\}\(\\mathrm\{AI\}\)&\\ell^\{\\,s\}\_\{i\}\(\\mathrm\{human\}\)&\\ell^\{\\,s\}\_\{i\}\(\\mathrm\{assisted\}\)&\\ell^\{\\,s\}\_\{i\}\(\\mathrm\{AI\}\)\\end\{bmatrix\}^\{\\\!\\top\}\.A multinomial logistic calibration model produces the CRF unary potential

ui​\(c\)=αc\+𝐰c⊤​𝐱i\+δc,u\_\{i\}\(c\)=\\alpha\_\{c\}\+\\mathbf\{w\}\_\{c\}^\{\\top\}\\mathbf\{x\}\_\{i\}\+\\delta\_\{c\},whereαc\\alpha\_\{c\}is a fitted intercept,𝐰c\\mathbf\{w\}\_\{c\}contains the fitted feature weights, andδc\\delta\_\{c\}is an optional class\-prior adjustment\. These parameters are fitted on held\-out calibration data and may vary between model releases\.

##### Linear\-chain CRF

The calibrated unary scores define a three\-state linear\-chain conditional random field\. For adjacent labelsaaandbb, the Potts transition potential at boundaryiiis

τi​\(a,b\)=\{0,a=b,min⁡\(0,−λ\+γ​mi\),a≠b,\\tau\_\{i\}\(a,b\)=\\begin\{cases\}0,&a=b,\\\\ \\min\\\!\\left\(0,\-\\lambda\+\\gamma m\_\{i\}\\right\),&a\\neq b,\\end\{cases\}wheremim\_\{i\}is the center\-weighted mixed log\-odds at the boundary,λ≥0\\lambda\\geq 0is the smoothness penalty, andγ\\gammacontrols how strongly mixed evidence can relax that penalty\. Both values are selected using calibration data\.

The maximum a posteriori label sequence is

𝐓^=arg⁡max𝐓∈\{0,1,2\}N​\[∑i=1Nui​\(Ti\)\+∑i=1N−1τi​\(Ti,Ti\+1\)\]\.\\widehat\{\\mathbf\{T\}\}=\\underset\{\\mathbf\{T\}\\in\\\{0,1,2\\\}^\{N\}\}\{\\arg\\max\}\\left\[\\sum\_\{i=1\}^\{N\}u\_\{i\}\(T\_\{i\}\)\+\\sum\_\{i=1\}^\{N\-1\}\\tau\_\{i\}\(T\_\{i\},T\_\{i\+1\}\)\\right\]\.

##### Decoding and marginals

Multiclass Viterbi decoding computes𝐓^\\widehat\{\\mathbf\{T\}\}\. A numerically stable forward–backward pass separately computes the token marginals

P​\(Ti=c∣all model observations\)\.P\(T\_\{i\}=c\\mid\\text\{all model observations\}\)\.Viterbi labels determine segment boundaries, while the marginals are retained as token\-level class probabilities\.

##### Sentence majority voting

Product output is constrained to sentence\-level resolution\. Sentences end at terminal punctuation \(including a trailing quote or bracket\), a blank line, or the end of the document\. Tokens are assigned to sentences by character midpoint\. Every token in a sentence is replaced by the sentence’s majority Viterbi label\. Deterministic tie\-breaking is used to make the operation fully reproducible\.

##### Minimum segment length

After sentence voting, the decoder repeatedly finds the first contiguous label run shorter than a configured minimumLminL\_\{\\min\}and merges it into a neighbor:

1. 1\.if only one neighbor exists, that neighbor’s label is used;
2. 2\.if both neighbors have the same label, that label is used;
3. 3\.otherwise, the longer neighboring run supplies the label; and
4. 4\.a deterministic rule resolves equal\-length ties\.

Merging continues until every remaining run has at leastLminL\_\{\\min\}tokens or the document contains only one run\. The value ofLminL\_\{\\min\}is a configurable postprocessing choice rather than a fixed property of the model\.

#### 4\.3\.4Segments and Document\-Level Prediction

Adjacent tokens with the same final constrained label are coalesced and mapped back to a complete partition of the original character sequence\. Document fractions are computed by character length:

fc=number of source characters assigned to class​cnumber of source characters in the document\.f\_\{c\}=\\frac\{\\text\{number of source characters assigned to class \}c\}\{\\text\{number of source characters in the document\}\}\.

#### 4\.3\.5Confidence

Token class marginals are computed from the CRF before sentence\-majority voting and minimum\-run merging\. Token confidence is the largest marginal probability, and segment confidence is the mean token confidence within the segment\. Therefore, confidence measures the peakedness of the unconstrained CRF posterior; it is not a recalibrated probability that the final constrained label is correct\.

## 5Evaluations

We evaluatePangram 4on human, AI\-generated, and AI\-assisted data over a range of in\-domain and challenging benchmarks\. The goal is to evaluate performance across representative real\-world use cases across varying domains and languages\.

### 5\.1Evaluation Metrics

Our final predictionZtok∈ℝS×3Z\_\{\\mathrm\{tok\}\}\\in\\mathbb\{R\}^\{S\\times 3\}whereSSis the number of segments as described in Section[4](https://arxiv.org/html/2607.27183#S4)must be decoded into a document\-level prediction for evaluation purposes\.

For purposes of evaluation, we decode the final tokenwise output into a documentwise output using the following rule\-based method:

y^=\{Human,if​fhuman≥0\.90,AI,if​fAI≥0\.80,Mixed,otherwise\.\\widehat\{y\}=\\begin\{cases\}\\mathrm\{Human\},&\\text\{if \}f\_\{\\mathrm\{human\}\}\\geq 0\.90,\\\\ \\mathrm\{AI\},&\\text\{if \}f\_\{\\mathrm\{AI\}\}\\geq 0\.80,\\\\ \\mathrm\{Mixed\},&\\text\{otherwise\}\.\\end\{cases\}
When we evaluate our model on fully human and fully AI text, we use standard binary classification metrics\. We define aFalse Positiveto be a prediction ofMixedorAIwhen the ground truth isHuman, and aFalse Negativeto be a prediction ofHumanorMixedwhen the ground truth is AI\. Note that a prediction ofMixedcounts as a failure for both false positive rate and false negative rate measurements, making our metrics stricter than threshold\-based calibration on a binary logit distribution as is typical for other methods\. For AUROC, TPR @ X% FPR and other rank\-based metrics, we directly use the post\-processed weighted AI fraction to rank the texts in the corpus\.

### 5\.2Overall False Negative Rate

We generate 520,000 AI outputs and run Pangram on them to identify false negatives\. At the production operating point,Pangram 4achieves anoverall false negative rate of 0\.3396%\. This is an improvement over Pangram 3\.3\.2, which reports a FNR of 1\.9942% on the same dataset\.

##### Synthetic benchmark construction

We evaluatePangram 4on a comprehensive synthetic benchmark of 520,000 examples generated by 26 frontier open\-source and closed\-weight models\. First, we useMistral\-Small\-24B\-Instruct\-2501\(Mistral AI,[2025](https://arxiv.org/html/2607.27183#bib.bib82)\)to filter prompts from the Chatbot Arena Conversations Dataset\(Zhenget al\.,[2023](https://arxiv.org/html/2607.27183#bib.bib1)\)\. We chose the Chatbot Arena Conversations dataset so that the prompts in the benchmark reflect the real\-world distribution of user prompts to LLMs\. We filtered for prompts that request an open\-ended generation or pose an open\-ended question\. Multiple\-choice questions, math, and coding\-related inquiries were considered out of scope for this benchmark\. After identifying 20,000 in\-scope prompts, we generate a response for each prompt using each of the 26 LLMs for a total of 520,000 attempted samples, of which 519,993 were successfully scored\. We found 1,766 false negatives, yielding an FNR of 0\.3396%, with a 95% Wilson confidence interval of \(0\.3242%, 0\.3558%\)\.

##### FNR by generation model

[Table 3](https://arxiv.org/html/2607.27183#S5.T3)displays results by generator model\. Anthropic models have the lowest FNR, at 0\.195%, and OpenAI models have an FNR of 0\.316%\. We find that there is no significant correlation between release date and FNR\.

CompanyModel familyGeneratorRelease datePangram 3\.3\.2Pangram 4AnthropicClaudeClaude Fable 5\(Anthropic,[2026a](https://arxiv.org/html/2607.27183#bib.bib12)\)Jun\. 9, 20261\.030%0\.255%Claude Opus 5\(Anthropic,[2026c](https://arxiv.org/html/2607.27183#bib.bib83)\)Jul\. 24, 20260\.585%0\.195%Claude Sonnet 5\(Anthropic,[2026e](https://arxiv.org/html/2607.27183#bib.bib17)\)Jun\. 30, 20260\.750%0\.140%Claude Opus 4\.8\(Anthropic,[2026b](https://arxiv.org/html/2607.27183#bib.bib16)\)May 28, 20260\.885%0\.235%Claude Sonnet 4\.6\(Anthropic,[2026d](https://arxiv.org/html/2607.27183#bib.bib15)\)Feb\. 17, 20261\.955%0\.215%Claude Haiku 4\.5\(Anthropic,[2025](https://arxiv.org/html/2607.27183#bib.bib14)\)Oct\. 15, 20250\.805%0\.130%Family aggregate–1\.002%0\.195%DeepSeekV4DeepSeek V4 Flash\(DeepSeek\-AI,[2026](https://arxiv.org/html/2607.27183#bib.bib18)\)Apr\. 24, 20261\.625%0\.455%DeepSeek V4 Pro\(DeepSeek\-AI,[2026](https://arxiv.org/html/2607.27183#bib.bib18)\)Apr\. 24, 20262\.050%0\.320%Family aggregate–1\.838%0\.388%GoogleGeminiGemini 3\.1 Pro Preview\(Google,[2026a](https://arxiv.org/html/2607.27183#bib.bib19)\)Feb\. 19, 20266\.380%0\.555%Gemini 3\.5 Flash\(Google,[2026b](https://arxiv.org/html/2607.27183#bib.bib20)\)May 19, 20263\.895%0\.440%Family aggregate–5\.138%0\.498%GoogleGemmaGemma 4 31B IT\(Google,[2026c](https://arxiv.org/html/2607.27183#bib.bib21)\)Apr\. 2, 20262\.910%0\.430%MetaLlamaLlama 3\.3 70B Instruct\(Meta,[2024](https://arxiv.org/html/2607.27183#bib.bib22)\)Dec\. 6, 20244\.405%0\.550%Mistral AIMistralMistral Medium 3\.5\(Mistral AI,[2026](https://arxiv.org/html/2607.27183#bib.bib23)\)Apr\. 29, 20260\.835%0\.400%Moonshot AIKimiKimi K2\.6\(Moonshot AI,[2026](https://arxiv.org/html/2607.27183#bib.bib24)\)Apr\. 20, 20262\.560%0\.395%NVIDIANemotronNemotron 3 Ultra 550B\-A55B\(NVIDIA,[2026](https://arxiv.org/html/2607.27183#bib.bib25)\)Jun\. 4, 20261\.320%0\.310%OpenAIGPTGPT\-5\.6 Sol\(OpenAI,[2026c](https://arxiv.org/html/2607.27183#bib.bib13)\)Jul\. 9, 20263\.240%0\.450%GPT\-5\.6 Terra\(OpenAI,[2026c](https://arxiv.org/html/2607.27183#bib.bib13)\)Jul\. 9, 20262\.040%0\.345%GPT\-5\.5\(OpenAI,[2026b](https://arxiv.org/html/2607.27183#bib.bib29)\)Apr\. 23, 20261\.645%0\.345%GPT\-5\.4\(OpenAI,[2026a](https://arxiv.org/html/2607.27183#bib.bib27)\)Mar\. 5, 20261\.145%0\.270%GPT\-5\.4 Mini\(OpenAI,[2026a](https://arxiv.org/html/2607.27183#bib.bib27)\)Mar\. 17, 20260\.900%0\.330%GPT\-OSS 120B\(OpenAI,[2025](https://arxiv.org/html/2607.27183#bib.bib26)\)Aug\. 5, 20250\.395%0\.155%Family aggregate–1\.561%0\.316%Alibaba CloudQwenQwen 3\.7 Max\(Qwen,[2026](https://arxiv.org/html/2607.27183#bib.bib30)\)May 19, 20262\.545%0\.315%TencentHunyuanHY3 Preview\(Tencent,[2026](https://arxiv.org/html/2607.27183#bib.bib31)\)Apr\. 23, 20261\.830%0\.330%Thinking MachinesInklingInkling\(Thinking Machines Lab,[2026](https://arxiv.org/html/2607.27183#bib.bib32)\)Jul\. 15, 20260\.300%0\.190%xAIGrokGrok 4\.3\(xAI,[2026](https://arxiv.org/html/2607.27183#bib.bib33)\)Apr\. 17, 20262\.645%0\.605%Z\.aiGLMGLM 5\.2\(Z\.ai,[2026](https://arxiv.org/html/2607.27183#bib.bib34)\)Jun\. 13, 20263\.175%0\.470%Overall1\.9942%0\.3396%

Table 3:False negative rate by generator model and family\.

### 5\.3Overall False Positive Rate

We evaluatePangram 4on over a million human\-written English texts and find that we have a false positive rate of 0\.0041%, with a 95% confidence interval of \(0\.0032%, 0\.0050%\)\. To put this in perspective, this is one false positive for every∼\\sim24,300 requests\. This is also an improvement to Pangram 3\.3\.2, which has a false positive rate of 0\.0539% on the same dataset\.

### 5\.4Performance by Length

AI detection is often harder on shorter texts, due to less text signal to compute predictions on\. We evaluate performance by length, finding thatPangram 4outperforms Pangram 3\.3\.2 at every length bucket, as displayed in[Figure 4](https://arxiv.org/html/2607.27183#S5.F4)\.

![Refer to caption](https://arxiv.org/html/2607.27183v1/x3.png)

![Refer to caption](https://arxiv.org/html/2607.27183v1/x4.png)

Figure 4:Error rates by document length for Pangram 3\.3\.2 andPangram 4\.
### 5\.5Performance on Mixed Authorship Text

#### 5\.5\.1Homogeneous Mixed Eval

##### FPR on AI\-Polished Text

We defineAI\-Assisted False Positive Rate \(FPR\)as the rate at which a model incorrectly flags minor edits made by AI as fully AI\-generated\. To evaluate the performance of our model at distinguishing lightly AI\-edited texts from fully AI\-generated texts, we create a dataset of AI\-proofread and AI\-polished texts\. We use consumer products \(Grammarly333[https://app\.grammarly\.com/](https://app.grammarly.com/), Apple Intelligence, and Gemini in Google Docs\) and frontier LLMs \(Opus 4\.8, Gemini 3\.1, and GPT\-5\.5\) to apply light edits to 11,363 pieces of human\-written student writing from ELLIPSE\(Crossleyet al\.,[2023](https://arxiv.org/html/2607.27183#bib.bib79)\), PELIC\(Juffset al\.,[2020](https://arxiv.org/html/2607.27183#bib.bib80)\), and ICNALE\(Ishikawa,[2019](https://arxiv.org/html/2607.27183#bib.bib81)\)\. For example, we used the Writing Tools feature embedded in Apple’s Pages word editor to “Proofread" some texts or prompted Opus 4\.8 to “Correct minor academic style issues while keeping the draft recognizably human\." For the full list of editing tools and prompts, see Table[16](https://arxiv.org/html/2607.27183#A2.T16)\. In Table[4](https://arxiv.org/html/2607.27183#S5.T4), we show that Pangram 3\.3\.2 classifies AI\-polished text asAI0\.18% of the time\.Pangram 4only flags AI\-polished text asAI0\.01% of the time—an 18x reduction in AI\-Assisted FPR\.

Table 4:Prediction distribution and AI\-Assisted FPR on our AI polish evaluation set\. Cells report count \(percentage\)\.nn= 11,363 texts\.
##### Recall on AI\-Edited Texts

We defineAI\-Assisted Recallas the rate at which a model classifies an AI\-assisted text asMixed\. Because there is no single definition for when a text should be considered AI\-edited, we create two datasets with different standards for AI assistance\.

We create the first dataset in a similar fashion to our AI Polish evaluation dataset\. We use the same consumer products and frontier LLMs to apply editing instructions or prompts that demonstrate a clear intent to substantially modify the source text\. For the full list of editing tools and prompts, see[Table 17](https://arxiv.org/html/2607.27183#A2.T17)\. As in our AI Polish dataset, these edits are applied to 14,990 human\-written student essays\. In[Table 5](https://arxiv.org/html/2607.27183#S5.T5), we show that Pangram 3\.3\.2 has an AI\-Assisted Recall of only 5\.54%, predictingHumanthe majority \(79\.23%\) of the time\.Pangram 4has a much\-improved AI\-Assisted Recall of 55\.01% on the same edited texts, predictingMixed10 times as often as Pangram 3\.3\.2\.

Table 5:Prediction distribution and AI\-Assisted Recall on our AI Editing Prompts evaluation set\. Cells report count \(percentage\)\.nn= 14,990 texts\.We also create a second dataset where AI\-assistance is defined by a quantitative metric: the Soft N\-grams Labeling approach from[subsection 3\.5](https://arxiv.org/html/2607.27183#S3.SS5)\. First, we mine the WildChat dataset\(Zhaoet al\.,[2024](https://arxiv.org/html/2607.27183#bib.bib78)\)using GPT\-5\.5 nano for over 5,000 unique editing prompts submitted by users that are neither domain\-specific \(i\.e\., related to a field or discipline\) nor source\-specific \(i\.e\., mentioning specific words, phrases, or arguments that should be modified\)\. Then we randomly apply these edits using one of Opus 4\.8, Gemini 3\.1, and GPT\-5\.5 to human\-authored Reddit posts, student essays, and creative writing\. Finally, we score all resulting edited texts using Soft N\-Grams Labeling and filter our dataset to 4,826 examples with an AI fraction of between 0\.25 and 0\.75\. We deem texts with this score to be AI\-assisted by any reasonable standard, as a score of 0\.25 means that half of the words have been modified by AI or 25% of the words have been newly generated by AI \(or some combination of the two\)\. Conversely, a score of 0\.75 indicates that the LLM generated 75% of the words in the human\-written source\. In table[6](https://arxiv.org/html/2607.27183#S5.T6), we show that Pangram 3\.3\.2 only classifies 12\.6% of these AI\-assisted texts asMixed\. Additionally, Pangram 3\.3\.2 displays polarized behavior, preferring to classify these AI\-assisted texts asHumanorAIrather thanMixed\. On the other hand,Pangram 4classifies the majority of these texts asMixed, achieving an AI\-Assisted Recall of 65\.7%, five times that of Pangram 3\.3\.2\.

Table 6:Prediction distribution and AI\-Assisted Recall on our WildChat Edits evaluation set\. Cells report count \(percentage\)\.nn= 4,826 texts\.

#### 5\.5\.2Heterogeneous Mixed Eval

##### Dataset Construction

For heterogeneous mixed eval, we aim to evaluate the resolution of our detector\. We create datasets with interleaved human and AI text\. These datasets contain documents that alternateNNfully human sentences withNNfully AI\-generated sentences, forN∈\{1,4,8,12,16,20\}\.N\\in\\\{1,4,8,12,16,20\\\}\.We create these documents by taking long human\-written documents from our test set, and replacing alternatingNNsentence chunks with AI sentences from a random choice of GPT\-5\.4, Claude Sonnet 4\.6, and Gemini 2\.5 Flash\. We sample the original human documents from our creative writing and formal academic writing splits\.

##### Results

In table[7](https://arxiv.org/html/2607.27183#S5.T7), we report the tokenwise binary classification metrics stratified byNNon this heterogeneous mixed dataset, as well as the document mean average error \(MAE\), which is the average difference between the predicted fraction of AI tokens and the ground\-truth fraction of AI tokens \(lower is better\)\.Pangram 4is significantly more accurate than Pangram 3\.3\.2 at all resolutions on these challenging interleaved texts\.

Qualitatively,Pangram 4is able to predict AI text at a much finer\-grained resolution than Pangram 3\.3\.2\. We display a representative example below in Figure[5](https://arxiv.org/html/2607.27183#S5.F5)\.

![Refer to caption](https://arxiv.org/html/2607.27183v1/figures/hetero_mixed_result.png)

Figure 5:Pangram 3\.3\.2 vs\. 4 on an excerpt from a heterogeneous mixed document\.Pangram 4is able to detect AI at a much finer\-grainedresolution\.

Table 7:Comparison of Pangram 3\.3\.2 with adaptive boundaries andPangram 4\. Bold values indicate the better result\. Precision and recall treat AI text as the positive class\.

### 5\.6Performance on Languages Other than English

[Table 8](https://arxiv.org/html/2607.27183#S5.T8)reports the FPR and FNR for supported languages\. FPRs are measured on pre\-2022 per\-language subsets of FineWeb2\(Penedoet al\.,[2025](https://arxiv.org/html/2607.27183#bib.bib62)\)\. While not reported, we test all 104 languages reported in FineWeb2, finding only 14 false positives among 996,273 examples, for anoverall multilingual FPR of 0\.0014%with a 95% Wilson confidence interval of \(0\.0008%, 0\.0024%\)\. FNRs are measured across 18 supported languages using a test set of 190,149 multilingual AI generations, which are synthetic mirrors of English data, including how\-to articles, news articles, and reviews\.Pangram 4’soverall multilingual FNR is 1\.24%with a 95% Wilson confidence interval of \(1\.19%, 1\.29%\)\. We hypothesize that higher false negative rates in languages such as Urdu \(5\.31%\) and Persian \(3\.17%\) may originate from differences in the base model’s tokenization; however, the effects of tokenization on multilingual AI\-text detection remain unknown\.

Table 8:Multilingual false positive and false negative rates by language\.
### 5\.7Performance on Non\-native English

##### Datasets

There has been speculation and anxiety that AI detectors are biased against non\-native English speakers\(Lianget al\.,[2023](https://arxiv.org/html/2607.27183#bib.bib48)\)\. To address this concern, we evaluatePangram 4’s false positive rate on several public corpora of non\-native English text: ELLIPSE\(Crossleyet al\.,[2023](https://arxiv.org/html/2607.27183#bib.bib79)\), ICNALE\(Ishikawa,[2019](https://arxiv.org/html/2607.27183#bib.bib81)\)\(a dialogue dataset\), PELIC\(Juffset al\.,[2020](https://arxiv.org/html/2607.27183#bib.bib80)\), and the TOEFL dataset byLianget al\.\([2023](https://arxiv.org/html/2607.27183#bib.bib48)\)\. In[Table˜9](https://arxiv.org/html/2607.27183#S5.T9), we show thatPangram 4has a single raw false positive out of 24,586 samples, resulting in a 0\.0041% FPR that is the same as our overall FPR reported in Section[5\.3](https://arxiv.org/html/2607.27183#S5.SS3)\.

Table 9:False positive rates on human\-authored English learner writing and speech transcripts\.

### 5\.8Public Benchmarks

BenchmarkAUROC /AUCTPR atfixed FPRFPRFNRMeanaccuracyF1BinaryUChicago\(Jabarian and Imas,[2025](https://arxiv.org/html/2607.27183#bib.bib71)\)✓\\checkmark✓\\checkmark✓\\checkmark———VUB\(Van Vlasselaeret al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib74)\)——————GEDE\(Gehring and Paaßen,[2025](https://arxiv.org/html/2607.27183#bib.bib72)\)✓\\checkmark✓\\checkmark————Perkins et al\.\(Perkinset al\.,[2024](https://arxiv.org/html/2607.27183#bib.bib73)\)————✓\\checkmark—Epoch AI Style Imitation\(Lee,[2026](https://arxiv.org/html/2607.27183#bib.bib75)\)——✓\\checkmark✓\\checkmark——DetectRL\(Wuet al\.,[2024](https://arxiv.org/html/2607.27183#bib.bib44)\)✓\\checkmark————✓\\checkmarkMELD\-eval\(Liet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib39)\)✓\\checkmark✓\\checkmark————Sem\-Detect\(Duarteet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib40)\)✓\\checkmark✓\\checkmark————MixedSaha Peer Review\(Sahaet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib42)\)—✓\\checkmark✓\\checkmark———OpAI\-Bench\(Bsharatet al\.,[2026](https://arxiv.org/html/2607.27183#bib.bib76)\)——————Table 10:Metric coverage across the binary and mixed\-authorship evaluations\. A checkmark means that Section 5 reports that metric forPangram 4\.##### Summary of results

[Table 12](https://arxiv.org/html/2607.27183#S5.T12)comparesPangram 4, Pangram 3, and the strongest non\-Pangram comparator on every split reported in the detailed benchmark tables\. Comparisons between benchmarks are challenging, as there is no unified metric of success; some choose FNR/FPR, while others look at AUROC or other locally defined metrics\. A summary of what metrics are presented in each benchmark is shown in[Table 10](https://arxiv.org/html/2607.27183#S5.T10)\.

BenchmarkEvaluation splitPangram 4resultPangram 3 resultNext\-best resultBinaryUChicagoStandard, full lengthTPR@1%FPR: 100\.00%99\.86%GPTZero: 98\.64%Standard, under 50 wordsTPR@1%FPR: 99\.70%88\.87%GPTZero: 0\.00%Humanizer, full lengthTPR@1%FPR: 98\.93%98\.08%GPTZero: 44\.32%Humanizer, under 50 wordsTPR@1%FPR: 73\.32%61\.74%GPTZero: 0\.00%Human controlsFPR: 0\.00%0\.00%Originality: 0\.11%VUBFully AI\-generatedRecall: 100%97\.4%GPTZero: 0%GEDEOverallTPR@1%FPR: 100\.0%98\.9%GPTZero: 77\.6%Fully generatedTPR@1%FPR: 100\.0%100\.0%GPTZero: 93\.2%AI\-improved humanTPR@1%FPR: 100\.0%97\.4%GPTZero: 54\.7%HumanizedTPR@1%FPR: 100\.0%100\.0%GPTZero: 92\.6%Perkins et al\.Baseline AIMean accuracy: 100\.0%—Copyleaks: 73\.9%Manipulated AIMean accuracy: 94\.1%—Copyleaks: 58\.7%Epoch AI Style ImitationOverallFNR / FPR: 2\.86% / 0\.00%5\.05% / 0\.00%GPTZero: 5\.72% / 0\.00%DetectRLMulti\-domainF1: 96\.9398\.05Rob\-Base: 99\.75Multi\-LLMF1: 97\.2598\.39Rob\-Base: 99\.58Multi\-attackF1: 96\.0097\.53Rob\-Large: 99\.03Domain generalizationF1: 96\.9598\.05Rob\-Base: 83\.00LLM generalizationF1: 97\.2598\.39X\-Rob\-Base: 92\.73Attack generalizationF1: 95\.9997\.53Rob\-Base: 92\.37Time, testF1: 91\.3093\.73Rob\-Large: 85\.23Human writingF1: 90\.7791\.95Rob\-Base: 97\.34 / Rob\-Large: 94\.63OverallAverage F1: 95\.3096\.70Rob\-Base: 93\.02MELD\-evalGPT\-5\.4\-MiniTPR@1%FPR: 99\.989%99\.962%MELD: 100\.0%Gemini\-3\-FlashTPR@1%FPR: 99\.991%99\.931%MELD: 99\.7%Claude\-Haiku\-4\.5TPR@1%FPR: 99\.989%99\.949%MELD: 100\.0%Qwen\-3\.6\-PlusTPR@1%FPR: 99\.993%99\.985%MELD: 99\.9%Generator overallTPR@1%FPR: 99\.990%99\.957%MELD: 99\.9%No attackTPR@1%FPR: 100\.00%99\.99%—HomoglyphTPR@1%FPR: 99\.97%99\.99%—Number perturbationTPR@1%FPR: 100\.00%99\.98%—SynonymTPR@1%FPR: 99\.96%99\.72%—Upper–lower flipTPR@1%FPR: 99\.99%99\.99%—WhitespaceTPR@1%FPR:99\.99%99\.99%—Zero\-width spaceTPR@1%FPR: 100\.00%99\.99%—Attack overallTPR@1%FPR: 99\.99%99\.95%—Sem\-DetectBinary detectionTPR@0\.1%FPR : 95\.5%100\.0%Sem\-Detect: 76\.0%MixedSaha Peer ReviewEasy, AI\-BPTPR: 98\.2%100\.0%Fast\-DetectGPT: 100\.0%Easy, AI\-EPTPR: 100\.0%100\.0%Fast\-DetectGPT: 100\.0%Easy, AI\-HITPR: 96\.3%100\.0%Fast\-DetectGPT \+ Context: 99\.3%Easy, H\-AIFPR: 1\.4%14\.9%Binoculars: 0\.0%Hard, AI\-BPTPR: 100\.0%100\.0%GPTZero: 96\.0%Hard, AI\-EPTPR: 100\.0%100\.0%GPTZero: 95\.8%Hard, AI\-HITPR: 98\.9%100\.0%GPTZero: 89\.4%Hard, H\-AIFPR: 2\.5%4\.5%LogLikelihood: 0\.0%HumanFPR: 0\.0%0\.0%LogLikelihood: 0\.0%OpAI\-Benchv​0v0, none \(target: 0\.00%\)AI\+Assisted mean: 0\.00%0\.26%—v​1v1, polish \(target: 20\.75%\)AI\+Assisted mean: 0\.01%0\.41%—v​2v2, paraphrase \(target: 30\.95%\)AI\+Assisted mean: 3\.18%1\.84%—v​3v3, style rewrite \(target: 46\.47%\)AI\+Assisted mean: 13\.59%4\.60%—v​4v4, compress \(target: 49\.61%\)AI\+Assisted mean: 5\.48%3\.70%—v​5v5, expand \(target: 68\.58%\)AI\+Assisted mean: 32\.57%12\.38%—v​6v6, style rewrite \(target: 81\.62%\)AI\+Assisted mean: 56\.17%20\.73%—v​7v7, paraphrase \(target: 92\.57%\)AI\+Assisted mean: 67\.94%22\.91%—v​8v8, polish \(target: 99\.35%\)AI\+Assisted mean: 69\.50%25\.07%—Table 12:Results on the native metrics reported for each public benchmark\. The next\-best entry is the strongest non\-Pangram result available for each metric\. Full comparisons appear in §[C](https://arxiv.org/html/2607.27183#A3)\.
##### UChicago

Jabarian and Imas\([2025](https://arxiv.org/html/2607.27183#bib.bib71)\)pairs authentic human writing from several genres with outputs from multiple LLMs across a range of document lengths\. It also includes dedicated evaluations of ultra\-short passages and texts processed by commercial humanizers\. Full detector comparisons and default\-threshold false positive rates are reported in[Table 18](https://arxiv.org/html/2607.27183#A3.T18)and[Table 19](https://arxiv.org/html/2607.27183#A3.T19)\.

##### VUB

Van Vlasselaeret al\.\([2026](https://arxiv.org/html/2607.27183#bib.bib74)\)focuses on long\-form academic papers and evaluates fully human\-written, fully AI\-generated, hybrid, and humanized AI\-generated documents\. Because the synthetic documents have known ground\-truth AI proportions, the benchmark can test whether detector scores reflect the extent of AI involvement rather than merely provide binary classifications\. Our evaluation uses the public fully AI\-generated subset, whose score distribution is reported in[Table 20](https://arxiv.org/html/2607.27183#A3.T20)\.

##### GEDE

Gehring and Paaßen\([2025](https://arxiv.org/html/2607.27183#bib.bib72)\)is an education\-focused benchmark containing more than 900 student\-written essays and over 12,500 LLM\-generated essays\. Its defining feature is a spectrum of student\-contribution levels, ranging from fully human writing through light LLM improvement and full generation to adversarial humanization\. Results for every evaluated split are reported in[Table 21](https://arxiv.org/html/2607.27183#A3.T21)\.

##### Perkins et al

Perkinset al\.\([2024](https://arxiv.org/html/2607.27183#bib.bib73)\)evaluates 15 original AI\-generated samples and 89 variants created using six detector\-evasion techniques\. It emphasizes adversarial robustness and the risks that unreliable detection poses to fairness and inclusivity in academic assessment\. Baseline and manipulated\-text results are reported in[Table 22](https://arxiv.org/html/2607.27183#A3.T22)\.

##### Epoch AI Style Imitation

Lee\([2026](https://arxiv.org/html/2607.27183#bib.bib75)\)contains 495 human passages and 594 AI passages: 297 generated from basic prompts and 297 prompted to imitate the styles of sampled authors\. We report document\-level false negative rates across both AI conditions, span\-level false negative rates for the Pangram models, and false positive rates on the human controls in[Table 23](https://arxiv.org/html/2607.27183#A3.T23)\.

##### DetectRL

Wuet al\.\([2024](https://arxiv.org/html/2607.27183#bib.bib44)\)evaluates detector robustness and generalization across domains, generators, attack strategies, text lengths, and real\-world factors in human writing\. Task 1 measures in\-domain performance across domains, generators, and attack types\. Task 4 applies paraphrasing, perturbation, and data mixing to human\-written texts to test how these transformations affect detector behavior\. The full zero\-shot and supervised leaderboard is reported in[Table 24](https://arxiv.org/html/2607.27183#A3.T24)\.

##### MELD\-eval

Liet al\.\([2026](https://arxiv.org/html/2607.27183#bib.bib39)\)introduces a controlled generator\-shift benchmark built from four current\-generation chat models and eight English domains\. It contains paired human texts, clean AI outputs, and attack\-augmented AI outputs, and it tests zero\-shot transfer to held\-out generators at low false positive rates\. Generator\-level and attack\-level results are reported in[Table 25](https://arxiv.org/html/2607.27183#A3.T25)and[Table 26](https://arxiv.org/html/2607.27183#A3.T26), respectively\.

##### Sem\-Detect

Duarteet al\.\([2026](https://arxiv.org/html/2607.27183#bib.bib40)\)introduces a dataset of more than 20,000 human\-written, AI\-generated, and LLM\-refined peer reviews from ICLR and NeurIPS, together with a detector that combines textual features with claim\-level semantic analysis\. The benchmark tests whether detectors can distinguish fully AI\-generated reviews from authentic or LLM\-refined human reviews whose original judgments are preserved\. Binary detection results are reported in[Table 27](https://arxiv.org/html/2607.27183#A3.T27)\.

##### Saha Peer Review

Sahaet al\.\([2026](https://arxiv.org/html/2607.27183#bib.bib42)\)simulates several levels of human–AI collaboration, including fully AI\-generated reviews, AI\-generated reviews conditioned on human\-provided key points, AI\-polished human reviews, and unmodified human reviews\. Its easy and hard subsets vary the diversity of generating models and prompts\. The full cohort\-level comparison is reported in[Table 28](https://arxiv.org/html/2607.27183#A3.T28)\.

##### OpAI\-Bench

Bsharatet al\.\([2026](https://arxiv.org/html/2607.27183#bib.bib76)\)constructs nine successive versions of each human\-written document using five AI editing operations and increasing predefined levels of AI coverage\. Because it provides character\-level authorship provenance, we compare the predicted AI\+Assisted fraction directly with the ground\-truth AI character fraction throughout the editing trajectory\. Stage\-level results are reported in[Table 29](https://arxiv.org/html/2607.27183#A3.T29)\.

### 5\.9Adversarial Robustness

We evaluate the robustness ofPangram 4to both humanizers and broader red\-teaming attacks\.

#### 5\.9\.1Robustness to Humanizer Attacks

We focus our evaluation of humanization on three domains in particular\. These domains are a manually collected set of commercial humanizers chosen by top web traffic in early 2026; a popular GitHub repository containing instructions for AI agents to humanize their text; and an academic benchmark of AI texts attacked in specific manners\.

##### How well does Pangram do at detecting humanized text?

We find Pangram is robust to humanization attempts\. We detect humanized text as AI\-generated 97\.67% of the time and as either Mixed or AI\-generated 98\.83% of the time\.

##### How well does Pangram distinguish between AI\-generated and humanized text?

We evaluate the humanizer classifier on paired unmodified and humanized AI\-generated texts\. For this evaluation, humanized AI is the positive class and the corresponding unmodified AI text is the negative class\. The humanizer classifier is only run if AI use is detected\.[Table 13](https://arxiv.org/html/2607.27183#S5.T13)reports aggregate performance\. For this separate AI\-versus\-humanized\-AI evaluation, the auxiliary head’s per\-system TPR ranges from 91\.52% to 99\.39% across commercial humanizers, as detailed alongside the primary\-detector results in[Table 14](https://arxiv.org/html/2607.27183#S5.T14)\.

AccuracyF1PrecisionRecallFPRFNR96\.82%0\.967898\.02%95\.57%1\.93%4\.43%Table 13:Auxiliary humanizer\-head performance on paired unmodified and humanized AI\-generated texts\. A false negative is a humanized text predicted as AI\-generated\. A false positive is AI\-generated text predicted as humanized\.Table 14:Performance ofPangram 4against different commercial evasion systems\. The first two columns report TPRs for humanized text from the Pangram AI\-use detector\. The rightmost column is the auxiliary humanizer head’s true\-positive rate\.
##### BLADER

BLADER444[blader/humanizeron GitHub](https://github.com/blader/humanizer)is a popular GitHub repository that contains instructions for LLMs on how to “remove signs of AI\-generated writing from text\.” As of publication, BLADER has nearly 32,000 stars on GitHub\. We report our false negative rate on text humanized using three frontier models prompted with BLADER’s instructions in[Table 15](https://arxiv.org/html/2607.27183#S5.T15)

Table 15:Performance ofPangram 4on text generated then rewritten using the BLADER series of agent skills\. False negative rates on BLADER\-adjusted AI\-generated text, grouped by the model used for humanization\.

#### 5\.9\.2Red\-Teaming

##### Manual Red\-Teaming

One week before launchingPangram 4, we gave all Pangram employees and a small group of early testers access to Pangram to stress\-test it for false positives and false negatives\. We used the feedback from the early access group to refine our calibration and postprocessing procedure and make small final adjustments to the model\.

##### Automated Red\-Teaming with AI Agents

We also gave two Codex agents with GPT\-5\.6 Sol access to thePangram 4API for 24 hours and put them on “Goal” mode with Full Access to attempt to foolPangram 4\. One agent was tasked with finding false positive gaps in our evaluation: kinds of text that are not explicitly present in our evaluation sets that could systematically have an elevated false positive rate\. The false positive search covered a wide variety of historical and contemporary literature, official and legal records, patents and filings, public minutes, engineering and incident records, medical reports, field notes, learner writing, and interviews\. Zero new false positives were found by the false positive red\-teaming agent in 24 hours\.

The second agent was tasked with finding systematic bypasses that could create false negatives repeatedly\. This agent tested several potential bypass hypotheses, such as imitating specific human voices and styles, using different operational and professional registers, imitating a notetaker, using much terser vocabulary, conditioning its output on detailed source writing, aggregating multiple AI documents together, reformatting data as PDFs, putting the text through multiple rounds of translation, and more\. The only bypass explicitly found by the agent was acting as a surgeon/pathologist dictating notes aloud\. We examined these false negatives by eye and deemed them to be out of scope: more context was given to the LLM than the output, and the outputs more resembled extremely terse factual bullets rather than open\-ended prose\.

From these two experiments, we deemed the model resilient to agent\-based adversarial attacks and sufficiently robust to release to the public\.

## 6Conclusion

We introduce our most powerful AI detector model,Pangram 4, with the purpose of detecting more granular AI usage and increasing both recall and specificity over Pangram 3\.3\.2\.Pangram 4is the first AI\-detection model that can predict not only document\-level AI usage, but also homogeneous and heterogeneous mixed text\. We intendPangram 4to increase visibility into how an author crafted a text, while still managing to keep our overall FPR at 0\.0041%\.

##### Limitations

WhilePangram 4is a state\-of\-the\-art AI detection model, its estimates are statistical, and both false positives and false negatives, although rare, do occur\. Its predictions are also to some degree a black box, and the reasons it makes particular predictions are difficult to understand\. Predictions for the same text in different contexts may also be inconsistent\. For example, a section of a paper run in isolation may receive a different prediction than it does within the full context of the paper\. In such cases, the additional context may give the model more information with which to make a holistic decision\. We are working to reduce these inconsistencies over time\. Finally,Pangram 4does not account for people who intentionally write like LLMs or who have absorbed elements of LLM style into their writing\. This form of data drift is a major focus of our ongoing research\.

Looking forward, we look to answer questions related to data drift, consistency, and interpretability, and to work toward the overall goal of understanding the provenance of AI\-generated text and how it influences the world around us\.

## AI Disclosure

We use AI agents extensively for coming up with research ideas, processing data, writing code, running experiments, and evaluating results\. We use some AI assistance in the writing of the technical report, for proofreading and review, but the majority of the writing is ours\. We claim full responsibility for the factual accuracy of the claims we make in this report\. For full disclosure, we provide a link to thePangram 4result on this technical report, and agree with its assessment of our authorship\.555[https://www\.pangram\.com/history/6c43abfc\-803e\-4dd5\-96d1\-8eba2768ba39](https://www.pangram.com/history/6c43abfc-803e-4dd5-96d1-8eba2768ba39)

![Refer to caption](https://arxiv.org/html/2607.27183v1/figures/pangram-result.png)Figure 6:Pangram AI result for this technical report\.
## Acknowledgements

We would like to thank the following people for their contributions toPangram 4\. We thank Artem Frenk for early discussions regarding tokenwise postprocessing and the HMM/CRF formulation\. We thank Fanyi Pan, Lu Lyu, Anna Lin, and Tianxu Zhou forPangram 4product support, design, UI/UX engineering, and testing to bringPangram 4to our users\. We thank Jocelyn James for graphic design contributions\. We thank Annamieka Aerts and James Freedman for helping with the technical writing and proofreading\. We thank Levi Goldstein and Lauryl Salami for data expertise and contributions\. We also thank our early testers for providing invaluable pre\-release feedback on the model\.

## References

- Claude Haiku 4\.5\.Note:Released October 15, 2025External Links:[Link](https://www.anthropic.com/news/claude-haiku-4-5)Cited by:[§3\.5](https://arxiv.org/html/2607.27183#S3.SS5.SSS0.Px1.p1.3),[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.7.7.3)\.
- Anthropic \(2026a\)Claude Fable 5 and Mythos 5\.Note:Released June 9, 2026External Links:[Link](https://www.anthropic.com/news/claude-fable-5-mythos-5)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p4.1),[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.2.2.3)\.
- Anthropic \(2026b\)Claude Opus 4\.8\.Note:Released May 28, 2026External Links:[Link](https://www.anthropic.com/news/claude-opus-4-8)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.5.5.3)\.
- Anthropic \(2026c\)Claude Opus 5\.Note:Large language modelExternal Links:[Link](https://www.anthropic.com/news/claude-opus-5)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.3.3.3)\.
- Anthropic \(2026d\)Claude Sonnet 4\.6\.Note:Released February 17, 2026External Links:[Link](https://www.anthropic.com/news/claude-sonnet-4-6)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.6.6.3)\.
- Anthropic \(2026e\)Claude Sonnet 5\.Note:Released June 30, 2026External Links:[Link](https://www.anthropic.com/news/claude-sonnet-5)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.4.4.3)\.
- E\. Artemova, J\. S\. Lucas, S\. Venkatraman, J\. Lee, S\. Tilga, A\. Uchendu, and V\. Mikhailov \(2025\)Beemo: Benchmark of Expert\-edited Machine\-generated Outputs\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 6992–7018\.External Links:[Link](https://aclanthology.org/2025.naacl-long.357/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.357),ISBN 979\-8\-89176\-189\-6Cited by:[§2\.3](https://arxiv.org/html/2607.27183#S2.SS3.p1.1)\.
- S\. M\. Bsharat, J\. Liu, X\. Zhao, T\. Yao, X\. Shang, Y\. Tang, J\. Cui, A\. Elhagry, S\. K\. A\. Khatib, H\. Li, S\. Khan, and Z\. Shen \(2026\)Operation\-Guided Progressive Human\-to\-AI Text Transformation Benchmark for Multi\-Granularity AI\-Text Detection\.External Links:2606\.06481,[Link](https://arxiv.org/abs/2606.06481)Cited by:[§2\.3](https://arxiv.org/html/2607.27183#S2.SS3.p1.1),[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px11.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.16.21.5.1)\.
- S\. Crossley, Y\. Tian, P\. Baffour, A\. Franklin, Y\. Kim, W\. Morris, M\. Benner, A\. Picou, and U\. Boser \(2023\)The English Language Learner Insight, Proficiency and Skills Evaluation \(ELLIPSE\) Corpus\.International Journal of Learner Corpus Research9\(2\),pp\. 248–269\.Note:Pre\-print available at[https://doi\.org/10\.5281/zenodo\.11217937](https://doi.org/10.5281/zenodo.11217937)External Links:[Document](https://dx.doi.org/10.1075/ijlcr.22026.cro)Cited by:[§5\.5\.1](https://arxiv.org/html/2607.27183#S5.SS5.SSS1.Px1.p1.1),[§5\.7](https://arxiv.org/html/2607.27183#S5.SS7.SSS0.Px1.p1.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-V4: Towards Highly Efficient Million\-Token Context Intelligence\.External Links:2606\.19348,[Link](https://arxiv.org/abs/2606.19348)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.10.10.3),[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.9.9.3)\.
- J\. Dolezal, S\. Alam, M\. Graham, and M\. Bohacek \(2026\)The Impact of AI\-Generated Text on the Internet\.External Links:2604\.26965,[Link](https://arxiv.org/abs/2604.26965)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p1.1)\.
- G\. Drayson, E\. Yilmaz, and V\. Lampos \(2025\)Machine\-generated text detection prevents language model collapse\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 29657–29673\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1506/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1506),ISBN 979\-8\-89176\-332\-6Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p3.1)\.
- A\. V\. Duarte, B\. Tufts, A\. Oke, F\. Fang, A\. L\. Oliveira, and L\. Li \(2026\)Sem\-Detect: Semantic Level Detection of AI Generated Peer\-Reviews\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=3S9eVockjL)Cited by:[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px9.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.14.14.3)\.
- N\. Dycke, M\. Sakharova, N\. Daheim, and I\. Gurevych \(2026\)’Your AI Text is not Mine’: Redefining and Evaluating AI\-generated Text Detection under Realistic Assumptions\.External Links:2606\.04906,[Link](https://arxiv.org/abs/2606.04906)Cited by:[§2\.3](https://arxiv.org/html/2607.27183#S2.SS3.p1.1)\.
- B\. Emi and M\. Spero \(2024\)Technical Report on the Pangram AI\-Generated Text Classifier\.External Links:2402\.14873,[Link](https://arxiv.org/abs/2402.14873)Cited by:[§2\.2](https://arxiv.org/html/2607.27183#S2.SS2.p1.1)\.
- L\. Gehring and B\. Paaßen \(2025\)Assessing LLM Text Detection in Educational Contexts: Does Human Contribution Affect Detection?\.External Links:2508\.08096,[Link](https://arxiv.org/abs/2508.08096)Cited by:[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px4.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.5.5.3)\.
- Google \(2026a\)Gemini 3\.1 Pro: A Smarter Model for Your Most Complex Tasks\.Note:Released in preview February 19, 2026External Links:[Link](https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-pro/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.12.12.3)\.
- Google \(2026b\)Gemini 3\.5: Frontier Intelligence with Action\.Note:Gemini 3\.5 Flash released May 19, 2026External Links:[Link](https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.13.13.3)\.
- Google \(2026c\)Gemma 4: Our Most Capable Open Models to Date\.Note:Released April 2, 2026External Links:[Link](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.15.15.3)\.
- S\. Ishikawa \(2019\)The ICNALE Spoken Dialogue: A new dataset for the study of Asian learners’ performance in L2 English interviews\.English Teaching74\(4\),pp\. 153–177\.External Links:[Document](https://dx.doi.org/10.15858/engtea.74.4.201912.153),[Link](https://files.eric.ed.gov/fulltext/EJ1284816.pdf)Cited by:[§5\.5\.1](https://arxiv.org/html/2607.27183#S5.SS5.SSS1.Px1.p1.1),[§5\.7](https://arxiv.org/html/2607.27183#S5.SS7.SSS0.Px1.p1.1)\.
- B\. Jabarian and A\. Imas \(2025\)Artificial Writing and Automated Detection\.NBER Working PaperTechnical Report34223,National Bureau of Economic Research\.External Links:[Document](https://dx.doi.org/10.3386/w34223),[Link](https://www.nber.org/papers/w34223)Cited by:[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px2.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.3.3.4)\.
- A\. Juffs, N\. Han, and B\. Naismith \(2020\)The University of Pittsburgh English Language Institute Corpus \(PELIC\)\.Zenodo\.Note:\[Data set\]External Links:[Document](https://dx.doi.org/10.5281/zenodo.3991977),[Link](https://github.com/ELI-Data-Mining-Group/PELIC-dataset)Cited by:[§5\.5\.1](https://arxiv.org/html/2607.27183#S5.SS5.SSS1.Px1.p1.1),[§5\.7](https://arxiv.org/html/2607.27183#S5.SS7.SSS0.Px1.p1.1)\.
- J\. Lee \(2026\)AI detectors rarely flag human writing, but sometimes miss AI text imitating real authors\.Note:Accessed: 2026\-07\-28External Links:[Link](https://epoch.ai/data-insights/ai-detectors-false-negatives)Cited by:[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px6.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.8.8.3)\.
- Y\. Leviathan, M\. Kalman, and Y\. Matias \(2025\)Prompt Repetition Improves Non\-Reasoning LLMs\.External Links:2512\.14982,[Link](https://arxiv.org/abs/2512.14982)Cited by:[§4\.1](https://arxiv.org/html/2607.27183#S4.SS1.SSS0.Px5.p1.2)\.
- C\. Li, C\. Wan, and J\. C\. Paetzold \(2026\)MELD: Multi\-Task Equilibrated Learning Detector for AI\-Generated Text\.External Links:2605\.06903,[Link](https://arxiv.org/abs/2605.06903)Cited by:[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px8.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.12.12.3)\.
- W\. Liang, M\. Yuksekgonul, Y\. Mao, E\. Wu, and J\. Zou \(2023\)GPT Detectors Are Biased Against Non\-Native English Writers\.Patterns4\(7\),pp\. 100779\.External Links:[Document](https://dx.doi.org/10.1016/j.patter.2023.100779),[Link](https://doi.org/10.1016/j.patter.2023.100779)Cited by:[§5\.7](https://arxiv.org/html/2607.27183#S5.SS7.SSS0.Px1.p1.1)\.
- E\. Masrour, B\. Emi, and M\. Spero \(2025\)DAMAGE: Detecting Adversarially Modified AI Generated Text\.External Links:2501\.03437,[Link](https://arxiv.org/abs/2501.03437)Cited by:[§3\.4](https://arxiv.org/html/2607.27183#S3.SS4.p1.1)\.
- Meta \(2024\)Llama 3\.3 70B Instruct Model Card\.Note:Released December 6, 2024External Links:[Link](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.16.16.3)\.
- Mistral AI \(2025\)Mistral Small 3\.Note:[https://mistral\.ai/news/mistral\-small\-3/](https://mistral.ai/news/mistral-small-3/)Cited by:[§5\.2](https://arxiv.org/html/2607.27183#S5.SS2.SSS0.Px1.p1.1)\.
- Mistral AI \(2026\)Mistral Medium 3\.5 Model Card\.Note:Released April 28, 2026External Links:[Link](https://docs.mistral.ai/models/model-cards/mistral-medium-3-5-26-04)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.17.17.3)\.
- Moonshot AI \(2026\)Kimi Research\.Note:Kimi K2\.6 released April 20, 2026External Links:[Link](https://www.kimi.com/blog/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.18.18.3)\.
- J\. Ng and V\. Abrecht \(2015\)Better Summarization Evaluation with Word Embeddings for ROUGE\.InProceedings of the 2015 Conference on Empirical Methods in Natural Language Processing,L\. Màrquez, C\. Callison\-Burch, and J\. Su \(Eds\.\),Lisbon, Portugal,pp\. 1925–1930\.External Links:[Link](https://aclanthology.org/D15-1222/),[Document](https://dx.doi.org/10.18653/v1/D15-1222)Cited by:[§3\.5](https://arxiv.org/html/2607.27183#S3.SS5.p2.2)\.
- NVIDIA \(2026\)NVIDIA Nemotron 3 Ultra Powers Faster, More Efficient Reasoning for Long\-Running Agents\.Note:Released June 4, 2026External Links:[Link](https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.19.19.3)\.
- OpenAI \(2025\)Introducing gpt\-oss\.Note:Released August 5, 2025External Links:[Link](https://openai.com/index/introducing-gpt-oss/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.25.25.3)\.
- OpenAI \(2026a\)Introducing GPT\-5\.4\.Note:Released March 5, 2026External Links:[Link](https://openai.com/index/introducing-gpt-5-4/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.23.23.3),[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.24.24.3)\.
- OpenAI \(2026b\)Introducing GPT\-5\.5\.Note:Released April 23, 2026External Links:[Link](https://openai.com/index/introducing-gpt-5-5/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.22.22.3)\.
- OpenAI \(2026c\)Introducing GPT\-5\.6\.Note:Generally available July 9, 2026External Links:[Link](https://openai.com/index/gpt-5-6/)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p4.1),[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.20.20.3),[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.21.21.3)\.
- Pangram Labs \(2025\)Pangram Predicts 21% of ICLR Reviews are AI\-Generated\.External Links:[Link](https://www.pangram.com/blog/pangram-predicts-21-of-iclr-reviews-are-ai-generated)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p1.1)\.
- Pangram Labs \(2026\)AI Content Is Everywhere on Social Media, Especially LinkedIn\.External Links:[Link](https://www.pangram.com/blog/ai-in-your-feed)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p1.1)\.
- G\. Penedo, H\. Kydlíček, V\. Sabolčec, B\. Messmer, N\. Foroutan, A\. H\. Kargaran, C\. Raffel, M\. Jaggi, L\. V\. Werra, and T\. Wolf \(2025\)FineWeb2: One Pipeline to Scale Them All — Adapting Pre\-Training Data Processing to Every Language\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=jnRBe6zatP)Cited by:[§5\.6](https://arxiv.org/html/2607.27183#S5.SS6.p1.1)\.
- M\. Perkins, J\. Roe, B\. H\. Vu, D\. Postma, D\. Hickerson, J\. McGaughran, and H\. Q\. Khuat \(2024\)Simple Techniques to Bypass GenAI Text Detectors: Implications for Inclusive Education\.International Journal of Educational Technology in Higher Education21\(1\),pp\. 53\.External Links:[Document](https://dx.doi.org/10.1186/s41239-024-00487-w),[Link](https://doi.org/10.1186/s41239-024-00487-w)Cited by:[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px5.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.6.6.2)\.
- G\. Quaremba, E\. Black, D\. Vrandečić, and E\. Simperl \(2026\)TSM\-Bench: Detecting LLM\-Generated Text in Real\-World Wikipedia Editing Practices\.External Links:2605\.31113,[Link](https://arxiv.org/abs/2605.31113)Cited by:[§2\.3](https://arxiv.org/html/2607.27183#S2.SS3.p1.1)\.
- Qwen \(2026\)Model Releases\.Note:Qwen 3\.7 Max released May 21, 2026External Links:[Link](https://docs.qwencloud.com/changelog/models)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.27.27.3)\.
- M\. Raj, J\. M\. Berg, and R\. Seamans \(2026\)The artificial intelligence disclosure penalty: Humans persistently devalue AI\-generated creative writing\.Journal of Experimental Psychology: General155\(4\),pp\. 896–915\.Note:Epub January 8, 2026\. PMID: 41505277External Links:[Document](https://dx.doi.org/10.1037/xge0001889),[Link](https://doi.org/10.1037/xge0001889)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p3.1)\.
- J\. Russell, M\. Karpinska, D\. Akinode, J\. Zhou, K\. Thai, B\. Emi, M\. Spero, and M\. Iyyer \(2026a\)AI use in American newspapers is widespread, uneven, and rarely disclosed\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 14554–14580\.External Links:[Link](https://aclanthology.org/2026.acl-long.663/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.663),ISBN 979\-8\-89176\-390\-6Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p1.1)\.
- J\. Russell, M\. Karpinska, and M\. Iyyer \(2025\)People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI\-generated text\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 5342–5373\.External Links:[Link](https://aclanthology.org/2025.acl-long.267/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.267),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p2.1)\.
- J\. Russell, R\. Rajendhran, C\. M\. Pham, M\. Iyyer, and J\. Wieting \(2026b\)StoryScope: Investigating idiosyncrasies in AI fiction\.External Links:2604\.03136,[Link](https://arxiv.org/abs/2604.03136)Cited by:[§3\.2](https://arxiv.org/html/2607.27183#S3.SS2.p3.1)\.
- R\. Saha, G\. Juneja, D\. Chaudhuri, N\. Sajeevan, N\. B\. Shah, and D\. Pruthi \(2026\)Policies Permitting LLM Use for Polishing Peer Reviews Are Currently Not Enforceable\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=znFIR7WFGt)Cited by:[§2\.3](https://arxiv.org/html/2607.27183#S2.SS3.p1.1),[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px10.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.16.16.3)\.
- S\. Saha and S\. Feizi \(2025\)Almost AI, Almost Human: The Challenge of Detecting AI\-Polished Writing\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 25414–25431\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.1303),[Link](https://aclanthology.org/2025.findings-acl.1303/)Cited by:[§2\.3](https://arxiv.org/html/2607.27183#S2.SS3.p1.1)\.
- C\. Shaib, T\. Chakrabarty, D\. Garcia\-Olano, and B\. C\. Wallace \(2026\)Measuring AI "Slop" in Text\.External Links:2509\.19163,[Link](https://arxiv.org/abs/2509.19163)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p1.1)\.
- Tencent \(2026\)Tencent Hunyuan Officially Releases Hy3, Advancing Agent Capabilities and Deeper Product Integration\.Note:Reports the Hy3 Preview launch on April 23, 2026External Links:[Link](https://www.tencent.com/tencent-hunyuan-officially-releases-hy3-advancing-agent-capabilities-and-deeper-product-integration/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.28.28.3)\.
- K\. Thai, B\. Emi, E\. Masrour, and M\. Iyyer \(2026\)EditLens: Quantifying the Extent of AI Editing in Text\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=gOkitaPCfZ)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p4.1),[§2\.3](https://arxiv.org/html/2607.27183#S2.SS3.p1.1),[§3\.2](https://arxiv.org/html/2607.27183#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2607.27183#S4.SS1.SSS0.Px3.p1.1)\.
- Thinking Machines Lab \(2026\)Inkling: Our Open\-Weights Model\.Note:Released July 15, 2026External Links:[Link](https://thinkingmachines.ai/news/introducing-inkling/)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.29.29.3)\.
- M\. Van Vlasselaer, F\. Van Droogenbroeck, and B\. Spruyt \(2026\)Who Wrote This? Evaluating the Reliability of AI Detection Tools in Higher Education\.International Journal for Educational Integrity22,pp\. 16\.External Links:[Document](https://dx.doi.org/10.1007/s40979-026-00226-w),[Link](https://doi.org/10.1007/s40979-026-00226-w)Cited by:[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px3.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.16.19.3.1)\.
- J\. Wu, R\. Zhan, D\. F\. Wong, S\. Yang, X\. Yang, Y\. Yuan, and L\. S\. Chao \(2024\)DetectRL: Benchmarking LLM\-Generated Text Detection in Real\-World Scenarios\.InThe Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=ZGMkOikEyv)Cited by:[§5\.8](https://arxiv.org/html/2607.27183#S5.SS8.SSS0.Px7.p1.1),[Table 10](https://arxiv.org/html/2607.27183#S5.T10.10.10.3)\.
- xAI \(2026\)Inference API: Models\.Note:The Grok 4\.3 endpoint reports creation time 1776556800, corresponding to April 19, 2026External Links:[Link](https://docs.x.ai/developers/rest-api-reference/inference/models)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.30.30.3)\.
- H\. Yu, D\. Kim, and Y\. Kim \(2026\)Retrieval Collapses When AI Pollutes the Web\.InProceedings of the ACM Web Conference 2026,WWW ’26,New York, NY, USA,pp\. 8745–8748\.External Links:ISBN 9798400723070,[Link](https://doi.org/10.1145/3774904.3792955),[Document](https://dx.doi.org/10.1145/3774904.3792955)Cited by:[§1](https://arxiv.org/html/2607.27183#S1.p3.1)\.
- Z\.ai \(2026\)GLM\-5\.2: Built for Long\-Horizon Tasks\.Note:Released June 16, 2026External Links:[Link](https://z.ai/blog/glm-5.2)Cited by:[Table 3](https://arxiv.org/html/2607.27183#S5.T3.1.1.31.31.3)\.
- Q\. Zhang, C\. Gao, D\. Chen, Y\. Huang, Y\. Huang, Z\. Sun, S\. Zhang, W\. Li, Z\. Fu, Y\. Wan, and L\. Sun \(2024\)LLM\-as\-a\-Coauthor: Can Mixed Human\-Written and Machine\-Generated Text Be Detected?\.InFindings of the Association for Computational Linguistics: NAACL 2024,K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 409–436\.External Links:[Link](https://aclanthology.org/2024.findings-naacl.29/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-naacl.29)Cited by:[§2\.3](https://arxiv.org/html/2607.27183#S2.SS3.p1.1)\.
- W\. Zhao, X\. Ren, J\. Hessel, C\. Cardie, Y\. Choi, and Y\. Deng \(2024\)WildChat: 1M ChatGPT Interaction Logs in the Wild\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Bl8u7ZRlbM)Cited by:[§5\.5\.1](https://arxiv.org/html/2607.27183#S5.SS5.SSS1.Px2.p3.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-bench and Chatbot Arena\.NIPS ’23,Red Hook, NY, USA\.Cited by:[§5\.2](https://arxiv.org/html/2607.27183#S5.SS2.SSS0.Px1.p1.1)\.

## Appendix ACitation reference

CitationPlease use the following BibTeX entry when citing this work:@techreport\{pangram2026pangram4,title=\{Pangram 4 Technical Report\},author=\{Ben Glickenhaus and Katherine Thai andJenna Russell and Elyas Masrour and Yue Han andMax Spero and Bradley Emi\},institution=\{Pangram Labs\},year=\{2026\}\}

## Appendix BMixed Authorship Text Dataset

In tables[16](https://arxiv.org/html/2607.27183#A2.T16)and[17](https://arxiv.org/html/2607.27183#A2.T17), we provide the prompts and instructions used to create our AI Polish and AI Editing Prompts evaluation datasets\.

Table 16:Editing instructions and prompts used to create our AI Polish evaluation set\.Table 17:Editing instructions and prompts used to create our AI Editing Prompts evaluation set\.
## Appendix CDetailed Benchmark Results

This section expands upon the results in[Table 12](https://arxiv.org/html/2607.27183#S5.T12)\.

### C\.1Binary

Table 18:Performance on the UChicago DetectionAI benchmark\. TPR is measured at a 1% false positive rate\.Note:Originality is excluded from both under\-50\-word splits because it does not support texts of that length\.NNis the number of observations for which each detector returned a usable score\. GPTZero and Originality sometimes rejected the requests\.

Table 19:Overall false positive rates on the UChicago DetectionAI human controls at each detector’s default 50% decision threshold\.Note:A document is classified as AI\-generated when its detector score is at least 50%\.

Table 20:Score distribution on the public fully AI\-generated subset of the VUB benchmark\.Table 21:Performance on the GEDE public benchmark\. TPR is measured at a maximum empirical false positive rate of 1%\.Table 22:Performance on the Perkins benchmark using the paper’s three\-method mean accuracy\.Table 23:Epoch AI Style Imitation benchmark\.*Strict verdicts:*For document\-level results, only a clean “AI” verdict counts as a detection; “Mixed” and “AI\-Assisted” verdicts count as non\-detections \(and as false positives on human text\)\. Span\-level results count any non\-AI span as a false negative in proportion to its length\.

Table 24:Performance on the DetectRL benchmark\. All values are reported as percentages\.Note:Baseline results are taken from the original DetectRL leaderboard\.Pangram 4, Pangram 3\.3\.2, and Open Pangram are evaluated in azero\-shotsetting without benchmark\-specific training and remain competitive with or outperform the supervised methods\. The average F1 is computed over the test datasets\.

Table 25:TPR@1%FPR \(×100\\times 100\) on MELD\-eval by generator\.Note:Results for all detectors other thanPangram 4, Pangram 3\.3\.2, and Open Pangram are taken from Table 4 of the MELD paper\.Pangram 4uses the derived AI likelihoodfractionAI\+0\.5​\(fractionAI​\-​assisted\)\.\\mathrm\{fraction\}\_\{\\mathrm\{AI\}\}\+0\.5\\left\(\\mathrm\{fraction\}\_\{\\mathrm\{AI\\text\{\-\}assisted\}\}\\right\)\.Confidence intervals for both Pangram models are 95% percentile\-bootstrap intervals based on 5,000 resamples\. The best and second\-best results in each column are shown in bold and underlined, respectively\.

Table 26:Pangram 4and Pangram 3\.3\.2 performance on MELD\-eval by attack type\. AUROC and TPR values are reported as percentages\.Note:Each attack is evaluated against the same 7,862 clean human controls\. Thresholds are calibrated independently at a target FPR of 1%\. Attacks are applied only to AI texts\.

Table 27:Performance on the Sem\-Detect peer\-review benchmark\. AUC and TPR values are reported as percentages\.Note:Baseline results are taken from Table 1 of the original paper\.Pangram 4uses the derived AI likelihood, calculated asfractionAI\+0\.5​\(fractionAI​\-​assisted\)\\mathrm\{fraction\}\_\{\\mathrm\{AI\}\}\+0\.5\\left\(\\mathrm\{fraction\}\_\{\\mathrm\{AI\\text\{\-\}assisted\}\}\\right\)\. Uncertainty is estimated using 1,000 bootstrap resamples\. All uncertainty terms are reported in percentage points\. Best and second\-best results are shown in bold and underlined, respectively\. Detectors marked with†are domain\-specific models trained or tuned on peer\-review data\.

### C\.2Mixed

Table 28:Performance on the Saha peer\-review benchmark\. All values are percentages\.Note:Baseline results are taken from Table 2 of the original paper\. For Pangram 3\.3\.2 andPangram 4, AI\-BP, AI\-EP, and AI\-HI are successfully detected when the prediction is either AI or Mixed\. For H\-AI, only an AI prediction is counted as a false positive; Mixed and Human predictions are accepted\. For fully human reviews \(H\), both AI and Mixed predictions are counted as false positives\. Pangram results are evaluated on the test partition of each subset\.

Table 29:Pangram 4and Pangram 3\.3\.2 predictions across progressive human–AI editing stages in the OpAI\-Bench test set\.Note:Each version contains 4,754 documents and represents a successive stage in a cumulative human–AI editing trajectory\. The actual AI character fraction is provided by the character\-level ground\-truth annotations in OpAI\-Bench\. ForPangram 4, the predicted AI\+Assisted fraction is calculated asfractionAI\+fractionAI​\-​assisted\\mathrm\{fraction\}\_\{\\mathrm\{AI\}\}\+\\mathrm\{fraction\}\_\{\\mathrm\{AI\\text\{\-\}assisted\}\}\. The mean and median are calculated independently across the 4,754 documents at each stage\.