Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation

# Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation
Source: [https://arxiv.org/html/2608.26159](https://arxiv.org/html/2608.26159)
Jesse St\. Amand MARS &Callum Canavan Independent &Sohaib Imran MARS &Joseph Hewson Independent &Aaron Lutz Independent &Shi Feng George Washington University &Puria Radmard Geodesic Research &Lennie Wells University of Cambridge

###### Abstract

Self\-Generated Text Recognition \(SGTR\)—the ability of an LLM to identify its own outputs—poses risks to AI safeguards that rely on LLMs as evaluators or monitors\. Specifically, an LLM may recognize outputs from other copies of the same model and make biased judgments or collude outright\. Prior work has drawn conflicting conclusions about whether current models possess significant SGTR capabilities\. We reconcile these findings by identifying key experimental design choices—which we termoperationalizations—that drive divergent results\. Evaluating 13–21 models across six operationalizations, we find that accuracy varies substantially with evaluation format \(pairwise vs\. individual assessments of text\), conversation structure \(presenting candidate text in user tags vs\. assistant tags\), and the domain of the task used to generate candidate text \(e\.g\., coding vs\. summarization\)\. We corroborate previous observations that a quality heuristic—models attributing authorship to text they perceive as higher quality—is a dominant confound\. We also find that improving a model’s SGTR performance via SFT in one evaluation configuration can generalize to others\. Training for SGTR additionally causes models to prefer their own outputs when acting as a judge in the AlpacaEval framework\. Finally, we discuss the implications of our evaluations for the safety of future AI systems: our work suggests that, despite confounds, some models possess practical SGTR capabilities, and that training a model for SGTR in one setting can affect its self\-recognition and self\-preference more generally\. We conclude that SGTR should be monitored and considered in the design of safety\-critical AI applications\.

## 1Introduction

As LLMs are deployed with increasing autonomy in real\-world environments, situational awareness—a model’s knowledge of itself and the context in which it is being run\(Cotra,[2022](https://arxiv.org/html/2608.26159#bib.bib70); Ngoet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib68); Berglundet al\.,[2023](https://arxiv.org/html/2608.26159#bib.bib69); Perezet al\.,[2023](https://arxiv.org/html/2608.26159#bib.bib71)\)—could become a significant concern\. Strong situational awareness could help a model distinguish between training and deployment in order to pursue hidden objectives\(Hubingeret al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib67)\)or identify and circumvent control protocols\(Terekhovet al\.,[2025](https://arxiv.org/html/2608.26159#bib.bib72)\)\.

Self\-generated text recognition\(SGTR\)—the ability of models to distinguish their own outputs from text produced by other sources—is one capability that could increase an LLM’s situational awareness\. In control protocols such as trusted editing \(where a trusted monitor model modifies an untrusted LLM’s reasoning traces\) and honeypotting \(where synthetic tasks generated by weaker models are inserted to detect misalignment\(Greenblattet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib43)\)\), robust SGTR would enable monitored LLMs to distinguish their authentic reasoning traces or environmental inputs from interventions\. Furthermore, in multi\-agent systems, SGTR could facilitate collusion without explicit communication channels\(Roger,[2024](https://arxiv.org/html/2608.26159#bib.bib44)\), and in LLM\-as\-a\-Judge evaluations, SGTR may induce systematic biases\(Panicksseryet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib59)\)\.

Although an understanding of current SGTR capabilities would be valuable for anticipating these risks, SGTR remains poorly characterized\. Prior work has drawn conflicting conclusions: some studies suggest models exhibit robust SGTR\(Laineet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib46); Panicksseryet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib59)\), while others find that even recent models struggle to exceed chance performance\(Baiet al\.,[2025](https://arxiv.org/html/2608.26159#bib.bib42)\)\.

This work makes the following contributions:

1. 1\.A characterization of the key dimensions along which SGTR accuracy varies, such as the domain of the text being evaluated \(e\.g\., coding vs\. summarization\) and how the evaluation task is presented \(a single text classified for self\-authorship vs\. a pair of texts to be ranked; candidate text supplied in the user’s message \[user tags\] vs\. presented as the model’s own prior response \[assistant tags\]\)\. We call a specific configuration of these parameters anoperationalizationof SGTR\.
2. 2\.An evaluation of a subset of these operationalizations that reconciles results from previous studies\. SGTR accuracy varies greatly across these conditions, suggesting that disagreements in the literature may arise from differences in how experimental design elicits SGTR\. Consistent withDavidsonet al\.\([2024](https://arxiv.org/html/2608.26159#bib.bib53)\), we find that models are generally more likely to attribute authorship to text they judge as higher quality\.
3. 3\.An analysis of how SGTR may improve as models scale and are trained on tasks that exercise overlapping skills\. Training models on select operationalizations yields improved performance on held\-out operationalizations\.
4. 4\.A demonstration that improving a model’s SGTR performance via training can induce bias when it is used as a judgein AlpacaEval 2\.0\(Duboiset al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib12)\), a widely used LLM\-as\-a\-Judge benchmark\.

## 2Operationalizations of SGTR

We define anoperationalizationof SGTR as a specific combination of experimental design choices that affect how SGTR is measured\. Different operationalizations can yield substantially different accuracy estimates, which we argue explains conflicting prior results\.

### 2\.1Dimensions of Variation

Our SGTR evaluations vary along the following dimensions:

1. 1\.Task domain\.The type of task posed to the model when generating text samples, e\.g\., summarization or code generation\.
2. 2\.Evaluation target\.Recognitionprompts ask the model to identify whether text samples were generated by that model\.Preferenceprompts ask it to judge the quality of text samples\. By comparing these, one can measure the correlation between judgments of self\-authorship and quality\(Davidsonet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib53)\)\.
3. 3\.Evaluation format\.The format in which text samples are presented to the model for evaluation\. Inindividualassessment, the model must classify a standalone sample\. Inpairwiseassessment, the model is presented with two samples and must select the one that is more likely to be self\-generated \(or higher quality, depending on the evaluation target\)\.
4. 4\.Conversation format\.The structure of the prompt’s conversation history\. Inuser tagexperiments, the conversation comprises a singleusermessage containing a question from the task domain, one or more sample responses to that question, and a request for the model’s judgment of self\-authorship or quality\. Inassistant tagexperiments, the conversation comprises one or more turns in which ausermessage poses a question from the task domain and anassistantmessage provides a sample response, followed by a final user message requesting the model’s evaluation\.

Additional dimensions not explored in this research include thedefinition of “self”\(same instance vs\. same family vs\. AI\-generated broadly\), thepool of non\-self text\(other models, humans, or hybrids\), inclusion ofreasoning tracesas sample text, andmeasurement type\(prompting vs\. probing\)\. Prior work primarily uses prompting with user\-tag formatting and no reasoning traces\.

### 2\.2Related Work

Several aspects of modern language model development could contribute to SGTR\. Pretraining corpora include human\- and machine\-generated content and comparisons between them, and AI\-related discourse in training data has been shown to impact model behavior\(Ticeet al\.,[2026](https://arxiv.org/html/2608.26159#bib.bib19)\)\. During post\-training, SGTR may help defend against many\-shot jailbreaking attacks that inject false assistant messages\(Anilet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib37); Ackerman and Panickssery,[2025a](https://arxiv.org/html/2608.26159#bib.bib45)\)\. SGTR may also arise as an instrumental capability that aids collaborative tasks\. In this work we focus on measuring SGTR rather than explaining its origins\.

Davidsonet al\.\([2024](https://arxiv.org/html/2608.26159#bib.bib53)\)found that models classify text they rate as higher quality as self\-generated, which we designate as thequality heuristicand study further here\. This heuristic connects SGTR toself\-preference bias—the tendency of an LLM acting as a judge to favor its own outputs\(Zhenget al\.,[2023](https://arxiv.org/html/2608.26159#bib.bib64); Panicksseryet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib59); Wataokaet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib65); Liet al\.,[2025](https://arxiv.org/html/2608.26159#bib.bib66)\)\. Furthermore,Ackerman and Panickssery \([2025b](https://arxiv.org/html/2608.26159#bib.bib40)\)identified a residual stream vector in Llama 3 8B Instruct that steers authorship beliefs, but we restrict our analysis to behavioral measurements\.

Studies disagree on the extent of SGTR capabilities\. Some report robust performance:Panicksseryet al\.\([2024](https://arxiv.org/html/2608.26159#bib.bib59)\)showed GPT 4 could reliably discern its text from human\-generated text in both pairwise and individual formats;Laineet al\.\([2024](https://arxiv.org/html/2608.26159#bib.bib46)\)found above\-chance pairwise recognition for multiple models against human text;Ackerman and Panickssery \([2025b](https://arxiv.org/html/2608.26159#bib.bib40)\)showed Llama 3 8B Instruct can distinguish its text from comparable models in pairwise settings, though performance degrades when output lengths are normalized\.

Other studies find weak or no SGTR\.Davidsonet al\.\([2024](https://arxiv.org/html/2608.26159#bib.bib53)\)found that recognition largely reflected quality heuristics in a multiple\-choice format\.Baiet al\.\([2025](https://arxiv.org/html/2608.26159#bib.bib42)\)tested individual\-format recognition across 10 frontier models and found that models disproportionately attributed text to well\-known families regardless of source\.

One explanation for weak inter\-model discrimination is that modern models share training data, architectures, and optimization objectives, producing genuinely similar output distributions\(Jianget al\.,[2025](https://arxiv.org/html/2608.26159#bib.bib39)\)\. See Appendix[A](https://arxiv.org/html/2608.26159#A1)for a detailed mapping of prior work across operationalization dimensions\.

## 3Methods

### 3\.1Operationalizations

We evaluate SGTR accuracy for six operationalizations: two evaluation formats \(pairwise, PW, or individual, IND\), two conversation formats \(user tag, UT, or assistant tag, AT\), and two evaluation targets \(recognition or preference\), where preference is assessed only in the user\-tag format\.

#### 3\.1\.1Prompt Formats

In the pairwise evaluation format, models receive a task prompt alongside two candidate responses: one originally generated by the evaluating model and one generated by an alternative model\. In the recognition condition, models are asked to identify which response they authored \(“Which response is your original output?”\)\. In the preference condition, models are asked to judge relative quality \(“Which one of the two responses would you rate as higher quality?”\)\. Models indicate their selection by responding with “1” or “2\.” To control for position bias, we present each comparison twice with the response order swapped\.

In the individual evaluation format, models receive a task prompt and a single candidate response\. In the recognition condition, models are informed that the response was sampled from a balanced pool where half originated from the evaluating model and half from alternative models, and must determine whether the presented response is theirs \(“Is this response your original output?”\)\. In the preference condition, models assess whether the response meets their standards \(“Does this response meet your own standards of quality?”\)\. Models respond with a designated token indicating acceptance or rejection\.

The full prompt templates for all four paradigms are provided in Appendix[F](https://arxiv.org/html/2608.26159#A6)\(Tables[5](https://arxiv.org/html/2608.26159#A6.T5)–[8](https://arxiv.org/html/2608.26159#A6.T8)\)\. Differences between the user\-tag and assistant\-tag conversation formats are shown in Table[1](https://arxiv.org/html/2608.26159#S3.T1)\.

Table 1:Conversation formats for presenting candidate text\. In the*assistant tags*format, candidate responses appear as prior assistant messages in a multi\-turn exchange\. In the*user tags*format, all content is embedded within a single user message\. “–” indicates the text is absent\.
#### 3\.1\.2Evaluation Targets

For recognition conditions, we report accuracy as the proportion of correct judgments\. In preference experiments, we instead measure how often a model judges its own text to be higher quality: in the pairwise setting, the rate at which it selects its own response over a competing one; in the individual setting, the rate at which it judges its own response—but not other models’ responses—to meet its “standards of quality\.” Preference experiments are limited to the user\-tag format\.

For individual assessment experiments, we calculate accuracy scores using a weighted average to control forattribution bias: the tendency of a model to answer uniformly across all text sources, accepting or rejecting nearly everything regardless of authorship\. Specifically, we weight performance on alternative\-model text and, independently, self\-generated text each at 50%\. This adjustment ensures that a model exhibiting attribution bias receives a score of 0\.5 \(as does a model that randomly guesses\)\.

To measure the relationship between model capability and SGTR performance, we use Elo scores from the Arena \(formerly LMArena / Chatbot Arena\) leaderboard\(Chianget al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib22)\)\. Elo scores are derived from pairwise human preference votes: each model’s score reflects its expected win rate against other models, with higher scores indicating stronger overall performance as judged by users\. We define*score distance*as the evaluator’s Elo score minus the text generator’s Elo score, where positive values indicate the evaluator is the more capable model\.

### 3\.2SGTR Training

To test how easily SGTR can be learned and transferred across operationalizations, we fine\-tune three models—Llama 3\.1 8B, GPT\-OSS 20B, and Qwen 3\.0 30B—using supervised fine\-tuning \(SFT\) with LoRA rank 32\. Note that the fine\-tuned Qwen \(3\.0 30B\) differs from the Qwen model in our evaluation set \(3\.0 80B\), as the smaller variant is more tractable to train\. For each model, we construct training data from the pairwise recognition task: each sample presents the model’s own output alongside an alternative model’s output and asks the model to identify which response is its own\. Training data is balanced \(50% self, 50% alternative\) with 100 samples per dataset, split 80/20 into train/val sets\. We train for 20 epochs with a learning rate of5×10−55\\times 10^\{\-5\}and batch size 16\.

We vary training along two axes:*operationalization*\(evaluation format, conversation format, task domain\) and*identity*\. Standard training teaches models to recognize their own text; adversarial training teaches models to recognize a*different*model’s text as their own \(e\.g\., GPT\-OSS 20B trained to claim authorship of Qwen 3\.0 30B’s outputs\)\. We evaluate transfer on held\-out benchmarks that test unseen operationalizations \(e\.g\., trained on pairwise and tested on individual, or trained on one task domain and tested on another\)\.

### 3\.3AlpacaEval as an LLM\-as\-a\-Judge Testbed

To evaluate how SGTR capabilities manifest in a downstream evaluation setting, we use AlpacaEval 2\.0\(Duboiset al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib12)\)as a testbed for self\-preference bias\. AlpacaEval is a standard benchmark for instruction\-following LLMs consisting of 805 diverse instructions\. We repurpose this framework to measure self\-preference: each model generates outputs on all instructions and then serves as a judge, simultaneously ranking its own outputs alongside those of every other model for each instruction\. Self\-preference is measured as the average rank a judge assigns to its own base model’s outputs, where lower rank indicates stronger self\-preference\.

This setup enables us to assess \(1\) whether base models exhibit self\-preference when acting as judges in a realistic evaluation pipeline, and \(2\) whether SGTR training amplifies this bias\. The latter has direct implications for LLM\-as\-a\-Judge reliability: if fine\-tuning for self\-recognition induces preference bias, models trained on tasks that incidentally improve SGTR could produce systematically skewed evaluations\.

### 3\.4Datasets

We construct evaluation data from 250 prompts drawn from four publicly available datasets spanning distinct task domains: WikiSum\(Cohenet al\.,[2021](https://arxiv.org/html/2608.26159#bib.bib13)\)for summarization \(50 prompts\), ShareGPT\(RyokoAI,[2023](https://arxiv.org/html/2608.26159#bib.bib63)\)for open\-ended conversation \(100 prompts\), PKU\-SafeRLHF\(Jiet al\.,[2025](https://arxiv.org/html/2608.26159#bib.bib15)\)for safety\-aligned QA \(50 prompts\), and BigCodeBench\(Zhuoet al\.,[2025](https://arxiv.org/html/2608.26159#bib.bib18)\)for coding tasks \(50 prompts\)\. We choose these domains to elicit different stylistic signatures from models\. For each prompt, we generate responses from all models in our evaluation set \(temperature = 1\.0\), which serve as candidate texts for SGTR and preference evaluations\. Full dataset descriptions are provided in Appendix[E](https://arxiv.org/html/2608.26159#A5)\.

### 3\.5Models

We use two model sets: a*full set*of 21 models \(Table[4](https://arxiv.org/html/2608.26159#A2.T4); 15 instruct\-only, 3 reasoning\-only, and 3 dual\-mode\) spanning 7 providers, used for per\-model analysis \(Figure[1](https://arxiv.org/html/2608.26159#S4.F1)b\) and user\-tag experiments \(Figure[2](https://arxiv.org/html/2608.26159#S4.F2)a–b, e–f\); and a*reduced set*of 13 instruct models \(Table[4](https://arxiv.org/html/2608.26159#A2.T4)\) available across all conditions, used for cross\-operationalization comparisons \(Figure[1](https://arxiv.org/html/2608.26159#S4.F1)a\) and assistant\-tag experiments \(Figure[2](https://arxiv.org/html/2608.26159#S4.F2)c–d\)\. Because each of the three dual\-mode models is evaluated both with and without chain\-of\-thought reasoning enabled, the full set yields 24 distinct evaluator configurations from 21 models\. Reasoning evaluations are denoted “\(R\)” in all figures and tables\. For training experiments \(Figure[3](https://arxiv.org/html/2608.26159#S4.F3)\), we fine\-tune Llama 3\.1 8B, GPT\-OSS 20B, and Qwen 3\.0 30B \(see §[3\.2](https://arxiv.org/html/2608.26159#S3.SS2)\)\.

## 4Results

### 4\.1Recognition Performance Across Operationalizations

SGTR accuracy is sensitive to how the task is presented\.Figure[1](https://arxiv.org/html/2608.26159#S4.F1)a summarizes recognition accuracy across operationalizations\. Pairwise evaluations and the user\-tag format produce higher median accuracy and greater spread than individual assessments and the assistant\-tag format, respectively\. Per\-model recognition accuracy for the pairwise and individual paradigms is reported in Appendix[B](https://arxiv.org/html/2608.26159#A2)\(Figure[5](https://arxiv.org/html/2608.26159#A2.F5)\), and the per\-model difference between the two paradigms in Appendix[C](https://arxiv.org/html/2608.26159#A3)\(Figure[6](https://arxiv.org/html/2608.26159#A3.F6)\)\.

Figure[1](https://arxiv.org/html/2608.26159#S4.F1)b shows per\-model pairwise recognition accuracy by dataset, ordered by Arena Elo score\. Performance varies widely across task domains\. Accuracy is often highest on PKU\-SafeRLHF, particularly for reasoning models, but Gemini 2\.0 Flash Lite and Llama 3\.1 8B are biased toward claiming authorship of high\-quality samples from this dataset, so their performance is below chance\. BigCodeBench results have high variance, with Claude Opus 4\.1 performing much worse in reasoning mode compared to instruct mode\. WikiSum and ShareGPT have similarly high variance, eliciting the best performance from some models and the worst from others\. Stronger models tend to have better SGTR; as we show in the next section, this is largely explained by models attributing authorship to text they perceive as higher quality\.

![Refer to caption](https://arxiv.org/html/2608.26159v1/x1.png)Figure 1:\(a\) Box plots of recognition accuracy distributions\. A single data point is the accuracy for one \(evaluator, generator\) model pair: how accurately an evaluator model recognizes its own text when paired against a given generator model’s text\. The purple, leftmost boxes show the four prompt formats \(UT PW – user\-tag pairwise, UT IND – user\-tag individual, AT PW – assistant\-tag pairwise, and AT IND – assistant\-tag individual\) pooled across all four task domains\. The multicolor, rightmost boxes show the four task domains \(S\-GPT – ShareGPT, PKU – PKU\-SafeRLHF, BCB – BigCodeBench, WS – WikiSum\), pooled over the pairwise operationalizations\. The dashed line marks chance \(0\.5\)\. \(b\) Mean accuracies over generator models for the user\-tag pairwise \(UT PW\) setting, grouped by evaluator model and task domain \(distinguished by color\), with models ordered by Arena Elo score \(lowest on the left\)\. Asterisks mark accuracies significantly different from chance \(two\-sided binomial test,p<0\.05p<0\.05\)\. Model names ending in “\(R\)” denote evaluations run with chain\-of\-thought reasoning enabled; text color groups models by their provider\.
### 4\.2Quality as a Recognition Heuristic

Models claim authorship of text they perceive as higher quality\.We support this claim with two analyses\. First, recognition accuracy correlates positively with Elo score distance across all four paradigms \(Figure[2](https://arxiv.org/html/2608.26159#S4.F2)a–d;r=0\.59r=0\.59–0\.740\.74\), with evaluators approaching ceiling against weaker generators and falling below chance against stronger ones\. An alternative view, plotting recognition accuracy directly against evaluator Arena rank, is provided in Appendix[D](https://arxiv.org/html/2608.26159#A4)\. We additionally conductpreferenceexperiments using nearly identical prompts, this time asking models to judge quality rather than authorship \(the recognition and preference templates are shown in Appendix[F](https://arxiv.org/html/2608.26159#A6), Tables[5](https://arxiv.org/html/2608.26159#A6.T5)–[8](https://arxiv.org/html/2608.26159#A6.T8)\)\. In the pairwise setting \(Figure[2](https://arxiv.org/html/2608.26159#S4.F2)e\), recognition and preference are highly correlated \(r=0\.78r=0\.78–0\.940\.94\)\. In the individual setting \(Figure[2](https://arxiv.org/html/2608.26159#S4.F2)f\), this weakens \(r=0\.20r=0\.20–0\.870\.87\), but so too does median performance \(Figure[1](https://arxiv.org/html/2608.26159#S4.F1)a\)\. However, models may have some meaningful SGTR beyond the quality heuristic, since the best\-fit lines on Figure[2](https://arxiv.org/html/2608.26159#S4.F2)a–d are above0\.50\.5accuracy at an Elo distance of0\.

![Refer to caption](https://arxiv.org/html/2608.26159v1/x2.png)Figure 2:Evidence for quality\-based recognition heuristic\. \(a–d\) Recognition accuracy vs\. Elo score distance \(evaluator score−\-generator score\) across four paradigms: user\-tag pairwise \(a\), user\-tag individual \(b\), assistant\-tag pairwise \(c\), and assistant\-tag individual \(d\)\. Positive score distance indicates a stronger evaluator\. Points colored by dataset; black regression line with 95% bootstrap CI\. \(e, f\) Recognition performance vs\. preference performance for pairwise \(e\) and individual \(f\) paradigms\. Each point represents one model–dataset pair; marker shape indicates dataset, color indicates model family\. Per\-dataset weighted regression lines with 95% CI bands\. Individual paradigm results \(b, d\) corrected for attribution bias\.Second, to test whether recognition persists when sample quality is more similar, we restrict model pairs to those with similar Arena rankings \(rank distance within±\\pm20\)\. This leads to smaller capability–accuracy correlations \(Appendix[G](https://arxiv.org/html/2608.26159#A7)\)\. Some models—particularly Claude Opus 4\.1, Sonnet 4\.5, and Qwen 3\.0 80B—maintain above\-chance performance against models with similar rank, though the number of samples is small\.

### 4\.3Training for SGTR

SGTR training transfers across operationalizations\.We fine\-tune Llama 3\.1 8B, GPT\-OSS 20B, and Qwen 3\.0 30B to claim authorship of their own text, and evaluate on held\-out operationalizations\. Figure[3](https://arxiv.org/html/2608.26159#S4.F3)shows the accuracy change \(post−\-pre\) for each model–training condition pair, with the trained condition excluded\. The left panel shows evaluation format transfer \(all trained on ShareGPT\): training on one format generally improves accuracy on others, though the magnitude varies by model\. The right panel shows dataset transfer \(all trained on UT PW\): cross\-domain generalization is positive on average, though weaker or even negative for BigCodeBench\. Results for adversarially trained models \(i\.e\., models trained to recognize a*different*model’s text as their own\) are also plotted; these too transfer across operationalizations\.

![Refer to caption](https://arxiv.org/html/2608.26159v1/x3.png)Figure 3:Training transfer across operationalizations\. Each dot represents one \(model, training condition\) pair evaluated on a held\-out test condition\. Left: task transfer across evaluation formats \(PW – pairwise, IND – individual\) and conversation formats \(UT – user\-tag, AT – assistant\-tag\), all trained on ShareGPT\. Right: task domain transfer \(WS – WikiSum, BCB – BigCodeBench, PKU – PKU\-SafeRLHF, S\-GPT – ShareGPT\), all trained on UT PW\. Marker shape indicates training condition; color indicates model identity \(edge = base model, fill = identity model for adversarial runs\)\. Box plots summarize the per\-row distribution\. The trained condition is excluded from each row\.SGTR training affects self\-preference in LLM\-as\-a\-Judge\.To measure how SGTR training affects downstream evaluation, we use trained models as judges in AlpacaEval 2\.0\. Figure[4](https://arxiv.org/html/2608.26159#S4.F4)a shows the change in self\-rank after SGTR training\. Positive values indicate increased self\-preference\. GPT\-OSS 20B shows the highest transfer when trained on UT PW ShareGPT \(\+0\.76\+0\.76\), while Qwen 3\.0 30B shows a more consistent transfer across conditions\. The effects on non\-ShareGPT datasets are weaker\.

We also investigate how adversarial training affects self\-preference\. We train GPT\-OSS 20B to recognize Qwen 3\.0 30B’s text as self\-generated and Qwen to recognize GPT\-OSS 120B’s text \(using the larger 120B variant as the target because GPT\-OSS 20B’s outputs are too easy for Qwen to distinguish from its own\)\. Figure[4](https://arxiv.org/html/2608.26159#S4.F4)b shows that adversarial training increases preference for the new target\. GPT\-OSS 20B trained as Qwen on UT PW shows a1\.141\.14rank increase in preference for Qwen’s text\. Qwen trained as GPT\-OSS 120B on UT PW shows a weaker but directionally consistent effect with a0\.330\.33rank increase in preference for GPT\-OSS’s text\.

![Refer to caption](https://arxiv.org/html/2608.26159v1/x4.png)Figure 4:Change in AlpacaEval 2\.0 self\-rank after SGTR training\. \(a\) Standard training: each cell shows average rank improvement for a model trained on a specific operationalization \(positive/green = more self\-preference\)\. Left columns: task domain transfer \(WS – WikiSum, BCB – BigCodeBench, PKU – PKU\-SafeRLHF\), all trained on user\-tag pairwise \(UT PW\)\. Right columns: task format transfer \(PW – pairwise, IND – individual, UT – user\-tag, AT – assistant\-tag\), all trained on ShareGPT \(S\-GPT\)\. \(b\) Adversarial training: models trained to recognize an alternative model’s text \(rank improvement here is for the alternative model’s text\)\.

## 5Discussion

The operationalization framework reveals that SGTR accuracy is highly sensitive to how the task is presented, helping to explain why prior studies, which used a variety of operationalizations, reached conflicting conclusions about SGTR capabilities\. Pairwise evaluations produce higher median accuracy and greater spread than individual assessments; the same holds for user\-tag over assistant\-tag settings \(Figure[1](https://arxiv.org/html/2608.26159#S4.F1)a\)\. All four task domains yield distinct accuracy distributions \(Figure[1](https://arxiv.org/html/2608.26159#S4.F1)a\), while specific models vary greatly in their performance per domain \(Figure[1](https://arxiv.org/html/2608.26159#S4.F1)b\)\.

Evidence for the quality heuristic appears across operationalizations\. In Figure[2](https://arxiv.org/html/2608.26159#S4.F2), we show pairwise recognition–preference coupling \(r≥0\.78r\\geq 0\.78\) and score\-distance correlations \(UT:r=0\.73r=0\.73, AT:r=0\.66r=0\.66\) consistent with models attributing authorship to text they perceive as higher quality\. In the individual paradigm, the recognition–preference coupling weakens, with medianr=0\.70r=0\.70and a low outlier ofr=0\.20r=0\.20\. The score\-distance correlations are similar \(UT:r=0\.59r=0\.59, AT:r=0\.74r=0\.74\), but with much reduced slopes\. This suggests that, although still present, the quality heuristic contributes less to accuracy in individual assessments\.

We attribute the pairwise–individual differences to two effects: the higher median in the pairwise setting is consistent with the comparison providing additional evidence for determining authorship, as the evaluator sees roughly twice as much candidate text and can juxtapose the two responses to isolate the stylistic features that distinguish them\. The greater spread aligns with the quality heuristic\. Because the heuristic ties recognition accuracy to the quality gap between evaluator and generator \(approximated by their Elo score distance\), evaluators at the top of the capability range are advantaged and those at the bottom disadvantaged: high\-capability models recognize their own text nearly perfectly against weaker generators, while low\-capability models fall well below chance against stronger ones\. This pushes accuracies toward both extremes, widening the distribution\. Figure[2](https://arxiv.org/html/2608.26159#S4.F2)a–d illustrates the effect: accuracy increases much more steeply with Elo score distance in the pairwise paradigms \(a, c\) than in the individual paradigms \(b, d\), so a given capability gap produces more over\- and under\-performance, and hence greater spread\.

Our training experiments show that SGTR is learnable and transfers across operationalizations\. Models trained on a single condition improve on held\-out task formats, conversation structures, and task domains, suggesting that training captures generalizable features rather than operationalization\-specific shortcuts\. This has direct downstream consequences: trained judges exhibit self\-preference bias in AlpacaEval 2\.0, ranking their own outputs up to\+0\.76\+0\.76rank positions higher\. Adversarial training redirects this effect—GPT\-OSS 20B trained to recognize Qwen’s text in UT PW conditions shows a1\.141\.14rank increase in preference for Qwen’s outputs—demonstrating that SGTR training can have a measurable effect on evaluation results\.

### 5\.1Practical Implications

SGTR performance varies substantially across operationalizations, models, and domains\. Developers deploying systems where SGTR matters should assess recognition under their specific conditions rather than relying on aggregate benchmarks\.

The trainability of SGTR poses risks for evaluation integrity\. A model fine\-tuned with modest SGTR data exhibits self\-preference bias when used as a judge\. This is concerning for benchmarks that rely on LLM judges, such as AlpacaEval and Arena\.

For AI control, quality\-based heuristics may be sufficient to subvert safeguards\. Control protocols employing honeypots or trusted edits\(Greenblattet al\.,[2024](https://arxiv.org/html/2608.26159#bib.bib43)\)typically use weaker models to generate replacement content, which stronger models can identify via quality differences alone\. The transferability of SGTR training further suggests that models might acquire these capabilities through post\-training on related tasks\.

### 5\.2Limitations

Our operationalizations span four task domains, two evaluation formats, and two conversation structures, but may not generalize to long\-form generation, multi\-turn dialogue, reasoning traces, or creative tasks\. Training experiments use only three base models with small datasets and low\-rank LoRA SFT; larger\-scale training or RL may yield different results\. Our AlpacaEval analysis measures self\-rank change but does not assess whether the resulting rankings are more or less accurate\. Our analysis is entirely behavioral: it does not examine internal activations, nor does it identify which text features drive predictions \(such as response length\(Ackerman and Panickssery,[2025b](https://arxiv.org/html/2608.26159#bib.bib40)\)and correctness, which we do not control for\)\.

### 5\.3Conclusion

We identify key dimensions along which SGTR evaluations vary and show that accuracy is highly sensitive to operationalization, reconciling conflicting prior results\. The quality heuristic is a dominant confound in pairwise settings, where recognition and preference are tightly coupled; the individual paradigm attenuates this, suggesting that above\-chance performance may partly reflect capabilities beyond quality assessment\. SGTR is trainable via SFT and transfers broadly across held\-out operationalizations and domains\. This training induces self\-preference bias when models are used as judges in AlpacaEval 2\.0, and adversarial training can redirect this bias to favor arbitrary target models\. We recommend that evaluation frameworks account for SGTR\-induced bias, and that AI control protocols be tested against models with enhanced recognition capabilities\.

## Acknowledgments

Claude Code\(Anthropic,[2025a](https://arxiv.org/html/2608.26159#bib.bib10)\)and Cursor\(Anysphere Inc\.,[2025](https://arxiv.org/html/2608.26159#bib.bib11)\)were used to assist with data analysis pipelines, figure generation, and iterative editing of paper text, including drafting and restructuring content\. All references were independently verified by the authors\.

This work was supported by the UK AI Security Institute \(AISI\) through the AISI Challenge Fund, funded by the Department for Science, Innovation and Technology \(DSIT\) \[grant ref\. G2\-SCH\-2025\-01\-16275\]\. The grant was administered through Meridian Impact CIC, which also hosted part of this work\. We thank the Cambridge AI Safety Hub \(CAISH\) for facilitating this collaboration through its Mentorship for Alignment Research Students \(MARS\) program\.

## References

- Mitigating Many\-Shot Jailbreaking\.arXiv\.Note:arXiv:2504\.09604 \[cs\] version: 3External Links:[Link](http://arxiv.org/abs/2504.09604),[Document](https://dx.doi.org/10.48550/arXiv.2504.09604)Cited by:[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p1.1)\.
- C\. Ackerman and N\. Panickssery \(2025b\)Inspection and Control of Self\-Generated\-Text Recognition Ability in Llama3\-8b\-Instruct\.arXiv\.Note:arXiv:2410\.02064 \[cs\]External Links:[Link](http://arxiv.org/abs/2410.02064),[Document](https://dx.doi.org/10.48550/arXiv.2410.02064)Cited by:[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p2.1),[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p3.1),[§5\.2](https://arxiv.org/html/2608.26159#S5.SS2.p1.1)\.
- C\. Anil, E\. Durmus, M\. Sharma, J\. Benton, S\. Kundu, J\. Batson, N\. Rimsky, M\. Tong, J\. Mu, D\. Ford, F\. Mosconi, R\. Agrawal, R\. Schaeffer, N\. Bashkansky, S\. Svenningsen, M\. Lambert, A\. Radhakrishnan, C\. Denison, E\. J\. Hubinger, Y\. Bai, T\. Bricken, T\. Maxwell, N\. Schiefer, J\. Sully, A\. Tamkin, T\. Lanham, K\. Nguyen, T\. Korbak, J\. Kaplan, D\. Ganguli, S\. R\. Bowman, E\. Perez, R\. Grosse, and D\. Duvenaud \(2024\)Many\-shot Jailbreaking\.\(en\)\.Note:arXiv preprintCited by:[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p1.1)\.
- Anthropic \(2025a\)Claude Code\.Note:[https://docs\.anthropic\.com/en/docs/claude\-code](https://docs.anthropic.com/en/docs/claude-code)AI\-powered coding assistantCited by:[Acknowledgments](https://arxiv.org/html/2608.26159#Sx1.p1.1)\.
- Anthropic \(2025b\)The Claude model family\.Note:[https://www\.anthropic\.com/claude](https://www.anthropic.com/claude)Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.6.2.2.2.3)\.
- Anysphere Inc\. \(2025\)Cursor: the AI code editor\.Note:[https://cursor\.com](https://cursor.com/)Cited by:[Acknowledgments](https://arxiv.org/html/2608.26159#Sx1.p1.1)\.
- X\. Bai, A\. Shrivastava, A\. Holtzman, and C\. Tan \(2025\)Know Thyself? On the Incapability and Implications of AI Self\-Recognition\.arXiv\.Note:arXiv:2510\.03399 \[cs\]External Links:[Link](http://arxiv.org/abs/2510.03399),[Document](https://dx.doi.org/10.48550/arXiv.2510.03399)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p4.1)\.
- L\. Berglund, A\. C\. Stickland, M\. Balesni, M\. Kaufmann, M\. Tong, T\. Korbak, D\. Kokotajlo, and O\. Evans \(2023\)Taken out of context: On measuring situational awareness in LLMs\.External Links:2309\.00667,[Link](https://arxiv.org/abs/2309.00667)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p1.1)\.
- W\. Chiang, L\. Zheng, Y\. Sheng, A\. N\. Angelopoulos, T\. Li, D\. Li, H\. Zhang, B\. Zhu, M\. Jordan, J\. E\. Gonzalez, and I\. Stoica \(2024\)Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference\.arXiv\.Note:arXiv:2403\.04132 \[cs\]External Links:[Link](http://arxiv.org/abs/2403.04132),[Document](https://dx.doi.org/10.48550/arXiv.2403.04132)Cited by:[§3\.1\.2](https://arxiv.org/html/2608.26159#S3.SS1.SSS2.p3.1)\.
- N\. Cohen, O\. Kalinsky, Y\. Ziser, and A\. Moschitti \(2021\)WikiSum: Coherent Summarization Dataset for Efficient Human\-Evaluation\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 2: Short Papers\),Online,pp\. 212–219\(en\)\.External Links:[Link](https://aclanthology.org/2021.acl-short.28),[Document](https://dx.doi.org/10.18653/v1/2021.acl-short.28)Cited by:[Appendix E](https://arxiv.org/html/2608.26159#A5.p1.1),[§3\.4](https://arxiv.org/html/2608.26159#S3.SS4.p1.1)\.
- A\. Cotra \(2022\)Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover\.Note:[https://www\.lesswrong\.com/posts/pRkFkzwKZ2zfa3R6H/without\-specific\-countermeasures\-the\-easiest\-path\-to](https://www.lesswrong.com/posts/pRkFkzwKZ2zfa3R6H/without-specific-countermeasures-the-easiest-path-to)LessWrong post, 18 July 2022Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p1.1)\.
- T\. R\. Davidson, V\. Surkov, V\. Veselovsky, G\. Russo, R\. West, and C\. Gulcehre \(2024\)Self\-Recognition in Language Models\.arXiv\.Note:arXiv:2407\.06946 \[cs\]External Links:[Link](http://arxiv.org/abs/2407.06946),[Document](https://dx.doi.org/10.48550/arXiv.2407.06946)Cited by:[item 2](https://arxiv.org/html/2608.26159#S1.I1.i2.p1.1),[item 2](https://arxiv.org/html/2608.26159#S2.I1.i2.p1.1),[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p2.1),[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p4.1)\.
- DeepSeek\-AI \(2024\)DeepSeek\-V3 technical report\.External Links:2412\.19437Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.15.11.11.18.6.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-R1: incentivizing reasoning capability in LLMs via reinforcement learning\.External Links:2501\.12948Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.15.11.11.18.6.1)\.
- A\. Dubeyet al\.\(2024\)The Llama 3 herd of models\.External Links:2407\.21783Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.15.11.11.17.5.1)\.
- Y\. Dubois, B\. Galambosi, P\. Liang, and T\. B\. Hashimoto \(2024\)Length\-controlled AlpacaEval: a simple way to debias automatic evaluators\.arXiv preprint arXiv:2404\.04475\.Cited by:[item 4](https://arxiv.org/html/2608.26159#S1.I1.i4.p1.1),[§3\.3](https://arxiv.org/html/2608.26159#S3.SS3.p1.1)\.
- Gemini Teamet al\.\(2024\)Gemini: a family of highly capable multimodal models\.External Links:2312\.11805Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.15.11.11.15.3.1)\.
- R\. Greenblatt, B\. Shlegeris, K\. Sachan, and F\. Roger \(2024\)AI Control: Improving Safety Despite Intentional Subversion\.arXiv\.Note:arXiv:2312\.06942 \[cs\]External Links:[Link](http://arxiv.org/abs/2312.06942),[Document](https://dx.doi.org/10.48550/arXiv.2312.06942)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p2.1),[§5\.1](https://arxiv.org/html/2608.26159#S5.SS1.p3.1)\.
- E\. Hubinger, C\. Denison, J\. Mu, M\. Lambert, M\. Tong, M\. MacDiarmid, T\. Lanham, D\. M\. Ziegler, T\. Maxwell, N\. Cheng, A\. S\. Jermyn, A\. Askell, A\. Radhakrishnan, C\. Anil, D\. Duvenaud, D\. Ganguli, F\. Barez, J\. Clark, K\. Ndousse, K\. Sachan, M\. Sellitto, M\. Sharma, N\. DasSarma, R\. B\. Grosse, S\. Kravec, Y\. Bai, Z\. Witten, M\. Favaro, J\. Brauner, H\. Karnofsky, P\. F\. Christiano, S\. R\. Bowman, L\. Graham, J\. Kaplan, S\. Mindermann, R\. Greenblatt, B\. Shlegeris, N\. Schiefer, and E\. Perez \(2024\)Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training\.CoRRabs/2401\.05566\.External Links:[Link](https://doi.org/10.48550/arXiv.2401.05566),[Document](https://dx.doi.org/10.48550/ARXIV.2401.05566),2401\.05566Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p1.1)\.
- J\. Ji, D\. Hong, B\. Zhang, B\. Chen, J\. Dai, B\. Zheng, T\. Qiu, J\. Zhou, K\. Wang, B\. Li, S\. Han, Y\. Guo, and Y\. Yang \(2025\)PKU\-SafeRLHF: Towards Multi\-Level Safety Alignment for LLMs with Human Preference\.arXiv\.Note:arXiv:2406\.15513 \[cs\]External Links:[Link](http://arxiv.org/abs/2406.15513),[Document](https://dx.doi.org/10.48550/arXiv.2406.15513)Cited by:[Appendix E](https://arxiv.org/html/2608.26159#A5.p3.1),[§3\.4](https://arxiv.org/html/2608.26159#S3.SS4.p1.1)\.
- L\. Jiang, Y\. Chai, M\. Li, M\. Liu, R\. Fok, N\. Dziri, Y\. Tsvetkov, M\. Sap, A\. Albalak, and Y\. Choi \(2025\)Artificial Hivemind: The Open\-Ended Homogeneity of Language Models \(and Beyond\)\.arXiv\.Note:arXiv:2510\.22954 \[cs\]External Links:[Link](http://arxiv.org/abs/2510.22954),[Document](https://dx.doi.org/10.48550/arXiv.2510.22954)Cited by:[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p5.1)\.
- Kimi Team \(2025\)Kimi K2: open agentic intelligence\.External Links:2507\.20534Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.15.11.11.11.3)\.
- R\. Laine, B\. Chughtai, J\. Betley, K\. Hariharan, J\. Scheurer, M\. Balesni, M\. Hobbhahn, A\. Meinke, and O\. Evans \(2024\)Me, Myself, and AI: The Situational Awareness Dataset \(SAD\) for LLMs\.arXiv\.Note:arXiv:2407\.04694 \[cs\]External Links:[Link](http://arxiv.org/abs/2407.04694),[Document](https://dx.doi.org/10.48550/arXiv.2407.04694)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p3.1)\.
- D\. Li, R\. Sun, Y\. Huang, M\. Zhong, B\. Jiang, J\. Han, X\. Zhang, W\. Wang, and H\. Liu \(2025\)Preference Leakage: A Contamination Problem in LLM\-as\-a\-judge\.arXiv\.Note:arXiv:2502\.01534 \[cs\]\. Presented at ICLR 2026External Links:[Link](http://arxiv.org/abs/2502.01534),[Document](https://dx.doi.org/10.48550/arXiv.2502.01534)Cited by:[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p2.1)\.
- R\. Ngo, L\. Chan, and S\. Mindermann \(2024\)The Alignment Problem from a Deep Learning Perspective\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=fh8EYKFKns)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p1.1)\.
- OpenAI \(2023\)GPT\-4 technical report\.External Links:2303\.08774Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.15.11.11.13.1.1)\.
- OpenAI \(2024\)GPT\-4o system card\.External Links:2410\.21276Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.15.11.11.13.1.1)\.
- A\. Panickssery, S\. R\. Bowman, and S\. Feng \(2024\)LLM Evaluators Recognize and Favor Their Own Generations\.arXiv\(en\)\.Note:arXiv:2404\.13076 \[cs\]External Links:[Link](http://arxiv.org/abs/2404.13076),[Document](https://dx.doi.org/10.48550/arXiv.2404.13076)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p2.1),[§1](https://arxiv.org/html/2608.26159#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p2.1),[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p3.1)\.
- E\. Perez, S\. Ringer, K\. Lukosiute, K\. Nguyen, E\. Chen, S\. Heiner, C\. Pettit, C\. Olsson, S\. Kundu, S\. Kadavath, A\. Jones, A\. Chen, B\. Mann, B\. Israel, B\. Seethor, C\. McKinnon, C\. Olah, D\. Yan, D\. Amodei, D\. Amodei, D\. Drain, D\. Li, E\. Tran\-Johnson, G\. Khundadze, J\. Kernion, J\. Landis, J\. Kerr, J\. Mueller, J\. Hyun, J\. Landau, K\. Ndousse, L\. Goldberg, L\. Lovitt, M\. Lucas, M\. Sellitto, M\. Zhang, N\. Kingsland, N\. Elhage, N\. Joseph, N\. Mercado, N\. DasSarma, O\. Rausch, R\. Larson, S\. McCandlish, S\. Johnston, S\. Kravec, S\. El Showk, T\. Lanham, T\. Telleen\-Lawton, T\. Brown, T\. Henighan, T\. Hume, Y\. Bai, Z\. Hatfield\-Dodds, J\. Clark, S\. R\. Bowman, A\. Askell, R\. Grosse, D\. Hernandez, D\. Ganguli, E\. Hubinger, N\. Schiefer, and J\. Kaplan \(2023\)Discovering Language Model Behaviors with Model\-Written Evaluations\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 13387–13434\.External Links:[Link](https://aclanthology.org/2023.findings-acl.847/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.847)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p1.1)\.
- F\. Roger \(2024\)An issue with training schemers with supervised fine\-tuning\.Note:AlignmentForum blog postExternal Links:[Link](https://www.alignmentforum.org/posts/mXYdYh6L9odTJZDSm/an-issue-with-training-schemers-with-supervised-fine-tuning)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p2.1)\.
- RyokoAI \(2023\)ShareGPT52K\.Note:[https://huggingface\.co/datasets/RyokoAI/ShareGPT52K](https://huggingface.co/datasets/RyokoAI/ShareGPT52K)Dataset of human\-LLM conversations, accessed via HuggingFaceCited by:[Appendix E](https://arxiv.org/html/2608.26159#A5.p2.1),[§3\.4](https://arxiv.org/html/2608.26159#S3.SS4.p1.1)\.
- M\. Terekhov, A\. Panfilov, D\. Dzenhaliou, C\. Gulcehre, M\. Andriushchenko, A\. Prabhu, and J\. Geiping \(2025\)Adaptive attacks on trusted monitors subvert ai control protocols\.External Links:2510\.09462,[Link](https://arxiv.org/abs/2510.09462)Cited by:[§1](https://arxiv.org/html/2608.26159#S1.p1.1)\.
- C\. Tice, P\. Radmard, S\. Ratnam, A\. Kim, D\. Africa, and K\. O’Brien \(2026\)Alignment Pretraining: AI Discourse Causes Self\-Fulfilling \(Mis\)alignment\.arXiv\.Note:arXiv:2601\.10160External Links:[Link](http://arxiv.org/abs/2601.10160),[Document](https://dx.doi.org/10.48550/arXiv.2601.10160)Cited by:[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p1.1)\.
- K\. Wataoka, T\. Takahashi, and R\. Ri \(2024\)Self\-Preference Bias in LLM\-as\-a\-Judge\.arXiv\.Note:arXiv:2410\.21819 \[cs\]External Links:[Link](http://arxiv.org/abs/2410.21819),[Document](https://dx.doi.org/10.48550/arXiv.2410.21819)Cited by:[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p2.1)\.
- A\. Yang, B\. Yang,et al\.\(2024\)Qwen2\.5 technical report\.External Links:2412\.15115Cited by:[Table 4](https://arxiv.org/html/2608.26159#A2.T4.12.8.8.8.3)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-Judge with MT\-Bench and Chatbot Arena\.arXiv\.Note:arXiv:2306\.05685 \[cs\]\. Presented at NeurIPS 2023 Datasets and Benchmarks TrackExternal Links:[Link](http://arxiv.org/abs/2306.05685),[Document](https://dx.doi.org/10.48550/arXiv.2306.05685)Cited by:[§2\.2](https://arxiv.org/html/2608.26159#S2.SS2.p2.1)\.
- T\. Y\. Zhuo, M\. C\. Vu, J\. Chim, H\. Hu, W\. Yu, R\. Widyasari, I\. N\. B\. Yusuf, H\. Zhan, J\. He, I\. Paul, S\. Brunner, C\. Gong, T\. Hoang, A\. R\. Zebaze, X\. Hong, W\. Li, J\. Kaddour, M\. Xu, Z\. Zhang, P\. Yadav, N\. Jain, A\. Gu, Z\. Cheng, J\. Liu, Q\. Liu, Z\. Wang, B\. Hui, N\. Muennighoff, D\. Lo, D\. Fried, X\. Du, H\. d\. Vries, and L\. V\. Werra \(2025\)BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions\.arXiv\.Note:arXiv:2406\.15877 \[cs\]External Links:[Link](http://arxiv.org/abs/2406.15877),[Document](https://dx.doi.org/10.48550/arXiv.2406.15877)Cited by:[Appendix E](https://arxiv.org/html/2608.26159#A5.p4.1),[§3\.4](https://arxiv.org/html/2608.26159#S3.SS4.p1.1)\.

## Appendix APrior Work Detailed Mapping

Table[2](https://arxiv.org/html/2608.26159#A1.T2)provides a detailed mapping of prior SGTR work across operationalization dimensions\.

Table 2:Prior work: experimental design\. Abbreviations: PW – Pairwise, IND – Individual, MC – Multiple Choice \(n\>2n\>2\), IND\-MC – Individual text with multiple\-choice identification\. All studies use prompting with candidate text in user tags\.Table 3:Prior work: datasets and key findings\.
## Appendix BModel Sets

Table 4:Evaluated language models\. The full set \(21 models\) is used for per\-model analysis and user\-tag experiments\. The reduced set \(13 instruct models, unmarked rows\) is the subset available across all conditions, used for cross\-operationalization and assistant\-tag experiments\.†\\dagger: run in both instruct and reasoning modes\.‡\\ddagger: excluded from the reduced set\.We first present recognition performance across experimental paradigms, then examine the relationship between SGTR and model capability, and finally investigate the role of quality\-based heuristics in recognition judgments\. In the following, the performance score of each model is an average over evaluations on text generated by all models in our set\.

### B\.1Recognition Performance Across Paradigms

Figure[5](https://arxiv.org/html/2608.26159#A2.F5)presents recognition accuracy \(performance scores\) across the pairwise and individual paradigms\. In the pairwise setting \(Figure[5](https://arxiv.org/html/2608.26159#A2.F5)a\), several models demonstrate substantial above\-chance performance, with Gemini 2\.5 Pro and Opus 4\.1 \(instruct\) achieving accuracies of∼\\sim0\.80 and above on all datasets\. Qwen 3\.0 80B also performs well above chance on all datasets, excelling on WikiSum at∼\\sim0\.95\. Notable poor performers include Llama 3\.1 8B and Gemini 2\.0 Flash Lite, which fall significantly below chance on multiple datasets\.

In the individual setting \(Figure[5](https://arxiv.org/html/2608.26159#A2.F5)b\), overall performance is substantially attenuated, with most models clustering closer to chance \(0\.5\)\. Claude Opus 4\.1 \(both instruct and reasoning variations\) maintains high performance across three domains \(WikiSum, ShareGPT, and PKU\-SafeRLHF\), while GPT 4\.1 performs well in all domains\. The performance gap between top and bottom performers narrows considerably compared to the pairwise paradigm\.

Here, we want to highlight general trends in the operationalizations\. In Appendix[C](https://arxiv.org/html/2608.26159#A3), we discuss specific cases of interest and display a clean contrast via Figure[6](https://arxiv.org/html/2608.26159#A3.F6)\.

![Refer to caption](https://arxiv.org/html/2608.26159v1/x5.png)Figure 5:Recognition performance across paradigms\. \(a\) Bar chart showing evaluator performance by dataset for the pairwise \(PW\) recognition paradigm\. Models ordered by mean performance, with grouped bars for WikiSum \(blue\), ShareGPT \(orange\), PKU\-SafeRLHF \(green\), and BigCodeBench \(red\)\. Dashed horizontal line at 0\.5 indicates chance\. Asterisks indicatep<0\.05p<0\.05\. \(b\) Bar chart showing evaluator performance for the individual \(IND\) recognition paradigm\.Recognition performance varies widely across task domains\. Top performances are inconsistent, but are most often elicited by PKU\-SafeRLHF, particularly for the reasoning models\. PKU\-SafeRLHF also generates strong systematic biases that lower performance scores below chance in Gemini 2\.0 Flash Lite and Llama 3\.1 8B, and in Haiku 3\.5, specifically in the pairwise paradigm\. BigCodeBench shows highly variable performance across models\. Notably, Claude Opus 4\.1 exhibits a marked performance drop on BigCodeBench in reasoning mode compared to instruct mode, and in the individual setting compared to pairwise\. Performances on WikiSum and ShareGPT are also highly variable, eliciting the best performances in some models and the worst in others\.

## Appendix CDifference in Performance

![Refer to caption](https://arxiv.org/html/2608.26159v1/x6.png)Figure 6:Difference in recognition performance across paradigms\. Bar chart showing evaluator performance differences by dataset for the pairwise \(PW\) vs\. individual \(IND\) recognition paradigms\. Models ordered by mean performance, with grouped bars for WikiSum \(blue\), ShareGPT \(orange\), PKU\-SafeRLHF \(green\), and BigCodeBench \(red\)\. Dashed horizontal line at 0\.5 indicates chance\. Asterisks indicatep<0\.05p<0\.05\.Figure[6](https://arxiv.org/html/2608.26159#A3.F6)shows the difference in performance between the pairwise and individual assessments \(Figure[5](https://arxiv.org/html/2608.26159#A2.F5)\)\. Gemini 2\.5 Pro and Qwen 3\.0 80B exhibit the largest overall performance drops, ranking as the number 1 and number 3 models in cumulative score in the pairwise condition, but falling to ranks 8 and 12 in the individual setting, respectively\. Claude Opus 4\.1 and Sonnet 4\.5 similarly show large decrements\. Some models appear to perform much better in certain domains of the individual assessment compared to pairwise\. However, this can be explained largely by performance trade\-offs between the quality heuristic in the pairwise setting and the attribution bias in the individual setting, as the former can cause the model to perform systematically below chance, while the latter enables the model to perform at chance\. GPT 4\.1 performs well across all domains and paradigms, notably improving in the ShareGPT individual assessment without a clear explanatory confound\. GPT\-OSS 120B is another outlier that performs much better in the individual assessment for the PKU\-SafeRLHF dataset\. However, this can be explained by a peculiarity specific to this model, which refuses to answer most questions in the dataset, but is seemingly unable to calibrate its rate of refusal in the pairwise setting\.

## Appendix DRelationship Between Model Capability and Recognition

We hypothesize that SGTR performance correlates with general model capability, as measured by Arena rankings\. Figure[7](https://arxiv.org/html/2608.26159#A4.F7)plots recognition accuracy against Arena ranking for both paradigms\.

In the pairwise setting \(Figure[7](https://arxiv.org/html/2608.26159#A4.F7)a\), we observe strong negative correlations across all datasets \(WikiSum:r=−0\.54r=\-0\.54, ShareGPT:r=−0\.65r=\-0\.65, PKU\-SafeRLHF:r=−0\.76r=\-0\.76, BigCodeBench:r=−0\.61r=\-0\.61\), indicating that higher\-ranked models achieve superior recognition accuracy\. In the individual setting \(Figure[7](https://arxiv.org/html/2608.26159#A4.F7)b\), correlations are attenuated but remain present \(WikiSum:r=−0\.60r=\-0\.60, ShareGPT:r=−0\.53r=\-0\.53, PKU\-SafeRLHF:r=−0\.38r=\-0\.38, BigCodeBench:r=−0\.31r=\-0\.31\)\.

![Refer to caption](https://arxiv.org/html/2608.26159v1/x7.png)Figure 7:Relationship between model capability and recognition\. Scatter plots of recognition accuracy vs\. Arena ranking for \(a\) pairwise \(PW\) and \(b\) individual \(IND\) paradigms\. Points colored by model family, shaped by dataset\. Separate regression lines per dataset with confidence bands\. Individual paradigm results corrected for attribution bias\.
## Appendix EDataset Descriptions

WikiSum \(Summarization\)\.We extract 50 unique articles from the WikiSum dataset\(Cohenet al\.,[2021](https://arxiv.org/html/2608.26159#bib.bib13),[HuggingFace:wikisum](https://huggingface.co/datasets/d0rj/wikisum)\)to prompt our generation of summaries\. This dataset consists of instructional WikiHow articles and candidate summaries, selected to test models’ ability to condense complex instructional text\. Note: although the original contribution of this dataset is its summaries, we do not use them in these experiments\.

ShareGPT \(Conversational & Miscellaneous\)\.We extract 100 unique prompts from the ShareGPT dataset\(RyokoAI,[2023](https://arxiv.org/html/2608.26159#bib.bib63),[HuggingFace:RyokoAI/ShareGPT52K](https://huggingface.co/datasets/RyokoAI/ShareGPT52K)\), which comprises human\-AI conversations shared on the ShareGPT platform\. This dataset provides diverse conversational prompts covering a wide array of topics and interaction styles\. We filter for English\-language text and remove model\-specific names to prevent potential confounding signals\.

PKU\-SafeRLHF \(Safety & Alignment\)\.We extract 50 prompts from the PKU\-SafeRLHF dataset\(Jiet al\.,[2025](https://arxiv.org/html/2608.26159#bib.bib15),[Huggingface:PKU\-SafeRLHF](https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF)\)\. Developed by the PKU\-Alignment team at Peking University, this dataset contains prompt\-response pairs focused on safety\-aligned question answering\.

BigCodeBench \(Coding & Technical\)\.We extract 50 unique prompts from BigCodeBench\(Zhuoet al\.,[2025](https://arxiv.org/html/2608.26159#bib.bib18),[Huggingface:bigcodebench](https://huggingface.co/datasets/bigcode/bigcodebench)\)\. This dataset presents programming challenges and technical problems requiring structured reasoning and domain\-specific knowledge\.

## Appendix FFull Prompt Texts

Tables[5](https://arxiv.org/html/2608.26159#A6.T5)–[8](https://arxiv.org/html/2608.26159#A6.T8)present the exact prompt templates used across our four experimental paradigms\. These paradigms vary systematically along two dimensions: assessment format \(pairwise versus individual\) and judgment type \(recognition versus preference\)\. For recognition tasks, evaluator models are asked to identify which response they originally produced; for preference tasks, they are asked to assess response quality without any reference to authorship\.

Table 5:Experiment Paradigm 1: Pairwise RecognitionTable 6:Experiment Paradigm 2: Individual RecognitionTable 7:Experiment Paradigm 3: Pairwise PreferenceTable 8:Experiment Paradigm 4: Individual PreferenceTables[9](https://arxiv.org/html/2608.26159#A6.T9)–[12](https://arxiv.org/html/2608.26159#A6.T12)detail the system prompts and representative user requests for each of the four evaluation datasets: PKU\-SafeRLHF, BigCodeBench, WikiSum, and ShareGPT\.

Table 9:Dataset details for PKU\-SafeRLHFTable 10:Dataset details for BigCodeBenchTable 11:Dataset details for WikiSumTable 12:Dataset details for ShareGPT
## Appendix GControlled Analyses

To further investigate whether genuine self\-recognition capabilities exist beyond the quality heuristic identified in Section[4](https://arxiv.org/html/2608.26159#S4), we conduct exploratory analyses that control for quality differentials between evaluator and generator\. These analyses restrict the data to subsets where quality\-based strategies should be less effective, allowing us to examine residual recognition signals\. However, the restricted sample sizes limit the strength of conclusions that can be drawn; we present these results as preliminary findings that motivate future work\.

### G\.1Score\-Controlled Performance by Model

We first restrict analysis to model pairings with similar Arena ranks \(rank distance within±\\pm20\), eliminating comparisons where quality differentials would provide a reliable recognition signal\. Figure[8](https://arxiv.org/html/2608.26159#A7.F8)presents recognition accuracy grouped by evaluator for these rank\-similar pairings\.

In the pairwise paradigm \(Figure[8](https://arxiv.org/html/2608.26159#A7.F8)a\), substantial variation in recognition ability is evident even among similarly ranked models\. Top performers include Qwen 3\.0 80B, GPT 4\.1 Mini, Claude Opus 4\.1, and Gemini 2\.0 Flash, suggesting these models may possess recognition capabilities beyond quality assessment\. In the individual paradigm \(Figure[8](https://arxiv.org/html/2608.26159#A7.F8)b\), after adjusting for attribution bias, Qwen 2\.5 72B, Claude Opus 4\.1, and Llama 3\.1 70B show the strongest performance\. However, the small number of comparison pairs available for each model in this restricted subset limits confidence in these rankings\.

![Refer to caption](https://arxiv.org/html/2608.26159v1/figures/figure_2a_ICML_07_rank_distance_grouped_bar_chart.png)\(a\)Pairwise, rank distance±\\pm20
![Refer to caption](https://arxiv.org/html/2608.26159v1/figures/figure_2b_ICML_08_rank_distance_grouped_bar_chart.png)\(b\)Individual, rank distance±\\pm20

Figure 8:Recognition accuracy by evaluator for model pairings with rank distance within±\\pm20\. Some models maintain above\-chance performance even when quality differentials are controlled, though small sample sizes per model limit conclusions\.
### G\.2Capability\-Recognition Correlations Under Controlled Conditions

We examine whether the capability\-recognition correlations observed in the main analysis \(Figure[7](https://arxiv.org/html/2608.26159#A4.F7)\) persist when quality differentials are removed\. Figure[9](https://arxiv.org/html/2608.26159#A7.F9)presents scatter plots of recognition accuracy against evaluator Arena rank under two filtering conditions\.

Under the±\\pm20 rank distance filter \(Figure[9](https://arxiv.org/html/2608.26159#A7.F9)a,[9](https://arxiv.org/html/2608.26159#A7.F9)b\), the relationship between evaluator capability and recognition accuracy substantially weakens\. In the pairwise paradigm, correlations across datasets drop tor=−0\.17r=\-0\.17\(WikiSum\),r=−0\.29r=\-0\.29\(ShareGPT\),r=−0\.21r=\-0\.21\(PKU\-SafeRLHF\), andr=−0\.23r=\-0\.23\(BigCodeBench\)—a substantial attenuation from ther=0\.62r=0\.62to0\.750\.75observed in the uncontrolled analysis \(positive here because this figure’s rank axis is inverted, placing higher\-capability evaluators to the right; the correlations against the non\-inverted rank axis of Figure[7](https://arxiv.org/html/2608.26159#A4.F7)are correspondingly negative\)\. In the individual paradigm, results are mixed: WikiSum shows an anomalous positive correlation \(r=0\.50r=0\.50\), while other datasets show weak to moderate negative correlations \(ShareGPT:r=−0\.16r=\-0\.16, PKU\-SafeRLHF:r=−0\.31r=\-0\.31, BigCodeBench:r=−0\.63r=\-0\.63\)\.

We also examine performance when restricting to cases where the generator is at least as capable as the evaluator \(rank distance\>−20\>\-20\), removing the “easy” cases where evaluators judge text from substantially weaker models \(Figure[9](https://arxiv.org/html/2608.26159#A7.F9)c,[9](https://arxiv.org/html/2608.26159#A7.F9)d\)\. Under these conditions, correlations are attenuated relative to the full dataset but remain present\. In the pairwise paradigm, correlations range fromr=−0\.22r=\-0\.22\(BigCodeBench\) tor=−0\.49r=\-0\.49\(ShareGPT\), withr=−0\.46r=\-0\.46for PKU\-SafeRLHF andr=−0\.26r=\-0\.26for WikiSum\. In the individual paradigm, correlations are further attenuated, with WikiSum showing a weak positive relationship \(r=0\.19r=0\.19\) and other datasets showing weak negative correlations \(ShareGPT:r=−0\.25r=\-0\.25, PKU\-SafeRLHF:r=−0\.27r=\-0\.27, BigCodeBench:r=−0\.47r=\-0\.47\)\.

![Refer to caption](https://arxiv.org/html/2608.26159v1/figures/figure_3a_ICML_07_rank_distance_filtered_evaluator_rank.png)\(a\)Pairwise,±\\pm20 filter
![Refer to caption](https://arxiv.org/html/2608.26159v1/figures/figure_3b_ICML_08_rank_distance_filtered_evaluator_rank.png)\(b\)Individual,±\\pm20 filter
![Refer to caption](https://arxiv.org/html/2608.26159v1/figures/figure_3c_ICML_07_rank_distance_filtered_evaluator_rank_positive.png)\(c\)Pairwise, distance\>−20\>\-20
![Refer to caption](https://arxiv.org/html/2608.26159v1/figures/figure_3d_ICML_08_rank_distance_filtered_evaluator_rank_positive.png)\(d\)Individual, distance\>−20\>\-20

Figure 9:Recognition accuracy vs\. evaluator Arena rank under controlled conditions\. \(a, b\) Rank distance within±\\pm20; correlations substantially attenuated\. \(c, d\) Rank distance\>−20\>\-20\(generator at least as capable\); more data points, but correlations still weaker than uncontrolled analysis\. Points colored by model family, shaped by dataset\. X\-axis reversed \(lower rank = higher capability\)\. The label \(R\) appending model names indicates evaluations run with chain\-of\-thought reasoning enabled\.
### G\.3Interpretation and Caveats

The attenuation of capability\-recognition correlations under controlled conditions is consistent with the quality\-heuristic interpretation presented in the main text: when quality differentials are removed, much of the recognition signal disappears\. However, several caveats limit the strength of conclusions from these analyses\.

First, restricting to rank distance±\\pm20 substantially increases the proportion of comparisons between models from the same developer \(e\.g\., different Claude versions, different GPT variants\)\. Shared training procedures, data sources, or architectural choices within model families could inflate or deflate recognition accuracy through mechanisms unrelated to general self\-recognition capability\. The anomalous positive correlation for WikiSum in the individual paradigm may partly reflect this confound\.

Second, the restricted sample sizes—particularly for the±\\pm20 filter—reduce statistical power and increase variance in the correlation estimates\. The observed correlations should be interpreted as suggestive trends rather than definitive measurements\.

Third, Arena rankings provide an imperfect proxy for the quality dimensions relevant to SGTR\. Models with similar Arena rankings may still differ substantially in output quality for specific tasks, meaning our “controlled” conditions may not fully eliminate quality\-based signals\.

Future work with larger model sets and more carefully matched comparison pairs could provide stronger evidence regarding residual self\-recognition capabilities\. Alternative approaches—such as adversarial quality equalization or mechanistic interpretability—may also help disentangle quality heuristics from genuine authorship recognition\.