OLMo-Detect：一个针对大语言模型成员推理的多阶段、混淆变量受控基准

# OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models
Source: [https://arxiv.org/html/2610.02986](https://arxiv.org/html/2610.02986)
###### Abstract

Membership inference on large language models \(LLMs\) aims to determine whether a given text sample was included in an LLM’s training data, without access to its training corpus\. Despite recent progress, existing benchmarks suffer from three limitations: limited coverage of training stages, insufficient distributional alignment between members and non\-members, and lack of rigorous filtering of non\-members against the training corpus\. To address these limitations, we proposeOLMo\-Detect, a multi\-stage, confounder\-controlled benchmark built upon the fully open OLMo 2 pipeline\.OLMo\-Detectspans pre\-training, mid\-training, and post\-training, explicitly aligns members and non\-members on three key axes, and rigorously filters non\-members via infini\-gram\. To assess robustness to distribution shifts, we further introduceOLMo\-Detect \(Shifted\), a variant where members are misaligned with non\-members\. We evaluate 15 unsupervised and 3 supervised membership inference attacks \(MIAs\) across the OLMo 2 family, finding that: \(i\) overall performance is limited: the best unsupervised and supervised MIAs both reach an AUC of only 0\.68, and supervised MIAs degrade under cross\-domain evaluation; \(ii\) MIA performance peaks at mid\-training and is lower at pre\-training and post\-training, a pattern driven by data type rather than a stage effect: curated math data is far more detectable than other types; \(iii\) overall scores improve from 1B to 13B but plateau at 32B; and \(iv\) no unsupervised MIA is robust to distribution shifts, with AUCs shifting by up to 0\.42\. Finally, we find that our findings on OLMo 2 generalize to OLMo 3 and non\-OLMo models\.

## 1Introduction

Membership inference on large language models \(LLMs\) aims to determine whether a given text sample was included in an LLM’s training data, without access to its training corpus\. Despite recent progress\([Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8);[Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14);[Zhang et al\., 2024a](https://arxiv.org/html/2610.02986#bib.bib9);[Dong et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib16);[Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18);[Fu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib19);[Tao et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib17);[Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12);[Zhang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib59);[Kim et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib13)\), there is still a lack ofmulti\-stage, confounder\-controlled benchmarks\.

As illustrated in Table[1](https://arxiv.org/html/2610.02986#S1.T1), existing benchmarks suffer from three limitations: \(i\)Limited coverage of training stages\.Modern LLM development\([Grattafiori et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib1);[OLMo et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib3);[Yang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib22)\)has evolved into a three\-stage pipeline of pre\-training, mid\-training, and post\-training, where mid\-training refines model capabilities through targeted high\-quality data and annealing\-style optimization\([Tu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib21)\)\. However, most existing benchmarks focus on the pre\-training stage, with only a few targeting post\-training\. \(ii\)Insufficient distributional alignment between members and non\-members\.Most benchmarks either introduce distribution shifts, such as temporal drift\([Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8);[Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14);[Zhang et al\., 2024b](https://arxiv.org/html/2610.02986#bib.bib10);[Fu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib19)\)and fine\-tuning\-induced differences\([Dong et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib16)\), or achieve only aggregate\-level alignment by construction, such as train/held\-out split\([Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18);[Tao et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib17)\), without explicitly controlling alignment on axes exploitable by existing membership inference attacks \(MIAs\)\. The only exception is OLMoMIA\([Kim et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib13)\), which aligns on semantic similarity\. However, most MIAs\([Carlini et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib25);[Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8);[Chang et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib29);[Zhang et al\., 2024a](https://arxiv.org/html/2610.02986#bib.bib9);[Zhang et al\., 2024b](https://arxiv.org/html/2610.02986#bib.bib10);[Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14);[Xie et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib11);[Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12);[Golchin and Surdeanu, 2025](https://arxiv.org/html/2610.02986#bib.bib15)\)exploit surface\-form signals that manifest through exact wording rather than meaning, so semantically aligned data may still exhibit lexical shifts\. \(iii\)Lack of rigorous filtering of non\-members against the training corpus\.Most benchmarks construct non\-members without filtering out those sharing substantialnn\-gram overlap with the model’s training data\. MIMIR\([Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18)\)is the sole exception but uses a 13\-gram filter with a permissive threshold of 80%, leaving its non\-members with potentially high residual overlap\.

To address these limitations, we proposeOLMo\-Detect, a multi\-stage, confounder\-controlled benchmark built upon the fully open training pipeline of OLMo 2\([OLMo et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib3)\)\. Specifically: \(i\) it spans pre\-training, mid\-training, and post\-training, enabling systematic analysis of how detection performance varies across training stages; \(ii\) it explicitly aligns members and non\-members on three key axes to minimize distribution shifts: text quality, temporal range, and lexical similarity; and \(iii\) it filters non\-members against OLMo 2’s training data via infini\-gram\([Liu et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib20)\)\-based 13\-gram filtering with a strict threshold of 20%, leaving non\-members with an average residual overlap of only 5\.31%\. Moreover, we introduceOLMo\-Detect \(Shifted\), a benchmark variant where members are misaligned with non\-members, to assess robustness to distribution shifts\.

BenchmarkStage\(s\)Domain\(s\)MemberNon\-MemberAssessmentsCAFWikiMIA\([Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8)\)Pre\-trainingWikipediaPre\-2017 dataPost\-2023 data✗✗✗StackMIA\([Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14)\)Pre\-trainingStack ExchangePre\-2017 dataPost\-2023 data✗✗✗DETCON\([Dong et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib16)\)Post\-trainingHumanEval, GSM8KOutputs of post\-SFT LLMsOutputs of base LLMs✗✗✗MIMIR\([Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18)\)Pre\-trainingThe Pile: 7 domainsTrain splitsHeld\-out data✗✗✓†PatentMIA\([Zhang et al\., 2024b](https://arxiv.org/html/2610.02986#bib.bib10)\)Pre\-trainingGoogle PatentsPre\-2023 dataPost\-2024 data✗✗✗WikiMIA\-24\([Fu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib19)\)Pre\-trainingWikipediaPre\-2017 dataPost\-2024 data✗✗✗RL\-MIA\([Tao et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib17)\)Post\-trainingAIME24/25, K&K, SATTraining dataHeld\-out data✗✗✗OLMoMIA\([Kim et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib13)\)Pre\-trainingDolmaPre\-checkpoint dataPost\-checkpoint data✗✓‡✗OLMo\-Detect\(ours\)Pre\-training,Mid\-training,Post\-training9 domains ofOLMo 2Distribution\-matchedtraining dataDistribution\-matched,rigorously filterednon\-members✓✓✓

Table 1:Comparison of existing benchmarks for membership inference on LLMs\. We assess on three criteria: \(i\)Coverage \(C\)of training stages \(pre\-training, mid\-training, and post\-training\); \(ii\)Explicit Alignment \(A\)of distributions between members and non\-members; \(iii\)Rigorous Filtering \(F\)of non\-members against the training corpus\. ✓†indicates that MIMIR uses a filter with a permissive threshold of 80%, leaving potentially high residual overlap\. ✓‡means that OLMoMIA aligns on semantic similarity, but most MIAs target exact wording rather than meaning\.We evaluate 15 unsupervised and 3 supervised MIAs across the OLMo 2 family \(1B/7B/13B/32B\) onOLMo\-Detect, finding that: \(i\)Overall performance is limited\.The best unsupervised method is Zlib\([Carlini et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib25)\), one of the simplest MIAs, while the best supervised method is Sup\-CAMIA\([Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12)\); both achieve an average AUC of 0\.68\. Moreover, supervised MIAs degrade under cross\-domain evaluation, indicating that learned signals do not transfer well\. \(ii\)Detectability is driven by data type rather than training stage\.Overall performance peaks at mid\-training and is lower at pre\-training and post\-training, but this ordering reflects data type rather than a stage effect: curated math data \(e\.g\., GSM8K\([Cobbe et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib66)\)\) is considerably more detectable than other types\. \(iii\) Overall scores improve from 1B to 13B but plateau at 32B, suggesting thatscale alone may not continue to make detection easier\.Additionally, we evaluate unsupervised MIAs onOLMo\-Detect \(Shifted\)and find thatno unsupervised MIA is robust to distribution shifts, with AUCs shifting by up to 0\.42\. Finally, we evaluate the top five unsupervised MIAs on OLMo 3 \(7B/32B\)\([Olmo et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib4)\)and two non\-OLMo families, DCLM\-Baseline \(1B/7B\)\([Li et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib38)\)and SmolLM2\-1\.7B\([Allal et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib67)\), on theOLMo\-Detectdomains they were trained on, and find thatour main findings generalize beyond OLMo 2\.

## 2Related Work

As summarized in Table[1](https://arxiv.org/html/2610.02986#S1.T1), existing benchmarks for membership inference on LLMs can be grouped into four categories based on how members and non\-members are constructed: \(i\) temporal\-cutoff\([Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8);[Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14);[Zhang et al\., 2024b](https://arxiv.org/html/2610.02986#bib.bib10);[Fu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib19)\); \(ii\) partition\-based\([Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18);[Tao et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib17)\); \(iii\) checkpoint\-based\([Kim et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib13)\); and \(iv\) model\-output\-based\([Dong et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib16)\)\. Detailed descriptions are provided in Appendix[A](https://arxiv.org/html/2610.02986#A1)\.

Existing MIAs can be grouped into unsupervised and supervised methods based on whether they use labeled members or non\-members\. Unsupervised MIAs can be classified into four categories: \(i\) likelihood\-based\([Yeom et al\., 2018](https://arxiv.org/html/2610.02986#bib.bib26);[Carlini et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib25);[Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8);[Zhang et al\., 2024a](https://arxiv.org/html/2610.02986#bib.bib9);[Zhang et al\., 2024b](https://arxiv.org/html/2610.02986#bib.bib10);[Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14);[Xie et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib11);[Fu et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib28);[Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12);[Kim et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib13)\); \(ii\) perturbation\-based\([Mattern et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib27)\); \(iii\) prompt\-based\([Golchin and Surdeanu, 2023](https://arxiv.org/html/2610.02986#bib.bib7);[Chang et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib29);[Deng et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib30);[Golchin and Surdeanu, 2025](https://arxiv.org/html/2610.02986#bib.bib15)\); and \(iv\) output\-distribution\-based\([Dong et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib16);[Tao et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib17)\)\. Supervised MIAs include a supervised variant of CAMIA\([Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12)\)and tuning\-based methods\([Fu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib19);[Zhang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib59)\)\. Details are provided in Appendix[B](https://arxiv.org/html/2610.02986#A2)\.

## 3Task Definition

Letℳ\\mathcal\{M\}denote an LLM trained on an unknown corpus𝒟\\mathcal\{D\}\. Given a candidate textxx, the task of membership inference on LLMs is to infer whetherx∈𝒟x\\in\\mathcal\{D\}via a scoring functions⁡\(x,ℳ\)s\(x;\\mathcal\{M\}\), thresholded atλ\\lambda:prediction\(x;ℳ\)=𝟙\[s\(x;ℳ\)≥λ\]\\text\{prediction\}\(x;\\mathcal\{M\}\)=\\mathds\{1\}\[s\(x;\\mathcal\{M\}\)\\geq\\lambda\], where11and00denote member and non\-member, respectively\. Following prior work\([Zhang et al\., 2024a](https://arxiv.org/html/2610.02986#bib.bib9)\), we assume grey\-box access toℳ\\mathcal\{M\}: output statistics \(loss, logits, token probabilities\) are available, internal parameters \(weights, gradients\) are not\. Some methods further assume the ability to fine\-tuneℳ\\mathcal\{M\}, described in Appendix[B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px2.SPx2)\.

## 4Benchmark Construction

### 4\.1Motivation for Using OLMo 2

Constructing benchmarks requires LLMs with open training data to verify members and filter non\-members against the corpus\. Among existing open\-source LLMs, GPT\-NeoX\([Black et al\., 2022](https://arxiv.org/html/2610.02986#bib.bib6)\)and Pythia\([Biderman et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib5)\)are limited to pre\-training; OLMo 1\([Groeneveld et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib2)\)additionally includes post\-training, but only for its 7B variant, and lacks mid\-training; only OLMo 2\([OLMo et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib3)\)and OLMo 3\([Olmo et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib4)\)provide complete access across all three stages\. We constructOLMo\-Detectbased on OLMo 2, as it was the available model when we started this work\. Nonetheless, we show that patterns found on OLMo 2 generalize to OLMo 3 \(Section[5\.5](https://arxiv.org/html/2610.02986#S5.SS5)\)\.

### 4\.2Training Data of OLMo 2

We summarize the training data of OLMo 2 as follows: \(i\)pre\-training: DCLM\-Baseline \(web\), StarCoder \(code\), peS2o and arXiv \(academic papers\), OpenWebMath \(web math\), Algebraic Stack \(math proofs and code\), and Wikipedia/Wikibooks \(encyclopedic\); \(ii\)mid\-training: three reused from pre\-training \(filtered DCLM\-Baseline, peS2o, and Wikipedia/Wikibooks\) and three new domains: FLAN \(instruction data\), Stack Exchange \(Q&A\), and Math Mix \(curated math\); and \(iii\)post\-training: curated data for supervised fine\-tuning \(SFT\), direct preference optimization \(DPO\), and reinforcement learning with verifiable rewards \(RLVR\)\. Details are provided in Appendix[C](https://arxiv.org/html/2610.02986#A3)\.

StageDomainSubsetsSplitSamplesTokensQualityTemporalLexicalPre\-trainingDCLM\-BaselineNM150217,117FastText: 0\.01809±\\pm0\.000072018\-01\-19±\\pm930d0\.07M150217,016FastText: 0\.01811±\\pm0\.000002018\-01\-18±\\pm940dpeS2oNM268216,981—2022\-12\-14±\\pm11d0\.10M268217,230—2022\-12\-15±\\pm13dOpenWebMathNM151217,183—2019\-10\-13±\\pm853d0\.11M151217,252—2019\-10\-17±\\pm864dStarCoderPython,JavaNM156217,156GitHub Stars:35\.65±\\pm66\.66Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.38±\\pm0\.05—0\.15M156217,222GitHub Stars:36\.15±\\pm64\.67Top\-1 Freq:0\.29±\\pm0\.01Top\-2 Freq:0\.36±\\pm0\.05—Mid\-trainingMath MixGSM8KNM1,040217,163——0\.04M1,040217,132——Stack ExchangeStack Overflow,Math Stack ExchangeNM430217,208Question Vote:3\.45±\\pm2\.19Answer Vote:5\.12±\\pm1\.992015\-03\-31±\\pm1378d0\.08M430217,202Question Vote:3\.52±\\pm1\.74Answer Vote:5\.54±\\pm1\.942015\-03\-30±\\pm1367dPost\-trainingSFT†WildChat,AyaNM225217,057——0\.11M225217,159——DPO†UF\-WildChatNM113217,045——0\.09M113217,287——RLVRCoT\-GSM8K,CoT\-MATHNM290217,110——0\.01M290217,025——

Table 2:Overall per\-domain statistics ofOLMo\-Detect\. Within each domain, members and non\-members are aligned on up to three axes: TextQuality,TemporalRange, andLexicalSimilarity\. Text Quality uses OLMo 2’s metric and is enforced for quality\-thresholded domains; Temporal Range is enforced when timestamps are available; and Lexical Similarity, measured by Jensen–Shannon divergence \(lower is better\), is enforced for all domains\. “—” denotes uncontrolled axes\.NM: non\-member;M: member\.Subsetslists constituent subsets; blanks indicate single\-subset domains whose subset shares the domain’s name\.†SFT and DPO are constructed per model size due to differing training data; values shown are for 1B\. Each domain is split into dev/test sets at approximately 1:9 sample and token ratios; only totals are shown\. Full statistics are in Appendix[D](https://arxiv.org/html/2610.02986#A4)\.
### 4\.3Benchmark Overview

As summarized in Table[2](https://arxiv.org/html/2610.02986#S4.T2),OLMo\-Detectcomprises nine domains spanning three training stages of OLMo 2’s training data\. Mid\-training domains reused from pre\-training are excluded; four domains are additionally excluded: pre\-training’s Wikipedia/Wikibooks \(no source\-matched non\-members available\), pre\-training’s arXiv and Algebraic Stack \(no 13\-gram\-filtered non\-members survived\), and mid\-training’s FLAN \(no instance\-length\-matched members available\)\.

### 4\.4Construction of Non\-Members

For each domain, we construct non\-members by \(i\) identifying source\-matched candidate pools, \(ii\) sampling candidates with strategies differing by domain type, \(iii\) retaining those that pass the infini\-gram\-based 13\-gram filter, and \(iv\) allocating token budgets\. Details are provided in Appendix[E](https://arxiv.org/html/2610.02986#A5)\.

Source\-Matched Candidate Pools\.We first identify domains for which non\-members can be drawn from the same underlying source as OLMo 2’s training data, from either held\-out data or the remainder of the original source corpus not used to train OLMo 2\. \(i\)Domains with Held\-Out Data\.Pre\-training: peS2o and OpenWebMath; mid\-training: Math Mix’s GSM8K; post\-training: RLVR’s CoT\-GSM8K and CoT\-MATH\. \(ii\)Domains with the Remainder of Original Source Corpora\.Pre\-training: DCLM\-Baseline and StarCoder; mid\-training: Stack Exchange; post\-training: SFT, DPO, and RLVR’s IFEval\. Among them, three are obtained by quality\-thresholding the original corpora: DCLM\-Baseline \(fastText score≥\\geq0\.01811\), StarCoder \(GitHub stars≥\\geq2; top\-1 word frequency≤\\leq30% and top\-2 word frequencies≤\\leq50%\), and Stack Exchange \(question votes≥\\geq3 and answer votes≥\\geq5\); the rest are obtained via random sampling from the original source corpora\. For domains with multiple sampleable subsets, we select two from the top\-5 subsets by training provenance to capture within\-domain diversity\.

Domain\-Dependent Sampling Strategies\.We then sample candidates based on whether a domain is quality\-thresholded\. \(i\)Random Sampling for Domains without Quality Thresholding\.For domains whose candidate pools come from either held\-out data or the remainder of original source corpora without quality thresholding, we randomly sample candidate non\-members\. \(ii\)Boundary Sampling for Quality\-Thresholded Domains\.For quality\-thresholded domains, random sampling would yield data of systematically lower quality than that used to train OLMo 2\. We thus applyboundary sampling, selecting candidates closest to OLMo 2’s quality threshold\. Since these candidates are drawn from the rejected remainder, they fall immediately below the threshold\.

Infini\-Gram\-based 13\-Gram Filtering\.Finally, we filter out candidates with substantial training\-data overlap via infini\-gram\. Infini\-gram\([Liu et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib20)\)is a suffix\-array\-based indexing system that supports efficientnn\-gram search over massive text corpora for arbitrarily largenn\. For each model size, we index its full training corpus, and apply 13\-gram filtering with a strict threshold of20%20\\%, a strategy established by[Xu et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib56), rather than the permissive 80% used by MIMIR\. Specifically, for each candidate, we extract all 13\-gramsGGand compute the fraction that appear in the index,η=\|\{g∈G:count⁡\(g\)\>0\}\|\|G\|\\eta=\\frac\{\|\\\{g\\in G:\\mathrm\{count\}\(g\)\>0\\\}\|\}\{\|G\|\}\. A candidate is retained as a non\-member ifη<20%\\eta<20\\%\.

Token Budget Allocation\.From the resulting non\-members, we sample toward a uniform budget of approximately 217K tokens per domain, determined by Math Mix’s non\-member token count, and allocate this budget proportionally to training provenance for multi\-subset domains\.

### 4\.5Construction of Members

Given the constructed non\-members per domain, we construct members that are distributionally aligned with them, via \(i\) identifying candidate pools, \(ii\) determining the alignment axes, and \(iii\) constrained sampling using simulated annealing\. We provide detailed descriptions in Appendix[F](https://arxiv.org/html/2610.02986#A6)\.

Candidate Pools\.For domains without quality thresholding, candidate pools are the corresponding OLMo 2 training data\. For quality\-thresholded domains, to obtain candidates matched in quality to their non\-member counterparts, we apply boundary sampling: since members are drawn from data that passed OLMo 2’s quality threshold, this selects candidates immediately above the threshold\.

Alignment Axes\.We identify three key axes for aligning members with non\-members: \(i\)text quality, already enforced via boundary sampling for quality\-thresholded domains; \(ii\)temporal range, enforced when timestamp data are available, since temporal drift can inflate detection scores\([Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18)\); and \(iii\)lexical similarity, measured by Jensen–Shannon divergence \(JSD\) between their token\-frequency distributions and enforced across all domains, as most MIAs target exact wording rather than meaning \(Section[1](https://arxiv.org/html/2610.02986#S1)\), making lexical mismatch a direct confounder\.

Pre\-trainingMid\-trainingPost\-trainingDCLM\-BaselineOpenWebMathpeS2oStarCoderMath MixStack ExchangeSFTDPORLVRAvgOverlap4\.07%8\.33%3\.11%9\.31%6\.65%5\.71%6\.32%2\.20%2\.06%5\.31%BoW AUC0\.490\.520\.480\.500\.520\.480\.510\.490\.510\.50

Table 3:Per\-domain verification results, averaged across model sizes\.Overlap: average 13\-gram overlap of each domain’s non\-members against OLMo 2’s training corpus\.BoW AUC: blind detectability, measured by the AUC of a BoW classifier\. Per\-model\-size results are in Tables[22](https://arxiv.org/html/2610.02986#A6.T22)and[24](https://arxiv.org/html/2610.02986#A6.T24)\.Constrained Sampling via Simulated Annealing\.Given the candidates, we select members that match their non\-member counterparts subject to two constraints: \(i\) alignment on axes beyond text quality; \(ii\) matched instance and token counts, eliminating average instance length as a confounder\. LetBBandAAdenote the sets of non\-members and candidate members for a domain\. We search for a member subsetA∗⊆AA^\{\*\}\\subseteq Athat minimizes a distributional lossℒ\\mathcal\{L\}via simulated annealing\([Lecchini\-Visintini et al\., 2007](https://arxiv.org/html/2610.02986#bib.bib58)\), which probabilistically accepts suboptimal moves to escape local minima:

A∗=arg⁡minA′⊆A\\displaystyle A^\{\*\}=\\arg\\min\_\{A^\{\\prime\}\\subseteq A\}JSD⁡\(pA′,pB\)\+𝟙​\[temporal metadata available\]⋅ℒtemp\\displaystyle\\mathrm\{JSD\}\(p\_\{A^\{\\prime\}\},p\_\{B\}\)\+\\mathds\{1\}\[\\text\{temporal metadata available\}\]\\cdot\\mathcal\{L\}\_\{\\text\{temp\}\}\(1\)s\.t\.\\displaystyle\\text\{s\.t\.\}\|A′\|=\|B\|,\|∑x∈A′tok\(x\)−∑x∈Btok\(x\)\|≤δ,\\displaystyle\|A^\{\\prime\}\|=\|B\|,\\quad\\left\|\\textstyle\\sum\_\{x\\in A^\{\\prime\}\}\\mathrm\{tok\}\(x\)\-\\sum\_\{x\\in B\}\\mathrm\{tok\}\(x\)\\right\|\\leq\\delta,wherepA′p\_\{A^\{\\prime\}\}andpBp\_\{B\}denote token distributions of the candidate member subset and non\-members, respectively;tok⁡\(x\)\\mathrm\{tok\}\(x\)denotes the token count of instancexx;δ\\deltais the token margin, which is set to 250;JSD\\mathrm\{JSD\}enforces lexical alignment and is bounded in\[0,1\]\[0,1\]; andℒtemp\\mathcal\{L\}\_\{\\text\{temp\}\}, which is normalized to the same scale asJSD\\mathrm\{JSD\}, aligns members and non\-members’ mean and standard deviation of timestamps\.

### 4\.6Benchmark Verification

Residual Overlap of Non\-Members\.Table[3](https://arxiv.org/html/2610.02986#S4.T3)shows that non\-members ofOLMo\-Detectexhibit an average 13\-gram overlap of 5\.31% against OLMo 2’s full training corpus, with a maximum of just 9\.31%, confirming the effectiveness of our infini\-gram\-based filtering scheme\.

Verification of Lexical Alignment\.Since lexical similarity has no ground truth and is measured using a summary statistic \(JSD\), residual lexical shifts may still remain\. To rule this out, we follow[Das et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib65)and probe each domain with a*blind*attack: a bag\-of\-words \(BoW\) classifier that predicts membership from text alone, without access to the LLM\. If residual lexical confounders exist, the AUC of BoW will exceed chance\. As shown in Table[3](https://arxiv.org/html/2610.02986#S4.T3), every domain attains a blind AUC within0\.020\.02of0\.500\.50, confirming that members and non\-members are lexically well aligned\.

### 4\.7OLMo\-Detect \(Shifted\)

To examine the effect of distribution shifts on detection robustness, we introduceOLMo\-Detect \(Shifted\)\. For each domain, non\-members remain unchanged, but members are randomly sampled to satisfy only the constraint of matched instance and token counts, without enforcing alignment on text quality, temporal range, or lexical similarity\. Mid\-training’s Math Mix and post\-training’s RLVR are excluded fromOLMo\-Detect \(Shifted\), as randomly sampled members do not yield substantially differentJSD\\mathrm\{JSD\}from their corresponding non\-members\. Overall statistics ofOLMo\-Detect \(Shifted\)are reported in Table[30](https://arxiv.org/html/2610.02986#A8.T30), with detailed descriptions in Appendix[G](https://arxiv.org/html/2610.02986#A7)\.

Method1B7B13B32BModel\-AvgPreMidPostAllPreMidPostAllPreMidPostAllPreMidPostAllPreMidPostAllPPLL0\.540\.640\.610\.600\.550\.720\.680\.650\.560\.790\.660\.670\.580\.800\.680\.690\.560\.740\.660\.65ZlibL0\.570\.670\.660\.630\.590\.760\.660\.670\.590\.830\.670\.700\.610\.850\.690\.720\.590\.780\.670\.68LowerL0\.570\.610\.590\.590\.570\.690\.620\.630\.580\.750\.650\.660\.580\.730\.670\.660\.580\.690\.630\.63MinKL0\.550\.650\.620\.610\.560\.730\.670\.650\.570\.800\.660\.680\.590\.810\.690\.700\.570\.750\.660\.66MinK\+\+L0\.480\.530\.590\.540\.500\.540\.620\.560\.510\.590\.590\.570\.540\.560\.630\.570\.510\.560\.610\.56DCPDDL0\.590\.610\.590\.600\.600\.760\.660\.670\.610\.830\.640\.690\.630\.830\.670\.710\.610\.760\.640\.67PACL0\.580\.650\.610\.610\.590\.730\.600\.640\.620\.810\.610\.680\.630\.800\.630\.690\.600\.750\.610\.66ReCaLLL0\.500\.430\.500\.480\.540\.290\.490\.440\.550\.340\.480\.450\.520\.370\.540\.480\.530\.360\.500\.46EMMIAL0\.560\.530\.610\.570\.580\.540\.610\.570\.570\.560\.600\.580\.610\.590\.470\.560\.580\.560\.570\.57CAMIAL0\.550\.670\.590\.600\.530\.730\.720\.660\.540\.800\.660\.670\.540\.820\.660\.670\.540\.760\.660\.65Neigh\.P0\.550\.590\.530\.560\.540\.640\.550\.580\.550\.730\.530\.600\.550\.760\.570\.630\.550\.680\.550\.59GuidedPr0\.470\.520\.500\.500\.500\.470\.500\.490\.470\.480\.490\.480\.480\.550\.460\.500\.480\.510\.490\.49DCQPr0\.520\.500\.460\.500\.410\.450\.400\.420\.490\.590\.480\.520\.560\.580\.470\.540\.500\.530\.450\.49CDDO0\.510\.510\.510\.510\.500\.540\.500\.520\.510\.600\.500\.540\.500\.620\.490\.540\.510\.570\.500\.52SelfCritO†——0\.460\.46——0\.500\.50——0\.440\.44——0\.480\.48——0\.470\.47Method\-Avg0\.540\.580\.560\.560\.540\.610\.590\.580\.550\.680\.580\.600\.570\.690\.590\.610\.550\.640\.580\.58

Table 4:AUCs of unsupervised MIAs onOLMo\-Detect, grouped by family:Llikelihood\-based,Pperturbation\-based,Prprompt\-based, andOoutput\-distribution\-based\.Pre/Mid/Postmacro\-average AUC over each stage’s domains, andAllmacro\-averages all stages\.Model\-Avgaverages across model sizes, andMethod\-Avgaverages across methods\. The best method per column is in bold\.†SelfCrit is evaluated on post\-training’s DPO and RLVR\. Per\-domain results are in Appendix[I](https://arxiv.org/html/2610.02986#A9)\.

## 5Experiments

### 5\.1Experimental Settings

We describe our experimental settings below, with details in Appendix[H](https://arxiv.org/html/2610.02986#A8)\. \(i\)Models\.We evaluate four instruction\-tuned OLMo 2 models: 1B, 7B, 13B, and 32B\. \(ii\)Compared Methods\.We evaluate 15 unsupervised MIAs across four families: \(a\) likelihood\-based: Perplexity \(PPL\)\([Yeom et al\., 2018](https://arxiv.org/html/2610.02986#bib.bib26)\), Zlib and Lowercase \(Lower\)\([Carlini et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib25)\), Min\-K% \(MinK\)\([Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8)\), Min\-K%\+\+ \(MinK\+\+\)\([Zhang et al\., 2024a](https://arxiv.org/html/2610.02986#bib.bib9)\), DCPDD\([Zhang et al\., 2024b](https://arxiv.org/html/2610.02986#bib.bib10)\), PAC\([Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14)\), ReCaLL\([Xie et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib11)\), EMMIA\([Kim et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib13)\), and CAMIA\([Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12)\); \(b\) perturbation\-based: Neighborhood Attack \(Neigh\.\)\([Mattern et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib27)\); \(c\) prompt\-based: Guided Instruction \(Guided\)\([Golchin and Surdeanu, 2023](https://arxiv.org/html/2610.02986#bib.bib7)\)and DCQ\([Golchin and Surdeanu, 2025](https://arxiv.org/html/2610.02986#bib.bib15)\); and \(d\) output\-distribution\-based: CDD\([Dong et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib16)\)and Self\-Critique \(SelfCrit\)\([Tao et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib17)\)\(evaluated only on DPO and RLVR, following its original setup\)\. We also evaluate three supervised MIAs: the supervised variant of CAMIA \(Sup\-CAMIA\)\([Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12)\), MIA\-Tuner\([Fu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib19)\), and FSD\([Zhang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib59)\)\. \(iii\)Evaluation Metrics\.Following prior work\([Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8)\), we report AUC and TPR@5%FPR\. \(iv\)Score Aggregation\.To aggregate subset\-level scores into domain\-level scores for multi\-subset domains, domain\-level scores into stage\-level scores, and stage\-level scores into final scores, we use macro\-averaging, following[Duan et al\. \(2024\)](https://arxiv.org/html/2610.02986#bib.bib18)\.

### 5\.2Overall Performance

Unsupervised Methods\.AUCs of unsupervised MIAs onOLMo\-Detectare reported in Table[4](https://arxiv.org/html/2610.02986#S4.T4), with TPR@5%FPR results in Appendix[J](https://arxiv.org/html/2610.02986#A10)\. Our main findings are: \(i\)Overall performance is limited\.The best method, Zlib, reaches an overall AUC of only 0\.68, while the average across methods is 0\.58\. This suggests that, under distribution\-matched settings with minimal confounders, membership signals are substantially harder to detect than existing benchmarks suggest\. Notably, Zlib, one of the earliest and simplest MIAs, outperforms every more sophisticated method\. \(ii\)Likelihood\-based methods outperform other families\.Across families, likelihood\-based MIAs generally perform best, with one exception: ReCaLL\. ReCaLL assumes that a non\-member prefix disrupts predictions on members more than on non\-members, but this effect is inverted on GSM8K, CoT\-GSM8K, and CoT\-MATH \(AUC 0\.10, 0\.09, and 0\.45; Tables[41](https://arxiv.org/html/2610.02986#A9.T41)and[49](https://arxiv.org/html/2610.02986#A9.T49)\): their rigid format turns the prefix into an in\-context demonstration, improving member predictions instead of disrupting them\. \(iii\)Token\-level normalization hurts performance\.MinK\+\+, which applies token\-level vocabulary normalization to MinK, performs markedly worse \(0\.56 vs\. 0\.66\), suggesting that fine\-grained normalization removes part of the membership signal under distribution\-matched settings\.

MethodSetting1B7B13B32BModel\-AvgPreMidPostAllPreMidPostAllPreMidPostAllPreMidPostAllPreMidPostAllSup\-CAMIAIn\-domain0\.560\.720\.610\.630\.540\.780\.720\.680\.550\.860\.720\.710\.550\.840\.670\.690\.550\.800\.680\.68LODO0\.560\.630\.520\.570\.600\.730\.620\.650\.570\.660\.550\.590\.590\.700\.620\.640\.580\.680\.580\.61MIA\-TunerIn\-domain0\.540\.600\.530\.560\.530\.610\.550\.560\.540\.600\.530\.560\.540\.530\.510\.520\.540\.590\.530\.55LODO0\.520\.500\.470\.500\.530\.480\.460\.490\.470\.480\.460\.470\.510\.530\.500\.520\.510\.500\.470\.49

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.65\+\+0\.00−\-0\.03\+\+0\.01−\-0\.03−\-0\.01−\-0\.04−\-0\.02−\-0\.02−\-0\.05−\-0\.03FSD \(Zlib\)0\.68−\-0\.04−\-0\.04−\-0\.02−\-0\.07−\-0\.04−\-0\.07−\-0\.03−\-0\.04−\-0\.07−\-0\.06FSD \(Lower\)0\.63−\-0\.03−\-0\.02−\-0\.03−\-0\.05−\-0\.03−\-0\.04−\-0\.02−\-0\.02−\-0\.05−\-0\.03FSD \(MinK\)0\.66−\-0\.05−\-0\.12−\-0\.07−\-0\.09−\-0\.08−\-0\.06−\-0\.03−\-0\.03−\-0\.06−\-0\.05

Table 5:Supervised MIAs onOLMo\-DetectunderIn\-domainandLODOsettings\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner; the best method per column is in bold\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members; we therefore report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\), averaged across stages\.Base AUC: base MIA’s AUC averaged across model sizes; positive gains are bolded\. Details are in Appendix[K](https://arxiv.org/html/2610.02986#A11)\.Supervised Methods\.We evaluate supervised MIAs in two settings: \(i\)In\-domain, trained on the evaluated domain’s dev set, which measures the gain from same\-distribution supervision over unsupervised MIAs; and \(ii\)LODO\(leave\-one\-domain\-out\), trained on the other eight domains’ dev sets and evaluated on the held\-out domain, which measures cross\-domain transfer\. AUCs of supervised MIAs onOLMo\-Detectare reported in Table[5](https://arxiv.org/html/2610.02986#S5.T5), with TPR@5%FPR results in Appendix[L](https://arxiv.org/html/2610.02986#A12)\. Our main findings are: \(i\)In\-domain supervision adds little over unsupervised methods\.Under theIn\-domainsetting, the best supervised method, Sup\-CAMIA, reaches an overall AUC of 0\.68, matching the best unsupervised MIA, Zlib \(0\.68\), and exceeding its unsupervised counterpart CAMIA \(0\.65\) by only 0\.03; MIA\-Tuner remains near chance \(0\.55\), and FSD lowers the AUC of every base MIA on average\. This indicates that matched\-distribution labels add little beyond unsupervised membership signals\. To rule out the small dev set as a confounder, we hold out 50% of each domain for evaluation and vary the supervision budget from 10% to 50% \(Appendix[M](https://arxiv.org/html/2610.02986#A13), Table[83](https://arxiv.org/html/2610.02986#A13.T83)\)\. We find that increasing the budget improves Sup\-CAMIA by only 0\.04–0\.06 AUC, confirming that the limited performance is not an artifact of the labeled\-data budget\. \(ii\)Learned signals do not transfer well across domains\.Compared withIn\-domain, all supervised MIAs perform worse or no better underLODO: Sup\-CAMIA drops to 0\.61 \(−0\.07\-0\.07\), MIA\-Tuner to 0\.49 \(−0\.06\-0\.06\), and FSD’s average gain over its base MIAs remains−0\.04\-0\.04\. This suggests that supervised MIAs largely learn domain\-specific statistics rather than transferrable membership signals\.

Domain\-Unknown Setting\.To simulate a real\-world setting in which an auditor is unaware of a text’s domain, we compute a single AUC over members and non\-members pooled with equal instance counts across all domains, retraining supervised MIAs on the pooled dev sets\. A full discussion is provided in Appendix[N](https://arxiv.org/html/2610.02986#A14)\. In summary, unsupervised and supervised MIAs degrade slightly, by 0\.02 and 0\.03 AUC respectively, with the supervised drop concentrated in Sup\-CAMIA \(−0\.06\-0\.06\)\. The relative ranking is largely preserved \(Spearmanρ=0\.95\\rho=0\.95over all 17 MIAs in Table[84](https://arxiv.org/html/2610.02986#A13.T84)\)\.

### 5\.3Performance Across Stages and Model Sizes

Cross\-Stage Performance\.As shown in theModel\-Avgcolumns of Tables[4](https://arxiv.org/html/2610.02986#S4.T4)and[5](https://arxiv.org/html/2610.02986#S5.T5), overall MIA performance peaks at mid\-training and is considerably lower at post\-training and pre\-training\. However, this pattern reflects data type rather than a stage effect: curated math data, i\.e\., instances drawn from purpose\-built math datasets rather than harvested from the web, is far more detectable than any other data type\. As the top half of Table[6](https://arxiv.org/html/2610.02986#S5.T6)shows, the three curated math subsets, GSM8K, CoT\-GSM8K, and CoT\-MATH, achieve the highest AUCs across all MIAs, and their advantage is even more pronounced among effective MIAs \(overall AUC≥0\.60\\geq 0\.60\), where all three exceed 0\.70 and CoT\-GSM8K reaches 0\.90\. Notably, the effect is specific to curated math data rather than to mathematical content in general: Math Stack Exchange \(free\-form forum math Q&A\) and OpenWebMath \(web math text\) score only 0\.61 and 0\.57 among effective MIAs, in line with non\-math subsets\. To test this difference statistically, we conduct a one\-sided Wilcoxon signed\-rank test withα=0\.01\\alpha=0\.01on the paired per\-MIA AUC differences between the curated math subsets and the others\. As the bottom half of Table[6](https://arxiv.org/html/2610.02986#S5.T6)shows, curated math subsets are significantly more detectable under both the all\-MIA \(p=0\.008p=0\.008\) and the effective\-MIA \(p=0\.004p=0\.004\) setting, and once they are excluded, the spread across stages shrinks substantially \(0\.10→0\.050\.10\\to 0\.05in the all\-MIA and0\.17→0\.060\.17\\to 0\.06in the effective\-MIA setting\), indicating that stage ordering is largely a composition effect\. Finally, we examine duplication count, i\.e\., how many times each member is seen during training, as a potential confounder, and find that it does not explain the observed pattern: the most duplicated domain, DCLM, is less detectable than CoT\-MATH, which has a duplication count of only 1\.5\.

Cross\-Model\-Size Performance\.TheMethod\-Avgrow of Table[4](https://arxiv.org/html/2610.02986#S4.T4)shows that the average AUC of unsupervised MIAs increases with model size, from 0\.56 at 1B to 0\.58 at 7B and 0\.60 at 13B, but reaches only 0\.61 at 32B\. This trend is more pronounced for effective unsupervised MIAs \(overall AUC≥0\.60\\geq 0\.60\): their average AUC rises from 0\.61 at 1B to 0\.68 at 13B, but gains only 0\.01 more at 32B\. Supervised methods \(Table[5](https://arxiv.org/html/2610.02986#S5.T5)\) even decline at 32B under the In\-domain setting: Sup\-CAMIA rises from 0\.63 at 1B to 0\.71 at 13B before falling to 0\.69 at 32B, while MIA\-Tuner remains at 0\.56 through 13B and drops to 0\.52 at 32B\. The increase from 1B to 13B is consistent with MIMIR\([Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18)\), which attributed a similar trend in Pythia models to greater overfitting\. Beyond 13B, gains diminish and In\-domain supervised performance declines, echoing[Hayes et al\. \(2026\)](https://arxiv.org/html/2610.02986#bib.bib68), who report that MIA AUC does not increase monotonically with model size\.

Pre\-trainingMid\-trainingPost\-trainingDCLMpeS2oOWMStarCoderMath MixSESFTDPORLVRPythonJavaGSM8KMath SESOAyaWildChatUF\-WildChatCoT\-GSM8KCoT\-MATHAUC\(all MIAs\)0\.560\.520\.540\.560\.590\.710\.590\.600\.570\.600\.500\.700\.63AUC\(eff\. MIAs\)0\.620\.500\.570\.590\.640\.880\.610\.610\.590\.620\.500\.900\.74Dup\. count9\.001\.121\.001\.001\.006\.006\.006\.002\.092\.001\.131\.251\.50

Stage \(all subsets\)Stage \(w/o curated math\)Curated math vs\. othersPreMidPostPreMidPostCurated mathOthersΔ\\Delta\(\#MIAs,pp\)AUC\(all MIAs\)0\.550\.650\.590\.550\.590\.540\.690\.56\+0\.12\+0\.12\(13/16, 0\.008\)AUC\(eff\. MIAs\)0\.580\.750\.650\.570\.610\.550\.860\.59\+0\.27\+0\.27\(8/8, 0\.004\)

Table 6:Cross\-stage analysis onOLMo\-Detect\.Top:Per\-subset AUCs averaged across model sizes and over all MIAs \(excluding FSD, which reports gain, and SelfCrit, which is evaluated on DPO/RLVR\) oreffective \(eff\.\)MIAs \(overall AUC≥0\.60\\geq 0\.60\); bold means AUC\>0\.70\>0\.70\.Dup\. count: training\-corpus occurrences per member in each subset\. DCLM: DCLM\-Baseline; OWM: OpenWebMath; SE: Stack Exchange; SO: Stack Overflow\.Bottom:Stage\-level AUCs with and without the three curated math subsets \(GSM8K, CoT\-GSM8K, CoT\-MATH\), and AUCs on the curated math subsets versus the others\.Δ\\Delta: average per\-MIA AUC gain on curated math subsets over the others; parentheses report the number of MIAs with positive gains and the one\-sided Wilcoxonpp\-value\.
### 5\.4Robustness to Distribution Shifts

To test the robustness of unsupervised MIAs to distribution shifts, we compare their per\-domain AUCs onOLMo\-Detect \(Shifted\)andOLMo\-Detect, testing each difference using a paired instance\-level bootstrap \(Table[7](https://arxiv.org/html/2610.02986#S5.T7)\)\. Our main findings are: \(i\)No unsupervised MIA is robust\.Every MIA’s AUC changes significantly on three to six of the seven domains, with each MIA’s largest change ranging from 0\.09 to 0\.42 \(mean 0\.22\); the largest changes concentrate in pre\-training domains\. \(ii\)Whether distribution shifts inflate or degrade AUC follows no consistent pattern\.For all methods except ReCaLL and CDD, whose significant AUC changes are all inflations, there is no consistent shifting pattern across method–domain pairs: OpenWebMath inflates PPL’s AUC by 0\.29 but degrades Zlib’s by 0\.31; peS2o inflates CAMIA’s AUC by 0\.28 yet degrades EMMIA’s by 0\.17 \(Table[85](https://arxiv.org/html/2610.02986#A15.T85)\)\. Thus, shifted benchmarks distort AUC unpredictably\. \(iii\)Likelihood\-based MIAs are most affected\.Eight of the ten likelihood\-based MIAs change by 0\.20–0\.42 AUC, while the methods from other families \(Neigh\., Guided, DCQ, and CDD\) change by at most 0\.12\.

PPLZlibLowerMinKMinK\+\+DCPDDReCaLLCAMIAPACNeigh\.EMMIADCQCDDGuidedI / D4 / 22 / 25 / 12 / 11 / 32 / 34 / 04 / 15 / 15 / 13 / 22 / 13 / 01 / 2LargestΔ\\Delta\+\+0\.29−\-0\.31−\-0\.30\+\+0\.23−\-0\.16\+\+0\.16\+\+0\.24\+\+0\.28−\-0\.20\+\+0\.12\+\+0\.42\+\+0\.09\+\+0\.12\+\+0\.10\(OWM\)\(OWM\)\(OWM\)\(OWM\)\(SC\)\(SC\)\(peS2o\)\(peS2o\)\(OWM\)\(SC\)\(OWM\)\(peS2o\)\(OWM\)\(SC\)

Table 7:Effect of distribution shift on unsupervised MIAs \(excluding SelfCrit\), over the seven domains with shifted variants\.I / D: number of domains on which the AUC increases / decreases significantly under the shift \(Δ\\Delta=OLMo\-Detect \(Shifted\)−\-OLMo\-Detect, averaged over model sizes; 95% paired instance bootstrap\)\.LargestΔ\\Delta: the largest change and the domain where it occurs\. OWM: OpenWebMath; SC: StarCoder\. Per\-domain values are in Appendix[O](https://arxiv.org/html/2610.02986#A15)\.DomainComparisonPPLZlibMinKDCPDDPACAvgGSM8KOLMo 2\-7B / OLMo 3\-7B0\.85 / 0\.970\.89 / 0\.980\.86 / 0\.970\.91 / 0\.980\.83 / 0\.950\.87 / 0\.97OLMo 2\-32B / OLMo 3\-32B0\.98 / 0\.990\.99 / 1\.000\.97 / 0\.990\.98 / 0\.980\.92 / 0\.980\.97 / 0\.99DCLM\-BaselineOLMo 2\-1B / DCB\-1B0\.60 / 0\.630\.59 / 0\.610\.57 / 0\.600\.57 / 0\.600\.61 / 0\.630\.59 / 0\.61OLMo 2\-1B / SmolLM2\-1\.7B0\.60 / 0\.620\.59 / 0\.620\.57 / 0\.590\.57 / 0\.610\.61 / 0\.640\.59 / 0\.62OpenWebMathOLMo 2\-1B / DCB\-1B0\.51 / 0\.510\.56 / 0\.540\.55 / 0\.540\.61 / 0\.580\.54 / 0\.560\.55 / 0\.54OLMo 2\-7B / DCB\-7B0\.51 / 0\.530\.57 / 0\.560\.55 / 0\.550\.57 / 0\.590\.56 / 0\.570\.55 / 0\.56

Table 8:AUCs of the five best\-performing unsupervised MIAs on OLMo 2 and other model families\.
### 5\.5Generalization Beyond OLMo 2

To show that our findings are not artifacts of OLMo 2, we evaluate the five best\-performing unsupervised MIAs \(Zlib, DCPDD, MinK, PAC, and PPL\) on OLMo 3\([Olmo et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib4)\)and two non\-OLMo model families, comparing them with size\-matched OLMo 2 models\. \(i\)OLMo 3\.The 7B and 32B instruction\-following OLMo 3 models share the GSM8K subset ofOLMo\-Detect’s Math Mix domain with OLMo 2, which occurs at mid\-training in both families\. We further verify thatOLMo\-Detect’s GSM8K members are included in OLMo 3’s training data while its non\-members are excluded\. As shown in Table[8](https://arxiv.org/html/2610.02986#S5.T8), GSM8K remains highly detectable on OLMo 3 \(AUC 0\.95–1\.00\), with a method ranking similar to that on OLMo 2, PAC being the weakest in both families\. While both families approach an AUC of 1\.0 at 32B, OLMo 3 is more detectable at 7B, likely due to OLMo 3’s stronger memorization capability\. \(ii\)Non\-OLMo Models\.Eligible non\-OLMo models must share training data with anOLMo\-Detectdomain, whose non\-members must either be excluded from the model’s training data by construction based on clear documentation or be verifiable as absent, which requires open access to its training corpus\. To the best of our knowledge, only two model families satisfy these strict conditions: DCLM\-Baseline\-1B/7B \(DCB\-1B/7B\)\([Li et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib38)\)and SmolLM2\-1\.7B\([Allal et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib67)\)\. DCB\-1B is trained on the full DCLM\-Baseline corpus and OpenWebMath, covering twoOLMo\-Detectdomains, whereas DCB\-7B is trained on an undisclosed subset of DCLM\-Baseline, leaving only OpenWebMath valid for evaluation; SmolLM2\-1\.7B is trained on the full DCLM\-Baseline corpus, covering the DCLM domain\. Conveniently, the entire training data of DCB\-1B/7B is a subset of OLMo 2’s pre\-training data, so the non\-members ofOLMo\-Detect’s DCLM and OpenWebMath domains are clean for these models by construction; for SmolLM2, we apply the same 13\-gram filter used in benchmark construction to each of its remaining training corpora, all of which are publicly released, and score both models on the non\-members that are verified to be clean \(Appendix[P](https://arxiv.org/html/2610.02986#A16)\)\. As shown in Table[8](https://arxiv.org/html/2610.02986#S5.T8), all three models closely track their size\-matched OLMo 2 counterparts: every method is within 0\.04 AUC, and both domains remain weakly detectable \(AUC 0\.51–0\.64\)\. Method rankings also transfer for all three models, with PAC performing best on DCLM and DCPDD best on OpenWebMath\.

## 6Conclusion

We introducedOLMo\-Detect, the first benchmark for membership inference on LLMs that spans pre\-training, mid\-training, and post\-training, explicitly aligns members and non\-members on three key axes, and rigorously filters non\-members against the training corpus\. After extensive evaluation, we find that existing MIAs remain limited, with the best MIA reaching an AUC of only 0\.68\. Moreover, curated math data is substantially more detectable than any other data type\. Importantly, the main patterns generalize to OLMo 3 and non\-OLMo models, showing that our findings are not artifacts of OLMo 2\. We hopeOLMo\-Detectserves as a rigorous testbed for future MIAs\.

### AI use statement

In this work, we used generative AI tools for the following purposes:

1. 1\.To aid or polish writing\.We used generative AI tools to refine wording, grammar, and clarity and to check the consistency of terminology and citation formatting\. All AI\-suggested edits were reviewed by the authors, who take full responsibility for the final text\.
2. 2\.For research ideation or execution\.We used generative AI tools to refine and double\-check our implementations of the 18 compared MIA methods\. All AI\-refined components were verified and tested for correctness by the authors before use in our experiments\. Additionally, we used generative AI tools to provide feedback on the benchmark construction methodology and experiments\. All feedback was critically evaluated by the authors, who made all final methodological and experimental decisions\.
3. 3\.To draft sections of the paper\.We used generative AI tools to draft Appendix[C](https://arxiv.org/html/2610.02986#A3)\(Detailed Descriptions of OLMo 2 Training Data\), which catalogs extensive factual details about OLMo 2’s publicly documented training pipeline\. The authors checked all factual claims in the draft against OLMo 2’s released documentation and wrote the final version themselves based on the draft\.

We have not used generative AI tools to generate synthetic datasets, help develop theoretical models or conceptual frameworks, propose or refine hypotheses, clean and reformat datasets, support qualitative and thematic data analysis, or interpret results\. Formulating mathematical claims, providing critical ingredients for proving mathematical claims, assisting in the writing of proofs, and assisting with translation are not applicable to this work\.

We take responsibility for the final content of this work, including text, claims, and artifacts produced with the aid of generative AI\.

### Ethics Statement

OLMo\-Detectis constructed entirely from data that OLMo 2 was trained on \(members\) or data drawn from the same source distributions \(non\-members\)\. We do not scrape or introduce data from sources beyond those in OLMo 2’s publicly documented training pipeline\. We do not perform additional PII screening ourselves, but two domains inherit filtering from their sources\([OLMo et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib3)\): peS2o applies regex\-based PII detection and removes documents containing substantial PII\([Soldaini et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib24)\); SFT includes WildChat, which anonymizes names, contact information, and other identifiers through a dedicated pipeline before release\([Zhao et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib57)\)\.

### Reproducibility statement

Benchmark Construction\.Benchmark construction is described in Sections[4\.3](https://arxiv.org/html/2610.02986#S4.SS3)–[4\.7](https://arxiv.org/html/2610.02986#S4.SS7), with detailed constructions of non\-members and members provided in Appendices[E](https://arxiv.org/html/2610.02986#A5)and[F](https://arxiv.org/html/2610.02986#A6), respectively\. Overall statistics forOLMo\-Detectare reported in Table[2](https://arxiv.org/html/2610.02986#S4.T2), with detailed per\-domain statistics provided in Appendix[D](https://arxiv.org/html/2610.02986#A4)\. Statistics forOLMo\-Detect \(Shifted\)are provided in Appendix[G](https://arxiv.org/html/2610.02986#A7)\.

Method Implementation and Evaluation\.The overall experimental settings, including evaluated models, compared methods, and evaluation metrics, are described in Section[5](https://arxiv.org/html/2610.02986#S5)\. Detailed implementation information is provided in Appendix[H](https://arxiv.org/html/2610.02986#A8), including \(i\) each method’s original public repository, tuned hyperparameters, and search spaces; \(ii\) determinism and statistical reporting; \(iii\) continuous scoring for categorical methods; and \(iv\) reasons for excluding certain existing MIAs\.

## References

- Allalet al\.\(2025\)L\. B\. Allal, A\. Lozhkov, E\. Bakouch, G\. M\. Blázquez, G\. Penedo, L\. Tunstall, A\. Marafioti, H\. Kydlíček, A\. P\. Lajarín, V\. Srivastav, J\. Lochner, C\. Fahlgren, X\. Nguyen, C\. Fourrier, B\. Burtenshaw, H\. Larcher, H\. Zhao, C\. Zakka, M\. Morlon, C\. Raffel, L\. von Werra, and T\. WolfSmolLM2: when smol goes big – data\-centric training of a small language model\.External Links:2502\.02737,[Link](https://arxiv.org/abs/2502.02737)Cited by:[§1](https://arxiv.org/html/2610.02986#S1.p4.1),[§5\.5](https://arxiv.org/html/2610.02986#S5.SS5.p1.1)\.
- Azerbayevet al\.\(2023\)Z\. Azerbayev, H\. Schoelkopf, K\. Paster, M\. D\. Santos, S\. McAleer, A\. Q\. Jiang, J\. Deng, S\. Biderman, and S\. WelleckLlemma: an open language model for mathematics\.arXiv preprint arXiv:2310\.10631\.Cited by:[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1)\.
- Bidermanet al\.\(2024\)D\. Biderman, J\. Portes, J\. J\. G\. Ortiz, M\. Paul, P\. Greengard, C\. Jennings, D\. King, S\. Havens, V\. Chiley, J\. Frankle, C\. Blakeney, and J\. P\. CunninghamLoRA learns less and forgets less\.External Links:2405\.09673,[Link](https://arxiv.org/abs/2405.09673)Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px4.p1.1)\.
- Bidermanet al\.\(2023\)S\. Biderman, H\. Schoelkopf, Q\. Anthony, H\. Bradley, K\. O’Brien, E\. Hallahan, M\. A\. Khan, S\. Purohit, U\. S\. Prashanth, E\. Raff, A\. Skowron, L\. Sutawika, and O\. van der WalPythia: a suite for analyzing large language models across training and scaling\.External Links:2304\.01373,[Link](https://arxiv.org/abs/2304.01373)Cited by:[§4\.1](https://arxiv.org/html/2610.02986#S4.SS1.p1.1)\.
- Blacket al\.\(2022\)S\. Black, S\. Biderman, E\. Hallahan, Q\. Anthony, L\. Gao, L\. Golding, H\. He, C\. Leahy, K\. McDonell, J\. Phang, M\. Pieler, U\. S\. Prashanth, S\. Purohit, L\. Reynolds, J\. Tow, B\. Wang, and S\. WeinbachGPT\-NeoX\-20B: an open\-source autoregressive language model\.InProceedings of BigScience Episode \#5 – Workshop on Challenges & Perspectives in Creating Large Language Models,A\. Fan, S\. Ilic, T\. Wolf, and M\. Gallé \(Eds\.\),virtual\+Dublin,pp\. 95–136\.External Links:[Link](https://aclanthology.org/2022.bigscience-1.9/),[Document](https://dx.doi.org/10.18653/v1/2022.bigscience-1.9)Cited by:[§4\.1](https://arxiv.org/html/2610.02986#S4.SS1.p1.1)\.
- Carliniet al\.\(2021\)N\. Carlini, F\. Tramer, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, U\. Erlingsson, A\. Oprea, and C\. RaffelExtracting training data from large language models\.External Links:2012\.07805,[Link](https://arxiv.org/abs/2012.07805)Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Appendix H](https://arxiv.org/html/2610.02986#A8.SS0.SSS0.Px4.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.3.2),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.4.2),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§1](https://arxiv.org/html/2610.02986#S1.p4.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Changet al\.\(2025\)H\. Chang, A\. S\. Shamsabadi, K\. Katevas, H\. Haddadi, and R\. ShokriContext\-aware membership inference attacks against pre\-trained large language models\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 7299–7321\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px2.SPx1.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.19.2),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.9.2),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§1](https://arxiv.org/html/2610.02986#S1.p4.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Changet al\.\(2023\)K\. Chang, M\. Cramer, S\. Soni, and D\. BammanSpeak, memory: an archaeology of books known to chatgpt/gpt\-4\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 7312–7327\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx3.p1.1),[Appendix H](https://arxiv.org/html/2610.02986#A8.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.External Links:2107\.03374,[Link](https://arxiv.org/abs/2107.03374)Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px4.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p4.1)\.
- Cuiet al\.\(2024\)G\. Cui, L\. Yuan, N\. Ding, G\. Yao, B\. He, W\. Zhu, Y\. Ni, G\. Xie, R\. Xie, Y\. Lin, Z\. Liu, and M\. SunUltraFeedback: boosting language models with scaled ai feedback\.External Links:2310\.01377,[Link](https://arxiv.org/abs/2310.01377)Cited by:[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.SSS0.Px2.p1.1)\.
- Daset al\.\(2025\)D\. Das, J\. Zhang, and F\. TramèrBlind baselines beat membership inference attacks for foundation models\.In2025 IEEE Security and Privacy Workshops \(SPW\),pp\. 118–125\.Cited by:[§4\.6](https://arxiv.org/html/2610.02986#S4.SS6.p2.1)\.
- Denget al\.\(2024\)C\. Deng, Y\. Zhao, X\. Tang, M\. Gerstein, and A\. CohanInvestigating data contamination in modern benchmarks for large language models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 8706–8719\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx3.p1.1),[Appendix H](https://arxiv.org/html/2610.02986#A8.SS0.SSS0.Px4.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1)\.
- Donget al\.\(2024\)Y\. Dong, X\. Jiang, H\. Liu, Z\. Jin, B\. Gu, M\. Yang, and G\. LiGeneralization or memorization: data contamination and trustworthy evaluation for large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 12039–12050\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px4.p1.1),[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx4.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.14.2),[Table 1](https://arxiv.org/html/2610.02986#S1.T1.2.1.5.1),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Duanet al\.\(2024\)M\. Duan, A\. Suri, N\. Mireshghallah, S\. Min, W\. Shi, L\. Zettlemoyer, Y\. Tsvetkov, Y\. Choi, D\. Evans, and H\. HajishirziDo membership inference attacks work on large language models?\.arXiv preprint arXiv:2402\.07841\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2610.02986#S1.T1.2.1.6.1),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p1.1),[§4\.5](https://arxiv.org/html/2610.02986#S4.SS5.p3.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1),[§5\.3](https://arxiv.org/html/2610.02986#S5.SS3.p2.1)\.
- Fuet al\.\(2024\)W\. Fu, H\. Wang, C\. Gao, G\. Liu, Y\. Li, and T\. JiangMembership inference attacks against fine\-tuned large language models via self\-prompt calibration\.Advances in Neural Information Processing Systems37,pp\. 134981–135010\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Appendix H](https://arxiv.org/html/2610.02986#A8.SS0.SSS0.Px4.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1)\.
- Fuet al\.\(2025\)W\. Fu, H\. Wang, C\. Gao, G\. Liu, Y\. Li, and T\. JiangMia\-tuner: adapting large language models as pre\-training text detector\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 27295–27303\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px1.p1.1),[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px2.SPx2.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.18.2),[Table 1](https://arxiv.org/html/2610.02986#S1.T1.2.1.8.1),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Gaoet al\.\(2020\)L\. Gao, S\. Biderman, S\. Black, L\. Golding, T\. Hoppe, C\. Foster, J\. Phang, H\. He, A\. Thite, N\. Nabeshima, S\. Presser, and C\. LeahyThe pile: an 800gb dataset of diverse text for language modeling\.External Links:2101\.00027,[Link](https://arxiv.org/abs/2101.00027)Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px2.p1.1)\.
- Golchin and Surdeanu \(2023\)S\. Golchin and M\. SurdeanuTime travel in llms: tracing data contamination in large language models\.arXiv preprint arXiv:2308\.08493\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx3.p1.1),[Appendix H](https://arxiv.org/html/2610.02986#A8.SS0.SSS0.Px3.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.15.2),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Golchin and Surdeanu \(2025\)S\. Golchin and M\. SurdeanuData contamination quiz: a tool to detect and estimate contamination in large language models\.Transactions of the Association for Computational Linguistics13,pp\. 809–830\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx3.p1.1),[Appendix H](https://arxiv.org/html/2610.02986#A8.SS0.SSS0.Px3.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.12.2),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. MaThe llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§1](https://arxiv.org/html/2610.02986#S1.p2.1)\.
- Groeneveldet al\.\(2024\)D\. Groeneveld, I\. Beltagy, E\. Walsh, A\. Bhagia, R\. Kinney, O\. Tafjord, A\. Jha, H\. Ivison, I\. Magnusson, Y\. Wang, S\. Arora, D\. Atkinson, R\. Authur, K\. Chandu, A\. Cohan, J\. Dumas, Y\. Elazar, Y\. Gu, J\. Hessel, T\. Khot, W\. Merrill, J\. Morrison, N\. Muennighoff, A\. Naik, C\. Nam, M\. Peters, V\. Pyatkin, A\. Ravichander, D\. Schwenk, S\. Shah, W\. Smith, E\. Strubell, N\. Subramani, M\. Wortsman, P\. Dasigi, N\. Lambert, K\. Richardson, L\. Zettlemoyer, J\. Dodge, K\. Lo, L\. Soldaini, N\. Smith, and H\. HajishirziOLMo: accelerating the science of language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15789–15809\.External Links:[Link](https://aclanthology.org/2024.acl-long.841/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.841)Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2610.02986#S4.SS1.p1.1)\.
- Guhaet al\.\(2025\)E\. Guha, R\. Marten, S\. Keh, N\. Raoof, G\. Smyrnis, H\. Bansal, M\. Nezhurina, J\. Mercat, T\. Vu, Z\. Sprague, A\. Suvarna, B\. Feuer, L\. Chen, Z\. Khan, E\. Frankel, S\. Grover, C\. Choi, N\. Muennighoff, S\. Su, W\. Zhao, J\. Yang, S\. Pimpalgaonkar, K\. Sharma, C\. C\. Ji, Y\. Deng, S\. Pratt, V\. Ramanujan, J\. Saad\-Falcon, J\. Li, A\. Dave, A\. Albalak, K\. Arora, B\. Wulfe, C\. Hegde, G\. Durrett, S\. Oh, M\. Bansal, S\. Gabriel, A\. Grover, K\. Chang, V\. Shankar, A\. Gokaslan, M\. A\. Merrill, T\. Hashimoto, Y\. Choi, J\. Jitsev, R\. Heckel, M\. Sathiamoorthy, A\. G\. Dimakis, and L\. SchmidtOpenThoughts: data recipes for reasoning models\.External Links:2506\.04178,[Link](https://arxiv.org/abs/2506.04178)Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px2.p1.1)\.
- Hayeset al\.\(2026\)J\. Hayes, I\. Shumailov, C\. A\. Choquette\-Choo, M\. Jagielski, G\. Kaissis, M\. Nasr, S\. Ghalebikesabi, M\. S\. M\. S\. Annamalai, N\. Mireshghallah, I\. Shilov, M\. Meeus, Y\. de Montjoye, K\. Lee, F\. Boenisch, A\. Dziedzic, and A\. F\. CooperExploring the limits of strong membership inference attacks on large language models\.External Links:2505\.18773,[Link](https://arxiv.org/abs/2505.18773)Cited by:[§5\.3](https://arxiv.org/html/2610.02986#S5.SS3.p2.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.SSS0.Px3.p1.1)\.
- Huet al\.\(2021\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.arXiv preprint arXiv:2106\.09685\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px4.p1.1)\.
- Ivisonet al\.\(2023\)H\. Ivison, Y\. Wang, V\. Pyatkin, N\. Lambert, M\. Peters, P\. Dasigi, J\. Jang, D\. Wadden, N\. A\. Smith, I\. Beltagy, and H\. HajishirziCamels in a changing climate: enhancing lm adaptation with tulu 2\.External Links:2311\.10702,[Link](https://arxiv.org/abs/2311.10702)Cited by:[§E\.1](https://arxiv.org/html/2610.02986#A5.SS1.SSS0.Px2.p4.1)\.
- Joulinet al\.\(2017\)A\. Joulin, E\. Grave, P\. Bojanowski, and T\. MikolovBag of tricks for efficient text classification\.InProceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers,pp\. 427–431\.Cited by:[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1)\.
- Kimet al\.\(2026\)G\. Kim, Y\. Li, E\. Spiliopoulou, J\. Ma, and W\. Y\. WangDetecting training data of large language models via expectation maximization\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1115–1129\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px3.p1.1),[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.11.2),[Table 1](https://arxiv.org/html/2610.02986#S1.T1.2.1.10.1),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Lambertet al\.\(2025\)N\. Lambert, J\. Morrison, V\. Pyatkin, S\. Huang, H\. Ivison, F\. Brahman, L\. J\. V\. Miranda, A\. Liu, N\. Dziri, S\. Lyu, Y\. Gu, S\. Malik, V\. Graf, J\. D\. Hwang, J\. Yang, R\. L\. Bras, O\. Tafjord, C\. Wilhelm, L\. Soldaini, N\. A\. Smith, Y\. Wang, P\. Dasigi, and H\. HajishirziTulu 3: pushing frontiers in open language model post\-training\.External Links:2411\.15124,[Link](https://arxiv.org/abs/2411.15124)Cited by:[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.SSS0.Px1.p1.1),[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.SSS0.Px3.p1.1),[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.p1.1),[Table 12](https://arxiv.org/html/2610.02986#A3.T12)\.
- Lecchini\-Visintiniet al\.\(2007\)A\. Lecchini\-Visintini, J\. Lygeros, and J\. MaciejowskiSimulated annealing: rigorous finite\-time guarantees for optimization on continuous domains\.Advances in neural information processing systems20\.Cited by:[§4\.5](https://arxiv.org/html/2610.02986#S4.SS5.p4.2)\.
- Leeet al\.\(2025\)C\. Lee, R\. Roy, M\. Xu, J\. Raiman, M\. Shoeybi, B\. Catanzaro, and W\. PingNv\-embed: improved techniques for training llms as generalist embedding models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 79310–79333\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2025\)J\. Li, A\. Fang, G\. Smyrnis, M\. Ivgi, M\. Jordan, S\. Gadre, H\. Bansal, E\. Guha, S\. Keh, K\. Arora, S\. Garg, R\. Xin, N\. Muennighoff, R\. Heckel, J\. Mercat, M\. Chen, S\. Gururangan, M\. Wortsman, A\. Albalak, Y\. Bitton, M\. Nezhurina, A\. Abbas, C\. Hsieh, D\. Ghosh, J\. Gardner, M\. Kilian, H\. Zhang, R\. Shao, S\. Pratt, S\. Sanyal, G\. Ilharco, G\. Daras, K\. Marathe, A\. Gokaslan, J\. Zhang, K\. Chandu, T\. Nguyen, I\. Vasiljevic, S\. Kakade, S\. Song, S\. Sanghavi, F\. Faghri, S\. Oh, L\. Zettlemoyer, K\. Lo, A\. El\-Nouby, H\. Pouransari, A\. Toshev, S\. Wang, D\. Groeneveld, L\. Soldaini, P\. W\. Koh, J\. Jitsev, T\. Kollar, A\. G\. Dimakis, Y\. Carmon, A\. Dave, L\. Schmidt, and V\. ShankarDataComp\-lm: in search of the next generation of training sets for language models\.External Links:2406\.11794,[Link](https://arxiv.org/abs/2406.11794)Cited by:[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p4.1),[§5\.5](https://arxiv.org/html/2610.02986#S5.SS5.p1.1)\.
- Liet al\.\(2023\)R\. Li, L\. B\. Allal, Y\. Zi, N\. Muennighoff, D\. Kocetkov, C\. Mou, M\. Marone, C\. Akiki, J\. Li, J\. Chim, Q\. Liu, E\. Zheltonozhskii, T\. Y\. Zhuo, T\. Wang, O\. Dehaene, M\. Davaadorj, J\. Lamy\-Poirier, J\. Monteiro, O\. Shliazhko, N\. Gontier, N\. Meade, A\. Zebaze, M\. Yee, L\. K\. Umapathi, J\. Zhu, B\. Lipkin, M\. Oblokulov, Z\. Wang, R\. Murthy, J\. Stillerman, S\. S\. Patel, D\. Abulkhanov, M\. Zocca, M\. Dey, Z\. Zhang, N\. Fahmy, U\. Bhattacharyya, W\. Yu, S\. Singh, S\. Luccioni, P\. Villegas, M\. Kunakov, F\. Zhdanov, M\. Romero, T\. Lee, N\. Timor, J\. Ding, C\. Schlesinger, H\. Schoelkopf, J\. Ebert, T\. Dao, M\. Mishra, A\. Gu, J\. Robinson, C\. J\. Anderson, B\. Dolan\-Gavitt, D\. Contractor, S\. Reddy, D\. Fried, D\. Bahdanau, Y\. Jernite, C\. M\. Ferrandis, S\. Hughes, T\. Wolf, A\. Guha, L\. von Werra, and H\. de VriesStarCoder: may the source be with you\!\.External Links:2305\.06161,[Link](https://arxiv.org/abs/2305.06161)Cited by:[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1),[§E\.1](https://arxiv.org/html/2610.02986#A5.SS1.SSS0.Px2.p2.1)\.
- Lin \(2004\)C\. LinROUGE: a package for automatic evaluation of summaries\.InText Summarization Branches Out,Barcelona, Spain,pp\. 74–81\.External Links:[Link](https://aclanthology.org/W04-1013/)Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx3.p1.1)\.
- Liuet al\.\(2025\)H\. Liu, J\. Li, H\. Zhu, K\. Zhang, Y\. Dong, and G\. LiSaturn: sat\-based reinforcement learning to unleash language model reasoning\.arXiv e\-prints,pp\. arXiv–2505\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2024\)J\. Liu, S\. Min, L\. Zettlemoyer, Y\. Choi, and H\. HajishirziInfini\-gram: scaling unbounded n\-gram language models to a trillion tokens\.arXiv preprint arXiv:2401\.17377\.Cited by:[§1](https://arxiv.org/html/2610.02986#S1.p3.1),[§4\.4](https://arxiv.org/html/2610.02986#S4.SS4.p4.1)\.
- Matternet al\.\(2023\)J\. Mattern, F\. Mireshghallah, Z\. Jin, B\. Schölkopf, M\. Sachan, and T\. Berg\-KirkpatrickMembership inference attacks against language models via neighbourhood comparison\.InFindings of the Association for Computational Linguistics: ACL 2023,pp\. 11330–11343\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx2.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.13.2),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Olmoet al\.\(2026\)T\. Olmo, :, A\. Ettinger, A\. Bertsch, B\. Kuehl, D\. Graham, D\. Heineman, D\. Groeneveld, F\. Brahman, F\. Timbers, H\. Ivison, J\. Morrison, J\. Poznanski, K\. Lo, L\. Soldaini, M\. Jordan, M\. Chen, M\. Noukhovitch, N\. Lambert, P\. Walsh, P\. Dasigi, R\. Berry, S\. Malik, S\. Shah, S\. Geng, S\. Arora, S\. Gupta, T\. Anderson, T\. Xiao, T\. Murray, T\. Romero, V\. Graf, A\. Asai, A\. Bhagia, A\. Wettig, A\. Liu, A\. Rangapur, C\. Anastasiades, C\. Huang, D\. Schwenk, H\. Trivedi, I\. Magnusson, J\. Lochner, J\. Liu, L\. J\. V\. Miranda, M\. Sap, M\. Morgan, M\. Schmitz, M\. Guerquin, M\. Wilson, R\. Huff, R\. L\. Bras, R\. Xin, R\. Shao, S\. Skjonsberg, S\. Z\. Shen, S\. S\. Li, T\. Wilde, V\. Pyatkin, W\. Merrill, Y\. Chang, Y\. Gu, Z\. Zeng, A\. Sabharwal, L\. Zettlemoyer, P\. W\. Koh, A\. Farhadi, N\. A\. Smith, and H\. HajishirziOlmo 3\.External Links:2512\.13961,[Link](https://arxiv.org/abs/2512.13961)Cited by:[§1](https://arxiv.org/html/2610.02986#S1.p4.1),[§4\.1](https://arxiv.org/html/2610.02986#S4.SS1.p1.1),[§5\.5](https://arxiv.org/html/2610.02986#S5.SS5.p1.1)\.
- OLMoet al\.\(2025\)T\. OLMo, P\. Walsh, L\. Soldaini, D\. Groeneveld, K\. Lo, S\. Arora, A\. Bhagia, Y\. Gu, S\. Huang, M\. Jordan, N\. Lambert, D\. Schwenk, O\. Tafjord, T\. Anderson, D\. Atkinson, F\. Brahman, C\. Clark, P\. Dasigi, N\. Dziri, A\. Ettinger, M\. Guerquin, D\. Heineman, H\. Ivison, P\. W\. Koh, J\. Liu, S\. Malik, W\. Merrill, L\. J\. V\. Miranda, J\. Morrison, T\. Murray, C\. Nam, J\. Poznanski, V\. Pyatkin, A\. Rangapur, M\. Schmitz, S\. Skjonsberg, D\. Wadden, C\. Wilhelm, M\. Wilson, L\. Zettlemoyer, A\. Farhadi, N\. A\. Smith, and H\. Hajishirzi2 olmo 2 furious\.External Links:2501\.00656,[Link](https://arxiv.org/abs/2501.00656)Cited by:[Table 10](https://arxiv.org/html/2610.02986#A3.T10),[Table 11](https://arxiv.org/html/2610.02986#A3.T11),[Table 13](https://arxiv.org/html/2610.02986#A3.T13),[Table 9](https://arxiv.org/html/2610.02986#A3.T9),[Appendix C](https://arxiv.org/html/2610.02986#A3.p1.1),[§E\.1](https://arxiv.org/html/2610.02986#A5.SS1.SSS0.Px1.p3.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§1](https://arxiv.org/html/2610.02986#S1.p3.1),[§4\.1](https://arxiv.org/html/2610.02986#S4.SS1.p1.1),[§6](https://arxiv.org/html/2610.02986#S6.SSx2.p1.1),[footnote 1](https://arxiv.org/html/2610.02986#footnote1)\.
- OpenAIet al\.\(2024\)OpenAI, :, A\. Hurst, A\. Lerer, A\. P\. Goucher, A\. Perelman, A\. Ramesh, A\. Clark, A\. Ostrow, A\. Welihinda, A\. Hayes, A\. Radford, A\. Mądry, A\. Baker\-Whitcomb, A\. Beutel, A\. Borzunov, A\. Carney, A\. Chow, A\. Kirillov, A\. Nichol, A\. Paino, A\. Renzin, A\. T\. Passos, A\. Kirillov, A\. Christakis, A\. Conneau, A\. Kamali, A\. Jabri, A\. Moyer, A\. Tam, A\. Crookes, A\. Tootoochian, A\. Tootoonchian, A\. Kumar, A\. Vallone, A\. Karpathy, A\. Braunstein, A\. Cann, A\. Codispoti, A\. Galu, A\. Kondrich, A\. Tulloch, A\. Mishchenko, A\. Baek, A\. Jiang, A\. Pelisse, A\. Woodford, A\. Gosalia, A\. Dhar, A\. Pantuliano, A\. Nayak, A\. Oliver, B\. Zoph, B\. Ghorbani, B\. Leimberger, B\. Rossen, B\. Sokolowsky, B\. Wang, B\. Zweig, B\. Hoover, B\. Samic, B\. McGrew, B\. Spero, B\. Giertler, B\. Cheng, B\. Lightcap, B\. Walkin, B\. Quinn, B\. Guarraci, B\. Hsu, B\. Kellogg, B\. Eastman, C\. Lugaresi, C\. Wainwright, C\. Bassin, C\. Hudson, C\. Chu, C\. Nelson, C\. Li, C\. J\. Shern, C\. Conger, C\. Barette, C\. Voss, C\. Ding, C\. Lu, C\. Zhang, C\. Beaumont, C\. Hallacy, C\. Koch, C\. Gibson, C\. Kim, C\. Choi, C\. McLeavey, C\. Hesse, C\. Fischer, C\. Winter, C\. Czarnecki, C\. Jarvis, C\. Wei, C\. Koumouzelis, D\. Sherburn, D\. Kappler, D\. Levin, D\. Levy, D\. Carr, D\. Farhi, D\. Mely, D\. Robinson, D\. Sasaki, D\. Jin, D\. Valladares, D\. Tsipras, D\. Li, D\. P\. Nguyen, D\. Findlay, E\. Oiwoh, E\. Wong, E\. Asdar, E\. Proehl, E\. Yang, E\. Antonow, E\. Kramer, E\. Peterson, E\. Sigler, E\. Wallace, E\. Brevdo, E\. Mays, F\. Khorasani, F\. P\. Such, F\. Raso, F\. Zhang, F\. von Lohmann, F\. Sulit, G\. Goh, G\. Oden, G\. Salmon, G\. Starace, G\. Brockman, H\. Salman, H\. Bao, H\. Hu, H\. Wong, H\. Wang, H\. Schmidt, H\. Whitney, H\. Jun, H\. Kirchner, H\. P\. de Oliveira Pinto, H\. Ren, H\. Chang, H\. W\. Chung, I\. Kivlichan, I\. O’Connell, I\. O’Connell, I\. Osband, I\. Silber, I\. Sohl, I\. Okuyucu, I\. Lan, I\. Kostrikov, I\. Sutskever, I\. Kanitscheider, I\. Gulrajani, J\. Coxon, J\. Menick, J\. Pachocki, J\. Aung, J\. Betker, J\. Crooks, J\. Lennon, J\. Kiros, J\. Leike, J\. Park, J\. Kwon, J\. Phang, J\. Teplitz, J\. Wei, J\. Wolfe, J\. Chen, J\. Harris, J\. Varavva, J\. G\. Lee, J\. Shieh, J\. Lin, J\. Yu, J\. Weng, J\. Tang, J\. Yu, J\. Jang, J\. Q\. Candela, J\. Beutler, J\. Landers, J\. Parish, J\. Heidecke, J\. Schulman, J\. Lachman, J\. McKay, J\. Uesato, J\. Ward, J\. W\. Kim, J\. Huizinga, J\. Sitkin, J\. Kraaijeveld, J\. Gross, J\. Kaplan, J\. Snyder, J\. Achiam, J\. Jiao, J\. Lee, J\. Zhuang, J\. Harriman, K\. Fricke, K\. Hayashi, K\. Singhal, K\. Shi, K\. Karthik, K\. Wood, K\. Rimbach, K\. Hsu, K\. Nguyen, K\. Gu\-Lemberg, K\. Button, K\. Liu, K\. Howe, K\. Muthukumar, K\. Luther, L\. Ahmad, L\. Kai, L\. Itow, L\. Workman, L\. Pathak, L\. Chen, L\. Jing, L\. Guy, L\. Fedus, L\. Zhou, L\. Mamitsuka, L\. Weng, L\. McCallum, L\. Held, L\. Ouyang, L\. Feuvrier, L\. Zhang, L\. Kondraciuk, L\. Kaiser, L\. Hewitt, L\. Metz, L\. Doshi, M\. Aflak, M\. Simens, M\. Boyd, M\. Thompson, M\. Dukhan, M\. Chen, M\. Gray, M\. Hudnall, M\. Zhang, M\. Aljubeh, M\. Litwin, M\. Zeng, M\. Johnson, M\. Shetty, M\. Gupta, M\. Shah, M\. Yatbaz, M\. J\. Yang, M\. Zhong, M\. Glaese, M\. Chen, M\. Janner, M\. Lampe, M\. Petrov, M\. Wu, M\. Wang, M\. Fradin, M\. Pokrass, M\. Castro, M\. O\. T\. de Castro, M\. Pavlov, M\. Brundage, M\. Wang, M\. Khan, M\. Murati, M\. Bavarian, M\. Lin, M\. Yesildal, N\. Soto, N\. Gimelshein, N\. Cone, N\. Staudacher, N\. Summers, N\. LaFontaine, N\. Chowdhury, N\. Ryder, N\. Stathas, N\. Turley, N\. Tezak, N\. Felix, N\. Kudige, N\. Keskar, N\. Deutsch, N\. Bundick, N\. Puckett, O\. Nachum, O\. Okelola, O\. Boiko, O\. Murk, O\. Jaffe, O\. Watkins, O\. Godement, O\. Campbell\-Moore, P\. Chao, P\. McMillan, P\. Belov, P\. Su, P\. Bak, P\. Bakkum, P\. Deng, P\. Dolan, P\. Hoeschele, P\. Welinder, P\. Tillet, P\. Pronin, P\. Tillet, P\. Dhariwal, Q\. Yuan, R\. Dias, R\. Lim, R\. Arora, R\. Troll, R\. Lin, R\. G\. Lopes, R\. Puri, R\. Miyara, R\. Leike, R\. Gaubert, R\. Zamani, R\. Wang, R\. Donnelly, R\. Honsby, R\. Smith, R\. Sahai, R\. Ramchandani, R\. Huet, R\. Carmichael, R\. Zellers, R\. Chen, R\. Chen, R\. Nigmatullin, R\. Cheu, S\. Jain, S\. Altman, S\. Schoenholz, S\. Toizer, S\. Miserendino, S\. Agarwal, S\. Culver, S\. Ethersmith, S\. Gray, S\. Grove, S\. Metzger, S\. Hermani, S\. Jain, S\. Zhao, S\. Wu, S\. Jomoto, S\. Wu, Shuaiqi, Xia, S\. Phene, S\. Papay, S\. Narayanan, S\. Coffey, S\. Lee, S\. Hall, S\. Balaji, T\. Broda, T\. Stramer, T\. Xu, T\. Gogineni, T\. Christianson, T\. Sanders, T\. Patwardhan, T\. Cunninghman, T\. Degry, T\. Dimson, T\. Raoux, T\. Shadwell, T\. Zheng, T\. Underwood, T\. Markov, T\. Sherbakov, T\. Rubin, T\. Stasi, T\. Kaftan, T\. Heywood, T\. Peterson, T\. Walters, T\. Eloundou, V\. Qi, V\. Moeller, V\. Monaco, V\. Kuo, V\. Fomenko, W\. Chang, W\. Zheng, W\. Zhou, W\. Manassra, W\. Sheu, W\. Zaremba, Y\. Patil, Y\. Qian, Y\. Kim, Y\. Cheng, Y\. Zhang, Y\. He, Y\. Zhang, Y\. Jin, Y\. Dai, and Y\. MalkovGPT\-4o system card\.External Links:2410\.21276,[Link](https://arxiv.org/abs/2410.21276)Cited by:[§C\.2](https://arxiv.org/html/2610.02986#A3.SS2.p1.1)\.
- Ouyanget al\.\(2022\)L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. Christiano, J\. Leike, and R\. LoweTraining language models to follow instructions with human feedback\.External Links:2203\.02155,[Link](https://arxiv.org/abs/2203.02155)Cited by:[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.SSS0.Px2.p1.1)\.
- Pasteret al\.\(2024\)K\. Paster, M\. Dos Santos, Z\. Azerbayev, and J\. BaOpenwebmath: an open dataset of high\-quality mathematical web text\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 20357–20379\.Cited by:[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1)\.
- Penedoet al\.\(2024\)G\. Penedo, H\. Kydlíček, L\. B\. allal, A\. Lozhkov, M\. Mitchell, C\. Raffel, L\. V\. Werra, and T\. WolfThe fineweb datasets: decanting the web for the finest text data at scale\.External Links:2406\.17557,[Link](https://arxiv.org/abs/2406.17557)Cited by:[§C\.2](https://arxiv.org/html/2610.02986#A3.SS2.p2.1)\.
- Penedoet al\.\(2023\)G\. Penedo, Q\. Malartic, D\. Hesslow, R\. Cojocaru, A\. Cappelli, H\. Alobeidli, B\. Pannier, E\. Almazrouei, and J\. LaunayThe refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only\.External Links:2306\.01116,[Link](https://arxiv.org/abs/2306.01116)Cited by:[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1)\.
- Qwenet al\.\(2025\)Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§C\.2](https://arxiv.org/html/2610.02986#A3.SS2.p1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.SSS0.Px2.p1.1)\.
- Schulmanet al\.\(2017\)J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. KlimovProximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.SSS0.Px3.p1.1)\.
- Sellamet al\.\(2020\)T\. Sellam, D\. Das, and A\. ParikhBLEURT: learning robust metrics for text generation\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 7881–7892\.External Links:[Link](https://aclanthology.org/2020.acl-main.704/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.704)Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx3.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§C\.3](https://arxiv.org/html/2610.02986#A3.SS3.SSS0.Px3.p1.1)\.
- Shiet al\.\(2023\)W\. Shi, A\. Ajith, M\. Xia, Y\. Huang, D\. Liu, T\. Blevins, D\. Chen, and L\. ZettlemoyerDetecting pretraining data from large language models\.arXiv preprint arXiv:2310\.16789\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px1.p1.1),[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.5.2),[Table 1](https://arxiv.org/html/2610.02986#S1.T1.2.1.3.1),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Soldainiet al\.\(2024\)L\. Soldaini, R\. Kinney, A\. Bhagia, D\. Schwenk, D\. Atkinson, R\. Authur, B\. Bogin, K\. Chandu, J\. Dumas, Y\. Elazar, V\. Hofmann, A\. Jha, S\. Kumar, L\. Lucy, X\. Lyu, N\. Lambert, I\. Magnusson, J\. Morrison, N\. Muennighoff, A\. Naik, C\. Nam, M\. Peters, A\. Ravichander, K\. Richardson, Z\. Shen, E\. Strubell, N\. Subramani, O\. Tafjord, E\. Walsh, L\. Zettlemoyer, N\. Smith, H\. Hajishirzi, I\. Beltagy, D\. Groeneveld, J\. Dodge, and K\. LoDolma: an open corpus of three trillion tokens for language model pretraining research\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15725–15788\.External Links:[Link](https://aclanthology.org/2024.acl-long.840/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.840)Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px2.p1.1),[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1),[§6](https://arxiv.org/html/2610.02986#S6.SSx2.p1.1)\.
- Soldaini and Lo \(2023\)L\. Soldaini and K\. LopeS2o \(Pretraining Efficiently on S2ORC\) Dataset\.Technical reportAllen Institute for AI\.Note:ODC\-By,[https://github\.com/allenai/pes2o](https://github.com/allenai/pes2o)Cited by:[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1)\.
- Taoet al\.\(2025\)Y\. Tao, T\. Wang, Y\. Dong, H\. Liu, K\. Zhang, X\. Hu, and G\. LiDetecting data contamination from reinforcement learning post\-training for large language models\.arXiv preprint arXiv:2510\.09259\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px2.p1.1),[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx4.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.16.2),[Table 1](https://arxiv.org/html/2610.02986#S1.T1.2.1.9.1),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Tuet al\.\(2025\)C\. Tu, X\. Zhang, R\. Weng, R\. Li, C\. Zhang, Y\. Bai, H\. Yan, J\. Wang, and X\. CaiA survey on llm mid\-training\.arXiv preprint arXiv:2510\.23081\.Cited by:[§1](https://arxiv.org/html/2610.02986#S1.p2.1)\.
- Weberet al\.\(2024\)M\. Weber, D\. Fu, Q\. Anthony, Y\. Oren, S\. Adams, A\. Alexandrov, X\. Lyu, H\. Nguyen, X\. Yao, V\. Adams, B\. Athiwaratkun, R\. Chalamala, K\. Chen, M\. Ryabinin, T\. Dao, P\. Liang, C\. Ré, I\. Rish, and C\. ZhangRedPajama: an open dataset for training large language models\.External Links:2411\.12372,[Link](https://arxiv.org/abs/2411.12372)Cited by:[§C\.1](https://arxiv.org/html/2610.02986#A3.SS1.p1.1)\.
- Weiet al\.\(2021\)J\. Wei, M\. Bosma, V\. Y\. Zhao, K\. Guu, A\. W\. Yu, B\. Lester, N\. Du, A\. M\. Dai, and Q\. V\. LeFinetuned language models are zero\-shot learners\.arXiv preprint arXiv:2109\.01652\.Cited by:[§C\.2](https://arxiv.org/html/2610.02986#A3.SS2.p1.1)\.
- Wortsmanet al\.\(2022\)M\. Wortsman, G\. Ilharco, S\. Y\. Gadre, R\. Roelofs, R\. Gontijo\-Lopes, A\. S\. Morcos, H\. Namkoong, A\. Farhadi, Y\. Carmon, S\. Kornblith, and L\. SchmidtModel soups: averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.External Links:2203\.05482,[Link](https://arxiv.org/abs/2203.05482)Cited by:[§C\.2](https://arxiv.org/html/2610.02986#A3.SS2.p3.1)\.
- Xieet al\.\(2025\)C\. Xie, Y\. Huang, C\. Zhang, D\. Yu, X\. Chen, B\. Y\. Lin, B\. Li, B\. Ghazi, and R\. KumarOn memorization of large language models in logical reasoning\.InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics,pp\. 2742–2785\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px2.p1.1)\.
- Xieet al\.\(2024\)R\. Xie, J\. Wang, R\. Huang, M\. Zhang, R\. Ge, J\. Pei, N\. Z\. Gong, and B\. DhingraRecall: membership inference via relative conditional log\-likelihoods\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 8671–8689\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.8.2),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Xuet al\.\(2025\)H\. Xu, J\. Liu, Y\. Choi, N\. A\. Smith, and H\. HajishirziINFINI\-gram mini: exact n\-gram search at the internet scale with fm\-index\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 24955–24980\.Cited by:[§4\.4](https://arxiv.org/html/2610.02986#S4.SS4.p4.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§1](https://arxiv.org/html/2610.02986#S1.p2.1)\.
- Yeet al\.\(2024\)W\. Ye, J\. Hu, L\. Li, H\. Wang, G\. Chen, and J\. ZhaoData contamination calibration for black\-box llms\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 10845–10861\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px1.p1.1),[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.10.2),[Table 1](https://arxiv.org/html/2610.02986#S1.T1.2.1.4.1),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Yeomet al\.\(2018\)S\. Yeom, I\. Giacomelli, M\. Fredrikson, and S\. JhaPrivacy risk in machine learning: analyzing the connection to overfitting\.In2018 IEEE 31st computer security foundations symposium \(CSF\),pp\. 268–282\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.2.2),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Zhanget al\.\(2025\)H\. Zhang, S\. Zhang, B\. Jing, and H\. WeiFine\-tuning can help detect pretraining data from large language models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 60902–60921\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px2.SPx2.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.17.2),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Zhanget al\.\(2024a\)J\. Zhang, J\. Sun, E\. Yeats, Y\. Ouyang, M\. Kuo, J\. Zhang, H\. F\. Yang, and H\. LiMin\-k%\+\+: improved baseline for detecting pre\-training data from large language models\.arXiv preprint arXiv:2404\.02936\.Cited by:[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.6.2),[§1](https://arxiv.org/html/2610.02986#S1.p1.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§3](https://arxiv.org/html/2610.02986#S3.p1.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Zhanget al\.\(2024b\)W\. Zhang, R\. Zhang, J\. Guo, M\. de Rijke, Y\. Fan, and X\. ChengPretraining data detection for large language models: a divergence\-based calibration method\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 5263–5274\.Cited by:[Appendix A](https://arxiv.org/html/2610.02986#A1.SS0.SSS0.Px1.p1.1),[Appendix B](https://arxiv.org/html/2610.02986#A2.SS0.SSS0.Px1.SPx1.p1.1),[Table 26](https://arxiv.org/html/2610.02986#A8.T26.2.1.7.2),[Table 1](https://arxiv.org/html/2610.02986#S1.T1.2.1.7.1),[§1](https://arxiv.org/html/2610.02986#S1.p2.1),[§2](https://arxiv.org/html/2610.02986#S2.p1.1),[§2](https://arxiv.org/html/2610.02986#S2.p2.1),[§5\.1](https://arxiv.org/html/2610.02986#S5.SS1.p1.1)\.
- Zhaoet al\.\(2024\)W\. Zhao, X\. Ren, J\. Hessel, C\. Cardie, Y\. Choi, and Y\. DengWildchat: 1m chatgpt interaction logs in the wild\.arXiv preprint arXiv:2405\.01470\.Cited by:[§6](https://arxiv.org/html/2610.02986#S6.SSx2.p1.1)\.

## Appendix ADetailed Descriptions of Existing Benchmarks for Membership Inference on LLMs

This section provides detailed descriptions of each benchmark surveyed in Section[2](https://arxiv.org/html/2610.02986#S2)\.

##### Temporal\-Cutoff Benchmarks\.

The central difficulty in benchmark construction is obtaining reliable membership labels without access to the model’s training corpus\. Temporal\-cutoff benchmarks solve this by exploiting the model’s training cutoff date: texts published before the cutoff are treated as members, while texts published after it are treated as non\-members\. WikiMIA\([Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8)\)first instantiated this strategy on Wikipedia event pages, treating pre\-2017 articles as members and post\-2023 articles as non\-members\. Subsequent benchmarks extend it along two axes: \(i\)broadening the coverage, where StackMIA\([Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14)\)applies the same January 2017 cutoff to Stack Exchange posts, treating those after May 2023 as non\-members, and PatentMIA\([Zhang et al\., 2024b](https://arxiv.org/html/2610.02986#bib.bib10)\)targets Chinese\-language patents, treating those published before January 2023 as members and those with priority dates after March 2024 as non\-members, to evaluate Chinese LLMs released in that window; and \(ii\)refreshing the cutoff, where WikiMIA\-24\([Fu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib19)\)retains WikiMIA’s original January 2017 members but updates its non\-members to articles published after March 2024, extending applicability to more recent models\. However, text naturally drifts over time in vocabulary, topic, and style, so pre\- and post\-cutoff splits differ in more than membership alone, entangling genuine membership signals with distribution shifts\. As a result, high performance on them likely reflects the shift rather than genuine membership detection\([Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18)\)\.

##### Partition\-Based Benchmarks\.

While temporal\-cutoff benchmarks infer membership from publication dates, partition\-based benchmarks obtain exact labels from a known training corpus, treating instances included in training as members and held\-out or later\-introduced instances as non\-members\. MIMIR\([Duan et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib18)\)partitions the documented corpora of open models, namely The Pile\([Gao et al\., 2020](https://arxiv.org/html/2610.02986#bib.bib31)\)and Dolma\([Soldaini et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib24)\), spanning a broad range of domains from general web text to academic and code corpora\. To decontaminate non\-members, it applies a Bloom\-filter–based 13\-gram deduplication filter against the training corpus, using a permissive overlap threshold of≤80%\\leq 80\\%: a candidate non\-member is discarded only if*more than*80%80\\%of its 13\-grams already appear among training data, so a document sharing up to80%80\\%of its 13\-grams with the training data is still retained as a “non\-member\.” Because the threshold is permissive, MIMIR reports substantial residual overlap between members and non\-members across all measured domains, consistently exceeding 30%\. In contrast,OLMo\-Detectapplies 13\-gram filtering with a far stricter≤20%\\leq 20\\%threshold, reducing residual overlap to5\.31%5\.31\\%on average \(Table[3](https://arxiv.org/html/2610.02986#S4.T3)\)\. Rather than partitioning an existing corpus, RL\-MIA\([Tao et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib17)\)manufactures its partition to simulate contamination during RL post\-training, drawing from AIME24, AIME25, K&K\([Xie et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib33)\), and SAT\([Liu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib34)\)\. Specifically, for AIME24 and AIME25, half of each benchmark’s items are injected into the OpenR1\-Math\-46K\([Guha et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib35)\)RL training corpus as members, with the remainder held out as non\-members, while for K&K and SAT, original training instances serve as members and additional items synthesized via the same generation procedure serve as non\-members\. However, similar to MIMIR, RL\-MIA provides no explicit mechanism to verify or control distributional alignment between its members and non\-members\.

##### Checkpoint\-Based Benchmarks\.

OLMoMIA\([Kim et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib13)\)constructs members and non\-members from the training checkpoints of OLMo\-7B\([Groeneveld et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib2)\): members are examples seen before step 100k, and non\-members are examples introduced between steps 400k and 500k, leaving the intervening steps unused\. OLMoMIA is the only existing benchmark to explicitly control distributional alignment between members and non\-members, via semantic similarity\. Specifically, it embeds members and non\-members using a text encoder, NV\-Embed\-v2\([Lee et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib37)\), clusters each split separately withkk\-means, and selects cluster pairs based on their inter\-cluster distance, producing settings that range from maximally separated to near\-indistinguishable\. However, most MIA methods exploit surface\-form signals that manifest through exact wording rather than meaning, while semantic similarity does not entail lexical similarity\. For example, two texts describing the same event using synonyms can lie close in NV\-Embed\-v2’s embedding space while sharing few overlappingnn\-grams\. Consequently, two instances considered well\-aligned under OLMoMIA’s construction may still differ substantially in surface\-form statistics that MIAs actually exploit, leading to a lexical shortcut that can inflate performance independent of membership signals\.

##### Model\-Output\-Based Benchmarks\.

Both preceding strategies label texts, model\-output\-based benchmarks instead label model outputs, constructing members and non\-members from the generations of models with and without exposure to the evaluation data\. DETCON\([Dong et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib16)\)simulates membership during the supervised fine\-tuning \(SFT\) stage by fine\-tuning a base LLM via low\-rank adaptation \(LoRA\)\([Hu et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib36)\)on HumanEval\([Chen et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib32)\)\(code generation\) and GSM8K\([Cobbe et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib66)\)\(mathematical reasoning\), varying the mixing ratio, learning rate, and occurrence count of these benchmarks in the fine\-tuning data\. Outputs from the fine\-tuned model are treated as members, while outputs from the corresponding base model are treated as non\-members\. However, since members and non\-members come from different model states rather than a single model, differences arising from the fine\-tuning procedure, such as LoRA’s own effect on output style independent of the injected content, may provide a shortcut to separate the data\. Moreover, LoRA fine\-tuning exhibits dynamics that diverge from full\-parameter training\([Biderman et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib61)\), so detection signals calibrated on DETCON’s LoRA\-simulated membership may not transfer to membership arising from real\-world LLM training\.

## Appendix BDetailed Descriptions of Existing MIAs

This section provides a taxonomy of existing MIAs\. Notably, we restrict our scope to those applicable in the gray\-box setting defined in Section[3](https://arxiv.org/html/2610.02986#S3)\. These methods can be grouped into unsupervised and supervised methods based on whether they require labeled members and non\-members\.

##### Unsupervised Methods\.

We classify unsupervised methods into four categories based on the signal they exploit: likelihood\-based, prompt\-based, perturbation\-based, and output\-distribution\-based\.

###### Likelihood\-based\.

These methods infer membership from the model’s token\-level likelihoods, exploiting the tendency of LLMs to assign higher probability to previously seen text\. The earliest instantiation, Perplexity\([Yeom et al\., 2018](https://arxiv.org/html/2610.02986#bib.bib26)\), directly thresholds the model’s loss on the candidate text, based on the observation that the gap between member and non\-member loss reflects overfitting\. However, raw likelihood conflates two causes: a text may receive high probability because it was memorized, or merely because it is inherently predictable\. Subsequent methods differ primarily in what they calibrate the likelihood against to remove this confound\. The first family calibrates against properties of the candidate text itself: Zlib and Lowercase\([Carlini et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib25)\)normalize the model’s perplexity by the text’s zlib\-compressed entropy and the perplexity of its lowercased variant, respectively; Min\-K%\([Shi et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib8)\)restricts scoring to thek%k\\%least\-probable tokens, hypothesizing that unseen text contains outlier tokens with abnormally low probability, and its extension Min\-K%\+\+\([Zhang et al\., 2024a](https://arxiv.org/html/2610.02986#bib.bib9)\)further normalizes each token’s log\-probability by the mean and standard deviation over the vocabulary at that position, exploiting the insight that maximum\-likelihood training pushes member tokens towards local maxima of the output distribution; DCPDD\([Zhang et al\., 2024b](https://arxiv.org/html/2610.02986#bib.bib10)\)calibrates token probabilities against a token\-frequency prior estimated from a reference corpus, computing the cross\-entropy between the two distributions to avoid misclassifying unseen text composed of frequent tokens\. The second family calibrates against augmented variants of the candidate: PAC\([Ye et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib14)\)extends Min\-K% by constructing adjacent samples via word\-level augmentation and computing a polarized distance that contrasts near and far probability regions to expose overfitted training samples; although it shares the augmentation idea with the perturbation\-based Neighborhood Attack below, its final score remains a Min\-K%\-style likelihood statistic, so we group it here\. The third family calibrates against other models: Smaller Ref\([Carlini et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib25)\)scores candidates by the ratio of the target model’s perplexity to that of a smaller model from the same family, exploiting the fact that smaller models memorize less of the shared training distribution, and SPV\-MIA\([Fu et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib28)\)removes the need for an externally trained reference by fine\-tuning one on text elicited from the target model itself through self\-prompting, scoring membership via a probabilistic\-variation signal computed over perturbed variants of the candidate\. Finally, the fourth family calibrates against the conditioning context rather than the text or model: ReCaLL\([Xie et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib11)\)measures the relative drop in conditional log\-likelihood when the candidate is prefixed with known non\-member text, based on the finding that such conditioning reduces the likelihood of members substantially more than that of non\-members; EMMIA\([Kim et al\., 2026](https://arxiv.org/html/2610.02986#bib.bib13)\)addresses ReCaLL’s dependence on prefix selection by iteratively refining prefix scores through expectation\-maximization; CAMIA\([Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12)\)extracts richer signals from the per\-token loss sequence, conditioning on prefix properties such as length, token diversity, and prefix perplexity, and computing loss\-dynamics statistics including slope, fluctuation, and outlier counts, motivated by the observation that member losses decrease more smoothly as context accumulates\.

###### Perturbation\-based\.

Rather than calibrating the likelihood of the candidate alone, perturbation\-based methods treat the candidate’s synthetic neighbors as the reference: the neighbors match the candidate in content and fluency, so any gap between the candidate and its neighbors isolates memorization of surface form from inherent predictability\. Neighborhood Attack\([Mattern et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib27)\)generates neighbors via masked\-word substitution with a mask\-filling model and scores membership by the difference between the candidate’s log\-likelihood and the mean log\-likelihood of its neighbors, flagging texts whose likelihood stands out from their own neighborhood\.

###### Prompt\-based\.

Rather than reading likelihoods, prompt\-based methods probe memorization through the model’s responses to prompts, differing primarily in how much they withhold and what they ask the model to do with the rest\. Guided Instruction\([Golchin and Surdeanu, 2023](https://arxiv.org/html/2610.02986#bib.bib7)\)withholds the most: by prompting the model with the dataset name, partition, and the initial segment of an instance, it measures whether the completion reproduces the entire withheld remainder, scoring the completion against the reference continuation with ROUGE\-L\([Lin, 2004](https://arxiv.org/html/2610.02986#bib.bib62)\)and BLEURT\([Sellam et al\., 2020](https://arxiv.org/html/2610.02986#bib.bib63)\)and additionally employing an LLM judge to identify exact or near\-exact matches\. Comparing this guided completion with one generated from a general instruction that omits dataset identifiers isolates the contribution of dataset\-specific memorization\. Narrowing the target from a free\-form continuation to a single slot,[Chang et al\. \(2023\)](https://arxiv.org/html/2610.02986#bib.bib29)employ a name\-cloze formulation in which a proper name is masked in a book passage and the model is asked to recover it without additional context, making successful recovery indicative of memorization of the source text, while TS\-Guessing\([Deng et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib30)\)adapts the same cloze principle, masking an incorrect answer option in a multiple\-choice question and flagging membership when the exact missing option is reproduced\. This narrowing reflects a trade\-off: free\-form completion probes more of the memorized content but requires soft matching, whereas slot filling admits exact\-match verification at the cost of probing only a fragment\. At the far end of this spectrum, DCQ\([Golchin and Surdeanu, 2025](https://arxiv.org/html/2610.02986#bib.bib15)\)withholds nothing and shifts from production to recognition: it casts membership inference as a five\-way multiple\-choice quiz in which the original candidate appears alongside four word\-level paraphrases that preserve its semantics\. Since all options are semantically equivalent, a consistent preference for the original is attributable to memorization of surface form rather than content understanding\.

###### Output\-distribution\-based\.

These methods share a common premise: membership leaves a fingerprint of abnormal concentration in the model’s output distribution\. CDD\([Dong et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib16)\)reads this fingerprint at the string level, sampling multiple outputs for the same input and measuring the peakedness of their distribution around the greedy output via edit distance, flagging membership when the outputs are abnormally concentrated\. Self\-Critique\([Tao et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib17)\)reads the same fingerprint in a setting where it is sharpest: membership introduced during RL post\-training\. Motivated by the observation that RL collapses the model’s output entropy onto narrow reasoning paths, it prompts the model to produce two distinct responses to the same problem and flags membership when the two exhibit high similarity in the entropy space\.

##### Supervised Methods\.

We classify supervised methods into two categories based on how they make use of labeled members and non\-members: likelihood\-based and tuning\-based\.

###### Likelihood\-based\.

A supervised variant of CAMIA\([Chang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib12)\)uses labeled members and non\-members to train a downstream classifier, leaving the target model untouched\. It represents each candidate as a feature vector of its per\-token loss\-dynamics signals and trains a logistic regression classifier to combine them, applying PCA within each signal group to control dimensionality\. The underlying likelihood\-derived signals are unchanged, only their combination is learned\.

###### Tuning\-based\.

These methods use labeled data to tune the target model, differing in what is tuned and what the tuning is used to achieve\. MIA\-Tuner\([Fu et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib19)\)keeps the model weights frozen and tunes a soft prompt injected into the target model, using both members and non\-members, so that the tuned model becomes the detector directly: an instruction\-tuning objective trains aligned models to answerYesorNoto a membership query, while a contrastive objective trains unaligned models, which cannot follow such instructions, to amplify the loss gap between members and non\-members\. FSD\([Zhang et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib59)\)instead fine\-tunes the model weights and is motivated by an empirical finding: briefly fine\-tuning the target model on a small set of non\-members disproportionately lowers non\-members’ scores under an existing method \(e\.g\. Perplexity, Min\-K%\), while leaving members’ scores largely unchanged\. Rather than tuning the model to become the detector itself, as MIA\-Tuner does, FSD uses this asymmetric shift as its detection signal: membership is inferred from a candidate’s score deviation before versus after fine\-tuning, and it requires only non\-members for tuning\.

## Appendix CDetailed Descriptions of OLMo 2 Training Data

This appendix provides a detailed breakdown of OLMo 2’s training data across the three stages described in Section[4\.2](https://arxiv.org/html/2610.02986#S4.SS2)\. Token counts, document counts, and filtering details are reproduced from the original OLMo 2 paper\([OLMo et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib3)\)\.

### C\.1Pre\-training: OLMo 2 Mix 1124

OLMo 2 Mix 1124 comprises seven domains drawn from three source corpora\. From DCLM\([Li et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib38)\), OLMo 2 uses DCLM\-Baseline \(web pages\), the top 10% quality subset of deduplicated DCLM\-RefinedWeb \(a RefinedWeb\([Penedo et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib64)\)\-style heuristically filtered version of the raw DCLM\-Pool corpus\), selected using a fastText classifier\([Joulin et al\., 2017](https://arxiv.org/html/2610.02986#bib.bib44)\)\. From Dolma 1\.7\([Soldaini et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib24)\), it incorporates five subsets: peS2o\([Soldaini and Lo, 2023](https://arxiv.org/html/2610.02986#bib.bib39)\)\(academic papers\), arXiv \(STEM papers; originally sourced from RedPajama\([Weber et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib40)\)\), OpenWebMath\([Paster et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib41)\)\(math web pages\), Algebraic Stack\([Azerbayev et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib42)\)\(math proofs and code\), and Wikipedia/Wikibooks \(encyclopedic content\)\. The seventh domain, StarCoder\([Li et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib43)\)\(code\), is derived from permissively licensed GitHub repositories\. StarCoder is filtered to retain only repositories with at least 2 GitHub stars; within the retained set, documents whose most\-frequent word constitutes over 30% of tokens or whose top\-2 most\-frequent words constitute over 50% are discarded\.

The full per\-domain composition of OLMo 2 Mix 1124 is summarized in Table[9](https://arxiv.org/html/2610.02986#A3.T9)\. The 1B and 7B models are pretrained on 4 trillion tokens of this mix \(1 epoch\), the 13B on 5 trillion \(1\.2 epochs\), and the 32B on 6 trillion \(1\.5 epochs\)\.

DomainTypeTokensWordsBytesDocsDCLM\-BaselineWeb pages3\.71T3\.32T21\.32T2\.95BStarCoder \(filtered\)Code83\.0B70\.0B459B78\.7MpeS2oAcademic papers58\.6B51\.1B413B38\.8MarXivSTEM papers20\.8B19\.3B77\.2B3\.95MOpenWebMathMath web pages12\.2B11\.1B47\.2B2\.89MAlgebraic StackMath proofs/code11\.8B10\.8B44\.0B2\.83MWikipedia/WikibooksEncyclopedic3\.7B3\.16B16\.2B6\.17MTotal3\.90T3\.48T22\.38T3\.08B

Table 9:Composition of OLMo 2 Mix 1124, the pre\-training corpus of the OLMo 2 suite, reproduced from Table 4 of[OLMo et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib3)\.DomainTypeTokensWordsBytesDocsHigh\-Quality Subset \(five domains\)Filtered DCLM\-BaselineHigh\-quality web752B670B4\.56T606MFLANInstruction data17\.0B14\.4B98\.2B57\.3MpeS2oAcademic papers58\.6B51\.1B413B38\.8MWikipedia/WikibooksEncyclopedic3\.7B3\.16B16\.2B6\.17MStack ExchangeQ&A1\.26B1\.14B7\.72B2\.48MHigh\-quality total832\.6B739\.8B5\.09T710\.8MDolmino Math Mix \(one domain, seven subsets\)TuluMathSynthetic math230M222M1\.03B220KDolminoSynthMathSynthetic math28\.7M35\.1M163M725KTinyGSM\-MINDSynthetic math6\.48B5\.68B25\.52B17MMathCoder2 Synth BooksSynthetic math3\.87B3\.71B18\.4B2\.83MMetamath \(OWM\-filtered\)Math84\.2M76\.6M741M383KCodeSearchNet \(OWM\-filtered\)†Code1\.78M1\.41M29\.8M7\.27KGSM8K \(training split\)Math2\.74M3\.00M25\.3M17\.6KDolmino Math Mix total10\.7B9\.73B45\.9B21\.37M

Table 10:Composition of Dolmino Mix 1124, the mid\-training corpus of the OLMo 2 suite, reproduced from Table 5 of[OLMo et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib3)\. The High\-Quality Subset contributes five domains directly; the Dolmino Math Mix is itself a single domain whose composition is given by seven subsets\. “OWM\-filtered” indicates that the subset is further filtered using the OpenWebMath classifier designed by[OLMo et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib3)\.†Although CodeSearchNet \(OWM\-filtered\) is listed as part of the Dolmino Math Mix in[OLMo et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib3), it was not used in the final training\.DomainTokens50B sample100B sample300B sampleDomain %Mix %Domain %Mix %Domain %Mix %Filtered DCLM\-Baseline752B3\.2347\.26\.8550\.220\.7851\.9FLAN17\.0B50\.016\.610016\.720011\.3Stack Exchange1\.26B1002\.452002\.474001\.68peS2o58\.6B5\.155\.8516\.79\.5210019\.4Wikipedia/Wikibooks3\.7B1007\.111003\.574004\.86Dolmino Math Mix10\.7B10020\.820017\.540010\.8

Table 11:Per\-domain composition of the three token\-budget samples \(50B, 100B, 300B\) drawn from Dolmino Mix 1124 for mid\-training, reproduced from Table 13 of[OLMo et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib3)\. For each domain: “Domain %” is the fraction of that domain’s full token count drawn into the sample, with values above 100 indicating repetition \(e\.g\., 400 = 4×\\timesrepeat\); “Mix %” is the domain’s share of the sample’s total tokens \(each Mix % column sums to 100%\)\.
### C\.2Mid\-training: Dolmino Mix 1124

Mid\-training is an intermediate annealing stage between pre\-training and post\-training, in which the pretrained model is further trained on a smaller, higher\-quality data mixture under a decaying learning rate to improve performance on targeted skills such as math and reasoning\. OLMo 2’s mid\-training mixture, Dolmino Mix 1124, comprises six domains organized into a High\-Quality Subset and a Dolmino Math Mix\. The High\-Quality Subset contains five domains: filtered DCLM\-Baseline, peS2o, Wikipedia/Wikibooks, FLAN\([Wei et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib46)\), and Stack Exchange\. Dolmino Math Mix \(abbreviated as Math Mix in the main paper\) is a heterogeneous mixture aggregating synthetic math data \(TuluMath, generated with GPT\-4o\([OpenAI et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib47)\); DolminoSynthMath and TinyGSM\-MIND, generated with Qwen2\.5\-7B\-Instruct\([Qwen et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib23)\); and MathCoder2\-Synthetic, collected from external sources and filtered using a FastText classifier trained on GPT\-4o annotations\), the GSM8K training split, OpenWebMath\-filtered Metamath, and OpenWebMath\-filtered CodeSearchNet\. The full per\-domain composition of Dolmino Mix 1124 is given in Table[10](https://arxiv.org/html/2610.02986#A3.T10)\.

Within the High\-Quality Subset, peS2o and Wikipedia/Wikibooks are reused from pre\-training without modification\. Filtered DCLM\-Baseline applies additional filtering on top of DCLM\-Baseline: documents are first retained by the DCLM FastText classifier with threshold 0\.0331 \(retaining∼65\.6%\\sim\\text\{65\.6\}\\%\), then restricted to FineWeb\-Edu\([Penedo et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib45)\)scores≥2\\geq\\text\{2\}\(∼20\.3%\\sim\\text\{20\.3\}\\%retained overall\)\. FLAN is decontaminated by removing training, validation, and test instances of evaluation benchmarks\. Stack Exchange is taken from the 2024\-09\-30 dump, restructured into Q&A format, and filtered to retain only pairs with an accepted answer, at least 3 question votes, and at least 5 answer votes\.

Three sample sizes \(50B, 100B, 300B tokens\) are drawn from the full mix, their per\-domain compositions are summarized in Table[11](https://arxiv.org/html/2610.02986#A3.T11)\. The 1B model anneals on a single 50B sample without averaging\. The 7B, 13B, and 32B models applymodel soup\([Wortsman et al\., 2022](https://arxiv.org/html/2610.02986#bib.bib54)\): multiple checkpoints are annealed on the same data mix using different randomized data orders, then averaged into the final midtrained model\. The 7B averages three checkpoints trained on 50B samples, while the 13B and 32B each average four checkpoints \(three trained on 100B samples and one on a 300B sample\)\.

### C\.3Post\-training: Tülu 3 Recipe

SubsetCount\# in SFTSampleableGeneralTülu 3 Hardcoded240240OpenAssistant88,8387,132✓No Robots9,5009,500WildChat \(GPT\-4 subset\)241,307100,000✓Knowledge RecallFLAN v289,98289,982SciRIFF35,35710,000✓TableGPT13,2225,000✓Math and ReasoningTülu 3 Persona MATH†149,960149,960Tülu 3 Persona GSM†49,98049,980Tülu 3 Persona Algebra20,00020,000OpenMathInstruct 221,972,79150,000✓NuminaMath\-TIR64,31264,312CodingTülu 3 Persona Python34,99934,999Evol CodeAlpaca107,276107,276Safety and Non\-ComplianceTülu 3 CoCoNot10,98310,983Tülu 3 WildJailbreak50,00050,000Tülu 3 WildGuardMix50,00050,000MultilingualAya202,285100,000✓Precise Instruction FollowingTülu 3 Persona IF29,98029,980Total23,221,012939,3446

Table 12:Subsets of OLMo 2’s SFT mix, reproduced from Table 7 of[Lambert et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib60)\. “Count” is the total number of prompts available in the upstream source corpus; “\# in SFT” corresponds totulu\-3\-sft\-olmo\-2\-mixture\(7B and 13B; 939,344 prompts\)\. The 1B and 32B models usetulu\-3\-sft\-olmo\-2\-mixture\-0225\(866,138 prompts\), which draws from the same 19 subsets but further removes date\-cutoff instructions and applies majority voting over 5 completions to the†\-marked subsets\. TheSampleablecolumn marks subsets with non\-member candidates available forOLMo\-Detect: ✓ = sampleable remainder available; blank = source corpus fully consumed \(\# in SFT = Count\)\.StageDomainSubsetPrompts∗1B7B13B32BSampleablePost\-trainingDPOSFT Reused117,025✓✓✓✓∼\\simSFT IF65,792✓✓✓✓WildChat Unused‡84,105✓✓✓✓✓WildChat Reused‡17,703✓✓✓✓✓WildChat IF‡10,794✓✓✓✓UltraFeedback \(cleaned\)60,816✓✓✓✓DaringAnteater IF1,618✓✓✓✓✓Tülu 3 Personas IF19,890✓✓✓✓✓Total prompts378\.3K366\.7K377\.7K377\.9K3

Table 13:Prompt subsets for OLMo 2’s DPO stage across all four model sizes\.∗Per\-subset prompt counts are reproduced from Table 27 of[OLMo et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib3)and apply to the 7B and 13B mixes; the 1B and 32B mixes draw from the same eight subset families with comparable but not identical per\-subset counts\. The 7B mix omits WildChat IF; chosen responses in the 1B and 32B mixes are additionally filtered to remove date\-cutoff mentions\. Subsets labeled “IF” contain prompts augmented with verifiable instruction\-following constraints \(IFEval\-style\)\. TheSampleablecolumn marks subsets with uncontaminated candidates available forOLMo\-Detect: ✓ = sampleable remainder available; blank = source corpus fully consumed;∼\\sim= unknown \(SFT Reused aggregates prompts from many SFT source datasets with varying sampleability, making it impossible to determine remainder without per\-source attribution\)\.‡All three WildChat subsets draw from the same upstream source corpus andtogether count as one of the three sampleable sources; the remaining two are DaringAnteater IF and Tülu 3 Personas IF\.The post\-training stage follows the Tülu 3 recipe\([Lambert et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib60)\)adapted to permissively\-licensed sources, and proceeds in three sequential phases: SFT, DPO, and RLVR\.

##### Supervised Fine\-Tuning \(SFT\)\.

Supervised fine\-tuning \(SFT\) trains a pretrained language model on a collection of \(prompt, response\) pairs using next\-token\-prediction loss, teaching the model to follow instructions and generate well\-formatted responses\. OLMo 2 draws its SFT prompts from the Tülu 3 SFT data sources\([Lambert et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib60)\), organized into two mixes:tulu\-3\-sft\-olmo\-2\-mixture\(applied to 7B and 13B models\), contains 939,344 prompts111The OLMo 2 paper\([OLMo et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib3)\)reports 939,104 prompts for this mix, whereas the released HuggingFace dataset \([https://huggingface\.co/datasets/allenai/tulu\-3\-sft\-olmo\-2\-mixture](https://huggingface.co/datasets/allenai/tulu-3-sft-olmo-2-mixture)\) contains 939,344 prompts\. In this paper, we adopt the released dataset as the ground\-truth count\.; andtulu\-3\-sft\-olmo\-2\-mixture\-0225\(applied to 1B and 32B models\), contains 866,138 prompts and applies two additional prompt\-level filters relative to the first mix: \(i\) prompts whose instruction text mentions an explicit date cutoff are removed, as training on such prompts caused the resulting model to hallucinate its own date cutoff and emit “As an AI language model…” prefaces at inference time; \(ii\) for the synthetic Persona MATH and Grade School Math subsets, five completions are sampled per prompt, and only prompts whose completions reach a majority answer are retained, reducing the rate of incorrect math answers absorbed during SFT\. Both mixes inherit the Tülu 3 subset structure, as detailed in Table[12](https://arxiv.org/html/2610.02986#A3.T12); the six subsets marked ✓: OpenAssistant, WildChat, SciRIFF, TableGPT, OpenMathInstruct 2, and Aya, are downsampled from larger upstream corpora\.

##### Direct Preference Optimization \(DPO\)\.

Direct preference optimization \(DPO\)\([Rafailov et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib53)\)is a preference\-alignment method that fine\-tunes a language model directly on pairwise \(chosen, rejected\) data, bypassing the reward\-model\-based RL training loop used in conventional reinforcement learning from human feedback \(RLHF\)\([Ouyang et al\., 2022](https://arxiv.org/html/2610.02986#bib.bib52)\)\. DPO preference data is constructed for each model size via a common synthetic\-data pipeline adapted from Tülu 3: \(i\) candidate responses are sampled from a pool of 20 external models together with on\-policy data generated from intermediate OLMo 2 SFT checkpoints; \(ii\) GPT\-4o\-2024\-08\-06 serves as the judge, scoring responses along four dimensions \(helpfulness, truthfulness, honesty, and instruction following\); and \(iii\) scores are binarized using the Argilla procedure\([Cui et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib48)\), which averages ratings across dimensions, selects the highest\-scoring completion as the chosen response, and samples one of the remaining completions as the rejected response\. All four mixes draw prompts from the eight Tülu 3 subset families listed in Table[13](https://arxiv.org/html/2610.02986#A3.T13), yielding 378\.3K prompts for the 1B, 366\.7K for the 7B \(excluding WildChat IF\), 377\.7K for the 13B, and 377\.9K for the 32B\. Chosen responses in the 1B and 32B mixes are additionally filtered to remove instances mentioning a date cutoff\.

##### Reinforcement Learning with Verifiable Rewards \(RLVR\)\.

RLVR\([Lambert et al\., 2025](https://arxiv.org/html/2610.02986#bib.bib60)\)is a reinforcement learning fine\-tuning method that replaces the learned reward model used in RLHF with a deterministic rule\-based verifier\. Completions receive a binary reward based on whether they can be programmatically verified as correct, e\.g\., through exact\-match comparison against a ground\-truth mathematical solution or satisfaction of explicit instruction\-following constraints\. The training data combines three subsets: the GSM8K training set, the MATH\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib49)\)training set, and prompts from[Lambert et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib60)containing verifiable instruction\-following constraints \(IFEval\-style\)\. GSM8K and MATH prompts are prepended with 8\-shot and 3\-shot chain\-of\-thought \(CoT\) in\-context exemplars, respectively\. The 7B and 13B models are trained with proximal policy optimization \(PPO\)\([Schulman et al\., 2017](https://arxiv.org/html/2610.02986#bib.bib50)\), with the value function initialized from a reward model trained on the same preference data used for DPO\. The 1B and 32B models are trained with group relative policy optimization \(GRPO\)\([Shao et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib51)\), which forgoes a reward model\. All four models receive a single RLVR pass on the combined mix; the 13B model additionally undergoes two further passes, one on GSM8K alone and one on MATH alone, with the value function re\-initialized from the reward model at each subsequent pass\. The 1B model additionally undergoes one MATH\-only pass \(see Table[14](https://arxiv.org/html/2610.02986#A3.T14)footnote\)\.

StageComponent1B7B13B32BPre\-trainingOLMo 2 Mix 1124✓✓✓✓Mid\-trainingDolmino Mix 1124, 50B sample×1\\times 1×3\\times 3Dolmino Mix 1124, 100B sample×3\\times 3×3\\times 3Dolmino Mix 1124, 300B sample×1\\times 1×1\\times 1SFTtulu\-3\-sft\-olmo\-2\-mixture\-0225×2\\times 2×2\\times 2tulu\-3\-sft\-olmo\-2\-mixture×2\\times 2×2\\times 2DPOModel\-specific preference mix∗✓✓✓✓RLVRCombined pass \(GSM8K \+ MATH \+ IFEval\)✓✓✓✓Additional GSM8K\-only pass✓Additional MATH\-only pass†✓✓

Table 14:Per\-model training corpora for infini\-gram indexing\. Entries denote inclusion in the model’s indexed corpus, with multiplicities reflecting epoch and model\-soup repetition \(✓ = included once;×N\\times N= includedNNtimes\)\.∗Model\-specific preference mixes:olmo\-2\-0425\-1b\-preference\-mix\(1B\),olmo\-2\-1124\-7b\-preference\-mix\(7B\),olmo\-2\-1124\-13b\-preference\-mix\(13B\),olmo\-2\-0325\-32b\-preference\-mix\(32B\)\.†The 1B model’s additional MATH\-only RLVR pass is inferred from the OLMo 2 training documentation in theopen\-instructrepository \([https://github\.com/allenai/open\-instruct/blob/main/docs/olmo2\.md](https://github.com/allenai/open-instruct/blob/main/docs/olmo2.md)\)\.

## Appendix DDetailed Per\-Domain Statistics ofOLMo\-Detect

We provide detailed per\-domain statistics forOLMo\-Detect, as introduced in Section[4\.3](https://arxiv.org/html/2610.02986#S4.SS3)\. Per\-domain statistics for pre\-training, mid\-training, and post\-training stages are reported in Tables[15](https://arxiv.org/html/2610.02986#A4.T15)–[17](https://arxiv.org/html/2610.02986#A4.T17), while per\-subset statistics for domains comprising multiple subsets \(StarCoder, Stack Exchange, SFT, and RLVR\) are reported in Tables[18](https://arxiv.org/html/2610.02986#A4.T18)–[21](https://arxiv.org/html/2610.02986#A4.T21)\. For each domain, we partition the data into a development set \(dev; for hyperparameter tuning\) and a test set \(test; for method evaluation\), with both sample and token ratios set to approximately 1:9\.

##### Interpreting the Lexical Column\.

JSD does not reach zero even between two samples drawn from the same source, because each sample misses some of the other’s rare tokens\. This floor rises as the token budget falls, and StarCoder Java has the smallest budget in the benchmark, 22\.1K tokens against 184K–217K elsewhere\. Splitting Java’s non\-members into two halves and comparing them against each other already yields 0\.40, above the 0\.30 reported for its members against its non\-members\. Restricting StarCoder Python to Java’s token budget reproduces the effect, raising its value from 0\.16 to 0\.36\. Java’s 0\.30 therefore reflects its size rather than residual misalignment, which the blind bag\-of\-words classifier confirms independently of sample size: it reaches an AUC of 0\.51 on Java \(Table[25](https://arxiv.org/html/2610.02986#A6.T25)\), indistinguishable from chance\.

StageDomainSubsetSplitPartitionSamplesTokensQualityTemporalLexicalPre\-trainingDCLM\-BaselineNMDev1621,745FastText: 0\.01808±\\pm0\.000042018\-01\-18±\\pm930d0\.07Test134195,372FastText: 0\.01809±\\pm0\.000082018\-01\-19±\\pm930dTotal150217,117FastText: 0\.01809±\\pm0\.000072018\-01\-19±\\pm930dMDev1621,721FastText: 0\.01811±\\pm0\.000002018\-01\-17±\\pm933dTest134195,295FastText: 0\.01811±\\pm0\.000002018\-01\-18±\\pm941dTotal150217,016FastText: 0\.01811±\\pm0\.000002018\-01\-18±\\pm940dpeS2oNMDev2721,793—2022\-12\-12±\\pm10d0\.10Test241195,188—2022\-12\-14±\\pm11dTotal268216,981—2022\-12\-14±\\pm11dMDev2721,804—2022\-12\-15±\\pm14dTest241195,426—2022\-12\-15±\\pm13dTotal268217,230—2022\-12\-15±\\pm13dOpenWebMathNMDev1521,761—2019\-10\-13±\\pm851d0\.11Test136195,422—2019\-10\-13±\\pm853dTotal151217,183—2019\-10\-13±\\pm853dMDev1521,793—2019\-10\-15±\\pm851dTest136195,459—2019\-10\-17±\\pm865dTotal151217,252—2019\-10\-17±\\pm864dStarCoderPython,JavaNMDev1621,536GitHub Stars:36\.19±\\pm67\.09Top\-1 Freq:0\.30±\\pm0\.01Top\-2 Freq:0\.37±\\pm0\.04—0\.15Test140195,620GitHub Stars:35\.59±\\pm66\.61Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.38±\\pm0\.05—Total156217,156GitHub Stars:35\.65±\\pm66\.66Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.38±\\pm0\.05—MDev1621,587GitHub Stars:36\.25±\\pm65\.18Top\-1 Freq:0\.28±\\pm0\.01Top\-2 Freq:0\.36±\\pm0\.04—Test140195,635GitHub Stars:36\.14±\\pm64\.61Top\-1 Freq:0\.29±\\pm0\.01Top\-2 Freq:0\.36±\\pm0\.05—Total156217,222GitHub Stars:36\.15±\\pm64\.67Top\-1 Freq:0\.29±\\pm0\.01Top\-2 Freq:0\.36±\\pm0\.05—

Table 15:Detailed per\-domain statistics for the pre\-training stage ofOLMo\-Detect\.NM: non\-member;M: member\.Dev: development set;Test: test set\. TheSubsetcolumn lists constituent subsets; blank entries indicate single\-subset domains whose subset shares the domain’s name\.StageDomainSubsetSplitPartitionSamplesTokensQualityTemporalLexicalMid\-trainingMath MixGSM8KNMDev10321,755——0\.04Test937195,408——Total1,040217,163MDev10321,729——Test937195,403——Total1,040217,132Stack ExchangeStack Overflow,Math Stack ExchangeNMDev4321,760Question Vote:3\.47±\\pm2\.20Answer Vote:5\.12±\\pm1\.942015\-03\-27±\\pm1378d0\.08Test387195,448Question Vote:3\.45±\\pm2\.19Answer Vote:5\.12±\\pm2\.002015\-03\-31±\\pm1378dTotal430217,208Question Vote:3\.45±\\pm2\.19Answer Vote:5\.12±\\pm1\.992015\-03\-31±\\pm1378dMDev4321,845Question Vote:3\.56±\\pm2\.18Answer Vote:5\.37±\\pm1\.892015\-03\-29±\\pm1302dTest387195,357Question Vote:3\.52±\\pm1\.68Answer Vote:5\.56±\\pm1\.942015\-03\-29±\\pm1374dTotal430217,202Question Vote:3\.52±\\pm1\.74Answer Vote:5\.54±\\pm1\.942015\-03\-29±\\pm1367d

Table 16:Sampling statistics for the mid\-training stage ofOLMo\-Detect\.NM: non\-member;M: member\.Dev: development set;Test: test set\. TheSubsetcolumn lists constituent subsets\.StageDomainSubsetModelSplitPartitionSamplesTokensQualityTemporalLexicalPost\-trainingSFTWildChat,Aya1B, 32BNMDev2121,697——0\.11Test204195,360——Total225217,057MDev2121,696——Test204195,463——Total225217,1597B, 13BNMDev1621,959——0\.14Test148195,165——Total164217,124MDev1621,738——Test148195,180——Total164216,918DPOUF\-WildChat1BNMDev1121,705——0\.09Test102195,340——Total113217,045MDev1121,722——Test102195,565——Total113217,2877BNMDev1021,704——0\.09Test100195,163——Total110216,867MDev1021,699——Test100195,327——Total110217,02613BNMDev1021,730——0\.12Test100195,321——Total110217,051MDev1021,720——Test100195,539——Total110217,25932BNMDev1021,838——0\.08Test100195,333——Total110217,171MDev1021,779——Test100195,498——Total110217,277RLVRCoT\-GSM8K,CoT\-MATH1B, 7B,13B, 32BNMDev2921,582——0\.01Test261195,528——Total290217,110MDev2921,543——Test261195,482——Total290217,025

Table 17:Detailed per\-stage statistics for the post\-training stage ofOLMo\-Detect\.NM: non\-member;M: member\.Dev: development set;Test: test set\. TheSubsetcolumn lists constituent subsets; blank entries indicate single\-subset domains whose subset shares the domain’s name\.StageDomainSubsetSplitPartitionSamplesTokensQualityTemporalLexicalPre\-trainingStarCoderPythonNMDev1019,364GitHub Stars:29\.50±\\pm61\.41Top\-1 Freq:0\.31±\\pm0\.01Top\-2 Freq:0\.36±\\pm0\.04—0\.16Test90175,594GitHub Stars:28\.53±\\pm59\.41Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.38±\\pm0\.06—Total100194,958GitHub Stars:28\.63±\\pm59\.62Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.37±\\pm0\.05—MDev1019,551GitHub Stars:28\.90±\\pm58\.22Top\-1 Freq:0\.29±\\pm0\.01Top\-2 Freq:0\.35±\\pm0\.04—Test90175,551GitHub Stars:29\.23±\\pm57\.87Top\-1 Freq:0\.29±\\pm0\.01Top\-2 Freq:0\.36±\\pm0\.06—Total100195,102GitHub Stars:29\.20±\\pm57\.90Top\-1 Freq:0\.29±\\pm0\.01Top\-2 Freq:0\.36±\\pm0\.06—JavaNMDev62,172GitHub Stars:47\.33±\\pm74\.29Top\-1 Freq:0\.30±\\pm0\.01Top\-2 Freq:0\.38±\\pm0\.03—0\.30Test5020,026GitHub Stars:48\.28±\\pm76\.28Top\-1 Freq:0\.30±\\pm0\.01Top\-2 Freq:0\.39±\\pm0\.03—Total5622,198GitHub Stars:48\.18±\\pm76\.07Top\-1 Freq:0\.30±\\pm0\.01Top\-2 Freq:0\.39±\\pm0\.03—MDev62,036GitHub Stars:48\.50±\\pm73\.75Top\-1 Freq:0\.28±\\pm0\.01Top\-2 Freq:0\.38±\\pm0\.03—Test5020,084GitHub Stars:48\.56±\\pm73\.64Top\-1 Freq:0\.28±\\pm0\.02Top\-2 Freq:0\.36±\\pm0\.04—Total5622,120GitHub Stars:48\.55±\\pm73\.65Top\-1 Freq:0\.28±\\pm0\.02Top\-2 Freq:0\.36±\\pm0\.04—

Table 18:Detailed per\-subset statistics for StarCoder in the pre\-training stage ofOLMo\-Detect\.NM: non\-member;M: member\.Dev: development set;Test: test set\.StageDomainSubsetSplitPartitionSamplesTokensQualityTemporalLexicalMid\-trainingStack ExchangeStack OverflowNMDev3718,489Question Vote:3\.49±\\pm2\.26Answer Vote:5\.14±\\pm2\.042015\-02\-06±\\pm1381\.39d0\.08Test331165,394Question Vote:3\.48±\\pm2\.24Answer Vote:5\.14±\\pm2\.052015\-02\-05±\\pm1379\.01dTotal368183,883Question Vote:3\.48±\\pm2\.25Answer Vote:5\.14±\\pm2\.052015\-02\-06±\\pm1379\.25dMDev3718,489Question Vote:3\.51±\\pm2\.23Answer Vote:5\.32±\\pm1\.952015\-02\-06±\\pm1329\.25dTest331165,471Question Vote:3\.53±\\pm1\.66Answer Vote:5\.58±\\pm1\.992015\-02\-06±\\pm1378\.98dTotal368183,960Question Vote:3\.52±\\pm1\.72Answer Vote:5\.55±\\pm1\.982015\-02\-06±\\pm1374\.07dMath Stack ExchangeNMDev63,271Question Vote:3\.33±\\pm1\.80Answer Vote:5\.00±\\pm1\.152016\-01\-22±\\pm1323\.12d0\.13Test5630,054Question Vote:3\.25±\\pm1\.81Answer Vote:4\.98±\\pm1\.642016\-02\-12±\\pm1329\.29dTotal6233,325Question Vote:3\.26±\\pm1\.80Answer Vote:4\.98±\\pm1\.602016\-02\-10±\\pm1328\.71dMDev63,356Question Vote:3\.83±\\pm1\.86Answer Vote:5\.67±\\pm1\.492016\-02\-03±\\pm1074\.89dTest5629,886Question Vote:3\.48±\\pm1\.81Answer Vote:5\.45±\\pm1\.672016\-01\-28±\\pm1302\.79dTotal6233,242Question Vote:3\.52±\\pm1\.82Answer Vote:5\.47±\\pm1\.652016\-01\-28±\\pm1282\.52d

Table 19:Detailed per\-subset statistics for Stack Exchange in the mid\-training stage ofOLMo\-Detect\.NM: non\-member;M: member\.Dev: development set;Test: test set\.StageDomainModelSubsetSplitPartitionSamplesTokensQualityTemporalLexicalPost\-trainingSFT1B, 32BWildchatNMDev716,404——0\.14Test64148,039——Total71164,443MDev716,438——Test64148,115——Total71164,553AyaNMDev145,293——0\.08Test14047,321——Total15452,614MDev145,258——Test14047,348——Total15452,6067B, 13BWildchatNMDev716,715——0\.18Test66148,305——Total73165,020MDev716,688——Test66148,362——Total73165,050AyaNMDev95,244——0\.06Test8246,860——Total9152,104MDev95,050——Test8246,818——Total9151,868

Table 20:Detailed per\-subset statistics for SFT in the post\-training stage ofOLMo\-Detect\.NM: non\-member;M: member\.Dev: development set;Test: test set\.StageDomainModelSubsetSplitPartitionSamplesTokensQualityTemporalLexicalPost\-trainingRLVR1B, 7B,13B, 32BCoT\-GSM8KNMDev1411,835——0\.01Test128107,609——Total142119,444MDev1411,796——Test128107,563——Total142119,359CoT\-MATHNMDev159,747——0\.01Test13387,919——Total14897,666MDev159,747——Test13387,919——Total14897,666

Table 21:Detailed per\-subset statistics for RLVR in the post\-training stage ofOLMo\-Detect\.NM: non\-member;M: member\.Dev: development set;Test: test set\.

## Appendix EDetailed Descriptions of Non\-Member Construction inOLMo\-Detect

This appendix details the construction of non\-members for each domain inOLMo\-Detect\.

### E\.1Source\-Matched Candidate Pools

##### Domains with Held\-Out Data\.

These domains have a natural held\-out split whose source already matches OLMo 2’s training data\.

Mid\-training:Math Mix, whose GSM8K subset has one held\-out file available,[test\-00000\-of\-00001\.parquet](https://huggingface.co/datasets/openai/gsm8k/blob/main/main/test-00000-of-00001.parquet)\. We additionally exclude the 200 instances designated by[OLMo et al\. \(2025\)](https://arxiv.org/html/2610.02986#bib.bib3)as a development set for evaluation\.

Post\-training:RLVR, whose held\-out files for CoT\-GSM8K and CoT\-MATH subsets can be constructed by prepending the same CoT exemplars used in OLMo 2’s RLVR setup \(8\-shot for GSM8K and 3\-shot for MATH\) to GSM8K’s[test\-00000\-of\-00001\.parquet](https://huggingface.co/datasets/openai/gsm8k/blob/main/main/test-00000-of-00001.parquet)and the test split of MATH’s[train\-00000\-of\-00001\-7320a6f3aba8ebd2\.parquet](https://huggingface.co/datasets/qwedsacf/competition_math/blob/main/data/train-00000-of-00001-7320a6f3aba8ebd2.parquet)\(separated from the training portion during preprocessing\), respectively\.

##### Domains with the Remainder of Original Source Corpora\.

These domains lack a natural held\-out split but are sampled from larger corpora\. Therefore, candidate non\-member pools are drawn from the portions of those original corpora that OLMo 2 did not use for training\.

Pre\-training: \(i\)DCLM\-Baseline\.DCLM\-Baseline is the top 10% quality subset of deduplicated DCLM\-RefinedWeb\. From the remainder of[DCLM\-RefinedWeb](https://data.commoncrawl.org/contrib/datacomp/DCLM-refinedweb/index.html), we randomly select three shards as candidate pools \(followed by deduplication\):global\-shard\_01\-local\-shard\_0,global\-shard\_05\-local\-shard\_6, andglobal\-shard\_07\-local\-shard\_3\. \(ii\)StarCoder\. OLMo 2’s StarCoder is a quality\-thresholded subset of the original StarCoder dataset\([Li et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib43)\), retaining only data from repositories with at least 2 GitHub stars, where the most\-frequent word constitutes at most 30% of tokens and the top\-2 most\-frequent words constitute at most 50%\. The top\-5 StarCoder subsets by training provenance are GitHub issues, Git commits, Markdown, Java, and Python, from which we select two distinct programming languages, Java and Python\. From the rejected remainder of these two subsets, we consider the following files as candidate pools: Java:[train\-00000\-of\-00087\.parquet](https://huggingface.co/datasets/bigcode/starcoderdata/blob/main/java/train-00000-of-00087.parquet)through[train\-00086\-of\-00087\.parquet](https://huggingface.co/datasets/bigcode/starcoderdata/blob/main/java/train-00086-of-00087.parquet); Python:[train\-00000\-of\-00059\.parquet](https://huggingface.co/datasets/bigcode/starcoderdata/blob/main/python/train-00000-of-00059.parquet)through[train\-00058\-of\-00059\.parquet](https://huggingface.co/datasets/bigcode/starcoderdata/blob/main/python/train-00058-of-00059.parquet)\.

Mid\-training:Stack Exchange\.OLMo 2’s Stack Exchange is a filtered version of[2024\-09\-30 dump](https://archive.org/details/stackexchange_20240930), retaining only data with an accepted answer, at least 3 question votes, and at least 5 answer votes\. The candidate non\-member pool is therefore the rejected remainder of the original 2024\-09\-30 dump, processed using the same pipeline as OLMo 2 to obtain matched Q&A format\.

Post\-training: \(i\)SFT\.Six subsets of OLMo 2’s SFT are randomly downsampled from larger upstream corpora: OpenAssistant, WildChat, SciRIFF, TableGPT, OpenMathInstruct 2, and Aya \(see Table[12](https://arxiv.org/html/2610.02986#A3.T12)for details\)\. Among them, we select two distinct subsets, WildChat and Aya\. The candidate pools for SFT are therefore the unused portions of original WildChat \([train\-00000\-of\-00014\.parquet](https://huggingface.co/datasets/allenai/WildChat-1M/blob/main/data/train-00000-of-00014.parquet)to[train\-00013\-of\-00014\.parquet](https://huggingface.co/datasets/allenai/WildChat-1M/blob/main/data/train-00013-of-00014.parquet)\) and original Aya \([test\-00000\-of\-00001\.parquet](https://huggingface.co/datasets/CohereLabs/aya_dataset/blob/main/data/test-00000-of-00001.parquet)and[train\-00000\-of\-00001\.parquet](https://huggingface.co/datasets/CohereLabs/aya_dataset/blob/main/data/train-00000-of-00001.parquet)\)\. \(ii\)DPO\.Five of OLMo 2’s DPO prompt subsets are randomly sampled from larger corpora: three WildChat variants \(Unused, Reused, and IF\), DaringAnteater, and Tülu 3 Personas IF \(see Table[13](https://arxiv.org/html/2610.02986#A3.T13)for details\)\. However, we only select WildChat \(Unused\) despite DPO’s multi\-subset nature\. This is because neither DaringAnteater nor Tülu 3 Personas IF yields sufficient non\-members after infini\-gram\-based 13\-gram filtering; meanwhile, the three WildChat variants are not independently selectable either: since all three draw from the same upstream WildChat corpus, they share a single non\-member remainder pool rather than three separate ones, and thus jointly count as one sampleable source\. We therefore retain WildChat \(Unused\) alone, considering the remaining parts of original WildChat not used for OLMo 2 training \([train\-00000\-of\-00014\.parquet](https://huggingface.co/datasets/allenai/WildChat-1M/blob/main/data/train-00000-of-00014.parquet)to[train\-00013\-of\-00014\.parquet](https://huggingface.co/datasets/allenai/WildChat-1M/blob/main/data/train-00013-of-00014.parquet)\) as candidate prompt pools\. We then apply OLMo 2’s UltraFeedback pipeline for response generation and preference annotation to obtain the source\-matched candidate pools, yielding the final subset named UltraFeedback\-WildChat \(abbreviated as UF\-Wildchat in the main paper\)\. \(iii\)RLVR\.The IFEval subset of OLMo 2’s RLVR is randomly sampled from the[Tülu 2 SFT mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture)\([Ivison et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib55)\)and augmented with verifiable instruction\-following constraints\. Its candidate pool is therefore the remainder of the Tülu 2 SFT mix, augmented via the same constraint\-construction pipeline\.

##### RLVR Subset Selection\.

As illustrated above, RLVR has three sampleable subsets: CoT\-GSM8K and CoT\-MATH, drawn from held\-out data, and IFEval, drawn from the remainder of the original source corpus\. IFEval would add the most within\-domain diversity, but we find that its non\-members cannot be aligned lexically with members\. Since alignment takes precedence over diversity, we select CoT\-GSM8K and CoT\-MATH, accepting that the two share a similar CoT construction\.

##### Excluded Domains\.

Pretraining’s Wikipedia/Wikibooks is excluded, as no source\-matched candidate non\-member pool is available\.

### E\.2Domain\-Dependent Sampling Strategies

For domains whose candidate pools originate from either held\-out files or the remainder of their source corpora without quality thresholding, sampling follows the random\-sampling procedure described in Section[4\.4](https://arxiv.org/html/2610.02986#S4.SS4)\. For quality\-thresholded domains, we use boundary sampling to obtain candidate non\-members whose quality is just below OLMo 2’s selection threshold\. Detailed descriptions are provided below\.

##### DCLM\-Baseline\.

We first identify the minimum fastText score in OLMo 2’s DCLM\-Baseline: 0\.01811\. Within the candidate pool, we then sample instances closest to this threshold, yielding over 350 candidate non\-members \(420K tokens\) with an average fastText score of around 0\.01809\.

##### StarCoder\.

From the candidate pool, we sample instances whose top\-1 word frequency is closest to 0\.3 and top\-2 word frequency is closest to 0\.5, while enforcing the same GitHub\-star constraint as OLMo 2, requiring each instance to have at least 2 stars\. This yields 56 candidate non\-members \(22K tokens\) for Java, with an average GitHub\-star count of 74 and top\-1/top\-2 word frequencies of 0\.30 and 0\.39, and 250 candidate non\-members \(300K tokens\) for Python, with an average GitHub\-star count of 28 and top\-1/top\-2 word frequencies of 0\.30 and 0\.37\. StarCoder is the only domain where the proportional allocation specified in Section[4\.4](https://arxiv.org/html/2610.02986#S4.SS4)cannot be met: although Java constitutes a larger share of OLMo 2’s training data than Python and would therefore be expected to contribute more tokens, too few Java candidates satisfy our filtering criteria to meet that target\. We therefore retain all available Java non\-members \(56 instances, 22K tokens\) and use Python to fill the remainder of the domain’s∼\\sim217K\-token budget\.

##### Stack Exchange\.

From the candidate pool, we sample instances satisfying either \(a\) question vote=2=\\text\{2\}and answer vote≥5\\geq\\text\{5\}, or \(b\) question vote≥3\\geq\\text\{3\}and answer vote=4=\\text\{4\}\. This yields over 400 candidate non\-members \(200K tokens\) for Stack Overflow, with an average question vote of 3\.48 and answer vote of 5\.14, and around 100 candidate non\-members \(50K tokens\) for Math Stack Exchange, with an average question vote of 3\.26 and answer vote of 4\.98\.

### E\.3Infini\-Gram\-Based 13\-Gram Filtering

After obtaining candidate non\-members for each domain, we perform 13\-gram filtering via infini\-gram to retain those with minimal training\-data overlap, as described in Section[4\.4](https://arxiv.org/html/2610.02986#S4.SS4)\. Here, we detail the index construction: we build an infini\-gram index over each OLMo 2 model’s full training corpus, defined as the union of all training\-stage documents with multiplicities reflecting epoch and model\-soup repetition\. The composition for each model size is summarized in Table[14](https://arxiv.org/html/2610.02986#A3.T14)\.

##### Excluded Domains\.

Although source\-matched candidate pools are identified for arXiv, using held\-out files[arXiv\_000\.jsonl\.zst](https://huggingface.co/datasets/EleutherAI/proof-pile-2/blob/main/arxiv/test/arXiv_000.jsonl.zst)through[arXiv\_099\.jsonl\.zst](https://huggingface.co/datasets/EleutherAI/proof-pile-2/blob/main/arxiv/test/arXiv_099.jsonl.zst)\(test\) and[arXiv\_000\.jsonl\.zst](https://huggingface.co/datasets/EleutherAI/proof-pile-2/blob/main/arxiv/validation/arXiv_000.jsonl.zst)through[arXiv\_099\.jsonl\.zst](https://huggingface.co/datasets/EleutherAI/proof-pile-2/blob/main/arxiv/validation/arXiv_099.jsonl.zst)\(validation\), and for Algebraic Stack, using held\-out files in its[test](https://huggingface.co/datasets/EleutherAI/proof-pile-2/tree/main/algebraic-stack/test)and[validation](https://huggingface.co/datasets/EleutherAI/proof-pile-2/tree/main/algebraic-stack/validation)directories, no candidates from either pool survive 13\-gram filtering\. Therefore, both arXiv and Algebraic Stack are excluded\.

### E\.4Token Budget Allocation

We sample from non\-members that survive 13\-gram filtering, following the rules in Section[4\.4](https://arxiv.org/html/2610.02986#S4.SS4)\. To ensure meaningful statistical analysis, each domain contains at least 100 non\-members, with at least 50 instances per subset in multi\-subset domains, to ensure meaningful statistical analysis\.

## Appendix FDetailed Descriptions of Member Construction inOLMo\-Detect

This appendix details the construction of members for each domain inOLMo\-Detect\.

### F\.1Candidate Pools

#### F\.1\.1Domains without Quality Thresholding

For the following domains, candidate pools are the corresponding training data used by OLMo 2\.

Mid\-training:Math Mix,whose only subset, GSM8K, has a candidate member pool drawn from its corresponding OLMo 2 training data,[0\.jsonl\.zst](https://huggingface.co/datasets/allenai/dolmino-mix-1124/blob/main/data/math/gsm8k/main/train/0.jsonl.zst)\.

Post\-training: \(i\)SFT\.Both of SFT’s subsets, WildChat and Aya, draw their candidate member pool from the corresponding OLMo 2 training data, filtered by source keyword:tulu\_v3\.9\_wildchat\_100kfor WildChat andtulu\_v3\.9\_aya\_100kfor Aya\. The 1B and 32B models draw shards 0–5 of[tulu\-3\-sft\-olmo\-2\-mixture\-0225](https://huggingface.co/datasets/allenai/tulu-3-sft-olmo-2-mixture-0225); the 7B and 13B models draw shards 0–5 of[tulu\-3\-sft\-olmo\-2\-mixture](https://huggingface.co/datasets/allenai/tulu-3-sft-olmo-2-mixture)\. \(ii\)DPO\.For each model size, candidate pools are drawn from OLMo 2’s corresponding DPO training data for DPO’s only subset, UltraFeedback\-WildChat, filtered using a model\-specific keyword: 1B draws shards 0–4 of[olmo\-2\-0425\-1b\-preference\-mix](https://huggingface.co/datasets/allenai/olmo-2-0425-1b-preference-mix)\(keyword:olmo2\-1b\-wildchat\-unused\-on\-policy\); 7B draws shards 0–5 of[olmo\-2\-1124\-7b\-preference\-mix](https://huggingface.co/datasets/allenai/olmo-2-1124-7b-preference-mix)\(keyword:wildchat\_v3\.9\_unused\_on\_policy\_olmo2\_7b\); 13B draws shards 0–5 of[olmo\-2\-1124\-13b\-preference\-mix](https://huggingface.co/datasets/allenai/olmo-2-1124-13b-preference-mix)\(keyword:wildchat\_v3\.9\_unused\_on\_policy\_olmo2\_13b\); and 32B draws shards 0–4 of[olmo\-2\-0325\-32b\-preference\-mix](https://huggingface.co/datasets/allenai/olmo-2-0325-32b-preference-mix)\(keyword:wildchat\_v3\.9\_unused\_off\_policy\_prompts\-olmo32\)\. \(iii\)RLVR\.The candidate pools for both the CoT\-GSM8K and CoT\-MATH subsets of RLVR originate from the single shard of[RLVR\-GSM\-MATH\-IF\-Mixed\-Constraints](https://huggingface.co/datasets/allenai/RLVR-GSM-MATH-IF-Mixed-Constraints), filtered using the keywordsgsm8kandMATH, respectively\.

#### F\.1\.2Quality\-Thresholded Domains

OLMo 2’s training data in the following domains has systematically higher quality than the corresponding non\-members, since only instances that passed OLMo 2’s quality threshold were selected for training\. To align text quality with non\-members, we select training data closest to the threshold as the candidate pool\.

##### DCLM\-Baseline\.

We restrict the candidate pool to the lowest\-scoring instances, those at the 10% quality threshold with a fastText score of 0\.01811, drawn from three shards of OLMo 2’s DCLM\-Baseline:global\-shard\_01\-local\-shard\_0,global\-shard\_05\-local\-shard\_6, andglobal\-shard\_07\-local\-shard\_3\. This yields a candidate member pool of over 300 instances \(580K tokens\)\.

StarCoder\.We restrict the candidate pool to instances whose top\-1 frequency lies within 0\.28–0\.3 or top\-2 frequency lies within 0\.48–0\.5\. Regarding GitHub stars, since candidates are drawn from OLMo 2’s StarCoder training data, the≥2\\geq 2GitHub\-star constraint, enforced for non\-members, is already satisfied by construction\. This yields a candidate member pool of over 8,000 instances \(7\.3M tokens\) for Java and over 5,500 instances \(15\.9M tokens\) for Python\.

Stack Exchange\.We restrict the candidate pool to instances with question vote=3=\\text\{3\}or answer vote=5=\\text\{5\}\. This yields a candidate member pool of over 300K instances \(200M tokens\) for Stack Overflow and over 30K instances \(19M tokens\) for Math Stack Exchange\.

### F\.2Alignment Axes

We align members and non\-members on three axes: text quality, temporal range, and lexical similarity, as discussed in Section[4\.5](https://arxiv.org/html/2610.02986#S4.SS5)\. Each axis is either a signal that existing MIAs are known to exploit or a by\-product of OLMo 2’s filtering that must therefore be controlled\. These three are not exhaustive\. Adding more axes is possible but carries a cost: each further constraint makes the simulated\-annealing search substantially harder and shrinks the feasible candidate pool, reducing subset sizes and therefore the reliability of the resulting statistical analysis\. We thus stop at the three most important axes, prioritizing subset size and statistical reliability over additional constraints\.

### F\.3Constrained Sampling via Simulated Annealing

This section provides a detailed description of the simulated\-annealing procedure used to sample members, introduced in Section[4\.5](https://arxiv.org/html/2610.02986#S4.SS5)\. The procedure is applied independently to each domain, or to each subset within multi\-subset domains, against its own corresponding non\-member set\.

##### Setup\.

Given a candidate member poolAAand a fixed non\-member split with token distributionpBp\_\{B\}, the goal is to select a subsetA′⊆AA^\{\\prime\}\\subseteq Asatisfying three constraints: \(i\)\|A′\|\|A^\{\\prime\}\|equals the non\-member instance count\|B\|\|B\|; \(ii\) the total token count ofA′A^\{\\prime\}is withinδ=250\\delta=250of the non\-member token count; and \(iii\) the distributional lossℒ\\mathcal\{L\}against the non\-member reference is minimized:

ℒ=JSD⁡\(pA′,pB\)\+𝟙​\[temporal metadata available\]⋅ℒtemp,\\mathcal\{L\}=\\mathrm\{JSD\}\(p\_\{A^\{\\prime\}\},p\_\{B\}\)\+\\mathds\{1\}\[\\text\{temporal metadata available\}\]\\cdot\\mathcal\{L\}\_\{\\text\{temp\}\},\(2\)wherepA′p\_\{A^\{\\prime\}\}is the token distribution ofA′A^\{\\prime\}, andJSD\\mathrm\{JSD\}denotes the Jensen–Shannon divergence, computed with base\-2 logarithms so thatJSD⁡\(pA′,pB\)∈\[0,1\]\\mathrm\{JSD\}\(p\_\{A^\{\\prime\}\},p\_\{B\}\)\\in\[0,1\], enforcing lexical alignment\. The temporal alignment term takes the form:

ℒtemp=\|μA′−μB\|\+\|σA′−σB\|σB,\\mathcal\{L\}\_\{\\text\{temp\}\}=\\frac\{\|\\mu\_\{A^\{\\prime\}\}\-\\mu\_\{B\}\|\+\|\\sigma\_\{A^\{\\prime\}\}\-\\sigma\_\{B\}\|\}\{\\sigma\_\{B\}\},\(3\)withμ\\muandσ\\sigmadenoting the mean and standard deviation of the temporal axis \(in days\), normalized by the non\-member standard deviationσB\\sigma\_\{B\}\.

##### Algorithm\.

We initialize by randomly partitioningAAinto\(A′,leftover\)\(A^\{\\prime\},\\text\{leftover\}\)such that\|A′\|\|A^\{\\prime\}\|matches the target instance count\|B\|\|B\|\. The algorithm then runs forTmax=107T\_\{\\max\}=10^\{7\}steps, with each step proposing a swap between one instance inA′A^\{\\prime\}and one in the leftover pool\. The procedure operates in two phases\. In thegreedy token\-matching phase, used when the current token count is outside theδ\\deltamargin, the swap is accepted only if it brings the token count closer to target\. In thealignment optimization phase, used once within margin, the swap is accepted if the post\-swap token count remains within margin and the swap satisfies the Metropolis criterion: accepted ifΔ​ℒ<0\\Delta\\mathcal\{L\}<0, otherwise accepted with probabilityexp\(−Δℒ/T\)\\exp\(\-\\Delta\\mathcal\{L\}/T\)\.

##### Temperature Schedule\.

The temperatureTTis initialized atT0=10−3T\_\{0\}=10^\{\-3\}and decays geometrically:T←0\.999995⋅TT\\leftarrow 0\.999995\\cdot Tafter each accepted swap in the alignment optimization phase\. This schedule provides broad exploration early in the search and progressive convergence as the temperature decreases\.

##### Reproducibility\.

All simulated\-annealing runs use a fixed random seed of 42, and the resulting subsetA∗A^\{\*\}is retained as the final member split\.

##### Excluded Domains\.

Mid\-training’s FLAN has sufficient candidate members and non\-members, but no member and non\-member pair can be matched on both instance count and total tokens, even with the token margin relaxed to 1,000\. We therefore exclude it fromOLMo\-Detect\.

Pre\-trainingMid\-trainingPost\-trainingDCLM\-BaselineOpenWebMathpeS2oStarCoderMath MixStack ExchangeSFTDPORLVRAvg1B4\.07±4\.878\.33±5\.693\.11±4\.019\.31±5\.246\.65±4\.775\.71±5\.346\.08±5\.242\.12±3\.012\.06±4\.555\.277B4\.07±4\.878\.33±5\.693\.11±4\.019\.31±5\.246\.66±4\.755\.71±5\.346\.54±5\.291\.98±3\.052\.06±4\.555\.3113B4\.07±4\.878\.33±5\.693\.11±4\.019\.31±5\.246\.65±4\.755\.71±5\.346\.55±5\.302\.52±3\.372\.06±4\.555\.3732B4\.07±4\.878\.33±5\.693\.11±4\.019\.31±5\.246\.64±4\.755\.71±5\.356\.12±5\.312\.20±3\.152\.06±4\.555\.28Avg4\.078\.333\.119\.316\.655\.716\.322\.202\.065\.31

Table 22:Per\-model\-size 13\-gram overlapη\\eta\(%; mean±std\{\}\_\{\\pm\\text\{std\}\}over non\-members\) for each domain against the training corpus of the corresponding OLMo 2 model size\. Multi\-subset domains are macro\-averaged across subsets \(Table[23](https://arxiv.org/html/2610.02986#A6.T23)\), with the domain\-level standard deviation computed as the root mean square of the subset\-level standard deviations\.DomainSubset1B7B13B32BAvgStarCoderPython7\.17±5\.287\.17±5\.287\.17±5\.287\.17±5\.287\.17Java11\.45±5\.2011\.45±5\.2011\.45±5\.2011\.45±5\.2011\.45Stack ExchangeMath Stack Exchange5\.67±4\.915\.67±4\.915\.67±4\.915\.68±4\.925\.67Stack Overflow5\.74±5\.745\.74±5\.745\.74±5\.755\.75±5\.745\.74SFTWildChat5\.16±4\.494\.53±4\.534\.55±4\.535\.14±4\.594\.84Aya7\.01±5\.898\.55±5\.958\.55±5\.967\.10±5\.957\.80DPOPrompt0\.00±0\.000\.13±1\.040\.00±0\.000\.01±0\.110\.04Chosen3\.55±3\.583\.06±3\.464\.20±4\.473\.73±4\.073\.63Rejected2\.80±3\.792\.75±3\.843\.37±3\.752\.85±3\.632\.94RLVRCoT\-GSM8K1\.45±3\.771\.45±3\.771\.45±3\.771\.45±3\.771\.45CoT\-MATH2\.67±5\.202\.67±5\.202\.67±5\.202\.67±5\.202\.67

Table 23:Per\-subset 13\-gram overlapη\\eta\(%; mean±std\{\}\_\{\\pm\\text\{std\}\}over non\-members\) for multi\-subset domains\. Domain\-level values are macro\-averaged across subsets; for DPO, they are averaged across the prompt, chosen, and rejected fields\.Pre\-trainingMid\-trainingPost\-trainingDCLM\-BaselineOpenWebMathpeS2oStarCoderMath MixStack ExchangeSFTDPORLVRAvg1B0\.490\.520\.480\.500\.520\.480\.510\.470\.510\.507B0\.490\.520\.480\.500\.520\.480\.520\.510\.510\.5013B0\.490\.520\.480\.500\.520\.480\.520\.490\.510\.5032B0\.490\.520\.480\.500\.520\.480\.510\.480\.510\.50Avg0\.490\.520\.480\.500\.520\.480\.510\.490\.510\.50

Table 24:Per\-model\-size blind detectability, measured by BoW classifier AUC\. Multi\-subset domains are macro\-averaged across subsets, with per\-subset AUCs reported in Table[25](https://arxiv.org/html/2610.02986#A6.T25)\.DomainSubset1B7B13B32BAvgStarCoderPython0\.480\.480\.480\.480\.48Java0\.510\.510\.510\.510\.51Stack ExchangeMath Stack Exchange0\.470\.470\.470\.470\.47Stack Overflow0\.490\.490\.490\.490\.49SFTWildChat0\.520\.540\.540\.520\.53Aya0\.500\.510\.510\.500\.50RLVRCoT\-GSM8K0\.510\.510\.510\.510\.51CoT\-MATH0\.510\.510\.510\.510\.51

Table 25:Per\-subset blind detectability for multi\-subset domains, measured by BoW classifier AUC\.

## Appendix GDetailed Per\-Domain Statistics of OLMo\-Detect \(Shifted\)

This appendix provides detailed statistics ofOLMo\-Detect \(Shifted\)\. Overall statistics are reported in Table[30](https://arxiv.org/html/2610.02986#A8.T30); detailed per\-domain statistics are reported in Tables[31](https://arxiv.org/html/2610.02986#A8.T31)–[33](https://arxiv.org/html/2610.02986#A8.T33); per\-subset statistics for multi\-subset domains, StarCoder, Stack Exchange, and SFT, are reported in Tables[34](https://arxiv.org/html/2610.02986#A8.T34)–[36](https://arxiv.org/html/2610.02986#A8.T36)\. Math Mix and RLVR are excluded, since random sampling does not yield members with meaningful distribution shifts in these domains\.

##### Analysis of Lexical Shift Across Domains\.

InOLMo\-Detect \(Shifted\), members are drawn at random rather than perturbed toward a target, so each domain shifts only along the axes on which its source corpus actually varies\. Web pages, academic papers, and Q&A posts remain lexically uniform across quality tiers and publication years, so DCLM\-Baseline, peS2o, and Stack Exchange shift on quality and temporal range instead: DCLM\-Baseline’s quality score rises from 0\.018 to 0\.131, peS2o’s mean date moves back by nearly nine years, and Stack Exchange’s question and answer votes roughly double, while their lexical values change by at most 0\.03\. By contrast, StarCoder, OpenWebMath, SFT, and DPO draw on corpora that vary lexically, through repository\-specific identifiers, heterogeneous web pages, and multi\-task multilingual data, and their lexical values rise two\- to fourfold, from 0\.09–0\.15 to 0\.32–0\.36\. In DPO, for instance, the dominant non\-English script changes from Chinese to Cyrillic\. Every domain is therefore shifted on at least one axis, and the shift is deliberately not maximized:it emulates the unaligned construction of existing benchmarks rather than a worst case\.

## Appendix HDetailed Descriptions of Experimental Settings

Table 26:Public repositories followed by our implementations of the compared MIAs\. The final three rows are the supervised methods; all others are unsupervised\.MethodHyperparameterSearch spaceSelected value1B7B13B32BMinKkk\{0\.05,…,1\.0\}\\\{0\.05,\\dots,1\.0\\\}0\.20\.2MinK\+\+kk\{0\.05,…,1\.0\}\\\{0\.05,\\dots,1\.0\\\}0\.20\.2DCPDDα\\alpha\{0\.001,…,0\.5\}\\\{0\.001,\\dots,0\.5\\\}0\.0010\.0010\.20\.20\.050\.050\.20\.2ReCaLL\# prefix shotsnn\{1,…,12\}\\\{1,\\dots,12\\\}101033771212CAMIApp\-value composition\{Edgington, Fisher, George\}GeorgeFisherGeorgeGeorgePAC\(k1,k2\)\(k\_\{1\},k\_\{2\}\)k1∈\{\.05,\.1,\.2\},k2∈\{\.1,\.2,\.3,\.4\}k\_\{1\}\\\!\\in\\\!\\\{\.05,\.1,\.2\\\},\\ k\_\{2\}\\\!\\in\\\!\\\{\.1,\.2,\.3,\.4\\\}\(0\.05,0\.10\)\(0\.05,\\,0\.10\)DCQscore type\{mean, min,κ\\kappa,κnp\\kappa\_\{\\mathrm\{np\}\}\}PcorrectP\_\{\\text\{correct\}\}minminκnp\\kappa\_\{\\mathrm\{np\}\}κ\\kappaNeigh\.score type\{dd,zz\}ddCDD\(α,L,r\)\(\\alpha,L,r\)α∈\{0,…,\.3\},L∈\{50,100,200\},r∈\{\.25,\.5,\.75\}\\alpha\\\!\\in\\\!\\\{0,\\dots,\.3\\\},L\\\!\\in\\\!\\\{50,100,200\\\},r\\\!\\in\\\!\\\{\.25,\.5,\.75\\\}\(\.025,50,\.5\)\(\.025,50,\.5\)\(\.075,50,\.5\)\(\.075,50,\.5\)\(\.15,50,\.25\)\(\.15,50,\.25\)\(\.3,50,\.25\)\(\.3,50,\.25\)Guidedscore type\{ROUGE\-guided, ROUGE\-Δ\\Delta\}ROUGE\-Δ\\DeltaROUGE\-Δ\\DeltaROUGE\-Δ\\DeltaROUGE\-guided

Table 27:Tunable hyperparameters, search spaces, and the values selected by development\-set AUC onOLMo\-Detect\.##### Implementation Details\.

We implement every compared method from its original public repository \(Table[26](https://arxiv.org/html/2610.02986#A8.T26)\)\. To ensure a fair comparison, every method is evaluated under an identical protocol with a batch size of11and a context length of4,0964\{,\}096tokens for the generative methods that require text generation \(Neigh\., Guided, DCQ, CDD, and SelfCrit\) and1,0241\{,\}024tokens for the remaining non\-generative methods\. All experiments are run on NVIDIA A100 \(80 GB\) GPUs, using two GPUs for the 32B model and a single GPU for 1B to 13B models\. Method\-specific hyperparameters, where applicable, are tuned on the held\-out development set; the search spaces and selected values are given in Table[27](https://arxiv.org/html/2610.02986#A8.T27)\. For reproducibility, all experiments use a fixed random seed of4242\.

##### Determinism and Statistical Reporting\.

We use a single random seed of4242because the majority of our evaluated methods are deterministic\. All non\-generative unsupervised methods \(PPL, Zlib, Lower, MinK, MinK\+\+, DCPDD, ReCaLL, CAMIA, EMMIA\) compute their statistics directly from token log\-probabilities, without any sampling\. Among the remaining methods: the two generative unsupervised methods, Guided and SelfCrit, decode greedily \(do\_sample=False, temperature00\)\. SelfCrit’s two responses are not sampled but produced from two different prompts, an initial answer and a follow\-up requesting an alternative solution, so greedy decoding yields two distinct yet deterministic outputs\. DCQ scores the target model with multiple\-choice log\-probabilities, which are likewise deterministic; its quiz options are generated once by an externalgpt\-4o\-minijudge and cached on disk, shared across all model sizes and runs, so the identical quiz is reused on every run\. Sup\-CAMIA fits a convex logistic\-regression classifier on deterministic features\. For all of these methods, the seed has no effect on the reported scores\.

The seed matters only for the methods that draw stochastic samples or train with random data ordering: Neigh\. \(neighbors sampled from a T5 mask\-filling model\), CDD \(completions sampled from the target model at temperature0\.80\.8\), PAC \(word\-level random\-swap augmentation\), and the two supervised fine\-tuning methods FSD \(a LoRA adapter\) and MIA\-Tuner \(a soft prompt\)\. Due to the computational cost of repeated runs, none of these works reports results across multiple seeds in their papers; we follow this convention and fix both the framework seed \(4242\) and each method’s own internal seed parameter where one exists \(e\.g\., PAC’s and Neigh\.’s\), holding their random draws, augmentations, and training runs fixed across runs\.

Since per\-instance scores are fixed under a single seed, we quantify the uncertainty from the finite evaluation set with a nonparametric bootstrap\. We resample members and non\-members with replacement within every subset, recompute the full subset\-to\-domain\-to\-stage aggregation on each of2,0002\{,\}000resamples, and report the2\.52\.5th and97\.597\.5th percentiles as the95%95\\%confidence interval, so the interval applies to the overall AUC rather than to any single evaluation\. This captures uncertainty from the finite evaluation set but not seed\-to\-seed variance for the five stochastic methods above; we leave a full multi\-seed analysis to future work\. Intervals are given in Tables[28](https://arxiv.org/html/2610.02986#A8.T28)and[29](https://arxiv.org/html/2610.02986#A8.T29)\.

MethodAUC95% CIPPL0\.65\[0\.64, 0\.67\]Zlib0\.68\[0\.67, 0\.70\]Lower0\.63\[0\.62, 0\.65\]MinK0\.66\[0\.64, 0\.67\]MinK\+\+0\.56\[0\.54, 0\.57\]DCPDD0\.67\[0\.65, 0\.68\]PAC0\.66\[0\.64, 0\.67\]ReCaLL0\.46\[0\.45, 0\.47\]EMMIA0\.57\[0\.56, 0\.58\]CAMIA0\.65\[0\.64, 0\.67\]Neigh\.0\.59\[0\.58, 0\.61\]Guided0\.49\[0\.48, 0\.50\]DCQ0\.49\[0\.48, 0\.51\]CDD0\.52\[0\.52, 0\.53\]Table 28:Bootstrap95%95\\%confidence intervals for the overall AUC of each unsupervised MIA onOLMo\-Detect, i\.e\. theAllcolumn of Table[4](https://arxiv.org/html/2610.02986#S4.T4)averaged over model sizes\. Members and non\-members are resampled with replacement within every subset and the full subset\-to\-domain\-to\-stage aggregation is recomputed on each of2,0002\{,\}000resamples, so the interval applies to the aggregate rather than to any single evaluation\. SelfCrit is omitted because it covers only post\-training\. Point estimates match Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.MethodSettingAUC95% CISup\-CAMIAIn\-domain0\.68\[0\.67, 0\.68\]Sup\-CAMIALODO0\.61\[0\.60, 0\.62\]MIA\-TunerIn\-domain0\.55\[0\.54, 0\.56\]MIA\-TunerLODO0\.49\[0\.48, 0\.50\]Table 29:Bootstrap95%95\\%confidence intervals for the overall AUC of the supervised MIAs onOLMo\-Detect, i\.e\. theAllcolumn of Table[5](https://arxiv.org/html/2610.02986#S5.T5)averaged over model sizes, computed as in Table[28](https://arxiv.org/html/2610.02986#A8.T28)\. FSD is omitted because it is reported as a gain over its base detector rather than as an absolute AUC\. Point estimates match Table[5](https://arxiv.org/html/2610.02986#S5.T5)\.
##### Continuous Scoring for Categorical Methods\.

DCQ\([Golchin and Surdeanu, 2025](https://arxiv.org/html/2610.02986#bib.bib15)\)and Guided\([Golchin and Surdeanu, 2023](https://arxiv.org/html/2610.02986#bib.bib7)\)originally produce categorical outputs that are not directly compatible with AUC computation\. For DCQ, we replace the original single\-letter output with the first\-token logprobs over\{A,B,C,D,E\}\\\{A,B,C,D,E\\\}from a single forward pass on the model under evaluation\. Each instance is presented through four bias compensator quiz \(BCQ\) prompts, with the original placed at slotsL∈\{A,B,C,D\}L\\in\\\{A,B,C,D\\\}respectively; the per\-slot scoreP⁡\(L∣BCQL\)P\(L\\mid\\mathrm\{BCQ\}\_\{L\}\)is the model’s recognition probability for that placement\. The final per\-instance score is the mean \(or minimum\) of these four probabilities, with the choice tuned per \(model, dataset\) on the dev set\. For Guided, we generate two greedy completions per instance: a*guided*completion \(prompted with the source corpus name\) and a*general*completion \(no source cited\)\. The per\-instance score is the ROUGE\-L F1 between the guided completion and the held\-out second piece of the original paragraph, or its difference against the general completion; the variant is selected per \(model, dataset\) on the dev set\. Both adaptations preserve the original methods’ computations and use the same underlying signals \(next\-token logprobs for DCQ; ROUGE\-L F1, the paper’s own metric, for Guided\); only the post\-processing step that maps these signals to a categorical decision is replaced with continuous score extraction\.

##### Excluded MIAs\.

Among existing MIAs that we have surveyed, four are excluded due to setting incompatibility: \(i\) methods that require access to a model other thanℳ\\mathcal\{M\}, whether an existing smaller model or one separately fine\-tuned, including Smaller Ref\([Carlini et al\., 2021](https://arxiv.org/html/2610.02986#bib.bib25)\)and SPV\-MIA\([Fu et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib28)\), since our setting assumes only grey\-box access toℳ\\mathcal\{M\}, optionally extended to fine\-tuningℳ\\mathcal\{M\}itself, not the construction of an additional model; \(ii\) narrowly focused methods, such as Name\-cloze\([Chang et al\., 2023](https://arxiv.org/html/2610.02986#bib.bib29)\), since it requires text containing proper nouns to mask; \(iii\) methods with no open\-source implementation, such as TS\-Guessing\([Deng et al\., 2024](https://arxiv.org/html/2610.02986#bib.bib30)\)\.

StageDomainSubsetsSplitSamplesTokensQualityTemporalLexicalPre\-trainingDCLM\-BaselineNon\-member150217,117FastText: 0\.01809±\\pm0\.000072018\-01\-19±\\pm930d0\.08Member150217,295FastText: 0\.13129±\\pm0\.182792017\-02\-21±\\pm1208dpeS2oNon\-member268216,981—2022\-12\-14±\\pm11d0\.13Member268217,039—2014\-02\-03±\\pm2657dOpenWebMathNon\-member151217,183—2019\-10\-13±\\pm853d0\.32Member151217,379—2011\-10\-01±\\pm2936dStarCoderPython,JavaNon\-member156217,156GitHub Stars:35\.65±\\pm66\.66Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.38±\\pm0\.05—0\.36Member156217,284GitHub Stars:10\.77±\\pm17\.94Top\-1 Freq:0\.09±\\pm0\.03Top\-2 Freq:0\.15±\\pm0\.04—Mid\-trainingStack ExchangeStack Overflow,Math Stack ExchangeNon\-member430217,208Question Vote:3\.45±\\pm2\.19Answer Vote:5\.12±\\pm1\.992015\-03\-31±\\pm1378d0\.08Member430217,209Question Vote:8\.12±\\pm7\.08Answer Vote:10\.02±\\pm6\.372013\-05\-31±\\pm1213dPost\-trainingSFT†WildChat,AyaNon\-member225217,057——0\.34Member225217,238——DPO†UF\-WildChatNon\-member113217,045——0\.32Member113217,154——

Table 30:Overall statistics ofOLMo\-Detect \(Shifted\), spanning seven of the nine domains ofOLMo\-Detect\(GSM8K and RLVR are excluded, as neither has any shiftable axis\)\. Non\-members are unchanged fromOLMo\-Detect, while members are randomly sampled subject only to matched instance and token counts, leaving TextQuality,TemporalRange, andLexicalSimilarity unaligned\. “—” denotes an axis that does not apply to a given domain, i\.e\., no quality thresholding is applied, or no timestamp metadata exists\.NM: non\-member;M: member\. TheSubsetscolumn lists constituent subsets; blank entries indicate single\-subset domains whose subset shares the domain’s name\.†SFT and DPO are constructed separately for each model size because their training data differ across sizes; the reported values correspond to the 1B model\.StageDomainSubsetSplitPartitionSamplesTokensQualityTemporalLexicalPre\-trainingDCLM\-BaselineNMDev1621,745FastText: 0\.01808±\\pm0\.000042018\-01\-18±\\pm930\.40d0\.08Test134195,372FastText: 0\.01809±\\pm0\.000082018\-01\-19±\\pm930\.19dTotal150217,117FastText: 0\.01809±\\pm0\.000072018\-01\-19±\\pm930\.21dMDev1621,745FastText: 0\.13136±\\pm0\.193092017\-02\-21±\\pm1204\.63dTest134195,550FastText: 0\.13128±\\pm0\.181522017\-02\-21±\\pm1208\.34dTotal150217,295FastText: 0\.13129±\\pm0\.182792017\-02\-21±\\pm1207\.94dpeS2oNMDev2721,793—2022\-12\-12±\\pm10\.45d0\.13Test241195,188—2022\-12\-14±\\pm11\.29dTotal268216,981—2022\-12\-14±\\pm11\.22dMDev2721,409—2014\-02\-03±\\pm2656\.90dTest241195,630—2014\-02\-03±\\pm2657\.15dTotal268217,039—2014\-02\-03±\\pm2657\.12dOpenWebMathNMDev1521,761—2019\-10\-13±\\pm850\.83d0\.32Test136195,422—2019\-10\-13±\\pm852\.94dTotal151217,183—2019\-10\-13±\\pm852\.73dMDev1521,820—2011\-10\-01±\\pm2943\.32dTest136195,559—2011\-10\-02±\\pm2935\.43dTotal151217,379—2011\-10\-01±\\pm2936\.22dStarCoderPython,JavaNMDev1621,536GitHub Stars:36\.19±\\pm67\.09Top\-1 Freq:0\.30±\\pm0\.01Top\-2 Freq:0\.37±\\pm0\.04—0\.36Test140195,620GitHub Stars:35\.59±\\pm66\.61Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.38±\\pm0\.05—Total156217,156GitHub Stars:35\.65±\\pm66\.66Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.38±\\pm0\.05—MDev1621,684GitHub Stars:10\.69±\\pm17\.33Top\-1 Freq:0\.09±\\pm0\.03Top\-2 Freq:0\.15±\\pm0\.03—Test140195,600GitHub Stars:10\.78±\\pm18\.00Top\-1 Freq:0\.09±\\pm0\.03Top\-2 Freq:0\.15±\\pm0\.04—Total156217,284GitHub Stars:10\.77±\\pm17\.94Top\-1 Freq:0\.09±\\pm0\.03Top\-2 Freq:0\.15±\\pm0\.04—

Table 31:Detailed per\-domain statistics for the pre\-training stage ofOLMo\-Detect \(Shifted\)\.NM: non\-member;M: member\.Dev: development set;Test: test set\. TheSubsetcolumn lists constituent subsets; blank entries indicate single\-subset domains whose subset shares the domain’s name\.StageDomainSubsetSplitPartitionSamplesTokensQualityTemporalLexicalMid\-trainingStack ExchangeStack Overflow,Math Stack ExchangeNMDev4321,760Question Vote:3\.47±\\pm2\.20Answer Vote:5\.12±\\pm1\.942015\-03\-27±\\pm1378\.74d0\.08Test387195,448Question Vote:3\.45±\\pm2\.19Answer Vote:5\.12±\\pm2\.002015\-03\-31±\\pm1378\.12dTotal430217,208Question Vote:3\.45±\\pm2\.19Answer Vote:5\.12±\\pm1\.992015\-03\-31±\\pm1378\.18dMDev4321,750Question Vote:8\.16±\\pm6\.92Answer Vote:10\.02±\\pm6\.412013\-06\-05±\\pm1206\.73dTest387195,459Question Vote:8\.12±\\pm7\.10Answer Vote:10\.02±\\pm6\.372013\-05\-31±\\pm1214\.10dTotal430217,209Question Vote:8\.12±\\pm7\.08Answer Vote:10\.02±\\pm6\.372013\-05\-31±\\pm1213\.37d

Table 32:Detailed per\-domain statistics for the mid\-training stage ofOLMo\-Detect \(Shifted\)\.NM: non\-member;M: member\.Dev: development set;Test: test set\. TheSubsetcolumn lists constituent subsets\.StageDomainSubsetModelSplitPartitionSamplesTokensQualityTemporalLexicalPost\-trainingSFTWildChat,Aya1B, 32BNMDev2121,697——0\.34Test204195,360——Total225217,057MDev2121,727——Test204195,511——Total225217,2387B, 13BNMDev1621,959——0\.34Test148195,165——Total164217,124MDev1621,736——Test148195,437——Total164217,173DPOUF\-WildChat1BNMDev1121,705——0\.32Test102195,340——Total113217,045MDev1121,554——Test102195,600——Total113217,1547BNMDev1021,704——0\.28Test100195,163——Total110216,867MDev1021,680——Test100195,345——Total110217,02513BNMDev1021,730——0\.31Test100195,321——Total110217,051MDev1021,720——Test100195,606——Total110217,32632BNMDev1021,838——0\.40Test100195,333——Total110217,171MDev1021,849——Test100195,320——Total110217,169

Table 33:Detailed per\-domain statistics for the post\-training stage ofOLMo\-Detect \(Shifted\)\.NM: non\-member;M: member\.Dev: development set;Test: test set\. TheSubsetcolumn lists constituent subsets\.StageDomainSubsetSplitPartitionSamplesTokensQualityTemporalLexicalPre\-trainingStarCoderPythonNMDev1019,364GitHub Stars:29\.50±\\pm61\.41Top\-1 Freq:0\.31±\\pm0\.01Top\-2 Freq:0\.36±\\pm0\.04—0\.37Test90175,594GitHub Stars:28\.53±\\pm59\.41Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.38±\\pm0\.06—Total100194,958GitHub Stars:28\.63±\\pm59\.62Top\-1 Freq:0\.30±\\pm0\.02Top\-2 Freq:0\.37±\\pm0\.05—MDev1019,295GitHub Stars:8\.60±\\pm17\.62Top\-1 Freq:0\.08±\\pm0\.02Top\-2 Freq:0\.14±\\pm0\.03—Test90175,746GitHub Stars:9\.02±\\pm18\.29Top\-1 Freq:0\.09±\\pm0\.03Top\-2 Freq:0\.15±\\pm0\.04—Total100195,041GitHub Stars:8\.98±\\pm18\.23Top\-1 Freq:0\.09±\\pm0\.03Top\-2 Freq:0\.15±\\pm0\.04—JavaNMDev62,172GitHub Stars:47\.33±\\pm74\.29Top\-1 Freq:0\.30±\\pm0\.01Top\-2 Freq:0\.38±\\pm0\.03—0\.48Test5020,026GitHub Stars:48\.28±\\pm76\.28Top\-1 Freq:0\.30±\\pm0\.01Top\-2 Freq:0\.39±\\pm0\.03—Total5622,198GitHub Stars:48\.18±\\pm76\.07Top\-1 Freq:0\.30±\\pm0\.01Top\-2 Freq:0\.39±\\pm0\.03—MDev62,389GitHub Stars:14\.17±\\pm16\.25Top\-1 Freq:0\.11±\\pm0\.03Top\-2 Freq:0\.17±\\pm0\.02—Test5019,854GitHub Stars:13\.94±\\pm17\.02Top\-1 Freq:0\.09±\\pm0\.03Top\-2 Freq:0\.15±\\pm0\.04—Total5622,243GitHub Stars:13\.96±\\pm16\.94Top\-1 Freq:0\.09±\\pm0\.03Top\-2 Freq:0\.15±\\pm0\.04—

Table 34:Detailed per\-subset statistics for StarCoder inOLMo\-Detect \(Shifted\)\.NM: non\-member;M: member\.Dev: development set;Test: test set\.StageDomainSubsetSplitPartitionSamplesTokensQualityTemporalLexicalMid\-trainingStack ExchangeStack OverflowNMDev3718,489Question Vote:3\.49±\\pm2\.26Answer Vote:5\.14±\\pm2\.042015\-02\-06±\\pm1381\.39d0\.09Test331165,394Question Vote:3\.48±\\pm2\.24Answer Vote:5\.14±\\pm2\.052015\-02\-05±\\pm1379\.01dTotal368183,883Question Vote:3\.48±\\pm2\.25Answer Vote:5\.14±\\pm2\.052015\-02\-06±\\pm1379\.25dMDev3718,488Question Vote:8\.00±\\pm6\.70Answer Vote:10\.00±\\pm6\.582013\-07\-30±\\pm1269\.85dTest331165,394Question Vote:8\.00±\\pm6\.98Answer Vote:10\.00±\\pm6\.552013\-07\-25±\\pm1281\.27dTotal368183,882Question Vote:8\.00±\\pm6\.95Answer Vote:10\.00±\\pm6\.552013\-07\-26±\\pm1280\.13dMath Stack ExchangeNMDev63,271Question Vote:3\.33±\\pm1\.80Answer Vote:5\.00±\\pm1\.152016\-01\-22±\\pm1323\.12d0\.14Test5630,054Question Vote:3\.25±\\pm1\.81Answer Vote:4\.98±\\pm1\.642016\-02\-12±\\pm1329\.29dTotal6233,325Question Vote:3\.26±\\pm1\.80Answer Vote:4\.98±\\pm1\.602016\-02\-10±\\pm1328\.71dMDev63,262Question Vote:9\.17±\\pm8\.07Answer Vote:10\.17±\\pm5\.212012\-06\-28±\\pm596\.92dTest5630,065Question Vote:8\.84±\\pm7\.73Answer Vote:10\.12±\\pm5\.192012\-07\-05±\\pm596\.75dTotal6233,327Question Vote:8\.87±\\pm7\.76Answer Vote:10\.13±\\pm5\.192012\-07\-04±\\pm596\.77d

Table 35:Detailed per\-subset statistics for Stack Exchange inOLMo\-Detect \(Shifted\)\.NM: non\-member;M: member\.Dev: development set;Test: test set\.StageDomainModelSubsetSplitPartitionSamplesTokensQualityTemporalLexicalPost\-trainingSFT1B, 32BWildchatNMDev716,404——0\.43Test64148,039——Total71164,443MDev716,388——Test64148,120——Total71164,508AyaNMDev145,293——0\.39Test14047,321——Total15452,614MDev145,339——Test14047,391——Total15452,7307B, 13BWildchatNMDev716,715——0\.41Test66148,305——Total73165,020MDev716,492——Test66148,578——Total73165,070AyaNMDev95,244——0\.43Test8246,860——Total9152,104MDev95,244——Test8246,859——Total9152,103

Table 36:Detailed per\-subset statistics for SFT inOLMo\-Detect \(Shifted\)\.NM: non\-member;M: member\.Dev: development set;Test: test set\.

## Appendix IPer\-Domain AUC Results for Unsupervised MIAs onOLMo\-Detect

This appendix provides detailed AUC results for unsupervised MIAs onOLMo\-Detectfor each domain, complementing the stage\-level aggregates in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\. Per\-domain AUC results are shown in Tables[37](https://arxiv.org/html/2610.02986#A9.T37)–[49](https://arxiv.org/html/2610.02986#A9.T49)\.

ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.600\.590\.600\.570\.460\.570\.610\.460\.520\.600\.640\.540\.500\.500\.557B0\.640\.640\.620\.610\.530\.640\.650\.500\.540\.650\.610\.490\.430\.490\.5713B0\.650\.650\.630\.630\.520\.640\.680\.620\.530\.660\.620\.470\.470\.510\.5932B0\.660\.670\.660\.640\.610\.650\.690\.530\.500\.640\.610\.470\.640\.510\.61Model\-Avg0\.640\.640\.630\.610\.530\.620\.660\.530\.520\.640\.620\.490\.510\.500\.58

Table 37:Comparison of unsupervised MIAs on the DCLM\-Baseline domain ofOLMo\-Detect, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.470\.490\.480\.500\.510\.520\.480\.510\.630\.500\.490\.440\.490\.500\.507B0\.470\.490\.480\.480\.490\.510\.470\.550\.700\.470\.510\.520\.520\.500\.5113B0\.480\.500\.500\.490\.500\.510\.510\.510\.700\.470\.510\.510\.540\.500\.5232B0\.490\.510\.500\.510\.480\.520\.510\.520\.710\.470\.530\.440\.640\.490\.52Model\-Avg0\.480\.500\.490\.500\.500\.510\.490\.520\.690\.480\.510\.480\.550\.500\.51

Table 38:Comparison of unsupervised MIAs on the peS2o domain ofOLMo\-Detect, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.580\.650\.600\.600\.490\.650\.690\.600\.530\.550\.530\.430\.590\.520\.577B0\.600\.650\.620\.610\.500\.670\.680\.600\.530\.540\.530\.550\.460\.540\.5813B0\.600\.640\.590\.600\.500\.680\.700\.600\.520\.600\.520\.450\.540\.560\.5832B0\.600\.630\.580\.620\.480\.690\.700\.560\.640\.610\.520\.580\.450\.540\.59Model\-Avg0\.590\.640\.600\.610\.490\.670\.690\.590\.560\.580\.530\.500\.510\.540\.58

Table 39:Comparison of unsupervised MIAs on the StarCoder domain ofOLMo\-Detect, evaluated via AUC \(macro\-averaged over its subsets\)\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.510\.560\.610\.550\.480\.610\.540\.440\.540\.560\.540\.470\.520\.510\.537B0\.510\.570\.590\.550\.500\.570\.560\.490\.530\.460\.520\.420\.240\.480\.5013B0\.520\.580\.610\.560\.530\.630\.580\.480\.520\.450\.550\.440\.420\.490\.5332B0\.570\.640\.590\.600\.590\.650\.600\.440\.600\.430\.550\.430\.510\.480\.55Model\-Avg0\.530\.590\.600\.560\.520\.610\.570\.460\.550\.470\.540\.440\.420\.490\.53

Table 40:Comparison of unsupervised MIAs on the OpenWebMath domain ofOLMo\-Detect, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.730\.760\.720\.760\.570\.690\.720\.320\.500\.750\.640\.490\.460\.520\.627B0\.850\.890\.830\.860\.520\.910\.830\.050\.510\.880\.720\.500\.510\.550\.6713B0\.980\.990\.930\.990\.580\.990\.960\.010\.480\.990\.890\.480\.530\.680\.7532B0\.980\.990\.930\.970\.510\.980\.920\.020\.500\.990\.910\.550\.510\.740\.75Model\-Avg0\.890\.910\.850\.890\.550\.890\.860\.100\.500\.900\.790\.510\.500\.620\.70

Table 41:Comparison of unsupervised MIAs on the GSM8K subset ofOLMo\-Detect’s Math Mix domain, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.550\.590\.500\.540\.490\.530\.590\.540\.560\.580\.550\.540\.540\.510\.547B0\.580\.640\.550\.600\.560\.610\.630\.530\.570\.590\.560\.450\.400\.530\.5613B0\.600\.670\.560\.620\.610\.670\.660\.660\.650\.610\.580\.490\.640\.520\.6132B0\.630\.700\.530\.650\.610\.680\.690\.730\.690\.640\.600\.540\.640\.500\.63Model\-Avg0\.590\.650\.530\.600\.570\.620\.640\.610\.620\.610\.570\.510\.550\.510\.59

Table 42:Comparison of unsupervised MIAs on the Stack Exchange domain ofOLMo\-Detect, evaluated via AUC \(macro\-averaged over its subsets\)\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.620\.640\.490\.630\.730\.560\.510\.610\.570\.490\.540\.480\.510\.510\.567B0\.730\.700\.550\.710\.770\.700\.590\.740\.590\.740\.600\.510\.430\.500\.6313B0\.650\.610\.500\.640\.700\.600\.550\.650\.550\.570\.510\.540\.440\.510\.5732B0\.640\.640\.500\.650\.690\.600\.510\.690\.550\.540\.520\.440\.450\.500\.57Model\-Avg0\.660\.650\.510\.660\.720\.610\.540\.670\.560\.580\.540\.490\.460\.510\.58

Table 43:Comparison of unsupervised MIAs on the SFT domain ofOLMo\-Detect, evaluated via AUC \(macro\-averaged over its subsets\)\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDSelfCrit†1B0\.440\.550\.640\.440\.450\.470\.620\.540\.600\.590\.540\.530\.450\.530\.430\.527B0\.410\.390\.500\.400\.400\.400\.430\.480\.500\.600\.460\.480\.300\.530\.530\.4513B0\.380\.470\.640\.380\.450\.390\.530\.590\.490\.570\.490\.450\.500\.530\.430\.4932B0\.460\.530\.620\.480\.470\.480\.620\.650\.390\.550\.600\.470\.470\.480\.550\.52Model\-Avg0\.420\.480\.600\.430\.440\.430\.550\.570\.490\.580\.520\.480\.430\.520\.480\.50

Table 44:Comparison of unsupervised MIAs on the UF\-Wildchat subset ofOLMo\-Detect’s DPO domain, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.†SelfCrit is evaluated only on DPO and RLVR\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDSelfCrit†1B0\.770\.790\.640\.790\.600\.740\.690\.360\.660\.680\.510\.500\.430\.480\.500\.617B0\.890\.880\.800\.900\.690\.870\.770\.250\.730\.820\.600\.530\.480\.480\.480\.6813B0\.950\.940\.810\.950\.620\.920\.750\.180\.760\.850\.580\.470\.500\.460\.460\.6832B0\.940\.910\.900\.940\.720\.920\.770\.280\.470\.890\.600\.470\.500\.490\.410\.68Model\-Avg0\.890\.880\.790\.900\.660\.860\.750\.270\.650\.810\.570\.490\.480\.480\.460\.66

Table 45:Comparison of unsupervised MIAs on the RLVR domain ofOLMo\-Detect, evaluated via AUC \(macro\-averaged over its subsets\)\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.†SelfCrit is evaluated only on DPO and RLVR\.SubsetModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDPython1B0\.540\.610\.580\.550\.500\.610\.690\.520\.490\.580\.520\.380\.570\.520\.557B0\.550\.620\.600\.560\.530\.610\.680\.510\.500\.580\.520\.480\.510\.510\.5613B0\.550\.620\.580\.560\.520\.620\.670\.560\.510\.620\.510\.480\.540\.520\.5632B0\.540\.600\.550\.550\.530\.610\.660\.530\.620\.600\.530\.560\.490\.560\.57Model\-Avg0\.540\.620\.580\.550\.520\.610\.680\.530\.530\.600\.520\.480\.530\.530\.56Java1B0\.620\.680\.610\.650\.470\.690\.700\.680\.570\.510\.540\.470\.610\.510\.597B0\.640\.680\.630\.650\.480\.730\.690\.680\.560\.500\.550\.630\.410\.570\.6013B0\.640\.650\.610\.650\.470\.740\.720\.640\.530\.580\.540\.430\.540\.600\.6032B0\.660\.660\.620\.690\.430\.770\.730\.600\.660\.620\.510\.600\.400\.510\.60Model\-Avg0\.640\.670\.620\.660\.460\.730\.710\.650\.580\.550\.530\.530\.490\.550\.60

Table 46:Comparison of unsupervised MIAs on the subsets of the StarCoder domain ofOLMo\-Detect, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.SubsetModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDMath Stack Exchange1B0\.550\.600\.490\.540\.490\.550\.620\.570\.580\.600\.570\.600\.590\.520\.567B0\.570\.630\.530\.580\.550\.590\.670\.510\.620\.610\.570\.410\.360\.540\.5513B0\.600\.670\.550\.620\.620\.680\.710\.630\.690\.640\.600\.450\.630\.510\.6132B0\.620\.710\.510\.650\.610\.700\.710\.790\.740\.660\.610\.570\.590\.500\.64Model\-Avg0\.590\.650\.520\.600\.570\.630\.680\.630\.660\.630\.590\.510\.540\.520\.59Stack Overflow1B0\.560\.580\.500\.540\.490\.520\.550\.510\.550\.570\.540\.490\.490\.490\.537B0\.590\.640\.560\.610\.570\.630\.600\.540\.510\.560\.550\.490\.430\.510\.5613B0\.600\.670\.560\.630\.600\.660\.610\.700\.600\.590\.570\.520\.650\.520\.6032B0\.630\.700\.550\.650\.610\.670\.660\.670\.630\.630\.580\.520\.680\.490\.62Model\-Avg0\.600\.650\.550\.610\.570\.620\.600\.600\.570\.590\.560\.500\.560\.500\.58

Table 47:Comparison of unsupervised MIAs on the subsets of the Stack Exchange domain ofOLMo\-Detect, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.SubsetModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDAya1B0\.560\.580\.500\.540\.670\.540\.500\.480\.580\.540\.540\.490\.480\.500\.537B0\.770\.700\.530\.720\.760\.720\.670\.690\.590\.760\.690\.520\.470\.530\.6513B0\.690\.600\.490\.660\.640\.620\.600\.560\.510\.670\.590\.510\.490\.550\.5932B0\.550\.560\.490\.540\.590\.550\.490\.600\.440\.500\.510\.380\.510\.510\.52Model\-Avg0\.640\.610\.510\.610\.660\.610\.570\.580\.530\.620\.580\.470\.490\.520\.57WildChat1B0\.690\.700\.480\.710\.790\.580\.510\.750\.550\.440\.540\.470\.540\.520\.597B0\.700\.710\.570\.700\.790\.690\.520\.790\.590\.730\.500\.490\.390\.480\.6213B0\.620\.620\.510\.620\.760\.570\.500\.740\.600\.460\.430\.570\.390\.470\.5632B0\.720\.720\.510\.760\.790\.650\.520\.770\.660\.570\.530\.510\.380\.500\.61Model\-Avg0\.680\.690\.520\.700\.780\.620\.510\.760\.600\.550\.500\.510\.430\.490\.60

Table 48:Comparison of unsupervised MIAs on the subsets of the SFT domain ofOLMo\-Detect, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.SubsetModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDSelfCrit†CoT\-GSM8K1B0\.850\.890\.730\.880\.620\.800\.780\.210\.800\.750\.490\.510\.420\.510\.470\.657B0\.980\.990\.870\.990\.780\.980\.890\.100\.970\.980\.620\.510\.500\.530\.500\.7513B1\.001\.000\.901\.000\.640\.990\.780\.011\.000\.980\.560\.480\.410\.490\.390\.7132B1\.001\.000\.951\.000\.791\.000\.870\.040\.500\.990\.610\.490\.510\.500\.350\.71Model\-Avg0\.960\.970\.860\.970\.710\.940\.830\.090\.820\.930\.570\.500\.460\.510\.430\.70CoT\-MATH1B0\.700\.680\.540\.710\.580\.690\.600\.510\.520\.610\.520\.500\.430\.460\.530\.577B0\.800\.770\.740\.810\.610\.750\.660\.410\.490\.660\.580\.540\.460\.430\.460\.6113B0\.900\.870\.720\.910\.600\.860\.730\.360\.510\.710\.600\.460\.580\.440\.520\.6532B0\.870\.820\.860\.880\.650\.850\.680\.520\.430\.790\.580\.440\.500\.480\.470\.65Model\-Avg0\.820\.790\.720\.830\.610\.790\.670\.450\.490\.690\.570\.490\.490\.450\.500\.62

Table 49:Comparison of unsupervised MIAs on the subsets of the RLVR domain ofOLMo\-Detect, evaluated via AUC\. Methods are grouped by family as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.†SelfCrit is evaluated only on DPO and RLVR\.
## Appendix JTPR@5% FPR Results for Unsupervised MIAs onOLMo\-Detect

This appendix provides detailed per\-domain TPR@5% FPR results for unsupervised MIAs onOLMo\-Detect\. Stage\-level aggregate results are reported in Table[50](https://arxiv.org/html/2610.02986#A10.T50), while per\-domain results are shown in Tables[51](https://arxiv.org/html/2610.02986#A10.T51)–[63](https://arxiv.org/html/2610.02986#A10.T63)\.

ModelStageLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDSelfCrit†1BPre0\.070\.090\.080\.090\.070\.120\.120\.070\.100\.090\.050\.030\.070\.06—0\.08Mid0\.140\.170\.150\.150\.090\.100\.140\.070\.090\.150\.100\.060\.070\.06—0\.11Post0\.200\.180\.120\.200\.150\.120\.130\.100\.090\.170\.050\.050\.040\.070\.070\.12All0\.130\.150\.120\.150\.100\.110\.130\.080\.090\.140\.070\.050\.060\.060\.070\.107BPre0\.080\.130\.100\.110\.090\.150\.130\.080\.180\.120\.060\.050\.050\.06—0\.10Mid0\.240\.330\.150\.280\.100\.390\.240\.020\.070\.270\.160\.050\.050\.07—0\.17Post0\.360\.320\.180\.340\.230\.300\.150\.090\.220\.270\.070\.050\.030\.070\.040\.18All0\.230\.260\.140\.240\.140\.280\.170\.060\.160\.220\.100\.050\.040\.060\.040\.1513BPre0\.100\.130\.090\.110\.070\.130\.170\.060\.190\.100\.080\.040\.050\.06—0\.10Mid0\.490\.570\.330\.510\.140\.550\.490\.050\.070\.500\.310\.060\.050\.11—0\.30Post0\.340\.340\.260\.330\.120\.280\.140\.150\.220\.280\.070\.050\.060\.080\.060\.19All0\.310\.350\.220\.310\.110\.320\.260\.090\.160\.300\.150\.050\.050\.080\.060\.1932BPre0\.120\.160\.070\.120\.100\.160\.200\.060\.200\.120\.080\.040\.070\.06—0\.11Mid0\.490\.580\.390\.490\.130\.560\.460\.080\.120\.520\.360\.110\.100\.14—0\.32Post0\.350\.310\.190\.370\.180\.320\.170\.130\.040\.310\.090\.040\.050\.070\.070\.18All0\.320\.350\.220\.320\.140\.350\.280\.090\.120\.310\.180\.060\.070\.090\.070\.20Model\-AvgPre0\.090\.130\.090\.100\.080\.140\.150\.070\.170\.110\.070\.040\.060\.06—0\.10Mid0\.340\.410\.260\.360\.110\.400\.330\.050\.090\.360\.240\.070\.070\.10—0\.23Post0\.310\.290\.190\.310\.170\.250\.150\.120\.140\.260\.070\.050\.040\.070\.060\.17All0\.250\.280\.180\.260\.120\.260\.210\.080\.130\.240\.120\.050\.060\.070\.060\.16

Table 50:Comparison of unsupervised MIAs onOLMo\-Detect, evaluated via TPR@5% FPR\. Methods are grouped by family: likelihood\-based, perturbation\-based \(Pert\.\), prompt\-based and output\-distribution\-based \(Out\.\-Dist\.\)\.Pre/Mid/Postreport macro\-averaged TPRs over the domains of each stage, whileAllmacro\-averages the three stages\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.†SelfCrit is evaluated only on DPO and RLVR and thus has onlyPostandAllvalues, averaged over these two domains\. Per\-domain results are provided in Tables[51](https://arxiv.org/html/2610.02986#A10.T51)–[63](https://arxiv.org/html/2610.02986#A10.T63)\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.090\.060\.140\.130\.010\.120\.160\.030\.060\.130\.090\.040\.030\.060\.087B0\.150\.110\.160\.210\.060\.180\.200\.030\.070\.220\.080\.060\.040\.040\.1213B0\.160\.130\.140\.200\.050\.150\.260\.060\.080\.230\.100\.040\.030\.060\.1232B0\.210\.180\.130\.220\.110\.180\.340\.060\.120\.260\.110\.040\.070\.070\.15Model\-Avg0\.150\.120\.140\.190\.060\.160\.240\.040\.080\.210\.100\.050\.040\.060\.12

Table 51:Comparison of unsupervised MIAs on the DCLM\-Baseline domain ofOLMo\-Detect, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.060\.070\.050\.090\.070\.070\.060\.050\.180\.060\.040\.050\.040\.050\.077B0\.070\.070\.060\.070\.070\.070\.070\.050\.470\.090\.050\.030\.100\.050\.0913B0\.070\.070\.080\.070\.050\.090\.100\.050\.440\.050\.070\.050\.070\.050\.0932B0\.080\.090\.080\.100\.080\.100\.090\.070\.480\.050\.060\.050\.110\.040\.10Model\-Avg0\.070\.070\.070\.080\.070\.080\.080\.050\.390\.060\.050\.050\.080\.050\.09

Table 52:Comparison of unsupervised MIAs on the peS2o domain ofOLMo\-Detect, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.080\.170\.070\.080\.130\.160\.170\.170\.090\.110\.050\.010\.160\.070\.117B0\.080\.270\.100\.080\.080\.230\.220\.220\.100\.110\.040\.050\.060\.120\.1313B0\.090\.240\.080\.090\.070\.200\.270\.090\.110\.100\.070\.030\.010\.070\.1132B0\.080\.190\.040\.080\.030\.240\.260\.080\.100\.120\.060\.040\.060\.100\.11Model\-Avg0\.080\.220\.070\.080\.080\.210\.230\.140\.100\.110\.060\.030\.070\.090\.11

Table 53:Comparison of unsupervised MIAs on the StarCoder domain ofOLMo\-Detect, evaluated via TPR@5% FPR \(macro\-averaged over its subsets\)\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.040\.070\.080\.060\.080\.120\.070\.010\.090\.070\.040\.030\.050\.070\.067B0\.040\.070\.070\.060\.140\.120\.020\.010\.090\.060\.070\.040\.000\.040\.0613B0\.060\.100\.040\.060\.110\.100\.040\.040\.150\.040\.070\.030\.080\.030\.0732B0\.100\.170\.040\.070\.150\.130\.110\.030\.100\.040\.070\.020\.040\.020\.08Model\-Avg0\.060\.100\.060\.060\.120\.120\.060\.030\.110\.050\.060\.030\.040\.040\.07

Table 54:Comparison of unsupervised MIAs on the OpenWebMath domain ofOLMo\-Detect, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.210\.250\.170\.250\.140\.120\.160\.030\.040\.220\.110\.040\.040\.070\.137B0\.430\.540\.210\.480\.130\.600\.370\.000\.050\.490\.220\.050\.060\.070\.2613B0\.920\.970\.540\.930\.210\.940\.780\.000\.030\.950\.530\.050\.040\.170\.5032B0\.890\.960\.610\.840\.140\.920\.670\.000\.060\.960\.630\.120\.060\.240\.51Model\-Avg0\.610\.680\.380\.630\.160\.650\.490\.010\.050\.660\.370\.060\.050\.140\.35

Table 55:Comparison of unsupervised MIAs on the GSM8K subset ofOLMo\-Detect’s Math Mix domain, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.060\.080\.120\.050\.030\.080\.120\.100\.130\.070\.100\.070\.100\.060\.087B0\.060\.120\.090\.080\.060\.180\.120\.030\.090\.060\.100\.050\.050\.060\.0813B0\.070\.170\.130\.080\.070\.160\.190\.100\.100\.050\.100\.070\.060\.050\.1032B0\.090\.200\.170\.130\.120\.210\.250\.160\.170\.070\.100\.100\.140\.050\.14Model\-Avg0\.070\.140\.130\.080\.070\.160\.170\.100\.120\.060\.100\.070\.090\.050\.10

Table 56:Comparison of unsupervised MIAs on the Stack Exchange domain ofOLMo\-Detect, evaluated via TPR@5% FPR \(macro\-averaged over its subsets\)\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDD1B0\.220\.140\.090\.210\.330\.120\.080\.120\.040\.150\.070\.050\.050\.050\.127B0\.400\.290\.130\.360\.500\.280\.170\.170\.070\.220\.060\.060\.020\.060\.2013B0\.190\.140\.070\.180\.260\.100\.060\.110\.060\.140\.090\.100\.070\.080\.1232B0\.280\.150\.070\.320\.330\.190\.050\.170\.070\.210\.070\.080\.000\.080\.15Model\-Avg0\.270\.180\.090\.270\.360\.170\.090\.140\.060\.180\.070\.070\.040\.070\.15

Table 57:Comparison of unsupervised MIAs on the SFT domain ofOLMo\-Detect, evaluated via TPR@5% FPR \(macro\-averaged over its subsets\)\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDSelfCrit†1B0\.070\.060\.140\.080\.040\.070\.120\.160\.080\.220\.040\.040\.030\.100\.120\.097B0\.020\.030\.070\.010\.020\.010\.010\.070\.090\.070\.040\.050\.000\.100\.040\.0413B0\.030\.060\.280\.040\.040\.030\.060\.330\.100\.080\.030\.030\.060\.110\.060\.0932B0\.040\.090\.090\.060\.070\.040\.130\.190\.010\.060\.030\.030\.070\.070\.110\.07Model\-Avg0\.040\.060\.150\.050\.040\.040\.080\.190\.070\.110\.030\.040\.040\.090\.080\.07

Table 58:Comparison of unsupervised MIAs on the UF\-Wildchat subset ofOLMo\-Detect’s DPO domain, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.†SelfCrit is evaluated only on DPO and RLVR\.ModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDSelfCrit†1B0\.290\.350\.120\.330\.080\.160\.190\.020\.150\.140\.030\.060\.040\.040\.020\.147B0\.660\.650\.330\.660\.160\.600\.250\.020\.510\.530\.110\.040\.050\.030\.050\.3113B0\.820\.810\.410\.780\.060\.710\.300\.000\.520\.620\.110\.020\.050\.050\.050\.3632B0\.710\.680\.430\.740\.150\.710\.340\.030\.040\.670\.170\.030\.060\.050\.020\.32Model\-Avg0\.620\.620\.320\.630\.110\.540\.270\.020\.300\.490\.100\.040\.050\.050\.040\.28

Table 59:Comparison of unsupervised MIAs on the RLVR domain ofOLMo\-Detect, evaluated via TPR@5% FPR \(macro\-averaged over its subsets\)\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.†SelfCrit is evaluated only on DPO and RLVR\.SubsetModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDPython1B0\.100\.230\.100\.100\.120\.200\.160\.090\.060\.170\.060\.000\.140\.080\.117B0\.100\.220\.120\.100\.120\.190\.230\.090\.080\.130\.060\.070\.050\.020\.1113B0\.110\.220\.100\.120\.070\.180\.260\.060\.080\.160\.040\.060\.010\.070\.1132B0\.100\.140\.030\.100\.070\.170\.220\.090\.100\.160\.040\.040\.060\.130\.10Model\-Avg0\.100\.210\.090\.100\.090\.180\.220\.080\.080\.150\.050\.040\.060\.070\.11Java1B0\.060\.100\.040\.060\.140\.120\.180\.260\.120\.060\.040\.020\.180\.070\.107B0\.060\.320\.080\.060\.040\.280\.200\.360\.120\.080\.020\.040\.080\.220\.1413B0\.060\.260\.060\.060\.080\.220\.280\.120\.140\.040\.100\.000\.000\.080\.1132B0\.060\.240\.040\.060\.000\.320\.300\.080\.100\.080\.080\.040\.060\.080\.11Model\-Avg0\.060\.230\.060\.060\.070\.230\.240\.200\.120\.070\.060\.030\.080\.110\.12

Table 60:Comparison of unsupervised MIAs on the subsets of the StarCoder domain ofOLMo\-Detect, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.SubsetModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDMath Stack Exchange1B0\.070\.070\.160\.040\.040\.110\.200\.140\.200\.110\.110\.090\.160\.080\.117B0\.050\.070\.070\.070\.020\.250\.160\.040\.140\.080\.110\.020\.020\.050\.0813B0\.070\.160\.150\.070\.040\.200\.270\.110\.110\.020\.120\.090\.070\.040\.1132B0\.110\.250\.210\.140\.110\.250\.390\.230\.200\.050\.110\.110\.120\.050\.17Model\-Avg0\.080\.140\.150\.080\.050\.200\.250\.130\.160\.060\.110\.080\.090\.060\.12Stack Overflow1B0\.060\.100\.090\.070\.030\.050\.050\.060\.060\.040\.080\.050\.050\.030\.067B0\.060\.160\.110\.090\.100\.110\.080\.030\.050\.050\.080\.080\.080\.070\.0813B0\.060\.180\.110\.080\.100\.130\.110\.100\.100\.080\.080\.050\.060\.070\.0932B0\.080\.160\.120\.110\.130\.170\.110\.090\.150\.080\.100\.090\.150\.040\.11Model\-Avg0\.060\.150\.100\.090\.090\.110\.090\.070\.090\.060\.090\.070\.080\.050\.09

Table 61:Comparison of unsupervised MIAs on the subsets of the Stack Exchange domain ofOLMo\-Detect, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.SubsetModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDAya1B0\.120\.020\.160\.160\.240\.090\.040\.090\.060\.140\.080\.060\.040\.050\.107B0\.480\.270\.120\.400\.350\.260\.200\.180\.090\.410\.070\.100\.050\.110\.2213B0\.280\.150\.060\.280\.230\.150\.100\.090\.040\.260\.090\.130\.110\.130\.1532B0\.170\.040\.110\.140\.100\.060\.040\.220\.070\.080\.100\.040\.010\.140\.09Model\-Avg0\.260\.120\.110\.250\.230\.140\.090\.150\.060\.220\.080\.080\.050\.110\.14WildChat1B0\.310\.250\.020\.260\.420\.140\.120\.140\.030\.160\.060\.050\.050\.050\.157B0\.320\.320\.140\.320\.640\.300\.150\.170\.050\.030\.050\.020\.000\.020\.1813B0\.090\.140\.070\.080\.290\.050\.030\.140\.080\.030\.090\.080\.030\.030\.0932B0\.390\.250\.030\.490\.560\.330\.060\.110\.080\.340\.050\.110\.000\.020\.20Model\-Avg0\.280\.240\.060\.290\.480\.200\.090\.140\.060\.140\.060\.060\.020\.030\.15

Table 62:Comparison of unsupervised MIAs on the subsets of the SFT domain ofOLMo\-Detect, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.SubsetModelLikelihood\-basedPert\.PromptOut\.\-Dist\.Method\-AvgPPLZlibLowerMinKMinK\+\+DCPDDPACReCaLLEMMIACAMIANeigh\.GuidedDCQCDDSelfCrit†CoT\-GSM8K1B0\.450\.550\.220\.430\.100\.190\.300\.000\.250\.140\.010\.060\.040\.030\.020\.197B0\.940\.960\.460\.970\.240\.910\.380\.000\.970\.950\.110\.020\.070\.040\.030\.4713B1\.001\.000\.591\.000\.090\.960\.410\.001\.000\.970\.060\.030\.020\.050\.020\.4832B1\.001\.000\.631\.000\.210\.980\.520\.000\.050\.950\.160\.030\.090\.060\.010\.45Model\-Avg0\.850\.880\.480\.850\.160\.760\.410\.000\.570\.750\.090\.030\.050\.040\.020\.40CoT\-MATH1B0\.140\.150\.030\.220\.050\.130\.080\.050\.050\.140\.060\.050\.040\.060\.020\.087B0\.370\.340\.200\.340\.080\.290\.110\.030\.050\.110\.110\.060\.040\.020\.060\.1513B0\.640\.620\.240\.570\.040\.450\.200\.010\.050\.270\.150\.010\.080\.060\.080\.2332B0\.420\.370\.230\.480\.090\.430\.160\.060\.030\.390\.180\.030\.030\.050\.040\.20Model\-Avg0\.390\.370\.170\.400\.060\.330\.140\.040\.040\.230\.120\.040\.050\.050\.050\.17

Table 63:Comparison of unsupervised MIAs on the subsets of the RLVR domain ofOLMo\-Detect, evaluated via TPR@5% FPR\. Methods are grouped by family as in Table[50](https://arxiv.org/html/2610.02986#A10.T50)\.Model\-Avgaverages across model sizes, whileMethod\-Avgaverages the methods with a value in that row\. The best\-performing method in each row is highlighted in bold\.†SelfCrit is evaluated only on DPO and RLVR\.
## Appendix KPer\-Domain AUC Results for Supervised MIAs onOLMo\-Detect

We report per\-domain AUCs for the supervised MIAs in Tables[64](https://arxiv.org/html/2610.02986#A11.T64)–[72](https://arxiv.org/html/2610.02986#A11.T72), one table per domain, complementing the per\-stage results of Table[5](https://arxiv.org/html/2610.02986#S5.T5)\. Each table follows the two\-part layout of Table[5](https://arxiv.org/html/2610.02986#S5.T5): the top part gives the AUCs of Sup\-CAMIA and MIA\-Tuner under both settings, with Zlib repeated as an unsupervised reference, and the bottom part gives FSD’s AUC gain over its base MIA, since FSD is only meaningful relative to the detector it is built on\.In\-domaintrains on the evaluated domain’s development set, andLODOtrains on the other eight domains’ development sets\.

Two patterns hold across domains\. First, supervision rarely pays: the better of Sup\-CAMIA and MIA\-Tuner exceeds Zlib on only five of the nine domains, ties on OpenWebMath, and falls below it on DCLM\-Baseline \(−0\.05\-0\.05\), SFT \(−0\.03\-0\.03\) and RLVR \(−0\.03\-0\.03\); the one substantial gain is on DPO \(\+0\.10\+0\.10\), the domain where Zlib is weakest at0\.480\.48\. Second, fine\-tuning on non\-members seldom helps its base detector: of the4545domain\-by\-base\-MIA combinations, FSD improves on its base in1313under In\-domain and1111under LODO, and its largest losses reach−0\.10\-0\.10or worse on DCLM\-Baseline, where every base detector degrades under both settings\.

MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.590\.640\.650\.670\.64Sup\-CAMIAIn\-domain0\.510\.550\.540\.560\.54LODO0\.550\.640\.580\.570\.58MIA\-TunerIn\-domain0\.450\.450\.530\.550\.50LODO0\.470\.510\.490\.470\.49

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.64−\-0\.02−\-0\.06−\-0\.06−\-0\.05−\-0\.05−\-0\.10−\-0\.04−\-0\.10−\-0\.18−\-0\.10FSD \(Zlib\)0\.64−\-0\.01−\-0\.06−\-0\.04−\-0\.05−\-0\.04−\-0\.09−\-0\.05−\-0\.08−\-0\.19−\-0\.10FSD \(Lower\)0\.63−\-0\.12−\-0\.09−\-0\.12−\-0\.09−\-0\.10−\-0\.09−\-0\.11−\-0\.11−\-0\.13−\-0\.11FSD \(MinK\)0\.61−\-0\.07−\-0\.07−\-0\.07−\-0\.01−\-0\.05−\-0\.08−\-0\.04−\-0\.07−\-0\.18−\-0\.09FSD \(MinK\+\+\)0\.53−\-0\.03−\-0\.02−\-0\.01−\-0\.03−\-0\.02−\-0\.01−\-0\.03−\-0\.04−\-0\.15−\-0\.06

Table 64:Supervised MIAs on the DCLM\-Baseline domain ofOLMo\-Detect, evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.490\.490\.500\.510\.50Sup\-CAMIAIn\-domain0\.550\.470\.520\.500\.51LODO0\.500\.520\.480\.550\.51MIA\-TunerIn\-domain0\.540\.570\.560\.570\.56LODO0\.550\.510\.390\.570\.51

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.48\+\+0\.07\+\+0\.04\+\+0\.05\+\+0\.03\+\+0\.05\+\+0\.08\+\+0\.05\+\+0\.08\+\+0\.07\+\+0\.07FSD \(Zlib\)0\.50\+\+0\.05\+\+0\.02\+\+0\.01−\-0\.01\+\+0\.02\+\+0\.04\+\+0\.03\+\+0\.05\+\+0\.05\+\+0\.04FSD \(Lower\)0\.49\+\+0\.07\+\+0\.04−\-0\.00\+\+0\.02\+\+0\.03\+\+0\.04\+\+0\.00−\-0\.01\+\+0\.00\+\+0\.01FSD \(MinK\)0\.50\+\+0\.06\+\+0\.01−\-0\.00\+\+0\.02\+\+0\.02\+\+0\.05\+\+0\.01\+\+0\.04\+\+0\.04\+\+0\.03FSD \(MinK\+\+\)0\.50\+\+0\.04\+\+0\.01−\-0\.01\+\+0\.02\+\+0\.01\+\+0\.01\+\+0\.01−\-0\.01\+\+0\.06\+\+0\.02

Table 65:Supervised MIAs on the peS2o domain ofOLMo\-Detect, evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.560\.570\.580\.640\.59Sup\-CAMIAIn\-domain0\.600\.590\.590\.580\.59LODO0\.540\.560\.530\.560\.55MIA\-TunerIn\-domain0\.570\.580\.520\.540\.55LODO0\.590\.540\.560\.510\.55

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.53\+\+0\.04\+\+0\.03\+\+0\.07\+\+0\.09\+\+0\.06\+\+0\.03\+\+0\.04\+\+0\.02\+\+0\.02\+\+0\.03FSD \(Zlib\)0\.59\+\+0\.02\+\+0\.00\+\+0\.02\+\+0\.01\+\+0\.01\+\+0\.00−\-0\.01−\-0\.05−\-0\.04−\-0\.02FSD \(Lower\)0\.60−\-0\.04−\-0\.04−\-0\.01−\-0\.07−\-0\.04−\-0\.10−\-0\.04−\-0\.11−\-0\.07−\-0\.08FSD \(MinK\)0\.56\+\+0\.01−\-0\.00\+\+0\.02\+\+0\.04\+\+0\.02\+\+0\.01\+\+0\.01−\-0\.02−\-0\.02−\-0\.01FSD \(MinK\+\+\)0\.52−\-0\.05−\-0\.01\+\+0\.02\+\+0\.00−\-0\.01−\-0\.01−\-0\.00−\-0\.04−\-0\.07−\-0\.03

Table 66:Supervised MIAs on the OpenWebMath domain ofOLMo\-Detect, evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.650\.650\.640\.630\.64Sup\-CAMIAIn\-domain0\.570\.540\.560\.560\.56LODO0\.640\.690\.670\.670\.67MIA\-TunerIn\-domain0\.610\.540\.530\.480\.54LODO0\.460\.550\.440\.500\.49

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.59\+\+0\.03\+\+0\.01−\-0\.00−\-0\.02\+\+0\.01\+\+0\.02−\-0\.01−\-0\.11−\-0\.01−\-0\.03FSD \(Zlib\)0\.64−\-0\.02−\-0\.01−\-0\.01−\-0\.05−\-0\.02−\-0\.03−\-0\.06−\-0\.16−\-0\.03−\-0\.07FSD \(Lower\)0\.60\+\+0\.02−\-0\.00−\-0\.02−\-0\.02−\-0\.01\+\+0\.04−\-0\.03−\-0\.03−\-0\.05−\-0\.02FSD \(MinK\)0\.61−\-0\.03−\-0\.04−\-0\.06−\-0\.09−\-0\.05−\-0\.05−\-0\.01−\-0\.12−\-0\.04−\-0\.05FSD \(MinK\+\+\)0\.49\+\+0\.00−\-0\.04−\-0\.00−\-0\.01−\-0\.01\+\+0\.03\+\+0\.06−\-0\.02\+\+0\.02\+\+0\.02

Table 67:Supervised MIAs on the StarCoder domain ofOLMo\-Detect\(macro\-averaged over its subsets\), evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.760\.890\.990\.990\.91Sup\-CAMIAIn\-domain0\.820\.941\.001\.000\.94LODO0\.680\.850\.770\.900\.80MIA\-TunerIn\-domain0\.510\.510\.500\.510\.51LODO0\.490\.510\.510\.520\.51

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.89−\-0\.05−\-0\.08\+\+0\.01−\-0\.06−\-0\.04−\-0\.16−\-0\.10−\-0\.01−\-0\.14−\-0\.10FSD \(Zlib\)0\.91−\-0\.07−\-0\.10\+\+0\.00−\-0\.06−\-0\.06−\-0\.19−\-0\.12−\-0\.01−\-0\.13−\-0\.11FSD \(Lower\)0\.85−\-0\.03\+\+0\.01\+\+0\.01\+\+0\.01−\-0\.00−\-0\.13−\-0\.05−\-0\.04−\-0\.04−\-0\.06FSD \(MinK\)0\.89−\-0\.12−\-0\.27−\-0\.06−\-0\.13−\-0\.15−\-0\.19−\-0\.08−\-0\.01−\-0\.12−\-0\.10FSD \(MinK\+\+\)0\.55−\-0\.02−\-0\.02−\-0\.02−\-0\.09−\-0\.04−\-0\.06\+\+0\.00−\-0\.00−\-0\.05−\-0\.03

Table 68:Supervised MIAs on the GSM8K subset ofOLMo\-Detect’s Math Mix domain, evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.590\.640\.670\.700\.65Sup\-CAMIAIn\-domain0\.610\.610\.720\.680\.66LODO0\.590\.610\.560\.500\.56MIA\-TunerIn\-domain0\.690\.720\.700\.550\.66LODO0\.510\.460\.440\.540\.49

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.59\+\+0\.01\+\+0\.06\+\+0\.11−\-0\.00\+\+0\.05−\-0\.08\+\+0\.03\+\+0\.01−\-0\.07−\-0\.03FSD \(Zlib\)0\.65−\-0\.02\+\+0\.03\+\+0\.03−\-0\.08−\-0\.01−\-0\.11−\-0\.02−\-0\.08−\-0\.14−\-0\.09FSD \(Lower\)0\.53−\-0\.01−\-0\.01−\-0\.03−\-0\.07−\-0\.03\+\+0\.01\+\+0\.06\+\+0\.08−\-0\.08\+\+0\.02FSD \(MinK\)0\.60\+\+0\.01\+\+0\.03\+\+0\.04−\-0\.01\+\+0\.02−\-0\.08\+\+0\.01−\-0\.01−\-0\.08−\-0\.04FSD \(MinK\+\+\)0\.57\+\+0\.00\+\+0\.00−\-0\.02−\-0\.03−\-0\.01−\-0\.03\+\+0\.01−\-0\.02−\-0\.03−\-0\.02

Table 69:Supervised MIAs on the Stack Exchange domain ofOLMo\-Detect\(macro\-averaged over its subsets\), evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.640\.700\.610\.640\.65Sup\-CAMIAIn\-domain0\.510\.730\.660\.550\.61LODO0\.550\.610\.560\.490\.55MIA\-TunerIn\-domain0\.530\.550\.500\.520\.53LODO0\.500\.460\.480\.510\.49

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.66\+\+0\.04−\-0\.05\+\+0\.02\+\+0\.06\+\+0\.02\+\+0\.06−\-0\.04\+\+0\.02\+\+0\.04\+\+0\.02FSD \(Zlib\)0\.65\+\+0\.01\+\+0\.02\+\+0\.08\+\+0\.05\+\+0\.04\+\+0\.04\+\+0\.02\+\+0\.09\+\+0\.04\+\+0\.05FSD \(Lower\)0\.51−\-0\.03\+\+0\.01−\-0\.05−\-0\.00−\-0\.02−\-0\.00−\-0\.04−\-0\.02\+\+0\.02−\-0\.01FSD \(MinK\)0\.66−\-0\.02−\-0\.07\+\+0\.02\+\+0\.06−\-0\.00\+\+0\.03−\-0\.06\+\+0\.02\+\+0\.05\+\+0\.01FSD \(MinK\+\+\)0\.72−\-0\.10−\-0\.06−\-0\.03−\-0\.01−\-0\.05−\-0\.06−\-0\.07−\-0\.02−\-0\.02−\-0\.04

Table 70:Supervised MIAs on the SFT domain ofOLMo\-Detect\(macro\-averaged over its subsets\), evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.550\.390\.470\.530\.48Sup\-CAMIAIn\-domain0\.600\.580\.580\.570\.58LODO0\.400\.420\.400\.440\.42MIA\-TunerIn\-domain0\.490\.550\.560\.470\.52LODO0\.410\.440\.420\.480\.44

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.42−\-0\.04−\-0\.01−\-0\.04−\-0\.02−\-0\.03−\-0\.03−\-0\.01−\-0\.06−\-0\.01−\-0\.03FSD \(Zlib\)0\.48−\-0\.13−\-0\.02−\-0\.13−\-0\.04−\-0\.08−\-0\.12−\-0\.01−\-0\.15−\-0\.06−\-0\.09FSD \(Lower\)0\.60−\-0\.08\+\+0\.02−\-0\.03−\-0\.06−\-0\.04−\-0\.06\+\+0\.05−\-0\.03−\-0\.10−\-0\.04FSD \(MinK\)0\.43−\-0\.07\+\+0\.00−\-0\.04−\-0\.02−\-0\.03−\-0\.06−\-0\.00−\-0\.06−\-0\.01−\-0\.03FSD \(MinK\+\+\)0\.44−\-0\.08−\-0\.01−\-0\.03\+\+0\.00−\-0\.03−\-0\.07−\-0\.01−\-0\.02\+\+0\.02−\-0\.02

Table 71:Supervised MIAs on the UF\-Wildchat subset ofOLMo\-Detect’s DPO domain, evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.790\.880\.940\.910\.88Sup\-CAMIAIn\-domain0\.730\.860\.910\.900\.85LODO0\.620\.830\.680\.940\.77MIA\-TunerIn\-domain0\.560\.540\.530\.540\.54LODO0\.510\.480\.480\.520\.50

MethodBase AUCIn\-domainΔ\\DeltaAUCLODOΔ\\DeltaAUC1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.89−\-0\.05−\-0\.21−\-0\.12−\-0\.31−\-0\.17−\-0\.02−\-0\.02−\-0\.03−\-0\.07−\-0\.03FSD \(Zlib\)0\.88−\-0\.11−\-0\.25−\-0\.14−\-0\.31−\-0\.20−\-0\.05−\-0\.02−\-0\.02−\-0\.05−\-0\.04FSD \(Lower\)0\.79−\-0\.04−\-0\.11−\-0\.05−\-0\.14−\-0\.08−\-0\.06−\-0\.07−\-0\.01−\-0\.01−\-0\.03FSD \(MinK\)0\.90−\-0\.19−\-0\.59−\-0\.48−\-0\.59−\-0\.46−\-0\.10−\-0\.07−\-0\.04−\-0\.11−\-0\.08FSD \(MinK\+\+\)0\.66−\-0\.02−\-0\.07−\-0\.02−\-0\.14−\-0\.06−\-0\.01−\-0\.02−\-0\.00−\-0\.06−\-0\.02

Table 72:Supervised MIAs on the RLVR domain ofOLMo\-Detect\(macro\-averaged over its subsets\), evaluated via AUC\.Top:AUCs of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its AUC gain \(Δ\\DeltaAUC\) over the base MIA \(in parentheses\) on this domain\.Base AUC: the base MIA’s AUC on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.
## Appendix LTPR@5%FPR Results for Supervised MIAs onOLMo\-Detect

The main text reports supervised MIAs by AUC only, so we give their TPR@5% FPR here\. Table[73](https://arxiv.org/html/2610.02986#A12.T73)is the overall table, aggregated by training stage, and Tables[74](https://arxiv.org/html/2610.02986#A12.T74)–[82](https://arxiv.org/html/2610.02986#A12.T82)give one table per domain\. All follow the two\-part layout of Table[5](https://arxiv.org/html/2610.02986#S5.T5): the top part reports Sup\-CAMIA and MIA\-Tuner under both settings, with Zlib repeated as an unsupervised reference, and the bottom part reports FSD’s gain over the base MIA it is built on\.In\-domaintrains on the evaluated domain’s development set, andLODOtrains on the other eight domains’ development sets\.

TPR@5% FPR reproduces the AUC picture and sharpens it\. Overall, the best supervised configuration, Sup\-CAMIA under In\-domain, reaches0\.260\.26against Zlib’s0\.280\.28, so supervision does not improve the operating point that matters for an attacker; moving to LODO halves it to0\.140\.14, and MIA\-Tuner reaches only0\.090\.09In\-domain and0\.050\.05under LODO, the latter indistinguishable from the0\.050\.05of a random detector\. Per domain, the better of Sup\-CAMIA and MIA\-Tuner exceeds Zlib on four of the nine domains and falls below it on the other five, with the largest shortfall on StarCoder \(−0\.10\-0\.10\) and the largest gains on Math Mix and DPO \(\+0\.08\+0\.08each\)\. FSD again rarely helps its base MIA: of the4545domain\-by\-base\-MIA combinations it improves in1212under In\-domain and1111under LODO\.

MethodSetting1B7B13B32BModel\-AvgPreMidPostAllPreMidPostAllPreMidPostAllPreMidPostAllPreMidPostAllZlib \(unsup\.\)—0\.090\.170\.180\.150\.130\.330\.320\.260\.130\.570\.340\.350\.160\.580\.310\.350\.130\.410\.290\.28Sup\-CAMIAIn\-domain0\.080\.240\.190\.170\.070\.380\.300\.250\.080\.570\.360\.340\.060\.560\.270\.300\.070\.440\.280\.26LODO0\.070\.090\.080\.080\.060\.220\.250\.180\.070\.070\.110\.080\.100\.240\.280\.210\.070\.160\.180\.14MIA\-TunerIn\-domain0\.070\.130\.100\.100\.080\.200\.090\.120\.040\.150\.080\.090\.060\.070\.060\.060\.060\.140\.080\.09LODO0\.070\.050\.050\.060\.040\.070\.050\.050\.040\.050\.060\.050\.050\.060\.060\.060\.050\.060\.050\.05

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.25−\-0\.00−\-0\.06\+\+0\.01−\-0\.10−\-0\.04−\-0\.02−\-0\.05−\-0\.02−\-0\.13−\-0\.06FSD \(Zlib\)0\.28−\-0\.03−\-0\.10−\-0\.06−\-0\.12−\-0\.08−\-0\.04−\-0\.07−\-0\.04−\-0\.14−\-0\.07FSD \(Lower\)0\.18−\-0\.02−\-0\.02−\-0\.00−\-0\.05−\-0\.02−\-0\.04−\-0\.02−\-0\.03−\-0\.05−\-0\.04FSD \(MinK\)0\.26−\-0\.05−\-0\.15−\-0\.11−\-0\.15−\-0\.11−\-0\.05−\-0\.08−\-0\.03−\-0\.14−\-0\.07FSD \(MinK\+\+\)0\.12−\-0\.04−\-0\.04−\-0\.01−\-0\.06−\-0\.04−\-0\.06−\-0\.04−\-0\.01−\-0\.04−\-0\.04

Table 73:Supervised MIAs onOLMo\-Detect, evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on the evaluated domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Pre/Mid/Postmacro\-average over each stage’s domains andAllmacro\-averages the three stages, as in Table[4](https://arxiv.org/html/2610.02986#S4.T4)\.Bottom:FSD’s gain \(Δ\\Delta\) over its base MIA \(in parentheses\) under theAllaggregation;Base TPR@5% FPRis the base MIA’s own score averaged across model sizes, and positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.060\.110\.130\.180\.12Sup\-CAMIAIn\-domain0\.010\.050\.040\.060\.04LODO0\.050\.080\.040\.170\.09MIA\-TunerIn\-domain0\.010\.010\.000\.060\.02LODO0\.020\.020\.030\.010\.02

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.15−\-0\.05−\-0\.10−\-0\.08−\-0\.13−\-0\.09−\-0\.06−\-0\.01−\-0\.09−\-0\.17−\-0\.08FSD \(Zlib\)0\.12−\-0\.01−\-0\.07−\-0\.10−\-0\.07−\-0\.06−\-0\.02\+\+0\.03−\-0\.04−\-0\.11−\-0\.04FSD \(Lower\)0\.14−\-0\.07−\-0\.04−\-0\.02−\-0\.03−\-0\.04−\-0\.07−\-0\.06\+\+0\.00−\-0\.04−\-0\.04FSD \(MinK\)0\.19−\-0\.11−\-0\.19−\-0\.15−\-0\.11−\-0\.14−\-0\.09−\-0\.11−\-0\.14−\-0\.18−\-0\.13FSD \(MinK\+\+\)0\.06\+\+0\.01−\-0\.01\+\+0\.01−\-0\.06−\-0\.01\+\+0\.04−\-0\.04−\-0\.01−\-0\.09−\-0\.03

Table 74:Supervised MIAs on the DCLM\-Baseline domain ofOLMo\-Detect, evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.070\.070\.070\.090\.07Sup\-CAMIAIn\-domain0\.080\.040\.050\.080\.06LODO0\.050\.030\.060\.050\.05MIA\-TunerIn\-domain0\.090\.080\.060\.090\.08LODO0\.070\.120\.010\.130\.08

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.07−\-0\.02−\-0\.02−\-0\.03−\-0\.04−\-0\.03−\-0\.01−\-0\.01\+\+0\.05−\-0\.01\+\+0\.01FSD \(Zlib\)0\.07−\-0\.02\+\+0\.00−\-0\.01−\-0\.05−\-0\.02−\-0\.02−\-0\.02\+\+0\.03−\-0\.04−\-0\.01FSD \(Lower\)0\.07\+\+0\.00−\-0\.03−\-0\.03−\-0\.01−\-0\.02\+\+0\.01−\-0\.03−\-0\.03−\-0\.04−\-0\.02FSD \(MinK\)0\.08−\-0\.03−\-0\.03−\-0\.04−\-0\.05−\-0\.04−\-0\.02−\-0\.03\+\+0\.00−\-0\.03−\-0\.02FSD \(MinK\+\+\)0\.07−\-0\.04−\-0\.00−\-0\.03−\-0\.01−\-0\.02−\-0\.02−\-0\.00\+\+0\.01\+\+0\.00−\-0\.00

Table 75:Supervised MIAs on the peS2o domain ofOLMo\-Detect, evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.070\.070\.100\.170\.10Sup\-CAMIAIn\-domain0\.100\.060\.100\.040\.07LODO0\.050\.090\.030\.090\.06MIA\-TunerIn\-domain0\.050\.090\.060\.050\.06LODO0\.110\.010\.060\.070\.06

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.06\+\+0\.04\+\+0\.06\+\+0\.04\+\+0\.06\+\+0\.05\+\+0\.01\+\+0\.00−\-0\.02−\-0\.05−\-0\.02FSD \(Zlib\)0\.10\+\+0\.02\+\+0\.04\+\+0\.02−\-0\.01\+\+0\.02−\-0\.01−\-0\.04−\-0\.05−\-0\.07−\-0\.04FSD \(Lower\)0\.06−\-0\.00−\-0\.05\+\+0\.01−\-0\.01−\-0\.01−\-0\.03−\-0\.03−\-0\.02−\-0\.00−\-0\.02FSD \(MinK\)0\.06\+\+0\.04\+\+0\.05\+\+0\.03\+\+0\.11\+\+0\.05\+\+0\.01−\-0\.00−\-0\.02−\-0\.03−\-0\.01FSD \(MinK\+\+\)0\.12−\-0\.03−\-0\.10−\-0\.07−\-0\.09−\-0\.07\+\+0\.01−\-0\.09−\-0\.08−\-0\.06−\-0\.06

Table 76:Supervised MIAs on the OpenWebMath domain ofOLMo\-Detect, evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.170\.270\.240\.190\.22Sup\-CAMIAIn\-domain0\.120\.130\.130\.080\.12LODO0\.120\.040\.140\.100\.10MIA\-TunerIn\-domain0\.120\.110\.020\.050\.08LODO0\.090\.020\.050\.010\.04

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.08−\-0\.03−\-0\.01−\-0\.01−\-0\.02−\-0\.02\+\+0\.06−\-0\.01−\-0\.03−\-0\.02\+\+0\.00FSD \(Zlib\)0\.22−\-0\.05−\-0\.17−\-0\.11−\-0\.05−\-0\.09−\-0\.03−\-0\.18−\-0\.15−\-0\.19−\-0\.14FSD \(Lower\)0\.07\+\+0\.06\+\+0\.02−\-0\.05\+\+0\.06\+\+0\.02\+\+0\.03\+\+0\.03−\-0\.05−\-0\.02\+\+0\.00FSD \(MinK\)0\.08−\-0\.06−\-0\.05−\-0\.03−\-0\.03−\-0\.04\+\+0\.07\+\+0\.02−\-0\.03−\-0\.02\+\+0\.01FSD \(MinK\+\+\)0\.08−\-0\.11−\-0\.02\+\+0\.00−\-0\.02−\-0\.04−\-0\.04\+\+0\.03−\-0\.01\+\+0\.02−\-0\.00

Table 77:Supervised MIAs on the StarCoder domain ofOLMo\-Detect\(macro\-averaged over its subsets\), evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.250\.540\.970\.960\.68Sup\-CAMIAIn\-domain0\.390\.690\.970\.970\.76LODO0\.130\.380\.070\.430\.25MIA\-TunerIn\-domain0\.060\.050\.060\.060\.06LODO0\.050\.070\.050\.060\.06

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.61−\-0\.02−\-0\.06\+\+0\.05−\-0\.16−\-0\.05−\-0\.17−\-0\.22−\-0\.05−\-0\.53−\-0\.24FSD \(Zlib\)0\.68−\-0\.08−\-0\.15\+\+0\.01−\-0\.21−\-0\.11−\-0\.20−\-0\.29−\-0\.03−\-0\.49−\-0\.25FSD \(Lower\)0\.38−\-0\.01\+\+0\.07\+\+0\.16\+\+0\.05\+\+0\.06−\-0\.12\+\+0\.01−\-0\.09−\-0\.18−\-0\.09FSD \(MinK\)0\.63−\-0\.11−\-0\.39−\-0\.21−\-0\.32−\-0\.26−\-0\.20−\-0\.25−\-0\.09−\-0\.41−\-0\.24FSD \(MinK\+\+\)0\.16−\-0\.04−\-0\.03−\-0\.04−\-0\.09−\-0\.05−\-0\.10−\-0\.07−\-0\.03−\-0\.07−\-0\.07

Table 78:Supervised MIAs on the GSM8K subset ofOLMo\-Detect’s Math Mix domain, evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.080\.120\.170\.200\.14Sup\-CAMIAIn\-domain0\.080\.060\.170\.160\.12LODO0\.050\.060\.080\.060\.06MIA\-TunerIn\-domain0\.190\.360\.240\.090\.22LODO0\.060\.080\.040\.050\.06

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.07\+\+0\.02\+\+0\.08\+\+0\.15\+\+0\.02\+\+0\.07−\-0\.03−\-0\.01\+\+0\.02−\-0\.04−\-0\.02FSD \(Zlib\)0\.14−\-0\.00\+\+0\.02−\-0\.03−\-0\.08−\-0\.02−\-0\.05−\-0\.08−\-0\.04−\-0\.15−\-0\.08FSD \(Lower\)0\.13−\-0\.07−\-0\.03−\-0\.00−\-0\.12−\-0\.06−\-0\.06−\-0\.00−\-0\.06−\-0\.11−\-0\.06FSD \(MinK\)0\.08\+\+0\.00−\-0\.01\+\+0\.08−\-0\.02\+\+0\.01−\-0\.02−\-0\.01\+\+0\.02−\-0\.07−\-0\.02FSD \(MinK\+\+\)0\.07−\-0\.02−\-0\.00−\-0\.00−\-0\.04−\-0\.01−\-0\.02\+\+0\.03\+\+0\.04−\-0\.04\+\+0\.00

Table 79:Supervised MIAs on the Stack Exchange domain ofOLMo\-Detect\(macro\-averaged over its subsets\), evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.140\.290\.140\.150\.18Sup\-CAMIAIn\-domain0\.090\.200\.140\.080\.13LODO0\.060\.230\.050\.100\.11MIA\-TunerIn\-domain0\.090\.100\.070\.040\.07LODO0\.050\.030\.070\.030\.04

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.27\+\+0\.08\+\+0\.03\+\+0\.16\+\+0\.01\+\+0\.07\+\+0\.19−\-0\.01\+\+0\.03−\-0\.08\+\+0\.03FSD \(Zlib\)0\.18\+\+0\.12−\-0\.00\+\+0\.07\+\+0\.12\+\+0\.08\+\+0\.18\+\+0\.08\+\+0\.07\+\+0\.05\+\+0\.09FSD \(Lower\)0\.09−\-0\.03−\-0\.01\+\+0\.00−\-0\.02−\-0\.02−\-0\.05−\-0\.05\+\+0\.01−\-0\.03−\-0\.03FSD \(MinK\)0\.27\+\+0\.09\+\+0\.01\+\+0\.13−\-0\.09\+\+0\.04\+\+0\.14−\-0\.06\+\+0\.08−\-0\.12\+\+0\.01FSD \(MinK\+\+\)0\.36−\-0\.12−\-0\.09\+\+0\.01−\-0\.20−\-0\.10−\-0\.28−\-0\.21−\-0\.04−\-0\.15−\-0\.17

Table 80:Supervised MIAs on the SFT domain ofOLMo\-Detect\(macro\-averaged over its subsets\), evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.060\.030\.060\.090\.06Sup\-CAMIAIn\-domain0\.210\.060\.210\.050\.13LODO0\.060\.060\.110\.080\.08MIA\-TunerIn\-domain0\.050\.080\.120\.060\.08LODO0\.060\.070\.070\.070\.07

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.04−\-0\.00−\-0\.01−\-0\.02\+\+0\.01−\-0\.01−\-0\.02\+\+0\.01−\-0\.01\+\+0\.01−\-0\.00FSD \(Zlib\)0\.06\+\+0\.03−\-0\.03−\-0\.04−\-0\.01−\-0\.01\+\+0\.00−\-0\.01−\-0\.03−\-0\.03−\-0\.02FSD \(Lower\)0\.15−\-0\.01\+\+0\.01−\-0\.01−\-0\.02−\-0\.01\+\+0\.01\+\+0\.05\+\+0\.00−\-0\.03\+\+0\.01FSD \(MinK\)0\.05−\-0\.02−\-0\.01−\-0\.02\+\+0\.05\+\+0\.00−\-0\.01\+\+0\.03−\-0\.02\+\+0\.04\+\+0\.01FSD \(MinK\+\+\)0\.04\+\+0\.01−\-0\.01\+\+0\.01\+\+0\.03\+\+0\.01−\-0\.01\+\+0\.01−\-0\.01\+\+0\.05\+\+0\.01

Table 81:Supervised MIAs on the UF\-Wildchat subset ofOLMo\-Detect’s DPO domain, evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.MethodSetting1B7B13B32BModel\-AvgZlib \(unsup\.\)—0\.350\.650\.810\.680\.62Sup\-CAMIAIn\-domain0\.270\.640\.720\.670\.58LODO0\.120\.450\.180\.660\.35MIA\-TunerIn\-domain0\.150\.100\.070\.070\.10LODO0\.040\.040\.030\.080\.05

MethodBase TPR@5% FPRIn\-domainΔ\\DeltaLODOΔ\\Delta1B7B13B32BModel\-Avg1B7B13B32BModel\-AvgFSD \(PPL\)0\.62−\-0\.06−\-0\.51−\-0\.33−\-0\.60−\-0\.37−\-0\.07−\-0\.05−\-0\.11−\-0\.06−\-0\.07FSD \(Zlib\)0\.62−\-0\.23−\-0\.53−\-0\.43−\-0\.59−\-0\.44−\-0\.08−\-0\.01−\-0\.09−\-0\.03−\-0\.05FSD \(Lower\)0\.32−\-0\.03−\-0\.18−\-0\.18−\-0\.31−\-0\.17−\-0\.06−\-0\.09−\-0\.01\+\+0\.09−\-0\.02FSD \(MinK\)0\.63−\-0\.21−\-0\.57−\-0\.77−\-0\.70−\-0\.56−\-0\.22−\-0\.19−\-0\.13−\-0\.23−\-0\.19FSD \(MinK\+\+\)0\.11−\-0\.01−\-0\.07−\-0\.01−\-0\.06−\-0\.04−\-0\.01−\-0\.01\+\+0\.00−\-0\.02−\-0\.01

Table 82:Supervised MIAs on the RLVR domain ofOLMo\-Detect\(macro\-averaged over its subsets\), evaluated via TPR@5% FPR\.Top:scores of Sup\-CAMIA and MIA\-Tuner under theIn\-domainsetting \(trained on this domain’s dev set\) and theLODOsetting \(trained on the other eight domains’ dev sets\); Zlib, the strongest unsupervised MIA, is repeated as a reference and is excluded from the bold, which marks the best supervised entry per column\.Bottom:FSD measures the change in a base MIA’s score after fine\-tuning the target model on non\-members, so we report its gain \(Δ\\Delta\) over the base MIA \(in parentheses\) on this domain\.Base TPR@5% FPR: the base MIA’s own score on this domain averaged across model sizes; positive gains are bolded\.Model\-Avgaverages across model sizes\.
## Appendix MEffect of the Labeled\-Data Budget on Supervised MIAs

The supervised methods in Section[5\.2](https://arxiv.org/html/2610.02986#S5.SS2)are trained on each domain’s dev split, i\.e\., 10% of its members and non\-members\. To test whether this budget limits their performance, we measure Sup\-CAMIA as a function of the number of labeled examples\. For each domain and model size, we hold out half of the members and half of the non\-members as a fixed evaluation set and train on nested subsets of the remaining half containing 10%, 20%, 30%, 40%, and 50% of the domain’s members and non\-members; the 10% point matches the budget of the main tables\. We repeat this over five random splits and report the mean AUC on the held\-out half\. Because the evaluation set comprises half of each domain rather than the 90% test split, absolute AUCs are not comparable to those in the main tables; only the trend is relevant\.

Table[83](https://arxiv.org/html/2610.02986#A13.T83)reports the results\. Raising the budget from 10% to 50% increases the macro\-averaged AUC by 0\.04–0\.05 at every model size\. The gains are confined to the smallest pre\-training domains \(DCLM\-Baseline, OpenWebMath, and StarCoder, with at most 151 members\), where the 10% budget provides only 15–16 labeled members and non\-members; peS2o and DPO stay within 0\.03 of their 10% value, and GSM8K and CoT\-GSM8K are saturated at every budget\. A five\-fold larger labeled set therefore does not change the conclusion of Section[5\.2](https://arxiv.org/html/2610.02986#S5.SS2): supervision from the matched distribution adds little beyond unsupervised membership signals\.

DomainNN1B7B13B32B10%20%30%40%50%10%20%30%40%50%10%20%30%40%50%10%20%30%40%50%DCLM\-Baseline1500\.520\.570\.560\.570\.580\.560\.580\.620\.620\.630\.580\.560\.590\.600\.600\.580\.600\.630\.630\.64peS2o2680\.520\.530\.550\.550\.550\.520\.550\.550\.550\.550\.490\.500\.490\.490\.480\.490\.490\.500\.490\.49OpenWebMath1510\.540\.570\.600\.620\.620\.520\.550\.580\.610\.610\.520\.570\.580\.600\.600\.550\.600\.610\.630\.63StarCoder1560\.610\.630\.650\.650\.660\.590\.610\.650\.670\.670\.590\.610\.640\.650\.660\.610\.640\.660\.660\.68Math Mix10400\.830\.830\.840\.840\.840\.940\.940\.950\.950\.951\.001\.001\.001\.001\.001\.001\.001\.001\.001\.00Stack Exchange4300\.580\.580\.580\.580\.590\.570\.590\.600\.610\.620\.630\.640\.670\.690\.700\.600\.630\.650\.670\.68SFT164–2250\.590\.620\.620\.640\.640\.670\.760\.770\.780\.770\.580\.640\.640\.640\.650\.580\.590\.620\.630\.64DPO110–1130\.610\.610\.630\.640\.650\.540\.590\.600\.610\.610\.640\.650\.660\.670\.670\.580\.590\.610\.610\.63RLVR2900\.730\.780\.800\.800\.800\.860\.870\.870\.880\.890\.940\.950\.950\.950\.950\.900\.920\.920\.920\.92Macro\-Avg0\.610\.630\.650\.650\.660\.640\.670\.690\.700\.700\.660\.680\.690\.700\.700\.650\.670\.690\.690\.70

Table 83:AUC of Sup\-CAMIA as a function of the labeled\-data budget onOLMo\-Detect\. For each subset and model size, half of the members and half of the non\-members are held out as a fixed evaluation set; Sup\-CAMIA is trained on nested subsets of the remaining half containing 10%–50% of the subset’s members and non\-members \(10% equals the dev\-set budget of the main tables\)\. Entries are the mean AUC on the held\-out half over five random splits, averaged over the domain’s subsets \(DPO over the chosen and rejected responses\);NNis the number of members \(equal to the number of non\-members\) in the domain\. SFT and DPO members come from the size\-specific post\-training mixes \(SFT: one mix shared by 1B and 32B and another by 7B and 13B; DPO: one preference mix per size\), so theirNNvaries across model sizes and is given as a range\. Macro\-Avg is the mean over the nine domains\. At the subset level the standard deviation across splits is below 0\.05 in 211 of 260 cases; it reaches 0\.17 for the smallest subsets \(at most 100 members\) at the 10%–20% budgets and is at most 0\.09 at budgets of 30% or more\. Absolute values are not comparable to Table[5](https://arxiv.org/html/2610.02986#S5.T5), which uses the 90% test split\.MethodGiven the domainDomain unknownAUC \(macro\)Acc\.AUC \(pooled\)Acc\.PPLL0\.600\.570\.560\.56ZlibL0\.640\.590\.590\.57LowerL0\.610\.570\.590\.57MinKL0\.610\.580\.580\.57MinK\+\+L0\.540\.520\.520\.53DCPDDL0\.630\.590\.600\.58PACL0\.630\.580\.590\.58ReCaLLL0\.480\.520\.500\.53EMMIAL0\.55—0\.51—CAMIAL0\.620\.590\.620\.60Neigh\.P0\.570\.530\.550\.54GuidedPr0\.490\.480\.490\.50DCQPr0\.490\.500\.500\.53CDDO0\.520\.500\.510\.50SelfCritO†0\.48—0\.48—Method\-Avg \(unsup\.\)0\.560\.550\.550\.55Sup\-CAMIA0\.660\.620\.600\.58MIA\-Tuner0\.560\.520\.550\.54Table 84:Two evaluation settings onOLMo\-Detect, averaged over model sizes \(1B–32B\)\. Every domain is subsampled to 100 members and 100 non\-members \(20 random draws; multi\-subset domains pooled over their subsets; methods, dev sets and hyperparameters exactly as in the main tables\)\.Given the domain: AUC per domain, macro\-averaged \(the paper’s protocol\), and accuracy with a decision threshold tuned within the evaluated domain \(2\-fold\)\.Domain unknown: one AUC over the pooled instances of all nine domains, and accuracy with a single threshold tuned on the pooled data; supervised MIAs are retrained on the pooled dev sets of all nine domains\. Thresholds maximise accuracy on their tuning data\. Best per column and block in bold\.†SelfCrit is evaluated on DPO and RLVR only\. Accuracies are omitted for EMMIA, which reports the best of its four base scores per subset and thus has no single score to threshold and for SelfCrit\.
## Appendix NDomain\-Unknown Evaluation

The main text evaluates MIAs*given the domain*: each domain is scored separately and the AUCs are macro\-averaged, which assumes the auditor knows which domain a text comes from\. In practice that is rarely true\. To simulate the realistic case, the*domain\-unknown*setting pools members and non\-members from all nine domains and computes a single AUC, so a score must separate members from non\-members in general rather than within a known domain\.222This differs from the LODO setting: the test instance still comes from a domain the MIA was tuned on, we simply do not know which one\.Table[84](https://arxiv.org/html/2610.02986#A13.T84)contrasts the two settings on the same subsampled data, 100 members and 100 non\-members per domain over 20 draws, with supervised MIAs retrained on the pooled dev sets\. For comparison on a common metric we also report accuracy, with a threshold tuned within each evaluated domain in the first setting and a single threshold tuned on the pooled data in the second\.

Pooling lowers AUC for 12 of the 15 unsupervised MIAs\. The three that rise, ReCaLL, Guided and DCQ, are at or below chance in both settings, so the increase carries no information\. The largest drops fall on Zlib \(0\.640\.64to0\.590\.59\), PPL \(0\.600\.60to0\.560\.56\) and EMMIA \(0\.550\.55to0\.510\.51\), while CAMIA is unaffected \(0\.620\.62in both\), making it the strongest method once the domain is unknown\. Averaged over unsupervised MIAs the cost is0\.020\.02AUC\. Supervised MIAs pay more, but unevenly: Sup\-CAMIA falls from0\.660\.66to0\.600\.60, having lost the ability to calibrate to a single domain, whereas MIA\-Tuner is almost unchanged at0\.560\.56to0\.550\.55\. Accuracy tells the same story, with the unsupervised average flat at0\.550\.55and Sup\-CAMIA falling from0\.620\.62to0\.580\.58\. Crucially the ranking survives: Spearmanρ=0\.95\\rho=0\.95between the two settings across the 17 MIAs\. Absolute performance therefore declines when the domain is unknown, but the conclusions of the main text do not depend on knowing it\.

## Appendix OPer\-Domain Matched\-vs\-Shifted Results

Table[85](https://arxiv.org/html/2610.02986#A15.T85)reports the per\-domain changes behind the aggregate counts of Table[7](https://arxiv.org/html/2610.02986#S5.T7)\. Each entry is the AUC onOLMo\-Detect \(Shifted\)minus the AUC onOLMo\-Detect, averaged over the four model sizes, with significance from a paired instance\-level bootstrap\. Two patterns are visible directly\. The direction of the change is a property of the method–domain pair rather than of the domain: OpenWebMath inflates PPL, MinK, CAMIA and EMMIA but degrades Zlib, Lower and PAC\. And the largest change of every MIA falls in a pre\-training domain, most often OpenWebMath \(7 of 14 MIAs\), then StarCoder \(4\) and peS2o \(3\), while DCLM never carries a method’s largest change despite being shifted on the same axes\.

MethodDCLMpeS2oOWMSCSESFTDPOLargestPPL\+\+0\.06‡\+\+0\.22‡\+\+0\.29‡\+\+0\.04†−\-0\.06‡−\-0\.08‡−\-0\.01OWMZlib\+\+0\.01−\-0\.20‡−\-0\.31‡\+\+0\.25‡\+\+0\.05‡−\-0\.01\+\+0\.01OWMLower\+\+0\.05‡\+\+0\.06‡−\-0\.30‡\+\+0\.14‡\+\+0\.04‡\+\+0\.06‡\+\+0\.01OWMMinK\+\+0\.02\+\+0\.21‡\+\+0\.23‡\+\+0\.03−\-0\.02−\-0\.06‡\+\+0\.01OWMMinK\+\+−\-0\.06‡−\-0\.03†−\-0\.02−\-0\.16‡\+\+0\.07‡−\-0\.02\+\+0\.01SCDCPDD\+\+0\.03\+\+0\.04‡−\-0\.05‡\+\+0\.16‡\+\+0\.02−\-0\.10‡−\-0\.04‡SCPAC\+\+0\.05†\+\+0\.09‡−\-0\.20‡\+\+0\.19‡−\-0\.00\+\+0\.04†\+\+0\.09‡OWMReCaLL\+\+0\.06‡\+\+0\.24‡\+\+0\.04‡\+\+0\.13‡\+\+0\.01\+\+0\.02\+\+0\.01peS2oEMMIA−\-0\.01−\-0\.17‡\+\+0\.42‡\+\+0\.06†−\-0\.06‡\+\+0\.06‡\+\+0\.02OWMCAMIA\+\+0\.04\+\+0\.28‡\+\+0\.18‡\+\+0\.23‡−\-0\.12‡\+\+0\.00\+\+0\.03†peS2oNeigh\.\+\+0\.03\+\+0\.10‡\+\+0\.04†\+\+0\.12‡−\-0\.03†\+\+0\.06‡\+\+0\.06‡SCGuided\+\+0\.03−\-0\.05‡\+\+0\.03\+\+0\.10‡\+\+0\.00−\-0\.03†−\-0\.01SCDCQ−\-0\.02\+\+0\.09‡−\-0\.03†\+\+0\.01\+\+0\.03†\+\+0\.02−\-0\.01peS2oCDD\+\+0\.01\+\+0\.05‡\+\+0\.12‡\+\+0\.04‡−\-0\.01−\-0\.01−\-0\.01OWM

Table 85:Per\-domain effect of distribution shift on unsupervised MIAs,Δ\\Delta= AUC onOLMo\-Detect \(Shifted\)minus AUC onOLMo\-Detect, averaged over the four model sizes\. Significance is assessed with a paired instance\-level bootstrap:†the95%95\\%interval excludes zero,‡the99%99\\%interval excludes zero; significant entries are in bold\.Largest: the domain carrying that MIA’s largest change\. GSM8K and RLVR have no shifted variant\. OWM: OpenWebMath; SC: StarCoder; SE: Stack Exchange\. Aggregate counts are in Table[7](https://arxiv.org/html/2610.02986#S5.T7)\.
## Appendix PNon\-Member Verification for SmolLM2

SmolLM2\-1\.7B is trained on DCLM together with FineWeb\-Edu, both publicly released\. Its members therefore overlapOLMo\-Detect’s DCLM domain by construction, but its non\-members are valid only if they are absent from*every*corpus SmolLM2 saw, not from DCLM alone\. BecauseOLMo\-Detect’s DCLM non\-members are drawn from 25 Common Crawl dumps spanning 2013–2021, and FineWeb\-Edu deduplicates within rather than across dumps, absence from DCLM does not imply absence from FineWeb\-Edu\.

We therefore applied the same 13\-gram filter used in benchmark construction, with the same0\.200\.20threshold, to the whole of FineWeb\-Edu: 2,410 files and1\.53×1091\.53\\times 10^\{9\}documents, sharded nine ways across the fleet\. Eight of the 134 DCLM test non\-members exceed the threshold, at overlaps of0\.8550\.855,0\.7970\.797,0\.7900\.790,0\.7460\.746,0\.7460\.746,0\.7330\.733,0\.6610\.661and0\.6160\.616, together with one of the 16 development non\-members\. The remaining 126 reach at most0\.1280\.128, so the separation is unambiguous rather than borderline\. The excluded indices are released with the benchmark\.

We evaluate SmolLM2 on the 126 verified non\-members, and re\-evaluate OLMo 2\-1B on exactly the same set so that the comparison in Table[8](https://arxiv.org/html/2610.02986#S5.T8)is paired\. Members are not filtered: a SmolLM2 member appearing in FineWeb\-Edu is expected and harmless, since it remains a member of that model’s training data\.