OriginBlame: Record- and Token-Level Data Provenance for AI Training Datasets

# OriginBlame: Record- and Token-Level Data Provenance for AI Training Datasets
Source: [https://arxiv.org/html/2607.13037](https://arxiv.org/html/2607.13037)
###### Abstract

When a data contributor requests removal, model trainers face a practical gap: unlearning algorithms require a forget set, yet no tool can locate which training records belong to a given author\. Existing provenance systems operate at file or dataset level, forcing catastrophic over\-deletion\. We presentob, a record\- and token\-level data provenance system that propagates author identity through data processing pipelines and resolves revocation requests into precise forget sets via deterministic queries\. Evaluation on 219,555 Wikipedia pages demonstrates that record\-level provenance eliminates dataset\-level over\-deletion \(from 101×\\timesto 1\.3×\\times\), while integration adds 1\.3–4\.0% throughput overhead \(HuggingFace\) and 2\.1–19\.0% \(Datatrove\) on wiki data\. On a 1\.7B model, provenance\-based forget sets improve unlearning by 42% over random baselines\.

OriginBlame: Record\- and Token\-Level Data Provenance for AI Training Datasets

Haolin Xuehxue@mail\.nwpu\.edu\.cn

## 1Introduction

When a data contributor requests removal, the model trainer must identify and remove that contributor’s data from the training set\. Machine unlearning methods such as Negative Preference Optimization \(NPO\)Zhanget al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib15)\)and Representation Misdirection for Unlearning \(RMU\)Liet al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib16)\)can degrade the influence of specific data once a forget set is provided, but all existing research assumes this forget set is already known\. In practice, training datasets are assembled from thousands of sources through data processing, tokenization, and packing; by the time the dataset reaches the trainer, the connection between individual training lines and their original contributors has been lost\. Without fine\-grained provenance, the only options are catastrophic over\-deletion \(discard the entire dataset\) or imprecise post\-hoc inferenceD’Angeloet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib5)\)\. forget\-set quality directly determines unlearning effectiveness\. Benchmarks such as TOFUMainiet al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib2)\)and MUSEShiet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib1)\)evaluate unlearning algorithms on synthetically curated forget sets, but their results cannot translate to real\-world compliance without a mechanism to locate affected data at training granularity\. Figure[1](https://arxiv.org/html/2607.13037#S1.F1)illustrates this gap between knowing*who*to forget and knowing*which data*to forget\.

![Refer to caption](https://arxiv.org/html/2607.13037v1/x1.png)Figure 1:The unlearning pipeline gap\. Existing methods assume a pre\-defined forget set, but locating affected data remains unsolved\. OriginBlame fills this gap\.As shown in Table[1](https://arxiv.org/html/2607.13037#S1.T1), data version control tools \(DVCIterative \([2024](https://arxiv.org/html/2607.13037#bib.bib10)\), LakeFSTreeverse \([2024](https://arxiv.org/html/2607.13037#bib.bib13)\), Delta LakeThe Linux Foundation \([2024](https://arxiv.org/html/2607.13037#bib.bib14)\)\) operate at file or dataset level\. Experiment management tools \(MLflowDatabricks \([2024](https://arxiv.org/html/2607.13037#bib.bib11)\), Weights & BiasesWeights & Biases \([2024](https://arxiv.org/html/2607.13037#bib.bib12)\)\) track metadata but not data origins\. Provenance tools like yProv4MLPadovaniet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib6)\)capture dataset\-level relationships, while DLProvPinaet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib8)\)requires extensive instrumentation\. None provide record\-level provenance with author attribution\.

OriginBlame \(ob\) tracks which training records derive from which source sections and which authors contributed to those sections, via a three\-layer architecture \(Figure[2](https://arxiv.org/html/2607.13037#S4.F2)\) with an independent token\-index layer\. Users integrate with oneob\.track\(\)call per record\. ob provides precise forget sets at both record and token granularity\.

We emphasize that ob does not discover authorship; it*propagates*it\. ob applies to data collection pipelines where author identity is already established by the source environment—for example, MediaWiki revision histories or GitHub commits—but this attribution is*lost*during tokenization and packing\. ob preserves the provenance chain across these transformations, enabling downstream queries \(e\.g\., “find all training lines from authoraia\_\{i\}”\) that would otherwise require post\-hoc inference\.

Table 1:Provenance Tools ComparisonThis paper makes three contributions\. First, we design a three\-layer content\-addressable architecture \(authors←\\leftarrowsections←\\leftarrowdocument\-index\) with no ML or GPU dependencies\. Second, we show that this architecture resolves author revocation requests into precise forget sets via deterministic queries and demonstrate that record\-level provenance eliminates the dataset\-level over\-deletion that file\-level tools incur\. Third, we introduce an independent token\-index layer that records source attribution during tokenization and packing, extending forget\-set production to token granularity without requiring document\-index records\.

## 2Problem Formulation

D’Angelo et al\.D’Angeloet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib5)\)formalize the Forget\-Set Identification \(ForSId\) problem: given a training setDD, a modelMDM\_\{D\}, an unwanted setDuD\_\{u\}, and a wanted setDwD\_\{w\}, find a forget setDf⊆DD\_\{f\}\\subseteq Dwhose removal maximally preserves model behavior onDwD\_\{w\}while altering it onDuD\_\{u\}\. ForSId requires model access to compute per\-sample influence via training\-data gradients\. More fundamentally, its input is*behavioral*evidence—samples whose predictions should change—not metadata\-level queries such as “forget all data by authoraia\_\{i\}\.”

When authoraia\_\{i\}initiates a withdrawal request, the target forget set isF=\{l∈D∣author​\(l\)=ai\}F=\\\{l\\in D\\mid\\text\{author\}\(l\)=a\_\{i\}\\\}\. ComputingFFprecisely without model access is the challenge ob addresses\. Dataset\-level provenance can only answer “data from authoraia\_\{i\}exists inDD,” forcing two extremes: delete the entire dataset or take no action\. Line\-level provenance resolves this by answering “authoraia\_\{i\}contributednnlines,” enabling targeted deletion—as we show in Section[5\.2](https://arxiv.org/html/2607.13037#S5.SS2), record\-level revocation reduces over\-deletion by up to 101×\\timescompared to dataset\-level approaches\.

From this gap, we derive three design requirements\.*Precision*: locate specific data lines, not files or datasets\.*Verifiability*: provenance must be independently auditable, not merely stored internally\.*Minimal invasiveness*: integration must require only a few lines of code—DLProvPinaet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib8)\)serves as a cautionary example, requiring extensive script instrumentation despite low runtime overhead\.

## 3Related Work

As shown in Table[1](https://arxiv.org/html/2607.13037#S1.T1), existing provenance tools stop at file or dataset granularity\. Chen et al\.Chenet al\.\([2026](https://arxiv.org/html/2607.13037#bib.bib7)\)propose fine\-grained sample\-level traceability with contribution scores across ML pipelines but focus on training\-stage data usage auditing rather than authorship attribution\. OriginBlame coexists with tools like DVC and HuggingFace Datasets: DVC manages file versions, while ob manages record\-level author provenance\.

Machine unlearning research similarly assumes the forget set is known \(Table[2](https://arxiv.org/html/2607.13037#S3.T2)\)\. Benchmarks TOFUMainiet al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib2)\)and OpenUnlearning use pre\-defined forget sets, while MUSEShiet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib1)\)evaluates unlearning on real\-world corpora with known forget/retain splits\. Unlearning methods like NPO, RMU, GradAscent, and exact unlearning algorithmsMuresanuet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib3)\)directly receive forget sets as input\. ForSIdD’Angeloet al\.\([2025](https://arxiv.org/html/2607.13037#bib.bib5)\)\(§2\) is the only work addressing forget\-set identification, but operates post\-hoc with model access\. OriginBlame captures origin information at recording time, requiring no model access\.

Table 2:Machine Unlearning Benchmarks: The Forget\-Set Location GapThis positioning complements methods like Attribute\-to\-DeleteGeorgievet al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib4)\), which use datamodels to simulate unlearning but still require the forget set as input\.

IPFS’s content\-addressing designTrautweinet al\.\([2022](https://arxiv.org/html/2607.13037#bib.bib9)\)inspired OriginBlame’s hash chain mechanism, but IPFS operates at chunk level \(256 KB blocks\) while OriginBlame refines granularity to lines\. OriginBlame uses plain JSONL rather than a special file system\. Compared to git\-blame, designed for code editing scenarios, OriginBlame targets generated data scenarios: supporting multi\-source attribution where a single line may fuse contributions from multiple authors, and author\-initiated revocation\.

## 4System Design

### 4\.1Architecture Overview

OriginBlame \(abbreviated as ob\) stores all provenance metadata in a\.ob/directory colocated with the project data\. All files are plain JSONL with no configuration files or central database\.

Three\-tier model\. ob employs a hierarchical storage architecture\. As illustrated in Figure[2](https://arxiv.org/html/2607.13037#S4.F2), three core layers are linked by content\-addressable hashes in a strict parent–child relationship:

Authors layer\(top\) stores identities and revocation tags:id\(SHA\-256 ofname\+email\),name,email, andrevoked\(boolean\)\. Therevokedfield is the single source of truth for revocation, cascading lazily at query time;emailserves as the lookup key forrevoke\.Sections layer\(middle\) stores file\-level copyright:section\_hash\(primary key, SHA\-256 of\{path, authors, license, year\}\),path\(source file path, groups records by file\),authors\(list of author ids\),license,year, andrevoked\(boolean\)\.Document\-index layer\(bottom\) stores per\-record provenance:line\_hash\(SHA\-256 of data content\),file\(output data file\),sources\(list of section hashes identifying contributing sources\),source\_type\(origin indicator, e\.g\.,"track"for explicit tracking\), andrevoked\(boolean\)\. The unique key is\(line\_hash, file, sources\)\. Each document\-index entry corresponds to one output record \(one line in the data file\), linked to its contributing sections and their authors\. Document\-index records do not store line numbers—blamecomputes hashes from data files at query time\.

Storage\. All core files are sharded into 256 buckets by the first two hexadecimal characters of their primary hash:document\-index/00–/ff,sections/00–/ff,authors/00–/ff\. Empty buckets are not created\. This design enables O\(1\) lookup—given aline\_hash,blamedirectly readsdocument\-index/\{first two hex chars\}without scanning unrelated buckets\. Multi\-process safety is achieved without locks: each process writes to isolated files \(docidx\.\{pid\},lock\.\{pid\}\), whichob cleanlater merges into sharded buckets\. Unmerged pid files block read operations until clean completes\. Anembeddings\.\{model\}/directory stores per\-model embedding vectors sharded into 256 buckets for reconcile \(§[4\.4](https://arxiv.org/html/2607.13037#S4.SS4)\); the core requires no ML libraries or GPU—embeddings are computed by the reconcile utility \(§[4\.4](https://arxiv.org/html/2607.13037#S4.SS4)\), not the user\.

Content\-addressable hashing\. ob hashes dict inputs by serializing to JSON with sorted keys then applying SHA\-256National Institute of Standards and Technology \([2015](https://arxiv.org/html/2607.13037#bib.bib21)\); str inputs are hashed as raw UTF\-8 bytes\. No normalization is performed\. The collision probability fornnrecords over akk\-bit hash space is bounded by the birthday approximationp≈n2/2k\+1p\\approx n^\{2\}/2^\{k\+1\}\(Katz and Lindell,[2020](https://arxiv.org/html/2607.13037#bib.bib20), Thm\. A\.4\)\. At the largest evaluated scale \(n=2\.2×105n=2\.2\\times 10^\{5\}records,k=256k=256\), this yieldsp<10−68p<10^\{\-68\}—negligible relative to any practical failure mode\.

Design principles\. ob follows two core principles\. First,*what is not recorded does not exist*: ob performs no post\-hoc inference and does not mark unprovenanced content as provenanced\. Second,*single\-hop provenance*: the system tracks only direct mappings from raw sources to final output, without tracing intermediate processing stages\. Deleting raw data files does not affect provenance or revocation\.

![Refer to caption](https://arxiv.org/html/2607.13037v1/x2.png)Figure 2:Three\-layer reference architecture with hash\-based sharding\.
### 4\.2Core Workflow

OriginBlame is designed as both a library and a CLI tool, with a Rust native implementation and optional Python bindings\. Users initialize a project withob init, creating the\.ob/directory; the Python init wrapper additionally writes a\.gitignorefor temporary files\. One\-time setup requires three CLI calls and one Python API call: \(1\)ob init, \(2\)ob author\.add NAME EMAIL, \(3\)ob register\.add \-\-path PATH \-\-authors AUTHOR \-\-license LICENSE \-\-year YEAR, \(4\)source\.append\(PATH\)\(Python API\)\. Integration into an existing data pipeline requires adding a singletrack\(\)call at the point where each training record is written to disk—for example, insertingtrack\(record, file="data/train\.jsonl"\)immediately before writing the record to the output file\. The user’s existing pipeline logic remains unchanged; ob only observes the data passing through\.

Source management:source\.append\(path\)resolves paths to section records and activates them, registering all sections associated with that path\. An optionalsectionparameter selects a single section by its hash\.source\.pop\(\)removes the most recently appended source, or a specific source by file path\.with ob\.sources\(\.\.\.\):provides scoped tracking\. Track operations associate with all active sources automatically—the caller does not pass sources explicitly\.

ob\.track\(\) workflow: Thetrack\(data, file\)library function records provenance for a single data entry\. Source attribution is resolved either from an explicitsource=parameter \(a file path that resolves to registered sections, or a list of section hashes\) or, by default, from the source stack\. As shown in Figure[3](https://arxiv.org/html/2607.13037#S4.F3), it \(1\) computes the SHA\-256 hash ofdata\(JSON serialization for all types\), \(2\) resolves the active source list into section hashes, \(3\) checks for duplicates \(idempotent\), and \(4\) writes to process\-isolated files via WAL \(lock\.\{pid\}→\\rightarrowdocidx\.\{pid\}\) in the Python API \(the Rust native implementation writes directly to sharded buckets\)\. If the process crashes before the lock is deleted, remaining data can be recovered byob clean\. An optionalembeddingparameter \(Python API\) stores a per\-record embedding vector alongside the document\-index record for later use by reconcile \(§[4\.4](https://arxiv.org/html/2607.13037#S4.SS4)\)\.

![Refer to caption](https://arxiv.org/html/2607.13037v1/x3.png)Figure 3:The track\(\) workflow\. After hash computation, the system checks for duplicates and writes to process\-isolated temporary files under WAL protection\.Query functionality: ob provides two orthogonal query methods\.ob blameperforms a forward query: given a file path and line number, it reads the line, computes SHA\-256 to obtainline\_hash, queriesdocument\-index/\{line\_hash\[:2\]\}, and traverses the provenance chain:line\_hash→\\rightarrowsources \(section\_hashes\)→\\rightarrowsections \(author\_ids, license, path\)→\\rightarrowauthors \(name, email\)\.ob showperforms a reverse query: given\-\-author, it traversesauthors→\\rightarrowsections→\\rightarrowdocument\-indexwithout reading data files\. With\-\-index, both commands use the bucket\-routing index \(Appendix[A](https://arxiv.org/html/2607.13037#A1)\) for faster lookup\. Additional filters include\-\-section\(filter by section hash\),\-\-license\(filter by license name\),\-\-revoked\(show only revoked entries\), and\-\-tokenizer\(token\-level granularity\); these can be combined for intersection queries\.ob statusdisplays aggregate author counts and, with\-\-tokenizer, token\-index statistics including entry counts, active/revoked breakdowns, and total token counts\.

Operation log: The\.ob/logfile records state\-changing operations \(author/section registration, revocation, purge, clean, merge\) as append\-only log lines with timestamps, providing an audit trail for administrative actions\. Read\-only operations \(blame, show, status\) and per\-record tracking are excluded to avoid volume overhead\.ob cleanrotates the log\.ob logqueries the log with optional\-\-opand\-\-sincefilters\.

### 4\.3Revocation and Purge

ob supports revocation at three granularity levels \(Figure[4](https://arxiv.org/html/2607.13037#S4.F4)\), all based on a tag model rather than bulk deletion\. The single source of truth for revocation resides on the author record; all authors are equal—a contributor of a single line holds the same revocation rights as a contributor of the entire dataset\.

Author\-level revoke\.ob revoke \-\-author "z@e\.com"setsrevoked=trueon the author record\. No other files are modified at this point\. When a subsequent query \(e\.g\.,blameorshow\) encounters a revoked author, it evaluates the revocation status lazily along the chain: scan authors forrevoked=true→\\rightarrowfind all sections whoseauthor\_idsinclude that author→\\rightarrowcollect all document\-index entries referencing those section hashes\. The result is a\(file, line\_hash\)list of every data line traceable to the revoked author\. The\-\-reverseflag undoes a revocation by toggling the tag back tofalse; with\-\-tokenizer, revocation also marks matching token\-index entries\.

Section\-level revoke\.ob revoke \-\-section HASHtogglesrevoked=trueon a specific section record, affecting only the document\-index entries that reference that section without revoking the author\. This handles cases where a single file or data source must be withdrawn without affecting other content from the same contributor\.

Line\-level revoke\.ob revoke \-\-line\-hash HASH \-\-file FILEtogglesrevoked=trueon a single document\-index entry, enabling granular line\-level withdrawal\. This is the finest revocation granularity, useful when a specific data line must be removed without affecting any other lines from the same section or author\.

Purge\.ob purge \-\-file FILEphysically deletes revoked data lines from the specified file, leaving document\-index cleanup toob cleanwhich archives revoked metadata intoarchive/\. A safety constraint enforcesno purge without prior revoke: purge can only delete lines that belong to revoked authors or revoked sections, preventing accidental data loss\. The\-\-indexvariant routes directly to the relevant index buckets for faster operation\. The\-\-dry\-runflag previews which lines would be deleted without modifying files\. The\-\-reverseflag restores previously purged lines from the archive directory\.

Lazy cascade rationale\. All revocation levels share a common design: the revoke operation modifies only a tag field on a single record\. By default, queries exclude revoked entries \(document\-index and token\-level alike\); the\-\-revokedflag explicitly requests revoked entries\. This yields two properties: \(1\) revoke is lightweight regardless of how many data lines are affected; \(2\) revocation is fully reversible—canceling a revoke removes the tag with immediate effect\.

![Refer to caption](https://arxiv.org/html/2607.13037v1/x4.png)Figure 4:Revocation levels in ob\. Author revoke cascades lazily from authors to sections to document\-index\. Section\-level revoke marks sections\. Purge physically deletes revoked entries after prior revoke\.
### 4\.4Package Ecosystem

The ob architecture separates a core package from an optional utility package for advanced features\.

Core packageprovides CLI commands for setup \(init,author\.add,register\.add\), source management via Python API \(source\.append/pop\), querying \(blame,show,status\), audit logging \(log\), revocation \(revoke,purge\), and maintenance \(clean,merge,index build,generate\-set,version\)\. The core compiles to a single native binary; users integrate viaauthor\.add \+ register\.add \+ source\.append \+ track\.

Utility features\(enabled via compile\-time feature flags\) provide optional parsers \(parse \-\-parser mediawiki\), reconcile \(reconcile\), and export \(export\-copyright\)\. The core binary performs zero imports of utility modules; when compiled without these features, ob omits the corresponding commands entirely\.

Reconcile mechanism: Reconcile operates in two phases \(Figure[5](https://arxiv.org/html/2607.13037#S4.F5)\)\. Pass 1 attempts hash exact matching: each line is hashed and looked up in the document\-index; a hit means unchanged content\. Pass 2 performs embedding\-based semantic matching via cosine similarity; a hit inherits the old record’s sources, and unmatched old records are marked as orphans\. Lines missing both phases require manualtrack\(\)\.

![Refer to caption](https://arxiv.org/html/2607.13037v1/x5.png)Figure 5:Two\-phase reconcile strategy\. Phase 1 attempts exact hash matching; Phase 2 falls back to embedding\-based semantic matching\. Unmatched lines are flagged as new\.
### 4\.5Token\-Level Provenance

The three\-layer architecture \(authors←\\leftarrowsections←\\leftarrowdocument\-index\) captures per\-record provenance, but tokenization and packing into binary shards eliminates record\-level identity\. ob addresses this with an independenttoken\-indexlayer that links each document’s token range in the packed binary to its source and author chain, operating in parallel to the document\-index\.

Token\-index entry\. Each entry storestoken\_count\(number of tokens produced from one document\),sources\(list of section hashes, linking to the author chain\),tokenizer\(identifier string, e\.g\., “gpt2” or “llama3”\), andrevoked\(boolean\)\. The token\-index is completely independent of the document\-index: entries share the same source references but require no line hashes, and the two layers can exist with or without each other\.

Two operating modes\. In*file mode*, the pipeline writes JSONL data files and callstrack\(\)to create document\-index records; a subsequent tokenize\+pack stage writes token\-index entries\. Both layers coexist, serving record\-level and token\-level queries respectively\. In*streaming mode*, the pipeline tokenizes documents directly without producing intermediate JSONL; only token\-index entries are written, making the token\-index the sole provenance record\.

Storage\. Token\-index entries follow the same PID file pattern as document\-index: each worker writes totoken\-index\.\{tokenizer\}\.\{pid\}, whichob cleanmerges into numbered files undertoken\-index\.\{tokenizer\}/\. To preserve position\-indexing correctness across parallel workers,ob cleanmerges PID files sequentially and computes cumulative token offsets only after all files are merged—not during write\. Different tokenizers are stored in separate directories, following the same pattern as the optional embeddings directory\. The tokenizer name is passed as a parameter; no tokenizer library is imported\.

Token\-level queries\. Because entries are written sequentially during tokenization, theii\-th entry’s token range in the packed binary can be reconstructed from the cumulative sum of preceding entries’token\_countvalues, with its length given by its owntoken\_count\. This makes the token\-index a position\-indexed provenance layer—no per\-token records are needed\.ob show \-\-author NAME \-\-tokenizer gpt2traverses author→\\rightarrowsection→\\rightarrowtoken\-index, summing token counts for all matching entries\.ob revoke \-\-author NAME \-\-tokenizer gpt2marks matching entries as revoked\.ob generate\-set \-\-tokenizer gpt2 \-o forget\.binproduces a binary bitmask where each bit corresponds to one token\-index entry \(1 = revoked, 0 = active\), directly usable by unlearning algorithms\.ob status \-\-tokenizer gpt2reports aggregate token counts\.

Immutability constraint\. Because entry positions encode token ranges, the token\-index does not support archive or purge operations\. Revoked entries remain in the index withrevoked=true; the bitmask produced bygenerate\-setis the sole mechanism for conveying revocation to downstream unlearning algorithms\. If the training pipeline is re\-run with different data, the token\-index is rebuilt from scratch rather than reconciled\.

Framework integration\. The integration pattern is framework\-agnostic: intercept each data record, register provenance metadata, and pass the record through unmodified\. We validate this with two frameworks\. For DatatrovePenedoet al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib17)\), we implement OBProvenanceBlock as a PipelineStep whoserun\(\)method receives a generator of Documents, registers authors and sections fromDocument\.metadata, writes token\-index entries, and yields each document to the next stage—the same interface used by Datatrove’s built\-in blocks \(e\.g\., LanguageFilter, TokensCounter\)\. For HuggingFace Datasets, an equivalent integration wraps the\.map\(\)callback: the function loads zhwiki data viaload\_dataset\(\), appliesauthor\_add \+ register\_section \+ trackper document, and writes provenance alongside output\. Both integrations require the same∼\\sim5\-line adapter and produce identical\.ob/directories; the downstream tokenizer requires no modification\. Section[5\.5](https://arxiv.org/html/2607.13037#S5.SS5)reports the overhead of both integrations\.

## 5Evaluation

This section validates that fine\-grained provenance at both record and token granularity is necessary for practical author revocation\. We measure revocation precision, reconcile recovery rates, storage overhead, query latency across four dataset scales, token\-index integration overhead in a real processing framework, and whether provenance\-based forget sets improve machine unlearning effectiveness over random baselines\. This is a functional validation on real data, not a comprehensive performance benchmark\.

### 5\.1Experimental Setup

The evaluation dataset originates from a Chinese Wikipedia \(zhwiki\) dump comprising 219,555 wiki pages authored by 482,543 unique contributors under CC\-BY\-SA\-4\.0 licenses\. The data construction pipeline has two stages\.

*Stage 1 — Parse\.*A stream parser extracts the latest\-revision wikitext and the full contributor list from each page in the XML dump\. Only namespace\-0 articles with≥\\geq50 characters and at least one contributor are retained, producing one source file per page and a metadata index \(title, authors, year\)\.

*Stage 2 — Track\.*At each target scale, wiki source texts are written as JSONL records via HuggingFace Datasets or Datatrove pipelines, withobprovenance tracking inserted at the document processing stage\. Each record receives author registration, section creation, and provenance linking through theOBTrackpipeline step\. The resulting JSONL lines each track to their wiki page sources through the three\-layer chain, while the Datatrove pipeline additionally produces token\-index entries for token\-level provenance queries\.

### 5\.2Revocation Precision

The central question is whether record\-level provenance granularity matters in practice\. To answer it, we simulate author revocation at the 10k scale for four contributors with widely different contribution shares, using a HuggingFace tokenization pipeline on zhwiki data\. Table[3](https://arxiv.org/html/2607.13037#S5.T3)reports the results\.

Table 3:Revocation precision at 10k scale \(HuggingFace pipeline\)\. Over\-deletion measures how much more data dataset\-level revocation destroys compared to ob’s record\-level precision\.Without record\-level provenance, the only available action is to discard the entire training set—there is no intermediate file\-level option, because wiki pages are collaborative documents that cannot be subdivided by author\. The resulting over\-deletion ranges from 1\.3×\\times\(InternetArchiveBot, 79\.5% coverage\) to 101×\\times\(HuangQQ, 1\.0%\)\. Since each generated record derives from exactly one wiki page, file\-level deletion is equivalent to record\-level deletion here; remaining over\-deletion comes from co\-authorship within pages\.

The over\-deletion factor follows a heavy\-tailed distribution across all authors in the dataset: the four benchmark authors in Table[3](https://arxiv.org/html/2607.13037#S5.T3)span two orders of magnitude in over\-deletion\. This pattern is consistent across all four tested scales \(1k–220k\): the largest\-share author \(InternetArchiveBot, 63–80% coverage\) consistently has 1\.3–1\.6×\\timesover\-deletion, while the smallest\-share authors \(≈\\approx1%\) consistently face≈\\approx100×\\timesover\-deletion under dataset\-level revocation\. All four authors have 100% of their records in multi\-author sections—meaning that even record\-level revocation affects shared content\. This co\-deletion is inherent to collaborative authorship\.

### 5\.3Reconcile Effectiveness

When data files are edited after initial tracking, provenance links break because content hashes no longer match\. We evaluate the reconcile mechanism \(§[4\.4](https://arxiv.org/html/2607.13037#S4.SS4)\) with deterministic mutations \(seed = 42\): 10% text edited, 5% deleted, 5% new lines, usingnomic\-embed\-text\-v1\.5at cosine threshold 0\.85\. Table[4](https://arxiv.org/html/2607.13037#S5.T4)reports the two\-phase recovery rates\.

Table 4:Reconcile recovery rates\. Hash = Pass 1 exact\. Semantic = Pass 2 embedding \(nomic\-embed\-text\-v1\.5, cosine≥\\geq0\.85\)\. Recovery = \(hash \+ semantic\) / mutated lines\. Mutations: seed = 42, 10% text edited, 5% deleted, 5% new\.Hash matching recovers 85–87% of original lines; embedding similarity recovers an additional 10–13%, bringing total recovery to 96–98% across all scales\. Recovery improves with dataset size \(96\.3% at 1k to 98\.2% at 100k\)\. The remaining 2–4% are genuinely new or heavily rewritten content requiring manual re\-tracking viaob\.track\(\)\.

### 5\.4Scalability

Table[5](https://arxiv.org/html/2607.13037#S5.T5)reports operation latency across four scales from 1k to 220k lines, averaged over three runs per scale, using real zhwiki data from the HuggingFace pipeline\. The implementation uses rayon for parallel bucket scanning and a binary index \(Appendix[A](https://arxiv.org/html/2607.13037#A1)\) that stores author\-to\-section routing\.showandpurgeuse parallel rayon workers to scan 256 bucket files concurrently; at 220k lines,showcompletes in 80 ms andrevokein 22 ms\.

Table 5:Operation latency across scales \(3\-run avg\., ms\)\. HuggingFace pipeline data: 1k–220k lines\.*idx*= binary index with author\-to\-section routing\.Line coverage is 100% at all scales, confirming every data line has a complete provenance chain\.

### 5\.5Token\-Level Provenance Evaluation

We evaluate the token\-index layer on three axes: integration overhead in a real processing framework, query performance, and metadata preservation\.

*Framework integration*\. We validate the integration pattern with two frameworks on zhwiki data at four scales: from 1k pages \(2\.8M gpt2 tokens, 4\.8k authors\) to 220k pages \(712M tokens, 483k authors\)\. For DatatrovePenedoet al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib17)\), Pipeline A is a vanilla pipeline \(reader→\\rightarrowtokenizer→\\rightarrowpacked output\); Pipeline B inserts OBProvenanceBlock between reader and tokenizer\. For HuggingFace Datasets, an equivalent\.map\(\)callback applies provenance tracking per document\. Table[6](https://arxiv.org/html/2607.13037#S5.T6)reports both frameworks across all four scales\.

Table 6:Token\-level provenance: integration overhead across four scales on zhwiki data\. Pipeline A: vanilla\. Pipeline B: with provenance tracking\. Query time isob show \-\-author \-\-tokenizer gpt2\.At the 1k\-page scale, the Datatrove provenance block adds 13\.8% throughput overhead while HuggingFace adds only 2\.0%, reflecting that provenance tracking is I/O\-bound and HuggingFace’s pipeline is already bottlenecked on tokenization\. Storage overhead follows a consistent trend: from 1\.33×\\timesat 1k pages to 1\.23×\\timesat 220k pages, reflecting that structured JSONL provenance metadata grows sublinearly relative to token data\. HuggingFace shows a 4\.0% throughput penalty at the 220k\-page scale \(712M tokens\)\. The storage ratio is higher at small scales because\.ob/stores structured metadata \(author names, emails, section hashes, licenses\) while the packed output is compact binary token IDs \(2 bytes each\)\.

*Query performance*\. After Pipeline B completes,ob show \-\-author NAME \-\-tokenizer gpt2traverses the full author→\\rightarrowsection→\\rightarrowtoken\-index chain in 3–69 ms across the four tested scales \(Table[6](https://arxiv.org/html/2607.13037#S5.T6)\)\. At the 220k\-page scale \(712M tokens, 483k authors\),ob revokecompletes in 16 ms\.

*Metadata preservation*\. Pipeline A produces packed binary output with document boundaries but*no author information*—consistent with the behavior of Datatrove’s DocumentTokenizer and Megatron\-LM’s preprocess\_data, which discard metadata during tokenization\. Pipeline B preserves the complete author→\\rightarrowsection→\\rightarrowtoken\-index chain, enabling provenance queries that are impossible in the vanilla pipeline\.

### 5\.6Cross\-Domain Generalization

To evaluate whether ob generalizes beyond wiki\-style text, we apply the same integration pattern to Linux kernel C source code\. The attribution source isgit blame \-e, which provides line\-level authorship for each\.c/\.hfile—a more accurate attribution than the file\-levelgit log \-\-name\-onlyapproach used by The StackKocetkovet al\.\([2022](https://arxiv.org/html/2607.13037#bib.bib18)\)and StarCoderLiet al\.\([2023](https://arxiv.org/html/2607.13037#bib.bib19)\)\. Files are first filtered by commit frequency \(topNNmost\-committed\.c/\.h/\.S/\.rs/\.dts/\.dtsifiles\), then attributed viagit blameon the top 1,000 files \(line\-level, budget\-capped\), with bulkgit logcommit\-frequency attribution for the remainder\. We test three scales: 1 000, 10 000, and all≈\\approx44k attributed source files \(Table[7](https://arxiv.org/html/2607.13037#S5.T7)\)\.

Table 7:Linux kernel cross\-domain benchmark\. Attribution viagit blame \-e\(line\-level authorship on topNN\.c/\.hfiles\)\. Over\-deletion compares file\-level against record\-level precision \(revokingtorvalds\)\.Both frameworks integrate ob provenance tracking with no code changes to the pipeline pattern established in §[5\.5](https://arxiv.org/html/2607.13037#S5.SS5)\. Storage overhead stays under 1\.06×\\timesacross scales, and all ob queries complete under 10 ms\. The HuggingFace throughput penalty is negligible \(<<1%\); Datatrove shows higher overhead \(25\.9–41\.2%\), driven by kernel source files being smaller and faster to process, which makes the provenance block a larger relative cost\.

The over\-deletion analysis reveals that file\-level deletion remains wasteful even with accurate attribution: at the smallest scale \(1 000 files, 671 authors\), revoking Linus Torvalds’ contributions at file granularity would delete 9×\\timesmore lines than necessary\. At the full scale \(≈\\approx44k files\), the over\-deletion factor drops to 1\.3×\\times\.

### 5\.7Application: Machine Unlearning

This subsection asks whether that precision translates to a downstream task: given a revocation request, does ob’s line\-level forget set enable more effective machine unlearning than a same\-size random baseline?

*Setup\.*We fine\-tune Qwen3\-1\.7B \(1\.7B parameters\) with QLoRA \(rank = 16, 4\-bit NF4 quantization, bf16\) on ChatML QA pairs generated from zhwiki data via the Zhipu GLM\-4\-Flash API\. The SFT checkpoint \(1 epoch, lr = 2×\\times10\-4, effective batch = 16\) serves as the shared starting point for all unlearning runs\. We select two contributors with different contribution shares—Berthe \(9\.3%, 14 882 lines\) and Antigng\-bot \(8\.6%, 11 618 lines\)—and construct paired forget sets for each:*line*uses ob’s provenance to select exactly the lines attributed to the target author, while*random*draws a same\-size random subset from the full corpus \(seed = 42\)\. Both forget sets contain the same number of lines; the only difference is*which*lines are selected\.

We apply three unlearning algorithms: NPOZhanget al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib15)\)\(β\\beta= 0\.1, 5 epochs, lr = 10\-5\), RMULiet al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib16)\)\(layer = 14,α\\alpha= 100, 80 steps\), and gradient ascent \(3 epochs, lr = 5×\\times10\-5\)\. Evaluation uses four metrics: forget\-set perplexity \(PPL, higher = better forgetting\), retain\-set PPL \(lower = better utility preservation\), forget\-set ROUGE\-L \(lower = less memorization\), and retain\-set ROUGE\-L \(higher = better preservation\)\. All metrics are computed on sampled subsets \(1 000 lines for PPL, 100 for ROUGE\-L\) from the respective splits\.

Table 8:Machine unlearning with ob line\-level provenance vs\. random baseline \(Qwen3\-1\.7B \+ QLoRA, zhwiki QA data\)\. Arrows indicate desired direction\.Bold: line\-level outperforms random\.†RMU is a known\-limitation baseline \(QLoRA incompatible\)\.‡Gradient ascent causes catastrophic collapse\.forget PPL↑\\uparrowretain PPL↓\\downarrowforget ROUGE↓\\downarrowretain ROUGE↑\\uparrow00\.50\.5111\.51\.5221\.41\.41\.391\.391\.591\.591\.31\.31\.451\.451\.221\.221\.961\.961\.111\.11Line / Random ratioBertheAntigngFigure 6:NPO advantage of ob line\-level forget sets over random baselines, per author\. Each bar shows the ratio \(line\-level / random\); metrics where lower is better are inverted so that\>\>1 always indicates line\-level outperforming random\. Dashed line at 1\.0 represents parity\. Both authors dominate on all four metrics\.*NPO results\.*Table[8](https://arxiv.org/html/2607.13037#S5.T8)shows that ob’s line\-level forget sets dominate the random baseline on*all four metrics*for both authors \(Figure[6](https://arxiv.org/html/2607.13037#S5.F6)\)\. Averaging across the two authors, NPO with line\-level provenance achieves forget PPL = 8\.97 \(2\.56×\\timesSFT baseline\) versus 6\.32 \(1\.80×\\times\) for random—a 42% improvement in forgetting\. Simultaneously, retain PPL is 4\.58 \(1\.52×\\times\) versus 5\.97 \(1\.98×\\times\)—a 23% improvement in utility preservation\. The ROUGE\-L metrics tell the same story: line\-level achieves forget ROUGE\-L = 0\.146 \(0\.37×\\timesSFT\) versus 0\.259 \(0\.65×\\times\), and retain ROUGE\-L = 0\.355 \(0\.87×\\times\) versus 0\.296 \(0\.73×\\times\)\. The consistency across both authors—who differ in contribution share and forget\-set size—indicates that the advantage stems from the*selection quality*of provenance\-based forget sets rather than any author\-specific artifact\.

*RMU results\.*All four RMU variants produce metrics nearly identical to the SFT baseline \(Table[8](https://arxiv.org/html/2607.13037#S5.T8)\), confirming the known incompatibility between RMU and QLoRALiet al\.\([2024](https://arxiv.org/html/2607.13037#bib.bib16)\): 4\-bit quantization introduces step sizes \(∼\\sim0\.05\) that dwarf RMU’s weight updates \(∼\\sim10\-4\), preventing the hidden\-state steering that RMU requires\. All published RMU results use full fine\-tuning\.

*Gradient Ascent\.*All gradient ascent runs produce catastrophic model collapse \(Table[8](https://arxiv.org/html/2607.13037#S5.T8)\), with forget PPL reaching 4\.4–4\.8×\\times108for line\-level and 1\.8×\\times105–6\.7×\\times106for random\. Retain PPL is similarly elevated, confirming total utility loss\. However, the ROUGE\-L metrics reveal a secondary signal: line\-level variants preserve retain ROUGE\-L at 0\.30–0\.32, while random variants collapse to 0\.06—a 5×\\timesdifference\. This suggests that precise forget sets create a more*bimodal*degradation pattern: extreme on the targeted content, relatively preserved elsewhere\. Random forget sets distribute the destructive gradient signal across all content, producing more uniform degradation\. Even with a destructive algorithm, provenance\-based targeting preserves more of the retain set\.

*Takeaway\.*Line\-level provenance enables NPO to achieve simultaneously better forgetting and better retention than random baseline forget sets, across two authors with different contribution profiles\. This demonstrates that the provenance problem—*which data to forget?*—directly affects unlearning quality\.

## 6Discussion

OriginBlame has four limitations\. \(1\) Incremental adoption is difficult: users cannot retroactively add provenance to existing datasets\. \(2\) The parser ecosystem is immature: only a MediaWiki parser is currently available, though our cross\-domain evaluation \(§[5\.6](https://arxiv.org/html/2607.13037#S5.SS6)\) confirms that the core provenance tracking is domain\-agnostic and works with arbitrary attribution sources like git blame\. \(3\) Single\-hop provenance: the system tracks only direct mappings from raw data to final output\. \(4\) The index must be rebuilt when data changes; indexed variants do not consistently outperform the full\-scan path due to rayon parallelism amortizing the scan cost\.

Future work will pursue three directions: expanding the parser ecosystem to support Common Crawl archives, PDF documents, and mainstream NLP dataset formats; exploring incremental adoption paths that allow users to partially embed OriginBlame in existing pipelines; and extending the token\-index layer to support per\-token attribution \(rather than per\-document token\-count attribution\) for integration with token\-level unlearning methods\.

#### Privacy and dual\-use\.

The authors layer stores contributor names and emails alongside their provenance records\. In production deployments handling sensitive contributor data, operators can strip PII from\.ob/by removing name and email fields from author records—leaving only the SHA\-256 identifier—and maintaining a separate encrypted mapping outside the provenance directory\. The operation log \(\.ob/log\) provides an audit trail for revocation and query commands, supporting access\-control policies that restrict provenance queries to authorized compliance personnel\. We note that fine\-grained author attribution could be misused to target individual contributors; deploying organizations should restrict provenance query access accordingly\.

## 7Conclusion

Machine unlearning research has overlooked a prerequisite: locating the affected data\. OriginBlame addresses this gap with a three\-layer content\-addressable architecture that propagates author identity through data processing pipelines, resolving revocation requests into precise forget sets without model access or post\-hoc inference\.

Evaluation across multiple domains and scales confirms practical deployability: record\-level provenance eliminates the 1–2 orders\-of\-magnitude over\-deletion inherent in dataset\-level approaches; integration overhead remains modest across frameworks; and two\-phase reconcile recovers 96–98% of provenance links after mutations\. On the downstream unlearning task, provenance\-based forget sets substantially outperform random baselines, demonstrating that the provenance problem directly affects unlearning quality\.

## Acknowledgements

The author used OpenCode \(GLM\-5\.1\) for language editing and code assistance, and takes responsibility for all AI\-assisted content\.

## References

- L\. Chen, M\. Liu, and H\. Fayek \(2026\)Fine\-grained traceability for transparent ML pipelines\.InProceedings of The Web Conference 2026 \(WWW ’26\),New York, NY, USA\.External Links:[Link](https://arxiv.org/abs/2601.14971)Cited by:[§3](https://arxiv.org/html/2607.13037#S3.p1.1)\.
- A\. D’Angelo, F\. Gullo, and G\. Stilo \(2025\)The forget\-set identification problem\.Machine Learning114\(247\)\.External Links:[Document](https://dx.doi.org/10.1007/s10994-025-06897-9)Cited by:[§1](https://arxiv.org/html/2607.13037#S1.p1.1),[§2](https://arxiv.org/html/2607.13037#S2.p1.8),[Table 2](https://arxiv.org/html/2607.13037#S3.T2.6.6.8.1.1),[§3](https://arxiv.org/html/2607.13037#S3.p2.1)\.
- Databricks \(2024\)MLflow: a platform for the machine learning lifecycle\.Note:[https://mlflow\.org](https://mlflow.org/)Cited by:[Table 1](https://arxiv.org/html/2607.13037#S1.T1.8.8.8.3),[§1](https://arxiv.org/html/2607.13037#S1.p2.1)\.
- K\. Georgiev, R\. Rinberg, S\. M\. Park, S\. Garg, A\. Ilyas, A\. Madry, and S\. Neel \(2024\)Attribute\-to\-delete: machine unlearning via datamodel matching\.arXiv preprint arXiv:2410\.23232\.External Links:[Link](https://arxiv.org/abs/2410.23232)Cited by:[§3](https://arxiv.org/html/2607.13037#S3.p3.1)\.
- Iterative \(2024\)DVC: data version control\.Note:[https://dvc\.org](https://dvc.org/)Cited by:[Table 1](https://arxiv.org/html/2607.13037#S1.T1.2.2.2.3),[§1](https://arxiv.org/html/2607.13037#S1.p2.1)\.
- J\. Katz and Y\. Lindell \(2020\)Introduction to modern cryptography\.3rd edition,CRC Press\.External Links:ISBN 978\-0\-367\-47562\-2Cited by:[§4\.1](https://arxiv.org/html/2607.13037#S4.SS1.p5.6)\.
- D\. Kocetkov, R\. Li, L\. Jivalds, T\. Jablonski, M\. de la Rosa, E\. Li, and et al\. \(2022\)The stack: 3 TB of permissively licensed source code\.Transactions on Machine Learning Research\.External Links:[Link](https://arxiv.org/abs/2211.15533)Cited by:[§5\.6](https://arxiv.org/html/2607.13037#S5.SS6.p1.2)\.
- N\. Li, A\. Pan, A\. Gopal, S\. Yue, D\. Berrios, A\. Gatti, J\. D\. Li, A\. Dombrowski, S\. Goel, G\. Mukobi, N\. Helm\-Burger, R\. Lababidi, L\. Justen, A\. B\. Liu, M\. Chen, I\. Barrass, O\. Zhang, X\. Zhu, R\. Tamirisa, B\. Bharathi, A\. Herbert\-Voss, C\. B\. Breuer, A\. Zou, M\. Mazeika, Z\. Wang, P\. Oswal, W\. Lin, A\. A\. Hunt, J\. Tienken\-Harder, K\. Y\. Shih, K\. Talley, J\. Guan, I\. Steneker, D\. Campbell, B\. Jokubaitis, S\. Basart, S\. Fitz, P\. Kumaraguru, K\. K\. Karmakar, U\. Tupakula, V\. Varadharajan, Y\. Shoshitaishvili, J\. Ba, K\. M\. Esvelt, A\. Wang, and D\. Hendrycks \(2024\)The WMDP benchmark: measuring and reducing malicious use with unlearning\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 28525–28550\.External Links:[Link](https://proceedings.mlr.press/v235/li24bc.html)Cited by:[§1](https://arxiv.org/html/2607.13037#S1.p1.1),[§5\.7](https://arxiv.org/html/2607.13037#S5.SS7.p3.5),[§5\.7](https://arxiv.org/html/2607.13037#S5.SS7.p5.3)\.
- R\. Li, L\. B\. Li, L\. Weber, M\. Xia, M\. del\-Rio, S\. Nie, and et al\. \(2023\)StarCoder: may the source be with you\!\.Transactions on Machine Learning Research\.External Links:[Link](https://arxiv.org/abs/2305.06161)Cited by:[§5\.6](https://arxiv.org/html/2607.13037#S5.SS6.p1.2)\.
- P\. Maini, Z\. Feng, A\. Schwarzschild, Z\. C\. Lipton, and J\. Z\. Kolter \(2024\)TOFU: a task of fictitious unlearning for LLMs\.InProceedings of the Conference on Language Modeling \(COLM\),External Links:[Link](https://arxiv.org/abs/2401.06121)Cited by:[§1](https://arxiv.org/html/2607.13037#S1.p1.1),[Table 2](https://arxiv.org/html/2607.13037#S3.T2.1.1.1.2),[§3](https://arxiv.org/html/2607.13037#S3.p2.1)\.
- A\. I\. Muresanu, A\. Thudi, M\. R\. Zhang, and N\. Papernot \(2025\)Fast exact unlearning for in\-context learning data for LLMs\.InProceedings of the International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.267,pp\. 45272–45288\.External Links:[Link](https://arxiv.org/abs/2402.00751)Cited by:[§3](https://arxiv.org/html/2607.13037#S3.p2.1)\.
- National Institute of Standards and Technology \(2015\)FIPS 180\-4: secure hash standard \(SHS\)\.Note:[https://doi\.org/10\.6028/NIST\.FIPS\.180\-4](https://doi.org/10.6028/NIST.FIPS.180-4)Cited by:[§4\.1](https://arxiv.org/html/2607.13037#S4.SS1.p5.6)\.
- G\. Padovani, V\. Anantharaj, and S\. Fiore \(2025\)yProv4ML: effortless provenance tracking for machine learning systems\.arXiv preprint arXiv:2507\.01078\.External Links:[Link](https://arxiv.org/abs/2507.01078)Cited by:[Table 1](https://arxiv.org/html/2607.13037#S1.T1.12.12.12.3),[§1](https://arxiv.org/html/2607.13037#S1.p2.1)\.
- G\. Penedo, H\. Kydlíček, A\. Cappelli, M\. Sasko, and T\. Wolf \(2024\)DataTrove: large scale data processing\.GitHub\.External Links:[Link](https://github.com/huggingface/datatrove)Cited by:[§4\.5](https://arxiv.org/html/2607.13037#S4.SS5.p7.1),[§5\.5](https://arxiv.org/html/2607.13037#S5.SS5.p2.2)\.
- D\. Pina, L\. Kunstmann, A\. Chapman, D\. de Oliveira, and M\. Mattoso \(2025\)DLProv: a suite of provenance services for deep learning workflow analyses\.PeerJ Computational Science\.External Links:[Document](https://dx.doi.org/10.7717/peerj-cs.2985)Cited by:[Table 1](https://arxiv.org/html/2607.13037#S1.T1.14.14.14.3),[§1](https://arxiv.org/html/2607.13037#S1.p2.1),[§2](https://arxiv.org/html/2607.13037#S2.p3.1)\.
- W\. Shi, J\. Lee, Y\. Huang, S\. Malladi, J\. Zhao, A\. Holtzman, D\. Liu, L\. Zettlemoyer, N\. A\. Smith, and C\. Zhang \(2025\)MUSE: machine unlearning six\-way evaluation for language models\.InProceedings of the International Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2407.06460)Cited by:[§1](https://arxiv.org/html/2607.13037#S1.p1.1),[Table 2](https://arxiv.org/html/2607.13037#S3.T2.2.2.2.2),[§3](https://arxiv.org/html/2607.13037#S3.p2.1)\.
- The Linux Foundation \(2024\)Delta Lake: open\-source storage layer for data lakes\.Note:[https://delta\.io](https://delta.io/)Cited by:[Table 1](https://arxiv.org/html/2607.13037#S1.T1.6.6.6.3),[§1](https://arxiv.org/html/2607.13037#S1.p2.1)\.
- D\. Trautwein, A\. Raman, G\. Tyson, I\. Castro, W\. Scott, M\. Schubotz, B\. Gipp, and Y\. Psaras \(2022\)Design and evaluation of IPFS: a storage layer for the decentralized web\.InProceedings of the ACM SIGCOMM Conference,External Links:[Document](https://dx.doi.org/10.1145/3544216.3544232)Cited by:[§3](https://arxiv.org/html/2607.13037#S3.p4.1)\.
- Treeverse \(2024\)LakeFS: data version control for data lakes\.Note:[https://lakefs\.io](https://lakefs.io/)Cited by:[Table 1](https://arxiv.org/html/2607.13037#S1.T1.4.4.4.3),[§1](https://arxiv.org/html/2607.13037#S1.p2.1)\.
- Weights & Biases \(2024\)Weights & biases\.Note:[https://wandb\.ai](https://wandb.ai/)Cited by:[Table 1](https://arxiv.org/html/2607.13037#S1.T1.10.10.10.3),[§1](https://arxiv.org/html/2607.13037#S1.p2.1)\.
- R\. Zhang, L\. Lin, Y\. Bai, and S\. Mei \(2024\)Negative preference optimization: from catastrophic collapse to effective unlearning\.arXiv preprint arXiv:2404\.05868\.External Links:[Link](https://arxiv.org/abs/2404.05868)Cited by:[§1](https://arxiv.org/html/2607.13037#S1.p1.1),[§5\.7](https://arxiv.org/html/2607.13037#S5.SS7.p3.5)\.

## Appendix ABucket\-Routing Index

showandpurgegrow super\-linearly because they scan all 256 document\-index shard files and expand each record’s section and author links\. The bucket\-routing index eliminates this full scan by pre\-computing which shard files are relevant for each author and section\.

#### Structure\.

The index is a fourth layer stored in\.ob/index/, sharded by the same 2\-character hex prefix as the other layers\. Each shard file contains JSONL records of the form\{"id": "<hash\>", "refs": \["ab", "cd", \.\.\.\]\}, whereidis either an author ID or a section hash, andrefslists the 2\-character bucket prefixes of the next layer’s relevant shards\. Specifically:

- •Author entries:id= author ID,refs= section bucket prefixes containing that author\.
- •Section entries:id= section hash,refs= document\-index bucket prefixes containing records that reference that section\.

The index is built by a single pass over the document\-index and section layers viaob index build\.

#### Query path\.

Forshow \-\- author \-\-index, the query proceeds in three hops, each reading only the relevant shard files:

1. 1\.Look up author entries in the index→\\toobtain section bucket prefixes\.
2. 2\.Read those section shards to find matching sections→\\tolook up section entries in the index→\\toobtain document\-index bucket prefixes\.
3. 3\.Read only those document\-index shards to collect results\.

purge \-\- author \-\-indexfollows the same three\-hop path but returns only\(line\_hash, file\)pairs without expanding section and author details\.

#### Implementation details\.

Two optimizations reduce I/O:

1. 1\.Batched bucket reads\.At each hop, IDs are grouped by their 2\-character prefix so each shard file is read at most once, even when hundreds of IDs map to the same bucket\.
2. 2\.Batch pre\-loading\.Forshow, all author records referenced by matched sections are loaded into memory in a single pass over the author shards before the document\-index loop, replacing per\-resultget\_author\(\)calls with constant\-time dict lookups\.

#### Unified binary index\.

The binary index \(OBIDXF02\) uses type\-tagged references to serve both the document\-index and token\-index layers\. Each reference has a 1\-byte type tag:0x00for document\-index shard prefixes and0x01for token\-index byte ranges\. A single section hash can reference both document\-index shards and token\-index ranges across multiple tokenizers\. Queries filter by type and tokenizer to access only the relevant layer\.

#### Performance\.

Line\-level query latency across all scales is reported in Table[5](https://arxiv.org/html/2607.13037#S5.T5)\(§[5\.4](https://arxiv.org/html/2607.13037#S5.SS4)\); token\-index query latency in Table[6](https://arxiv.org/html/2607.13037#S5.T6)\(§[5\.5](https://arxiv.org/html/2607.13037#S5.SS5)\)\. The index is opt\-in via the\-\-indexflag; without it, both commands fall back to the parallel full\-scan path\.

## Appendix BReconcile Mutation Algorithm

The reconcile benchmark \(§[5\.3](https://arxiv.org/html/2607.13037#S5.SS3)\) simulates data edits by applying three operations to the tracked data file with a deterministic random seed \(42\):

1. 1\.Edit\(10%\): Each line is independently classified by drawingr∼U​\(0,1\)r\\sim U\(0,1\)\. Lines withr<0\.10r<0\.10are edited by replacing 10% of their alphabetic characters with random lowercase letters\. For JSON records, only thetextfield is modified; other fields \(metadata, section hashes\) are preserved\.
2. 2\.Delete\(5%\): Lines with0\.10≤r<0\.150\.10\\leq r<0\.15are removed entirely\.
3. 3\.Insert\(5%\):0\.05×N0\.05\\times Nnew lines are generated by copying random existing lines and modifying 15% of their alphabetic characters\. Insertions are placed at random positions\.

This design preserves the character\-set distribution of the original data \(unlike random byte\-flipping, which would produce invalid UTF\-8 in Chinese text\) while ensuring that edited and inserted lines have no hash overlap with original document\-index entries, isolating the embedding\-based semantic matching phase\.