TeXFix-Bench: An Empirically Grounded Multi-Format Benchmark for LLM-Based Document Source Repair

# An Empirically Grounded Multi-Format Benchmark for LLM-Based Document Source Repair
Source: [https://arxiv.org/html/2608.07617](https://arxiv.org/html/2608.07617)
Prajwal S\. Venkateshmurthy

###### Abstract\.

Scientific and technical writing depends on markup sources that must*compile*: LaTeX, Typst, and Markdown pipelines fail when documents contain missing delimiters, mismatched environments, broken imports, or package conflicts\. Existing document\-repair evaluations inject faults with ad\-hoc text edits that lack an empirically derived fault model\. We present*TeXFix\-Bench*, a multi\-format benchmark for LLM\-based full\-source document repair grounded in a mined fault taxonomy\. First, a Grounded\-Theory study of localized hard\-crash LaTeX faults mined from TeX Stack Exchange, GitHub commits, and package documentation \(168 verified faults, dual open coding atκ=0\.34\\kappa\{=\}0\.34with a documented granularity analysis\) yields an 18\-category taxonomy that we instantiate asDocMut: 48 AST\-aware operators across three formats \(the taxonomy is mined for LaTeX, and Typst and Markdown operators extend it by analogy\)\. A three\-model cross\-benchmark showsDocMutfaults are 5\.6–9\.2 pp harder to repair than pattern\-based mutations on the same seeds, and a real\-error case study \(88 mined human crashes, 67\.0% repair success\) brackets both synthetic sets from below\. Second, we construct 10,437 mutation instances from 743 openly licensed seeds and evaluate seven contemporary LLMs under a fixed zero\-shot protocol with provider\-pinned routing, collecting 48,651 attempts at a total inference cost of≈\{\\approx\}$200\. A complete 6,613\-instance×\\times7\-model balanced matrix confirms all rankings\. A pinned engine gate yields a 27\.5\-point intention\-to\-treat compile spread \(56\.7–84\.2%\)\. Typst is markedly harder than LaTeX and Markdown\. Third, a restoration oracle over 28,129 compiling repairs shows that 13\.6–18\.5% of compiling repairs materially alter document text, and that restoration rank diverges from compile rank: the model with the lowest compile rate restores content best among its successes\. Compile success alone overstates repair quality\. We release the taxonomy,DocMut, and all campaign artifacts\.

automated program repair, LaTeX, Typst, Markdown, mutation testing, fault taxonomy, benchmarking, large language models, document compilation

††copyright:rightsretained††journalyear:2027††ccs:Software and its engineering Software testing and debugging††ccs:Computing methodologies Natural language processing## 1\.Introduction

Markup sources dominate scientific and technical authoring: LaTeX for papers and theses, Markdown for documentation and computational notebooks \(Jupyter, R Markdown, Quarto\), and Typst as a fast\-growing newcomer\. When sources fail to compile or convert, authors face opaque diagnostics, package/engine interactions, and multi\-file project complexity\. LLM\-based assistants are increasingly offered as a fix: paste the broken source, receive a repaired document\. How well this works, and for which formats, faults, and models, has not been measured with the discipline used for code repair\.

Automated program repair \(APR\) research has produced strong executable benchmarks such as Defects4J\(Just et al\.,[2014](https://arxiv.org/html/2608.07617#bib.bib8)\), ManyBugs\(Le Goues et al\.,[2015](https://arxiv.org/html/2608.07617#bib.bib10)\), and SWE\-bench\(Jimenez et al\.,[2024](https://arxiv.org/html/2608.07617#bib.bib7)\)\. Their contracts rest on unit tests or issue\-reproduction environments\. Document markup has a different oracle: engines and converters \(Tectonic/pdfTeX, Typst, pandoc\) decide whether a source is buildable\. Fault patterns also differ from typical programming bugs: unbalanced groups, environment mismatches, missing packages, invalid set\-rules, and shell\-escape requirements\.

Mutation\-based evaluation solves the supply problem, because real broken documents are scarce, poorly licensed, and multi\-file\. But it introduces a scientific risk:*unrealistic mutants*\. If injected faults do not resemble errors authors actually make, repair scores misstate field performance\. Code mutation addressed this by grounding operators in fault taxonomies\(Offutt et al\.,[1996](https://arxiv.org/html/2608.07617#bib.bib12); Jia and Harman,[2011](https://arxiv.org/html/2608.07617#bib.bib6)\)\. Document repair has lacked an analogous foundation\. Prior document\-repair benchmarks \(including our own v0\.3 pilot\(Venkateshmurthy,[2026b](https://arxiv.org/html/2608.07617#bib.bib18)\)\) injected faults with pattern\-based text edits that are cheap and reproducible but unvalidated against human fault distributions\.

This paper closes that loop end to end \(Figure[1](https://arxiv.org/html/2608.07617#S3.F1)\): we*mine*real compilation faults,*derive*a taxonomy,*instantiate*it as a mutation library,*validate*that the library produces harder and more realistic repair tasks than pattern\-based injection, and then*evaluate*seven contemporary LLMs at scale under both a compile oracle and a content restoration oracle\.

#### Contributions\.

1. \(1\)A mined fault taxonomy for LaTeX compilation crashes\(Section[3](https://arxiv.org/html/2608.07617#S3)\): a dual\-sampled mining pipeline over TeX\.SE, GitHub commits, and package documentation with hard\-crash verification and explicit exclusion funnels\. The result is 168 localized faults dual\-coded into 18 axial categories, with an honest reliability analysis \(κ=0\.34\\kappa\{=\}0\.34under free\-form open coding, diagnosed as granularity asymmetry, plus a codebook\-operationalizability check atκ=0\.64\\kappa\{=\}0\.64\)\.
2. \(2\)DocMut\(Venkateshmurthy,[2026a](https://arxiv.org/html/2608.07617#bib.bib17)\)\(Section[4](https://arxiv.org/html/2608.07617#S4)\): a 48\-operator, multi\-format mutation library \(25 LaTeX \+ 15 Typst \+ 8 Markdown\) with hard/soft tracks, difficulty tiers, deterministic PRNG control, and a render\-diff equivalence guard \(controlled F1=0\.937\{=\}0\.937\)\.
3. \(3\)Operator validation\(Section[5](https://arxiv.org/html/2608.07617#S5)\): a three\-model cross\-benchmark on matched seeds showsDocMutfaults are5\.6–9\.2 pp harder to repairthan pattern\-based mutations for every model family tested\. A real\-error case study \(88 mined human hard crashes, 67\.0% repair success\) shows human faults are harder still\.
4. \(4\)TeXFix\-Bench v0\.4\(Section[6](https://arxiv.org/html/2608.07617#S6)\): 10,437 unique hard instances from 743 openly licensed seeds drawn from five public source families \(template packs, Overleaf, CTAN, GitHub, Typst Universe\) plus pandoc\-derived Markdown, engine\-gated under pinned Tectonic/Typst/pandoc\.
5. \(5\)A seven\-model hosted evaluation\(Sections[7](https://arxiv.org/html/2608.07617#S7),[8](https://arxiv.org/html/2608.07617#S8)\): 48,651 attempts at a transparently accounted≈\{\\approx\}$200 total inference cost with provider\-pinned routing \(allow\_fallbacks=false\) and a per\-request provider ledger\. Primary claims verified on acomplete 6,613\-instance×\\times7\-model balanced matrix\.
6. \(6\)A two\-oracle result\(Section[9](https://arxiv.org/html/2608.07617#S9)\): beyond a 27\.5\-point ITT compile spread and a clear Typst difficulty gap, a restoration oracle over 28,129 compiling repairs shows 13\.6–18\.5% of compiling repairs materially alter document text, with restoration rank diverging from compile rank\. Compile\-only leaderboards overstate repair quality\.

#### Scope\.

We evaluate*full\-source*repair under a fixed harness\. We do not claim a universal model ranking, visual/layout PDF equivalence, or ecological coverage of all real authoring errors\. The mined taxonomy targets*localized hard crashes*under a fixed engine\. Multi\-hunk reconfigurations are an explicit out\-of\-scope class \(Section[3](https://arxiv.org/html/2608.07617#S3)\)\.

#### Paper roadmap\.

Section[2](https://arxiv.org/html/2608.07617#S2)positions related work\. Sections[3](https://arxiv.org/html/2608.07617#S3)–[5](https://arxiv.org/html/2608.07617#S5)derive and validate the fault model\. Sections[6](https://arxiv.org/html/2608.07617#S6)–[7](https://arxiv.org/html/2608.07617#S7)describe the benchmark and campaign\. Section[8](https://arxiv.org/html/2608.07617#S8)answers the research questions\. Sections[10](https://arxiv.org/html/2608.07617#S10)–[11](https://arxiv.org/html/2608.07617#S11)discuss implications and threats\.

## 2\.Related Work

### 2\.1\.APR benchmarks

Defects4J, ManyBugs/IntroClass, and SWE\-bench fix task contracts around tests or GitHub issues\(Just et al\.,[2014](https://arxiv.org/html/2608.07617#bib.bib8); Le Goues et al\.,[2015](https://arxiv.org/html/2608.07617#bib.bib10); Jimenez et al\.,[2024](https://arxiv.org/html/2608.07617#bib.bib7)\)\. Smith et al\. warn that test\-passing patches can overfit\(Smith et al\.,[2015](https://arxiv.org/html/2608.07617#bib.bib15)\)\. Monperrus surveys the broader APR literature\(Monperrus,[2018](https://arxiv.org/html/2608.07617#bib.bib11)\)\. We adopt the same caution for compile oracles\. A minimal stub can “pass” compilation, so we address this directly with a restoration oracle \(Section[9](https://arxiv.org/html/2608.07617#S9)\)\.

### 2\.2\.Mutation testing

Surveys by Jia and Harman and by Papadakis et al\. frame operator design, equivalent mutants, and determinism\(Jia and Harman,[2011](https://arxiv.org/html/2608.07617#bib.bib6); Papadakis et al\.,[2019](https://arxiv.org/html/2608.07617#bib.bib14)\)\. Offutt et al\. established that operator selection should be grounded in empirical sufficiency\(Offutt et al\.,[1996](https://arxiv.org/html/2608.07617#bib.bib12)\)\. Code mutators \(PIT, Stryker, Universal Mutator\(Groce et al\.,[2023](https://arxiv.org/html/2608.07617#bib.bib3)\)\) target application languages\. Markup constructs such as braces, environments, packages, and engine flags have no direct analogue in those toolkits, which motivates deriving a domain taxonomy*before*operator design\.

### 2\.3\.LLM\-based code repair

DeepFix and diagnostic\-feedback repair address student programs\(Gupta et al\.,[2017](https://arxiv.org/html/2608.07617#bib.bib4); Yasunaga and Liang,[2020](https://arxiv.org/html/2608.07617#bib.bib23)\)\. Conversational APR systems show strong results on classic bug datasets\(Xia and Zhang,[2023](https://arxiv.org/html/2608.07617#bib.bib22)\)\. These works motivate LLM repair but use different oracles and artifacts\.

### 2\.4\.Document\-adjacent systems

EqFix repairs individual equations\(Zhu and He,[2022](https://arxiv.org/html/2608.07617#bib.bib24)\)\. TexOCR reconstructs LaTeX from page images\(Wang et al\.,[2026](https://arxiv.org/html/2608.07617#bib.bib20)\)\. Editor assistants target writing workflows rather than compile oracles\(Wen et al\.,[2024](https://arxiv.org/html/2608.07617#bib.bib21); Hou et al\.,[2026](https://arxiv.org/html/2608.07617#bib.bib5)\)\. Tan and Rigger study cross\-engine inconsistencies among*successfully*typeset documents\(Tan and Rigger,[2024](https://arxiv.org/html/2608.07617#bib.bib16)\)\. Our setting is the dual: sources that do not produce output at all\. We pin engines accordingly\.

### 2\.5\.Qualitative coding in empirical SE

Our dual open coding follows Grounded Theory practice and thematic analysis guidance\(Braun and Clarke,[2006](https://arxiv.org/html/2608.07617#bib.bib2)\)\. We report Cohen’sκ\\kappawith Landis and Koch bands\(Landis and Koch,[1977](https://arxiv.org/html/2608.07617#bib.bib9)\)and discuss how free\-form open coding depressesκ\\kapparelative to fixed\-codebook designs\(Viera and Garrett,[2005](https://arxiv.org/html/2608.07617#bib.bib19)\)\.

### 2\.6\.Positioning

*To our knowledge*, TeXFix\-Bench is the first multi\-format \(LaTeX, Typst, Markdown\) engine\-gated repair benchmark whose mutation operators are validated against a mined human fault taxonomy, and the first to report compile and content\-restoration oracles jointly at this scale\.

## 3\.An Empirically Grounded Fault Taxonomy

Mine human faultsTeX\.SE, GitHub, docs168 localizedTaxonomy18 categoriesdual\-codedDocMut48 operators3 formatsBenchmark10,437 instances743 seeds7\-model campaign48,651 attemptsprovider\-pinnedTwo oraclescompile gaterestorationSpdfS\_\{\\mathrm\{pdf\}\}validation: 3\-model cross\-benchmark,real\-error case study

Figure 1\.End\-to\-end pipeline: mined faults ground the taxonomy, the taxonomy grounds the operators, and the resulting benchmark is scored under both a compile oracle and a content\-restoration oracle\.We target*localized hard crashes*: single\-site \(or tightly coupled\) faults that produce non\-zero engine exit codes under Tectonic\. We deliberately exclude multi\-hunk reconfigurations, warning\-only diagnostics, and multi\-file asset puzzles that confound mutation oracles\.

### 3\.1\.Mining pipeline

We mine three*human*sources for taxonomy derivation and keep a fourth source \(AI repair failures\) for difficulty analysis only\.

#### TeX Stack Exchange \(dual sampling\)\.

From the TeX\.SE data dump we select questions taggederrorsorcompilation\-errorwith accepted answers \(N=2,655N\{=\}2\{,\}655universe\):Popular\(top 500 by views\) andRandom\(500 seeded draws, zero overlap\)\. Dual sampling reduces the risk that the taxonomy reflects only highly upvoted threads\(Braun and Clarke,[2006](https://arxiv.org/html/2608.07617#bib.bib2)\)\.

#### GitHub commits with tangled\-commit isolation\.

Four search queries yield 1,662 unique commits across 1,122 repositories \(median patch size 337 lines\)\. A raw “fix \+\.tex” commit is almost never a pure compilation repair, so we apply a three\-stage high\-precision funnel: \(1\) exactly one\.texfile, under 20 non\-context lines, and a message matching fix/compile/error/broken, leaving 71 survivors, \(2\) retain only fault\-fix lines \(delimiters, packages, commands, braces\), and \(3\) manual verification on a random 50 \(heuristic prelabel accuracy 0\.94\)\. The 20\-line threshold sits on the commit\-size CDF: only 13\.3% of commits are that small, and raising it to 30 lines adds 2\.8 pp of candidates while admitting prose\-mixed hunks\. The 95\.7% exclusion rate is a deliberate precision bias: multi\-file and large commits systematically confound localization and are the wrong seed material for single\-operator mutation design\.

#### Package documentation\.

For 13 widely used packages we extract 30 hard\-crash troubleshooting patterns from manuals/CTAN \(29 verified hard\-crash\)\. These seed missing\-package, option\-clash, and engine\-requirement faults that forums under\-sample\.

#### Hard\-crash verification and localization\.

Every candidate is compiled with Tectonic\. Only non\-zero exits survive \(237verified crashes, 86\.2% survival among reconstructions\)\. We then classify on the*post\-fix*\(compiling\) version: diff broken↔\\leftrightarrowfixed, map changes to coarse AST node types, keep single\-node changes\. Result:168 localized\(70\.9%\), and69 reconfigurationsexcluded, a distinct multi\-concern class \(babel option \+ package changes, biblatex/preamble rewrites\) that violates the single\-operator contract and is explicit future work, not discarded evidence\. By source: TeX\.SE 143, GitHub 14, package docs 11\.

### 3\.2\.Dual coding and reliability

Annotator 1 \(author\) and Annotator 2 \(independent\) labeled all 168 faults with free\-form open codes and*no shared codebook*\. Open strings were then mapped to 18 axial categories via documented synonym rules \(coding handbook released\)\.

#### Reliability\.

On axial labels after synonym merge, Cohen’sκ=0\.34\\kappa\{=\}\\mathbf\{0\.34\}\(po=0\.41p\_\{o\}\{=\}0\.41,pe=0\.11p\_\{e\}\{=\}0\.11\),*fair*under Landis and Koch\(Landis and Koch,[1977](https://arxiv.org/html/2608.07617#bib.bib9)\)\. The primary cause is open\-code granularity asymmetry: Annotator 1 produced 22 consolidated labels, while Annotator 2 produced 120 near\-instance\-specific phrases\. Many\-to\-one synonym collapse cannot recover agreements that never existed at matching granularity\. Residual disagreements \(e\.g\., “undefined control sequence” symptom vs\. missing package root cause\) were resolved with diagnostic\-priority and specificity rules \(audit trail released\)\. Fairκ\\kappaunder free\-form open coding with asymmetric aggregation is expected\(Braun and Clarke,[2006](https://arxiv.org/html/2608.07617#bib.bib2); Viera and Garrett,[2005](https://arxiv.org/html/2608.07617#bib.bib19)\)\. We report it as a finding about coding methodology and treat a second\-round closed\-codebook human annotation as future work\.

#### Codebook operationalizability check\.

As a complementary check,*not*a second inter\-rater measurement, we implemented the 18 categories plus the resolution rules as a deterministic classifier over compiler diagnostics and fault\-site features\. It reproduces Annotator 1’s axial labels atκ=0\.64\\kappa\{=\}0\.64\(po=0\.70p\_\{o\}\{=\}0\.70\): the categories are well\-defined enough to be assigned mechanically from diagnostics at substantial agreement with a human coder\. Because the classifier encodes the resolution rules, this does not substitute for human–human reliability\.

### 3\.3\.Taxonomy and frequencies

Figure[2](https://arxiv.org/html/2608.07617#S3.F2)shows the resolved distribution\. Top categories among the 168 resolved faults: UndefinedControlSequence \(18\.5%, Wilson 95% CI\[13\.3,25\.0\]\[13\.3,25\.0\]\), OtherFatalCompile \(10\.1%\), InputEncodingFault \(9\.5%\), BraceGroupFault and MathModeFault \(8\.9% each\), GraphicsIncludeFault \(7\.1%\)\.

UCSOtherEncBraceMathGfxPkgTabDocFontEnvUEnvMFileShellBabelAmsPreamClash0101020203030313117171616151515151212998888776666664433222211Faults \(N=168N\{=\}168\)Figure 2\.Frequency of resolved axial categories among the 168 localized hard crashes\. UCS = UndefinedControlSequence, Other = OtherFatalCompile, Enc = InputEncoding, Brace = BraceGroup, Math = MathMode, Gfx = GraphicsInclude, Pkg = MissingRequiredPackage, Tab = TableAlignment, Doc = DocumentBoundary, Font = FontEngine, EnvU = UndefinedEnvironment, EnvM = EnvironmentMismatch, File = MissingExternalFile, Shell = ShellEscape, Babel = BabelLanguage, Ams = AmsmathStructure, Pream = PreambleMisuse, Clash = PackageOptionClash\.Of 18 nonempty categories, 13 \(72\.2%\) are targeted by at least oneDocMutLaTeX operator, andinstance coverageis 76\.2% \(CI\[69\.2,82\.0\]\[69\.2,82\.0\]\)\. Uncovered categories \(InputEncodingFault, MissingExternalFile, BabelLanguageFault, AmsmathStructureFault\) are first\-class backlog items\.

## 4\.The DocMut Mutation Library

DocMutv0\.2\.0\(Venkateshmurthy,[2026a](https://arxiv.org/html/2608.07617#bib.bib17)\)instantiates the taxonomy as 48 operators: 25 LaTeX, 15 Typst, and 8 Markdown \(Table[1](https://arxiv.org/html/2608.07617#S4.T1)\)\. Operators are implemented against format\-specific structure \(not blind regex\) and labeled with*tier*and*hard/soft track*\. Hard operators are retained only when the golden compiles and the broken source fails under the designated engine\. Soft operators target diagnostic/semantic faults that may still compile\. Site selection uses a deterministic PRNG\. The mined taxonomy is LaTeX\-only\. Typst/Markdown operators share the adequacy framework and are designed by analogy \(a scoped external\-validity note, Section[11](https://arxiv.org/html/2608.07617#S11)\)\.

Table 1\.DocMut operator inventory\.#### Example\.

A Tier\-1TEX\-BRC\-DRP\(BraceDrop\) mutation removes one closing brace at a PRNG\-selected site, leaving a source that Tectonic rejects with a runaway\-argument error:

> golden:\(\\vec\{u\}\-\\vec\{w\}\) broken:\(\\vec\{u\}\-\\vec\{w\)

A Tier\-3TEX\-SHL\-ESCmutation instead injects aminteddependency whose repair requires an engine\-flag or package substitution rather than a local token edit, spanning the two ends of the difficulty range\.

#### Difficulty tiers\.

Tier 1 \(surface/structural\):Delimiters, unclosed environments/blocks, missing class or import declarations\.

Tier 2 \(semantic/structure\):Wrong commands or types, argument drops, dictionary keys, soft environment swaps\.

Tier 3 \(realistic author patterns\):Shell\-escape requirements, glossary/font/package\-order issues, HTML/image mistakes\.

#### Taxonomy\-driven refinement\.

Pilot operators existed before the formal Grounded\-Theory study\. The study validates and refines them: \(1\) BraceDrop split into runaway vs\. extra\-brace subpatterns, \(2\) PackageDrop refined into missing\-package vs\. option\-clash hard patterns \(excluding warning\-only hyperref load\-order styles\), \(3\) soft FontSwap deprecated for hard\-crash evaluation\.

#### Render\-diff equivalence guard\.

Equivalent mutants must be excluded from hard repair sets\. We compile golden and mutant under Tectonic, extract text with a fixedpdftotextbounding box, normalize \(form\-feed strip, whitespace collapse, no lowercasing\), and compare\. On 120 controlled pairs \(true equivalents such as identity, whitespace, and comment edits, and true non\-equivalents such as body text, math, and preamble edits\), precision is 0\.88, recall 1\.0,F1=0\.937\{=\}0\.937, over\-normalization rate 0\. Structure and math scopes reach F1=1\.0\{=\}1\.0\. Preamble edits reach F1=0\.85\{=\}0\.85\(style changes can leave body text unchanged\)\.

## 5\.Operator Validation

Does taxonomy\-guided injection actually change measured repair difficulty? We answer with a controlled cross\-benchmark, a per\-operator analysis, and a real\-error case study\.

### 5\.1\.Cross\-benchmark: pattern\-based vs\. DocMut

From the same seed pool we build two equal\-sized hard\-fail mutation sets \(n=391n\{=\}391each\):Set A, pattern\-based mutations \(delete brace / misspell command / drop package line\), extending the injection style of earlier document\-repair benchmarks\.Set B,DocMuthard\-track operators on matched seeds\. Both sets require broken sources that fail to compile\. Empty API responses count as missing, not successful repairs\. We repair both sets with three models from distinct families \(Table[2](https://arxiv.org/html/2608.07617#S5.T2)\)\.

Table 2\.Cross\-benchmark: compile success \(exact match\) by model,n=391n\{=\}391per set\. Gap==Set A−\-Set B compile success\.The direction replicates in all three families:DocMutfaults are5\.6–9\.2 pp harderunder compile success and 2\.8–14\.6 pp harder under exact match, with nearly identical API completion \(≥\\geq97\.7%\) and cost\. The strongest model \(Grok\) shows the smallest gap, but no model findsDocMutfaults easier\. All three gaps are statistically significant under exact McNemar tests on the seed\-paired outcomes: DeepSeekb=60b\{=\}60/c=24c\{=\}24withp<0\.001p\{<\}0\.001, Grokb=42b\{=\}42/c=20c\{=\}20withp=0\.007p\{=\}0\.007, and Codestralb=65b\{=\}65/c=30c\{=\}30withp<0\.001p\{<\}0\.001, wherebbcounts seeds repaired only in Set A andcconly in Set B\.*How*faults are injected changes measured repair difficulty by up to ten points under a fixed model and seed pool\.

### 5\.2\.Per\-operator difficulty \(Set B, DeepSeek\)

High compile success: BraceStray 95\.6%, EnvRename 93\.1%, ItemMisplaced 91\.3%\. Low: ShellEscapeReq47\.2%, MathDisplay 50\.0% \(smallnn\), DocumentClassDrop 65\.4%\. Exact\-match rates sit far below compile rates throughout \(e\.g\., PackageDrop 77\.8% compile but 5\.6% exact\): models often compile via non\-minimal rewrites, inventing substitute packages or deleting dependent body code rather than restoring the original line\. This gap motivates the restoration oracle of Section[9](https://arxiv.org/html/2608.07617#S9)\.

### 5\.3\.Real\-error case study

For ecological validity we also repaired mined human hard crashes \(DeepSeek V4 Pro\)\. From the 168 localized faults we stratified 100 cases\. Excluding custom\-class documents and one non\-failing sandbox case leaves88evaluated cases \(dummy assets provisioned,\.texunmodified\)\. Compile success is67\.0%\(59/88\), belowDocMutSet B \(81\.3%\) and far below pattern\-based Set A \(90\.5%\) \(Table[3](https://arxiv.org/html/2608.07617#S5.T3)\)\. In\-sample: math delimiters 9/9 and brace balance 5/5, but input encoding 3/8, undefined control sequences 4/10, missing files 1/4\.Case study only, not a population estimate, but the ordering \(pattern\>\>DocMut\>\>real\) indicates taxonomy\-guided injection moves synthetic evaluation*toward*real difficulty, not past it\.

Table 3\.Same model, three fault sources \(DeepSeek V4 Pro\)\.

## 6\.Benchmark Construction

### 6\.1\.Dataset construction

Seeds are real, openly licensed documents \(no LLM\-authored templates\): template packs \(CC0\), Overleaf Gallery \(CC BY / CC0\), CTAN examples \(LPPL\), GitHub LaTeX/Typst repositories \(MIT / Apache / CC0\), Typst Universe packages, and pandoc\-converted Markdown derived from simple LaTeX\. After filtering and SHA\-256 deduplication of broken sources, the stratified freeze contains10,437instances from743unique seeds \(Table[4](https://arxiv.org/html/2608.07617#S6.T4)\)\. Instance\-level licenses: MIT \(4,543\), CC\-BY\-4\.0 \(2,575\), CC0\-1\.0 \(1,583\), LPPL\-1\.3c \(1,045\), Apache\-2\.0 \(508\), MIT\-0 \(183\)\.

Table 4\.TeXFix\-Bench v0\.4 corpus \(frozen evaluation set\)\.
### 6\.2\.Instance generation

Each instance recordsid, format,operator\_code, tier, track,golden\_source,broken\_source, content hashes, mutation site metadata, andcompile\_engine\. Construction uses a fixed salt \(20260802\) for deterministic PRNG site selection\. TheDocMutcatalog defines 48 operators\. The frozen benchmark instantiates 39 of them with nonzero instance support after gate filtering\. The other nine produced zero retained mutants under the construction budget \(no eligible sites, gate failures, or inadequate soft mutations\), and one \(TEX\-MTH\-OPS\) has single\-instance support\. The engine\-gate ledger therefore covers38operators \(Appendix[A](https://arxiv.org/html/2608.07617#A1)\)\. Tier counts are 6,364 \(Tier 1\), 2,620 \(Tier 2\), and 1,453 \(Tier 3\)\.

### 6\.3\.Engine\-gate oracle

Designated engines match construction: Tectonic 0\.17\.0 \(LaTeX\), Typst 0\.15\.1 \(Typst\), pandoc 3\.10\.1 \(Markdown, HTML conversion primary\)\. Each compile uses a fresh temporary directory and a 30 s wall\-clock timeout\. Controls on campaign\-touched instances \(N=7,007N\{=\}7\{,\}007\): golden compile rate100%\. Broken residual compile rate5\.7%, concentrated in soft\-track and Markdown operators retained for structural change even when conversion still succeeds\. Equivalent\-mutant detection compares SHA\-256 of the candidate \(after light fence unwrap\) to the golden hash\.

## 7\.Evaluation Methodology

### 7\.1\.Models and providers

Table[5](https://arxiv.org/html/2608.07617#S7.T5)lists the seven models accessed through OpenRouter\(OpenRouter,[2026](https://arxiv.org/html/2608.07617#bib.bib13)\)withallow\_fallbacks=false\(no silent provider substitution\)\. Because hosted delivery is part of our results, we release a per\-request provider ledger\. Table[5](https://arxiv.org/html/2608.07617#S7.T5)summarizes it\. Four models were served≥\\geq99\.9% by their first\-party providers, so the delivery failures reported below are properties of those serving stacks, not marketplace routing noise\.

Table 5\.Model panel with upstream serving providers \(from the per\-request audit ledger,allow\_fallbacks=false\)\.
### 7\.2\.Prompt design

We use a single zero\-shot prompt for all models, temperature0,max\_tokens=4096\{=\}4096\(ablated at 16,384 in Section[8](https://arxiv.org/html/2608.07617#S8)\), no tools, no streaming\. The system message \(format substituted\) is:

> *You are a document repair expert\. You are given a\{FORMAT\}document that FAILS to compile to PDF\. Return ONLY the fully corrected\{FORMAT\}source code\. Do not add commentary, do not use code fences, do not add markdown formatting\. Preserve all content; change only what is needed to make the document compile cleanly\.*

The user message is the rawbroken\_source\.

### 7\.3\.API campaign and the balanced matrix

The planned matrix is10,437×7=73,05910\{,\}437\\times 7=73\{,\}059requests \(one repetition per pair\)\. The campaign ran with checkpointed scheduling at concurrency 10, retrying only transient 5xx/429/timeout responses\. All outcomes are retained under intention\-to\-treat\. Evaluation recorded48,651attempts at a total provider cost of≈\{\\approx\}$200 \(per\-model cost breakdown in the artifact\), full replication of the campaign is therefore accessible to any modestly resourced group\. The schedule prioritized complete model coverage per instance over complete instance coverage: every model covered 7,006–7,007 instances except DeepSeek\-Flash \(6,613\)\. The intersection yields acomplete balanced matrix of 6,613 instances×\\times7 models\(46,291 attempts, 95\.1% of recorded volume\)\. All primary claims are stated on this balanced matrix, with full\-ITT numbers in the artifact\. Remaining schedule cells are unfinished, not silently dropped successes\.

### 7\.4\.Metrics

API delivery:Fraction of attempts that return a usable candidate \(ledger statusok\)\.

CRapi:Compile success among API\-ok attempts \(conditional skill\)\.

CRitt:Compile success among*all*attempts \(deployment\-facing rate that embeds delivery\)\.

Equivalent\-mutant rate:Fraction of API\-ok candidates whose SHA\-256 equals the golden \(exact reverts\)\.

RestorationSpdfS\_\{\\mathrm\{pdf\}\}:Token\-LCS Dice between candidate and goldenpdftotextoutput for compiling non\-revert candidates \(Section[9](https://arxiv.org/html/2608.07617#S9)\)\.

We report rates with explicit denominators, and Wilson 95% CIs on CRitt\. CRapiis the model\-skill metric, CRittadditionally reflects each serving stack\. We do not treat either alone as a universal ranking\.

## 8\.Results: Compile Oracle

Unless noted, numbers are from the balanced matrix \(Section[7\.3](https://arxiv.org/html/2608.07617#S7.SS3)\)\. Full\-set ITT rates differ by at most 0\.6 pp and preserve all orderings\.

### 8\.1\.RQ1: How do contemporary LLMs perform?

Table[6](https://arxiv.org/html/2608.07617#S8.T6)and Figure[3](https://arxiv.org/html/2608.07617#S8.F3)summarize primary outcomes\. Under CRitt, Grok\-4\.3 leads at84\.2%with perfect delivery, while Qwen3\.7\-max is lowest at56\.7%despite the highest conditional CRapi\(94\.4%\) among delivering attempts, a 27\.5\-point spread\. Codestral and Llama\-4 Maverick combine≥\\geq99\.7% delivery with mid\-70% ITT\. GLM and DeepSeek\-Pro show low delivery with high conditional compile rates\. Because Qwen and DeepSeek\-Pro were served 100% by their first\-party providers \(Table[5](https://arxiv.org/html/2608.07617#S7.T5)\), their delivery deficits are serving\-stack properties, not router artifacts\.

Table 6\.Overall results by model on the balanced matrix\. Delivery = API\-ok/N/N, CRapi= compile among API\-ok, CRitt= compile among all attempts\. 95% Wilson CIs on CRitt\.GrokCodestralMaverickDS\-FlashDS\-ProGLMQwen02020404060608080100100Rate \(%\)CRittCRapiFigure 3\.Deployment\-facing CRittvs\. conditional CRapion the balanced matrix\. High conditional skill with low delivery \(Qwen, GLM, DS\-Pro\) yields weak end\-to\-end rates\.
### 8\.2\.RQ2: Which document formats are hardest?

On the full recorded set, Markdown is easiest under CRitt\(90\.2%,N=2,789N\{=\}2\{,\}789\), then LaTeX \(74\.2%,N=30,814N\{=\}30\{,\}814\), then Typst \(60\.3%,N=15,048N\{=\}15\{,\}048\)\. Conditional on API\-ok, Markdown is near ceiling \(99\.9%\), LaTeX 85\.5%, Typst 77\.7%\. The Typst gap appears across nearly all models \(per\-model Typst CRittspans 42\.8–86\.1%\) and is consistent with newer syntax and less public training mass, though we do not observe training corpora directly\.

### 8\.3\.RQ3: Does difficulty scale with tier?

The tier gradient is modest under CRitt\(T1 71\.5%, T2 71\.4%, T3 66\.9%\) and clearer conditionally \(CRapi86\.0/84\.6/76\.1%\)\. Tier ordering also varies across formats \(Markdown Tier 3 is its*easiest*cell\)\. For cross\-format tier ordering, RQ3 is therefore largely a null result\. We treat tiers as per\-format design labels rather than a universal difficulty scale, and the per\-operator view \(RQ4\) carries more signal\.

### 8\.4\.RQ4: Which operators are most challenging?

Among operators with≥\\geq20 API\-ok attempts, the hardest are Typst import drop \(TYP\-IMP\-DRP, 17\.5% CRapi\), LaTeX shell\-escape requirement \(TEX\-SHL\-ESC, 31\.8%\), and Typst unclosed math \(48\.4%\)\. Structural import/package faults remain harder than local token typos, matching the taxonomy’s prediction that nonlocal dependency restoration is the hard core\. Operators at 100% CRapiare mostly small\-nnMarkdown soft\-track cells and should not be read as ceilings\.

### 8\.5\.RQ5: What are the dominant failure modes?

#### API\-level\.

Of 48,651 recorded attempts, 4,352 returned empty, 2,258 hit the completion cap, 1,051 failed in transport, and 58 hit residual rate limits\. Qwen and GLM concentrate empty and length failures\. Grok, Codestral, and Maverick are nearly clean\.

#### Truncation and the 16k ablation\.

The 2,258 length failures concentrate in Qwen \(830, 11\.8% of its attempts\), DeepSeek\-Pro \(760\), DeepSeek\-Flash \(399\), and GLM \(260\)\. A fixed 4,096\-token completion budget is a protocol choice that systematically truncates long full\-document rewrites, so we*re\-ran*the length\-failed cells atmax\_tokens=16,384\{=\}16\{,\}384under the same prompt and provider pinning\. The rerun covered 1,947 of 2,258 length cells, sampling the highest\-cost route \(Qwen: 519 of its 830 cells\), 40 cells were lost to provider rate limits during the rerun\. Results \(Table[7](https://arxiv.org/html/2608.07617#S8.T7)\):77% of rerun truncation cells produce a compiling repair once the cap is lifted\(1,504/1,947\), reaching90%for Qwen\. Truncation was therefore predominantly a protocol artifact, not model inability\. Folding observed conversions back into ITT accounting, Qwen’s full\-set CRittrises from 56\.1% to at least 62\.7% observed \(≈66\.7%\{\\approx\}66\.7\\%extrapolating its 90% sample rate to all 830 cells\), and DeepSeek\-Pro’s from 65\.8% to 73\.8%, while Grok, Codestral, and Maverick are unaffected \(zero or near\-zero length cells\)\. Two caveats: 49 GLM cells migrated from*length*to*empty*at 16k \(failure class migration, not repair\), and 73 of the new compiles are exact golden reverts\. Model rankings under a 16k protocol would compress but not invert: Grok’s lead persists, but the gap to the truncation\-affected models roughly halves\.

Table 7\.16k truncation ablation: formerlengthcells rerun atmax\_tokens=16,384\{=\}16\{,\}384\(Qwen sampled 519/830\)\.Conv\. rate==compile//rerun\. 39 DS\-Flash and 1 GLM cells lost to rate limits, 4 Codestral cells still truncate at 16k\.

#### Equivalent mutants / golden reverts\.

1,527 API\-ok candidates \(3\.7%\) match the golden SHA\-256 exactly, ranging from 0\.9% \(Codestral\) to 7\.7% \(GLM\)\. Reversion compiles but is a weak repair signal: it solves the instance by undoing the operator\. Candidate explanations \(edit style, delivery selection, training familiarity\) are observationally indistinguishable here\. The restoration oracle below provides the complementary continuous view\.

## 9\.Results: Restoration Oracle

Compile success is necessary but not sufficient: a candidate can compile while deleting or rewriting content\(Smith et al\.,[2015](https://arxiv.org/html/2608.07617#bib.bib15); Venkateshmurthy,[2026b](https://arxiv.org/html/2608.07617#bib.bib18)\)\. We therefore score every compiling, non\-revert candidate with a restoration oracle: compile the candidate, extract text \(pdftotext\), and compute token\-LCS Dice similaritySpdfS\_\{\\mathrm\{pdf\}\}against the golden’s extracted text\.

#### Scorability accounting\.

Of 32,937 candidates, 4,708 rows \(14\.3%\) are*unscorable*: their golden compiles to a valid but textless PDF\. These concentrate in Typst \(4,435 rows from template/library seeds whose top\-level file defines functions without rendering content\) plus graphics\-only LaTeX \(259\)\. Restoration similarity is undefined, not zero, for such instances, so they are excluded with this explicit accounting\. A further 100 candidates failed re\-compilation\. The scored set is28,129candidates\. Markdown “restoration” compares pandoc HTML output rather than PDF text and is reported separately\.

### 9\.1\.RQ6: Do compiling repairs preserve content?

Not reliably \(Figure[4](https://arxiv.org/html/2608.07617#S9.F4)\)\. Per model,13\.6–18\.5% of compiling repairs fall belowSpdfS\_\{\\mathrm\{pdf\}\}=0\.95\{=\}0\.95: they lose or alter a material fraction of document text \(Table[8](https://arxiv.org/html/2608.07617#S9.T8)\)\. MedianSpdfS\_\{\\mathrm\{pdf\}\}is 1\.0 for every model: most repairs are faithful, but a heavy tail of content\-destroying rewrites persists everywhere\.

Table 8\.Restoration among compiling non\-revert repairs \(goldens with extractable text, LaTeX\+Typst\+Markdown rows, means over per\-candidateSpdfS\_\{\\mathrm\{pdf\}\}\)\.525254545656585860606262646466666868707072727474767678788080828284848686888890900\.950\.950\.960\.960\.970\.97GrokCodestralMaverickDS\-FlashDS\-ProGLMQwenCRitt\(%, balanced matrix\)MeanSpdfS\_\{\\mathrm\{pdf\}\}\(compiling repairs\)Figure 4\.Compile rate vs\. content restoration\. The two oracles disagree in the tails: Qwen \(lowest CRitt\) restores best among its successes, Maverick \(third\-highest CRitt\) restores worst\.#### Restoration rank diverges from compile rank\.

Qwen3\.7\-max,*last*under CRitt, has the*best*restoration among its compiling repairs\. Llama\-4 Maverick, third under CRitt, has the worst \(18\.5% below 0\.95\)\. Grok combines the top compile rate with near\-top restoration\. The ranking is robust to the threshold choice: atτ=0\.90/0\.95/0\.99\\tau\{=\}0\.90/0\.95/0\.99the per\-model below\-threshold shares span 8\.1–11\.9%, 13\.6–18\.5%, and 26\.9–35\.5% respectively, with Maverick worst and the Grok/Qwen/DeepSeek\-Pro cluster best at everyτ\\tau\. The divergence is modest in means \(0\.955–0\.971\) but systematic in tails, and it inverts for specific models: optimizing leaderboard position under a compile\-only oracle rewards exactly the aggressive full\-rewrite behavior that the restoration oracle penalizes\.

#### Per\-format and per\-tier\.

MeanSpdfS\_\{\\mathrm\{pdf\}\}: LaTeX 0\.975 \(n=21,300n\{=\}21\{,\}300\), Typst 0\.970 \(n=4,328n\{=\}4\{,\}328\), Markdown HTML\-similarity 0\.847 \(n=2,501n\{=\}2\{,\}501, a different construct: models reformat Markdown freely even when conversion succeeds\)\. Tier 3 shows the worst restoration tail \(13\.5% below 0\.95 vs\. 10\.4% for Tier 1\), consistent with harder faults inducing larger rewrites\.

## 10\.Discussion

#### Fault models change conclusions\.

The validation results \(Section[5](https://arxiv.org/html/2608.07617#S5)\) quantify what taxonomy\-grounded injection buys: 5\.6–9\.2 pp harder compile targets and substantially harder exact restoration, replicated across three model families, with real human faults harder still\. Benchmarks built from brace typos alone will overstate field performance\.

#### Two oracles, one leaderboard\.

Compile\-only scoring and restoration scoring disagree about models in the tails\. End\-to\-end document\-repair products should report both: CRittfor deployment reality \(including serving\-stack delivery\), CRapifor model skill, and a restoration statistic for content fidelity\.

#### Delivery is a first\-order outcome, and attributable\.

With provider\-pinned routing and a released provider ledger, the Qwen and DeepSeek\-Pro delivery deficits are attributable to first\-party serving stacks\. Cost intensity is likewise first\-order: recorded provider cost per 1,000 attempts ranges from≈\{\\approx\}$0\.45 \(DS\-Flash\) and $0\.79 \(Maverick\) to $4\.98 \(Grok\) and$14\.74\(Qwen\)\. The most expensive model in the panel yields the weakest ITT rate, a caution for cost\-unaware leaderboards\.

#### Typst gap\.

The Typst deficit appears across most models and is a useful signal of training\-data currency for newer languages\.

#### Matrix completeness\.

Unevaluated schedule cells are disclosed, not hidden: the balanced 6,613×\\times7 matrix removes any unevenness concern for primary claims, and released checkpoints allow anyone to evaluate the remaining freeze instances under the identical protocol at proportional cost\.

## 11\.Threats to Validity

#### Construct validity\.

Compile success is not semantic or visual correctness\.SpdfS\_\{\\mathrm\{pdf\}\}measures extracted\-text similarity, not layout, and its tokenizer does not yet fold Unicode normalization or ligatures\. Tier labels are design choices whose ordinal meaning varies by format \(RQ3\)\. The taxonomy targets localized hard crashes\. Reconfiguration\-class faults \(29\.1% of mined candidates\) are out of scope by design\. The Markdown restoration construct differs from PDF text similarity\. The zero\-shot, no\-diagnostics protocol is deliberately conservative: real repair tools have the compiler error available, so the absolute rates reported here are lower bounds on assisted repair, and a diagnostics\-in\-prompt condition is a planned extension\.

#### Internal validity\.

Mutations are synthetic\. Residual broken compiles \(5\.7%\) show soft\-track and Markdown oracles are imperfect\. Engine versions are pinned on one machine\. Other TeX Live / Typst versions may differ\(Tan and Rigger,[2024](https://arxiv.org/html/2608.07617#bib.bib16)\)\. Open\-codingκ=0\.34\\kappa\{=\}0\.34reflects granularity asymmetry \(Section[3\.2](https://arxiv.org/html/2608.07617#S3.SS2)\)\. The operationalizability check does not substitute for a second human codebook round\. Annotator 1 is the author\. The truncation ablation reallocates length\-failed cells but cannot rule out residual prompt–model interaction\.

#### External validity\.

Seven OpenRouter routes are not all LLMs\. Frontier closed models beyond Grok are absent\. Seeds are open templates and examples, not full multi\-file research monographs with private assets\. In addition, 1,158 seeds compile to textless PDFs and are excluded from restoration with explicit accounting\. The mined taxonomy is LaTeX\-only\. Typst/Markdown operators are analogues\. TeX\.SE dominates the mined corpus \(143/168\) despite dual sampling\. English\-language bias remains\. The real\-error evaluation is an 88\-case study, not a population sample\.

## 12\.Conclusion

TeXFix\-Bench closes the loop from mined human faults to large\-scale LLM evaluation: an 18\-category taxonomy derived from 168 verified hard crashes with transparent reliability analysis,DocMut, a 48\-operator multi\-format mutation library whose faults are 5\.6–9\.2 pp harder than pattern\-based injection across three model families and bracketed from below by real human faults \(67%\), a 10,437\-instance benchmark, and a seven\-model, provider\-pinned campaign whose primary claims hold on a complete 6,613×\\times7 balanced matrix\. Under the compile oracle we find a 27\.5\-point ITT spread and a persistent Typst gap\. Under the restoration oracle we find that 13\.6–18\.5% of compiling repairs materially alter document text, with restoration rank diverging from compile rank\. Document repair needs both oracles, and fault models grounded in evidence rather than convenience\. All artifacts are released\.

###### Acknowledgements\.

Independent second\-annotator coding is gratefully acknowledged\.

## Data Availability

DocMutv0\.2\.0 \(MIT\):[https://github\.com/prajwal\-svm/docmut](https://github.com/prajwal-svm/docmut)\. The taxonomy annotation data \(open codes, synonym handbook,κ\\kappaaudit trail\), campaign ledgers \(including the per\-request provider ledger\), engine\-gate and restoration results, cross\-benchmark sets, and harness scripts are archived in a companion Zenodo research package \([https://doi\.org/10\.5281/zenodo\.21831797](https://doi.org/10.5281/zenodo.21831797)\)\. Provider API keys and Authorization headers are excluded\.

## References

- \(1\)
- Braun and Clarke \(2006\)Virginia Braun and Victoria Clarke\. 2006\.Using thematic analysis in psychology\.*Qualitative Research in Psychology*3, 2 \(2006\), 77–101\.
- Groce et al\.\(2023\)Alex Groce et al\.2023\.Universal Mutator\.[https://github\.com/agroce/universalmutator](https://github.com/agroce/universalmutator)\.
- Gupta et al\.\(2017\)Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish Shevade\. 2017\.DeepFix: Fixing Common C Language Errors by Deep Learning\. In*Proc\. AAAI*\. 1345–1351\.
- Hou et al\.\(2026\)Junyi Hou et al\.2026\.PaperDebugger: A Plugin\-Based Multi\-Agent System for In\-Editor Academic Writing\. In*Companion Proc\. ACM Web Conf\.*144–147\.
- Jia and Harman \(2011\)Yue Jia and Mark Harman\. 2011\.An Analysis and Survey of the Development of Mutation Testing\.*IEEE Trans\. Softw\. Eng\.*37, 5 \(2011\), 649–678\.[https://doi\.org/10\.1109/TSE\.2010\.62](https://doi.org/10.1109/TSE.2010.62)
- Jimenez et al\.\(2024\)Carlos E\. Jimenez et al\.2024\.SWE\-bench: Can Language Models Resolve Real\-World GitHub Issues?\. In*Proc\. ICLR*\.
- Just et al\.\(2014\)René Just, Darioush Jalali, and Michael D\. Ernst\. 2014\.Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs\. In*Proc\. ISSTA*\.[https://doi\.org/10\.1145/2610384\.2628055](https://doi.org/10.1145/2610384.2628055)
- Landis and Koch \(1977\)J\. Richard Landis and Gary G\. Koch\. 1977\.The measurement of observer agreement for categorical data\.*Biometrics*33, 1 \(1977\), 159–174\.
- Le Goues et al\.\(2015\)Claire Le Goues et al\.2015\.The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs\.*IEEE Trans\. Softw\. Eng\.*41, 12 \(2015\), 1236–1256\.[https://doi\.org/10\.1109/TSE\.2015\.2454513](https://doi.org/10.1109/TSE.2015.2454513)
- Monperrus \(2018\)Martin Monperrus\. 2018\.Automatic Software Repair: A Bibliography\.ACM Comput\. Surv\.\.Extended bibliography of APR\.
- Offutt et al\.\(1996\)A\. Jefferson Offutt, Ammei Lee, Gregg Rothermel, Roland H\. Untch, and Christian Zapf\. 1996\.An experimental determination of sufficient mutant operators\.*ACM Transactions on Software Engineering and Methodology*5, 2 \(1996\), 99–118\.
- OpenRouter \(2026\)OpenRouter\. 2026\.OpenRouter Unified LLM API\.[https://openrouter\.ai/](https://openrouter.ai/)\.
- Papadakis et al\.\(2019\)Mike Papadakis, Marinos Kintis, Jie Zhang, Yue Jia, Yves Le Traon, and Mark Harman\. 2019\.Mutation Testing Advances: An Analysis and Survey\.*Adv\. Comput\.*112 \(2019\), 275–378\.
- Smith et al\.\(2015\)Edward K\. Smith, Earl T\. Barr, Claire Le Goues, and Yuriy Brun\. 2015\.Is the Cure Worse Than the Disease? Overfitting in Automated Program Repair\. In*Proc\. ESEC/FSE*\.[https://doi\.org/10\.1145/2786805\.2786825](https://doi.org/10.1145/2786805.2786825)
- Tan and Rigger \(2024\)Jirong Tan and Manuel Rigger\. 2024\.Inconsistencies in TeX\-Produced Documents\. In*Proc\. ISSTA*\. 1415–1427\.[https://doi\.org/10\.1145/3650212\.3680370](https://doi.org/10.1145/3650212.3680370)
- Venkateshmurthy \(2026a\)Prajwal S\. Venkateshmurthy\. 2026a\.DocMut: Document Mutation Library\.[https://github\.com/prajwal\-svm/docmut](https://github.com/prajwal-svm/docmut)\.v0\.2\.0, MIT\.
- Venkateshmurthy \(2026b\)Prajwal S\. Venkateshmurthy\. 2026b\.TeXFix\-Bench v0\.3: An Audited Synthetic Benchmark for Single\-Fault LaTeX Compilation Repair\.Zenodo\.[https://doi\.org/10\.5281/zenodo\.21764870](https://doi.org/10.5281/zenodo.21764870)
- Viera and Garrett \(2005\)Anthony J\. Viera and Joanne M\. Garrett\. 2005\.Understanding interobserver agreement: The kappa statistic\.*Family Medicine*37, 5 \(2005\), 360–363\.
- Wang et al\.\(2026\)Cheng Wang, Lin Fu, Zhiyuan Kuang, and Yu Zhao\. 2026\.TexOCR: Advancing Document OCR Models for Compilable Page\-to\-LaTeX Reconstruction\. In*Proc\. ACL*\. 35821–35845\.
- Wen et al\.\(2024\)Haoran Wen et al\.2024\.OverleafCopilot: Empowering Academic Writing in Overleaf with Large Language Models\.arXiv:2403\.09733\.
- Xia and Zhang \(2023\)Chunqiu Steven Xia and Lingming Zhang\. 2023\.Keep the Conversation Going: Fixing 162 out of 337 bugs for $0\.42 each using ChatGPT\. In*Proc\. ISSTA*\.Conversational APR / ChatRepair line of work\.
- Yasunaga and Liang \(2020\)Michihiro Yasunaga and Percy Liang\. 2020\.Graph\-Based, Self\-Supervised Program Repair from Diagnostic Feedback\. In*Proc\. ICML*\. 10799–10808\.
- Zhu and He \(2022\)Fengmin Zhu and Fei He\. 2022\.EqFix: Fixing LaTeX Equation Errors by Examples\. In*SETTA**\(LNCS, Vol\. 13649\)*\. Springer, 106–124\.

## Appendix AOperator Codes Observed in the Freeze

Table[9](https://arxiv.org/html/2608.07617#A1.T9)lists operator codes with nonzero instance counts in the v0\.4 freeze\. Full rationales ship with theDocMutcatalog\.

Table 9\.Operators with support in the v0\.4 instance freeze\.