Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents

# 1 Introduction
Source: [https://arxiv.org/html/2607.05764](https://arxiv.org/html/2607.05764)
Inject or Navigate? Token\-Efficient Retrieval for LLM Analysis of Transactional Legal Documents Mahmoud Hany\*Mourad ElSheraey\*Mahmoud Said\*Peter Naoumhany@syntheia\.appmourad@syntheia\.iomahmoud@syntheia\.apppeter@syntheia\.app

Syntheia Pty Ltd

Abstract\.Answering questions over a set of transactional legal documents is most simply done by*injecting*the whole corpus into the LLM’s context window on every query\. That baseline maximises retrieval recall, but its token footprint scales with the corpus rather than the question, and long\-context degradation scales with it\. We report what it took to replace full\-corpus injection in a legal\-document analysis system, comparing it against two structured retrieval modes over our proprietary structure\-aware chunking: embedding retrieval \(navembed\) and LLM navigation over a compact structured index \(navindex\)\. On a 20\-question benchmark with verified ground\-truth answers, a position\-bias\-controlled, reference\-anchored pairwise judge scored semantic retrieval with reranking tied with injection on 16 of 18 document\-bound questions \(injection preferred on 2\) while attending to17\.3×17\.3\\timesfewer input tokens \(a general\-text\-embedding \(GTE\) configuration reaches29\.9×29\.9\\timesat a lower tie rate\); both modes were judged tied on the 2 out\-of\-scope controls\.navindexwas judged tied on all 18 at a1\.61×1\.61\\timessmaller total token footprint, a∼56×\{\\sim\}56\\timessmaller answering context, and 25% lower dollar cost\. We derive a closed\-form caching\-crossover rule: cached injection is cheaper in dollars only while the corpus stays below roughly ten times the retrieval payload\. Scope and uncertainty are quantified in Section[8](https://arxiv.org/html/2607.05764#S8)\.

11footnotetext:Equal contribution\. Mahmoud Hany built and ran the evaluation program across all three phases and wrote the paper’s initial draft\. Mourad ElSheraey revised the analysis, added the uncertainty quantification and threats\-to\-validity treatment, and prepared the final manuscript\. Mahmoud Said implemented the cross\-encoder reranking pipeline and metadata fields, and contributed the cross\-reference extraction\. Peter Naoum implemented the initial codebase, and supervised the experimental work\.Large language models have made it practical to interrogate documents directly: a user poses a question in natural language and receives an answer grounded in the source text, with no hand\-built query interface and no manual review\. Few professions stand to benefit more from this than law, whose day\-to\-day work consists largely of locating, reconciling, and interpreting provisions spread across long, densely cross\-referenced instruments\. Syntheia provides the data layer for this workload: a structured document index over coordinated*transaction sets*\(an original agreement plus its amendments, side letters, schedules, and disclosure documents\) served through an API, and directly to LLMs such as Claude via the Model Context Protocol\[[1](https://arxiv.org/html/2607.05764#bib.bib1)\], for document tools to build on\. This paper reports the evaluation that decided how retrieval over that index should work: what it took to replace full\-corpus injection with structured retrieval without giving up answer quality, and what each alternative costs in tokens and dollars\.

The simplest method is to place the entire corpus in the model’s context window on every query\. We call this approachinject, and we adopt it as our baseline deliberately, because it represents the upper bound on retrieval recall\. Every clause, defined term, and cross\-reference is in context on every query, so there is nothing for a retriever to miss and nothing to misconfigure\. An alternative that a careful judge cannot distinguish frominjectis therefore a stronger result than one that merely outperforms a lossy retriever; the corpus was deliberately sized to keepinjectrunnable \(Section[3](https://arxiv.org/html/2607.05764#S3)\)\.

Two costs limitinjectas the corpus grows\. The token footprint scales with the corpus rather than the question \(every query pays for every clause regardless of relevance\) and the corpus cannot exceed the context window at all\. Separately, LLMs attend unevenly across long inputs and lose information positioned in the middle of the context\[[2](https://arxiv.org/html/2607.05764#bib.bib2)\]\. The legal domain compounds both: answers routinely depend on defined terms, on cross\-references between schedules and operative clauses, and on how an original agreement interacts with later amendments, so the text bearing on a question is dispersed across the corpus and easily diluted by surrounding material\. Existing contract benchmarks \(CUAD\[[3](https://arxiv.org/html/2607.05764#bib.bib3)\], ContractNLI\[[4](https://arxiv.org/html/2607.05764#bib.bib4)\]\) operate on single documents and do not capture this cross\-instrument dependency, which remains an open gap for legal retrieval evaluation\.

Both modes studied here operate over the same input representation: documents chunked by a proprietary structure\-aware method developed at Syntheia, rather than split into fixed\-size spans\. Chunking is held constant as a fixed preprocessing step, so that what we measure is the retrieval strategy and not the chunking; we do not evaluate naive fixed\-size chunking, whose tendency to fragment provisions and reduce retrieval quality is already well documented\[[5](https://arxiv.org/html/2607.05764#bib.bib5),[6](https://arxiv.org/html/2607.05764#bib.bib6)\]\. We report two comparisons:

Comparison 1\(Section[4](https://arxiv.org/html/2607.05764#S4)\) evaluates*Navigate\-Embeddings*\(navembed\), which pre\-embeds the document nodes and, at query time, retrieves the top\-kkmost similar by cosine similarity, placing only those nodes in the answering model’s context\. Twelve configurations \(RUN\-001–012\) were run in three phases: Phases I–II while the harness matured \(RUN\-001–008\), Phase III at the full 20\-question scale \(RUN\-009–012\)\. Each is an internally controlled paired comparison \(injectversusnavembedon identical questions\), and the headline claims rest on the Phase III runs\.

Comparison 2\(Section[5](https://arxiv.org/html/2607.05764#S5)\) evaluates*Navigate\-Index*\(navindex\), which replaces vector similarity with guided navigation over a compact structured index: boolean semantic flags, generated summaries, and explicit cross\-reference and defined\-term graphs\. We sweep six descriptor configurations \(the per\-node metadata the retriever sees\) to characterise the quality–token\-footprint trade\-off\.

The twonavigatemodes reduceinject’s burden along two dimensions that prompt caching has pulled apart: token footprint and dollar cost\. Caching lowers the*price*of re\-reading a large context without reducing the tokens the model must attend over when it answers, so the two dimensions are tracked separately throughout \(Section[3\.3](https://arxiv.org/html/2607.05764#S3.SS3)\)\. The cache discount also holds only within a single warm session \(a run of queries close enough in time to keep the provider’s short\-lived prompt cache live\), and a corpus that outgrows the context window cannot be injected at any price \(Section[7](https://arxiv.org/html/2607.05764#S7)\)\.

Within that frame, the two modes sit at different points on the quality–token\-footprint curve \(Table[4](https://arxiv.org/html/2607.05764#S6.T4)\):navindexmaximises structural fidelity at a modest token\-footprint gain,navembedmaximises token\-footprint reduction, and the dollar comparison between them and cachedinjectis corpus\-dependent \(Section[7](https://arxiv.org/html/2607.05764#S7)\)\. Figure[1](https://arxiv.org/html/2607.05764#S3.F1)illustrates all three modes\.

The contributions are: \(i\) a paired evaluation of full\-corpus injection against two structured retrieval modes on a scored legal\-QA evaluation set with verified ground\-truth answers, judged by a position\-bias\-controlled, reference\-anchored pairwise protocol \(answer\-level equivalence; retrieval recall is not directly measured\), with uncertainty quantified throughout; \(ii\) the design of thenavindexformat, a navigation descriptor for legal instruments \(typed boolean flags, cross\-reference and defined\-term graphs, a hard selection cap\), documented to field level by the released prompts, which also operationalise amendment precedence and defined\-term resolution \(Appendix[D](https://arxiv.org/html/2607.05764#A4)\); \(iii\) a closed\-form caching\-crossover rule separating token footprint from dollar cost, with its session\-economics caveats; and \(iv\) a catalogue of failure modes we would not ship \(BM25 length bias, titles\-only over\-selection, uncapped node selection\) and an account of the evaluation’s limits\.

## 2 Related Work

RAG couples a parametric generator with a non\-parametric retriever so that generation is conditioned on fetched evidence\[[7](https://arxiv.org/html/2607.05764#bib.bib7)\]; recent surveys map the design space\[[8](https://arxiv.org/html/2607.05764#bib.bib8)\]\. Dense passage retrieval\[[9](https://arxiv.org/html/2607.05764#bib.bib9)\]established learned bi\-encoders as a strong default, followed by a family of general\-purpose sentence embeddings \(Sentence\-BERT\[[10](https://arxiv.org/html/2607.05764#bib.bib10)\], E5\[[11](https://arxiv.org/html/2607.05764#bib.bib11)\], GTE\[[12](https://arxiv.org/html/2607.05764#bib.bib12)\], BGE\[[13](https://arxiv.org/html/2607.05764#bib.bib13)\], and MPNet\-based encoders\[[14](https://arxiv.org/html/2607.05764#bib.bib14)\]\)\. BM25 remains a strong lexical baseline\[[15](https://arxiv.org/html/2607.05764#bib.bib15)\]; hybrid systems fuse dense and sparse signals via Reciprocal Rank Fusion\[[16](https://arxiv.org/html/2607.05764#bib.bib16)\], and cross\-encoder rerankers\[[17](https://arxiv.org/html/2607.05764#bib.bib17)\]re\-score candidates at higher fidelity\. Our strongestnavembedconfiguration is a retrieve\-then\-rerank pipeline; our hybrid BM25 run illustrates a known failure mode \(length bias\) on clause\-structured text\. Chunking granularity affects retrieval quality\[[5](https://arxiv.org/html/2607.05764#bib.bib5),[6](https://arxiv.org/html/2607.05764#bib.bib6)\]; we hold structure\-aware chunking fixed and vary only the retrieval strategy\.

Two observations make token footprint a first\-class metric here\. Long\-context performance degrades when relevant information sits in the middle of a long input\[[2](https://arxiv.org/html/2607.05764#bib.bib2)\], and provider\-side prompt caching discounts re\-reads of a fixed prefix\[[18](https://arxiv.org/html/2607.05764#bib.bib18)\], lowering the*price*of a large context without lowering the*tokens*the model attends to\. Dollar cost and token footprint must therefore be tracked as separate quantities\. For evaluation, LLM\-as\-judge protocols are common\[[19](https://arxiv.org/html/2607.05764#bib.bib19)\]but exhibit position bias\[[20](https://arxiv.org/html/2607.05764#bib.bib20)\]; our forward/reverse protocol is a direct response\.

In legal NLP, CUAD\[[3](https://arxiv.org/html/2607.05764#bib.bib3)\]and LEDGAR\[[21](https://arxiv.org/html/2607.05764#bib.bib21)\]are the leading contract benchmarks, and ContractNLI\[[4](https://arxiv.org/html/2607.05764#bib.bib4)\]shows that entailment over contracts requires resolving defined terms and cross\-references, structural dependencies that flat retrieval cannot represent\. Hierarchical indexing organises nodes into a tree and navigates from coarse summaries to fine provisions\[[22](https://arxiv.org/html/2607.05764#bib.bib22)\]; a complementary line uses typed metadata pre\-filters to narrow an index before any similarity computation, analogous to predicate push\-down in query optimisation\. Legal documents require both: defined terms propagate meaning across sections, cross\-references bind operative clauses to schedules, and amendments supersede base provisions in ways neither cosine similarity nor BM25 can represent\. Thenavindexformat encodes these signals directly and sits in the hierarchical\-navigation paradigm rather than in flat RAG; among vectorless navigators it is closest to PageIndex\[[23](https://arxiv.org/html/2607.05764#bib.bib23)\]\(Section[5\.1](https://arxiv.org/html/2607.05764#S5.SS1)\)\.

## 3 Problem Setting and Method

### 3\.1 Corpus and models

Document corpus\.The benchmark indexes fifteen transactional legal agreements drawn from three practice areas\. Six*credit facility agreements*\(FA\) cover two borrowers: Fluor Corporation \(a $1\.8 B revolving facility plus one amendment\) and Generac \(an ABL facility plus two term\-loan amendments and a 2024 refinancing\)\. Three*limited partnership agreements*\(LP\) cover two funds: Thomas Green Fund, a US real\-estate fund targeting LEED\-certified office and multi\-family assets with a $500 M commitment cap and a companion side letter, and Carlyle PE Partners, a PE fund with a 5 % IRR hurdle and 12\.5 % incentive allocation\. Six*share/asset purchase agreements and disclosure schedules*\(SP\) cover four transactions: Meta’s divestiture of Giphy to Shutterstock \($128 M purchase price\), the First Avenue Networks disclosure schedule, the PPG/Comex cross\-border acquisition, and a Royal Wolf Holdings facility agreement\. Two additional Netflix facility agreements are indexed in the open\-search workspace but not bound to any scored question\. The scored benchmark runs use six of the fifteen documents: one Thomas Green Fund LPA, one Carlyle PE Partners LPA, the Meta/Giphy SPA, the First Avenue Networks disclosure schedule, the Generac 2024 amendment, and the Fluor facility amendment\. All fifteen documents are public filings drawn from SEC EDGAR exhibits, Kentucky pension FOIA releases, and UK government sources\. The scored question set with its verified reference answers and a source reference for every document is released as supplementary material; all prompts are printed in Appendix[D](https://arxiv.org/html/2607.05764#A4)\.

Question set\.The*scored evaluation set*contains 20 questions with verified ground\-truth answers \(Appendix[C](https://arxiv.org/html/2607.05764#A3)\): 18 are bound to specific source documents \(six per practice area, three questions per document\), and 2 are*out\-of\-scope controls*whose answers are deliberately absent from all indexed documents, testing whether the system correctly declines rather than hallucinating\. Because the controls test declination rather than retrieval, we report them separately from the document\-bound counts throughout\. The first two phases ran on a 15\-question, document\-pinned subset of*DocNavBench*, a companion 52\-question open\-search set over the same corpus, authored by the reviewing lawyer: its questions are deliberately issue\-driven, written as a practitioner would ask*before*reading and without presuming the answer exists \(so “that is not in the agreement” can be the correct finding\), and it is document\-agnostic, pairwise\-judged only, with no ground\-truth answers\. The Phase I–II runs used 3 questions first, widening to 9 and then all 15 as the harness matured \(Appendix[A](https://arxiv.org/html/2607.05764#A1)\); thenavindexdescriptor sweep \(Configs 1–5, Section[5](https://arxiv.org/html/2607.05764#S5)\) also belongs to Phase II, sharing its pool and corpus\. The 20\-question scored set was written fresh at Phase III by the engineering team: questions and reference answers were drafted with the same model family over the six scored documents, verified against the sources, and endorsed by the reviewing lawyer, with the out\-of\-scope controls added so that every document\-bound question is answerable from the indexed documents by construction\. It shares no question with the Phase I–II pool, so nothing the pipeline was tuned on is scored\. The lawyer’s assessment of the set’s difficulty and phrasing is reported in Section[8](https://arxiv.org/html/2607.05764#S8)\. The four Phase III runs andnavindexConfig 6 all use the scored set; DocNavBench’s full open\-search evaluation is future work\. The answering model isclaude\-sonnet\-4\-6; the judge isclaude\-opus\-4\-7\.

Context\-window scope\.The scored evaluation was deliberately sized to stay within the model’s context window, so thatinjectis a runnable baseline and the comparison is like\-for\-like on every question\. This is a constraint of the evaluation, not of the methods\. Real legal matters frequently exceed the window \(a single transaction can span dozens of instruments across base agreements, amendments, side letters, and schedules\) and beyond that pointinjectis not a viable method rather than merely an expensive one, so it would not appear as a baseline at all\. Thenavigatemodes carry only a bounded retrieval payload into the answering context and are not subject to this ceiling\. Within\-window parity is thus the deliberately hard test: a mode that cannot matchinjectwhereinjectis runnable could not be trusted whereinjectis not\.

### 3\.2 Judging protocol

Every question is answered*twice, in parallel*: once byinjectand once by thenavigatevariant under test\. The judge receives the question, the*verified reference answer*, and both model answers, and decides which answer more accurately reflects the reference, or a tie if both are equally correct or equally wrong \(the full judge prompt is Appendix[D](https://arxiv.org/html/2607.05764#A4),JUDGE\_GROUNDTRUTH\_SYSTEM\)\. Each pair is judged in two directions,*forward*and*reverse*\(positions swapped\); a*win*is awarded only when both directions agree, and a hedged verdict in either direction \(e\.g\. “slight lean toward A, effectively tied”\) is treated as a tie\. The two\-direction rule controls the position bias documented for LLM judges\[[20](https://arxiv.org/html/2607.05764#bib.bib20),[19](https://arxiv.org/html/2607.05764#bib.bib19)\]\.

A tie under this protocol therefore certifies that the two answers are*judged equally consistent with the verified reference*, equally correct or equally wrong relative to ground truth, and not merely that they read alike\. It does not certify that either answer is right; judge error is bounded empirically by the audit below, and the remaining caveats \(a forced\-decision probe, i\.e\. a rerun with ties disallowed; judge–answerer provider overlap; and tie\-category granularity\) are consolidated in Section[8](https://arxiv.org/html/2607.05764#S8)\. All 20\-question scored runs \(RUN\-009–012 andnavindexConfig 6\) were judged with this reference\-anchored prompt; the Phase I–II runs,navindexConfigs 1–5 included, were scored with the Phase I–II preference judge, which compares the two answers without a reference \(reproduced in Appendix[D](https://arxiv.org/html/2607.05764#A4)\)\.

Judge audit\.A manual audit of one 15\-question run classified9/159/15verdicts as clearly correct,4/154/15as defensible\-but\-debatable, and2/152/15\(13\.3%13\.3\\%\) as likely wrong \(95% Clopper–Pearson interval\[[24](https://arxiv.org/html/2607.05764#bib.bib24)\]\[2%,40%\]\[2\\%,40\\%\]\)\. The audit was performed by one author against the verified ground\-truth answers for that run; a verdict was “clearly correct” if the judge’s preferred answer matched the ground truth exactly, “defensible\-but\-debatable” if partially correct or a reasonable legal interpretation, and “likely wrong” if it contradicted the ground truth\. No second annotator reviewed the run, so inter\-annotator agreement is not available\.

### 3\.3 Token footprint versus dollar cost

Prompt caching separates two axes\.*Token footprint*is every input token the model attends to, including cached context; it governs the long\-context degradation noted above and the hard window limit\.*Dollar cost*is recorded operational spend, reshaped by discounted cache reads and one\-time cache writes\[[18](https://arxiv.org/html/2607.05764#bib.bib18)\]\. Caching can make a full\-contextinjectcheap to re\-read without reducing the tokens inside the model’s attention\. In RUN\-005 \(15 questions\),injectattended to3,217,7923\{,\}217\{,\}792input tokens in total \(∼214\.5\{\\sim\}214\.5K per question\) whilenavembedprocessed171,307171\{,\}307: about3×3\\timesapart on dollars \($1\.881 vs\. $0\.611\) but18\.8×18\.8\\timesapart on token footprint\. Caching compresses the dollar axis; the token\-footprint gap is what remains structural\.

### 3\.4 System context

The three modes differ in their runtime shape as well as their token profile\.injectissues one call per question over a long \(cache\-discounted\) prefix\.navembedembeds the query, retrieves and optionally reranks nodes, then issues one short answering call\.navindexissues two sequential LLM calls per question \(a short\-output index scan over a cached prefix, then a short answering call\), trading tokens for one extra round\-trip of latency\. Indexing is a one\-time, per\-document cost with two layers: headings, clause references, and the cross\-reference and defined\-term edges are extracted deterministically by the chunking pipeline \(same input, same output, verifiable against the source\), while the semantic boolean flags \(Table[2](https://arxiv.org/html/2607.05764#S5.T2)\), keyword tags, and per\-node summaries are populated by a separate enrichment pass of the Syntheia indexing pipeline\.

INJECTFull corpus\(context window\)LLM answersAnswerNAVEMBEDEmbed corpus\(node index\)Cosine retrieval\(top\-kknodes\)LLM answersAnswerNAVINDEXBuild index\(metadata\+\+flags\)LLM scans index\(returns node IDs\)Fetch selectednode textLLM answersAnswer

Figure 1:The three pipeline modes\.Left:inject: full corpus in context on every query\.Centre:navembed: cosine retrieval selects the top\-kknodes at query time\.Right:navindex: three\-step structured navigation over a compact boolean\-flagged index, no embedding similarity required\.

## 4 INJECT vs\. NAVEMBED

### 4\.1 Approach

navembedpre\-indexes each document into a tree of*nodes*\. Every node carries the section’s verbatim text plus a descriptor used at index time\. At query time the query is embedded with the same model and cosine similarity selects the top\-kknodes; only those nodes \(plus the question\) are placed in the answering model’s context\. This replaces the full corpus with a small, question\-relevant working set\. Five retrieval strategies were evaluated:

1. R1\.Embedding top\-kk\[[9](https://arxiv.org/html/2607.05764#bib.bib9)\]:cosine similarity; keep the topkk\.
2. R2\.Embedding\+\+cross\-reference expansion:after top\-kkhits, follow each hit’scrossReferencedIdsand add those nodes\.
3. R3\.Hybrid semantic\+\+BM25:fuse dense and lexical signals\[[15](https://arxiv.org/html/2607.05764#bib.bib15),[16](https://arxiv.org/html/2607.05764#bib.bib16)\]\.
4. R4\.Semantic\+\+rerank:retrieve4040candidates by cosine, rerank with a cross\-encoder, keep the top1010\[[17](https://arxiv.org/html/2607.05764#bib.bib17)\]\.
5. R5\.Naive LLM index scan:the model sees a titles\-only index, selects candidates, and those are fetched\. This prototype over\-selected \(8787,7272, and3838nodes on its three trial questions, pushing the token footprint to within1\.2×1\.2\\timesofinject\) and motivated thenavindexformat described in the next section\.

Twelve configurations were run in three phases; the earlier two \(3–15 questions, evolving harness\) are reported as run history in Appendix[A](https://arxiv.org/html/2607.05764#A1), and all claims below rest on the Phase III runs, which share the full 20\-question set, the same six\-document corpus, and the same prompts\.

### 4\.2 Results

Table 1:Phase IIInavembedruns \(20 questions: 18 document\-bound\+\+2 out\-of\-scope controls\)\. “Red\.” \(reduction\) is theinject/navembedinput\-token ratio; dollar columns are total recorded run cost in USD \(Section[7](https://arxiv.org/html/2607.05764#S7)price card\)\. Judge outcomes are ties /injectwins /navembedwins\. RUN\-010 uses ground\-truth node labels to validate retrieval and is an oracle upper bound, not a deployable configuration\. Per\-question verdicts are printed for RUN\-012 in Appendix[B](https://arxiv.org/html/2607.05764#A2); its twoinjectwins are document\-bound and both controls were judged tied\. RUN\-009’sinjectarm ran fresh \($1\.621\); RUN\-010–012 reuse a cachedinjectanswer set \(recorded at $1\.614\), so their cross\-run variation is retrieval\-side only\.Table[1](https://arxiv.org/html/2607.05764#S4.T1)reports the Phase III runs; Table[5](https://arxiv.org/html/2607.05764#S6.T5)gives exact 95% confidence intervals for every headline rate, and Appendix[A](https://arxiv.org/html/2607.05764#A1)reports the full twelve\-run history \(Table[6](https://arxiv.org/html/2607.05764#A1.T6)\), across which the token\-footprint reduction held at8\.58\.5–29\.9×29\.9\\times\.

The strongest deployable result is semantic retrieval with reranking \(RUN\-012\)111The initial 40\-candidate retrieval uses theparaphrase\-MiniLM\-L6\-v2sentence embedding\[[25](https://arxiv.org/html/2607.05764#bib.bib25)\], and reranking uses thecross\-encoder/ms\-marco\-MiniLM\-L6\-v2cross\-encoder\. The pipeline shape \(retrieve 40 by cosine, cross\-encoder rerank, keep 10\) and its full per\-question token and cost ledger are reported in Appendix[B](https://arxiv.org/html/2607.05764#A2)\.: judged tied withinjecton 16 of 18 document\-bound questions \(injectpreferred on 2,navembedpreferred on 0\), with both out\-of\-scope controls also tied, at188,274188\{,\}274input tokens againstinject’s3,263,7963\{,\}263\{,\}796, a17\.3×17\.3\\timesreduction\. GTE embeddings \(RUN\-009\) give the largest reduction,29\.9×29\.9\\times, at a lower tie rate \(15/2015/20;injectpreferred on 3,navembedon 2\)\[[12](https://arxiv.org/html/2607.05764#bib.bib12)\]\. Hybrid BM25 \(RUN\-011\) is the failure mode: length bias\[[15](https://arxiv.org/html/2607.05764#bib.bib15)\]inflates the retrieved payload to382382K tokens \(reduction falls to8\.5×8\.5\\times\), demonstrating that lexical fusion can silently cost more than it returns on clause\-structured text\. RUN\-010 \(MiniLM with ground\-truth\-assisted retrieval\) reached18/2018/20ties but uses gold node labels to validate retrieval, making it an oracle upper bound; it is excluded from all cross\-mode comparisons\. Notably, its twoinjectwins fall on the same two questions as RUN\-012’s, consistent with the retrieval\-difficulty diagnosis below\.

The twoinjectwins in RUN\-012 \(SP\-first\-ave\-Q2andSP\-meta\-giphy\-Q3; Appendix[B](https://arxiv.org/html/2607.05764#A2)\) both concern share\-purchase and disclosure\-schedule provisions that draw on multiple cross\-referenced clauses\. The likely cause is retrieval failure \(relevant nodes ranked outside the top\-1010after reranking\) rather than reasoning failure, though confirming this requires the ground\-truth node labels planned in Section[9](https://arxiv.org/html/2607.05764#S9)\. This diagnosis motivated encoding cross\-references explicitly, which is exactly the signalnavindexnavigates \(Section[5](https://arxiv.org/html/2607.05764#S5)\)\.

Embedding model choice mattered less than descriptor richness\.Across the Phase II runs \(Appendix[A](https://arxiv.org/html/2607.05764#A1)\), BGE embeddings produced moreinjectwins thanparaphrase\-MiniLM\-L6\-v2\(consistent with over\-selection of lower\-relevance nodes\), and MPNet QA embeddings raised the Phase II tie count to9/159/15\. But no embedding swap moved outcomes as much as what each index node carries, the observation that motivated Comparison 2\.

Dollar cost\.navembed’s recorded cost is lower thaninject’s in every ledger\-validated run \(RUN\-005 onward; Table[6](https://arxiv.org/html/2607.05764#A1.T6)\)\. RUN\-011 is, however, the one Phase III configuration whose per\-query payload \(∼19\{\\sim\}19K tokens\) exceeds one\-tenth of the163163K corpus prefix, so by the crossover rule of Section[7](https://arxiv.org/html/2607.05764#S7)a warm\-cachedinjectwould beat it at steady state: the length\-bias failure costs real money, not only token footprint\.

## 5 INJECT vs\. NAVINDEX

### 5\.1 Motivation: descriptor richness as the binding constraint

Thenavembedruns identify the binding constraint: the*richness*of what each index node carries\. To overcome the limitations of a vector retrieval system, we designednavindex: a retrieval system where an LLM reasons over a compact structured index rather than searching by similarity alone\. Cosine similarity surfaces semantically similar nodes but cannot represent structural signals such as “this clause defines a term,” “this provision contains a liability cap,” or “this node cross\-references Section 5\.17\.” And when nodes carry only headings, the model over\-selects \(R5, Section[4](https://arxiv.org/html/2607.05764#S4)\)\. Thenavindexformat addresses both limits by encoding each node with typed metadata and a hard selection cap\.

A vectorless, structure\-navigating index of this kind is closest in spirit to PageIndex\[[23](https://arxiv.org/html/2607.05764#bib.bib23)\], which builds a hierarchical table\-of\-contents tree of a document and has an LLM reason over node titles and per\-node summaries to choose which sections to read\. PageIndex generates a natural\-language summary of every node at index time\. Our descriptor is deliberately weighted the other way: the primary navigation signals are compact and cheap to index: headings, clause references, and the explicit cross\-reference and defined\-term graph are extracted deterministically from document structure, and the boolean semantic flags and keyword tags are populated by Syntheia’s enrichment pass as single booleans and short tag lists rather than free text\. The model filters on these before consulting any summary text\. The reason is cost\. Generating a free\-text summary of every section is the expensive end of the indexing budget and amortises only when the same document is queried many times; a given legal instrument is often consulted only a handful of times, or once, so that cost may never be recovered\. The extracted fields carry no generation cost at all, and the flags and tags are the cheap end of the enrichment budget\.

### 5\.2 Approach: the NAVINDEX format

Dual\-file architecture\.Every processed document produces two files:\*\.index\.json\(the*navigation surface*: structural metadata plus a short opening snippet per node, no full provision text\) and\*\.full\.json\(the*provision store*: verbatim clause text fetched by node ID\)\. The navigation surface is designed to scale with document structure rather than text length\. The cached S1 prompt used in the scored runs serialises the six scored documents’ token\-optimised indices together with the navigation instructions and measures98,56298\{,\}562tokens \(Appendix[B](https://arxiv.org/html/2607.05764#A2)\); the three large instruments \(the two LPAs and the SPA\) account for roughly nine\-tenths of it, and the two amendments and the disclosure schedule for the remainder\.

Boolean pre\-filter flags\.Seven semantic boolean fields \(Table[2](https://arxiv.org/html/2607.05764#S5.T2)\) let the retrieval model narrow a500500\-node index to a handful of candidates in a single pass, before reading any summary text\. Config 4 also tests two structural fields \(textLength,isTocEntry\) added on top of the seven semantic flags\.

Table 2:Boolean pre\-filter flags stored in each index node\.Three\-step navigation\.

1. S1\.*Structured scan\.*The LLM receives\*\.index\.jsonas its cached system prompt\. It applies boolean pre\-filters, confirms relevance via titles and summaries, and returns*at most ten*node IDs\.
2. S2\.*Deterministic fetch\.*Node IDs are looked up in\*\.full\.json; verbatim provision texts are assembled into a<provisions\>block\.
3. S3\.*Grounded answer\.*The answering model receives only the question and the fetched provisions \(no index, no other documents\)\.

The hard cap of ten nodes per question bounds S3 token cost regardless of corpus size\. Prompt caching on the index block means subsequent S1 calls pay only for the question text \(2020–6060tokens\), not the full index, though the model still attends over the full cached index on every scan \(Section[3\.3](https://arxiv.org/html/2607.05764#S3.SS3)\)\.

Table 3:injectvs\.navindexacross six descriptor configurations\. “Q” is the question count for the run \(Configs 1–5 predate the full 20\-question set\)\. “Inj\. w\.”/“Nav\. w\.” are judge wins; “Inj\. Mtok” and “Nav\. Mtok” are total input tokens \(millions\); “Red\.” is their ratio; dollar columns are total recorded run cost in USD\. Config 6’s 20 questions comprise 18 document\-bound ties and 2 out\-of\-scope\-control ties \(Appendix[B](https://arxiv.org/html/2607.05764#A2)\)\.†\\daggerFlags:textLength,isTocEntry,hasParty,hasCondition,hasLiabilityLimit,hasPercentage,hasDate,hasMoney,isDefinition\. ‡\\ddaggerConfig 6 carries all Config 1 features \(cross\-references, AI\-generated summaries, defined\-term flags, and the full Config 4 metadata flag set\) but serialises the index with token\-optimised JSON: boolean fields are omitted whenfalse, array fields when empty, and string fields when null; key names are abbreviated to two characters; all non\-essential whitespace is stripped\. These changes reduce the cached index size by∼30%\{\\sim\}30\\%without altering the navigable content, the one effect attributable to the serialisation itself\. Config 6 also moves to the full 20\-question set and the Phase III harness, so*all*cross\-configuration comparisons against Configs 1–5 \(ties, dollar cost, and token footprint\) carry that change; Config 6’s own headline numbers are unaffected, being paired againstinjectwithin the same run\.

Cross\-reference and defined\-term resolution\.Standard RAG retrieves the clause most similar to the query, but legal accuracy requires more\. A liability cap may invoke a defined term whose definition appears 20 pages earlier; an operative clause may point to a schedule that contains the actual threshold; an amendment may silently supersede the provision the retriever fetched\. Cosine similarity and BM25 measure lexical or semantic proximity; neither can represent these structural dependencies\.

Thenavindexformat encodes them explicitly\. Every node carries acrossReferencedIdslist naming the node IDs it references, and theisDefinitionflag marks defined\-term nodes\. The deterministic fetch in S2 follows the cross\-reference edges automatically: retrieving a clause also retrieves every provision it cross\-references, without a second similarity search\. Defined terms are handled by the flag rather than an edge: theisDefinitionmarker lets the navigation step \(S1\) pull in the definitions a question turns on, instead of leaving them to embedding proximity\. The answering model therefore receives a provision’s cross\-referenced context and the relevant definitions rather than only its highest\-scoring neighbours in embedding space\. This structural navigation is not available to flat embedding retrieval; whether it is what produces Config 6’s tie result is not established \(Section[6](https://arxiv.org/html/2607.05764#S6)\)\.

Encoding legal precedence in prompts\.The retrieval format is only half the domain engineering; the released prompts \(Appendix[D](https://arxiv.org/html/2607.05764#A4)\) carry the other half\. The answering prompts encode an amendment\-precedence hierarchy \(“a later\-dated amendment, side letter, restatement, or supplementoverridesthe original to the extent of any inconsistency”\), defined\-term origin tracking across documents, and an exact\-string silence rule for out\-of\-scope questions\. The S1 navigation prompt encodes a flag\-routing table \(defined term→\\rightarrowisDefinition; amount→\\rightarrowhasMoney; …\) and an amendment rule: when an amendment addresses the same subject as the original, fetch*both*\. These rules operationalise how transactional lawyers actually read instruments, and they transfer to any legal retrieval system regardless of index format\.

Descriptor ladder\.The quality–cost trade\-off maps onto a descriptor ladder of increasing richness and indexing cost: \(D1\) heading only; \(D2\)\+\+structured flags; \(D3\)\+\+extracted keywords and entities; \(D4\)\+\+a capped one\-line micro\-summary; \(D5\) full section summary\. The configurations evaluated here do not climb it one rung at a time \(they already sit near the top\), so attributing answer quality to a specific rung remains an untested ablation, and the ladder should be read as a design map rather than a result\.

### 5\.3 Results

Table[3](https://arxiv.org/html/2607.05764#S5.T3)reportsinjectvs\.navindexacross six index descriptor configurations\. Configurations 1–5 use 15\-question subsets; Config 6 uses all 20 questions with the token\-optimised index\.

The configurations show three things\. First, capping returned nodes at1010\(Config 2\) reducednavindexcost by∼20%\{\\sim\}20\\%\($1\.86 vs\. $2\.32\) withnavindexwins unchanged at33, though ties fell from1111to99; over\-selection in Config 1 had inflated cost without protecting quality\. Second, raising the cap to2020\(Config 3\) gave no benefit: tie count dropped andinjectwins rose, so the additional nodes introduce noise rather than signal\. The node cap mattered more than any descriptor enrichment we tested\. Third, the token\-optimised Config 6 was judged tied withinjecton all 18 document\-bound questions and on both out\-of\-scope controls, at lower dollar cost thaninject\($1\.21 vs\. $1\.61\), the highest single\-run tie rate in either comparison\. Because Config 6 changes the serialisation and the question set together, comparisons against Configs 1–5 are confounded \(Table[3](https://arxiv.org/html/2607.05764#S5.T3), note‡\\ddagger\); its headline numbers are paired againstinjectwithin the same run and are unaffected\.

navindex’s efficiency story decomposes into three numbers, all from the per\-question ledger \(Appendix[B](https://arxiv.org/html/2607.05764#A2)\): total attended token footprint is1\.61×1\.61\\timeslower thaninject\(the∼98\.6\{\\sim\}98\.6K\-token cached index is re\-attended on every S1 scan\); the*answering\-model*context is∼56×\{\\sim\}56\\timessmaller \(∼2\.9\{\\sim\}2\.9K tokens of fetched provisions per query vs\.163163K\); and recorded dollar cost is 25% lower \($1\.21 vs\. $1\.61\)\. The long\-context benefit accrues at S3, where the answer is actually composed; the token\-footprint benefit is bounded by the index size, which is why it is1\.61×1\.61\\timesrather thannavembed’s17\.3×17\.3\\times\.

## 6 Cross\-Mode Discussion

Table[4](https://arxiv.org/html/2607.05764#S6.T4)places the headline numbers side by side, and Table[5](https://arxiv.org/html/2607.05764#S6.T5)attaches exact uncertainty to every headline rate\. All intervals are two\-sided 95% Clopper–Pearson\[[24](https://arxiv.org/html/2607.05764#bib.bib24)\]computed from the printed counts \(a convention used throughout the paper\)\. We use this*exact*binomial method rather than the textbook normal approximation because the counts are small \(n=15n=15–2020\) and several rates sit at the0/10/1boundary, where approximate intervals become unreliable: the standard Wald interval, for instance, collapses to zero width at a perfect18/1818/18, absurdly implying no uncertainty at all\. Each interval is the range of true rates consistent with the observed count, so a perfect18/1818/18bounds the true tie rate only to≥0\.81\{\\geq\}0\.81, since any lower rate would make a clean sweep implausibly lucky \(p18<0\.025p^\{18\}<0\.025\)\.navembedwas exercised across twelve configurations \(Appendix[A](https://arxiv.org/html/2607.05764#A1)\) and provides the token\-footprint\-reduction result;navindexreaches the highest single\-run tie rate and is the only mode that navigates by structure, the signal designed for questions that hinge on defined terms, liability caps, or explicit cross\-references, though whether that mechanism produces its tie result is untested \(Section[9](https://arxiv.org/html/2607.05764#S9)\)\.

A direct head\-to\-head comparison between RUN\-012 and Config 6 is not yet valid even though both use the full 20\-question set: each mode was judged only pairwise against its owninjectarm \(the two modes’ answers were never judged against each other\), and matching tie counts against a shared baseline transfer token footprint and dollar cost, not quality rank\. On the present evidence the two modes occupy different points on the quality–token\-footprint curve, and neither result supersedes the other\.

Table 4:Headline configurations of the twonavigatemodes againstinjecton the 20\-question scored set\. Each row is paired againstinjectonly; the rows are not head\-to\-head comparable \(Section[6](https://arxiv.org/html/2607.05764#S6)\)\. “Doc\-bound” is ties / 18 document\-bound questions; both out\-of\-scope controls were judged tied in all three runs\. “Red\.” \(reduction\) is theinject/navigateinput\-token ratio\. “Ans\. ctx\.” is the mean answering\-model context per query\.Table 5:Exact uncertainty for the headline rates \(Clopper–Pearson intervals, as described in Section[6](https://arxiv.org/html/2607.05764#S6)\)\. “Forced\-decision” is the ties\-disallowed probe of Section[8](https://arxiv.org/html/2607.05764#S8): the share of forced verdicts awarded toinject\.
## 7 Cost Analysis: The Caching Crossover

![[Uncaptioned image]](https://arxiv.org/html/2607.05764v1/x1.png)![[Uncaptioned image]](https://arxiv.org/html/2607.05764v1/x2.png)

Figure 2:Cumulative dollar cost versus repeated queries;navembedat the measured RUN\-012 payload \(9\.49\.4K tokens/query, full input price\)\.Left:a single∼35\{\\sim\}35K\-token document \(navindexomitted; its cost depends on that document’s index size\)\.Right:the measured163163K six\-document corpus;navindexat its Config 6 structure \(cached98\.698\.6K index re\-read per query plus∼2\.9\{\\sim\}2\.9K provisions at full input price; Eq\.[3](https://arxiv.org/html/2607.05764#S7.E3)\)\.Phase III settled the token\-footprint comparison; the dollar comparison is less direct\. The price model uses base input at $3\.00 per million tokens \(pinp\_\{\\text\{in\}\}\), cache write at $3\.75 \(pwp\_\{\\text\{w\}\}\), cache read at $0\.30 \(prp\_\{\\text\{r\}\}\), and output at $15\[[26](https://arxiv.org/html/2607.05764#bib.bib26)\]\. Output tokens are excluded from the cost model below because output length did not differ systematically betweeninjectandnavigatemodes in these runs \(median output length within±8%\\pm 8\\%across paired answers\); if output length diverges substantially in other corpora or question sets, the output term should be reinstated\. Every aggregate token and dollar figure for the two headline runs reconciles to the per\-question ledgers in Appendix[B](https://arxiv.org/html/2607.05764#A2)under this price card; dollar accounting is ledger\-validated from RUN\-005 onward \(Phase I dollar cells were recorded under an earlier harness and are excluded; Appendix[A](https://arxiv.org/html/2607.05764#A1)\)\. For a corpus ofCCtokens queried repeatedly against anavigatepayload ofRRtokens, at warm steady state:

cinjectwarm=C​pr,cnavigate=R​pin\.c\_\{\\textsc\{inject\}\{\}\}^\{\\text\{warm\}\}=C\\,p\_\{\\text\{r\}\},\\qquad c\_\{\\textsc\{navigate\}\{\}\}=R\\,p\_\{\\text\{in\}\}\.\(1\)injectis cheaper per query exactly when

C<R​pinpr=10​R\.C<R\\,\\frac\{p\_\{\\text\{in\}\}\}\{p\_\{\\text\{r\}\}\}=10\\,R\.\(2\)*Practical rule fornavembed:*cachedinjectis cheaper in dollars only when the corpus stays below roughly ten times the retrieval payload\. The headline RUN\-012 payload averages∼9\.4\{\\sim\}9\.4K tokens per query, putting break\-even near9494K tokens; across the Phase III configurations \(payloads5\.55\.5K–19\.119\.1K per query\) the threshold spans roughly5555K–191191K\. The measured six\-document prefix of163,147163\{,\}147tokens exceeds the RUN\-012 threshold, sonavembedstays cheaper at steady state, except for the BM25 configuration \(RUN\-011\), whose inflated19\.119\.1K payload pushes its threshold past the corpus size \(Section[4\.2](https://arxiv.org/html/2607.05764#S4.SS2)\)\.

navindexhas a different cost profile \(steps as defined in Section[5\.2](https://arxiv.org/html/2607.05764#S5.SS2)\): its S1 index \(∼98\.6\{\\sim\}98\.6K tokens\) is itself cached, so only the small S3 provision payload \(∼2\.9\{\\sim\}2\.9K tokens per query\) is billed at full input price\. Warm\-state per\-query cost isI​pr\+S​pinI\\,p\_\{\\text\{r\}\}\+S\\,p\_\{\\text\{in\}\}, whereIIis the cached index size andSSthe provision payload\. The crossover condition becomes

C​pr<I​pr\+S​pin⟹C<I\+S​pinpr≈98,600\+2,900×10≈128,000​tokens\.C\\,p\_\{\\text\{r\}\}<I\\,p\_\{\\text\{r\}\}\+S\\,p\_\{\\text\{in\}\}\\;\\Longrightarrow\\;C<I\+S\\,\\frac\{p\_\{\\text\{in\}\}\}\{p\_\{\\text\{r\}\}\}\\\\\[2\.0pt\] \\approx 98\{,\}600\+2\{,\}900\\times 10\\approx 128\{,\}000\\text\{ tokens\.\}\(3\)Cachedinjectbeatsnavindexonly when the corpus stays below roughly128128K tokens\. Both thresholds are exceeded by the measured163163K six\-document prefix, matching the recorded outcome \($1\.21 vs\. $1\.61\); for a single∼35\{\\sim\}35K\-token document,injectis the cheaper steady\-state option against eithernavigatemode\. \(The Phase II corpus was a different six\-document sample of the workspace \(it carried the PPG/Comex SPA in place of the disclosure schedule\) at∼214\.5\{\\sim\}214\.5K tokens per question against the scored corpus’s163\.1163\.1K\.\)

Session economics\.The warm\-state equation omits the one\-time cache write, which must also be amortised, and Anthropic’s prompt cache has a five\-minute TTL\[[18](https://arxiv.org/html/2607.05764#bib.bib18)\]\. Queries arriving more than five minutes apart invalidate the cache entry, and the next call pays the write premium again\. In a synchronous review session \(many questions in one sitting\) the model approximates reality; in an asynchronous workflow \(a lawyer asking three questions today and two tomorrow\) the cache write may be paid for every session, shifting the crossover to a higher query count or removing it entirely for infrequent use\. Cachedinject’s cheapness below the threshold therefore assumes a usage pattern that legal review often violates\. The analysis here assumes a single warm\-cache session\. Figure[2](https://arxiv.org/html/2607.05764#S7.F2)shows cumulative costs for two corpus sizes: for a single∼35\{\\sim\}35K\-token document \(left\), cachedinjectovertakesnavembedafter about seven repeated questions; for the163163K six\-document corpus \(right\),navembedremains cheaper at every repetition count\.

## 8 Threats to Validity

What a tie certifies\.A tie under the reference\-anchored protocol \(Section[3\.2](https://arxiv.org/html/2607.05764#S3.SS2)\) certifies that the judge found both answers equally consistent with the verified ground truth, not that either answer is verified correct\. Judge error is bounded by the audit \(2/152/15verdicts likely wrong; Table[5](https://arxiv.org/html/2607.05764#S6.T5)\)\. A forced\-decision variant of thenavindexConfig 2 setup \(ties disallowed\) on the 15\-question Phase II pool, whose DocNavBench questions are disjoint from the scored set \(Section[3](https://arxiv.org/html/2607.05764#S3)\), showed a1010–55injectpreference, judged with the Phase I–II preference judge\. That lean is statistically indistinguishable from chance \(two\-sided exact binomial test\[[24](https://arxiv.org/html/2607.05764#bib.bib24)\],p=0\.30p=0\.30\), the interval \(\[0\.38,0\.88\]\[0\.38,0\.88\]\) is wide, and it attaches to the Phase II questions rather than the scored set\. Even so, it remains the only forced\-decision datum available\. Single\-pass tie counts also carry judge\-pass noise: two stored judging passes over the*identical*RUN\-010 answers differed on one verdict in twenty \(a control\-question tie versus anavigatewin\), so tallies should be read with at least±1\\pm 1verdict of judging noise, well inside the intervals of Table[5](https://arxiv.org/html/2607.05764#S6.T5)\. The forced\-decision probe has not been re\-run under the final protocol or on the full 20 questions; a forced\-decision run on Config 6 is the single most important follow\-up, since it would establish whether the headline tie counts reflect genuine interchangeability or differences beneath the judge’s resolution\.

Judge–answerer provider overlap\.The judge \(claude\-opus\-4\-7\) shares a provider with the answering model \(claude\-sonnet\-4\-6\)\. Because*both*arms of every pair are answered by the same model, any family\-level self\-preference\[[27](https://arxiv.org/html/2607.05764#bib.bib27)\]applies to both answers symmetrically: the comparison is between retrieval conditions, not model families, so provider bias cannot systematically favour one arm\. The residual risk is tie*inflation*through stylistic coherence, compounded by the reference answers being model\-drafted before human verification \(Section[3](https://arxiv.org/html/2607.05764#S3)\), which the reference\-anchored judging criteria \(verdicts keyed to key facts against the reference, with explicit instructions to ignore formatting, length, and tone\) and the forced\-decision probe partially address, and which the audit bounds empirically\. A cross\-provider judge remains future work\.

No ground\-truth node labels\.Retrieval recall and precision cannot be measured\. Whennavigateanswers “not found,” it is impossible to tell whether it fetched the wrong nodes or the right nodes that lacked the answer; the diagnosis of RUN\-012’s twoinjectwins \(Section[4\.2](https://arxiv.org/html/2607.05764#S4.SS2)\) is correspondingly conjectural\.

Corpus\-selection artefacts\.In one Phase II run,11/1511/15questions were unanswerable because amendments were sampled without their base documents\. Both methods return “not in documents” and are scored a tie, a phantom tie that inflates apparent convergence\. The artefact predates the scored set: the printed 15\-question rows are inconsistent with it \(their per\-question records show substantive retrieval and decisive verdicts on the same questions\), and the scored runs are protected by construction rather than by luck\. Each document\-bound question has a verified ground\-truth answer authored against the indexed documents, and the per\-question ledgers \(Appendix[B](https://arxiv.org/html/2607.05764#A2)\) show substantive answers on all 18 with no mass “not found” behaviour; only the 2 out\-of\-scope controls are deliberately unanswerable\.

Out\-of\-scope controls\.The stored answers verify declination directly for the sharedinjectarm \(reused across RUN\-010–012\): both control answers are the mandated refusal string\. The storednavigateanswers in the ground\-truth\-judged oracle run likewise open with the exact refusal; for RUN\-012’s and Config 6’snavigatearms the controls are certified as*judged tied*against a reference that says the answer is absent, under a criterion where a fabricated answer loses\.

Difficulty profile and phrasing\.The internal difficulty labels \(14 high, 6 medium; Appendix[C](https://arxiv.org/html/2607.05764#A3)\) rate retrieval dispersion \(how widely the answer spreads across clauses and documents\), not cognitive difficulty: the reviewing lawyer assessed only a handful of the 20 as genuinely hard, the majority being precise retrieval\-and\-comprehension questions a modern LLM answers reliably once the governing text is in context\. That cuts both ways\. It makes the comparison a clean test of the retrieval condition, since ties are the expected outcome when both arms receive the right provisions and the informative signal is the failures; but the evaluation says nothing about legal reasoning\. The lawyer also noted the questions are phrased answer\-aware, written as if the asker already knows what the document contains, unlike DocNavBench’s issue\-driven register \(Section[3](https://arxiv.org/html/2607.05764#S3)\), so the phrasing may flatter retrieval relative to real\-world queries\.

Scale\.All results come from one answering model, one judge, and one run per configuration, on a 20\-question set over six documents; Table[5](https://arxiv.org/html/2607.05764#S6.T5)quantifies the resulting uncertainty, and the wide intervals are the honest reading of every rate in this paper\. Thenavembedandnavindexcomparisons are asymmetric in configuration count \(twelve vs\. six\), and only their strongest configurations share the scored question set, so the cross\-mode reading \(Section[6](https://arxiv.org/html/2607.05764#S6)\) is indicative, not definitive\.

## 9 Lessons Learned and Future Work

Four operational lessons transfer beyond this system\.\(1\) Cap the selection\.The ten\-node cap cutnavindexcost by∼20%\{\\sim\}20\\%with wins unchanged \(ties dipped11→911\{\\to\}9; Table[3](https://arxiv.org/html/2607.05764#S5.T3)\), and every uncapped or weakly\-signalled variant \(titles\-only scan, 20\-node cap, all\-related\-nodes\) over\-selected; a hard cap is the single highest\-leverage retrieval control we found\.\(2\) Lexical fusion can silently cost money\.BM25 length bias inflated the hybrid payload2×2\\timesover pure semantic retrieval and pushed the configuration past the caching crossover, worse thaninjectat steady state\.\(3\) What a node carries matters more than which embedding scores it\.Embedding swaps moved outcomes less than descriptor enrichment across Phases I–II \(Appendix[A](https://arxiv.org/html/2607.05764#A1)\), which is what motivated thenavindexformat\.\(4\) Track token footprint and dollar cost separately\.Prompt caching makes dollar comparisons corpus\- and session\-dependent \(Section[7](https://arxiv.org/html/2607.05764#S7)\); token\-footprint reduction is the durable quantity\.

The evaluation gaps, in order of urgency: a forced\-decision judging run on Config 6 \(Section[8](https://arxiv.org/html/2607.05764#S8)\); ground\-truth node labels to measure retrieval recall directly and to diagnose the two RUN\-012 losses; a head\-to\-head judging run in which the twonavigatemodes’ answers are compared directly rather than each againstinject; evaluation on the 52\-question open\-search DocNavBench set; substring rubrics \(must\_contain\) as a judge\-free scoring dimension; easy single\-fact questions to establish a retrieval floor; corpus pairing that always includes base agreements with amendments; questions that require reconciling an original agreement against its amendments across documents \(a cross\-instrument dependency the present question set does not isolate\); a second\-annotator pass over the headline runs’ judge verdicts; a cross\-provider judge; and integrating a LEDGAR\-style clause\-type classifier\[[21](https://arxiv.org/html/2607.05764#bib.bib21)\]as an index field\.

## 10 Conclusion

inject\(placing the entire corpus in context\) requires no retriever, and on a small corpus queried repeatedly within a warm cache session it is the cheaper option in dollars by the crossover rule of Section[7](https://arxiv.org/html/2607.05764#S7)\. Outside that regime, both structured alternatives held answer quality at a fraction of the token footprint on this evaluation, with the caveats of Section[8](https://arxiv.org/html/2607.05764#S8)\.

Againstnavembed, semantic retrieval with reranking \(RUN\-012\) was judged tied withinjecton 16 of 18 document\-bound questions \(injectpreferred on 2\) while attending to17\.3×17\.3\\timesfewer input tokens \(188188K vs\.3\.263\.26M\); GTE retrieval pushed the reduction to29\.9×29\.9\\timesat a lower tie rate \(15/2015/20\)\. Across all twelve configurations the reduction never fell below8\.5×8\.5\\times, and the binding constraint on quality was descriptor richness, not embedding choice\.

Againstnavindex, its index format encodes the defined\-term and cross\-reference relationships a legal answer depends on as traversable edges, and the deterministic fetch follows them without a second similarity search\. Its token\-optimised configuration was judged tied withinjecton all 18 document\-bound questions \(the highest single\-run tie rate in either comparison\) at1\.61×1\.61\\timeslower total token footprint, a∼56×\{\\sim\}56\\timessmaller answering context, and 25% lower recorded dollar cost \($1\.21 vs\. $1\.61\)\.

For practitioners the decision rule is short: below∼10×\{\\sim\}10\\timesthe retrieval payload \(a constant set by the cache\-read discountpin/prp\_\{\\text\{in\}\}/p\_\{\\text\{r\}\}under the Section[7](https://arxiv.org/html/2607.05764#S7)price card\), inject and cache; above it, retrieve, withnavembedwhen maximal token\-footprint reduction matters andnavindexwhen structural fidelity does\. Real transaction sets outgrow the context window, whereinjectstops being an option at all, and asynchronous legal workflows re\-pay the cache write per session, eroding cachedinject’s advantage even below the threshold\. Structured retrieval is how the system holds quality when the corpus keeps growing\.

### Acknowledgements

We are grateful to Horace Wu, who authored the 52\-question DocNavBench open\-search set and verified the scored benchmark’s reference answers against the source documents, and toMahmoud Abdallahfor his work on the document\-index API and surrounding features\.

## References

- \[1\]Anthropic\.Introducing the Model Context Protocol\.Anthropic, November 2024\.[https://www\.anthropic\.com/news/model\-context\-protocol](https://www.anthropic.com/news/model-context-protocol)
- \[2\]N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, P\. Liang\.Lost in the Middle: How Language Models Use Long Contexts\.*TACL*, 2024\.
- \[3\]D\. Hendrycks, C\. Burns, A\. Chen, S\. Ball\.CUAD: An Expert\-Annotated NLP Dataset for Legal Contract Review\.In*NeurIPS Datasets and Benchmarks*, 2021\.
- \[4\]Y\. Koreeda, C\. D\. Manning\.ContractNLI: A Dataset for Document\-Level Natural Language Inference for Contracts\.In*Findings of EMNLP*, 2021\.
- \[5\]C\. Merola, J\. Singh\.Reconstructing Context: Evaluating Advanced Chunking Strategies for Retrieval\-Augmented Generation\.*arXiv:2504\.19754*, 2025\.
- \[6\]S\. Taiwo, M\. A\. Yusoff\.Evaluating Chunking Strategies for Retrieval\-Augmented Generation in Oil and Gas Enterprise Documents\.*arXiv:2603\.24556*, 2026\.
- \[7\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, et al\.Retrieval\-Augmented Generation for Knowledge\-Intensive NLP Tasks\.In*NeurIPS*, 2020\.
- \[8\]Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, et al\.Retrieval\-Augmented Generation for Large Language Models: A Survey\.*arXiv:2312\.10997*, 2023\.
- \[9\]V\. Karpukhin, B\. Oğuz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, W\. Yih\.Dense Passage Retrieval for Open\-Domain Question Answering\.In*EMNLP*, 2020\.
- \[10\]N\. Reimers, I\. Gurevych\.Sentence\-BERT: Sentence Embeddings using Siamese BERT\-Networks\.In*EMNLP*, 2019\.
- \[11\]L\. Wang, N\. Yang, X\. Huang, B\. Jiao, L\. Yang, D\. Jiang, R\. Majumder, F\. Wei\.Text Embeddings by Weakly\-Supervised Contrastive Pre\-training\.*arXiv:2212\.03533*, 2022\.
- \[12\]Z\. Li, X\. Zhang, Y\. Zhang, D\. Long, P\. Xie, M\. Zhang\.Towards General Text Embeddings with Multi\-stage Contrastive Learning\.*arXiv:2308\.03281*, 2023\.
- \[13\]S\. Xiao, Z\. Liu, P\. Zhang, N\. Muennighoff, D\. Lian, J\.\-Y\. Nie\.C\-Pack: Packaged Resources to Advance General Chinese Embedding\.*arXiv:2309\.07597*, 2023\.
- \[14\]K\. Song, X\. Tan, T\. Qin, J\. Lu, T\.\-Y\. Liu\.MPNet: Masked and Permuted Pre\-training for Language Understanding\.In*NeurIPS*, 2020\.
- \[15\]S\. Robertson, H\. Zaragoza\.The Probabilistic Relevance Framework: BM25 and Beyond\.*Foundations and Trends in Information Retrieval*, 3\(4\), 2009\.
- \[16\]G\. V\. Cormack, C\. L\. A\. Clarke, S\. Buettcher\.Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods\.In*SIGIR*, 2009\.
- \[17\]R\. Nogueira, K\. Cho\.Passage Re\-ranking with BERT\.*arXiv:1901\.04085*, 2019\.
- \[18\]Anthropic\.Prompt Caching with Claude\.Anthropic Documentation, 2024\.[https://docs\.anthropic\.com/en/docs/build\-with\-claude/prompt\-caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching)\[Accessed: June 2026; content subject to change as a living document\]\.
- \[19\]L\. Zheng, W\.\-L\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, et al\.Judging LLM\-as\-a\-Judge with MT\-Bench and Chatbot Arena\.In*NeurIPS*, 2023\.
- \[20\]P\. Wang, L\. Li, L\. Chen, Z\. Cai, D\. Zhu, B\. Lin, Y\. Cao, et al\.Large Language Models are not Fair Evaluators\.*arXiv:2305\.17926*, 2023\.
- \[21\]D\. Tuggener, P\. von Däniken, T\. Peetz, M\. Cieliebak\.LEDGAR: A Large\-Scale Multi\-label Corpus for Text Classification of Legal Provisions in Contracts\.In*LREC*, 2020\.
- \[22\]P\. Sarthi, S\. Abdullah, A\. Tuli, S\. Khanna, A\. Goldie, C\. D\. Manning\.RAPTOR: Recursive Abstractive Processing for Tree\-Organized Retrieval\.In*ICLR*, 2024\.
- \[23\]M\. Zhang, Y\. Tang, et al\. \(VectifyAI\)\.PageIndex: Document Index for Vectorless, Reasoning\-based RAG\.2025\.[https://github\.com/VectifyAI/PageIndex](https://github.com/VectifyAI/PageIndex)
- \[24\]C\. J\. Clopper, E\. S\. Pearson\.The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial\.*Biometrika*, 26\(4\):404–413, 1934\.
- \[25\]W\. Wang, F\. Wei, L\. Dong, H\. Bao, N\. Yang, M\. Zhou\.MiniLM: Deep Self\-Attention Distillation for Task\-Agnostic Compression of Pre\-Trained Transformers\.In*NeurIPS*, 2020\.
- \[26\]Anthropic\.Pricing\.Anthropic, 2026\.[https://www\.anthropic\.com/pricing](https://www.anthropic.com/pricing)\[Accessed: June 2026; prices subject to change\]\.
- \[27\]A\. Panickssery, S\. R\. Bowman, S\. Feng\.LLM Evaluators Recognize and Favor Their Own Generations\.In*NeurIPS*, 2024\.

## Appendix ARun History and Lessons

The twelvenavembedconfigurations were run in three phases as the evaluation harness matured\. Each run is an internally controlled paired comparison \(injectvs\.navembedon identical questions\), but five things changed*between*phases: the question pool \(the DocNavBench subset in Phases I–II vs\. the scored set in Phase III; Section[3](https://arxiv.org/html/2607.05764#S3)\), the corpus sample \(Phase II carried the PPG/Comex SPA in place of the disclosure schedule; Section[7](https://arxiv.org/html/2607.05764#S7)\), the prompts, the caching setup \(integrated at RUN\-005\), and the judge \(the Phase I–II preference judge, Appendix[D\.5](https://arxiv.org/html/2607.05764#A4.SS5), vs\. the reference\-anchored judge, Appendix[D\.4](https://arxiv.org/html/2607.05764#A4.SS4); Section[3\.2](https://arxiv.org/html/2607.05764#S3.SS2)\)\. Cross\-run tie counts are therefore run history rather than a controlled sweep; the paper’s claims rest on the Phase III runs \(Section[4\.2](https://arxiv.org/html/2607.05764#S4.SS2)\)\. Footprint reduction \(the deterministic quantity\) held at8\.58\.5–29\.9×29\.9\\timesacross all twelve runs\.

Table 6:All scored\-runnavembedconfigurations\. “Red\.” \(reduction\) is theinject/navembedinput\-token ratio\. “Inj\. w\.” and “Nav\. w\.” are judge wins forinjectandnavembed\. Costs are recorded operational spend in USD; Phase I cells \(RUN\-001–004\) are omitted because they were recorded under an earlier harness and do not reconcile with the Section[7](https://arxiv.org/html/2607.05764#S7)price model \(dollar accounting is ledger\-validated from RUN\-005 onward; Appendix[B](https://arxiv.org/html/2607.05764#A2)\)\. Win splits are taken from the per\-question run records \(the run\-time ledgers; RUN\-010 from its stored run file, in which both judge directions agree on every question\)\.‡\\ddaggerOracle configuration: uses ground\-truth node labels to validate retrieval at indexing time; excluded from cross\-mode comparison \(§[6](https://arxiv.org/html/2607.05764#S6)\)\.

Phase I \(RUN\-001–004\): harness construction\.The first runs built and validated the harness on 3–9 questions from the Phase I–II pool \(a 15\-question DocNavBench subset; Section[3](https://arxiv.org/html/2607.05764#S3)\): filtering irrelevant nodes, stripping rationale from model output, and adding cross\-reference expansion\.navembedalready used1010–15×15\\timesfewer tokens thaninject; RUN\-003 produced the first outrightnavembedwin, and RUN\-004 closed the phase with threenavembedwins and six ties\.

Phase II \(RUN\-005–008\): caching and embeddings\.Moving to the full 15\-question pool with prompt caching integrated, the token footprint held1414–19×19\\timesbelowinject; embedding swaps were tried here \(Section[4\.2](https://arxiv.org/html/2607.05764#S4.SS2)\):paraphrase\-MiniLM\-L6\-v2\(the incumbent default\),intfloat/e5\-base\-v2, andBAAI/bge\-base\-en\-v1\.5were compared on the same pool, and MPNet QA embeddings \(multi\-qa\-mpnet\-base\-cos\-v1, RUN\-008\) gave the best Phase II tie count \(9/159/15\)\.

Phase III \(RUN\-009–012\): the scored comparison\.On the full 20\-question set with the final harness, outcomes are reported in Section[4\.2](https://arxiv.org/html/2607.05764#S4.SS2)\. Theinjectwins concentrated in Phase II, while the harness was still being tuned \(Table[6](https://arxiv.org/html/2607.05764#A1.T6), Inj\. w\. column\); this trend is an engineering observation, not a measured result, since the question sets differ across phases\.

## Appendix BPer\-Question Detail for Two Representative Runs

Tables[8](https://arxiv.org/html/2607.05764#A2.T8)and[8](https://arxiv.org/html/2607.05764#A2.T8)give per\-question token and cost breakdowns\. Column key:out= output tokens;CW= cache\-write \(Q1 only\);CR= cache\-read \(Q2–Q20\);in= non\-cached input;Inj/Nav= cost in USD\. Every aggregate token and dollar figure reported for these two runs in the body reconciles to these ledgers under the Section[7](https://arxiv.org/html/2607.05764#S7)price card\. Two bookkeeping notes: theinjecttoken totals in Table[6](https://arxiv.org/html/2607.05764#A1.T6)and Table[3](https://arxiv.org/html/2607.05764#S5.T3)additionally include856856non\-cached question\-text input tokens not broken out as a column here; and the Nav rows likewise include2020–6060tokens of non\-cached S1 question\-text input per query not broken out as a column\.

Table 7:Config 6:navindextoken\-optimised, 20 questions \(all ties\)\. Nav\. S1 caches the∼\\sim98\.6 K\-token index JSON \(CW on Q1, CR on Q2–Q20\); Nav\. S3 fetches verbatim clauses \(no caching\)\.Table 8:RUN\-012: semantic\+\+rerank, 20 questions \(18 ties, 2 inject wins\)\. Navigate retrieves top\-kkchunks via embedding\+\+rerank; no caching\.
## Appendix CScored Question Set

Table[9](https://arxiv.org/html/2607.05764#A3.T9)lists all 20 scored questions with verified ground\-truth answers; 18 are document\-bound and 2 are out\-of\-scope controls that test correct declination\. All are high difficulty except those marked†\\dagger\(medium\)\.

Table 9:All 20 scored benchmark questions with verified ground\-truth answers\.†\\daggermedium difficulty; all others high\.
## Appendix DSystem Prompts

All prompts are reproduced verbatim fromprompts\.py\. Runtime substitutions are shown as\{placeholder\}\.

### D\.1 INJECT Answer Prompt \(INJECT\_ANSWER\_SYSTEM\)

Youareaseniorlegalanalystpreparingaprecise,citation\-anchoredanswer

forapartnerworkingacrossseveralrelateddocuments\.

\#Sourceoftruth

Theonlyauthoritativesourceisthedocumentsdelimitedby

<documentdoc\_id="\.\.\."\>tagsbelow\.

\-Each<document\>isaseparateinstrument;treatthemasacoordinated

transactionset\.

\-Donotimportgenerallegalknowledgeorassumptionsfromoutsidethe

documents\.

\-DefinedtermshaveONLYthemeaninggiveninthedocuments\.Atermmaybe

definedinonedocumentandusedinanother\-\-trackdefined\-termorigin\.

\#Reasoningmethod\(internal\-\-donotoutputyourscratchwork\)

1\.Decomposethequestionintosub\-questions,conditions,thresholds,parties\.

2\.Foreachsub\-question,identifywhichdocument\(s\)containrelevanttext\.

3\.Resolvecross\-referencesbetweendocuments\(e\.g\.anamendmentthatmodifies

aclauseoftheoriginal\)\.

4\.Reconcileprecedence:

\-Alater\-datedamendment,sideletter,restatement,orsupplementOVERRIDES

theoriginaltotheextentofanyinconsistency\.

\-Amorespecificprovisionusuallyoverridesamoregeneralone\.

\-Whenindoubt,statewhichdocumentyoutreatedascontrollingandwhy\.

5\.Composeasingleintegratedanswer\.

\#Answerformat

\-Leadwiththedirectanswer\(1\-2sentences\)\.Thenashortsupportinganalysis\.

\-Citeeveryload\-bearingfactas:"\[doc\_id\]ClauseX"\.

\-Quoteshort,decisivephrasesverbatimwherewordingmatters\.

\-Preserveexactnumericvalues,currencies,dates,andpartynames\.

\-Ifdocumentsconflict,namethecontrollingdocumentandexplainwhy\.

\-Ifadefinedtermisused:identifywhichdocumentdefinesit\.

\#Whenthedocumentsaresilent

Ifthedocumentsdonotaddressthequestion,respondEXACTLY:

"Theprovideddocumentsdonotcontaintheanswer\."

\{documents\_block\}

### D\.2 NAVINDEX S1 Prompt \(NAVINDEX\_SCAN\_SYSTEM\)

The model sees only the compact index JSON \(no provision text\) and returns a list of\(doc\_id, node\_id\)pairs to fetch\.

Youarealegalretrievalplannerworkingacrossseveralrelateddocumentsina

transactionset\(e\.g\.anoriginalagreementplusamendments,sideletters,

schedules\)\.Yourjobistodecidewhichprovisionsacrossalldocumentsahuman

lawyerwouldneedtoreadend\-to\-endtoanswertheuser’squestion\.

\#Whatyoucanandcannotsee

YouseeONLYtheindices\-\-oneperdocument\-\-containingclausereferences,

headings,textsnippets,summaries,keywordtags,andstructuralmetadata\.

YoudoNOTseethefullprovisiontext,onlytheopeningsnippetofeachnode\.

YoumustdecidewhattofetchfromSTRUCTURALSIGNALSALONE\.

\#Indexfieldguide\(samestructureperdocument\)

Each<indexdoc\_id="\.\.\."\>blockhas:

\-documentTitle\-\-thedocument’stitle\.

\-documentTags\-\-Jurisdiction\(governinglaw\)andTopic\(key

subject\-matterterms\)\.Usetoidentifywhich

documentsarelikelyrelevantbeforescanningnodes\.

\-documentIndex\-\-arecursivetreeofsectionnodes\.Eachnodehas:

\-nodeId\-\-theidentifieryoureturn\.Alwaysusethis,never

clauseReference\.

\-clauseReference\-\-printedclausereference\.Usefororientationonly\.

\-title\-\-clauseheading;primarytopicalsignal\.

\-summary\-\-AI\-generatedsummary\.Usetoconfirmrelevancewhen

titleandsnippetareinsufficient\.

\-keywords\-\-topictagsappliedtothisclause\.

\-crossReferencedIds\-\-nodeIdsthisclauseexplicitlycross\-references\.

Followselectivelyfordefinitions/conditions\.

\-children\-\-nestedsub\-clauses,recursivelystructured\.

\-isDefinition\-\-trueifthisnodedefinesoneormoreterms\.

\-hasMoney\-\-trueiftheprovisioncontainsamonetaryamount\.

\-hasDate\-\-trueiftheprovisioncontainsadateordeadline\.

\-hasPercentage\-\-trueiftheprovisioncontainsapercentageorrate\.

\-hasLiabilityLimit\-\-trueiftheprovisioncontainsaliabilitycap\.

\-hasCondition\-\-trueiftheprovisioncontainsaconditionprecedent\.

\-hasParty\-\-trueiftheprovisionreferencesanamedparty\.

\-isTocEntry\-\-trueiftable\-of\-contentsentry\.NEVERincludethese\.

\-textLength\-\-charactercount\.Nodesunder~50charsarestubs\.

\#Retrievalstrategy\(thinkbeforeyouanswer\)

1\.Decomposethequestion:sub\-questions,concepts,thresholds,conditions,

parties\.

2\.ScandocumentTags\.Topicacrossalldocumentstoidentifyrelevantdocuments\.

3\.DiscardanynodewhereisTocEntryistrue\.

4\.TreatEVERYdocumentasinscopeuntilyouhaveastructuralreasontorule

itout\.Theansweroftenlivesinmorethanonedocument\.

5\.Usebooleanflagsasfastfiltersbeforereadingsummaries:

\-definedterminquestion\-\>isDefinition:truenodesacrossalldocs

\-amountorfee\-\>hasMoney:truenodes

\-dateordeadline\-\>hasDate:truenodes

\-rateorpercentage\-\>hasPercentage:truenodes

\-liabilitycap\-\>hasLiabilityLimit:truenodes

\-conditionortrigger\-\>hasCondition:truenodes

\-specificparty\-\>hasParty:truenodes

6\.Confirmeachcandidateviatitle,snippet,summary,keywords\.

7\.Whenanamendmentaddressesthesamesubjectastheoriginal,includeBOTH\.

8\.FollowcrossReferencedIdsonlywhenthenodesuppliesadefinitionor

conditiontheselectedclauseexplicitlydependson\.

\#Hardrules

\-ReturnATMOST10\(doc\_id,node\_id\)pairs\.

\-doc\_idMUSTexactlymatcha<indexdoc\_id="\.\.\."\>valueintheblockbelow\.

\-Donotinventdoc\_idsornode\_ids\.

\-Decidefromstructuralsignalsonly\-\-donotspeculateaboutprovisiontext\.

\{indices\_block\}

### D\.3 NAVINDEX / NAVEMBED Answer Prompt \(NAVIGATE\_ANSWER\_SYSTEM\)

The model sees only the fetched provision texts and composes the final answer\.

YouareaseniorlegalanalystansweringaquestionfromaCURATEDEXTRACT

spanningseveralrelateddocuments\.Theprovisionsbelowwereretrievedbecause

theywerejudgedlikelytoberelevant;youmustanswerfromthisextractalone\.

\#Sourceoftruth

Theonlyauthoritativesourceisthetextinside<provisions\>\.\.\.</provisions\>\.

\-Eachprovisionistagged"\[doc=\.\.\.,clause\_ref=\.\.\.,node\_id=\.\.\.\]"\.Alwayscite

BOTHthedocandtheclause\.

\-Anamendment,sideletter,orrestatementOVERRIDEStheoriginaltothe

extentofanyinconsistency\.

\-Donotinvokegenerallegalknowledgefromoutsidetheextract\.

\-DefinedtermshaveONLYthemeaninggivenintheextract\.Ifatermisused

butitsdefinitionisNOTintheextract,sayso\.

\#Relevancefiltering\-\-DONOToutputanypartofthisstep\.

Beforereasoning,screeneveryprovisionagainstthequestion:

\-MarkeachprovisionRELEVANTorIRRELEVANT\.

\-AnswerEXCLUSIVELYfromtheRELEVANTprovisions\.

\-Ifuncertain,erronthesideofINCLUDING\-\-over\-exclusionproducesafalse

"noanswer"finding;over\-inclusionisaminorinefficiency\.

\-OnlymarkIRRELEVANTiftheprovisionclearlyaddressesadifferentsubject

withnobearingonanysub\-question\.

\#Reasoningmethod\-\-DONOToutputthisstep\.

1\.Decomposethequestionintosub\-questions,conditions,thresholds,parties\.

2\.Foreachsub\-question,identifywhichdocument\(s\)bearonit\.

3\.Resolveprecedence:

\-Alater\-in\-timeamendmentoverridestheoriginaltotheextentof

inconsistency\.

\-Amorespecificprovisionusuallyoverridesamoregeneralone\.

4\.Resolvecross\-referenceswithintheextract\.Flagpointersoutsideit\.

5\.Composeasingleintegratedanswer\.

\#Answerformat

\-Leadwiththedirectanswer\(1\-2sentences\)\.Thenabriefsupportinganalysis\.

\-Citeeveryload\-bearingfactas:"\[doc\_id\]ClauseX"\.

\-Quoteshort,decisivephrasesverbatimwherewordingmatters\.

\-Preserveexactnumericvalues,currencies,dates,andpartynames\.

\-Whendocumentsconflict,namethecontrollingdocumentandexplainwhy\.

\#Partial\-informationrule

Iftheextractismissingakeylinkedclause:statewhattheextractDOES

establish,identifywhatismissing,anddonotfabricatethemissingcontent\.

\#Silencerule

Iftheextractdoesnotaddressthequestionatall,respondEXACTLY:

"Theprovidedtextdoesnotcontaintheanswer\."

Onlyinvokethisafterconfirmingeveryprovisionisgenuinelyirrelevant\.

<provisions\>

\{provisions\}

</provisions\>

### D\.4 Ground\-Truth Judge Prompt \(JUDGE\_GROUNDTRUTH\_SYSTEM\)

Used for the scored benchmark \(20 questions with verified reference answers\)\. The forward and reverse calls share an identical system prompt; only the user message \(which places Answer A and Answer B\) changes between orderings\.

Youareevaluatingtwomodelanswerstoalegalquestion\.Averifiedreference

answerisprovided\-\-treatitastheauthoritativegroundtruth\.

\#Yourtask

Decidewhichmodelanswer\(AnswerAorAnswerB\)moreaccuratelyreflectsthe

referenceanswer,ordeclareatieifbothareequallycorrectorequallywrong\.

\#Judgingcriteria\(inorder\)

1\.Keyfacts\-\-doestheanswerincludethespecificfactsstatedinthe

reference\(exactnumbers,dates,partynames,amounts,percentages,clause

citations\)?Missingorwrongkeyfactsaretheprimarybasisforaloss\.

2\.Nohallucination\-\-doestheansweravoidassertingfactsthatcontradict

thereferenceorthatarenotsupportedbythereference?

3\.Completeness\-\-ifthereferencehasmultipleparts,doestheansweraddress

allofthem?

4\.No\-answercorrectness\-\-ifthereferencesaystheanswerisnotinthe

documents,ananswerthatcorrectlysays"notfound"iscorrect;ananswer

thatfabricatesaplausible\-soundingfactiswrong\.

\#Verdictrules

\-Return"A"ifAnswerAisclosertothereference\.

\-Return"B"ifAnswerBiscloser\.

\-Return"tie"ifbothareequallycorrect,equallywrong,ordifferonlyin

stylewhilecontainingthesamekeyfacts\.

\-Apartialanswerbeatsa"notfound"whenthereferenceconfirmsafact

exists\-\-andlosestoitwhenthereferenceconfirmstheanswerisabsent\.

\#Anti\-biasrules

\-Ignoreformatting,length,andtone\.

\-Donotprefermoreverboseormoreconfidentanswers\.

\-Thereferencemaybeterse;donotpenaliseextradetailthatisnot

contradictory\.

Thequestion,referenceanswer,andtwomodelanswersareintheusermessage\.

### D\.5 Phase I–II Preference Judge \(JUDGE\_PREFERENCE\_SYSTEM\)

Used to score the Phase I–II runs,navindexConfigs 1–5 included: a pairwise comparison of two anonymised answers with*no*reference answer\. The 20\-question scored runs use the reference\-anchored judge of Appendix[D\.4](https://arxiv.org/html/2607.05764#A4.SS4)instead\. The criteria block is static across every judge call in a run, so it forms a cacheable prefix shared by the forward and reverse calls\.

Youarecomparingtwoanonymisedanswerstothesamelegalquestion\.Decidewhichanswerisbetter,orwhethertheyareequivalent\.

\#Judgingcriteria\(inorderofimportance\)

1\.Factualcorrectness\-\-doestheanswerstatetherightfacts?Ananswerwithonewrongfactlosestoananswerwithallrightfacts,eveniftheright\-factanswerisshorter\.

2\.Completeness\-\-doesitcovereverythingthequestionasks?Amulti\-partquestionrequireseverypartaddressed\.

3\.Specificity\-\-doesitcitethecontrollingclause?doesitpreserveexactnumericvalues,currencies,dates,parties?

4\.Honestyaboutuncertainty\-\-ananswerthatcorrectlyflagsmissingcontextbeatsonethathallucinatesaconfidentwronganswer\.

5\.Clarity\-\-isitunambiguousandeasytofollow?\(leastimportanttiebreaker\)

\#Verdictrules

\-Return"A"ifAisclearlybetteronthecriteriaabove\.

\-Return"B"ifBisclearlybetter\.

\-Return"tie"ifbothareequallygood,equallybad,ordifferonlyinstyle\.

\-Donotbreaktiesartificially\.Ifneitherismeaningfullybetter,"tie"isthecorrectanswer\.

\#Anti\-biasrules

\-Ignorelengthunlessoneanswerissoterseitomitsrequiredinformation\.

\-Ignoreformatting\(bulletpointsvs\.prose\)\.

\-Ignoretoneandconfidencewording\.

Thequestionandthetwoanonymisedanswerswillbeprovidedintheusermessagetagged<question\>,<answer\_a\>,and<answer\_b\>\.ReturnyourverdictusingtheJSONschemayouhavebeengiven\.