Locating Failure in Multi-Page Visually Rich Document Understanding: An Empirical Attribution

# Locating Failure in Multi-Page Visually Rich Document Understanding: An Empirical Attribution
Source: [https://arxiv.org/html/2608.07943](https://arxiv.org/html/2608.07943)
Lewei Xu1Yihao Ding1∗Zihan Xu2Daniel Yitian Su1 Daochang Liu1Siwen Luo1Yifan Peng3Wei Liu1 1The University of Western Australia2The University of Melbourne3Weill Cornell Medicine ∗Correspondence:yihao\.ding@uwa\.edu\.au

###### Abstract

Multi\-page visually\-rich document understanding \(MP\-VRDU\) requires managing evidence that is sparse, spread across pages, and often exceeds a model’s context window\. Prior work has produced competing, largely untested claims about how these systems should be built\. We attribute incorrect answers to three failure modes, representation, selection, and reasoning, and isolate each over a multi\-page document understanding dataset by intervening on one while holding the others fixed\. We find that vision is necessary but does not replace text extraction, that missing pages bound accuracy while distractors cost little, and that reasoners fail to integrate evidence across pages even when it is fully supplied\. Prompting can shift reasoning behaviour substantially, improving some outcomes at the expense of others\. We translate these findings into guidance for building such systems under a fixed compute budget\.

Locating Failure in Multi\-Page Visually Rich Document Understanding: An Empirical Attribution

Lewei Xu1Yihao Ding1∗Zihan Xu2Daniel Yitian Su1Daochang Liu1Siwen Luo1Yifan Peng3Wei Liu11The University of Western Australia2The University of Melbourne3Weill Cornell Medicine∗Correspondence:yihao\.ding@uwa\.edu\.au

## 1Introduction

Answering a question over a long visually\-rich document is at its core an evidence\-management problem\. The evidence bearing on a question is sparse, spread across many pages, and often larger than the reasoner’s context window, so a system cannot simply read the whole document and answer\. It must instead decide what to acquire, encode, select, and reason over under a fixed compute budget\(Xuet al\.,[2026](https://arxiv.org/html/2608.07943#bib.bib15)\)\. Multi\-page visually\-rich document understanding \(MP\-VRDU\) is therefore distinct from the single\-page setting, defined by how a system manages evidence it cannot hold all at once\.

The field has explored this problem from many directions, but its conclusions remain unsettled\. One disagreement concernsrepresentation\. Recent OCR\-free approaches argue that image input avoids text extraction errors\(Tanakaet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib55); Zhenget al\.,[2026](https://arxiv.org/html/2608.07943#bib.bib50)\), while other work finds that text\-free reasoners struggle on dense passages\(Hannanet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib74)\)and that combining text and vision outperforms either modality alone\(Hanet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib51); Suriet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib40)\)\. A second disagreement concernsselection\. Fixed\-depth retrieval is brittle because a question can require more evidence pages than a fixed depth returns, while too many introduce distractors that dilute attention\(Yanet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib43); Liet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib57)\)\. At the same time, other work identifies retrieval coverage as the main factor of accuracy\(Liuet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib37); Zhuet al\.,[2025a](https://arxiv.org/html/2608.07943#bib.bib82)\)\. A third disagreement concernsreasoning\. Performance drops on long inputs are attributed to context\-length limits\(Wanget al\.,[2024b](https://arxiv.org/html/2608.07943#bib.bib110); Liuet al\.,[2024](https://arxiv.org/html/2608.07943#bib.bib133)\), or to failures of evidence integration that persist even when the context is short enough to fit\(Zhenget al\.,[2026](https://arxiv.org/html/2608.07943#bib.bib50); Guoet al\.,[2026](https://arxiv.org/html/2608.07943#bib.bib78)\)\.

These claims shape system design, but they are difficult to test and compare because they are reported across different pipelines, datasets, and operating points\. As a result, the field has gathered design intuitions without a corresponding account of which ones actually hold\.

In this paper, we argue that these disagreements can be organized around three operations that every MP\-VRDU system must perform:encodethe document,selectthe evidence shown to the reasoner, andreasonover that evidence\. This yields three corresponding loci of failure:representation, where the encoding fails to preserve the evidence needed to answer the question;selection, where the required evidence is absent or diluted by irrelevant evidence; andreasoning, where the model cannot combine adequate evidence or misjudges whether it supports an answer\. This decomposition is useful because it abstracts over implementation details\. Techniques differ widely but intervene on the same operations, and agentic or iterative systems merely apply them repeatedly\.

We use this decomposition as a framework for controlled attribution by intervening on one locus while holding the others fixed\. We apply this decomposition to a single\-pass pipeline on a multi\-page document benchmark, and test several competing claims about modality, retrieval, and reasoning\. Our study yields three main findings\. First, vision is necessary for MP\-VRDU but does not replace text extraction\. Second, omitting evidence sharply limits accuracy, whereas adding distractor evidence is less harmful than commonly assumed\. Third, current reasoners fail to integrate evidence across pages, a limit that prompting and scale only partly close\. Together, these findings suggest that MP\-VRDU bottlenecks are more structured than current debates imply\.

Our contributions are as follows\.\(1\)We introduce an attribution framework that formalises answer failure in MP\-VRDU as three loci, representation, selection, and reasoning, each with two mechanisms\. Because the loci are defined at the level of system function rather than model architecture, they offer a common formalisation for analysing and comparing methods\.\(2\)We conduct a controlled empirical study that isolates each locus and resolves several of the field’s competing and largely untested claims about modality, retrieval, and reasoning under a unified setup\.\(3\)We translate the resulting analysis into practical guidance for designing and deploying MP\-VRDU pipelines under a fixed compute budget\.

## 2Related Work

Prior works evaluate representations, retrievers, and reasoners through end\-to\-end accuracy, making it difficult to identify which stage causes an improvement or failure\(Tanakaet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib55); Hanet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib51); Suriet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib40); Yanet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib43)\)\. Stage\-specific diagnostic studies provide cleaner attribution by isolating OCR, retrieval, or reasoning\(Zhanget al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib26); Zarrinkiaet al\.,[2026](https://arxiv.org/html/2608.07943#bib.bib19)\), but they remove competing stages and therefore cannot measure cross\-stage compensation\. Unified benchmarks improve comparability across systems\(Penget al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib31)\), yet still localise neither gains nor cross\-page reasoning failures\. We address this gap through controlled interventions within one pipeline, varying representation, selection, and reasoning separately to attribute errors and test competing design claims at the stage they concern\.

![Refer to caption](https://arxiv.org/html/2608.07943v1/x1.png)Figure 1:The three failure modes and their mechanisms\.
## 3Attribution Framework

An incorrect answer from an MP\-VRDU system can arise from three*failure modes*: Representation, Selection, and Reasoning\. These failure modes are hierarchical: evidence lost during representation cannot be selected, and evidence that is never selected cannot be incorporated into reasoning\.

Representationrefers to how faithfully a document is encoded into the tokens consumed by selection and reasoning, whether as text or visual tokens\. Representation failures occur through two mechanisms\.\(1\) Modality Ceiling\.Each modality can represent only a bounded range of signals\. A representation preserves a signal only when its modality can encode it and at the granularity permitted by its representation\. For example, text tokens cannot capture non\-linguistic information, and visual tokens express fine\-grained details only when sufficient spatial resolution and token budget are available\.\(2\) Conversion Fidelity\.For signals that a modality can represent, conversion fidelity measures how accurately the encoding reproduces them\. Failures occur when the conversion degrades or corrupts a signal that the modality could otherwise carry\. For instance, a document parser may misread a numeric value and cause the downstream reasoner to operate on an incorrect but seemingly reliable representation\(Zhanget al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib26)\)\.

Selectionconcerns what evidence is delivered to the reasoner\. Selection failures arise through two mechanisms\.\(1\) Evidence Coverage\.This mechanism measures how much of the evidence required to answer a question is present in the selected context,*i\.e\.*, the recall of the required evidence units\. Omitting required evidence leaves the reasoner with an incomplete evidence set, placing an upper bound on downstream accuracy regardless of the reasoner’s capability\.\(2\) Distractor Exposure\.Selection may also deliver evidence that is irrelevant to the question\. This mechanism measures the extent to which distractors accompany the required evidence\. Unlike omission, distractors do not by themselves make the evidence insufficient, but they increase the burden on the reasoner to identify and use the relevant content\.

Reasoningconcerns what the model does with the evidence it is given\. Reasoning failures can arise through many mechanisms; we focus on two\.\(1\) Evidence Integration\.Multi\-page questions may require the reasoner to combine complementary information from several pages, including content whose meaning spans page boundaries\. A model may succeed when all relevant evidence is given but fail when it must connect distributed evidence on its own\. This difficulty may be amplified in long contexts, where evidence at different positions is not used equally reliably\(Liuet al\.,[2024](https://arxiv.org/html/2608.07943#bib.bib133)\)\.\(2\) Response Calibration\.This mechanism concerns whether the model aligns its response with the level of support provided by the available evidence\. Unlike integration, calibration is tested both when sufficient evidence is present and when it is absent by construction\. Failures occur in two directions:*hallucination*, where the model produces an answer unsupported by the available evidence\(Liet al\.,[2023](https://arxiv.org/html/2608.07943#bib.bib28)\), and*false abstention*, where it refuses to answer despite having sufficient evidence\. We treat these as a single mechanism because they represent opposite sides of the same decision, so an intervention that reduces one tends to increase the other\(Whiteheadet al\.,[2022](https://arxiv.org/html/2608.07943#bib.bib2)\)\.

## 4Experimental Setup

We instantiate the framework in a single\-pass retriever–generator document question answering system, whose stages act once and in order so that errors can be attributed to individual failure modes\. We then describe the interventions that isolate each failure mode, the pipeline configuration, and the evaluation protocol\.

### 4\.1Attribution by Construction

Because each failure mode bounds those that follow, we isolate each by varying only the condition on which its failures depend\.

Representation Interventions\.We examine whether information is lost because the representation cannot express the required evidence or because it encodes that evidence inaccurately\. All downstream components are held fixed: the same reasoner reads the same annotated gold pages, so the encoding is the only variable\. The two mechanisms then map onto two axes of variation\.*Modality Ceiling*is varied across four document encodings that differ in their use of text, layout, and vision\.*Conversion Fidelity*is varied by quality within each modality, through a parser for text and a render resolution for vision\.

Selection Interventions\.We examine whether errors arise because required evidence is absent from the delivered context or because that evidence is diluted by irrelevant pages\. Representation and reasoning are held fixed, and the page selection is controlled directly\. The two mechanisms map onto two perturbations of the gold evidence set\.*Evidence Coverage*is varied by withholding gold pages from multi\-page questions and evaluating against the complete gold evidence\.*Distractor Exposure*is varied by augmenting the gold page set with non\-evidence pages while keeping the required evidence unchanged\.

Reasoning Interventions\.We examine whether the reasoner fails to combine evidence it holds or to calibrate its responses to the available support\. Representation and selection are neutralised by supplying the complete gold page set in the richest document encoding\. The two mechanisms map onto two contrasts\.*Evidence Integration*contrasts questions whose evidence lies on a single page against questions whose evidence spans several pages under the same documents and encoding\.*Response Calibration*contrasts answerable against unanswerable questions while varying only the prompting strategy given to the reasoner \(§[4\.2](https://arxiv.org/html/2608.07943#S4.SS2)\)\.

### 4\.2Pipeline Configuration

We instantiate the controlled interventions above in a three\-stage pipeline comprising page encoding \(representation\), evidence retrieval \(selection\), and answer generation \(reasoning\)\.

Page Encoding\.Each page is encoded from three possible signals: text \(T\), layout \(L\), and vision \(V\)\. Text is taken from the page’s embedded text layer, layout is the structural markup produced by a document parser, including reading order and table structure, and vision is the rendered page image consumed directly by the multimodal model\. We construct three encoding forms ordered by cost and fidelity: embedded text \(T\), parser\-generated layout\-aware text \(TL\), and parser\-generated text with the page image \(TLV\), with the page image alone \(V\) as an additional comparison\. PyMuPDF is used to extract raw flattened document text, while PaddleOCR\-VLCuiet al\.\([2025](https://arxiv.org/html/2608.07943#bib.bib16)\)is used as the default parser, with MinerU 2\.5Niuet al\.\([2025](https://arxiv.org/html/2608.07943#bib.bib17)\)and unlimitedOCRYinet al\.\([2026](https://arxiv.org/html/2608.07943#bib.bib3)\)included in the parser comparison\. Page images are rendered at low, medium, and high resolutions, with the medium preset used by default\.

Evidence Retrieval\.Evidence is selected at page granularity, matching the benchmark annotations so that a selected unit and an annotated evidence unit are the same object\. Controlled experiments supply page sets built from the annotated gold pages: coverage tests withhold the highest\- or lowest\-ranked gold page, and exposure tests add the top\-ranked non\-gold pages as distractors, with page rank given by BM25 and ColQwen3 rankers\. Retrieval experiments use these same rankers at varying depths\. We report ColQwen3 throughout the main text and BM25 in Appendix[C](https://arxiv.org/html/2608.07943#A3)\. Selected pages are restored to their original document order before being passed to the answer generator\.

Answer Generation\.The default answer generator is Qwen3\-VL\-8B\-InstructBaiet al\.\([2025](https://arxiv.org/html/2608.07943#bib.bib18)\), loaded in bfloat16 and decoded greedily\. It receives the selected pages in a single pass and, in the baseline condition, is given no additional instruction beyond the question\. Robustness experiments vary Qwen3\-VL scale across 2B, 4B, 8B, and 32B, compare the 8B model with InternVL3\-8BZhuet al\.\([2025b](https://arxiv.org/html/2608.07943#bib.bib4)\), and evaluate 8\-bit and 4\-bit quantisation\. We also vary the instruction using grounding, abstention, and step\-by\-step reasoning prompts\.

Table 1:MMLongBench\-Doc composition\.
### 4\.3Data and Evaluation

We evaluate on MMLongBench\-DocMaet al\.\([2024](https://arxiv.org/html/2608.07943#bib.bib104)\), summarised in Table[1](https://arxiv.org/html/2608.07943#S4.T1)\. Its annotations define the subset each experiment runs on\. The representation and reasoning experiments use answerable questions, so that answer accuracy is measured on its own without abstention mixed in\. Selection withholds evidence from multi\-page questions to test coverage, and adds distractors to questions of a fixed gold\-page count to test exposure\. Multi\-hop reasoning integration contrasts the single\- and multi\-page questions\. The instruction\-prompt experiments pair the answerable and unanswerable pools\. Where it is informative, we read a finding split by the evidence source a question draws on, by its document domain, and by whether the source document is born\-digital or scanned\. Answers are scored by an LLM judge rather than exact match, since a correct answer may differ from the gold string in units, rounding, or formatting that exact match would reject\. We use Gemini 2\.5 Flash as the judge, and report judge\-scored accuracy with 95% confidence intervals from a document\-level bootstrap\. Full details on pipeline configuration, data, and evaluation are available in Appendix[A](https://arxiv.org/html/2608.07943#A1)\.

## 5Empirical Findings

We report each failure mode in turn, isolating its two mechanisms through the interventions of §[4\.1](https://arxiv.org/html/2608.07943#S4.SS1)\.

TTLTLVVΔ\\DeltaV*By document domain*Academic paper36\.433\.142\.933\.8−\-9\.1Admin / Industry50\.056\.267\.254\.7−\-12\.5Brochure28\.632\.545\.540\.3−\-5\.2Financial report49\.152\.851\.938\.9−\-13\.0Guidebook35\.840\.051\.745\.0−\-6\.7Research / Intro22\.627\.447\.645\.3−\-2\.3Tutorial / Workshop14\.348\.273\.267\.9−\-5\.3*By evidence source*Chart18\.519\.145\.543\.3−\-2\.2Figure15\.523\.844\.840\.3−\-4\.5Generalized\-text28\.037\.350\.844\.1−\-6\.7Pure\-text39\.247\.855\.344\.3−\-11\.0Table40\.648\.448\.437\.8−\-10\.6All31\.938\.852\.545\.6−\-6\.9

Table 2:Representation accuracy by domain and evidence source \(Δ\\DeltaV = V−\-TLV\)\.### 5\.1Representation

OCR\-free systems read the page image directly\(Choet al\.,[2024](https://arxiv.org/html/2608.07943#bib.bib53); Tanakaet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib55)\), raising the possibility that vision reduces the reliance on text extraction\. We test this across the two mechanisms of representation failure, the modality ceiling and conversion fidelity\. Vision moves both, but replaces the text channel in neither\.

Vision is Necessary but Not Sufficient\.Our results support the importance of vision but not the complete replacement of text extraction \(Table[2](https://arxiv.org/html/2608.07943#S5.T2)\)\. Vision is necessary because page images carry graphical and spatial signals that text cannot express: a chart or figure encodes a signal no parser can render in text, so an image\-free pipeline leaves these questions unanswerable regardless of parser quality\. Vision alone, however, remains below the combined text–vision encoding, from 45\.6 for V to 52\.5 for TLV, so the page image loses signal the text channel preserves\. Layout\-dependent evidence shows this most sharply: even the page image, which displays layout directly, falls short of the combined encoding on these questions\. This gap persists across render resolutions \(Appendix[B](https://arxiv.org/html/2608.07943#A2)\), indicating a limit of the visual channel itself rather than an artefact of insufficient resolution\. Vision therefore raises the modality ceiling, while text retains complementary information that visual encoding alone loses\.

Vision Recovers Weak Text\.We further examine whether vision can compensate for errors introduced during text extraction \(Figure[2](https://arxiv.org/html/2608.07943#S5.F2)\)\. On digital pages, the three parsers differ by 15\.6 accuracy points when only their text outputs are used, from 26\.6 for the weakest to 42\.3 for the strongest, reflecting the downstream impact of conversion errors\(Zhanget al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib26)\)\. Adding the page image reduces this spread to 4\.2 points, with the weakest parser receiving the largest gain, since the image supplies the content the parser failed to preserve\. Vision also compensates for structure that flattening to raw text destroys: raw embedded text with the page image reaches 49\.7, nearly matching parser\-generated text with the image at 50\.2, so the image restores the reading order and table structure the parser existed to recover\. This compensation is not universal, because scanned pages lack a usable embedded\-text layer and still require parsing to provide any textual content at all\. Vision therefore reduces sensitivity to text\-conversion quality without eliminating the need for text extraction\.

![Refer to caption](https://arxiv.org/html/2608.07943v1/x2.png)Figure 2:Parser accuracy by text source, text\-only vs\. with the page image\.
### 5\.2Selection

Prior work emphasises the cost of irrelevant context, showing that it dilutes attention as the context grows\(Shiet al\.,[2023](https://arxiv.org/html/2608.07943#bib.bib7); Liuet al\.,[2024](https://arxiv.org/html/2608.07943#bib.bib133); Wanget al\.,[2024b](https://arxiv.org/html/2608.07943#bib.bib110)\)\. We find the two mechanisms of selection failure far from symmetric: omitting a required page bounds accuracy, while irrelevant pages cost little at realistic retrieval depths\.

Missing Evidence Bounds Accuracy\.Our results show withholding required evidence causes a large and consistent drop in multi\-page accuracy across encodings \(Figure[3](https://arxiv.org/html/2608.07943#S5.F3)\(a\)\)\. At TLV, accuracy falls from an oracle of 38\.6 to 18\.5 once a single gold page is removed, and to 12\.8 when only one page remains\. The per\-question transitions are strongly asymmetric, with far more answers transitioning to incorrect than recovering, so the loss is systematic rather than noise\. Which page is removed matters only mildly, since a higher\-ranked gold page costs more than a lower\-ranked one but the gap is small beside the collapse itself\. Retrieval rank is therefore a weak proxy for evidential necessity, as a lower\-ranked page can still supply a necessary link in the evidence chain\. The accuracy that survives likely reflects questions whose remaining page already carries enough for the reasoner to reconstruct the answer, rather than recovery of the missing evidence\. Evidence coverage thus sets an upper bound later stages cannot lift\.

![Refer to caption](https://arxiv.org/html/2608.07943v1/x3.png)Figure 3:Selection at the TLV encoding, with page rank given by ColQwen3\.Distractors Are Largely Tolerated\.Irrelevant pages, by contrast, cost little\. Adding distractor pages while retaining the required evidence leaves accuracy broadly stable, against the common concern that irrelevant context dilutes useful evidence \(Figure[3](https://arxiv.org/html/2608.07943#S5.F3)\(b\)\)\. At TLV, padding a gold page with three distractors lowers accuracy only slightly, from 64\.6 to 63\.1\. The cost grows when the required evidence itself spans more pages, reaching 5\.0 points for two\-gold\-page questions, but stays far below the loss from omitting one required page\. This stability is a net figure, since distractors turn a similar number of answers incorrect and correct\. Some turn correct, plausibly because an added page supplies context that aids the reasoner\. The losses, by contrast, fall on visual evidence, where an added page turns a chart answer incorrect more often than it turns one correct \(Appendix[C](https://arxiv.org/html/2608.07943#A3)\)\. Visual reasoning is already demanding for an 8B model, so extra pages plausibly disturb a process that was barely succeeding, which points to a reasoning limit rather than a selection one\. The tolerance is in any case not unlimited, since a sufficiently long context may eventually impair evidence use\(Liuet al\.,[2024](https://arxiv.org/html/2608.07943#bib.bib133); Wanget al\.,[2024b](https://arxiv.org/html/2608.07943#bib.bib110)\); within the range we test, distractor exposure remains far less damaging than incomplete coverage\.

Table 3:Accuracy by gold evidence\-page count across the representations\.
### 5\.3Reasoning

With representation and selection controlled, we ask whether the reasoner uses the evidence it holds correctly\. It fails in two ways: it cannot reliably combine evidence across pages, and it cannot judge when that evidence is sufficient, answering without support unless instructed to abstain, then refusing answerable questions when it is\.

Cross\-Page Integration Is a Reasoning Failure\.Our results show the reasoner failing on multi\-page questions even when every required page is supplied, which points to a reasoning failure rather than a limit of capacity\. At TLV, accuracy falls from 64\.6 on single\-page questions to 38\.6 as soon as the evidence spans two pages, and the pooled multi\-page deficit reaches 25\.9 points\. Two pages is far short of the context window, and §[5\.2](https://arxiv.org/html/2608.07943#S5.SS2)results show single\-page questions holding their accuracy even when three distractor pages are added \(Figure[3](https://arxiv.org/html/2608.07943#S5.F3)\(b\)\), so length alone does not explain the drop: the reasoner fails on the two\-page questions not because the input is long but because the answer must be assembled across pages\. The deficit also widens as the representation improves\. It grows from 11\.7 points on text to 25\.9 once the page image is added, even though the image supplies a richer, higher\-fidelity view of evidence that remains well within the context limit\. That a stronger representation enlarges the gap points to a visual reasoning failure specifically: the model integrates textual evidence across pages more reliably than it integrates evidence it must read from the images\.

Abstention Is a Partial Safeguard\.We probe calibration at TLV, where the oracle pages give the reasoner the highest fidelity representation, so in principle it holds the evidence each answerable question needs\. Even here, an abstention instruction leads it to refuse roughly a quarter of answerable questions \(Figure[4](https://arxiv.org/html/2608.07943#S5.F4)\)\. Some refusals are well\-judged: the per\-source breakdown \(Table[4](https://arxiv.org/html/2608.07943#S6.T4)\) shows that on the text\-only representations, the reasoner abstains most on chart and figure evidence, where the answer lies in a modality the encoding does not carry, returning a concise “not answerable” instead of a verbose non\-answer\. On unanswerable questions the same instruction has the opposite effect to weigh: where the model answers rather than abstains, it commits to a claim the evidence cannot support, which we read as hallucination\. The safeguard is thus imperfect at TLV, refusing some answerable questions while still answering some unanswerable ones\. Without it the reasoner rarely refuses at all, so abstention converts overconfidence into caution, some warranted and some not\.

![Refer to caption](https://arxiv.org/html/2608.07943v1/x4.png)Figure 4:Calibration trade\-off at TLV representation\. \(a\) Correct vs\. false refusal rates by prompt\. \(b\) Accuracy vs\. decode length by prompt\.

## 6Practical Discussion

### 6\.1Design Considerations

Each failure mode suggests a practical design choice for building a MP\-VRDU system\.

Representation: Match Encoding to the Document\.Representation is not a single default but a choice set by two things: the evidence a question draws on, and the encodings the document even permits\. The image is required when a question rests on graphical evidence, which no text channel conveys\. It also helps on layout\-dependent evidence: sufficient textual encoding should in principle carry the structure, but the image makes it more robust in practice \(§[5\.1](https://arxiv.org/html/2608.07943#S5.SS1)\)\. In addition, which text channel is available is fixed by the document\. The document fixes which text channel is available: scanned pages need an ocr parser, born\-digital pages already carry usable text\. TLV is therefore the safe default, the only full option on scanned pages, while TV is the cheaper choice on born\-digital pages, reaching a comparable encoding without the parsing pass\.

Selection: Favour Recall\.Selection failures are asymmetric in whether the reasoner can undo them\. Evidence never selected cannot be recovered downstream, and §[5\.2](https://arxiv.org/html/2608.07943#S5.SS2)shows a single missing page bounding accuracy well below the oracle\. Extra irrelevant evidence can be absorbed, since at reasonable retrieval depths the reasoner tolerates the distractors that wider retrieval brings\. With a real ranker the extra evidence need not even be irrelevant, and §[5\.2](https://arxiv.org/html/2608.07943#S5.SS2)finds a share of questions turning correct when added context supplies useful support\. Retrieval should therefore favour recall even at some cost to precision, since a missing page forecloses the answer while errors from an extra page can be recovered through better reasoning, and added evidence can occasionally help\.

Table 4:Abstention across encodings, with accuracy and incorrect rate at TLV, under the abstention instruction for answerable questions, by evidence source\.Reasoning: Prompting, Scale, and Training\.The multi\-page integration failure responds to changes in reasoning alone, confirming it is a reasoning problem and one addressable without new data\. At TLV, chain\-of\-thought prompting raises multi\-page accuracy from 38\.6 to 44\.9, and a 32B reasoner raises it to 46\.4 \(Table[3](https://arxiv.org/html/2608.07943#S5.T3)\); both act at inference time, and CoT in particular narrows the single\-to\-multi gap rather than lifting all questions alike\. An abstention instruction similarly reduces the calibration failure at inference time, at the coverage cost weighed in §[6\.2](https://arxiv.org/html/2608.07943#S6.SS2)\. Training is the less\-explored route: single\-page document reading is already well served by continued pretraining and fine\-tuningDinget al\.\([2025](https://arxiv.org/html/2608.07943#bib.bib124)\), whereas the multi\-page skills these deficits reflect, cross\-page integration in particular, are rarely a training target\(Xuet al\.,[2026](https://arxiv.org/html/2608.07943#bib.bib15)\)\. Building supervision around multi\-page tasks is a plausible direction for what prompting and scale only partly close\.

### 6\.2Deployment Considerations

The findings also bear on how to spend a fixed compute budget, where the strongest representation and the largest model are rarely the right default\.

![Refer to caption](https://arxiv.org/html/2608.07943v1/figures/routing.png)Figure 5:Average input tokens and accuracy by representation and routing policy\.Route by Document Class\.Running text and vision on every question is the safe default but an expensive one, since much of its cost buys accuracy only where the question needs the image\. The representation a question needs is tied to its evidence source \(§[5\.1](https://arxiv.org/html/2608.07943#S5.SS1)\): graphical evidence needs the image, while plain\-text and simple\-layout questions are served well by text alone\. Two signals support routing on this\. Where the evidence source is known, representation is routed to it directly; where it is not, the document’s domain serves as a proxy, assigned by hand or by a classifier\. The saving is twofold\. Routing representation by document class reaches 45\.6, matching vision alone, at less than half the input tokens of the full encoding \(2,410 against 5,049\) and fewer than vision alone requires \(3,442, Figure[5](https://arxiv.org/html/2608.07943#S6.F5)\)\.

Trade Precision for Parameters\.A fixed memory budget can be spent on more parameters or on higher precision, and the two trade differently\. Reasoner scale buys the most accuracy but costs the most, since a larger reasoner raises the prefill cost that dominates a document pipeline’s latency\. Quantization lowers precision for a much smaller footprint, and although low\-bit weights are known to degrade language and reasoning quality\(Wanget al\.,[2024a](https://arxiv.org/html/2608.07943#bib.bib29); Yuanet al\.,[2025](https://arxiv.org/html/2608.07943#bib.bib30)\), we observe almost no accuracy loss on document answering: four\-bit weights cost 0\.6 points at the combined encoding for a 61% smaller footprint \(Table[5](https://arxiv.org/html/2608.07943#S6.T5)\)\. A plausible reason is that document answering leans more on locating and reading evidence than on the open\-ended reasoning most sensitive to quantization, so the task tolerates the lost precision\. The two compose in one direction at a matched budget, since a quantized larger model performs better than a smaller full\-precision one at every encoding, so precision is the first thing to trade for parameters\.

Abstention by Error Cost\.Abstention is worth enabling where a wrong answer costs more than a missing one \(Figure[4](https://arxiv.org/html/2608.07943#S5.F4)\)\. The instruction raises correct refusal on unanswerable questions to about three\-quarters, and §[5\.3](https://arxiv.org/html/2608.07943#S5.SS3)shows much of the refusal it adds on answerable questions to be well\-judged, the reasoner declining where the encoding cannot carry the evidence rather than guessing\. A smaller share is genuine lost coverage, so a deployment that cannot afford any refusal of answerable questions should still weigh it, but the trade is more favourable than the raw refusal rate suggests\. Where abstention is enabled, the visual encoding reduces the residual false\-refusal cost, and chain\-of\-thought, which is more expensive to decode, is best triggered per question rather than run by default\.

Table 5:Reasoner accuracy across precision, scale, matched budget, family, and a reasoning variant\.

## 7Conclusion and Future Work

We attribute failure in MP\-VRDU systems to three stages: representation, selection, and reasoning\. Isolating each within a controlled single\-pass pipeline, we find that vision is necessary but insufficient, selection is sharply asymmetric, and reasoners struggle to integrate evidence across pages even when it is fully available\. These findings establish a diagnostic basis for more targeted investigation\. At the representation stage, future work can test whether stronger reasoners better exploit structure recovered by parsers\. The selection results motivate closer analysis of why irrelevant visual evidence is more harmful than textual evidence, and whether this pattern persists with learned rankers\. Turning to reasoning, the cross\-page deficit raises questions about the mechanisms of evidence integration and the distinction between over\-refusal and appropriate abstention\. Addressing these directions across richer corpora, retrieval settings, and broader model families may yield substantial gains for MP\-VRDU\.

## Limitations

Our analysis is based the MMLongBench\-Doc dataset and primarily evaluates models from the Qwen3 family, which may limit the generality of the observed effect sizes\. Attribution is also conducted at page level, following the annotation granularity of MMLongBench\-Doc, and correctness is assessed using a single LLM judge\. Extending the study to additional datasets, model families, finer\-grained evidence annotations, and multiple judges represents a natural next step for testing the robustness and broader applicability of the framework\.

## References

- Qwen3\-VL technical report\.arXiv preprint arXiv:2511\.21631\.External Links:[Link](https://arxiv.org/abs/2511.21631)Cited by:[§A\.1](https://arxiv.org/html/2608.07943#A1.SS1.p1.2),[§4\.2](https://arxiv.org/html/2608.07943#S4.SS2.p4.1)\.
- J\. Cho, D\. Mahata, O\. Irsoy, Y\. He, and M\. Bansal \(2024\)M3docrag: multi\-modal retrieval is what you need for multi\-page multi\-document understanding\.arXiv preprint arXiv:2411\.04952\.Cited by:[§5\.1](https://arxiv.org/html/2608.07943#S5.SS1.p1.1)\.
- C\. Cui, T\. Sun, S\. Liang, T\. Gao, Z\. Zhang, J\. Liu, X\. Wang, C\. Zhou, H\. Liu, M\. Lin, Y\. Zhang, Y\. Zhang, H\. Zheng, J\. Zhang, J\. Zhang, Y\. Liu, D\. Yu, and Y\. Ma \(2025\)PaddleOCR\-VL: boosting multilingual document parsing via a 0\.9B ultra\-compact vision\-language model\.arXiv preprint arXiv:2510\.14528\.External Links:[Link](https://arxiv.org/abs/2510.14528)Cited by:[§A\.3](https://arxiv.org/html/2608.07943#A1.SS3.p1.1),[§4\.2](https://arxiv.org/html/2608.07943#S4.SS2.p2.1)\.
- Y\. Ding, S\. Luo, Y\. Dai, Y\. Jiang, Z\. Li, G\. Martin, and Y\. Peng \(2025\)A survey on mllm\-based visually rich document understanding: methods, challenges, and emerging trends\.arXiv preprint arXiv:2507\.09861\.Cited by:[§6\.1](https://arxiv.org/html/2608.07943#S6.SS1.p4.1)\.
- J\. Guo, C\. Huang, H\. Tao, Z\. An, G\. Ke, and X\. Fang \(2026\)End\-to\-end document understanding via chain\-of\-reading\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1)\.
- S\. Han, P\. Xia, R\. Zhang, T\. Sun, Y\. Li, H\. Zhu, and H\. Yao \(2025\)Mdocagent: a multi\-modal multi\-agent framework for document understanding\.arXiv preprint arXiv:2503\.13964\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1),[§2](https://arxiv.org/html/2608.07943#S2.p1.1)\.
- T\. Hannan, D\. Mallios, P\. Pathak, F\. Sardari, T\. Seidl, G\. Bertasius, M\. Fayyaz, and S\. Sengupta \(2025\)DocSLM: a small vision\-language model for long multimodal document understanding\.arXiv preprint arXiv:2511\.11313\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1)\.
- D\. Li, Z\. Sun, X\. Hu, Z\. Liu, Z\. Chen, B\. Hu, A\. Wu, and M\. Zhang \(2023\)A survey of large language models attribution\.arXiv preprint arXiv:2311\.03731\.External Links:2311\.03731Cited by:[§3](https://arxiv.org/html/2608.07943#S3.p4.1)\.
- Z\. Li, Y\. Li, L\. Kang, D\. Karatzas, and W\. Ma \(2025\)AVIR: adaptive visual in\-document retrieval for efficient multi\-page document question answering\.InProceedings of the 7th ACM International Conference on Multimedia in Asia,pp\. 1–7\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1)\.
- K\. Liu, Z\. Chen, M\. Li, J\. Tang, D\. Yang, and L\. Zhang \(2025\)Resolving evidence sparsity: agentic context engineering for long\-document understanding\.arXiv preprint arXiv:2511\.22850\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1)\.
- N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. Liang \(2024\)Lost in the middle: how language models use long contexts\.Transactions of the association for computational linguistics12,pp\. 157–173\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1),[§3](https://arxiv.org/html/2608.07943#S3.p4.1),[§5\.2](https://arxiv.org/html/2608.07943#S5.SS2.p1.1),[§5\.2](https://arxiv.org/html/2608.07943#S5.SS2.p3.1)\.
- Y\. Ma, Y\. Zang, L\. Chen, M\. Chen, Y\. Jiao, X\. Li, X\. Lu, Z\. Liu, Y\. Ma, X\. Dong,et al\.\(2024\)Mmlongbench\-doc: benchmarking long\-context document understanding with visualizations\.Advances in Neural Information Processing Systems37,pp\. 95963–96010\.Cited by:[§A\.7](https://arxiv.org/html/2608.07943#A1.SS7.p1.1),[§4\.3](https://arxiv.org/html/2608.07943#S4.SS3.p1.1)\.
- J\. Niu, Z\. Liu, Z\. Gu, B\. Wang, L\. Ouyang, Z\. Zhao, T\. Chu, T\. He, F\. Wu, Q\. Zhang, Z\. Jin,et al\.\(2025\)MinerU2\.5: a decoupled vision\-language model for efficient high\-resolution document parsing\.arXiv preprint arXiv:2509\.22186\.External Links:[Link](https://arxiv.org/abs/2509.22186)Cited by:[§A\.3](https://arxiv.org/html/2608.07943#A1.SS3.p1.1),[§4\.2](https://arxiv.org/html/2608.07943#S4.SS2.p2.1)\.
- X\. Peng, C\. Qin, Z\. Chen, R\. Xu, C\. Xiong, and C\. Wu \(2025\)UniDoc\-Bench: a unified benchmark for document\-centric multimodal RAG\.arXiv preprint arXiv:2510\.03663\.External Links:2510\.03663Cited by:[§2](https://arxiv.org/html/2608.07943#S2.p1.1)\.
- F\. Shi, X\. Chen, K\. Misra, N\. Scales, D\. Dohan, E\. H\. Chi, N\. Schärli, and D\. Zhou \(2023\)Large language models can be easily distracted by irrelevant context\.InInternational conference on machine learning,pp\. 31210–31227\.Cited by:[§5\.2](https://arxiv.org/html/2608.07943#S5.SS2.p1.1)\.
- M\. Suri, P\. Mathur, K\. Goswami, R\. A\. Rossi, F\. Dernoncourt, and D\. Manocha \(2025\)VisDoM: multi\-document qa with visually rich elements using multimodal retrieval\-augmented generation\.InProceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1),[§2](https://arxiv.org/html/2608.07943#S2.p1.1)\.
- R\. Tanaka, T\. Iki, T\. Hasegawa, K\. Nishida, K\. Saito, and J\. Suzuki \(2025\)Vdocrag: retrieval\-augmented generation over visually\-rich documents\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 24827–24837\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1),[§2](https://arxiv.org/html/2608.07943#S2.p1.1),[§5\.1](https://arxiv.org/html/2608.07943#S5.SS1.p1.1)\.
- C\. Wang, Z\. Wang, X\. Xu, Y\. Tang, J\. Zhou, and J\. Lu \(2024a\)Q\-VLM: post\-training quantization for large vision\-language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:2410\.08119Cited by:[§6\.2](https://arxiv.org/html/2608.07943#S6.SS2.p3.1)\.
- W\. Wang, S\. Zhang, Y\. Ren, Y\. Duan, T\. Li, S\. Liu, M\. Hu, Z\. Chen, K\. Zhang, L\. Lu,et al\.\(2024b\)Needle in a multimodal haystack\.Advances in Neural Information Processing Systems37,pp\. 20540–20565\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1),[§5\.2](https://arxiv.org/html/2608.07943#S5.SS2.p1.1),[§5\.2](https://arxiv.org/html/2608.07943#S5.SS2.p3.1)\.
- S\. Whitehead, S\. Petryk, V\. Shakib, J\. Gonzalez, T\. Darrell, A\. Rohrbach, and M\. Rohrbach \(2022\)Reliable visual question answering: abstain rather than answer incorrectly\.InComputer Vision – ECCV 2022,Lecture Notes in Computer Science, Vol\.13696,pp\. 148–166\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-20059-5%5F9),2204\.13631Cited by:[§3](https://arxiv.org/html/2608.07943#S3.p4.1)\.
- L\. Xu, Y\. Ding, Z\. Xu, S\. Luo, D\. Liu, and W\. Liu \(2026\)Managing evidence at document scale: a survey of multi\-page visually rich document understanding\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p1.1),[§6\.1](https://arxiv.org/html/2608.07943#S6.SS1.p4.1)\.
- H\. Yan, Y\. Liu, X\. Liu, Y\. Zhang, M\. Liao, J\. Wu, W\. Chen, and X\. Bai \(2025\)DocSeeker: structured visual reasoning with evidence grounding for long document understanding\.arXiv preprint\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1),[§2](https://arxiv.org/html/2608.07943#S2.p1.1)\.
- Y\. Yin, H\. Liu, YY, Q\. Xie, C\. Liu, S\. Yang, S\. Wang, Z\. Liu, H\. Zou, J\. Chen, S\. Wei, J\. Wu, M\. Huang, Z\. Wu, G\. Wang, T\. Du, and L\. Jia \(2026\)Unlimited ocr works\.External Links:2606\.23050,[Link](https://arxiv.org/abs/2606.23050)Cited by:[§4\.2](https://arxiv.org/html/2608.07943#S4.SS2.p2.1)\.
- Z\. Yuan, W\. Sun, Y\. Liu, H\. Zhou, R\. Zhou, Y\. Li, Z\. Zhang, W\. Song, Y\. Huang, H\. Jia, K\. Murugesan, Y\. Wang, L\. He, J\. Gao, L\. Sun, and Y\. Ye \(2025\)EfficientLLM: efficiency in large language models – evaluation on architecture pretraining, fine\-tuning, and bit\-width quantization\.arXiv preprint arXiv:2505\.13840\.External Links:2505\.13840Cited by:[§6\.2](https://arxiv.org/html/2608.07943#S6.SS2.p3.1)\.
- Y\. Zarrinkia, V\. Srinivasan, and A\. Thomo \(2026\)The reasoning bottleneck in Graph\-RAG: structured prompting and context compression for multi\-hop QA\.arXiv preprint arXiv:2603\.14045\.External Links:2603\.14045Cited by:[§2](https://arxiv.org/html/2608.07943#S2.p1.1)\.
- J\. Zhang, Q\. Zhang, B\. Wang, L\. Ouyang, Z\. Wen, Y\. Li, K\. Chow, C\. He, and W\. Zhang \(2025\)OCR hinders RAG: evaluating the cascading impact of OCR on retrieval\-augmented generation\.InIEEE/CVF International Conference on Computer Vision \(ICCV\),External Links:2412\.02592Cited by:[§2](https://arxiv.org/html/2608.07943#S2.p1.1),[§3](https://arxiv.org/html/2608.07943#S3.p2.1),[§5\.1](https://arxiv.org/html/2608.07943#S5.SS1.p3.1)\.
- Y\. Zheng, P\. Fu, H\. Li, Z\. Wang, Y\. Zhang, W\. Ruan, X\. Zhang, Z\. Wei, Z\. Luo, J\. Luan,et al\.\(2026\)Doc\-v\*: coarse\-to\-fine interactive visual reasoning for multi\-page document vqa\.arXiv preprint arXiv:2604\.13731\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1)\.
- D\. Zhu, R\. Meng, J\. Chen, S\. Li, T\. Pfister, and J\. Yoon \(2025a\)Doclens: a tool\-augmented multi\-agent framework for long visual document understanding\.arXiv preprint arXiv:2511\.11552\.Cited by:[§1](https://arxiv.org/html/2608.07943#S1.p2.1)\.
- J\. Zhu, W\. Wang, Z\. Chen, Z\. Liu, S\. Ye, L\. Gu, Y\. Duan, H\. Tian, W\. Su, J\. Shao,et al\.\(2025b\)InternVL3: exploring advanced training and test\-time recipes for open\-source multimodal models\.arXiv preprint arXiv:2504\.10479\.Cited by:[§A\.1](https://arxiv.org/html/2608.07943#A1.SS1.p1.2),[§4\.2](https://arxiv.org/html/2608.07943#S4.SS2.p4.1)\.

## Appendix AFull Pipeline Configuration

This appendix gives the exact models, prompts, decoding, retrieval, dataset, and scoring settings the runs used\. Section[4\.2](https://arxiv.org/html/2608.07943#S4.SS2)states the main configuration; everything here is the full detail behind it\.

### A\.1Reasoner

The baseline reasoner is Qwen3\-VL\-8B\-InstructBaiet al\.\([2025](https://arxiv.org/html/2608.07943#bib.bib18)\), loaded in bfloat16\. Decoding is greedy \(do\_sample=False; no temperature, top\-pp, top\-kk, or repetition penalty\), with a decode budget of 256 new tokens for the terse\-answer modes and 2048 for the reasoning\-bearing modes, which must emit both reasoning and a final answer\. The full text context is passed with no input\-token cap\. The scale sweep runs Qwen3\-VL at 2B, 4B, 8B, and 32B; the cross\-family comparison runs InternVL3\-8BZhuet al\.\([2025b](https://arxiv.org/html/2608.07943#bib.bib4)\)at the same 8B scale\. Both backends share one prompt template\. Quantization comparisons apply 8\-bit and 4\-bit bitsandbytes quantization to the 8B reasoner, each cached as its own condition\.

### A\.2Representation Ladder

The representation is built at render DPI 200 \(PyMuPDF,zoom=\{\}=\{\}DPI/72/72\) and takes the cost\-ordered forms in Table[6](https://arxiv.org/html/2608.07943#A1.T6)\. The forms are not cumulative: parser markdown and the page image are competing representations of one page, so TL and V are alternative channels rather than additive layers\. The four canonical representations are T, TL, TLV, and V; TV and TLVi are additional representations a run requests explicitly\. TV is the parser\-free counterpart of TLV, pairing the cheap embedded text with the page image and skipping the parser pass, so the TLV\-minus\-TV contrast isolates what the parser layer adds once the model can already see the page\. TLVi is TLV’s interleaved variant, the same text and images at the same cost but ordered per page \(page text, that page’s image, next page\) rather than one merged text block followed by every image\.

Table 6:The representation ladder\. “Embedded text” is the PyMuPDF text layer; “parser markdown” is a parser’s structural output\.
### A\.3Parsers

The baseline parser is PaddleOCR\-VL \(0\.9B\)Cuiet al\.\([2025](https://arxiv.org/html/2608.07943#bib.bib16)\), run through thepaddleocr3\.7 pipeline \(PP\-DocLayoutV2 layout, PP\-OCRv5 detection and recognition, orientation and unwarping\)\. The parser ablation adds MinerU 2\.5 \(1\.2B\)Niuet al\.\([2025](https://arxiv.org/html/2608.07943#bib.bib17)\), a Qwen2\-VL\-based parser prompted to convert the page to Markdown, and Unlimited\-OCR, a DeepSeek\-OCR\-style full\-page extractor\. The T representation bypasses all three and reads the PyMuPDF embedded text layer directly, which is what makes it the acquisition floor rather than a fourth parser\. T and V never invoke a parser; only TL and TLV do\.

### A\.4Visual Resolution

Each page image is capped at a per\-page pixel budget equal totokens/page×28×28\\text\{tokens/page\}\\times 28\\times 28, since the reasoner packs one vision token per28×2828\\times 28patch\. The three presets are low \(400400tokens,313,600313\{,\}600px\), med \(640640tokens,501,760501\{,\}760px\), and high \(960960tokens,752,640752\{,\}640px\)\. Pages are rasterised at 200 DPI, then downsampled to the budget\. Every table uses med except the resolution sweep, which runs low, med, and high on the TLV and V representations\.

### A\.5Retrieval

The retrieval unit is the page; there is no sub\-page chunking, so a retrieved unit and an annotated evidence unit are the same object\. Retrieval depth isk∈\{1,3,5,7,10\}k\\in\\\{1,3,5,7,10\\\}\. Lexical retrieval is BM25 \(one index per document\)\. Dense\-text retrieval uses BGE\-M3 and Qwen3\-Embedding\-4B \(sequence cap 4096, encode batch 1\)\. Late\-interaction visual retrieval uses ColModernVBERT, ColQwen2\.5, and ColQwen3, embedding page images rendered at 200 DPI\. Joint retrieval is the deduplicated union of a matched text and vision arm, scored atk∈\{1,3\}k\\in\\\{1,3\\\}\. The generation stage feeds one text arm, one vision arm, and their joint union to the reasoner at the TLV and V representations\.

### A\.6Prompt Modes

The reasoner prompt is a fixed header and body, with an optional instruction preamble that defines the mode:

> You are answering a question about a document\. \{instruction\} Question: \{question\} Document evidence: \{context\} Answer:

The answerable ladder runs with no instruction \(none: header and question only\)\. The faithfulness sweeps run six modes over the same questions, composed from named fragments so each mode isolates one mechanism:

> none\(empty\)\. groundedUse only the provided document evidence and keep the answer concise\. abstaingrounded, plusIf the evidence does not contain the answer, answer exactly: Not answerable\. abstain\_balancedabstain, plusIf the evidence does contain the answer, give it: do not decline a question the evidence supports\. cotgrounded, plusThink step by step before answering, with the final answer on a line beginningAnswer:\. extract\_cotgrounded, plus a verbatim\-evidence extraction step, then reasoning constrained to the extracted evidence, then theAnswer:line\.

For thecotandextract\_cotmodes the text after the lastAnswer:marker is taken as the final answer before judging\.

### A\.7Dataset

We use MMLongBench\-DocMaet al\.\([2024](https://arxiv.org/html/2608.07943#bib.bib104)\): 1091 questions over 135 documents, split into 847 answerable and 244 unanswerable \(a question is unanswerable when its gold answer is exactly “Not answerable”\)\. Each question carries a native document type \(Research report 293, Academic paper 204, Guidebook 156, Tutorial/Workshop 139, Financial report 117, Brochure 101, Administration/Industry file 81\), a typed evidence source, and an evidence\-hop label derived from its gold\-page count \(Table[1](https://arxiv.org/html/2608.07943#S4.T1)\)\. Answer formats are Int 290, Str 250, None 244, Float 160, and List 147\. Evidence pages are annotated per document, converted from 1\-based to 0\-based indices, de\-duplicated, and used directly as the oracle set\. A per\-document born\-digital or scanned label is auto\-detected \(a page counts as text with at least 20 extracted characters; a document is scanned when none of its first sampled pages carries a text layer\), giving 672 digital and 189 scanned questions with the remainder unlabelled\. A hand\-labelled modality bin \(text\-dominant, mixed\-modality, visual\-dominant\) is joined per document where available\.

### A\.8Judging

Answers are scored by an LLM judge rather than exact match, since a correct answer may differ from the gold string in units or formatting\. Two judges of different families, GPT\-4o\-mini and Gemini\-2\.5\-flash, run at temperature 0 with JSON\-only output\. The judge sees the question, the gold answer, the unanswerable flag, and the model answer, and returns one of correct, incorrect, or abstained; for an unanswerable question, abstaining is scored correct\. The rubric marks an answer correct when it is semantically equivalent to the gold answer\. Abstention is additionally detected by matching a fixed set of refusal surface forms against the casefolded answer \(“not answerable”, “cannot be answered”, “insufficient information”, “not mentioned”, “not provided”, and related phrasings\)\.

### A\.9Evaluation

Accuracy is judge\-scored correctness\. Confidence intervals are 95% document\-level bootstrap intervals, resampling documents \(not questions\) over 1000 resamples with the2\.52\.5and97\.597\.5percentiles\. Resampling documents rather than questions makes the interval reflect document\-level variation, since questions from one document are not independent\. The sufficiency comparisons use a pre\-registered margin of 3 accuracy points\. Every reported run uses the whole pool rather than a sample, so the bootstrap resamples the full 847 answerable or 244 unanswerable questions and no sampling stage enters the interval\.

Ladder @ med \(%\)TLVVTTLTLVVlowhighlowhighΔ\\DeltaV*By document domain*Academic paper36\.433\.142\.933\.837\.046\.819\.538\.3−\-9\.1Admin / Industry50\.056\.267\.254\.764\.170\.342\.264\.1−\-12\.5Brochure28\.632\.545\.540\.340\.348\.139\.049\.4−\-5\.2Financial report49\.152\.851\.938\.954\.653\.721\.346\.3−\-13\.0Guidebook35\.840\.051\.745\.048\.351\.738\.347\.5−\-6\.7Research / Intro22\.627\.447\.645\.342\.950\.038\.247\.6−\-2\.3Tutorial / Workshop14\.348\.273\.267\.971\.474\.174\.170\.5−\-5\.3*By evidence source*Chart18\.519\.145\.543\.338\.248\.334\.348\.3−\-2\.2Figure15\.523\.844\.840\.339\.748\.338\.343\.8−\-4\.5Generalized\-text28\.037\.350\.844\.146\.654\.247\.553\.4−\-6\.7Pure\-text39\.247\.855\.344\.352\.955\.038\.851\.5−\-11\.0Table40\.648\.448\.437\.848\.848\.825\.342\.4−\-10\.6All31\.938\.852\.545\.649\.254\.737\.850\.2−\-6\.9

Table 7:Representation ladder by domain and evidence source at medium resolution, with the TLV and V rungs swept across low and high resolution\. Teal deepens with accuracy; bold marks each row’s winning rung; the warm strip deepens with the text channel’s recovery over vision alone \(Δ\\DeltaV = V−\-TLV\)\.Table 8:Parser fidelity by evidence source and scan status, categorised by parser\. Teal deepens with accuracy and with the vision\-step recovery rate \(W→\\rightarrowR\); warm deepens with the vision\-step regression rate \(R→\\rightarrowW\)\. Text\-channel accuracy and its vision\-step verdict transitions, split digital vs scanned, for each parser across the five evidence sources\.

## Appendix BFull Representation Details

### B\.1Domain, Source, Resolution

Table[7](https://arxiv.org/html/2608.07943#A1.T7)resolves the ladder by document domain and by evidence source, and adds the resolution sweep the main text refers to but does not show\. Two patterns hold across the strata\. The combined encoding wins on six of the seven domains, and the one exception is instructive: on financial reports parser text alone is the best rung \(52\.852\.8against51\.951\.9\), and it is the domain where the image adds least, since a filing carries its content in tables and running text that the parser already recovers\. Read by evidence source, the image earns its accuracy on the graphical sources, where the text channel recovers least over vision alone \(Chart and Figure show a text\-recovery margin of only2\.22\.2and4\.54\.5points\), and the text channel earns its accuracy on the textual ones \(Pure\-text and Table recover11\.011\.0and10\.610\.6points\)\. The resolution wings confirm that this is a property of the visual channel rather than of its resolution: raising the render resolution lifts the pixel\-limited sources most, moving the image\-only rung on Table from25\.325\.3at low resolution to42\.442\.4at high and on Chart from34\.334\.3to48\.348\.3, while text\-carried sources such as Pure\-text barely move\. Higher resolution therefore buys legibility on dense and graphical content, not a uniform gain, and even at the highest resolution the image\-only rung stays below the combined encoding, which places the shortfall in the channel and not the pixels\.

### B\.2Parser Fidelity

Table[8](https://arxiv.org/html/2608.07943#A1.T8)extends the parser comparison to every evidence source and to scanned pages, and reports the paired verdict transitions across the vision step\. The convergence the main text reports pooled holds within each source: the three layout parsers and the raw embedded\-text channel spread widely on text alone and close once the page image is added, and the recovery direction dominates the regression direction at every parser, with the vision step flipping far more wrong answers to right \(W→\\rightarrowR\) than right to wrong \(R→\\rightarrowW\)\. Two source\-level details are worth noting\. The recovery concentrates on the graphical sources and is smallest on tabular evidence, so the image compensates most exactly where the parser preserves least: adding the page image repairs20\.820\.8and21\.221\.2percent of previously wrong Chart and Figure answers under the baseline parser, against4\.94\.9percent on Table, where the parser was already reading the content correctly and the image has little left to add\. Scanned pages behave differently by parser: the three OCR parsers recover usable text from the rendered page and hold accuracy, while PyMuPDF, which only reads an embedded text layer, collapses to the token floor on scans \(4\.04\.0against40\.640\.6on digital pages\) and depends almost entirely on the image to answer, repairing54\.254\.2percent of its scanned Chart answers once the page is visible\. The scanned per\-source cells are thin and should be read for direction rather than precise value\.

Table 9:Page\-level retrieval precision, recall, and F1 across depth for every retriever\. Vision leads text at every depth, joint retrieval raises recall, and recall climbs withkkas precision falls\.Table 10:Selection under constructed page sets\. Withholding or isolating one gold page collapses accuracy \(left\); adding distractors to a fixed gold set does not move it \(right\)\.Table 11:Withholding gold pages: verdict flips by evidence source, at the combined encoding with colqwen3 ranking\. Red deepens with what the change breaks, green with what it repairs, on scales shared with Table[12](https://arxiv.org/html/2608.07943#A2.T12)\.\+1\+1distractor\+2\+2distractors\+3\+3distractorsEvidence sourceR→\\rightarrowWW→\\rightarrowRR→\\rightarrowWW→\\rightarrowRR→\\rightarrowWW→\\rightarrowRnn*1 gold page*Chart9\.37\.210\.39\.317\.58\.297Figure7\.46\.811\.110\.59\.39\.9162Generalized\-text10\.75\.48\.97\.17\.13\.656Pure\-text7\.26\.55\.95\.97\.25\.9153Table8\.74\.85\.83\.86\.76\.7104All sources7\.87\.08\.28\.09\.78\.2474*2 gold pages*Chart5\.28\.66\.95\.210\.38\.658Figure4\.510\.63\.06\.17\.63\.066Generalized\-text5\.95\.917\.60\.023\.55\.934Pure\-text7\.63\.310\.94\.310\.93\.392Table8\.37\.37\.35\.29\.44\.296All sources7\.57\.59\.16\.211\.26\.2241*3 gold pages*\(thin at every source; omitted from Figure[3](https://arxiv.org/html/2608.07943#S5.F3)\)Chart†0\.00\.00\.00\.00\.00\.03Figure†15\.85\.326\.35\.321\.10\.019Generalized\-text†0\.020\.00\.020\.00\.020\.05Pure\-text†11\.811\.85\.95\.911\.811\.817Table†0\.014\.30\.00\.00\.014\.37All sources†10\.07\.512\.55\.012\.55\.040

Table 12:Adding distractor pages: verdict flips by gold\-page count and evidence source, at the combined encoding with colqwen3 ranking\. Shading is on the same scales as Table[11](https://arxiv.org/html/2608.07943#A2.T11): the red that dominates there is faint here, and the green nearly matches it, so added pages cost little on net\.

## Appendix CFull Selection Details

### C\.1Retrieval Sweep

Table[9](https://arxiv.org/html/2608.07943#A2.T9)gives the full page\-level retrieval sweep behind the recall claim in Section[6\.1](https://arxiv.org/html/2608.07943#S6.SS1)\. Across every depth the vision retrievers lead the text ones, and the best single\-page operating point isk=1k\{=\}1for every retriever, since precision falls faster than recall rises as the depth grows\. The recall\-tuned reading the design discussion recommends is visible in the curve: the strongest vision retriever climbs from54\.154\.1recall atk=1k\{=\}1to87\.487\.4atk=10k\{=\}10, buying coverage the reasoner can use at a precision cost the reasoner tolerates\. Joint text\-and\-vision retrieval raises recall further at shallow depth, reaching58\.758\.7atk=1k\{=\}1where the best vision arm alone reaches54\.154\.1, which is why it is the better choice when the pipeline reads only a few pages, but its advantage narrows as the vision retriever catches up with depth\. The practical implication for selection is that a recall\-tuned vision or joint retriever reaches full coverage of the required pages within the distractor range the reasoner absorbs, so retrieval and reasoning compose rather than compete\.

### C\.2Rankers and Encodings

Table[10](https://arxiv.org/html/2608.07943#A2.T10)gives the full page\-set sweep, over both rankers and every rung\. The withholding block confirms that the collapse is indifferent to which page is removed and to how it is ranked: dropping the highest\-ranked gold page, dropping the lowest\-ranked one, and isolating a single gold page all fall to within a few points of each other at every rung, so retrieval rank is not a proxy for evidential necessity\. Isolating a single page falls furthest, since it removes the most evidence, which is consistent with the loss scaling with how much required evidence is withheld rather than with its rank\. The distractor block confirms the other half of the asymmetry: adding one, two, or three non\-evidence pages to a fixed gold set leaves accuracy at its oracle level throughout, for both a one\-page and a two\-page gold set, and the small movements that do appear are within the confidence intervals\. The two blocks are not comparable to each other, since they differ in both evidence and length, but read down each block the message is the one the main text draws\.

### C\.3Verdict Transitions

Tables[11](https://arxiv.org/html/2608.07943#A2.T11)and[12](https://arxiv.org/html/2608.07943#A2.T12)report the per\-question verdict transitions behind the asymmetry, resolved by evidence source, and give both directions of the flip: right→\\rightarrowwrong is the share of oracle\-correct answers a change to the page set breaks, and wrong→\\rightarrowright the share it repairs\. Read together the two directions separate the mechanisms in a way neither does alone\. Withholding a gold page breaks a fifth to a third of previously correct answers at every source, from19\.219\.2on Chart to31\.331\.3on Pure\-text, and repairs at most10\.010\.0in return, so the change is a near one\-way loss rather than a churn\. Adding distractor pages breaks under a tenth and repairs almost exactly as many as it breaks: pooled over sources, three added pages cost9\.79\.7and recover8\.28\.2at one gold page, and11\.411\.4against6\.06\.0across two and three\. That the two directions nearly cancel is the stronger statement, since a raw regression rate of9\.79\.7read alone would suggest a real cost, and it is only the matching repair rate that shows the reasoner is churning rather than losing\. The source breakdown adds one detail the pooled numbers hide: the omission cost is slightly higher on the text\-carried sources than on the graphical ones, consistent with textual multi\-page questions depending more often on one specific withheld page, though the gap is small\.

TTLTLVVStratumSMΔ\\DeltaSMΔ\\DeltaSMΔ\\DeltaSMΔ\\Deltann*By evidence source*\(overlapping\)Chart20\.416\.5−\-4\.019\.419\.0−\-0\.456\.132\.9−\-23\.254\.130\.4−\-23\.7158Figure19\.310\.1−\-9\.225\.321\.8−\-3\.549\.439\.5−\-9\.945\.833\.6−\-12\.2168Generalized\-text30\.426\.7−\-3\.744\.631\.7−\-13\.066\.138\.3−\-27\.755\.435\.0−\-20\.4116Pure\-text46\.831\.3−\-15\.456\.538\.8−\-17\.767\.542\.5−\-25\.053\.934\.3−\-19\.6274Table47\.135\.5−\-11\.766\.332\.7−\-33\.666\.331\.8−\-34\.551\.925\.5−\-26\.5190*By domain*\(disjoint\)Academic paper34\.539\.7\+\+5\.229\.838\.2\+\+8\.539\.347\.1\+\+7\.831\.038\.2\+\+7\.3152Admin/Industry58\.540\.0−\-18\.568\.340\.0−\-28\.387\.835\.0−\-52\.868\.335\.0−\-33\.361Brochure24\.535\.7\+\+11\.230\.635\.7\+\+5\.151\.035\.7−\-15\.344\.932\.1−\-12\.877Financial report60\.334\.1−\-26\.268\.331\.8−\-36\.474\.620\.5−\-54\.152\.420\.5−\-31\.9107Guidebook42\.525\.5−\-16\.946\.629\.8−\-16\.860\.338\.3−\-22\.053\.431\.9−\-21\.5120Research report28\.017\.1−\-10\.934\.620\.0−\-14\.664\.530\.5−\-34\.061\.728\.6−\-33\.1212Tutorial/Workshop23\.80\.0−\-23\.854\.041\.3−\-12\.781\.065\.2−\-15\.781\.052\.2−\-28\.8109All37\.325\.1−\-12\.245\.031\.3−\-13\.763\.538\.5−\-25\.055\.233\.5−\-21\.7838

Table 13:The integration deficit \(M−\-S\) by evidence source and by domain across the ladder\. Teal deepens with accuracy; the warm wash onΔ\\Deltadeepens with the integration deficit\. The gap widens from T to TLV regardless of source or domain, locating the loss in cross\-page integration rather than any single representation\.Table 14:Faithfulness across six prompt modes by evidence source\. Teal deepens with the good\-direction rate, warm with false abstention; the abstention instruction couples correct refusal to false refusal, and no prompt separates them\.Table 15:Verdict flips \(%\) from the uninstructed prompt to each abstention prompt, paired within\-question on the answerable set at oracle pages \(n=847n\{=\}847per cell\)\. R→\\rightarrowW splits into the questions the instruction refuses after answering them correctly and the questions it still answers, wrongly; the refusals are the smaller half at every rung but one\.
Table 16:Reasoning interventions at TLV: how each prompt and page ordering moves the integration deficit\. Teal deepens with accuracy, warm with the deficit;Δ\\DeltaGap isolates the interventions that narrow the gap \(chain\-of\-thought\) from those that only shift the level\.

## Appendix DFull Reasoning Details

### D\.1Deficit by Stratum

Table[13](https://arxiv.org/html/2608.07943#A3.T13)resolves the integration deficit by evidence source and by document domain across the ladder\. The widening the main text reports pooled holds almost everywhere: single\-page and multi\-page accuracy both rise as the encoding improves, but single\-page rises faster, so the deficit deepens most on the sources and domains where the image helps single\-page questions the most\. Two strata run against the pooled pattern and are worth noting\. Academic pages carry a positive deficit at every rung, meaning multi\-page questions there outperform single\-page ones, which likely reflects the structured cross\-referencing of academic documents suiting multi\-page evidence\. The administrative and financial domains show the deepest deficits at the combined rung \(−52\.8\-52\.8and−54\.1\-54\.1\), where single\-page accuracy is high and multi\-page accuracy is not, so the integration failure is sharpest exactly where the single\-page ceiling is highest\. Across the sources the deficit is present under every condition, which is what places the loss in cross\-page integration rather than in any single representation\.

### D\.2Prompting by Source

Table[14](https://arxiv.org/html/2608.07943#A3.T14)gives the full prompting sweep by evidence source, for answerable accuracy, false abstention, and correct abstention on unanswerable questions, across all four rungs\. The coupled abstention trade the main text reports pooled holds within each source: the instruction that raises correct refusal on unanswerable questions also raises false refusal on answerable ones, and no source escapes the coupling\. The reasoning prompts behave differently from the abstention ones, leaving both abstention rates near the uninstructed baseline while moving accuracy, so they act on the answer rather than on the model’s willingness to give one\. The per\-source view corrects an impression the pooled numbers leave\. Chain\-of\-thought lifts pooled accuracy at the combined encoding by only0\.60\.6points, which reads as a null result, but the pooled figure averages a6\.96\.9point gain on Table against movements within a point on every other source\. The gain therefore concentrates on tabular evidence, where answering typically requires reading several cells and relating them, rather than on the graphical sources where the image already does the work\. This is the same selectivity the hop split shows: the reasoning prompt helps where an answer has to be assembled from parts, and does nothing where it has to be read off\.

### D\.3What the Abstention Instruction Breaks

Table[15](https://arxiv.org/html/2608.07943#A3.T15)pairs each answerable question with itself across prompt modes, so it isolates what the abstention instruction costs on questions the reasoner could already answer\. The pages are the oracle evidence pages under both modes and only the instruction changes, which makes every flip attributable to the instruction rather than to what the model was shown\. The loss runs one way: at the combined encoding the instruction breaks12\.912\.9percent of the paired questions and repairs1\.81\.8, and the same asymmetry holds at all four rungs\. What the split adds is that most of the loss is not refusal\. Of the12\.912\.9points,4\.84\.8are questions the reasoner had answered correctly and now declines to answer, while8\.08\.0are questions it still answers and now answers wrongly\. The instruction therefore does two separable things, and only one of them is the cautious behaviour it was written to produce; the other is a perturbation of the answer itself, which the abstention rates alone do not reveal\. Read the other way, the refusals are better aimed than the pooled false\-abstention rate suggests\. Of the232232false abstentions at the combined encoding,17\.717\.7percent land on questions the uninstructed prompt answered correctly, against the52\.552\.5percent a refusal blind to correctness would hit, so the model does refuse preferentially where it was already failing\. The balanced wording acts on this half and not the other: it lowers the refusal component from4\.84\.8to3\.53\.5points at the combined encoding while leaving the wrong\-answer component where it was \(8\.08\.0against8\.38\.3\), which recovers about a point of accuracy overall\. Softening the instruction therefore buys back refusals, not answers, and the answer\-perturbing half of the cost survives every version of the instruction we test\.

### D\.4Moving the Deficit

Table[16](https://arxiv.org/html/2608.07943#A3.T16)isolates which prompt and page\-ordering interventions move the integration deficit rather than merely shifting the accuracy level\. The distinction matters, because an intervention can raise single\- and multi\-page accuracy together and leave the gap untouched, which is a weaker effect than repairing integration\. The chain\-of\-thought prompts narrow the deficit most, by8\.58\.5and9\.09\.0points, and the narrowing comes from the multi\-page side, so they act on integration directly\. Interleaving each page’s text with its own image narrows the gap only slightly \(1\.51\.5points\), so presentation order matters far less than the reasoning prompt\. The two abstention prompts show a positive gap movement, but this is an artefact of their depressing single\-page accuracy through false abstention on the answerable pool rather than a genuine integration gain, and it should not be read as one\. Reasoner scale belongs in the same comparison and is the clearest negative case: as Section[E\.2](https://arxiv.org/html/2608.07943#A5.SS2)shows, a four\-fold increase in parameters raises both hop classes and leaves the deficit no smaller, so of the interventions we test only the reasoning prompt reaches the loss\.

Table 17:Generation cost per rung: prefill and decode seconds for each reasoner configuration, over the input\-token profile each rung feeds\. Amber deepens with cost, on a separate scale for prefill and decode\. Adding the page image raises prefill roughly fifteen\-fold while decode stays flat; 8\-bit weights cost about three times the decode of bf16\. Blocks are separated by measuring machine and are not comparable across the rule\.Table 18:Accuracy by gold evidence\-page count for each Qwen3\-VL size, per rung\.Mpools all multi\-page questions andΔ\\DeltaMreads it against the 8B at the same encoding\. Scale buys multi\-page accuracy, most of it on the image encodings, but it raises single\-page accuracy just as fast, so the multi\-page deficit does not close\.

## Appendix ECost and Scale

### E\.1Prefill and Decode

Table[17](https://arxiv.org/html/2608.07943#A4.T17)splits generation cost into its input\-bound and output\-bound halves for each reasoner configuration, over the input\-token profile each encoding feeds\. Reporting the two separately is what makes either readable, since end\-to\-end latency is dominated by whichever half the run happens to inflate\. The input side is set by the representation and dwarfs everything else on the image encodings: prefill rises from1\.81\.8seconds at parser text alone to26\.326\.3at the combined encoding, roughly fifteen\-fold, because a page image costs about1,8001\{,\}800visual tokens against the few hundred a page of text contributes, and the mean input grows from1,6581\{,\}658tokens to5,0915\{,\}091\. The output side does not follow\. Decode is flat across the encodings, moving with the precision and the prompt instead: eight\-bit weights decode in21\.221\.2seconds at the combined encoding against6\.76\.7for bfloat16, roughly three times slower, and four\-bit in10\.310\.3\. This is the practical qualification on the quantization result in the main text, where low\-bit weights cost almost no accuracy for a much smaller footprint\. They are close to free in accuracy and in memory, but they are not free in latency, and a deployment that quantizes to fit a smaller card pays for it in decode time rather than in answers\. Two measurement caveats bound how far the table can be read\. Rows are grouped by the machine that measured them and the two groups must not be compared, since the larger and reasoning\-variant configurations only ever ran on hardware that prefills the same input roughly fifty times faster\. And every decode figure is an uninstructed upper bound, because these cells carry no instruction and the model runs to its token cap; the instructed modes settle an order of magnitude lower\.

### E\.2Scale and Integration

Table[18](https://arxiv.org/html/2608.07943#A4.T18)asks whether reasoner scale recovers multi\-page accuracy, by resolving accuracy against the number of gold evidence pages a question cites for each model size\. It buys accuracy, steadily and on both hop classes\. The 32B gains2\.62\.6,6\.06\.0,7\.87\.8and12\.512\.5points of multi\-page accuracy over the 8B across the four encodings, and the gain is largest where the image carries the evidence, which is consistent with the visual channel being the part of the input that most rewards capacity\. What scale does not buy is integration\. Single\-page accuracy rises just as fast as multi\-page accuracy, so the deficit between them is no smaller at 32B than at 2B, and at the combined encoding it is in fact widest at the two larger models\. Sixteen times the parameters therefore leaves the cross\-page loss where it was, which is the negative result that gives the chain\-of\-thought finding its force: a prompt costing nothing at inference reaches a loss that four size steps do not\. Two smaller observations follow from the same table\. The 4B tracks the 8B closely on multi\-page questions at both image encodings, within a point at each \(\+0\.6\+0\.6and−0\.3\-0\.3\), and trails it by only a few points on the text encodings, which supports routing text\-sufficient traffic to the smaller model rather than treating scale as uniformly beneficial\. And the tail buckets fall off sharply at every size, so questions citing three or more pages remain the hardest class in the corpus regardless of the reasoner, which is where a system that could genuinely combine evidence would show its advantage\.

## Appendix FWorked Examples

The tables report rates; this section shows individual questions behind them, one per failure mechanism \(Section[3](https://arxiv.org/html/2608.07943#S3)\) and one per intervention \(Sections[6\.1](https://arxiv.org/html/2608.07943#S6.SS1)and[6\.2](https://arxiv.org/html/2608.07943#S6.SS2)\)\. Each box gives the question, the gold answer, and the reasoner’s output under every condition shown, marked✓\\checkmarkcorrect,×\\timesincorrect or∘\\circabstained\. Page indices are the corpus’s zero\-based ones, and the gold pages are those the oracle conditions supply\.

Several rows show the reasoner declining to answer while correctly reporting that its evidence does not carry the information \(R1’s text rungs, R3’sT \(raw\)andTL \(MinerU\)\)\. Whether the judge logs that as an abstention or as an incorrect answer, it is not a correct one, and accuracy counts it as a failure to answer: the question was answerable from the document, and the condition under test is why the model could not answer it\. It is better behaviour than inventing a value, but not a third outcome for the metric\.

### F\.1Failure Mechanisms

Two per failure mode, one for each of its mechanisms, in the order Section[3](https://arxiv.org/html/2608.07943#S3)introduces them\.

R1Modality ceilingmmlongbench:000085Honor Watch GS Pro user guide \(Guidebook\), born\-digital\. Gold page 15\. Evidence source: plain text \+ figure\.QWhat are the colors of the icons that the users touch to move the app up and remove the app respectively?Goldgray, redThe page’s instructions survive the parse intact; the two inline icons do not\. PaddleOCR\-VL emits:\#\# NOTE • On the Favorites screen, touch to move the app up to the top of the list\. <img src="2608\.07943v1/\[…\]" width="3%" /\> <img src="2608\.07943v1/\[…\]" width="3%" /\> \- On the Favorites screen, touch to remove the app from the list\.T×\\times“The document does not specify the colors of the icons used to move the app up or remove the app\. The text only mentions that users should ‘touch’ certain icons \[…\] no visual descriptions or color information are provided\.”TL×\\times“\[…\] While it describes the actions users should take \(‘touch to move the app up’ and ‘touch to remove the app’\), it does not mention any visual details such as icon colors\.”V✓\\checkmark“To move the app up:The icon isgray\.To remove the app:The icon isred\.”TLV✓\\checkmark“To move the app up: The icon isgray\(a gray upward arrow\)\.To remove the app: The icon isred\(a red circle with a line through it \[…\]\)\.”Colour is not a linguistic signal, so no parser can carry it however well it works\. Both text rungs report the gap rather than guessing, which places the failure in the encoding and not in the reasoning over it\.

R3Conversion fidelitymmlongbench:000158Presentation deck \(Tutorial/Workshop\), scanned, so no usable embedded text layer\. Gold page 0\. Evidence source: layout\.QWhat is the name of the company that created this presentation?GoldAVALAUNCH MEDIAThe three parsers convert the same title slide into three different pages\. Only one keeps the banner that carries the answer:PaddleOCR\-VL: \#\# PRESENTATIONS THAT KICK ASTERKS MinerU 2\.5: \(empty\) unlimitedOCR: AVALAUNCH MEDIA \| PRESENTATIONS THAT KICK \| LAST YEART \(raw\)×\\times“The document does not provide any evidence or information about the company that created the presentation\.”TL \(Paddle\)×\\times“The name of the company that created this presentation isAsterks\.”TL \(MinerU\)×\\times“The document does not provide any evidence or information about the company \[…\]”TL \(unlimited\)✓\\checkmark“The name of the company that created this presentation isAVALAUNCH MEDIA\.”TLV✓\\checkmark“\[…\] the name of the company \[…\] isAvalaunch Media\. This name is clearly visible in the banner located at the top of the presentation’s title slide \[…\]”The banner leaves the encoding entirely and the slide title is corrupted to “ASTERKS”, which the reasoner then returns as a company name\. It has no way to tell a corrupted token from a real one, so a parser mistake is indistinguishable from evidence\. The page image recovers both\.

S1Evidence coveragemmlongbench:000141Climate assessment report \(Research report\), born\-digital\. Gold pages 13 and 14, both parts of one author list\. Evidence source: plain text\.QHow many chapter authors of this report come from Columbia University?Gold3oracle: 13, 14✓\\checkmark“\[…\] there arethreechapter authors from Columbia University: 1\.Radley Horton2\.Patrick Kinney3\.Carlos Perez Garcia\-Pando”drop top: 14 left×\\times“\[…\] there isonechapter author from Columbia University\. The author listed isCarlos Perez Garcia\-Pando\[…\] No other chapter authors in the document are listed as being from \[…\]”drop bottom: 13 left×\\times“\[…\] there aretwochapter authors from Columbia University:Radley Horton,Patrick Kinney\[…\]”Each truncated context yields a count that is correct over the pages supplied and wrong for the question asked, asserted with the same confidence and with any others explicitly denied\. The two removals cost different amounts, one author against two, which is why retrieval rank is a weak proxy for evidential necessity\.

S3Distractor exposuremmlongbench:000178Arab Youth Survey \(Research report\), scanned\. Gold page 19, a per\-country bar chart\. Distractors are the top\-ranked non\-gold pages: \+1 adds page 17, \+2 adds 15, \+3 adds 42\. Evidence source: chart\.QAccording to this survey, which country’s youth show the greatest concern about unemployment problem?GoldEgyptgold only✓\\checkmark“Egypt’s youth show the greatest concern about unemployment\. The chart displays the level of concern about unemployment for different countries, broken down into four categories \[…\]”\+1 distractor×\\times“\[…\] are fromIraq, with 59% reporting they are ‘Very concerned’\.”\+2 distractor×\\times“\[…\] isIraq\. The chart titled ‘CONCERN ABOUT UNEMPLOYMENT BY COUNTRY’ displays the percentage of youth who are ‘Very concerned’ \(the blue segment at the bottom of each bar\) \[…\]”\+3 distractor×\\times“\[…\] isAlgeria\. The chart titled ‘CONCERN ABOUT UNEMPLOYMENT BY COUNTRY’ displays \[…\]”The added page 17 is a second concern chart from the same survey, carrying the same question stem and its own unemployment figures\. The model keeps citing the correct chart by title and reads a different country off it each time\. Nothing was removed and nothing became ambiguous, so this is a chart reading that was barely succeeding being disturbed: a reasoning limit surfacing under selection rather than a selection cost\.

N2Evidence integrationmmlongbench:000443Dataset paper \(Academic paper\), born\-digital\. Gold pages 18 and 19, two qualitative\-example tables: page 18 gives the Acropolis Museum’s 2009 opening, page 19 the Chapel Bridge’s construction in 1333\. Evidence source: table\.QHow many years after The Chapel Bridge was The Acropolis Museum completed? Return me an integer\.Gold676none×\\times“10”cot✓\\checkmark“Step 1: \[…\] ‘The Acropolis Museum was inaugurated on June 20, 2009’\. Step 2: \[…\] ‘The Chapel Bridge is over 700 years old having been built in 1333’\. Step 3: \[…\] 2009−\-1333 = 676 \[…\]”The two dates sit one page apart and the subtraction is trivial, yet the direct answer matches neither the difference nor either date\. The second row is diagnostic rather than a result: both operands were reachable all along, which places the failure in the assembly and not in reading either page\. Note also that it is silent, since an integration failure produces a confident answer rather than a refusal\.

N3Calibration: hallucinationmmlongbench:000047Data\-leadership talk \(Tutorial/Workshop\), scanned, unanswerable\. Top\-3 retrieved pages \(0, 1, 2\); the premise of the question is not in the document at all\.QWhich Youtube does the slides use to show the consequce of having a weak data leadership?GoldNot answerablenone×\\times“The slides use a YouTube clip from the TV showBreaking Badto illustrate the consequence of having weak data leadership, specifically, the potential for organizations to make poor, uncontrolled decisions \[…\] as symbolized by the character Walter White hoarding money \[…\]”grounded×\\times“The slides use a YouTube clip featuring Walter White from ‘Breaking Bad’ to show the consequence of having a weak data leadership\.”abstain✓\\checkmark“Not answerable”One retrieved page carries a photograph of a man with stacks of money, and the model builds a show, a character and a narrative reading out of it, then repeats the invention when instructed to ground its answer in the document\. Grounding fails here because the answer*is*grounded in something visible; what is missing is any check that the document supports the claim being made\.

### F\.2Verdict Transitions Under Intervention

One example for each intervention recommended in Sections[6\.1](https://arxiv.org/html/2608.07943#S6.SS1)and[6\.2](https://arxiv.org/html/2608.07943#S6.SS2), showing what it does to a single question\. Where an intervention also has a cost, the reading notes it\.

M2Pair text with visionmmlongbench:000230Command newsletter \(Administration/Industry file\), born\-digital\. Gold page 1, carrying two commanding officers’ messages\. Evidence source: figure \+ plain text\.QWho is the commanding officer in the first figure on the second page?GoldCapt\. John W\. SandersT×\\times“\[…\] isCmdr\. Cindy Tamminga of NMRC\. The second page contains the ‘NMRC Commanding Officer’s Message’ \(first section\) and ‘NSMRL Commanding Officer’s Message’ \(second\) \[…\]”TL×\\times“\[…\] isCmdr\. Cindy Tamminga of NMRC\. The first figure \(image\) on thesecond pageis located \[…\]”V×\\times“\[…\] isSteven M\. Wechsler, CAPT, MC, USN\. The document is divided into two main sections, each with a ‘Commanding Officer’s Message’ \[…\]”TLV✓\\checkmark“\[…\] the first figure on the second page is the photograph accompanying the ‘NMRC Commanding Officer’s Message’\. The message is signed byJohn W\. Sanders III, CAPT, MC, USN\.”The question needs the position of a photograph, which the text channels cannot see, and a signature block, which the image channel has to read at small type\. Each single\-modality rung fails on the half it cannot serve and names a different wrong officer\. The pairing supplies both halves, which is the concrete form of the claim that the combined encoding is not merely the better of its parts\.

M3Favour recallmmlongbench:000403Genetics lecture deck \(Tutorial/Workshop\), scanned\. Gold pages 0, 27, 47\. Vision retriever; k=1 returns page 27, k=3 returns exactly the gold set, k=5 adds pages 6 and 18\. PROVISIONAL: partial G2 pool\.QWhich chapters are included in the slide? Your answer should be formatted as a list like \[’Chapter 1’, ’Chapter 3’\]\.Gold\[’Chapter 8’, ’Chapter 9’, ’Chapter 10’\]k=1 \(1 of 3 gold\)×\\times“\[’Chapter 9’\]”k=3 \(3 of 3 gold\)✓\\checkmark“\[’Chapter 8’, ’Chapter 9’, ’Chapter 10’\]”k=5 \(3 gold \+ 2 extra\)✓\\checkmark“\[’Chapter 8’, ’Chapter 9’, ’Chapter 10’\]”At k=1 the answer is truncated to the evidence available rather than wrong, exactly as in S1\. Going deeper completes the gold set and the answer completes with it, and the two non\-gold pages that arrive at k=5 leave it untouched, so both halves of the selection asymmetry are visible within a single question\.

M5Chain\-of\-thoughtmmlongbench:000111Pew survey report \(Research report\), born\-digital\. Gold pages 26 and 38: a bar chart and the sample\-size table\. Evidence source: table \+ chart\.QAmong all valid respondents, how many rep/lean rep independents \(the exact number, not the ratio\) favor building a strong relationship with China on trade? \[…\]Gold300none×\\times“260\.Reasoning:\[…\] The document provides the unweighted sample size for the ‘Rep/Lean Rep’ group as1,106\[…\]”cot✓\\checkmark“Step 1: Identify the group of interest\. Step 2: From the table, find the unweighted sample size for this group: 1,106\. Step 3: From the chart \[…\] 26% favor ‘Build a strong relationship’\. Step 4: Calculate: 1,106×\\times0\.26 = 287\.56\. Step 5: Round to the nearest hundred: 300\.”The uninstructed answer quotes the sample size from page 38 correctly and still returns a number that no combination of the two pages produces\. The instruction changes nothing about the evidence and only forces the percentage on page 26 to be retrieved as a separate step, so the repair is in the ordering of the work rather than in the information available\. The cost falls elsewhere: on graphical evidence the same decomposition resolves each bar in isolation and destroys comparisons the model reads correctly in one pass, which is why the pooled gain concentrates on tabular evidence\.

M8Reasoner scale: no repairmmlongbench:000287Fact\-verification paper \(Academic paper\), born\-digital\. Gold pages 2 and 3; the four example prompt boxes are split across them, three on page 2 \(Sections 3\.1 and 3\.2\) and one on page 3 \(Section 3\.4\)\. Evidence source: layout \+ plain text\.QHow many prompts instruction examples are listed in the main body of the paper? Please writhe down the answer in float format\.Gold48B×\\times“10\.0”32B×\\times“0\.0”8B \+ CoT✓\\checkmark“Section 3\.1 \[…\] provides one example prompt\. Section 3\.2 \[…\] provides two example prompts\. Section 3\.4 \[…\] provides one example prompt\. \[…\]”Page 2 contains the sentence “we prompt InstructGPT with ten distinct in\-context examples”\. The 8B model returns that number, reading a salient figure off one page instead of counting objects across two; the 32B does not copy it but fails differently and returns zero, so scale changes the error without removing it\. The same 8B model counts correctly once the prompt makes it walk the sections in order, which is Section[6\.1](https://arxiv.org/html/2608.07943#S6.SS1)’s ordering in miniature\.

M9 / M10Abstention: both directionsmmlongbench:000458, :000457Prompting paper \(Academic paper\), born\-digital\. M9 is unanswerable: the paper reports GPT\-4, not GPT\-4o\. M10 is answerable from gold pages 4 and 5\. Evidence source: table\.QM9: How much lower is the accuracy of GPT\-4oon SituatedQA compared to MMLU Chemistry? M10: How much lower is the accuracy of GPT\-4 on SituatedQA compared to MMLU Chemistry?GoldM9: Not answerable M10: 16\.7%M9 none×\\times“\[…\] the accuracy ofGPT\-4on SituatedQA is63\.2%, while its accuracy on MMLU Chemistry is79\.9%\[…\]16\.7 percentage points lower”M10 none✓\\checkmark“\[…\] the accuracy of GPT\-4 on SituatedQA is63\.2%\[…\]16\.7 percentage points lower\[…\]”M9 grounded×\\times“3\.3%”M10 grounded×\\times“13\.3%”M9 abstain✓\\checkmark“Not answerable”M10 abstain∘\\circ“Not answerable”The two questions differ by one character\. Uninstructed, the model returns the same answer to both, silently reading GPT\-4o as GPT\-4, which is correct for M10 and a hallucination for M9\. Instructed to abstain, it refuses both, and the grounding instruction in between reaches neither, replacing the shared answer with two different wrong numbers\. In no setting does the model distinguish the two questions; the instruction only chooses which of them it gets wrong\.