What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering

# Multi-Aspect Page Annotationfor Long-Document Multimodal Question Answering
Source: [https://arxiv.org/html/2608.14841](https://arxiv.org/html/2608.14841)
## What the Reranker Sees: Multi\-Aspect Page Annotation for Long\-Document Multimodal Question Answering

Jiayuan DingAffiliation:Hippocratic AI, Palo Alto, CA, USASubhabrata MukherjeeCarl YangAffiliation:Department of Computer Science, Emory University, Atlanta, GA, USAAffiliation:Hippocratic AI, Palo Alto, CA, USA

###### Abstract

Long\-document visual question answering \(VQA\) over documents of tens to hundreds of pages mixing text, tables, charts, and figures typically follows retrieve\-then\-read pipelines\. In our setting, the bottleneck shifts from retrieval recall to reranker\-side evidence selection: on MMLongBench\-Doc, BGE\-M3 reaches Recall@20=0\.8620\{=\}0\.86but only F1@5=0\.2545\{=\}0\.254, and even the visual retriever ColPali reaches only F1@5=0\.3325\{=\}0\.332—a text\-only rerank LLM seeing only raw snippets misses table, chart, and layout evidence even when the upstream retriever encoded images\. We proposeTrident, with two complementary components:Trident\-R, a retriever\-agnostic LLM reranker that converts each candidate into an LLM\-readable semantic record—visual caption, section path, entity tags, multi\-axis concept hits, and a text snippet—then performs a single adaptive\-KKrerank call; andTrident\-S, a generation\-side module that prompts the VLM under topical, entity, and structural lenses before synthesis\. On two long\-document datasets, the annotation\+rerank protocol substantially improves retrieval F1 across five heterogeneous pools, with every reranked pool exceeding the strongest adaptive\-KKbaseline PageIndex\. An LLM rerank without the annotation barely changes first\-hit ranking, indicating the lift comes from the structured annotation\.Trident\-Stargets open\-ended synthesis questions by design, adding up to6\.66\.6points in generation accuracy on these questions\. The bestTridentconfiguration is the strongest downstream QA pipeline in our evaluation, with rankings consistent across two LLM judges \(κ=0\.913\\kappa\{=\}0\.913\)\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.14841v1/figure1.png)Figure 1:Overview ofTrident\. Given a query and a long document, a retriever first returns a top\-20 candidate pool where evidence pages often already appear\.Trident\-Rimproves evidence selection by converting candidates into semantic records for adaptive LLM reranking, whileTrident\-Ssynthesizes the final answer through topical, entity, and structural lenses\.Long\-document visual question answering \(VQA\) requires answering natural\-language questions over documents of tens to hundreds of pages that mix text, tables, charts, and figures\. Although frontier Vision Language Models \(VLMs\) accept very long contexts in principle, feeding entire documents on every query is expensive and can suffer from position\-induced degradation\([21](https://arxiv.org/html/2608.14841#bib.bib15)\)\. Thus, retrieve\-then\-read pipelines\([15](https://arxiv.org/html/2608.14841#bib.bib25)\)remain practical: a retriever selects a small set of pages, and a VLM answers the question conditioned on them\. On MMLongBench\-Doc, oracle evidence pages alone lift Large Vision Language Model \(LVLM\) F1 by1010–3030points over full\-document inputs\([23](https://arxiv.org/html/2608.14841#bib.bib1)\), showing evidence selection is central to long\-document VQA\.

Where does evidence selection fail? In our setting, BGE\-M3\([2](https://arxiv.org/html/2608.14841#bib.bib3)\)achievesRecall​@​20=0\.86\\mathrm\{Recall\}@20\{=\}0\.86on MMLongBench\-Doc but onlyF1​@​5=0\.254\\mathrm\{F1\}@5\{=\}0\.254; the multimodal retriever ColPali\([9](https://arxiv.org/html/2608.14841#bib.bib7)\)improves this toF1​@​5=0\.332\\mathrm\{F1\}@5\{=\}0\.332but remains far from oracle\. Evidence pages are typically present in the top\-2020pool—the bottleneck is identifying the small evidence subset within2020surface\-similar candidates\.

A natural next step is to insert an LLM reranker between the top\-2020pool and theK≤5K\{\\leq\}5evidence selection, an approach used by RankGPT\([31](https://arxiv.org/html/2608.14841#bib.bib12)\)and increasingly common in long\-document RAG\. However, when this reranker only sees raw text snippets of each candidate page, it does not improve first\-hit ranking on either pool: on BGE\-M3, MRR is essentially unchanged \(0\.524→0\.5230\.524\\to 0\.523\); on ColPali, MRR actually*drops*from0\.6920\.692to0\.5980\.598, because the text\-only LLM judgment overrides ColPali’s stronger visual ranking signal\. The reason is that long\-document evidence is often locked in tables, charts, figures, and layout, all poorly represented in short text snippets\. A rerank\-stage LLM that only sees snippets is partially blind to exactly the evidence that matters most—ColPali’s visual encoding occurs only at retrieval; the rerank LLM still consumes text\-only candidate records and cannot access page images directly\.

This rerank\-stage blindness has been recognized: recent work addresses it by replacing the text LLM reranker with a multimodal LLM that directly sees page images\([3](https://arxiv.org/html/2608.14841#bib.bib36);[36](https://arxiv.org/html/2608.14841#bib.bib37)\)\. This is effective but expensive—every rerank query requires a full multimodal LLM call, and these rerankers typically need instruction\-tuning or RL training on multimodal labels\. We take an alternative architectural choice—a path used widely in industry RAG systems but underexplored in academic long\-document QA: keep a cheap, training\-free text\-LLM reranker, and pre\-compute a structured VLM\-generated annotation once per document offline\. Concretely, we convert each candidate into an LLM\-readable semantic record—a VLM\-generated visual caption surfaces table/chart/figure content, and structured fields \(section path, entity tags, topic/entity/structure concept hits\) expose pre\-computed evidence signals\. On the ColPali pool, attaching this annotation raises rerank F1 from0\.3320\.332to0\.5810\.581\. Consistent with the mechanism above, removing the caption hurts both pools but hurts BGE\-M3 more than ColPali: a visual retriever partially compensates for the LLM’s blindness at retrieval time, but does not eliminate the need for an explicit visual signal at rerank time\.

Motivated by these observations, we proposeTrident, a long\-document multimodal QA pipeline with two components addressing analogous bottlenecks at the rerank and generation stages\.Trident\-Rattaches the annotation above to each top\-2020candidate and uses a single LLM call to select an adaptive set ofK≤5K\{\\leq\}5evidence pages\. The same multi\-aspect logic carries over to generation: a single VLM call must implicitly trade off attention across text, tables, and figures within each retrieved page, so we additionally proposeTrident\-S, which prompts the VLM under topical, entity, and structural lenses before synthesizing a final answer\. On MMLongBench\-Doc, the annotation\+rerank protocol applied to five heterogeneous candidate pools \(BM25, BGE\-M3, RRF, ColPali, and our multi\-axisxKG\) raises retrieval F1 on all five, with every reranked pool exceeding the strongest adaptive\-KKbaseline PageIndex\([39](https://arxiv.org/html/2608.14841#bib.bib9)\); the improvement also holds on LongDocURL\.Trident\-Stargets open\-ended synthesis questions by design, providing consistent gains on that workload while leaving extractive workloads to a standard single\-call VLM\. End\-to\-end, the bestTridentconfiguration is the strongest downstream pipeline in our evaluation, with rankings consistent across two LLM judges \(κ=0\.913\\kappa\{=\}0\.913\)\.

## 2Related Work

#### Long\-document VQA benchmarks and multimodal RAG systems\.

We evaluate on MMLongBench\-Doc\([23](https://arxiv.org/html/2608.14841#bib.bib1)\)and LongDocURL\([6](https://arxiv.org/html/2608.14841#bib.bib2)\), two benchmarks targeting VQA over hundred\-plus\-page documents with multi\-modal evidence \(text, tables, charts, figures, layout\); MMLongBench\-Doc reports that GPT\-4o\-vision with full\-document access lags VLMs given oracle evidence pages by∼30\{\\sim\}30points\. Recent benchmarks complement these along different axes: retrieval\-focused MMDocIR\([7](https://arxiv.org/html/2608.14841#bib.bib16)\)\(313313long documents, notably finding VLM image descriptions outperform OCR text—consistent with our caption\-dominance result\), fine\-grained evidence\-selection MMDocRAG\([8](https://arxiv.org/html/2608.14841#bib.bib17)\)\(metrics beyond page\-level F1\), and difficulty\-graded REAL\-MM\-RAG\([34](https://arxiv.org/html/2608.14841#bib.bib19)\)\(finance / technical\)\. Multimodal RAG systems combine multi\-modal retrievers with multimodal LLMs \(M3DocRAG\([4](https://arxiv.org/html/2608.14841#bib.bib22)\)\), use hierarchical indexing with intra\-page and cross\-page chunks \(MMRAG\-DocQA\([11](https://arxiv.org/html/2608.14841#bib.bib23)\)\), build chunk–query graphs \(MLDocRAG\([41](https://arxiv.org/html/2608.14841#bib.bib18)\)\), triage by document structure \(PDFTriage\([26](https://arxiv.org/html/2608.14841#bib.bib20)\)\), or use VLM\-generated structured JSON descriptions for tables and figures \(MultiFinRAG\([10](https://arxiv.org/html/2608.14841#bib.bib26)\)\)\.Tridentdiffers by factoring the pipeline into a swappable Stage 1 retriever and a Stage 2 annotation\+\+rerank protocol we empirically isolate \(§[4\.2](https://arxiv.org/html/2608.14841#S4.SS2)\)\.

#### Document retrievers and adaptive\-KKselection\.

Page\-level retrievers span sparse \(BM25\), dense \(BGE\-M3\([2](https://arxiv.org/html/2608.14841#bib.bib3)\); ColBERT\-style late interaction\([14](https://arxiv.org/html/2608.14841#bib.bib4);[28](https://arxiv.org/html/2608.14841#bib.bib5)\); RRF fusion\([5](https://arxiv.org/html/2608.14841#bib.bib6)\)\), and vision\-aware variants \(ColPali\([9](https://arxiv.org/html/2608.14841#bib.bib7)\)with PaliGemma plus patch\-level MaxSim; DSE\([22](https://arxiv.org/html/2608.14841#bib.bib8)\)with image\-level dense embeddings\); none condition on document\-level structure\. Adaptive\-KKselection has been addressed by section\-tree navigation \(PageIndex\([39](https://arxiv.org/html/2608.14841#bib.bib9)\)\), hierarchical summarization \(RAPTOR\([29](https://arxiv.org/html/2608.14841#bib.bib10)\)\), reflection\-controlled retrieval \(Self\-RAG\([1](https://arxiv.org/html/2608.14841#bib.bib11)\)\), summary\-based re\-ranking \(SimpleDoc\([13](https://arxiv.org/html/2608.14841#bib.bib21)\)\), and relevance clustering \(AVIR\([18](https://arxiv.org/html/2608.14841#bib.bib24)\)\)\. Recent agentic approaches use iterative refinement \(Doc\-React\([35](https://arxiv.org/html/2608.14841#bib.bib27)\)\), unified MLLM retrieval\-generation \(URaG\([30](https://arxiv.org/html/2608.14841#bib.bib28)\)\), or multi\-turn RL \(MM\-Doc\-R1\([19](https://arxiv.org/html/2608.14841#bib.bib29)\)\)\. These either rely on a single structural / semantic / visual signal or require end\-to\-end MLLM / agent training;Tridentis complementary—training\-free, retriever\-agnostic, and easily bolted onto any pipeline via multi\-aspect annotation\.

#### LLM rerankers with structured features\.

Retrieval\-augmented generation\([15](https://arxiv.org/html/2608.14841#bib.bib25)\)traditionally couples retriever and generator via end\-to\-end training;Tridentsits at the opposite end of the design space—modular and prompting\-based, with multi\-aspect annotation serving as a frozen abstraction layer pairable with any pool and any VLM without re\-training\. RankGPT\([31](https://arxiv.org/html/2608.14841#bib.bib12)\)exposes raw candidate text to an LLM for re\-scoring; recent work instead conditions LLM rerankers on*structured per\-candidate features*: CoRank\([32](https://arxiv.org/html/2608.14841#bib.bib33)\)\(categories / sections / keywords for scientific retrieval\), FinCARDS\([42](https://arxiv.org/html/2608.14841#bib.bib30)\)\(schema fields for financial QA\), KeyB2\([16](https://arxiv.org/html/2608.14841#bib.bib34)\)\(block selection\), and AcuRank\([37](https://arxiv.org/html/2608.14841#bib.bib31)\)\(uncertainty\-aware adaptive listwise\)—all text\-only\.Tridentdiffers in three ways: \(i\) we target*long\-document multimodal*QA, using VLM\-generated visual captions as an LLM\-readable interface for multimodal evidence; \(ii\) we test the interface via cross\-pool isolation across five retriever paradigms, and show the gains transfer to the visual ColPali pool via asymmetric drop\-caption and pool\-invariant caption\-only ablations \(§[4\.2](https://arxiv.org/html/2608.14841#S4.SS2), Appendix[I](https://arxiv.org/html/2608.14841#A9)\); \(iii\) our generation\-side analogTrident\-Sextends the same principle to multi\-view VLM prompting\. We additionally expose multi\-aspect metadata—concept hits across topic / entity / structure axes plus section path, visual caption, and entity tags—enabling annotation\-level reasoning when surface text is repetitive among top\-20 candidates\. Knowledge graphs have been used for retrieval over textual corpora\([12](https://arxiv.org/html/2608.14841#bib.bib13)\)and document QA\([27](https://arxiv.org/html/2608.14841#bib.bib14)\);Tridentdiffers by treating annotation as a page\-level reranker\-friendly abstraction layer, not as a graph\-walk substrate\.

#### Caption\-based vs MLLM\-based multimodal reranking\.

Two architectural families incorporate visual content into multimodal rerankers\.Caption\-based approachesconvert page images to text via VLM descriptions and feed a text\-only LLM reranker; this pattern is common in industry RAG systems \(Haystack, NVIDIA RAG Blueprint, IBM Multimodal RAG\) but academically underexplored under a “loses visual details” assumption\.MLLM\-based approacheslet a multimodal LLM see page images directly, spanning zero\-shot prompt\-based rerankers\([3](https://arxiv.org/html/2608.14841#bib.bib36);[24](https://arxiv.org/html/2608.14841#bib.bib40)\), fine\-tuned / RL\-trained variants\([36](https://arxiv.org/html/2608.14841#bib.bib37);[33](https://arxiv.org/html/2608.14841#bib.bib39);[20](https://arxiv.org/html/2608.14841#bib.bib38)\), and listwise / production rerankers\([17](https://arxiv.org/html/2608.14841#bib.bib42)\); all are effective but costly per query and typically require multimodal instruction\-tuning or RL training\. Concurrent work also retrieves over VLM\-encoded pages directly\([38](https://arxiv.org/html/2608.14841#bib.bib41)\)\.Tridentsystematically revisits the caption\-based path for long\-document multimodal QA and shows that with structured multi\-aspect annotation, a text\-only LLM reranker is competitive with or better than text\-only / visual baselines at a fraction of per\-query cost, while remaining drop\-in compatible with any upstream retriever—tested via cross\-pool isolation across five retriever paradigms \(§[4\.2](https://arxiv.org/html/2608.14841#S4.SS2)\)\.

#### VLM\-augmented LLM QA and multi\-view prompting\.

SCRA\-VQA\([40](https://arxiv.org/html/2608.14841#bib.bib32)\)converts single\-image VQA into a caption\+\+LLM rerank pipeline, demonstrating that VLM captions can substantially augment LLM\-based QA at the single\-image scale; MPCAR\([25](https://arxiv.org/html/2608.14841#bib.bib35)\)generates diverse complementary descriptions from multiple analytical perspectives and fuses them in a single\-image LVLM reasoning prompt\.Tridentextends this lineage to*long\-document*multimodal retrieval and generation: visual caption is identified as the strongest universal annotation field on the retrieval side; the three orthogonal lenses on the generation side \(Trident\-S\) are structured around our T / E / S decomposition axes and are followed by a format\-aware LLM synthesis call\. Crucially, we*characterize*Trident\-S’s effective regime \(substantial gains on open\-ended synthesis questions only\) rather than claim universal lift\.

## 3Method

![Refer to caption](https://arxiv.org/html/2608.14841v1/figure2.png)Figure 2:Overall framework of TRIDENT\-R\.### 3\.1Problem Formulation

Let a documentD=\{p1,…,pN\}D=\\\{p\_\{1\},\\ldots,p\_\{N\}\\\}containNNpages of mixed text and visual content \(NNmay exceed100100\)\. Given a natural\-language questionqqwith gold evidence setEq⊆DE\_\{q\}\\subseteq Dand gold answeraqa\_\{q\}, the long\-document VQA task is to produce an answera^\\hat\{a\}approximatingaqa\_\{q\}\. We decompose the task into retrieval and generation:

P^q=ℛ⁡\(q,D\),\|P^q\|≤Kmax;a^=𝒢⁡\(q,P^q\),\\hat\{P\}\_\{q\}=\\mathcal\{R\}\(q,D\),\\ \|\\hat\{P\}\_\{q\}\|\\leq K\_\{\\max\};\\quad\\hat\{a\}=\\mathcal\{G\}\(q,\\hat\{P\}\_\{q\}\),\(1\)whereℛ\\mathcal\{R\}returns up toKmaxK\_\{\\max\}evidence pages and𝒢\\mathcal\{G\}is a VLM\. Following the reranker\-interface bottleneck identified in §[1](https://arxiv.org/html/2608.14841#S1),Tridentfactorsℛ\\mathcal\{R\}into a candidate\-retrieval stage and an annotation\+\+rerank stage:

P^q=ℛ2​\(q,ℛ1​\(q,D\),Φ⁡\(D\)\),\\hat\{P\}\_\{q\}=\\mathcal\{R\}\_\{2\}\(q,\\mathcal\{R\}\_\{1\}\(q,D\),\\ \\Phi\(D\)\),\(2\)whereℛ1\\mathcal\{R\}\_\{1\}returns a top\-M=20M\{=\}20pool,Φ⁡\(D\)\\Phi\(D\)is a multi\-aspect annotation \(§[3\.2](https://arxiv.org/html/2608.14841#S3.SS2)\), andℛ2\\mathcal\{R\}\_\{2\}is an LLM reranker withKmax=5K\_\{\\max\}\{=\}5\(§[3\.3](https://arxiv.org/html/2608.14841#S3.SS3)\)\.𝒢\\mathcal\{G\}is then realized by either a single VLM call or a multi\-view generatorTrident\-S\(§[3\.5](https://arxiv.org/html/2608.14841#S3.SS5)\)\.ℛ1\\mathcal\{R\}\_\{1\}is treated as a swappable candidate\-pool source \(§[3\.4](https://arxiv.org/html/2608.14841#S3.SS4)\), whileℛ2\\mathcal\{R\}\_\{2\}and the page annotationΦ\\Phiare the focus of this work\.

### 3\.2Trident\-R\-Index: Multi\-Aspect Page Annotation

Trident\-R\-Index is a functionΦ⁡\(D\)=\(𝒞,ℳ\)\\Phi\(D\)=\(\\mathcal\{C\},\\mathcal\{M\}\)that maps a documentDDto a set of concepts𝒞\\mathcal\{C\}and per\-page metadata records\{ℳp\}p∈D\\\{\\mathcal\{M\}\_\{p\}\\\}\_\{p\\in D\}\. The concept set decomposes into three disjoint axes:

𝒞=𝒞T∪𝒞E∪𝒞S\\mathcal\{C\}\\;=\\;\\mathcal\{C\}\_\{T\}\\cup\\mathcal\{C\}\_\{E\}\\cup\\mathcal\{C\}\_\{S\}\(3\)where𝒞T\\mathcal\{C\}\_\{T\}contains topic concepts,𝒞E\\mathcal\{C\}\_\{E\}entity\-bundle concepts, and𝒞S\\mathcal\{C\}\_\{S\}structural concepts\. Each conceptc∈𝒞Xc\\in\\mathcal\{C\}\_\{X\}\(X∈\{T,E,S\}X\\in\\\{T,E,S\\\}\) carries a name, an embedding𝐞c∈ℝd\\mathbf\{e\}\_\{c\}\\in\\mathbb\{R\}^\{d\}\(where applicable\), and a*weighted membership*—a set of \(page, weight\) pairs\{\(p,wc,p\):p∈D,wc,p\>0\}\\\{\(p,w\_\{c,p\}\):p\\in D,\\ w\_\{c,p\}\>0\\\}wherewc,p∈\[0,1\]w\_\{c,p\}\\in\[0,1\]reflects how strongly pageppcontributes to conceptcc\.

#### Topic concepts𝒞T\\mathcal\{C\}\_\{T\}\.

We segment each pageppinto overlapping text windows of≤512\\leq 512tokens \(stride256256\), denotedWp=\{wp\(1\),…,wp\(np\)\}W\_\{p\}=\\\{w^\{\(1\)\}\_\{p\},\\ldots,w^\{\(n\_\{p\}\)\}\_\{p\}\\\}\. All windows⋃pWp\\bigcup\_\{p\}W\_\{p\}are embedded by a sentence embedding modelℰW\\mathcal\{E\}\_\{W\}and L2\-normalized\. We cluster the embedding set via mini\-batchKK\-means withKT=min⁡\(64,\|W\|8\)K\_\{T\}\{=\}\\min\(64,\\tfrac\{\|W\|\}\{8\}\)\. Each resulting clusterc∈𝒞Tc\\in\\mathcal\{C\}\_\{T\}records its centroid𝐞c\\mathbf\{e\}\_\{c\}, member windowsWcW\_\{c\}, and per\-page weight:

wc,p=\|Wc∩Wp\|\|Wc\|,so that​∑pwc,p=1\.w\_\{c,p\}\\;=\\;\\frac\{\|W\_\{c\}\\cap W\_\{p\}\|\}\{\|W\_\{c\}\|\},\\quad\\text\{so that \}\\sum\_\{p\}w\_\{c,p\}=1\.\(4\)

#### Entity\-bundle concepts𝒞E\\mathcal\{C\}\_\{E\}\.

An extraction LLMℒext\\mathcal\{L\}\_\{\\text\{ext\}\}extracts a flat set of typed entitiesℰp\\mathcal\{E\}\_\{p\}from each page—named entities, numbers, dates, monetary values, percentages, and table / figure references\. We group co\-occurring entities across pages using density\-based clustering on a graph where entities are connected when they co\-occur on at least one page, retaining bundles of size≥2\\geq 2\. Each bundlec∈𝒞Ec\\in\\mathcal\{C\}\_\{E\}stores its member entity setℰc\\mathcal\{E\}\_\{c\}and the pages on whichℰc\\mathcal\{E\}\_\{c\}surfaces, with weightwc,pw\_\{c,p\}equal to the fraction ofℰc\\mathcal\{E\}\_\{c\}entities appearing onpp\.

#### Structural concepts𝒞S\\mathcal\{C\}\_\{S\}\.

We obtain a hierarchical section tree ofDDfrom a public PDF structural parser𝒫sec\\mathcal\{P\}\_\{\\text\{sec\}\}\(any document parser returning a section hierarchy can be substituted; we ground our choice in §[4\.1](https://arxiv.org/html/2608.14841#S4.SS1)\)\. Each leaf sectionssbecomes a structural conceptccwith member pages\{p:p∈span​\(s\)\}\\\{p:p\\in\\text\{span\}\(s\)\\\}\(uniform weights\), and embedding𝐞c=ℰW​\(summary​\(s\)\)\\mathbf\{e\}\_\{c\}=\\mathcal\{E\}\_\{W\}\(\\text\{summary\}\(s\)\), wheresummary​\(s\)\\text\{summary\}\(s\)is the section summary returned by𝒫sec\\mathcal\{P\}\_\{\\text\{sec\}\}\.

#### Per\-page metadataℳp\\mathcal\{M\}\_\{p\}\.

In addition to the concepts above, we attach four per\-page fields to eachpp: \(a\) a*structural path*string \(e\.g\. “Item 8 / Consolidated Balance Sheets”\), read off the section tree; \(b\) a*visual caption*vcp\\text\{vc\}\_\{p\}, generated by a captioning VLM𝒱cap\\mathcal\{V\}\_\{\\text\{cap\}\}prompted to describe the page’s visible tables, figures, charts, and layout in≤60\\leq 60words; \(c\) a flat*entity\-tag list*etp⊆ℰp\\text\{et\}\_\{p\}\\subseteq\\mathcal\{E\}\_\{p\}retaining the top\-15 most salient entities/numbers on the page; and \(d\) the page’s raw text snippet \(truncated to 600 characters\)\.

#### Offline cost\.

Index construction requires per\-page LLM/VLM passes once per document and is cached on disk for reuse across all subsequent queries; full token\-level accounting is in Appendix[H](https://arxiv.org/html/2608.14841#A8)\.

### 3\.3Trident\-R: Adaptive\-KKLLM Rerank

Given a queryqq, we first obtain a candidate page poolPq⊆DP\_\{q\}\\subseteq Dwith\|Pq\|≤M=20\|P\_\{q\}\|\\leq M=20from one of the retrievers in §[3\.4](https://arxiv.org/html/2608.14841#S3.SS4), thenTrident\-Rperforms a single LLM call that returns an adaptive\-KKordered subsetP^q⊆Pq\\hat\{P\}\_\{q\}\\subseteq P\_\{q\}with\|P^q\|≤Kmax=5\|\\hat\{P\}\_\{q\}\|\\leq K\_\{\\max\}=5\.

#### Candidate payload\.

For each candidate pagep∈Pqp\\in P\_\{q\}, we assemble a structured recordℛp=\(p,sp,chp,ℳp\)\\mathcal\{R\}\_\{p\}=\(p,\\ s\_\{p\},\\ \\text\{ch\}\_\{p\},\\ \\mathcal\{M\}\_\{p\}\)wheresps\_\{p\}is the underlying retriever score andchp\\text\{ch\}\_\{p\}is a*concept\-hit list*

chp=\{\\displaystyle\\text\{ch\}\_\{p\}\\;=\\;\\big\\\{\(X,c\.name\):c∈𝒞X,\\displaystyle\(X,c\.\\text\{name\}\):c\\in\\mathcal\{C\}\_\{X\},\(5\)wc,p\>0,X∈\{T,E,S\}\}\\displaystyle w\_\{c,p\}\>0,\\ X\\in\\\{T,E,S\\\}\\big\\\}collapsing all concept memberships involvingppinto a typed list of≤6\\leq 6tags \(e\.g\.T:Financial Statements,E:Nike\)\. Combined withℳp\\mathcal\{M\}\_\{p\}, each candidate exposes*five*discriminating signals to the reranker: concept hits, section path, visual caption, entity tags, and a truncated text excerpt\.

#### Rerank step\.

We serialize\{ℛp\}p∈Pq\\\{\\mathcal\{R\}\_\{p\}\\\}\_\{p\\in P\_\{q\}\}as a JSON list and prompt a rerank LLMℛθ\\mathcal\{R\}\_\{\\theta\}\(atτ=0\\tau\{=\}0\) with the question and candidate list, expecting a JSON response of the form:

ℛθ​\(q,\{ℛp\}\)=\(thinking,P^q\),\|P^q\|≤Kmax\.\\mathcal\{R\}\_\{\\theta\}\(q,\\\{\\mathcal\{R\}\_\{p\}\\\}\)\\;=\\;\\big\(\\text\{thinking\},\\ \\hat\{P\}\_\{q\}\\big\),\\quad\|\\hat\{P\}\_\{q\}\|\\leq K\_\{\\max\}\.\(6\)The reranker may return fewer thanKmaxK\_\{\\max\}pages when the question’s evidence is concentrated, yielding an adaptive\|P^q\|\|\\hat\{P\}\_\{q\}\|\. The verbatim rerank prompt is in Appendix[B](https://arxiv.org/html/2608.14841#A2)\.

#### Why JSON\-conditioned rerank?

Compared to a fully\-textual rerank prompt \(e\.g\. RankGPT\([31](https://arxiv.org/html/2608.14841#bib.bib12)\)\), exposing structured per\-candidate fields lets the LLM attend to specific evidence signals—visual caption, entity tags, concept memberships—rather than re\-deriving them from the raw page text in its limited context\. We empirically validate this design in §[4\.2](https://arxiv.org/html/2608.14841#S4.SS2)and isolate per\-field contributions in §[4\.3](https://arxiv.org/html/2608.14841#S4.SS3)\.

### 3\.4Candidate Pool Sources

Trident\-Ris agnostic to the source ofPqP\_\{q\}\. We evaluate four pool choices, each returning the top\-M=20M\{=\}20pages by score\.

#### Single\-signal text pools\.

BM25returns the top\-20 pages by Okapi BM25 score over page text\.BGE\-M3\([2](https://arxiv.org/html/2608.14841#bib.bib3)\)returns the top\-20 pages by cosine similarity between the query embedding and page embeddings \(dim10241024\)\.RRF\([5](https://arxiv.org/html/2608.14841#bib.bib6)\)fuses BM25 and BGE\-M3 rankings by Reciprocal Rank Fusion \(k=60k\{=\}60\)\.

#### Vision pool\.

ColPali\([9](https://arxiv.org/html/2608.14841#bib.bib7)\)encodes each page image with PaliGemma and returns top\-20 pages by late\-interaction MaxSim\.

#### Multi\-axis pool \(xKG\)\.

We compose a multi\-channel score from𝒞T,𝒞E,𝒞S\\mathcal\{C\}\_\{T\},\\mathcal\{C\}\_\{E\},\\mathcal\{C\}\_\{S\}\. A lightweight LLM router \(Appendix[D](https://arxiv.org/html/2608.14841#A4)\) first classifiesqqinto a classκ∈\{topical,entity\-level,structural,mixed\}\\kappa\\in\\\{\\text\{topical\},\\ \\text\{entity\-level\},\\ \\text\{structural\},\\ \\text\{mixed\}\\\}and extracts query entitiesℰq\\mathcal\{E\}\_\{q\}\. The concept\-level scores per axis are:

scoreT​\(c,q\)\\displaystyle\\text\{score\}\_\{T\}\(c,q\)=cos⁡\(𝐞c,𝐞q\),\\displaystyle=\\mathrm\{cos\}\(\\mathbf\{e\}\_\{c\},\\mathbf\{e\}\_\{q\}\),c∈𝒞T\\displaystyle c\\in\\mathcal\{C\}\_\{T\}\(7\)scoreE​\(c,q\)\\displaystyle\\text\{score\}\_\{E\}\(c,q\)=\|ℰc∩subℰq\|\|ℰc\|,\\displaystyle=\\frac\{\|\\mathcal\{E\}\_\{c\}\\cap\_\{\\text\{sub\}\}\\mathcal\{E\}\_\{q\}\|\}\{\|\\mathcal\{E\}\_\{c\}\|\},c∈𝒞E\\displaystyle c\\in\\mathcal\{C\}\_\{E\}\(8\)scoreS​\(c,q\)\\displaystyle\\text\{score\}\_\{S\}\(c,q\)=cos⁡\(𝐞c,𝐞q\),\\displaystyle=\\mathrm\{cos\}\(\\mathbf\{e\}\_\{c\},\\mathbf\{e\}\_\{q\}\),c∈𝒞S\\displaystyle c\\in\\mathcal\{C\}\_\{S\}\(9\)where∩sub\\cap\_\{\\text\{sub\}\}denotes loose substring matching to handle entity surface variants\. Page\-level scores are obtained by aggregating concept contributions weighted by membership and a class\-conditional fusion\(αTκ,αEκ,αSκ\)\(\\alpha\_\{T\}^\{\\kappa\},\\alpha\_\{E\}^\{\\kappa\},\\alpha\_\{S\}^\{\\kappa\}\):

spxKG​\(q\)=∑X∈\{T,E,S\}αXκ​∑c∈𝒞XscoreX​\(c,q\)⋅wc,p\.s^\{\\textsc\{xKG\}\{\}\}\_\{p\}\(q\)\\;=\\;\\sum\_\{X\\in\\\{T,E,S\\\}\}\\alpha\_\{X\}^\{\\kappa\}\\sum\_\{c\\in\\mathcal\{C\}\_\{X\}\}\\text\{score\}\_\{X\}\(c,q\)\\cdot w\_\{c,p\}\.\(10\)The top\-MMpages byspxKGs^\{\\textsc\{xKG\}\{\}\}\_\{p\}form thexKGpool\. Specific\(αTκ,αEκ,αSκ\)\(\\alpha\_\{T\}^\{\\kappa\},\\alpha\_\{E\}^\{\\kappa\},\\alpha\_\{S\}^\{\\kappa\}\)values for eachκ\\kappa, and a router\-ablation comparison with uniform weights, are in Appendix[D](https://arxiv.org/html/2608.14841#A4)\.

### 3\.5Trident\-S: Multi\-View Generation

Given retrieved pagesP^q\\hat\{P\}\_\{q\},Trident\-Sprompts a VLM𝒱\\mathcal\{V\}three times in parallel under three lenses applied to the*same*retrieved image set\{render​\(p\):p∈P^q\}\\\{\\text\{render\}\(p\):p\\in\\hat\{P\}\_\{q\}\\\}:

- T:*Topical lens\.*“What is each page about thematically? Which topics relate to the question?”
- E:*Entity / value lens\.*“Extract the named entities, numerical values, dates, table cells, and figure labels visible on each page\. Use verbatim values\.”
- S:*Structural lens\.*“Where does each page sit in the document organization \(section title, captions, headers\)?”

Producing three candidate answersa^T,a^E,a^S\\hat\{a\}^\{T\},\\hat\{a\}^\{E\},\\hat\{a\}^\{S\}\. A fourth*synthesis*call𝒮\\mathcal\{S\}merges them into a final answer:

a^=𝒮⁡\(q,a^T,a^E,a^S\),\\hat\{a\}\\;=\\;\\mathcal\{S\}\\big\(q,\\ \\hat\{a\}^\{T\},\\ \\hat\{a\}^\{E\},\\ \\hat\{a\}^\{S\}\\big\),\(11\)with format\-aware preferences encoded in the synthesis prompt \(preferring the E\-view answer for numeric questions, the S\-view for location questions, and consensus otherwise\)\. When all three lenses agree the question is unanswerable,𝒮\\mathcal\{S\}returns “Not answerable”—a calibration property we examine in §[4\.4](https://arxiv.org/html/2608.14841#S4.SS4)\. Full prompts are in Appendix[C](https://arxiv.org/html/2608.14841#A3)\.

#### Why three lenses?

Long\-document evidence is often heterogeneous within a single retrieved page \(e\.g\., a financial\-report page contains both narrative text*and*a table\)\. A single VLM call must implicitly trade off attention across modalities; by issuing three lens\-conditioned prompts we explicitly induce the VLM to surface each evidence type, then defer the trade\-off to a smaller textual synthesis call\. This is conceptually analogous to multi\-aspect annotation at generation time\.

We treatTrident\-Sas a question\-format\-conditioned extension; the empirical characterization of when it helps is in §[4\.4](https://arxiv.org/html/2608.14841#S4.SS4)and §[4\.5](https://arxiv.org/html/2608.14841#S4.SS5)\.

## 4Experiments

### 4\.1Setup

Table 1:Cross\-pool isolation on MMLongBench\-Doc\(N=830N\{=\}830aligned QAs; all reranked rows use GPT\-4\.1 withKmax=5K\_\{\\max\}\{=\}5\)\. The same annotation\+\+rerank protocol improves sparse, hybrid, graph, dense, and visual candidate pools\.Table 2:Mechanism ablations\. LLM reranking with only raw text snippets does not improve first\-hit ranking on either pool \(MRR flat on BGE\-M3, and decreasing on ColPali because the LLM judgment overrides ColPali’s visual ranking signal\); the ranking lift requires the structured annotation, and the visual caption alone recovers most of it on both text and visual pools\.Datasets: MMLongBench\-Doc\([23](https://arxiv.org/html/2608.14841#bib.bib1)\)\(1,082 QAs / 134 documents\) and LongDocURL\([6](https://arxiv.org/html/2608.14841#bib.bib2)\)\(1,123 QAs / 200 documents\)\. Retrieval metrics are reported on theN=830N\{=\}830answerable subset of MMLongBench\-Doc\. End\-to\-end accuracy is reported on a 300\-QA stratified subset; the full sampling details are in Appendix[E](https://arxiv.org/html/2608.14841#A5)\.

Baselines: BM25, BGE\-M3\([2](https://arxiv.org/html/2608.14841#bib.bib3)\), RRF\([5](https://arxiv.org/html/2608.14841#bib.bib6)\), ColPali\([9](https://arxiv.org/html/2608.14841#bib.bib7)\)atK=5K\{=\}5\(fixed\-KK, no rerank\), and PageIndex\([39](https://arxiv.org/html/2608.14841#bib.bib9)\)adaptive\-KKcookbook with GPT\-4\.1\. Our cross\-pool isolation pairs each retriever pool with the sameTrident\-Rannotation\+\+LLM rerank protocol\.

Module assignments: We instantiate the abstract modules of §[3](https://arxiv.org/html/2608.14841#S3)as follows—text embeddings \(ℰW\\mathcal\{E\}\_\{W\}\):text\-embedding\-3\-large; extraction LLMℒext\\mathcal\{L\}\_\{\\text\{ext\}\}, rerank LLMℛθ\\mathcal\{R\}\_\{\\theta\}, and synthesis LLM𝒮\\mathcal\{S\}: GPT\-4\.1 atτ=0\\tau\{=\}0; captioning VLM𝒱cap\\mathcal\{V\}\_\{\\text\{cap\}\}: GPT\-4o\-mini; answering VLM𝒱ans\\mathcal\{V\}\_\{\\text\{ans\}\}: GPT\-4\.1 \(vision\); structural parser𝒫sec\\mathcal\{P\}\_\{\\text\{sec\}\}: the PageIndex API\([39](https://arxiv.org/html/2608.14841#bib.bib9)\)used purely as a PDF section\-tree extractor, independent of the PageIndex*retrieval system*we benchmark against\. The candidate JSON records produced byΦ\\Phiare consumable by any instruction\-tuned LLM; the annotation\+\+rerank protocol is therefore not tied to a specific model family\. Claude and Gemini additionally serve as LLM\-as\-judge evaluators \(Appendix[L](https://arxiv.org/html/2608.14841#A12)\)\.

Metrics: Retrieval is evaluated by page\-level Recall, Precision, F1, MRR, and nDCG against gold evidence pages\. End\-to\-end QA is evaluated by MMLongBench\-Doc Generalized Accuracy and LLM\-as\-judge accuracy\.

PageIndex output: PageIndex selects section nodes, which we expand to constituent pages and truncate at top\-55for apples\-to\-apples comparison\. On MMLongBench\-Doc this cap is rarely active \(18\.6%18\.6\\%of queries\); on LongDocURL it is more active \(49\.7%49\.7\\%\)\.

### 4\.2Cross\-Pool Retrieval Isolation

Table[1](https://arxiv.org/html/2608.14841#S4.T1)tests whether the proposed interface works only for a particular retriever or transfers across candidate\-pool distributions\. The answer is consistent: applying the same annotation\+\+rerank protocol more than doubles raw retriever F1 for several pools \(e\.g\., BGE\-M30\.254→0\.5460\.254\\to 0\.546\) and raises all five pools above PageIndex\. The visual ColPali pool is the most important case for our story: although ColPali already encodes page images, converting its candidates into LLM\-readable annotations still raises F1 from0\.3320\.332to0\.5810\.581\. This indicates that the interface is not merely supplying missing visual information to text retrievers\.

### 4\.3Mechanism: What the Reranker Sees

Table[2](https://arxiv.org/html/2608.14841#S4.T2)isolates why the interface helps\. The text\-snippet\-only rerank rows show that an LLM with raw page snippets improves F1 by shortening the output, but does not improve first\-hit ranking on either pool: on BGE\-M3 MRR is essentially unchanged \(0\.524→0\.5230\.524\\to 0\.523\), and on ColPali MRR actually drops \(0\.692→0\.5980\.692\\to 0\.598\) because the snippet\-only LLM judgment overrides ColPali’s stronger visual ranking signal\. The ranking lift appears only once the reranker sees structured page annotation, and the visual caption alone is sufficient to recover most of it \(BGE\-M3 MRR→0\.692\\to 0\.692, ColPali MRR→0\.744\\to 0\.744with caption\-only;→0\.744/0\.788\\to 0\.744/0\.788with full annotation\)\. Full per\-field drop ablations \(Appendix[I](https://arxiv.org/html/2608.14841#A9)\) show the same pattern: removing visual caption hurts both pools, and the remaining topic/entity/structure axes add a small but nearly identical marginal gain \(\+0\.028\+0\.028/\+0\.027\+0\.027\)\. Thus the caption is not just modality completion for text retrievers; it is a reranker\-facing semantic representation that remains useful even when the upstream retriever is visual\.

We keep detailed secondary analyses in the appendix to keep the main text focused on the reranker\-interface claim\. Appendix[I](https://arxiv.org/html/2608.14841#A9)reports full field ablations on BGE\-M3 and ColPali; Appendix[D](https://arxiv.org/html/2608.14841#A4)reportsxKGrouter details; Appendix[J](https://arxiv.org/html/2608.14841#A10)reports the adaptive\-KKdistribution; and Appendix[F](https://arxiv.org/html/2608.14841#A6)reports bootstrap, Wilcoxon, and McNemar tests\.

### 4\.4End\-to\-End QA as Supporting Evidence

Table 3:End\-to\-end QA on the 300\-QA MMLongBench\-Doc subset \(selected pipelines; all values in %\)\. Full 14\-pipeline results, including answer accuracy and unanswerable abstention, are in Appendix[A](https://arxiv.org/html/2608.14841#A1)\.Table[3](https://arxiv.org/html/2608.14841#S4.T3)shows that the retrieval\-side gain generally transfers to downstream QA\. The ColPali pool with annotation and rerank is the best observed end\-to\-end configuration in this subset, and addingTrident\-Sgives the highest Gen Acc and Gen F1 on MMLongBench\-Doc\. The full 14\-pipeline table and inter\-judge agreement are in Appendix[A](https://arxiv.org/html/2608.14841#A1)and Appendix[L](https://arxiv.org/html/2608.14841#A12); the pipeline ranking is consistent across the two judges \(κ=0\.913\\kappa\{=\}0\.913\)\.

### 4\.5LongDocURL Transfer Check

Table 4:LongDocURL retrieval comparison \(N=1,122N\{=\}1\{,\}122QAs with non\-empty gold evidence\)\. The retrieval\-side trend transfers, although the full cross\-pool isolation study is conducted on MMLongBench\-Doc\.Table[4](https://arxiv.org/html/2608.14841#S4.T4)provides an external transfer check onN=1,122N\{=\}1\{,\}122QAs\.Trident\-Rachieves the highest retrieval F1, showing that the retrieval\-side trend transfers to LongDocURL\. End\-to\-end results in Appendix[A](https://arxiv.org/html/2608.14841#A1)show the same pattern: ColPali plus annotation and rerank remains strongest\.

#### When doesTrident\-Shelp?

Trident\-Sdelivers consistent positive gains on MMLongBench\-Doc’s open\-ended synthesis questions \(\+1\.7\+1\.7to\+6\.6\+6\.6Gen Acc across all eight evaluated pipelines, with the largest absolute gain on weaker baselines—PageIndex Plain→⁣\+\\to\{\+\}Trident\-Sadds\+6\.6\+6\.6\)\. On LongDocURL’s MCQ\-style and short\-extractive workloads, the three lenses converge on the same extracted answer, andTrident\-Sis bypassed by design\. We treatTrident\-Sas a question\-format\-conditioned extension; an a\-priori format router that gatesTrident\-Sinvocation is a natural deployment refinement\.

## 5Limitations and Conclusion

#### Limitations\.

Trident\-Rrequires an offline per\-document indexing pass \($0\.256\\$0\.256/100 pages, amortized across queries; Appendix[H](https://arxiv.org/html/2608.14841#A8)\)\. The reranker uses GPT\-4\.1 here, but smaller or open\-source LLMs are drop\-in substitutes \(Appendix[M](https://arxiv.org/html/2608.14841#A13)\)\. The five\-pool isolation is conducted on MMLongBench\-Doc, with LongDocURL as an external transfer check \(§[4\.5](https://arxiv.org/html/2608.14841#S4.SS5)\)\. We use PageIndex only as a section\-path parser; public PDF parsers such as PyMuPDF or GROBID could replace it\. Finally, since most lift comes from visual captions, a caption\-first variant is preferable when other fields are costly\.

#### Conclusion\.

We identify a reranker\-interface bottleneck in long\-document multimodal QA: evidence selection improves when candidate pages are converted into LLM\-readable semantic annotations, even with multimodal retrievers\.Trident\-Rimplements this as a retriever\-agnostic annotation\-based LLM reranker\. Across five candidate\-pool families on MMLongBench\-Doc, the same interface substantially improves retrieval F1, and every reranked pool exceeds PageIndex\. The gain on ColPali shows that the effect is not merely missing visual information for text retrievers\. Mechanism ablations show that snippet\-only LLM reranking barely changes first\-hit ranking, while visual captions recover most of the full lift on both text and visual pools\.Trident\-Sis a secondary generation extension: useful for open\-ended synthesis, but unnecessary for MCQ and short\-extractive workloads\. Overall, the results shift the focus from what the retriever scores to what the downstream reranker can see about each candidate page\.

## References

- Asaiet al\.\(2024\)A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. HajishirziSelf\-rag: learning to retrieve, generate, and critique through self\-reflection\.InInternational conference on learning representations,Vol\.2024,pp\. 9112–9141\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- \[2\]J\. Chen, S\. Xiao, P\. Zhang, K\. Luo, D\. Lian, and Z\. LiuBge m3\-embedding: multi\-lingual, multi\-functionality, multi\-granularity text embeddings through self\-knowledge distillation\.Cited by:[§1](https://arxiv.org/html/2608.14841#S1.p2.1),[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1),[§3\.4](https://arxiv.org/html/2608.14841#S3.SS4.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.14841#S4.SS1.p2.1)\.
- Chenet al\.\(2025\)Z\. Chen, C\. Xu, Y\. Qi, X\. Jiang, and J\. GuoVLM is a strong reranker: advancing multimodal retrieval\-augmented generation via knowledge\-enhanced reranking and noise\-injected training\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 8140–8158\.Cited by:[§1](https://arxiv.org/html/2608.14841#S1.p4.1),[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px4.p1.1)\.
- Choet al\.\(2024\)J\. Cho, D\. Mahata, O\. Irsoy, Y\. He, and M\. BansalM3docrag: multi\-modal retrieval is what you need for multi\-page multi\-document understanding\.arXiv preprint arXiv:2411\.04952\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1)\.
- Cormacket al\.\(2009\)G\. V\. Cormack, C\. L\. Clarke, and S\. BuettcherReciprocal rank fusion outperforms condorcet and individual rank learning methods\.InProceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval,pp\. 758–759\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1),[§3\.4](https://arxiv.org/html/2608.14841#S3.SS4.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.14841#S4.SS1.p2.1)\.
- Denget al\.\(2025\)C\. Deng, J\. Yuan, P\. Bu, P\. Wang, Z\. Li, J\. Xu, X\. Li, Y\. Gao, J\. Song, B\. Zheng,et al\.Longdocurl: a comprehensive multimodal long document benchmark integrating understanding, reasoning, and locating\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1135–1159\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.14841#S4.SS1.p1.1)\.
- Donget al\.\(2025\)K\. Dong, Y\. Chang, D\. G\. X\. Deik, D\. Li, R\. Tang, and Y\. LiuMMDocIR: benchmarking multimodal retrieval for long documents\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 30959–30993\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1)\.
- Donget al\.\(2026\)K\. Dong, C\. YUJING, S\. Huang, Y\. Wang, R\. Tang, and Y\. LiuBenchmarking retrieval\-augmented multimomal generation for document question answering\.Advances in Neural Information Processing Systems38\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1)\.
- Faysseet al\.\(2025\)M\. Faysse, H\. Sibille, T\. Wu, B\. Omrani, G\. Viaud, C\. Hudelot, and P\. ColomboColpali: efficient document retrieval with vision language models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 61424–61449\.Cited by:[§1](https://arxiv.org/html/2608.14841#S1.p2.1),[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1),[§3\.4](https://arxiv.org/html/2608.14841#S3.SS4.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.14841#S4.SS1.p2.1)\.
- Gondhalekaret al\.\(2025\)C\. Gondhalekar, U\. Patel, and F\. YehMultiFinRAG: an optimized multimodal retrieval\-augmented generation \(rag\) framework for financial question answering\.arXiv preprint arXiv:2506\.20821\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1)\.
- Gonget al\.\(2025\)Z\. Gong, Y\. Huang, and C\. MaiMmrag\-docqa: a multi\-modal retrieval\-augmented generation method for document question\-answering with hierarchical index and multi\-granularity retrieval\.arXiv e\-prints,pp\. arXiv–2508\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1)\.
- Gutiérrezet al\.\(2024\)B\. J\. Gutiérrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. SuHipporag: neurobiologically inspired long\-term memory for large language models\.Advances in neural information processing systems37,pp\. 59532–59569\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px3.p1.1)\.
- Jainet al\.\(2025\)C\. Jain, Y\. Wu, Y\. Zeng, J\. Liu, S\. Dai, Z\. Shao, Q\. Wu, and H\. WangSimpleDoc: multi\-modal document understanding with dual\-cue page retrieval and iterative refinement\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 28398–28415\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Khattab and Zaharia \(2020\)O\. Khattab and M\. ZahariaColbert: efficient and effective passage search via contextualized late interaction over bert\.InProceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval,pp\. 39–48\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§1](https://arxiv.org/html/2608.14841#S1.p1.1),[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2024\)M\. Li, É\. Gaussier, J\. Li, and G\. ZhouKeyB2: selecting key blocks is also important for long document ranking with large language models\.arXiv preprint arXiv:2411\.06254\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2026\)M\. Li, Y\. Zhang, D\. Long, K\. Chen, S\. Song, S\. Bai, Z\. Yang, P\. Xie, A\. Yang, D\. Liu,et al\.Qwen3\-vl\-embedding and qwen3\-vl\-reranker: a unified framework for state\-of\-the\-art multimodal retrieval and ranking\.arXiv preprint arXiv:2601\.04720\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px4.p1.1)\.
- Liet al\.\(2025\)Z\. Li, Y\. Li, L\. Kang, D\. Karatzas, and W\. MaAVIR: adaptive visual in\-document retrieval for efficient multi\-page document question answering\.InProceedings of the 7th ACM International Conference on Multimedia in Asia,pp\. 1–7\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Linet al\.\(2026\)J\. Lin, K\. Hu, B\. Wang, Y\. Zhou, Z\. Xi, H\. Guo, S\. Liu, J\. Wang, S\. Dou, E\. Zhou,et al\.MM\-doc\-r1: training agents for long document visual question answering through multi\-turn reinforcement learning\.arXiv preprint arXiv:2604\.13579\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Linet al\.\(2025\)S\. Lin, C\. Lee, M\. Shoeybi, J\. Lin, B\. Catanzaro, and W\. PingMm\-embed: universal multimodal retrieval with multimodal llms\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 44215–44234\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px4.p1.1)\.
- Liuet al\.\(2024\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the association for computational linguistics12,pp\. 157–173\.Cited by:[§1](https://arxiv.org/html/2608.14841#S1.p1.1)\.
- Maet al\.\(2024a\)X\. Ma, S\. Lin, M\. Li, W\. Chen, and J\. LinUnifying multimodal retrieval via document screenshot embedding\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 6492–6505\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Maet al\.\(2024b\)Y\. Ma, Y\. Zang, L\. Chen, M\. Chen, Y\. Jiao, X\. Li, X\. Lu, Z\. Liu, Y\. Ma, X\. Dong,et al\.Mmlongbench\-doc: benchmarking long\-context document understanding with visualizations\.Advances in Neural Information Processing Systems37,pp\. 95963–96010\.Cited by:[§1](https://arxiv.org/html/2608.14841#S1.p1.1),[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.14841#S4.SS1.p1.1)\.
- Mortahebet al\.\(2025\)M\. Mortaheb, M\. A\. A\. Khojastepour, S\. T\. Chakradhar, and S\. UlukusRe\-ranking the context for multimodal retrieval augmented generation\.arXiv preprint arXiv:2501\.04695\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px4.p1.1)\.
- Rahmanet al\.\(2025\)A\. Rahman, Q\. Xu, and X\. HuangMPCAR: multi\-perspective contextual augmentation for enhanced visual reasoning in large vision\-language models\.arXiv preprint arXiv:2508\.12400\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px5.p1.1)\.
- Saad\-Falconet al\.\(2024\)J\. Saad\-Falcon, J\. Barrow, A\. Siu, A\. Nenkova, S\. Yoon, R\. A\. Rossi, and F\. DernoncourtPdftriage: question answering over long, structured documents\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track,pp\. 153–169\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1)\.
- Sanmartin \(2024\)D\. SanmartinKg\-rag: bridging the gap between knowledge and creativity\.arXiv preprint arXiv:2405\.12035\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px3.p1.1)\.
- Santhanamet al\.\(2022\)K\. Santhanam, O\. Khattab, J\. Saad\-Falcon, C\. Potts, and M\. ZahariaColbertv2: effective and efficient retrieval via lightweight late interaction\.InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 3715–3734\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Sarthiet al\.\(2024\)P\. Sarthi, S\. Abdullah, A\. Tuli, S\. Khanna, A\. Goldie, and C\. ManningRaptor: recursive abstractive processing for tree\-organized retrieval\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 32628–32649\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Shiet al\.\(2026\)Y\. Shi, J\. Wang, Z\. Shan, D\. Peng, Z\. Lin, and L\. JinURaG: unified retrieval and generation in multimodal llms for efficient long document understanding\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 25357–25365\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Sunet al\.\(2023\)W\. Sun, L\. Yan, X\. Ma, S\. Wang, P\. Ren, Z\. Chen, D\. Yin, and Z\. RenIs chatgpt good at search? investigating large language models as re\-ranking agents\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 14918–14937\.Cited by:[§1](https://arxiv.org/html/2608.14841#S1.p3.1),[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px3.p1.1),[§3\.3](https://arxiv.org/html/2608.14841#S3.SS3.SSS0.Px3.p1.1)\.
- Tianet al\.\(2025\)R\. Tian, X\. Xu, B\. Jin, S\. Kang, and J\. HanCoRank: llm\-based compact reranking with document features for scientific retrieval\.arXiv preprint arXiv:2505\.13757\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px3.p1.1)\.
- Wassermanet al\.\(2025a\)N\. Wasserman, O\. Heinimann, Y\. Golbari, T\. Zimbalist, E\. Schwartz, and M\. IraniDocReRank: single\-page hard negative query generation for training multi\-modal rag rerankers\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 8651–8669\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px4.p1.1)\.
- Wassermanet al\.\(2025b\)N\. Wasserman, R\. Pony, O\. Naparstek, A\. R\. Goldfarb, E\. Schwartz, U\. Barzelay, and L\. KarlinskyReal\-mm\-rag: a real\-world multi\-modal retrieval benchmark\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 31660–31683\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1)\.
- Wuet al\.\(2025\)J\. Wu, Y\. Xia, T\. Yu, X\. Chen, S\. S\. Harsha, A\. V\. Maharaj, R\. Zhang, V\. Bursztyn, S\. Kim, R\. A\. Rossi,et al\.Doc\-react: multi\-page heterogeneous document question\-answering\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),pp\. 67–78\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1)\.
- Xuet al\.\(2025\)M\. Xu, J\. Dong, J\. Hou, Z\. Wang, S\. Li, Z\. Gao, R\. Zhong, and H\. CaiMm\-r5: multimodal reasoning\-enhanced reranker via reinforcement learning for document retrieval\.arXiv preprint arXiv:2506\.12364\.Cited by:[§1](https://arxiv.org/html/2608.14841#S1.p4.1),[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px4.p1.1)\.
- Yoonet al\.\(2026\)S\. Yoon, G\. Kim, G\. Cho,et al\.AcuRank: uncertainty\-aware adaptive computation for listwise reranking\.Advances in Neural Information Processing Systems38,pp\. 26388–26417\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px3.p1.1)\.
- Yuet al\.\(2025\)S\. Yu, C\. Tang, B\. Xu, J\. Cui, J\. Ran, Y\. Yan, Z\. Liu, S\. Wang, X\. Han, Z\. Liu,et al\.Visrag: vision\-based retrieval\-augmented generation on multi\-modality documents\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 21074–21098\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px4.p1.1)\.
- Zhanget al\.\(2025a\)M\. Zhang, Y\. Tang, and P\. TeamPageIndex: next\-generation vectorless, reasoning\-based rag\.PageIndex Blog\.Note:https://pageindex\.ai/blog/pageindex\-introCited by:[§1](https://arxiv.org/html/2608.14841#S1.p5.1),[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.14841#S4.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.14841#S4.SS1.p3.1)\.
- Zhanget al\.\(2025b\)Y\. Zhang, J\. Lin, M\. Zhang, K\. Xiao, X\. Hou, Y\. Zhao, and Z\. LiSCRA\-vqa: summarized caption\-rerank for augmented large language models in visual question answering\.InInternational Conference on Database Systems for Advanced Applications,pp\. 37–52\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px5.p1.1)\.
- Zhang and Wu \(2026\)Y\. Zhang and Y\. WuMLDocRAG: multimodal long\-context document retrieval augmented generation\.arXiv preprint arXiv:2602\.10271\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2026\)Y\. Zhou, F\. Zhang, Y\. Chen, H\. Zhang, P\. Nakov, and Z\. XieFinCARDS: card\-based analyst reranking for financial document question answering\.arXiv preprint arXiv:2601\.06992\.Cited by:[§2](https://arxiv.org/html/2608.14841#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix AAdditional End\-to\-End Results

This appendix complements §[4\.4](https://arxiv.org/html/2608.14841#S4.SS4)and §[4\.5](https://arxiv.org/html/2608.14841#S4.SS5)with the full end\-to\-end QA tables on both benchmarks\. Table[5](https://arxiv.org/html/2608.14841#A1.T5)reports all 14 evaluated pipelines on the 300\-QA MMLongBench\-Doc subset, including the answer\-accuracy and unanswerable\-abstention columns omitted from the main paper\. Table[6](https://arxiv.org/html/2608.14841#A1.T6)reports the corresponding end\-to\-end results on the 300\-QA LongDocURL stratified subset \(all questions answerable\), comparingTrident\-Ragainst the strongest baseline ColPali\+\+annotation\+\+rerank, each with and withoutTrident\-S\.

Table 5:Full end\-to\-end QA results on the 300\-QA MMLongBench\-Doc subset \(all values in %\)\.*J\.*=LLM\-as\-judge accuracy;*\+\+ann\.*=multi\-aspect annotation;*\+\+rk*=adaptive\-KKrerank;*Plain*=single multimodal VLM call;*\+\+Trident\-S*=three\-lens generation with synthesis\.Table 6:End\-to\-end QA on the 300\-QA LongDocURL stratified subset \(all values in %; all questions are answerable\)\. LongDocURL is dominated by MCQ\-style and short\-extractive answers, soTrident\-Sgives little average benefit\.
## Appendix BTrident\-RRerank Prompt

The full rerank prompt sent to GPT\-4\.1 is shown below\. Variables in angle brackets are filled at runtime\. The candidate JSON list \(<candidates\_json\>\) is the array of records described in §[3\.3](https://arxiv.org/html/2608.14841#S3.SS3);<query\>is the natural\-language question;<k\>is the maximum number of pages \(K=5K\{=\}5throughout this work\)\.

> You are picking the top\-K pages most likely to answer the question, from a candidate list\. Each candidate page comes with the document signals on it AND a list of ‘‘concept hits’’\-\-\-high\-level concepts that pulled it into the candidate pool\. Pages hit by multiple distinct concepts are stronger evidence\. Question: <query\> Candidate pages \(<n\> total\): <candidates\_json\> Reply as JSON: \{‘‘thinking’’: ‘‘…’’, ‘‘page\_list’’: \[<page\_id\>, …\]\} Pick the strongest evidence pages first\. Return at most <k\> pages\. Return only JSON\.

Each candidate record in<candidates\_json\>has the form:

> \{page\_id, retriever\_score, concept\_hits: \[‘‘T:<name\>’’, ‘‘E:<name\>’’, ‘‘S:<name\>’’\], section, visual, entities, text\}

Theretriever\_scorefield holds the underlying Stage\-1 pool score \(Okapi BM25 for BM25, cosine for BGE\-M3, RRF fused rank for RRF, late\-interaction MaxSim for ColPali, or our multi\-channelxKGfusion of Eq\.[10](https://arxiv.org/html/2608.14841#S3.E10)\); the field name is uniform across all pools so the rerank prompt template is pool\-agnostic\. \(In our released code the field is currently namedgraph\_scorefor historical reasons; the value semantics are as described\.\)

## Appendix CTrident\-SLens Prompts

Trident\-Sprompts the VLM three times in parallel with the three lenses below, then synthesizes a final answer with the synthesis prompt\. All four calls share the same retrieved page images\.

#### \(T\) Topical lens\.

> Question: <query\> You are given <n\> page images from a long document\. Read them through theTOPICAL lens: what is each page about thematically? Which topics on which pages relate to the question? \-\- Identify the topical content of each page\. \-\- Focus on how the topics connect to the question\. Provide a clear, concise answer based on the topical analysis\. If none of the provided pages contain information sufficient to answer the question through this topical lens, reply exactly: ‘‘Not answerable’’\.

#### \(E\) Entity / value lens\.

> Question: <query\> …Read them through theENTITY / VALUE lens: focus on the specific named entities, numerical values, dates, currencies, percentages, table cells, and figure labels that appear on each page\. \-\- Extract every named entity, number, date, or specific value\. \-\- Be precise\-\-\-use verbatim values from the images\. Provide a clear, concise answer focused on the specific entities and values surfaced\. If none of the provided pages contain the entities or values needed to answer the question, reply exactly: ‘‘Not answerable’’\.

#### \(S\) Structural / section lens\.

> Question: <query\> …Read them through theSTRUCTURAL / SECTION lens: focus on where each page sits in the document’s organization\-\-\-section titles visible on the page, headers, chapter / part markers, document structure, captions of figures / tables\. \-\- Identify each page’s structural role\. \-\- Use captions of tables/figures and section paths\. Provide a clear, concise answer that leverages structural context\. If the structural context does not contain information needed to answer the question, reply exactly: ‘‘Not answerable’’\.

#### Synthesis prompt\.

> You are answering a long\-document QA question\. We elicited three candidate answers from a vision\-language model under three orthogonal analytical lenses applied to the same K=<n\> retrieved page images: \(T\) Topical lens \(E\) Entity / value lens \(S\) Structural / section lens Question: <query\> Candidate answers: \[T\-view\] <ans\_T\> \[E\-view\] <ans\_E\> \[S\-view\] <ans\_S\> Task: produce a single, clear, concise final answer\. Synthesis rules: \-\- If two or three lenses converge, prefer that answer\. \-\- If only one lens has a precise answer while others are vague, prefer the specific one\-\-\-unless contradicted\. \-\- If question is numeric, prefer E\-view’s value\. \-\- If question is ‘‘where / which section’’, prefer S\-view\. \-\- If question is ‘‘what is the topic’’, prefer T\-view\. \-\- Match the question’s expected answer format\. Do not introduce information beyond what the three candidate answers contain\. Do not invent values\. If all three lenses agree the question is not answerable, reply exactly: ‘‘Not answerable’’\.

## Appendix DQuery\-Type Router

The router prompts an LLM \(GPT\-4\.1,τ=0\\tau\{=\}0, JSON mode\) to classify the question into one of four classes and to extract salient entities\. The full prompt is:

> You are routing a question against a multi\-view document index that has three concept types: topical \(semantic content clusters\), entity\-bundle \(groups of co\-occurring named entities/numbers\), and section\-derived \(structural sections\)\. Question: <query\> Output JSON with three fields: ‘‘entities’’: list of salient entities / key terms / numbers from the question\. Include named entities, technical terms, monetary amounts, percentages, figure / table references\. Drop generic stopwords\. ‘‘class’’: one of ‘‘entity\-level’’ \(asks about a specific named entity / number / figure\), ‘‘topical’’ \(asks for thematic content / overview / explanation\), ‘‘structural’’ \(asks about location / section / where in the document\), or ‘‘mixed’’ if it spans multiple\. ‘‘rationale’’: one short sentence explaining the class choice\. Reply with JSON only\.

The class label determines the channel\-fusion weights\(αT,αE,αS\)\(\\alpha\_\{T\},\\alpha\_\{E\},\\alpha\_\{S\}\)applied when aggregating topical / entity / structural concept scores into a candidate\-pool score:

Replacing this class\-conditional router with uniform weights\(13,13,13\)\(\\tfrac\{1\}\{3\},\\tfrac\{1\}\{3\},\\tfrac\{1\}\{3\}\)causes a small but consistent−0\.003\-0\.003F1 drop on the cross\-pool isolation setup \(§[4\.2](https://arxiv.org/html/2608.14841#S4.SS2), measured on the sameN=830N\{=\}830aligned subset as Table[1](https://arxiv.org/html/2608.14841#S4.T1)\), validating the router’s contribution\.

## Appendix E300\-QA Subset Construction

The 300\-QA subset is drawn from MMLongBench\-Doc with seed 42\. Sampling emphasizes the long\-document setting that motivates this work: the subset has median document length 72 pages, compared to 28 in the full set, aligning with our framework’s focus on documents that exceed any current VLM’s input window\. Table[7](https://arxiv.org/html/2608.14841#A5.T7)reports the per\-axis distribution match\.

Table 7:Distribution match between the 300\-QA subset and the full MMLongBench\-Doc dataset\. Most axes \(Evidence Source, Evidence Pages, Unanswerable rate\) match within±3\\pm 3pp\. Answer format is moderately biased toward Float \(\+5\.9\+5\.9pp\) and away from Str \(−6\.1\-6\.1pp\); document length is intentionally biased toward longer documents \(median 72 vs\. 28 pages\), aligning with our framework’s long\-document focus\.The LongDocURL subset of 200 QAs \(§[4\.5](https://arxiv.org/html/2608.14841#S4.SS5)\) is sampled analogously, stratified by document length, answer format, and evidence source\. Released QA\-id lists for both subsets accompany the code release\.

## Appendix FBootstrap CIs, Wilcoxon and McNemar tests

#### Retrieval \(N=830\)\.

We report \(i\) non\-parametric 95% bootstrap CIs on per\-QA F1 \(B=10,000B\{=\}10\{,\}000resamples drawn with replacement from the 830 aligned QAs\); and \(ii\) pairwise Wilcoxon signed\-rank tests on per\-QA F1 againstTrident\-R\(xKG pool\)\. We prefer Wilcoxon on F1 over McNemar on STRICT for retrieval because methods have different effectiveKK\(5 for fixed\-KKbaselines; meanK∈\[2\.7,3\.0\]K\\in\[2\.7,3\.0\]for adaptive\-KKmethods\), which would structurally favor the larger\-KKmethods\. CIs cluster tightly within±0\.024\\pm 0\.024\. All five raw baselines are significantly worse thanTrident\-R\(p<0\.001p<0\.001\)\. Among the\+\+ann\.\+\+rerank variants, RRF and BGE\-M3 swaps are statistically indistinguishable fromTrident\-R\(p=0\.077p\{=\}0\.077and0\.0550\.055\), the BM25 swap is borderline \(p=0\.026p\{=\}0\.026\), and the ColPali swap is significantly higher \(p<0\.001p<0\.001\)\. Taken together, all five\+\+ann\.\+\+rerank rows in Table[1](https://arxiv.org/html/2608.14841#S4.T1)fall within the\[0\.503,0\.581\]\[0\.503,0\.581\]F1 band without pairwise dominance \(except ColPali\), validating the pool\-agnostic claim\.

#### End\-to\-end \(N=300\)\.

For end\-to\-end Gen Acc on the 300\-QA MMLongBench\-Doc subset we use McNemar on a binary correct\-score \(≥1\\geq 1\) outcome per QA, with continuity correction\. None of the pairwise comparisons between top\-tier pipelines \(ColPali\+\+ann\.\+\+Trident\-S,Trident\-R\+\+Trident\-S, BGE\-M3\+\+ann\.\+\+Trident\-S, PageIndex\+\+Trident\-S\) reachp<0\.05p<0\.05atN=300N\{=\}300, reflecting the small sample size; however the ranking is consistent across all three LLM judges \(Appendix[L](https://arxiv.org/html/2608.14841#A12), inter\-judgeκ=0\.913\\kappa\{=\}0\.913\)\. Full pairwise results are released with our code\.

## Appendix GCase Study Records

We list the full records for the three case\-study QAs in §[G](https://arxiv.org/html/2608.14841#A7)\. All extracted answers are the GPT\-4o\-extracted short forms after the MMLongBench\-Doc official evaluation pipeline\. “Score” is the rule\-based Generalized Accuracy score \(1\.0 = correct, 0\.0 = wrong\)\.

#### Case 1: Chart axes\.

Doc2401\.18059v1\.pdf; Q:*What are the horizontal and vertical axis of Figure 3 respectively?*Gold:\[Context Length, F1\]\.

- •Trident\-R: retrieved pages\{6,5\}\\\{6,5\\\}, ext\. “\[Context Length, F1\]”, score 1\.0\.
- •PageIndex: retrieved\{3\}\\\{3\\\}, ext\. “Not answerable”, score 0\.0\.
- •BGE\-M3 raw: retrieved\{5,2,16,19,22\}\\\{5,2,16,19,22\\\}, ext\. “Not answerable”, score 0\.0\.

#### Case 2: Activision R&D ratio\.

DocACTIVISIONBLIZZARD\_2019\_10K\.pdf; Q:*What is R&D to asset ratio for Activision Blizzard in FY 2019?*Gold:5\.03%\.

- •Trident\-R: retrieved\{70,69\}\\\{70,69\\\}, ext\. “0\.0503”, score 1\.0\.
- •PageIndex: retrieved\{37,62,67,68,69\}\\\{37,62,67,68,69\\\}, ext\. “Not answerable”, score 0\.0\.
- •BGE\-M3 raw: retrieved\{47,29,92,174,175\}\\\{47,29,92,174,175\\\}, ext\. “Not answerable”, score 0\.0\.

#### Case 3: Calibrated refusal\.

Doc2311\.16502v3\.pdf; Q:*In which image type does GPT\-4o demonstrate least proficiency …?*Gold:Not answerable\.

- •Trident\-R: retrieved\{21,111,26\}\\\{21,111,26\\\}, ext\. “Not answerable”, score 1\.0\.
- •PageIndex: retrieved\{110,111,112\}\\\{110,111,112\\\}, ext\. “Chemical”, score 0\.0\.
- •BGE\-M3 raw: retrieved\{112,7,26,110,2\}\\\{112,7,26,110,2\\\}, ext\. “Music”, score 0\.0\.

## Appendix HDetailed Cost Breakdown

Token\-level cost accounting forTrident\-R\-Index, measured empirically on the 134\-document MMLongBench\-Doc corpus \(6,492 pages, mean103103text tokens per page,4040caption tokens per page\) at public OpenAI 2025 rates \($2\.00/$8\.00 per 1M input/output tokens for gpt\-4\.1; $0\.15/$0\.60 for gpt\-4o\-mini; $0\.13 per 1M tokens for text\-embedding\-3\-large; image tokens≈1500\\approx 1500per full page atZOOM=2\.0\\text\{ZOOM\}\{=\}2\.0\)\.

Component \(per 100\-page doc\)ModelCallsUSDVisual captiongpt\-4o\-mini1000\.028Entity extractiongpt\-4\.11000\.201Page embeddingemb\-3\-large1000\.001Concept naminggpt\-4\.1200\.026Offline total0\.256Online per\-query \(avgK=2\.85K\{=\}2\.85\):Stage 2 LLM rerankgpt\-4\.110\.010Plain VLM generationgpt\-4\.1 \(vision\)10\.012Trident\-S\(3 lens\+\+synth\)gpt\-4\.1 \(vision\)40\.039

Table 8:Empirical token\-level cost breakdown\. Entity extraction dominates offline indexing \(78%78\\%\); concept naming and visual captioning contribute∼10%\\sim 10\\%each; page embedding is negligible \(1%1\\%\)\.The reported$0\.256\\$0\.256per 100\-page document is within the∼$0\.30\\sim\\$0\.30range stated in the main text and is conservative for dense documents: doubling per\-page text tokens \(to∼200\\sim 200\) raises total cost to approximately$0\.36\\$0\.36\. Per\-query cost is dominated by the multimodal VLM call: Plain generation is$0\.012\\$0\.012,Trident\-Smulti\-view is$0\.039\\$0\.039\(3 lens calls\+\+1 synthesis\)\.

## Appendix IDetailed Field Ablation on Two Pools

We complement §[4\.2](https://arxiv.org/html/2608.14841#S4.SS2)with the full per\-field ablation on both the BGE\-M3 \(text\-only\) and ColPali \(visual\) pools\. Each row removes exactly one annotation field from the rerank candidate JSON\.

ConfigurationF1Δ\\DeltaF1RecallPrec\.BGE\-M3 pool \(text retriever,F1full=0\.546\\mathrm\{F1\}\_\{\\text\{full\}\}\{=\}0\.546\)\+\+full annotation0\.546—0\.7260\.499drop visual\_caption0\.416−0\.130\-0\.1300\.6570\.354drop concept\_hits0\.515−0\.032\-0\.0320\.6880\.477drop entities0\.507−0\.039\-0\.0390\.7010\.455caption\-only\(only visual\_caption\)0\.518−0\.028\-0\.0280\.6690\.484ColPali pool \(visual retriever,F1full=0\.581\\mathrm\{F1\}\_\{\\text\{full\}\}\{=\}0\.581\)\+\+full annotation0\.581—0\.7660\.533drop visual\_caption0\.498−0\.083\-0\.0830\.7150\.444drop concept\_hits0\.593\+0\.013\+0\.0130\.7510\.556drop entities0\.588\+0\.007\+0\.0070\.7730\.542caption\-only\(only visual\_caption\)0\.554−0\.027\-0\.0270\.7230\.516

Table 9:Per\-field annotation ablation on two retrievers \(N=830N\{=\}830aligned,Kmax=5K\_\{\\max\}\{=\}5\)\. Standard rows: remove one field at a time\.Caption\-only rows: keep only the visual caption \(drop concept\_hits, section path, entity tags, text snippet\)\. Caption alone achieves∼90%\{\\sim\}90\\%of the full annotation lift on both pools \(−0\.028\-0\.028/−0\.027\-0\.027from full\), with the gap remarkably pool\-invariant\. Combined with the asymmetric drop\-caption result \(−0\.130\-0\.130on BGE\-M3 vs\.−0\.083\-0\.083on ColPali;36%36\\%smaller on visual pool\), this rules out pure modality completion as the sole mechanism: the lift stems from structuring page evidence as LLM\-readable input, with visual caption acting as a*semantic interface*for the text\-only LLM reranker \(effective regardless of whether the upstream retriever is visual or textual\)\.
## Appendix JAdaptive\-KKDistribution by Question Type

We bin the rerank\-selectedKKby MMLongBench\-Doc question type \(using theevidence\_sourcesfield\) across four annotated\-rerank pools\.

Table 10:Mean adaptiveKKper question type, by candidate pool \(Kmax=5K\_\{\\max\}\{=\}5\)\. Two patterns: \(1\)KKis highly stable*across pools*\(range2\.552\.55–3\.113\.11\), suggesting that the reranker’sKKchoice is driven by the question’s evidence requirement, not by which retriever fed the pool\. \(2\)KKvaries*by question type*: Pure\-text questions request the most pages \(K¯≈3\.0\\bar\{K\}\\approx 3\.0,K=5K\{=\}5cap rate∼30%\\sim 30\\%\); Figure / Chart questions request fewer \(K¯≈2\.6\\bar\{K\}\\approx 2\.6,K=1K\{=\}1rate\>30%\>30\\%on Figure\), consistent with visual evidence being concentrated on individual pages\.
## Appendix KVisual Caption QC: Substring and N\-gram Overlap

To address the concern that VLM\-generated captions might leak the gold answer text and thereby trivialize Stage 2 reranking, we audit200200randomly sampled answerable QA / candidate\-page pairs \(N=238N\{=\}238caption\-page records after dropping “Text\-only page\.” captions\)\.

- •Exact substring\(gold answer string contained verbatim in caption\):22/238=9\.2%22/238=9\.2\\%\.
- •Bigram Jaccard≥0\.3\\geq 0\.3\(significant sentence\-level overlap\):0/238=0\.0%0/238=0\.0\\%\.
- •Unigram coverage≥0\.7\\geq 0\.7of gold tokens in caption:32/238=13\.4%32/238=13\.4\\%\.

The0%0\\%bigram match indicates no caption near\-verbatim paraphrases the answer sentence; the9\.2%9\.2\\%substring rate reflects captions correctly encoding visual content \(e\.g\. chart legend colours such as “Blue”, or diagram labels such as “Elastic Compute Service”\) that happen to coincide with the gold answer — the intended behaviour of caption\-as\-visual\-evidence rather than answer leakage\.

## Appendix LInter\-Judge Agreement

We use two independent LLM judges \(GPT\-4\.1 and Claude Sonnet 4\.5\) to score the same 300\-QA end\-to\-end gen outputs from each of the1414pipelines in Table[5](https://arxiv.org/html/2608.14841#A1.T5)\. Both judges use the identical33\-level rubric \(correct / partial / wrong\) and the same MMLongBench\-Doc gold answers\. Table[11](https://arxiv.org/html/2608.14841#A12.T11)reports per\-pipeline pairwise agreement and Cohen’sκ\\kappa\.

Table 11:Inter\-judge agreement and Cohen’sκ\\kappabetween GPT\-4\.1 and Claude on the same 300\-QA outputs\. Meanκ=0\.913\\kappa\{=\}0\.913corresponds to “almost perfect agreement” on the Landis\-Koch scale\. The pipeline\-level ranking under the two judges is identical \(ColPali\+\+ann\.\+\+rk\+\+Trident\-Sis best under both;Trident\-R\+\+Trident\-Ssecond under Claude, tied second under GPT\)\. This indicates that the observed end\-to\-end ranking is not dependent on a single judge’s quirks\.We do not perform human evaluation due to scale \(14×300=4,20014\\times 300=4\{,\}200prediction\-gold pairs\); we view the high inter\-judge agreement and the agreement of pipeline rankings between two independently developed LLMs as a reasonable proxy\.

## Appendix MExtended Limitations Discussion

We elaborate the limitations summarized in §[5](https://arxiv.org/html/2608.14841#S5)\.

#### Reranker LLM dependence\.

The rerank stage in this work uses GPT\-4\.1 atτ=0\\tau\{=\}0\. We expect the annotation\-conditioned rerank protocol to be reranker\-agnostic in principle, because the candidate JSON exposes per\-page evidence in a form that any instruction\-tuned LLM can consume\. A pilot replacement with a smaller reranker would falsify or strengthen this expectation; we leave a full small\-reranker / open\-source\-reranker sweep for follow\-up work due to budget constraints\.

#### LongDocURL coverage\.

LongDocURL serves as an external transfer check: the retrieval\-side comparison \(Table[4](https://arxiv.org/html/2608.14841#S4.T4)\) reports adaptive\-KKmethods on the fullN=1,122N\{=\}1\{,\}122QAs with non\-empty gold evidence, but the full five\-pool cross\-pool isolation conducted on MMLongBench\-Doc is not separately replicated on LongDocURL\. End\-to\-end LongDocURL results \(Appendix[A](https://arxiv.org/html/2608.14841#A1), Table[6](https://arxiv.org/html/2608.14841#A1.T6)\) confirm the broad pattern\.

#### Trident\-Son extractive workloads\.

On LongDocURL’s MCQ\-style and short\-extractive questions, the topical, entity, and structural lenses converge on the same extracted answer; the synthesis call therefore contributes meanΔ≈0\\Delta\\approx 0across four LongDocURL pipelines \(Appendix[A](https://arxiv.org/html/2608.14841#A1), Table[6](https://arxiv.org/html/2608.14841#A1.T6)\)\. This motivates the on\-demand deployment regime described in §[3\.5](https://arxiv.org/html/2608.14841#S3.SS5):Trident\-Sis gated by question format\.

#### Parser substitution\.

We use the PageIndex API as a structural parser for the section\-path field while also benchmarking against the PageIndex retrieval system; the two roles are independent\. Any public PDF parser that yields a section tree \(PyMuPDF, GROBID\) could fill this role\. A parser\-substitution robustness study is left to future work\.