MCite-RL: Towards Reliable Multimodal RAG via Citation-enhanced Agentic Reinforcement Learning

# MCite-RL: Towards Reliable Multimodal RAG via Citation-enhanced Agentic Reinforcement Learning
Source: [https://arxiv.org/html/2608.21808](https://arxiv.org/html/2608.21808)
Suifeng ZhaoZida LiuAffiliation:Key Laboratory of High Confidence Software Technologies, CS, Peking University, ChinaXinyu LeiAffiliation:Key Laboratory of Computational Linguistics\(MOE\), CS, Peking University, ChinaLei SunAffiliation:Panasonic Connect Co\., Ltd\. Tokyo, Japan\{sfzhao25,zdliu25,xinyulei25\}@stu\.pku\.edu\.cn,sun\.lei@jp\.panasonic\.com,\{gaojun,lisujian\}@pku\.edu\.cnJun Gao, Sujian LiThanks:Corresponding AuthorsAffiliation:Key Laboratory of Computational Linguistics\(MOE\), CS, Peking University, China

###### Abstract

Multimodal Retrieval\-Augmented Generation \(RAG\) with visual citation is crucial for ensuring the traceability and verifiability of MLLMs\. However, current RAG and SFT\-based methods struggle to achieve robust cross\-modal reasoning, causing imprecise visual citations or decoupling between the citation and the generated answers\. To address these limitations, we propose MCite\-RL, a citation\-enhanced agentic reinforcement learning framework designed for reliable multimodal RAG\. MCite\-RL introduces an Agentic Refinement module for visual citation that employs iterative retrieval, reasoning, and recursive cropping to progressively narrow the search space, transforming citation into a dynamic, evidence\-driven reasoning process rather than a static step\. Furthermore, we incorporate a Citation\-enhanced Reward mechanism that integrates both process\-level and outcome\-level feedback within a reinforcement learning paradigm to jointly optimize answer accuracy and source traceability\. Extensive experiments on benchmarks such as Wiki\-VISA, FinRAGBench\-V, and MMLongBench\-Doc demonstrate that MCite\-RL effectively achieves the joint optimization of citation precision and answer quality\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.21808v1/intro_fig_7.png)Figure 1:Typical success and failure patterns of visual citation in multimodal RAG\. A reliable case \(top\) exhibits precise visual citation that directly supports the generated answer\. In contrast, existing models frequently suffer from imprecise citations \(middle\) or scenarios where citations become decoupled from the generated answer \(bottom\)\.Recent advances in Multimodal Large Language Models \(MLLMs\) have enabled Retrieval\-Augmented Generation \(RAG\) systems\([5](https://arxiv.org/html/2608.21808#bib.bib6);[9](https://arxiv.org/html/2608.21808#bib.bib7);[29](https://arxiv.org/html/2608.21808#bib.bib8);[35](https://arxiv.org/html/2608.21808#bib.bib9)\)to move beyond text\-only settings and incorporate visual information into the generation process, allowing models to produce richer and more informative responses\([25](https://arxiv.org/html/2608.21808#bib.bib12);[36](https://arxiv.org/html/2608.21808#bib.bib10);[21](https://arxiv.org/html/2608.21808#bib.bib11)\)\. Similar to text\-based RAG systems\([4](https://arxiv.org/html/2608.21808#bib.bib35);[41](https://arxiv.org/html/2608.21808#bib.bib36);[38](https://arxiv.org/html/2608.21808#bib.bib37)\), accurate and well\-grounded citations are critical for multimodal RAG\([20](https://arxiv.org/html/2608.21808#bib.bib38)\), as they improve the verifiability and traceability of generated outputs by explicitly linking responses to the underlying visual evidence\.

Multimodal RAG with Visual Citation has recently attracted initial research interest as a framework aimed at enhancing both accuracy and verifiability\. Initial efforts have primarily focused on defining the task and establishing preliminary benchmarks\. For instance, VISA[14](https://arxiv.org/html/2608.21808#bib.bib2)pioneers the concept of visual source attribution by linking answers to fine\-grained visual evidence through SFT\-based methods, and FinRAGBench\-V[40](https://arxiv.org/html/2608.21808#bib.bib1), which extends this task by introducing automated evaluation for visual citations in complex documents\. For these benchmarks, the adopted baseline methods are still relatively simple, typically relying on vanilla RAG or SFT\-based approaches to implement basic citation capabilities\.

Despite these emerging efforts, we argue that multimodal RAG with visual citation entails more fundamental challenges that remain insufficiently explored\.First, there remains a substantial gap in citation precision\. This difficulty stems from the need to localize relevant evidence at an appropriate level of granularity, where both overly coarse and overly fine\-grained citation can lead to incorrect visual support\([12](https://arxiv.org/html/2608.21808#bib.bib40);[14](https://arxiv.org/html/2608.21808#bib.bib2);[37](https://arxiv.org/html/2608.21808#bib.bib41)\)\. As illustrated in Figure[1](https://arxiv.org/html/2608.21808#S1.F1), MLLMs frequently fail to localize the correct visual evidence, generating bounding boxes that are spatially misaligned, too coarse, or too detailed\. Such imprecise citations fail to serve as verifiable evidence\.Second, visual citations are often misaligned with the generated answers they are meant to support\. As shown in the bottom example of Figure[1](https://arxiv.org/html/2608.21808#S1.F1), the model may match a numerical value yet cite a visual region from an entirely different context\. This mismatch reveals that the answer and the citation lack a shared evidence basis, fundamentally undermining the trustworthiness of citation\-enabled multimodal RAG systems\.

Based on the above analysis, we conclude that a reliable multimodal RAG system requires far more beyond accurate answer, it also requires precise evidence localization and consistent answer\-citation alignment\. However, achieving such fine\-grained citation in complex multimodal documents is inherently difficult within a static, single\-step generation paradigm\([38](https://arxiv.org/html/2608.21808#bib.bib37);[20](https://arxiv.org/html/2608.21808#bib.bib38)\)\. Existing approaches, whether based on prompting or static supervised fine\-tuning, often fail to provide the appropriate granular feedback necessary to align these objectives\. We require both a holistic feedback mechanism to ensure citation reliability and an iterative agentic workflow to progressively refine visual evidence\. Motivated by this insight, we proposeMCite\-RL, a novel citation\-enhanced agentic reinforcement learning framework\. Unlike previous methods, we explicitly treat citation precision as a primary optimization objective by incorporating it as a dedicated component in the reward function, rather than considering it treating it merely as a by\-product of answer generation\. Inspired by prior work on iterative retrieval and reasoning \(e\.g\., Search\-R1[8](https://arxiv.org/html/2608.21808#bib.bib4); VRAG\-RL[26](https://arxiv.org/html/2608.21808#bib.bib3)\), we also design an agentic refinement workflow for visual citation, which progressively narrows the visual search space through multi\-step evidence refinement, allowing answers and their corresponding citations to emerge from a shared, evidence\-driven reasoning process\.

We evaluate the effectiveness of our method on three benchmarks, including Wiki\-VISA[14](https://arxiv.org/html/2608.21808#bib.bib2), FinRAGBench\-V[40](https://arxiv.org/html/2608.21808#bib.bib1), and MMLongBench\-Doc[15](https://arxiv.org/html/2608.21808#bib.bib5)\. Experimental results show that MCite\-RL consistently improves citation precision across all benchmarks, achieving an average gain of 26\.16% on the Intersection\-over\-Union \(IoU\) metric compared to the vanilla RAG baseline\. Moreover, the integration of citation\-based rewards also leads to an average increase of 5\.89% in answer accuracy, demonstrating that explicit citation supervision directly contributes to answer reliability\.

Our contributions are summarized as follows:

- •We propose MCite\-RL, a framework that improves the reliability of multimodal RAG by integrating citation\-enhanced agentic reinforcement learning\.
- •We introduce an agentic workflow for multimodal RAG with visual citation, where iterative evidence refinement enables answers and citations to emerge from a shared reasoning process\.
- •We develop a citation\-enhanced reward function that explicitly optimizes both reasoning accuracy and citation precision\. By incorporating citation feedback at multiple levels, our method leverages citation precision as a central driver for model training\.
- •Extensive experiments demonstrate the effectiveness of our approach, showing improvements not only in citation precision but also in answer accuracy\.

## 2Related Work

#### Multimodal RAG with Visual Citation

Early Multimodal RAG systems primarily focused on coarse\-grained retrieval over documents or images, aiming to augment generation with multimodal context\([32](https://arxiv.org/html/2608.21808#bib.bib16);[3](https://arxiv.org/html/2608.21808#bib.bib15);[13](https://arxiv.org/html/2608.21808#bib.bib14)\)\. More recent studies have shifted toward visual citation, which requires grounding generated content in fine\-grained image regions to enable traceability and verification\. VISA[14](https://arxiv.org/html/2608.21808#bib.bib2)formalized this setting through visual source attribution by aligning generated text spans with detected visual regions, while FinRAGBench\-V[40](https://arxiv.org/html/2608.21808#bib.bib1)systematically evaluated visual citation performance in visually dense financial charts and tables\. Despite these advances, most existing approaches rely on static and passive grounding pipelines, where visual regions are pre\-selected prior to generation and loosely aligned afterward, limiting citation precision and robustness in complex multimodal reasoning scenarios\.

#### Agentic Reasoning and Reinforcement Learning

The rise of agentic Large Language Models has transformed RAG from one\-shot retrieval into iterative reasoning–action loops\([33](https://arxiv.org/html/2608.21808#bib.bib34);[27](https://arxiv.org/html/2608.21808#bib.bib33);[22](https://arxiv.org/html/2608.21808#bib.bib32)\), exemplified by the ReAct paradigm[34](https://arxiv.org/html/2608.21808#bib.bib13)and subsequent efforts on agentic reasoning\([30](https://arxiv.org/html/2608.21808#bib.bib21);[24](https://arxiv.org/html/2608.21808#bib.bib22);[10](https://arxiv.org/html/2608.21808#bib.bib23);[11](https://arxiv.org/html/2608.21808#bib.bib24);[39](https://arxiv.org/html/2608.21808#bib.bib28);[7](https://arxiv.org/html/2608.21808#bib.bib27);[19](https://arxiv.org/html/2608.21808#bib.bib26);[6](https://arxiv.org/html/2608.21808#bib.bib25);[31](https://arxiv.org/html/2608.21808#bib.bib31);[28](https://arxiv.org/html/2608.21808#bib.bib30)\)\. In text\-only setting, Search\-R1[8](https://arxiv.org/html/2608.21808#bib.bib4)employs chain\-of\-thought reasoning to refine retrieval queries, while in multimodal scenarios, VRAG\-RL[26](https://arxiv.org/html/2608.21808#bib.bib3)incorporates reinforcement learning to iteratively crop and attend to visual regions\. However, these methods predominantly optimize answer correctness or task\-level rewards, treating visual grounding as an implicit intermediate signal rather than an explicit optimization target\. Consequently, models may internally attend to relevant regions without producing precise, verifiable visual citations\.

![Refer to caption](https://arxiv.org/html/2608.21808v1/main_fig_5.png)Figure 2:Architecture of MCite\-RL\.It features a two\-stage pipeline: \(1\) Trajectory Synthesis with rigorous quality control for supervised fine\-tuning \(SFT\), and \(2\) RL\-based Agentic Refinement for visual citation, which leverages multi\-step rollouts \(retrieval and cropping\) and citation\-enhanced rewards to jointly optimize answer accuracy and citation precision\.

## 3MCite\-RL

Figure[2](https://arxiv.org/html/2608.21808#S2.F2)illustrates the overall architecture of MCite\-RL, which consists of two core components: an agentic refinement module for visual citation \(Section[3\.2](https://arxiv.org/html/2608.21808#S3.SS2)\) and a citation\-enhanced reward calculation module \(Section[3\.3](https://arxiv.org/html/2608.21808#S3.SS3)\)\. Based on this architecture, the model is trained via a two\-stage process \(Section[3\.4](https://arxiv.org/html/2608.21808#S3.SS4)\) comprising SFT Cold\-Start and Citation\-enhanced RL\.

### 3\.1Task Formulation of Multimodal RAG with Citation

We formally define the task of Multimodal Retrieval\-Augmented Generation with Citation \(MMRAG with Citation\) as a constrained generation problem\. Given a user queryqqand an initial imageii, the objective is to generate a response tuple\(y,cf​i​n​a​l\)\(y,c\_\{final\}\), whereyyrepresents the textual answer andcf​i​n​a​l=\{\(x1,y1\),\(x2,y2\)\}c\_\{final\}=\\\{\(x\_\{1\},y\_\{1\}\),\(x\_\{2\},y\_\{2\}\)\\\}denotes the visual citation in the form of bounding box coordinates withiniithat provide precise visual evidence for the answeryy\. The overall process can be formulated as a function:

ℱ:\(q,i\)→\(y,cf​i​n​a​l\)\.\\mathcal\{F\}:\(q,i\)\\to\(y,c\_\{final\}\)\.\(1\)
To effectively navigate the vast search space and ensure grounding precision, the generation process is decomposed into the synergistic interplay between iterative retrieval and progressive visual cropping\. Unlike standard VQA, this task requires the joint optimization of two critical objectives:answer accuracy, which measures the fidelity of the answeryyto the queryqq, andcitation precision, which ensures the spatial alignment between the citationcf​i​n​a​lc\_\{final\}and the corresponding segments inyy\. Since optimizingℱ\\mathcal\{F\}involves mastering the interplay between reasoning and tool manipulation, we adopt an agentic refinement workflow to manage this iterative process\.

### 3\.2Agentic Workflow for Visual Citation

As illustrated in lower\-left part of Figure[2](https://arxiv.org/html/2608.21808#S2.F2), we employ an agentic reasoning pipeline to MMRAG outputs with precise visual citations\. While iterative retrieval and cropping have been explored to enhance general QA performance\([8](https://arxiv.org/html/2608.21808#bib.bib4);[26](https://arxiv.org/html/2608.21808#bib.bib3);[16](https://arxiv.org/html/2608.21808#bib.bib29)\), we adapt this workflow to enforce both answer accuracy and citation precision\.

Within our framework, this iterative process progressively collects the visual and textual clues needed to answer the question\. By alternating between image retrieval and visual grounding, the agent successively crops the visual field by pruning irrelevant regions, thereby narrowing its focus onto the precise coordinates required to anchor the final answer\.

Iterative Visual Grounding\.We formulate inference as a sequential decision\-making trajectory:

τ=\{\(s0,a0\),\(s1,a1\),…,\(sT,aT\)\},\\tau=\\\{\(s\_\{0\},a\_\{0\}\),\(s\_\{1\},a\_\{1\}\),\\dots,\(s\_\{T\},a\_\{T\}\)\\\},\(2\)wherests\_\{t\}denotes the agent’s state at stepttandata\_\{t\}denotes the corresponding action\. The action space includes image retrieval, visual cropping, and generating the final answer with visual citations\.

Each statests\_\{t\}is represented as:

st=\(q,ht,vct\),s\_\{t\}=\(q,h\_\{t\},v\_\{c\_\{t\}\}\),\(3\)whereqqis the input query,hth\_\{t\}summarizes the interaction history up to steptt\(encompassing past actions and retrieved contexts\), andvctv\_\{c\_\{t\}\}is the visual observation, which is cropped based on the generated coordinatesctc\_\{t\}at steptt\.

As shown in the lower\-left part of Figure[2](https://arxiv.org/html/2608.21808#S2.F2), the agent progressively refines its visual observation through iterative retrieval and cropping\. The goal of this refinement is to identify an optimal visual regionv^ct\\hat\{v\}\_\{c\_\{t\}\}\. This region is selected to maximize the likelihood of the evidence given the interaction history and the query:

v^ct=arg⁡maxv⁡P⁡\(v∣τ<t,q\)\\hat\{v\}\_\{c\_\{t\}\}=\\arg\\max\_\{v\}P\(v\\mid\\tau\_\{<t\},q\)\(4\)whereτ<t=\{\(s0,a0\),…,\(st−1,at−1\)\}\\tau\_\{<t\}=\\\{\(s\_\{0\},a\_\{0\}\),\\dots,\(s\_\{t\-1\},a\_\{t\-1\}\)\\\}represents the historical trajectory of states and actions\.

#### Joint Answer and Citation Generation\.

Based on the final converged visual regionvcTv\_\{c\_\{T\}\}, the model jointly generates the textual answeryyand its corresponding visual citationcf​i​n​a​lc\_\{final\}in a single step:

\(y^,c^f​i​n​a​l\)=argmaxy,cf​i​n​a​lP\(y,cf​i​n​a​l∣vcT,q\)\(\\hat\{y\},\\hat\{c\}\_\{final\}\)=\\arg\\max\_\{y,c\_\{final\}\}P\(y,c\_\{final\}\\mid v\_\{c\_\{T\}\},q\)\(5\)Both the answer and the citation are derived from the shared visual context, with the citation spatially localized withinvcTv\_\{c\_\{T\}\}\. This joint generation mechanism ensures that the answer is intrinsically grounded in visual evidence, establishing strong alignment between the answer and its supporting region while mitigating the risk of misaligned or hallucinated citations\.

### 3\.3Citation\-enhanced Reward

In MCite\-RL, we design a citation\-enhanced reward function tailored to the agentic interaction trajectory described in Section[3\.2](https://arxiv.org/html/2608.21808#S3.SS2)\. This reward jointly captures two aspects:process\-levelintermediate grounding feedback andoutcome\-levelcitation precision, thereby incentivizing the generation of precise visual citation\.

#### Process\-Level Citation Reward\.

To supervise the dynamic reasoning trajectory, we introduce the process\-level rewardRcitprocR\_\{\\text\{cit\}\}^\{\\text\{proc\}\}, which evaluates the quality of the converged search space during the agentic loop\. While the agent performs multiple “think\-and\-crop” iterations and gradually narrow the search space, this reward is finally anchored on the terminal cropcTc\_\{T\}\. We employ the Intersection\-over\-Ground\-truth \(IoGT\) metric to quantify this reward:

Rcitproc=IoGT​\(cT,c∗\)=\|cT∩c∗\|\|c∗\|,R\_\{\\text\{cit\}\}^\{\\text\{proc\}\}=\\text\{IoGT\}\(c\_\{T\},c^\{\*\}\)=\\frac\{\|c\_\{T\}\\cap c^\{\*\}\|\}\{\|c^\{\*\}\|\},\(6\)wherec∗c^\{\*\}denotes the ground\-truth citation and\|⋅\|\|\\cdot\|represents the region area\. Unlike the outcome\-level reward that emphasizes precise localization, the process\-level IoGT specifically incentivizes the preservation of core evidence throughout the visual grounding refinement process\. By providing this supervision signal, the reward penalizes any information loss during the iterative pruning of the visual field\. This ensures that the agent maintains a high\-recall focus on the ground\-truth region, thereby establishing a robust and undistorted evidence foundation for the subsequent generation of the textual answer and its corresponding visual citation\.

#### Outcome\-Level Citation Reward\.

The outcome\-level rewardRcitoutR\_\{\\text\{cit\}\}^\{\\text\{out\}\}evaluates the precision of the final predicted citationccagainst the ground\-truthc∗c^\{\*\}using Intersection\-over\-Union \(IoU\):

Rcitout​\(cf​i​n​a​l,c∗\)=IoU​\(cf​i​n​a​l,c∗\)=\|cf​i​n​a​l∩c∗\|\|cf​i​n​a​l∪c∗\|\.R\_\{\\text\{cit\}\}^\{\\text\{out\}\}\(c\_\{final\},c^\{\*\}\)=\\text\{IoU\}\(c\_\{final\},c^\{\*\}\)=\\frac\{\|c\_\{final\}\\cap c^\{\*\}\|\}\{\|c\_\{final\}\\cup c^\{\*\}\|\}\.\(7\)Unlike the process\-level citation reward, which emphasizes full coverage of the ground truth in intermediate crops, the outcome\-level reward focuses on the accuracy of the final prediction\. An ideal final citation should be tightly aligned with the ground truth, capturing all necessary information while excluding irrelevant regions\. By enforcing this IoU\-based criterion, we aim to make the final citation both accurate and verifiable\. Together with the process\-level reward, this design guides the agent to progressively hone in on the relevant region during intermediate steps, while ultimately producing a highly precise final citation, thereby reinforcing the structural alignment between the answer and its supporting evidence\.

#### Final Reward Function\.

To achieve holistic optimization, the final reward functionR⁡\(τ\)R\(\\tau\)integrates retrieval efficiency, answer accuracy, and citation rewards at both outcome and process levels, along with the format penalty\. We define the total rewardR⁡\(τ\)R\(\\tau\)as follows:

R⁡\(τ\)=𝕀v​\(τ\)⋅\(α​Rret\+β​Rans\+γ1​Rcitout\+γ2​Rcitproc\)R\(\\tau\)=\\mathbb\{I\}\_\{\\text\{v\}\}\(\\tau\)\\cdot\(\\alpha R\_\{\\text\{ret\}\}\+\\beta R\_\{\\text\{ans\}\}\+\\gamma\_\{1\}R\_\{\\text\{cit\}\}^\{\\text\{out\}\}\+\\gamma\_\{2\}R\_\{\\text\{cit\}\}^\{\\text\{proc\}\}\)\(8\)where each component is defined below\. All reward components are normalized to the range\[0,1\]\[0,1\]\.

- •RretR\_\{\\text\{ret\}\}represents theretrieval efficiency reward\. Inspired by[26](https://arxiv.org/html/2608.21808#bib.bib3), we employ nDCG as the metric, to encourage the agent to acquire the relevant image in the earliest possible steps during iterative retrieval\.
- •RansR\_\{\\text\{ans\}\}represents theanswer accuracy reward, calculated based on Exact Match \(EM\) between the generated response and the ground\-truth\.
- •RcitoutR\_\{\\text\{cit\}\}^\{\\text\{out\}\}andRcitprocR\_\{\\text\{cit\}\}^\{\\text\{proc\}\}are thecitation precision rewardsat the outcome and process levels respectively, as described above\.
- •𝕀v​\(τ\)∈\{0,1\}\\mathbb\{I\}\_\{\\text\{v\}\}\(\\tau\)\\in\\\{0,1\\\}acts as aformat constraint\. A rollout trajectory is valid if it covers the required agentic sequence: retrieval, cropping, and final response with answer and visual citation\.

### 3\.4Training Method

#### SFT Cold\-Start\.

To adapt the model to the required agentic reasoning format and establish a high\-quality initial policy for subsequent reinforcement learning, we conduct a cold\-start phase using supervised fine\-tuning \(SFT\)\. As existing multi\-modal datasets generally lack explicit reasoning trajectories, we synthesize data that formalizes the chain of thought for both textual answers and visual citations\. Specifically, given an input queryqq, the ground\-truth answery∗y^\{\*\}, and golden citation coordinatesc∗c^\{\*\}, we leverage GPT\-5 to generate step\-wise agentic trajectories\. These trajectories adhere strictly to a predefined workflow: iterative image retrieval, visual cropping, and final answer generation integrated with visual citations\. To ensure data fidelity, we apply rigorous filtering criteria \(detailed in Table[5](https://arxiv.org/html/2608.21808#A2.T5)of Appendix[B](https://arxiv.org/html/2608.21808#A2)\) to exclude hallucinated or format\-violating samples\. This process enables the model to internalize the mandatory structural sequence prior to the RL phase\.

#### Citation\-enhanced RL\.

Building upon the SFT\-tuned model, we transition to reinforcement learning to further optimize the agent’s accuracy and citation precision\. We employ the Group Relative Policy Optimization \(GRPO\) algorithm[18](https://arxiv.org/html/2608.21808#bib.bib39), which optimizesπθ\\pi\_\{\\theta\}by maximizing:

JG​R​P​O​\(θ\)\\displaystyle J\_\{GRPO\}\(\\theta\)=𝔼⁡\[q∼P⁡\(Q\),\{τi\}i=1G∼πθold\]\\displaystyle=\\mathbb\{E\}\\big\[q\\sim P\(Q\),\\\{\\tau\_\{i\}\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\theta\_\{\\text\{old\}\}\}\\big\]\(9\)1G∑i=1G\(ℒclip\(θ\)−β𝔻K​L\(πθ\|\|πr​e​f\)\)\\displaystyle\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\big\(\\mathcal\{L\}\_\{\\text\{clip\}\}\(\\theta\)\-\\beta\\mathbb\{D\}\_\{KL\}\(\\pi\_\{\\theta\}\|\|\\pi\_\{ref\}\)\\big\)where the advantageA^i\\hat\{A\}\_\{i\}inℒclip\\mathcal\{L\}\_\{\\text\{clip\}\}is computed by normalizing our citation\-enhanced rewardR⁡\(τi\)R\(\\tau\_\{i\}\)\(as defined in Eq\.[8](https://arxiv.org/html/2608.21808#S3.E8)\) within the group:

A^i=R⁡\(τi\)−mean​\(\{R⁡\(τj\)\}j=1G\)std​\(\{R⁡\(τj\)\}j=1G\)\\hat\{A\}\_\{i\}=\\frac\{R\(\\tau\_\{i\}\)\-\\text\{mean\}\(\\\{R\(\\tau\_\{j\}\)\\\}\_\{j=1\}^\{G\}\)\}\{\\text\{std\}\(\\\{R\(\\tau\_\{j\}\)\\\}\_\{j=1\}^\{G\}\)\}\(10\)By directly optimizing for these rewards, MCite\-RL achieves a synergistic improvement of precise visual citation and answer accuracy\.

Table 1:Main results of MCite\-RL compared with baseline methods on Wiki\-Visa, FinRAGBench\-V, and MMLongBench\-Doc\. NOTE: the proprietary models are only tested under the setting of MCite\-RL \(w/o Training\)\.Boldandunderlinedenote the best and second\-best performance, respectively\. Theblue\-shadedrows represent our method and its variants\.

## 4Experimental Settings

### 4\.1Datasets

To evaluate performance across diverse domains, we conduct experiments on subsets of three multimodal RAG benchmarks, specifically filtered single\-image samples to evaluate performance across diverse domains: \(i\)Wiki\-VISA[14](https://arxiv.org/html/2608.21808#bib.bib2): An open\-domain dataset requiring long image pages retrieval from Wikipedia; \(ii\)FinRAGBench\-V[40](https://arxiv.org/html/2608.21808#bib.bib1): A multimodal financial domain benchmark featuring rich charts and tables; \(iii\)MMLongBench\-Doc[15](https://arxiv.org/html/2608.21808#bib.bib5): A long\-context benchmark where we focus on document understanding tasks\. For citation evaluation, Wiki\-VISA is assessed using the original annotations; FinRAGBench\-V is evaluated on a provided subset with citation ground truth; and MMLongBench\-Doc is evaluated on 300 samples manually annotated and verified\.Detailed statistics of datasets are in Appendix[D\.4](https://arxiv.org/html/2608.21808#A4.SS4)\.

### 4\.2Baselines and Model Variants

To evaluate its effectiveness, we compare MCite\-RL against three categories of methods:

\(i\)Proprietary Models: We evaluate state\-of\-the\-art closed\-source models \(GPT\-4o[17](https://arxiv.org/html/2608.21808#bib.bib18), Gemini\-2\.0\-Flash[2](https://arxiv.org/html/2608.21808#bib.bib19), Claude\-4\.5\-Sonnet[1](https://arxiv.org/html/2608.21808#bib.bib20)\) in a zero\-shot setting, using the identical prompting strategy as our “MCite\-RL \(w/o training\)” setting\.

\(ii\)Reproduced Strong Baselines: We implement standard paradigms \(Vanilla RAG, ReAct\) and reproduce strong baselines \(Search\-R1, and VRAG\-RL\) using the Qwen2\.5\-VL\-Instruct models \(3B and 7B\) trained on our specific dataset\.

\(iii\)Ablation Variants: We evaluate MCite\-RL and its different training variants \(w/o Proc, w/o RL, and w/o Training, etc\.\) to analyze the individual contributions of each component\. Specifically,w/o Procremoves the process\-level citation rewards, utilizing only the final answer correctness for optimization\.w/o RLrefers to the model derived purely from Supervised Fine\-Tuning \(SFT\) without the subsequent RL training stage\. Finally,w/o Trainingdenotes the direct inference results from the off\-the\-shelf base model\.

### 4\.3Evaluation Metrics

We evaluate performance along two dimensions: answer accuracy and citation precision\.

#### Answer Accuracy\.

To ensure a fair comparison, we follow the evaluation protocols defined by each original benchmark, incorporating both rule\-based metrics and model\-based semantic judgments\. Detailed mappings of metrics to specific datasets are provided in Appendix[D\.5](https://arxiv.org/html/2608.21808#A4.SS5)\.

#### Citation Precision\.

To quantify the precision of visual citation, we measure the alignment between predicted bounding boxes and ground\-truth annotations using the average Intersection over Union \(IoU\)\.

### 4\.4Training Details

We implement our MCite\-RL framework using Qwen2\.5\-VL\-3B\-Instruct and Qwen2\.5\-VL\-7B\-Instruct[23](https://arxiv.org/html/2608.21808#bib.bib17)as backbone models, and employ ColQwen[3](https://arxiv.org/html/2608.21808#bib.bib15)as our retriever model\. The training pipeline consists of two stages, both utilizing subsets of the Wiki\-VISA dataset, while the other two datasets are reserved for out\-of\-domain \(OOD\) evaluation\.

We first conduct Supervised Fine\-Tuning \(SFT\) on 2,417 curated samples to establish the basic citation capabilities\. Then, we further optimize the agent policy via Group Relative Policy Optimization \(GRPO\) using a dataset of 1,000 samples to align the model with our citation\-enhanced rewards objectives\. Detailed hyperparameters and hardware configurations are provided in Appendix[D](https://arxiv.org/html/2608.21808#A4)\.

## 5Experimental Results and Analysis

### 5\.1Main Experiment

Table[1](https://arxiv.org/html/2608.21808#S3.T1)compares MCite\-RL with representative baselines on Wiki\-VISA, FinRAGBench\-V, and MMLongBench\-Doc, evaluated in terms of Answer Accuracy \(Ans\) and Citation Precision \(Cit\)\. We summarize three key insights\.

#### Multimodal citation remains challenging\.

As shown in Table[1](https://arxiv.org/html/2608.21808#S3.T1), vanilla RAG consistently yields low citation scores across datasets, indicating that coarse retrieval without explicit visual cropping is insufficient for citation tasks\. Moreover, introducing agentic reasoning with iterative retrieval and cropping alone does not resolve this limitation\. For instance, GPT\-4o achieves only 3\.40% citation accuracy on Wiki\-VISA, while Qwen2\.5\-VL\-7B\-Instruct attains merely 0\.04%\. This implies that inference\-time optimization without training is inadequate for reliable multimodal citation\.

#### Citation\-enhanced RL effectively improves citation precision\.

By explicitly incorporating citation\-based rewards directly into the reinforcement learning objective, MCite\-RL substantially improves citation performance across all datasets\. Specifically, on Qwen2\.5\-VL\-7B\-Instruct backbone, it leads to double\-digit improvements \(over 10%\) compared to previous strong baselines, demonstrating the effectiveness of our citation\-enhanced RL\.

#### Improving citation precision generally leads to better answer accuracy\.

In most settings, citation improvements are accompanied by gains in answer accuracy\. For Qwen2\.5\-VL\-7B\-Instruct, MCite\-RL achieves the best answer precision in all datasets, including 60\.00% in Wiki\-VISA, 21\.81% in FinRAGBench\-V, and 35\.04% in MMLongBench\-Doc, which corresponds to a 4–6% improvement\. For Qwen2\.5\-VL\-3B\-Instruct, however, clear gains in citation fail to yield corresponding answer accuracy improvements in some cases\. This may imply that the limited capacity of the model restricts the generalization of citation benefits to the accuracy of the answer\.

Overall, these results indicate that explicitly optimizing citation precision at the objective level is critical for reliable multimodal RAG, and that citation\-enhanced reinforcement learning provides a systematic solution to this challenge\. We further provide case study of our method in Appendix[A\.2](https://arxiv.org/html/2608.21808#A1.SS2)\.

\(a\)Answer Accuracy\(b\)Outcome\-level Citation\(c\)Process\-level Citation\(d\)Entropy Loss
Figure 3:Training dynamics comparison across four key metrics\.The orange solid lines represent MCite\-RL, while the blue solid lines denote the baseline VRAG\-RL \(w/o Citation Reward\)\.

### 5\.2Training Analysis

To investigate the mechanisms driving the performance improvements, we analyze the training dynamics of MCite\-RL against the highly competitive baseline \(VRAG\-RL w/o Citation\), as illustrated in Figure[3](https://arxiv.org/html/2608.21808#S5.F3)\. We derive two additional observations regarding the learning process\. This analysis provides further insights into how citation\-enhanced RL influences the learning behavior\.

#### Citation rewards reduce generation uncertainty and improves overall quality\.

TheEntropy Losscurve \(Figure[3](https://arxiv.org/html/2608.21808#S5.F3)\(d\)\) shows that although both models reduce entropy during training, MCite\-RL consistently converges to a lower entropy level\. This indicates that citation constraints serve as an effective regularizer, narrowing the feasible solution space and yielding more confident and robust generation\.

#### Explicit citation rewards amplify the intrinsic correlation between answering and citation\.

In Figure[3](https://arxiv.org/html/2608.21808#S5.F3)\(b\) and \(c\), baseline curves \(blue lines\) reveal a slight natural increase in citation precision even without explicit citation supervision, suggesting an implicit link between answering and visual grounding\. However, this signal remains weak and plateaus early, especially on out\-of\-domain datasets\. In contrast, MCite\-RL \(orange lines\), equipped with citation rewards, transforms this weak correlation into an explicit reinforcement loop, improving performance in both in\-domain and out\-of\-domain scenarios\.

### 5\.3Ablation and Citation Analysis

To disentangle the contributions of different components and address potential entanglement between multi\-term rewards, we conduct a component\-wise ablation on the Qwen2\.5\-VL\-7B\-Instruct backbone \(Table[2](https://arxiv.org/html/2608.21808#S5.T2)\)\. We further evaluate whether the generated citations provide semantically sufficient evidence for the answers through an answer\-citation consistency analysis\.

#### Role of Training Stages\.

SFT establishes the foundational agentic workflow and mitigates the cold\-start challenge\. However, removing the RL stage \(w/o RL\) results in significant degradation in both answer and citation quality\. This confirms that strategic reward optimization, rather than mere instruction following or format adherence, is the primary driver of MCite\-RL’s performance gains\.

ConfigurationAns\. \(%\)Cit\. \(%\)MCite\-RL \(Full\)60\.0036\.05Reward Componentsw/o Process\-level Cit\.57\.6034\.50w/o Outcome\-level Cit\.54\.0013\.15w/o All Cit\. Rewards54\.2020\.56w/o Ans\. Reward53\.2041\.65w/o Search Reward54\.8035\.95w/o Format Constraints55\.2033\.41Training Ablationw/o RL Stage \(SFT only\)45\.8018\.66w/o Training \(Base\)9\.600\.04Table 2:Component\-wise Ablation for Reward Function on Wiki\-VISA \(7B\)\. We categorize components into reward items and training constraints to analyze their contributions\.
#### Effectiveness of Visual Citation Rewards\.

Our ablation results \(Table[2](https://arxiv.org/html/2608.21808#S5.T2)\) demonstrate the significant impact of visual citation\-enhanced rewards on overall system performance\. Removing all visual citation rewards results in a sharp decline in citation precision from 36\.05% to 20\.56%, while answer accuracy concurrently drops from 60\.00% to 54\.20%\. This confirms that visual citation rewards not only enhance localization quality but also improve answer generation through evidence\-based reasoning\. Within this framework, the outcome\-level reward serves as the dominant anchor by providing precise IoU feedback, while the process\-level reward provides additional refinement by constraining intermediate cropping\. Crucially, without outcome\-level anchors \(w/o Outcome\-level Vis\-Cit\.\), the model tends to generate overly expansive crops to satisfy intermediate coverage signals at the expense of precision, leading to a substantial degradation in citation quality \(13\.15%\)\. Ultimately, the synergy between both reward levels is essential for maintaining precise, stable, and verifiable agentic visual citation in multimodal RAG\.

#### Answer–Citation Consistency Evaluation\.

Since bounding\-box IoU only measures spatial alignment, we further evaluate whether the cited regions semantically support the generated answers\. We adopt an LLM\-as\-a\-judge protocol that assesses whether the cited evidence is sufficient, non\-redundant, and directly supports the answer\. MCite\-RL consistently improves this metric over baselines, indicating better semantic alignment between generated answers and visual citations \(detailed results in Appendix[A\.1](https://arxiv.org/html/2608.21808#A1.SS1)Table[3](https://arxiv.org/html/2608.21808#A1.T3)\)\.

## 6Conclusion

Accurate citation is essential for reliable multimodal RAG systems, yet it remains challenging in practice\. In this work, we propose MCite\-RL, a framework that integrates agentic workflow with a citation\-enhanced reward function\. By explicitly supervising citation at both the process and outcome levels, MCite\-RL enables joint optimization of citation and answer quality\.

## Limitations

While MCite\-RL improves the precision of visual citation through citation\-aware reinforcement learning, it has several limitations\. The current training process introduces some additional computational overhead, and the formulation focuses on bounding\-box–based visual citation on single images and does not explicitly model more expressive citation structures; we leave these extensions for future work\.

## Ethical Consideration

This work focuses on improving the traceability and verifiability of multimodal retrieval\-augmented generation through explicit visual citation\. By encouraging models to ground their outputs in identifiable visual evidence, our approach aims to reduce hallucination and improve transparency\. These goals are broadly aligned with responsible AI practices\. Nevertheless, the system may inherit biases present in the underlying datasets, including those related to visual representations and annotation practices, which could potentially affecting citation behavior\. In addition, while visual citation enhances interpretability, it does not guarantee factual correctness of the retrieved content itself\. Caution should be exercised when deploying such systems in high\-stakes domains, and human oversight remains necessary\.

## Acknowledgement

We thank the anonymous reviewers for their helpful comments on this paper\. This work was partially supported by National Natural Science Foundation of China projects \(No\. 62476010\), National Natural Science Foundation of China \(No\. 62272008\), and the Fundamental Research Funds for the Central Universities, Peking University\.

## References

- Anthropic \(2024\)AnthropicClaude 3\.5 sonnet raises the industry bar for intelligence\.Note:[https://www\.anthropic\.com/news/claude\-3\-5\-sonnet](https://www.anthropic.com/news/claude-3-5-sonnet)Accessed: 2025\-09\-07Cited by:[§4\.2](https://arxiv.org/html/2608.21808#S4.SS2.p2.1)\.
- Comaniciet al\.\(2025\)G\. Comanici, E\. Bieber, M\. Schaekermann, I\. Pasupat, N\. Sachdeva, I\. S\. Dhillon, M\. Blistein, O\. Ram, D\. Zhang, E\. Rosen, L\. Marris, S\. Petulla, C\. Gaffney, A\. Aharoni, N\. Lintz, T\. C\. Pais, H\. Jacobsson, I\. Szpektor, N\. Jiang, K\. Haridasan, A\. Omran, N\. Saunshi, D\. Bahri, G\. Mishra, E\. Chu, T\. Boyd, B\. Hekman, A\. Parisi, C\. Zhang, K\. Kawintiranon, T\. Bedrax\-Weiss, O\. Wang, Y\. Xu, O\. Purkiss, U\. Mendlovic, I\. Deutel, N\. Nguyen, A\. Langley, F\. Korn, L\. Rossazza, A\. Ramé, S\. Waghmare, H\. Miller, N\. Byrd, A\. Sheshan, R\. H\. S\. Bhardwaj, P\. Janus, T\. Rissa, D\. Horgan, S\. Silver, A\. Wahid, S\. Brin, Y\. Raimond, K\. Kloboves, C\. Wang, N\. B\. Gundavarapu, I\. Shumailov, B\. Wang, M\. Pajarskas, J\. Heyward, M\. Nikoltchev, M\. Kula, H\. Zhou, Z\. Garrett, S\. Kafle, S\. Arik, A\. Goel, M\. Yang, J\. Park, K\. Kojima, P\. Mahmoudieh, K\. Kavukcuoglu, G\. Chen, D\. Fritz, A\. Bulyenov, S\. Roy, D\. Paparas, H\. Shemtov, B\. Chen, R\. Strudel, D\. Reitter, A\. Roy, A\. Vlasov, C\. Ryu, C\. Leichner, H\. Yang, Z\. Mariet, D\. Vnukov, T\. Sohn, A\. Stuart, W\. Liang, M\. Chen, P\. Rawlani, C\. Koh, J\. Co\-Reyes, G\. Lai, P\. Banzal, D\. Vytiniotis, J\. Mei, and M\. CaiGemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.CoRRabs/2507\.06261\.External Links:[Link](https://doi.org/10.48550/arXiv.2507.06261),[Document](https://dx.doi.org/10.48550/ARXIV.2507.06261),2507\.06261Cited by:[§4\.2](https://arxiv.org/html/2608.21808#S4.SS2.p2.1)\.
- Faysseet al\.\(2024\)M\. Faysse, H\. Sibille, T\. Wu, B\. Omrani, G\. Viaud, C\. Hudelot, and P\. ColomboColPali: efficient document retrieval with vision language models\.External Links:2407\.01449,[Link](https://arxiv.org/abs/2407.01449)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px1.p1.1),[§4\.4](https://arxiv.org/html/2608.21808#S4.SS4.p1.1)\.
- Gaoet al\.\(2023\)T\. Gao, H\. Yen, J\. Yu, and D\. ChenEnabling large language models to generate text with citations\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6\-10, 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),pp\. 6465–6488\.External Links:[Link](https://doi.org/10.18653/v1/2023.emnlp-main.398),[Document](https://dx.doi.org/10.18653/V1/2023.EMNLP-MAIN.398)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.
- Gaoet al\.\(2024\)Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, M\. Wang, and H\. WangRetrieval\-augmented generation for large language models: a survey\.External Links:2312\.10997,[Link](https://arxiv.org/abs/2312.10997)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.
- Jianget al\.\(2025a\)P\. Jiang, X\. Xu, J\. Lin, J\. Xiao, Z\. Wang, J\. Sun, and J\. HanS3: you don’t need that much data to train a search agent via rl\.ArXivabs/2505\.14146\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278769452)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Jianget al\.\(2025b\)Y\. Jiang, L\. Shen, L\. Niu, S\. Zhao, W\. Su, and B\. ZhengQAgent: a modular search agent with interactive query understanding\.ArXivabs/2510\.08383\.External Links:[Link](https://api.semanticscholar.org/CorpusID:281951207)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Jinet al\.\(2025\)B\. Jin, H\. Zeng, Z\. Yue, D\. Wang, H\. Zamani, and J\. HanSearch\-r1: training llms to reason and leverage search engines with reinforcement learning\.CoRRabs/2503\.09516\.External Links:[Link](https://doi.org/10.48550/arXiv.2503.09516),[Document](https://dx.doi.org/10.48550/ARXIV.2503.09516),2503\.09516Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p4.1),[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2608.21808#S3.SS2.p1.1)\.
- Jinet al\.\(2023\)Z\. Jin, P\. Cao, Y\. Chen, K\. Liu, and J\. ZhaoInstructoR: instructing unsupervised conversational dense retrieval with large language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 6649–6675\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.443/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.443)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.
- Kimi \(2025\)KimiKimi\-researcher: end\-to\-end rl training for emerging agentic\.External Links:[Link](https://moonshotai.github.io/Kimi-Researcher/)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2025\)X\. Li, G\. Dong, J\. Jin, Y\. Zhang, Y\. Zhou, Y\. Zhu, P\. Zhang, and Z\. DouSearch\-o1: agentic search\-enhanced large reasoning models\.arXiv preprint arXiv:2501\.05366\.Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Loisonet al\.\(2026\)A\. Loison, Q\. Macé, A\. Edy, V\. Xing, T\. Balough, G\. Moreira, B\. Liu, M\. Faysse, C\. Hudelot, and G\. ViaudViDoRe V3: A comprehensive evaluation of retrieval augmented generation in complex real\-world scenarios\.CoRRabs/2601\.08620\.External Links:[Link](https://doi.org/10.48550/arXiv.2601.08620),[Document](https://dx.doi.org/10.48550/ARXIV.2601.08620),2601\.08620Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p3.1)\.
- Maet al\.\(2024a\)X\. Ma, S\. Lin, M\. Li, W\. Chen, and J\. LinUnifying multimodal retrieval via document screenshot embedding\.External Links:2406\.11251,[Link](https://arxiv.org/abs/2406.11251)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px1.p1.1)\.
- Maet al\.\(2025\)X\. Ma, S\. Zhuang, B\. Koopman, G\. Zuccon, W\. Chen, and J\. LinVISA: retrieval augmented generation with visual source attribution\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2025, Vienna, Austria, July 27 \- August 1, 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),pp\. 30154–30169\.External Links:[Link](https://aclanthology.org/2025.acl-long.1456/)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p2.1),[§1](https://arxiv.org/html/2608.21808#S1.p3.1),[§1](https://arxiv.org/html/2608.21808#S1.p5.1),[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.21808#S4.SS1.p1.1)\.
- Maet al\.\(2024b\)Y\. Ma, Y\. Zang, L\. Chen, M\. Chen, Y\. Jiao, X\. Li, X\. Lu, Z\. Liu, Y\. Ma, X\. Dong, P\. Zhang, L\. Pan, Y\. Jiang, J\. Wang, Y\. Cao, and A\. SunMMLongBench\-doc: benchmarking long\-context document understanding with visualizations\.External Links:2407\.01523,[Link](https://arxiv.org/abs/2407.01523)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p5.1),[§4\.1](https://arxiv.org/html/2608.21808#S4.SS1.p1.1)\.
- Meiet al\.\(2025\)L\. Mei, Z\. Yang, and C\. ChenAI\-searchplanner: modular agentic search via pareto\-optimal multi\-objective reinforcement learning\.ArXivabs/2508\.20368\.External Links:[Link](https://api.semanticscholar.org/CorpusID:280949856)Cited by:[§3\.2](https://arxiv.org/html/2608.21808#S3.SS2.p1.1)\.
- OpenAI \(2023\)OpenAIGPT\-4 technical report\.CoRRabs/2303\.08774\.External Links:[Link](https://doi.org/10.48550/arXiv.2303.08774),[Document](https://dx.doi.org/10.48550/ARXIV.2303.08774),2303\.08774Cited by:[§4\.2](https://arxiv.org/html/2608.21808#S4.SS2.p2.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.03300),[Document](https://dx.doi.org/10.48550/ARXIV.2402.03300),2402\.03300Cited by:[§3\.4](https://arxiv.org/html/2608.21808#S3.SS4.SSS0.Px2.p1.1)\.
- Shiet al\.\(2025\)Y\. Shi, S\. Li, C\. Wu, Z\. Liu, J\. Fang, H\. Cai, A\. Zhang, and X\. WangSearch and refine during think: autonomous retrieval\-augmented reasoning of llms\.ArXivabs/2505\.11277\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278715010)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Sunet al\.\(2025\)Y\. Sun, C\. Peng, Y\. Yan, S\. Yu, Z\. Liu, C\. Chen, Z\. Liu, and M\. SunVisRAG 2\.0: evidence\-guided multi\-image reasoning in visual retrieval\-augmented generation\.CoRRabs/2510\.09733\.External Links:[Link](https://doi.org/10.48550/arXiv.2510.09733),[Document](https://dx.doi.org/10.48550/ARXIV.2510.09733),2510\.09733Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1),[§1](https://arxiv.org/html/2608.21808#S1.p4.1)\.
- Suriet al\.\(2025\)M\. Suri, P\. Mathur, F\. Dernoncourt, K\. Goswami, R\. A\. Rossi, and D\. ManochaVisDoM: multi\-document QA with visually rich elements using multimodal retrieval\-augmented generation\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 6088–6109\.External Links:[Link](https://aclanthology.org/2025.naacl-long.310/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.310),ISBN 979\-8\-89176\-189\-6Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.
- Tanet al\.\(2025\)Z\. Tan, J\. Huang, Q\. Wu, H\. Zhang, C\. Zhuang, and J\. GuRAG\-r1: incentivizing the search and reasoning capabilities of llms through multi\-query parallelism\.External Links:[Link](https://api.semanticscholar.org/CorpusID:280148675)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Team \(2025\)Q\. TeamQwen2\.5\-vl\.External Links:[Link](https://qwenlm.github.io/blog/qwen2.5-vl/)Cited by:[§4\.4](https://arxiv.org/html/2608.21808#S4.SS4.p1.1)\.
- Teamet al\.\(2025\)T\. D\. Team, B\. Li, B\. Zhang, D\. Zhang, F\. Huang, G\. Li, G\. Chen, H\. Yin, J\. Wu, J\. Zhou,et al\.Tongyi deepresearch technical report\.arXiv preprint arXiv:2510\.24701\.Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025a\)Q\. Wang, R\. Ding, Z\. Chen, W\. Wu, S\. Wang, P\. Xie, and F\. ZhaoViDoRAG: visual document retrieval\-augmented generation via dynamic iterative reasoning agents\.External Links:2502\.18017,[Link](https://arxiv.org/abs/2502.18017)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.
- Wanget al\.\(2025b\)Q\. Wang, R\. Ding, Y\. Zeng, Z\. Chen, L\. Chen, S\. Wang, P\. Xie, F\. Huang, and F\. ZhaoVRAG\-RL: empower vision\-perception\-based RAG for visually rich information understanding via iterative reasoning with reinforcement learning\.CoRRabs/2505\.22019\.External Links:[Link](https://doi.org/10.48550/arXiv.2505.22019),[Document](https://dx.doi.org/10.48550/ARXIV.2505.22019),2505\.22019Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p4.1),[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1),[1st item](https://arxiv.org/html/2608.21808#S3.I1.i1.p1.1),[§3\.2](https://arxiv.org/html/2608.21808#S3.SS2.p1.1)\.
- Wanget al\.\(2025c\)Y\. Wang, Y\. Zhao, R\. Duan, J\. Li, Y\. Wang, and K\. ZhangGenR1\-searcher: curriculum reinforcement learning for dynamic retrieval and document generation\.Proceedings of the 34th ACM International Conference on Information and Knowledge Management\.External Links:[Link](https://api.semanticscholar.org/CorpusID:282906833)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025d\)Z\. Wang, X\. Zheng, K\. An, C\. Ouyang, J\. Cai, Y\. Wang, and Y\. WuStepSearch: igniting llms search ability via step\-wise proximal policy optimization\.ArXivabs/2505\.15107\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278782142)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Weiet al\.\(2025\)Z\. Wei, W\. Chen, and Y\. MengInstructRAG: instructing retrieval\-augmented generation via self\-synthesized rationales\.External Links:2406\.13629,[Link](https://arxiv.org/abs/2406.13629)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.
- Wuet al\.\(2025a\)J\. Wu, J\. Zhu, Y\. Liu, M\. Xu, and Y\. JinAgentic reasoning: a streamlined framework for enhancing LLM reasoning with agentic tools\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 28489–28503\.External Links:[Link](https://aclanthology.org/2025.acl-long.1383/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1383),ISBN 979\-8\-89176\-251\-0Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Wuet al\.\(2025b\)J\. Wu, X\. Zhong, J\. Sun, B\. Li, B\. Jin, J\. Han, and Q\. ZengStructure\-r1: dynamically leveraging structural knowledge in llm reasoning through reinforcement learning\.ArXivabs/2510\.15191\.External Links:[Link](https://api.semanticscholar.org/CorpusID:282203523)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Xiaet al\.\(2025\)P\. Xia, K\. Zhu, H\. Li, T\. Wang, W\. Shi, S\. Wang, L\. Zhang, J\. Zou, and H\. YaoMMed\-rag: versatile multimodal rag system for medical vision language models\.External Links:2410\.13085,[Link](https://arxiv.org/abs/2410.13085)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px1.p1.1)\.
- Xieet al\.\(2025\)R\. Xie, D\. Qiu, D\. Gopinath, D\. Lin, Y\. Sun, C\. Wang, S\. Potdar, and B\. DhingraInterleaved reasoning for large language models via reinforcement learning\.ArXivabs/2505\.19640\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278904627)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.External Links:2210\.03629,[Link](https://arxiv.org/abs/2210.03629)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Yoranet al\.\(2023\)O\. Yoran, T\. Wolfson, O\. Ram, and J\. BerantMaking retrieval\-augmented language models robust to irrelevant context\.ArXivabs/2310\.01558\.External Links:[Link](https://api.semanticscholar.org/CorpusID:263608822)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.
- Yuet al\.\(2025a\)S\. Yu, C\. Tang, B\. Xu, J\. Cui, J\. Ran, Y\. Yan, Z\. Liu, S\. Wang, X\. Han, Z\. Liu, and M\. SunVisRAG: vision\-based retrieval\-augmented generation on multi\-modality documents\.External Links:2410\.10594,[Link](https://arxiv.org/abs/2410.10594)Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.
- Yuet al\.\(2025b\)W\. Yu, W\. Chen, G\. Qi, W\. Li, Y\. Li, L\. Sha, D\. Xia, and J\. HuangBBox docvqa: A large scale bounding box grounded dataset for enhancing reasoning in document visual question answer\.CoRRabs/2511\.15090\.External Links:[Link](https://doi.org/10.48550/arXiv.2511.15090),[Document](https://dx.doi.org/10.48550/ARXIV.2511.15090),2511\.15090Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p3.1)\.
- Zhanget al\.\(2026\)J\. Zhang, X\. Lv, L\. Feng, L\. Hou, and J\. LiChaining the evidence: robust reinforcement learning for deep search agents with citation\-aware rubric rewards\.CoRRabs/2601\.06021\.External Links:[Link](https://doi.org/10.48550/arXiv.2601.06021),[Document](https://dx.doi.org/10.48550/ARXIV.2601.06021),2601\.06021Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1),[§1](https://arxiv.org/html/2608.21808#S1.p4.1)\.
- Zhanget al\.\(2025\)Q\. Zhang, S\. Yang, L\. Gao, H\. Chen, X\. Hu, J\. Chen, J\. Wang, S\. Guo, B\. Zheng, H\. Wang, and J\. ZhaoLeTS: learning to think\-and\-search via process\-and\-outcome reward hybridization\.ArXivabs/2505\.17447\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278886413)Cited by:[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2025\)S\. Zhao, Z\. Jin, S\. Li, and J\. GaoFinRAGBench\-v: A benchmark for multimodal RAG with visual citation in the financial domain\.CoRRabs/2505\.17471\.External Links:[Link](https://doi.org/10.48550/arXiv.2505.17471),[Document](https://dx.doi.org/10.48550/ARXIV.2505.17471),2505\.17471Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p2.1),[§1](https://arxiv.org/html/2608.21808#S1.p5.1),[§2](https://arxiv.org/html/2608.21808#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.21808#S4.SS1.p1.1)\.
- Zhaoet al\.\(2026\)Y\. Zhao, J\. Zhu, Z\. Ding, and X\. LiGRACE: reinforcement learning for grounded response and abstention under contextual evidence\.CoRRabs/2601\.04525\.External Links:[Link](https://doi.org/10.48550/arXiv.2601.04525),[Document](https://dx.doi.org/10.48550/ARXIV.2601.04525),2601\.04525Cited by:[§1](https://arxiv.org/html/2608.21808#S1.p1.1)\.

## Appendix AAdditional Experimental Analysis

### A\.1Answer\-Citation Consistency Evaluation

IoU\-based citation metrics primarily measure spatial alignment between predicted and ground\-truth regions, but do not directly assess whether the cited region semantically supports the generated answer\. To address this limitation, we introduce an additional answer\-citation consistency evaluation based on an LLM\-as\-a\-judge paradigm\. Given the cropped citation region, question, and generated answer, the evaluator judges whether the citation provides sufficient, non\-redundant, and directly supporting evidence\.

Table 3:Answer\-citation consistency \(%\) of MCite\-RL compared with other methods\. Higher scores indicate better semantic alignment between generated answers and cited regions\.As shown in Table[3](https://arxiv.org/html/2608.21808#A1.T3), MCite\-RL consistently outperforms baselines across all three datasets under this evaluation, demonstrating that it not only localizes relevant regions more precisely but also identifies evidence that is more directly supportive of the generated answers\.

### A\.2Case Study

To vividly illustrate the effectiveness of our proposed method, Figure[4](https://arxiv.org/html/2608.21808#A1.F4)presents a qualitative comparison between models with and without citation\-based rewards on a query regarding the “Jharkhand national movement”\. Without the citation\-enhanced reward, the model fails to localize the relevant evidence, instead cropping an irrelevant footer region, which consequently leads to the incorrect answer “Marty Sahid Lal and Vishwanath Shandeo”\. In contrast, with the citation\-enhanced reward, our MCite\-RL method effectively guides the model to precisely localize the key “Jharkhand revolts” paragraph\. This accurate evidence localization enables the model to correctly identify the answer “Sidhu and Kanhu”, supported by an accurate visual citation, thereby demonstrating the effectiveness of our proposed method\.

![Refer to caption](https://arxiv.org/html/2608.21808v1/case_study.png)Figure 4:This is a qualitative comparison showing the impact of the Citation\-enhanced Reward\. Without the reward \(top\), incorrect cropping leads to information loss and hallucinations; with the reward \(bottom\), the model precisely locates critical evidence to generate accurate answers and supporting citations\.

## Appendix BDetails for SFT Trajectory Synthesis and Filtering

To construct effective process supervision for the agentic SFT stage, we synthesize trajectories following a structured search–crop–answer workflow\. Each trajectory contains intermediate retrieval and visual grounding actions, followed by the final answer with visual citation\. To ensure that the synthesized trajectories provide reliable and learnable supervision signals, we further apply strict yet practical filtering criteria to remove noisy or inconsistent samples\.

### B\.1Trajectory Synthesis

We synthesize agentic trajectories using a multimodal teacher model\. Given a question and the corresponding document image, the teacher is instructed to first retrieve relevant evidence, progressively refine the visual region through hierarchical cropping, and finally produce the answer with the corresponding citation bounding box\.

The generation process follows the structured workflow shown in Table[4](https://arxiv.org/html/2608.21808#A2.T4), which specifies the available tools, search\-crop refinement strategy, and required output format\. This design encourages trajectories with explicit intermediate evidence localization steps, providing process\-level supervision for agentic SFT\.

Table 4:Trajectory generation prompt used for synthesizing agentic SFT trajectories\.
### B\.2Trajectory Filtering

Although the teacher model can generate diverse trajectories, some samples may contain incorrect retrieval decisions, inaccurate answers, or invalid citation regions\. Therefore, we apply a set of filtering criteria \(Table[5](https://arxiv.org/html/2608.21808#A2.T5)\) to ensure the quality of the synthesized supervision data\.

Specifically, we filter trajectories based on retrieval correctness, answer correctness, and bounding box validity\. Only trajectories satisfying all quality constraints are retained for SFT training\. In total, we generate 28,175 candidate trajectories and retain 2,417 trajectories after filtering, resulting in an acceptance rate of approximately 8\.6%\. Table[6](https://arxiv.org/html/2608.21808#A3.T6)shows an example of filtered trajectory\.

Table 5:Filtering criteria for trajectory synthesis\. A trajectory is retained only if it satisfies all the conditions\.

## Appendix CVisual Citation Annotation Guidelines

To evaluate the quality of visual citation on MMLongBench\-Doc, we developed a strict annotation protocol to annotate 300 samples\. Annotators are provided with a triplet:\(User Query, Correct Answer, Source Image\)\. The task is to draw a bounding box that visually substantiates the answer\. The guidance is listed in Table[7](https://arxiv.org/html/2608.21808#A3.T7)\. We have two annotators to double check each data\.

Table 6:Synthesized Trajectory Data Sample\.This table visualizes a complete multi\-turn interaction where the model performs search and progressive cropping to answer the user query\. Note how the bounding box coordinates refine from coarse to fine\.Table 7:Annotation Principles for Visual Citation\.The ground truth bounding box must balance completeness, compactness, and relevance\.
## Appendix DImplementation Details

### D\.1Hardware and Software Environment

All experiments were conducted on a high\-performance computing cluster, using GPU: 8\*NVIDIA A800 \(80GB\), and vLLM as the inference engine\. SFT takes about 4 hours, and RL completes in approximately 8 hours under our current configuration \(1,000 samples, 1 epoch\)\.

### D\.2Reward Configuration

The composite reward function aims to balance retrieval quality, reasoning correctness, and visual alignment\. The specific weights for each component are listed in Table[8](https://arxiv.org/html/2608.21808#A4.T8)\. We enforce a format pattern constraint: if the model output fails to match the required structural pattern, the total reward is penalized to 0\.

Table 8:Reward components, corresponding metrics, and weights used in RL optimization\.
### D\.3Training Hyperparameters

We conducted a two\-stage training process: Supervised Fine\-Tuning \(SFT\) followed by Reinforcement Learning \(RL\)\.

Supervised Fine\-Tuning \(SFT\)\.We fine\-tuned the Qwen2\.5\-VL\-7B\-Instruct model using the configuration detailed in Table[9](https://arxiv.org/html/2608.21808#A4.T9)\.

Table 9:Hyperparameters for SFTReinforcement Learning \(RL\)\.In the RL stage, we utilized the GRPO algorithm\. The detailed hyperparameters are provided in Table[10](https://arxiv.org/html/2608.21808#A4.T10)\.

HyperparameterValueAlgorithmGRPOTraining Epochs1Actor Learning Rate1\.0×10−61\.0\\times 10^\{\-6\}KL Coefficient0\.01Global Train Batch Size8Max Prompt Length8,192Max Response Length2,048Rollout \(NN\)1Number of Agents5Training Samples1000Table 10:Hyperparameters for RL
### D\.4Dataset Statistics

We use the subset of Wiki\-Visa dataset for training, and subsets of FinRAGBench\-V and MMLongBench\-Doc as the OOD evaluation sets\. The retrieval corpus consists of the source documents from which the model retrieves information\. The detailed statistics are shown in Table[11](https://arxiv.org/html/2608.21808#A4.T11)\.

Table 11:Data Statistics of the specific retrieval corpus size and validation set size for each dataset\.
### D\.5Details of Evaluation Metrics

To ensure the reproducibility of our experiments, we follow the official evaluation protocols for each benchmark\. For Wiki\-VISA and MMLongBench\-Doc, we adopt Relaxed Exact Match \(EM\) and ROUGE\-L, as their answers are predominantly short and fact\-based with limited lexical variation\. In contrast, for FinRAGBench\-V, which contains a mixture of short factual and long\-form reasoning responses, we employ a GPT\-4o\-based judge for semantic evaluation\. These settings align with the specific characteristics of each dataset\.