Structured Role-Aware Policy Optimization for Multimodal Reasoning

# Structured Role-Aware Policy Optimization for Multimodal Reasoning
Source: [https://arxiv.org/html/2605.07274](https://arxiv.org/html/2605.07274)
Bingqing JiangSchool of Computing & Data Science, The University of Hong Kong\. Email: bingqingjiang@connect\.hku\.hkDifan ZouSchool of Computing & Data Science and Institute of Data Science, The University of Hong Kong\. Email: dzou@hku\.hk

###### Abstract

Reinforcement learning from verifiable rewards \(RLVR\), especially with Group Relative Policy Optimization \(GRPO\), has shown strong potential for improving the reasoning capabilities of large vision\-language models \(LVLMs\)\. However, in multimodal reasoning, final\-answer rewards are typically assigned at the sequence level and do not distinguish the functional roles of different tokens, making it difficult to determine whether a correct answer is supported by task\-relevant visual evidence\. In this paper, we revisit multimodal RLVR from the perspective of role\-aware token\-level credit assignment, where structured responses are decomposed into perception tokens for extracting visual evidence and reasoning tokens for deriving answers from that evidence\. Based on this perspective, we proposeStructuredRole\-awarePolicyOptimization \(SRPO\), which refines the sequence\-level GRPO advantage into role\-aware token\-level advantages without changing the reward function\. Specifically, SRPO assigns role\-specific credit by using self\-distilled on\-policy contrasts: perception tokens are emphasized according to their visual dependency under original versus corrupted visual inputs, while reasoning tokens are emphasized according to their consistency with the generated perception\. These role\-specific signals are further unified through a shared trajectory\-level baseline, yielding positive token weights that adjust relative update magnitudes while preserving the original GRPO reward and optimization direction, without requiring external reward models or separate teachers\. Experiments across diverse multimodal reasoning benchmarks show that SRPO improves evidence\-grounded reasoning, highlighting the importance of moving beyond uniform sequence\-level credit toward role\-aware optimization for reliable multimodal reasoning\.

## 1Introduction

Reinforcement learning from verifiable rewards \(RLVR\)\(Guoet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib47)\), particularly with online policy optimization algorithms such as Group Relative Policy Optimization \(GRPO\)\(Shaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib33)\), has recently emerged as an effective paradigm for improving the reasoning capabilities of large language models\. Motivated by its success in text\-only settings, a growing body of work has begun to extend RLVR to large vision\-language models \(LVLMs\) for multimodal reasoning\(Huanget al\.,[2026b](https://arxiv.org/html/2605.07274#bib.bib1); Zhanget al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib2); Yaoet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib3); Wanget al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib6); Zhaet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib23)\)\. Despite encouraging progress, multimodal reasoning remains substantially more challenging than its text\-only counterpart: producing a reliable answer requires the model to first identify task\-relevant visual evidence and then use that evidence to support subsequent reasoning\(Liuet al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib4); Wanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib5); Huanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib21); Miaoet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib13)\)\.

For evidence\-dependent multimodal reasoning, sequence\-level answer supervision has a central limitation: it verifies only whether the final response reaches the correct answer, without assessing whether the intermediate tokens are grounded in task\-relevant visual evidence\(Liet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib35); Wanget al\.,[2026d](https://arxiv.org/html/2605.07274#bib.bib11); Chenet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib22)\)\. In practice, LVLMs can produce correct or plausible answers by exploiting language priors, dataset regularities, or shallow textual heuristics, rather than faithfully grounding their responses in the image\(Ghoshet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib48); Fanget al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib49)\)\. For example, as illustrated in Fig\.[1](https://arxiv.org/html/2605.07274#S1.F1), our MathVerse\(Zhanget al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib42)\)evaluation shows that GRPO can reach the correct final answer while exhibiting flawed intermediate grounding\. In the first example, the model misinterprets side\-length expressions as angle expressions, leading to an incorrect perception step and shortcut\-like reasoning\. In the second example, the model derives the correct final formula despite misidentifying the underlying 3D visual structure\. As a result, these rollouts may still receive positive sequence\-level rewards, even though their intermediate perception or reasoning is not fully supported by the visual evidence\. Thus, improved answer accuracy under sequence\-level supervision does not necessarily imply improved image\-grounded reasoning\.

![Refer to caption](https://arxiv.org/html/2605.07274v1/x1.png)Figure 1:MathVerse examples where GRPO reaches correct answers despite flawed intermediate grounding\.Recent efforts have begun to address this problem by explicitly separating visual perception from downstream reasoning\(Huanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib21); Miaoet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib13); Wanget al\.,[2026d](https://arxiv.org/html/2605.07274#bib.bib11); Zhouet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib51)\)\. This direction highlights a basic requirement for grounded multimodal reasoning: the model should first identify task\-relevant visual evidence and then answer based on that evidence\. Accordingly, a structured response can be organized into two functional stages: a*perception*stage for extracting question\-relevant visual evidence, and a*reasoning*stage for deriving the final answer from that evidence\(Sharmaet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib50)\)\. Crucially, these two stages serve distinct functional roles and therefore require different credit assignment criteria\(Chenet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib22); Yuet al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib24); Dinget al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib20)\)\. The perception stage should be credited for faithfully extracting task\-relevant visual evidence, whereas the reasoning stage should be credited for deriving a valid answer from that evidence\. However, existing optimization methods often treat these stages uniformly, ignoring their role distinction and conflating evidence extraction with evidence\-based inference\(Huanget al\.,[2026b](https://arxiv.org/html/2605.07274#bib.bib1); Zhanget al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib2); Yaoet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib3); Miaoet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib13); Wanget al\.,[2026d](https://arxiv.org/html/2605.07274#bib.bib11)\)\. This motivates a role\-aware policy optimization framework that preserves verifiable answer\-level supervision while assigning token\-level credit according to the functional role of each response segment\.

In this paper, we proposeSRPO\(StructuredRole\-awarePolicyOptimization\), a token\-level policy optimization method for multimodal RLVR that accounts for the functional heterogeneity of generated responses\. SRPO asks the policy to produce a structured response with a*perception*segment for extracting task\-relevant visual evidence and a*reasoning*segment for deriving the answer from that evidence\.Instead of treating the whole response as a homogeneous token sequence, SRPO converts the sequence\-level GRPO advantage into role\-aware token\-level advantages through positive modulation weights\. These weights are obtained by aself\-distilled credit assignmentprinciple, where the frozen rollout policy rescores the same generated tokens under role\-specific information conditions\. For perception tokens, SRPO measures visual dependency by contrasting likelihoods under the original and corrupted images; for reasoning tokens, it measures grounding consistency by contrasting perception\-conditioned prediction with direct image–question grounding\. These role\-specific scores are unified through a shared response\-level baseline over the entire valid trajectory and mapped to bounded positive multipliers, preserving the original verifiable reward and the reward\-induced optimization direction of GRPO while making token update magnitudes role\-aware\. SRPO therefore requires no external reward model, auxiliary supervision, or separate teacher, but restructures the policy\-gradient signal to reflect visual evidence extraction and evidence\-grounded reasoning\. In summary, our contributions are threefold:

- •Compared with conventional sequence\-level optimization that assigns a uniform credit signal to the whole response, we introduce a*role\-decomposed perspective*for multimodal token\-level credit assignment\. This perspective treats multimodal responses as functionally heterogeneous trajectories, where the*perception*stage extracts task\-relevant visual evidence and the*reasoning*stage derives answers from that evidence\.
- •We propose SRPO, which instantiates role\-decomposed token\-level credit assignment within GRPO\. SRPO estimates perception\- and reasoning\-stage credit through role\-specific self\-distilled on\-policy contrasts, then calibrates these signals with a shared trajectory\-level baseline and converts them into bounded positive weights to reweight the sequence\-level GRPO advantage\. This preserves the original reward and optimization direction without external reward models or teachers\.
- •We demonstrate that SRPO improves evidence\-grounded multimodal reasoning across multiple benchmarks, with ablations confirming the importance of role\-specific credit signals and unified trajectory\-level modulation\.

## 2Related Work

#### Multimodal Reinforcement Learning for Reasoning\.

RLVR, especially with policy optimization algorithms such as PPO and GRPO, has become an effective paradigm for improving reasoning in large language models\(Guoet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib47); Shaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib33)\)\. Motivated by this success, recent work extends RL\-style post\-training to LVLMs to improve multimodal reasoning beyond supervised imitationAchiamet al\.\([2023](https://arxiv.org/html/2605.07274#bib.bib56)\); Baiet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib32)\); Gemini Teamet al\.\([2024](https://arxiv.org/html/2605.07274#bib.bib55)\); Zhuet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib57)\)\. Existing methods explore improved multimodal training dataHuanget al\.\([2026b](https://arxiv.org/html/2605.07274#bib.bib1)\); Yaoet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib3)\), rollout diversificationLiuet al\.\([2026a](https://arxiv.org/html/2605.07274#bib.bib4)\); Wanget al\.\([2026a](https://arxiv.org/html/2605.07274#bib.bib5)\), replay mechanismsWanget al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib6)\), and more stable optimization heuristicsZhaet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib23)\), collectively demonstrating that RL\-style post\-training can substantially enhance multimodal reasoning\.

#### Perception\-Aware and Structured Multimodal Reasoning\.

Since correct final answers do not necessarily imply visually grounded intermediate reasoning\(Ghoshet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib48)\), prior methods strengthen visual grounding through perception\-aware rewardsXiaet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib7)\); Xiaoet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib8)\); Wanget al\.\([2026d](https://arxiv.org/html/2605.07274#bib.bib11)\), explicit perceptual scaffoldsCaoet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib9)\); Sarchet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib10)\); Suet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib27)\); Sunet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib28)\), or RL mechanisms that encourage stronger reliance on image evidenceYuet al\.\([2026a](https://arxiv.org/html/2605.07274#bib.bib24)\); Wanget al\.\([2026c](https://arxiv.org/html/2605.07274#bib.bib14)\); Liuet al\.\([2026b](https://arxiv.org/html/2605.07274#bib.bib30)\)\. Another related direction decomposes multimodal generation into structured stages, such as perception, evidence extraction, reasoning, and answer predictionChenet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib22)\); Liet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib12)\); Miaoet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib13)\); Dinget al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib20)\); Heet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib31)\)\. These works highlight the importance of explicit perceptual evidence and perception–reasoning separation\. However, such structure is mainly used as a modeling or supervision scaffold, while its role in determining token\-level policy updates remains less explored\. Our work incorporates explicit perception–reasoning structure into policy optimization and assigns distinct credit signals to the two components\.

#### Fine\-Grained Credit Assignment in Reinforcement Learning\.

Beyond sequence\-level supervision, the granularity of credit assignment has been increasingly recognized as important for reasoning optimization\. In text\-based settings, prior work studies structured credit allocation over intermediate reasoning processes or informative token subsetsGuoet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib15)\); Tranet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib16)\); Wanget al\.\([2026b](https://arxiv.org/html/2605.07274#bib.bib17)\); Xuet al\.\([2025](https://arxiv.org/html/2605.07274#bib.bib25)\); Zhanget al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib26)\); Yanget al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib29)\)\. Similar trends have emerged in multimodal reasoning, where token\-level or perception\-aware optimization is used to better align learning signals with visually grounded reasoning behaviorHuanget al\.\([2026a](https://arxiv.org/html/2605.07274#bib.bib21)\); Luet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib18)\); Yeet al\.\([2026](https://arxiv.org/html/2605.07274#bib.bib19)\)\. These studies suggest that fine\-grained optimization can reduce the tendency of models to exploit shortcut signals under coarse sequence\-level rewards\. Our work is closely related, but differs in explicitly handling functional heterogeneity within multimodal responses\. Rather than using a shared token\-importance criterion, we distinguish*perception tokens*from*reasoning tokens*and assign credit according to their distinct roles, while keeping the GRPO reward and its reward\-aligned optimization direction unchanged\.

## 3Method

### 3\.1Preliminary: Group Relative Policy Optimization

GRPO is a value\-free reinforcement learning algorithm that estimates policy\-gradient advantages by comparing multiple responses sampled from the same prompt\(Shaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib33)\)\. Given an image–question pair\(I,q\)\(I,q\), the behavior policyπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}samples a group ofGGresponses,

\{yi\}i=1G,yi=\(yi,1,…,yi,Ti\)∼πθold\(⋅∣I,q\)\.\\\{y\_\{i\}\\\}\_\{i=1\}^\{G\},\\qquad y\_\{i\}=\(y\_\{i,1\},\\ldots,y\_\{i,T\_\{i\}\}\)\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid I,q\)\.A verifier assigns each complete response a scalar reward that combines format compliance and answer correctness:

Ri=λfmt​Rifmt\+λacc​Riacc,R\_\{i\}=\\lambda\_\{\\mathrm\{fmt\}\}R\_\{i\}^\{\\mathrm\{fmt\}\}\+\\lambda\_\{\\mathrm\{acc\}\}R\_\{i\}^\{\\mathrm\{acc\}\},whereRifmtR\_\{i\}^\{\\mathrm\{fmt\}\}measures whether the response follows the required output format, andRiacc=𝟙​\{eq​\(a^i,a\)\}R\_\{i\}^\{\\mathrm\{acc\}\}=\\mathbbm\{1\}\\\{\\mathrm\{eq\}\(\\hat\{a\}\_\{i\},a\)\\\}measures whether the extracted answera^i\\hat\{a\}\_\{i\}matches the ground\-truth answeraaunder an equivalence checkereq​\(⋅,⋅\)\\mathrm\{eq\}\(\\cdot,\\cdot\)\. GRPO estimates the advantage of each response by normalizing rewards within the group:

A^i=Ri−μRσR\+ϵnorm,μR=1G​∑j=1GRj,σR=1G​∑j=1G\(Rj−μR\)2,\\hat\{A\}\_\{i\}=\\frac\{R\_\{i\}\-\\mu\_\{R\}\}\{\\sigma\_\{R\}\+\\epsilon\_\{\\mathrm\{norm\}\}\},\\qquad\\mu\_\{R\}=\\frac\{1\}\{G\}\\sum\_\{j=1\}^\{G\}R\_\{j\},\\qquad\\sigma\_\{R\}=\\sqrt\{\\frac\{1\}\{G\}\\sum\_\{j=1\}^\{G\}\(R\_\{j\}\-\\mu\_\{R\}\)^\{2\}\},whereϵnorm\\epsilon\_\{\\mathrm\{norm\}\}is a small constant for numerical stability\. The policy is then optimized with a PPO\-style clipped surrogate objective:

𝒥GRPO​\(θ\)=𝔼​\[1G​∑i=1G1Ti​∑t=1Timin⁡\(ρi,t​\(θ\)​A^i,clip​\(ρi,t​\(θ\),1−ϵ,1\+ϵ\)​A^i\)\],\\mathcal\{J\}\_\{\\mathrm\{GRPO\}\}\(\\theta\)=\\mathbb\{E\}\\left\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\frac\{1\}\{T\_\{i\}\}\\sum\_\{t=1\}^\{T\_\{i\}\}\\min\\left\(\\rho\_\{i,t\}\(\\theta\)\\hat\{A\}\_\{i\},\\,\\mathrm\{clip\}\\bigl\(\\rho\_\{i,t\}\(\\theta\),1\-\\epsilon,1\+\\epsilon\\bigr\)\\hat\{A\}\_\{i\}\\right\)\\right\],whereρi,t​\(θ\)=πθ​\(yi,t∣I,q,yi,<t\)πθold​\(yi,t∣I,q,yi,<t\)\\rho\_\{i,t\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid I,q,y\_\{i,<t\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{i,t\}\\mid I,q,y\_\{i,<t\}\)\}\. In practice, a KL penalty to a reference policyπref\\pi\_\{\\mathrm\{ref\}\}can be added to constrain policy drift\.

Despite its effectiveness, GRPO performs credit assignment at the sequence level: the same scalar advantageA^i\\hat\{A\}\_\{i\}is applied uniformly to all valid tokens in a response\. This coarse\-grained design ignores the functional heterogeneity of multimodal reasoning trajectories\. In particular, perception tokens should be encouraged according to their dependence on visual evidence, whereas reasoning tokens should be evaluated by whether the subsequent inference remains supported by the perceived content\. A uniform sequence\-level advantage cannot distinguish these roles\. This motivates a structured token\-level modulation mechanism that preserves the global optimization direction provided by the outcome reward, while redistributing update magnitudes across tokens\.

Table 1:Diagnostic evaluation on LogicVista\(Xiaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib40)\)under controlled visual conditions\. Numbers in parentheses denote the accuracy change relative to the original\-image setting\.
### 3\.2Structured Role\-aware Policy Optimization

We now introduce SRPO, which refines the sequence\-level GRPO signal into role\-aware token\-level advantages without changing the reward function or introducing an external teacher\. In multimodal reasoning, however, final\-answer correctness provides only an outcome\-level supervision signal and does not by itself specify whether the response is supported by the matched visual evidence\. To illustrate this limitation, we conduct a diagnostic evaluation on LogicVista using Qwen2\.5\-VL\-3B and Qwen2\.5\-VL\-7B under four visual conditions: the original image, a masked image, a randomly mismatched image, and a text\-only input\(Baiet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib32); Xiaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib40)\)\. As shown in Table[1](https://arxiv.org/html/2605.07274#S3.T1), accuracy decreases noticeably when the image is degraded by masking, indicating that visual information contributes to task performance\. In contrast, replacing the image with a randomly mismatched one yields little to no degradation relative to the original\-image setting\(Faveroet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib52); Luoet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib53); Liet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib54)\)\. This observation suggests that final\-answer accuracy is not always tightly coupled with image–text consistency, and that outcome\-level rewards may assign positive credit to responses that bypass or only weakly exploit the relevant visual evidence\. Recent token\-level or perception\-aware optimization methods suggest that fine\-grained learning signals can mitigate shortcut behavior under coarse sequence\-level rewards\(Huanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib21); Wanget al\.,[2026d](https://arxiv.org/html/2605.07274#bib.bib11); Luet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib18); Yeet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib19)\)\. SRPO shares this fine\-grained optimization view, but differs by explicitly decomposing multimodal responses into perception and reasoning stages rather than using a shared token\-importance criterion\. This role decomposition allows visual grounding and subsequent inference to receive distinct credit, which SRPO converts into self\-distilled token\-level signals while preserving the original GRPO outcome reward and reward\-induced optimization direction\.

For each on\-policy rollout batch, SRPO rescores the generated tokens with the behavior policyπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}under controlled changes of visual or textual context\. These rescoring computations are treated as stop\-gradient signals and are used only for credit modulation\. For each sampled responseyi=\(yi,1,…,yi,Ti\)y\_\{i\}=\(y\_\{i,1\},\\ldots,y\_\{i,T\_\{i\}\}\), we assume a structured format that can be parsed into two contiguous spans:

yi=\(yiperc,yireas\),y\_\{i\}=\(y\_\{i\}^\{\\mathrm\{perc\}\},y\_\{i\}^\{\\mathrm\{reas\}\}\),whereyipercy\_\{i\}^\{\\mathrm\{perc\}\}denotes perception tokens that describe or extract task\-relevant visual evidence, andyireasy\_\{i\}^\{\\mathrm\{reas\}\}denotes reasoning tokens that use the perceived content to derive the final answer\. SRPO then derives role\-specific token scores for the two spans and converts them into token\-level modulation weights for the GRPO advantage\.

#### Self\-distilled visual dependency for perception tokens\.

For perception tokens, we estimate visual reliance by comparing the behavior policy under two visual conditions\. The first uses the original imageII, and the second uses a corrupted imageImaskI^\{\\mathrm\{mask\}\}obtained by masking a subset of image patches\. Model parameters are identical across the two evaluations; only the visual input is changed\. For each perception\-token positiont∈𝒫it\\in\\mathcal\{P\}\_\{i\}, we first compute the stop\-gradient log\-likelihood contrast

δi,tperc=sg⁡\[log⁡πθold​\(yi,t∣I,q,yi,<t\)−log⁡πθold​\(yi,t∣Imask,q,yi,<t\)\]\.\\delta\_\{i,t\}^\{\\mathrm\{perc\}\}=\\operatorname\{sg\}\\left\[\\log\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{i,t\}\\mid I,q,y\_\{i,<t\}\)\-\\log\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{i,t\}\\mid I^\{\\mathrm\{mask\}\},q,y\_\{i,<t\}\)\\right\]\.\(3\.1\)A larger positiveδi,tperc\\delta\_\{i,t\}^\{\\mathrm\{perc\}\}indicates stronger dependence on the original visual input, while a small or negative value indicates weak visual dependence\. Under proportional perception\-credit allocation, SRPO converts this contrast into positive perception scores:

ri,tperc=exp⁡\(sign⁡\(A^i\)⋅δi,tperc\),t∈𝒫i\.r\_\{i,t\}^\{\\mathrm\{perc\}\}=\\exp\\left\(\\operatorname\{sign\}\(\\hat\{A\}\_\{i\}\)\\cdot\\delta\_\{i,t\}^\{\\mathrm\{perc\}\}\\right\),\\qquad t\\in\\mathcal\{P\}\_\{i\}\.\(3\.2\)WhenA^i\>0\\hat\{A\}\_\{i\}\>0, visually dependent perception tokens receive larger modulation scores and are therefore reinforced more strongly\. WhenA^i<0\\hat\{A\}\_\{i\}<0, the sign reversal assigns smaller scores to visually dependent perception tokens and larger scores to weakly visual\-dependent ones, so failed trajectories mainly suppress perception tokens that are insufficiently tied to the image\. The stop\-gradient operation ensures that these self\-distilled signals are used only for credit modulation and do not introduce an auxiliary optimization objective\.

![Refer to caption](https://arxiv.org/html/2605.07274v1/x2.png)Figure 2:Overview of SRPO\. SRPO decomposes each on\-policy response into perception and reasoning tokens, obtains a sequence\-level advantage from verifier rewards, and converts it into role\-aware token\-level advantages\. Perception tokens are scored by visual dependency, while reasoning tokens are scored by perception\-supported grounding consistency\. The resulting scores produce bounded positive modulation weights for token\-level reweighting, and policy optimization further includes a response\-token entropy regularizer for stability\.
#### Self\-distilled grounding consistency for reasoning tokens\.

For reasoning tokens, the desired property is not direct visual sensitivity, but whether the generated perception provides useful support for subsequent inference\. We therefore compare the behavior policy under two on\-policy conditioning contexts\. The first is the full rollout context, which contains the image, question, generated perception, and previous reasoning tokens\. The second is a perception\-ablated rescoring context, which removes the generated perception and rescoring is performed on the same rollout tail tokens\. This comparison estimates how much the perception segment supports each reasoning token beyond what is already explained by the image–question pair and the preceding reasoning context\. For each reasoning token positiont∈ℛit\\in\\mathcal\{R\}\_\{i\}, we define

δi,treas=sg⁡\[log⁡πθold​\(yi,t∣I,q,yiperc,yi,<treas\)−log⁡πθold​\(yi,t∣I,q,yi,<treas\)\],\\delta\_\{i,t\}^\{\\mathrm\{reas\}\}=\\operatorname\{sg\}\\left\[\\log\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\\bigl\(y\_\{i,t\}\\mid I,q,y\_\{i\}^\{\\mathrm\{perc\}\},y\_\{i,<t\}^\{\\mathrm\{reas\}\}\\bigr\)\-\\log\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\\bigl\(y\_\{i,t\}\\mid I,q,y\_\{i,<t\}^\{\\mathrm\{reas\}\}\\bigr\)\\right\],\(3\.3\)whereyi,<treasy\_\{i,<t\}^\{\\mathrm\{reas\}\}denotes the reasoning\-prefix tokens before positiontt\. This residual measures the perception\-induced change in the likelihood of the current reasoning token\. A positiveδi,treas\\delta\_\{i,t\}^\{\\mathrm\{reas\}\}indicates that conditioning on the generated perception makes the reasoning token more likely under the same image–question input and reasoning prefix, suggesting perception\-supported inference\. A small or negative value suggests that the reasoning token is weakly supported, or even suppressed, by the generated perception\. As in perception\-token modulation, we use this self\-distilled residual in an outcome\-aware manner by interpreting it under the sign of the trajectory\-level advantage and mapping it to a positive modulation factor:

ri,treas=exp⁡\(sign⁡\(A^i\)⋅δi,treas\),t∈ℛi\.r\_\{i,t\}^\{\\mathrm\{reas\}\}=\\exp\\left\(\\operatorname\{sign\}\(\\hat\{A\}\_\{i\}\)\\cdot\\delta\_\{i,t\}^\{\\mathrm\{reas\}\}\\right\),\\qquad t\\in\\mathcal\{R\}\_\{i\}\.\(3\.4\)WhenA^i\>0\\hat\{A\}\_\{i\}\>0, perception\-supported reasoning tokens with largerδi,treas\\delta\_\{i,t\}^\{\\mathrm\{reas\}\}receive larger modulation factors, thereby reinforcing successful inference patterns that are grounded in the generated perception\. WhenA^i<0\\hat\{A\}\_\{i\}<0, the sign reversal makes perception\-supported reasoning tokens receive smaller modulation factors, preventing unsuccessful perception\-supported patterns from being strengthened\.

#### Unified trajectory\-level modulation\.

We combine the role\-specific raw scores into a single valid\-token sequence:

ri,t=\{ri,tperc,t∈𝒫i,ri,treas,t∈ℛi,r\_\{i,t\}=\\begin\{cases\}r\_\{i,t\}^\{\\mathrm\{perc\}\},&t\\in\\mathcal\{P\}\_\{i\},\\\\ r\_\{i,t\}^\{\\mathrm\{reas\}\},&t\\in\\mathcal\{R\}\_\{i\},\\end\{cases\}where𝒫i\\mathcal\{P\}\_\{i\}andℛi\\mathcal\{R\}\_\{i\}denote the perception and reasoning token\-position sets of responseyiy\_\{i\}, respectively\. Let𝒱​\(yi\)\\mathcal\{V\}\(y\_\{i\}\)denote the set of all valid token positions\. SRPO computes one response\-level baseline over the entire valid trajectory:

r¯i=1\|𝒱​\(yi\)\|​∑t∈𝒱​\(yi\)ri,t\.\\bar\{r\}\_\{i\}=\\frac\{1\}\{\|\\mathcal\{V\}\(y\_\{i\}\)\|\}\\sum\_\{t\\in\\mathcal\{V\}\(y\_\{i\}\)\}r\_\{i,t\}\.This baseline is shared by perception and reasoning tokens, rather than computed separately for each span\. The centered raw scores are then converted into bounded positive multipliers:

wi,t=clip​\(1\+λmod​\(ri,t−r¯i\),mmin,mmax\),t∈𝒱​\(yi\),w\_\{i,t\}=\\mathrm\{clip\}\\left\(1\+\\lambda\_\{\\mathrm\{mod\}\}\\left\(r\_\{i,t\}\-\\bar\{r\}\_\{i\}\\right\),\\;m\_\{\\min\},\\,m\_\{\\max\}\\right\),\\qquad t\\in\\mathcal\{V\}\(y\_\{i\}\),\(3\.5\)whereλmod\\lambda\_\{\\mathrm\{mod\}\}is a modulation\-strength hyperparameter and\[mmin,mmax\]\[m\_\{\\min\},m\_\{\\max\}\]keeps the multipliers bounded and positive\. The centering step makes each multiplier depend on the token’s relative credit within the same response, and the clipping range is chosen such thatwi,t\>0w\_\{i,t\}\>0\. The token\-level advantage is then defined as

A~i,t=A^i​wi,t\.\\tilde\{A\}\_\{i,t\}=\\hat\{A\}\_\{i\}w\_\{i,t\}\.Sincewi,tw\_\{i,t\}is strictly positive, SRPO preserves the reward\-induced direction of GRPO while affecting only the token\-wise update magnitude\.

### 3\.3Optimization Objective

SRPO keeps the GRPO clipped surrogate unchanged, but replaces the uniform sequence\-level advantage with the token\-level advantageA~i,t=A^i​wi,t\\tilde\{A\}\_\{i,t\}=\\hat\{A\}\_\{i\}w\_\{i,t\}:

𝒥SRPO​\(θ\)=𝔼​\[1G​∑i=1G1Ti​∑t=1Timin⁡\(ρi,t​\(θ\)​A~i,t,clip​\(ρi,t​\(θ\),1−ϵ,1\+ϵ\)​A~i,t\)\],\\mathcal\{J\}\_\{\\mathrm\{SRPO\}\}\(\\theta\)=\\mathbb\{E\}\\left\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\frac\{1\}\{T\_\{i\}\}\\sum\_\{t=1\}^\{T\_\{i\}\}\\min\\left\(\\rho\_\{i,t\}\(\\theta\)\\tilde\{A\}\_\{i,t\},\\,\\mathrm\{clip\}\\bigl\(\\rho\_\{i,t\}\(\\theta\),1\-\\epsilon,1\+\\epsilon\\bigr\)\\tilde\{A\}\_\{i,t\}\\right\)\\right\],whereρi,t​\(θ\)\\rho\_\{i,t\}\(\\theta\)is the PPO importance ratio defined in Section[3\.1](https://arxiv.org/html/2605.07274#S3.SS1)\. The modulation weightswi,tw\_\{i,t\}are computed from stop\-gradient contrasts underπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}and are fixed during the policy update\. Therefore, SRPO preserves the GRPO optimization form while introducing role\-aware token\-level reweighting\.

However, redistributing update magnitudes across response tokens may amplify updates on unstable or low\-confidence tokens in multimodal reasoning trajectories, increasing rollout uncertainty and degrading training stability\. To mitigate this issue, we introduce a lightweight response\-token uncertainty penalty along the sampled trajectory\. This term is used only in the final optimization objective and does not affect the stop\-gradient credit scores or the modulation weights\. Concretely, for each sampled response, we define the response\-token uncertainty surrogate as

𝒥resp​\(θ\)=𝔼​\[1G​∑i=1G1\|𝒱​\(yi\)\|​∑t∈𝒱​\(yi\)−log⁡πθ​\(yi,t∣I,q,yi,<t\)\]\.\\mathcal\{J\}\_\{\\mathrm\{resp\}\}\(\\theta\)=\\mathbb\{E\}\\left\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\frac\{1\}\{\|\\mathcal\{V\}\(y\_\{i\}\)\|\}\\sum\_\{t\\in\\mathcal\{V\}\(y\_\{i\}\)\}\-\\log\\pi\_\{\\theta\}\\left\(y\_\{i,t\}\\mid I,q,y\_\{i,<t\}\\right\)\\right\]\.The final regularized objective is defined as

𝒥total​\(θ\)=𝒥SRPO​\(θ\)−η​𝒥resp​\(θ\),\\mathcal\{J\}\_\{\\mathrm\{total\}\}\(\\theta\)=\\mathcal\{J\}\_\{\\mathrm\{SRPO\}\}\(\\theta\)\-\\eta\\,\\mathcal\{J\}\_\{\\mathrm\{resp\}\}\(\\theta\),\(3\.6\)whereη\\etacontrols the strength of the uncertainty penalty\. We maximize𝒥total​\(θ\)\\mathcal\{J\}\_\{\\mathrm\{total\}\}\(\\theta\)during policy optimization\. See Appendix[A](https://arxiv.org/html/2605.07274#A1)for the full training procedure\.

## 4Experiments

### 4\.1Experimental Setup

#### Models, Data, and Baselines\.

We evaluate our method on Qwen2\.5\-VL\-3B and Qwen2\.5\-VL\-7B\(Baiet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib32)\)\. Both models are trained on ViRL39K\(Wanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib5)\), a collection of approximately 39K verifiable multimodal reasoning problems covering diverse visual inputs such as diagrams, charts, and natural images\. We compare against recent open\-source multimodal reasoning models at comparable scales, including Vision\-SR1\(Liet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib12)\), ThinkLite\-VL, PAPO\(Wanget al\.,[2026d](https://arxiv.org/html/2605.07274#bib.bib11)\), Perception\-R1\(Xiaoet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib8)\), Vision\-Matters\(Liet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib35)\), VL\-Rethinker\(Wanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib5)\), VPPO\(Huanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib21)\), and PRCO\(Miaoet al\.,[2026](https://arxiv.org/html/2605.07274#bib.bib13)\)\. For controlled comparisons, we also train GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib33)\)and DAPO\(Yuet al\.,[2026b](https://arxiv.org/html/2605.07274#bib.bib34)\)baselines using the same backbones, data, reward function, and rollout configuration\.

#### Training Details\.

All models are trained with the EasyR1\(Zhenget al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib36)\)codebase using AdamW\(Loshchilov and Hutter,[2017](https://arxiv.org/html/2605.07274#bib.bib37)\)with a learning rate of1×10−61\\times 10^\{\-6\}\. We train each model for two epochs on ViRL39K\(Wanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib5)\)with a rollout batch size of 384 and sample 8 responses per prompt for group\-relative advantage estimation\. All controlled baselines and ablation variants use the same backbone, data, rollout budget, and optimizer settings unless otherwise specified\. Detailed SRPO hyperparameters, including visual masking, modulation clipping, and regularization settings, are provided in Appendix[B\.1](https://arxiv.org/html/2605.07274#A2.SS1)\.

#### Evaluation Benchmarks\.

We evaluate on nine multimodal reasoning benchmarks covering math\-related and general multimodal reasoning tasks\. The math\-related benchmarks include MathVista\(Luet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib58)\), MathVision\(Wanget al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib41)\), We\-Math\(Qiaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib39)\), MathVerse\(Zhanget al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib42)\), and DynaMath\(Zouet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib44)\)\. The general\-task benchmarks include MMMU\-Pro\(Yueet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib38)\), MM\-Vet\(Yuet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib46)\), LogicVista\(Xiaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib40)\), and NaturalBench\(Liet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib54)\)\. All models are evaluated with VLMEvalKit\(Duanet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib43)\)using a fixed greedy decoding configuration with temperature 0 and top\-pp1\.0\. We report single\-sample performance under each benchmark’s official VLMEvalKit metric, referred to as accuracy for simplicity\. When a benchmark requires LLM\-as\-a\-judge evaluation, we use GPT\-4o\-mini as the judge model\. Additional details are provided in Appendix[B\.2](https://arxiv.org/html/2605.07274#A2.SS2)\.

### 4\.2Main Results

![Refer to caption](https://arxiv.org/html/2605.07274v1/x3.png)Figure 3:Comparison of training dynamics on the accuracy reward\. Solid lines indicate running averages with a stepping window size of 20\.Figure[3](https://arxiv.org/html/2605.07274#S4.F3)and Table[4\.2](https://arxiv.org/html/2605.07274#S4.SS2)jointly demonstrate the effectiveness of SRPO from optimization and downstream evaluation perspectives\. During training, SRPO achieves consistently higher accuracy rewards than GRPO and DAPO on both the 3B and 7B backbones, maintaining a clear advantage after the early optimization stage until convergence\. This indicates that role\-aware token\-level credit assignment provides a more effective learning signal than broadcasting the same sequence\-level reward to all tokens\. By assigning targeted credits to perception and reasoning tokens according to their functional roles, SRPO offers denser and more informative supervision for policy optimization\. This optimization advantage translates into consistent downstream gains: across Qwen2\.5\-VL\-3B and Qwen2\.5\-VL\-7B, SRPO improves over the base models and standard RL baselines, while also achieving the best overall average among recent multimodal reasoning methods\. The gains hold across math\-related and general\-task categories, suggesting that SRPO improves multimodal reasoning broadly rather than specializing to a narrow class of benchmarks\.

Table 2:Main results on nine multimodal reasoning benchmarks with Qwen2\.5\-VL\-3B and Qwen2\.5\-VL\-7B backbones\. We report benchmark scores on math\-related benchmarks, general\-task benchmarks, and their overall average\. The best and second\-best results within each backbone are highlighted in bold and underlined, respectively\.### 4\.3Pass@k Evaluation

![Refer to caption](https://arxiv.org/html/2605.07274v1/x4.png)Figure 4:Pass@k performance comparison on MathVision and MMMU\-Pro for 3B and 7B models\.We further evaluate pass@k performance on MathVision\(Wanget al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib41)\)and MMMU\-Pro\(Yueet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib38)\)to assess whether the model can benefit from increased sampling budgets\. As shown in Figure[4](https://arxiv.org/html/2605.07274#S4.F4), SRPO consistently achieves strong pass@k performance across both benchmarks and model scales\. The advantage is especially clear for the 3B models, where SRPO outperforms competing methods across different values ofkk\. For the 7B models, SRPO remains competitive on MathVision and shows consistent gains on MMMU\-Pro\. These results indicate that SRPO improves not only single\-sample accuracy but also the likelihood of sampling correct reasoning trajectories, suggesting stronger reasoning robustness under multiple attempts\.

### 4\.4Ablation Study

Table 3:Ablation studies on Qwen2\.5\-VL\-7B\. The base model is included for reference, followed by ablations on design components and entropy coefficient\. The full SRPO setting is highlighted in blue\.SettingMath\-RelatedGeneral TaskMathVistaMathVisionWe\-MathMathVerseDynaMathMMMU\-ProMM\-VetLogicVistaNaturalBenchQwen2\.5\-VL\-7B69\.8027\.3035\.1433\.3751\.2930\.7355\.3742\.9577\.67\\rowcolorblockgrayComponent Ablation\+ SRPO w/o perception credit73\.6030\.2640\.8642\.7656\.4137\.6457\.2947\.6578\.18\+ SRPO w/o reasoning credit73\.1029\.2739\.9043\.0255\.9435\.7257\.3445\.1978\.07\+ SRPO w/o unified modulation72\.0026\.6439\.8140\.7354\.5334\.7955\.8244\.5278\.05\\rowcolorfullblue \+ SRPO76\.3034\.5446\.8644\.9258\.2238\.0960\.5750\.5678\.62\\rowcolorblockgrayEntropy\-Coefficient Ablationη=0\\eta=073\.3027\.3029\.3337\.1856\.3134\.5157\.2944\.0778\.02η=0\.1\\eta=0\.174\.9031\.2540\.3843\.4056\.5235\.7857\.7147\.4278\.14\\rowcolorfullblueη=0\.03\\eta=0\.0376\.3034\.5446\.8644\.9258\.2238\.0960\.5750\.5678\.62

![Refer to caption](https://arxiv.org/html/2605.07274v1/x5.png)Figure 5:Training dynamics of SRPO ablations on Qwen2\.5\-VL\-7B\. The left panel compares component ablations, and the right panel compares entropy\-coefficient variants\. SRPO with all components andη=0\.03\\eta=0\.03achieves the strongest and most stable training trajectory\. The 3B ablation results are presented in Appendix[C](https://arxiv.org/html/2605.07274#A3)\.
#### Ablation Study on SRPO Components\.

We first evaluate the contribution of the three core components in SRPO: self\-distilled visual dependency for perception tokens, self\-distilled grounding consistency for reasoning tokens, and unified trajectory\-level modulation\. As shown in the upper block of Table[4\.4](https://arxiv.org/html/2605.07274#S4.SS4), removing any component degrades performance on Qwen2\.5\-VL\-7B, while the full SRPO variant achieves the best results across all reported benchmarks\. This indicates that SRPO’s gains are not driven by a single isolated signal, but by the coordinated use of role\-aware credit assignment and unified token\-level modulation\. Removing perception\-specific credit weakens the visual\-dependency signal used to identify visually grounded perception tokens, leading to lower performance across both math\-related and general\-task benchmarks\. Removing reasoning\-specific credit also reduces performance, suggesting that measuring whether reasoning tokens are supported by the generated perception is important for perception\-conditioned inference\. The largest degradation is observed when unified modulation is removed, indicating that role\-specific scores should be centered at the trajectory level and converted into bounded positive weights to preserve the reward\-induced update direction while adjusting token\-wise update magnitudes\. The left panel of Figure[5](https://arxiv.org/html/2605.07274#S4.F5)provides consistent optimization evidence: the full SRPO variant maintains a stronger training accuracy trajectory than its component\-ablated variants\. Further ablations on the 3B backbone are presented in Appendix[C](https://arxiv.org/html/2605.07274#A3)\.

#### Effect of Response\-Token Uncertainty Penalty\.

We further study the coefficient of the response\-token uncertainty penalty in the lower block of Table[4\.4](https://arxiv.org/html/2605.07274#S4.SS4)\. Without this regularization, i\.e\.,η=0\\eta=0, SRPO still improves over the base model but underperforms the regularized variants on most benchmarks, suggesting that role\-aware token reweighting benefits from an additional stability constraint on sampled response tokens\. This is consistent with our objective design: token\-level modulation redistributes update magnitudes across long multimodal reasoning trajectories, and the uncertainty penalty helps prevent unstable or low\-confidence tokens from being overly amplified during policy optimization\. However, using a larger coefficient,η=0\.1\\eta=0\.1, is also inferior to the default settingη=0\.03\\eta=0\.03\. This indicates that an overly strong uncertainty penalty can over\-constrain the policy update and weaken reward\-directed adaptation\. A moderate coefficient therefore provides a better balance between stable token\-level optimization and effective reward maximization\. The right panel of Figure[5](https://arxiv.org/html/2605.07274#S4.F5)further supports this conclusion:η=0\.03\\eta=0\.03follows a stronger and more stable training trajectory than both the unregularized variant and the variant with an overly large penalty\. Additional ablation results on the 3B backbone are presented in Appendix[C](https://arxiv.org/html/2605.07274#A3)\.

### 4\.5Case Study

![Refer to caption](https://arxiv.org/html/2605.07274v1/x6.png)Figure 6:Qualitative case study of SRPO on representative multimodal reasoning examples\. Purple highlights indicate perception evidence tokens, while orange highlights indicate perception\-supported reasoning tokens\. SRPO first extracts task\-relevant visual evidence in the perception segment and then uses it to support the subsequent reasoning process\.Figure[6](https://arxiv.org/html/2605.07274#S4.F6)presents two qualitative examples illustrating the role\-aware credit assignment behavior of SRPO\. Purple highlights denote perception tokens that extract task\-relevant visual evidence, while orange highlights denote reasoning tokens that operate on the perceived evidence\. In case \(a\), SRPO first identifies the geometric roles in the cone diagram, including the heightA​OAO, the base radiusO​B=5OB=5, and the slant heightA​B=9\.434AB=9\.434\. These perceived facts are then used to form the right triangleA​O​BAOBand apply the Pythagorean theorem, leading to the correct heightA​O=8AO=8\. In case \(b\), SRPO extracts the question\-relevant chart evidence, including item names, store categories, and bar values for each item\. The reasoning segment then applies the condition “less than 5 units in at least one store” to each item and counts the satisfying items\. These examples show that SRPO assigns larger credit to tokens that form an evidence\-grounded reasoning chain, rather than uniformly reinforcing all tokens from a successful response\. More qualitative analyses are provided in Appendix[F](https://arxiv.org/html/2605.07274#A6)\.

## 5Conclusion

In this paper, we present SRPO, a role\-aware policy optimization method for multimodal RLVR\. SRPO decomposes responses into perception and reasoning stages, derives self\-distilled token\-level credit for each role, and reweights the GRPO advantage through bounded positive modulation\. This preserves the verifiable reward and optimization direction without external reward models or separate teachers\. Experiments across diverse multimodal reasoning benchmarks show that SRPO improves training dynamics and downstream performance, with ablations validating its role\-aware design\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang, H\. Zhong, Y\. Zhu, M\. Yang, Z\. Li, J\. Wan, P\. Wang, W\. Ding, Z\. Fu, Y\. Xu, J\. Ye, X\. Zhang, T\. Xie, Z\. Cheng, H\. Zhang, Z\. Yang, H\. Xu, and J\. Lin \(2025\)Qwen2\.5\-vl technical report\.arXiv preprint arXiv:2502\.13923\.Cited by:[§B\.1](https://arxiv.org/html/2605.07274#A2.SS1.SSS0.Px2.p1.2),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- Ground\-r1: incentivizing grounded visual reasoning via reinforcement learning\.arXiv preprint arXiv:2505\.20272\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Chen, L\. Li, T\. Xi, L\. Zeng, and J\. Wang \(2025\)Perception before reasoning: two\-stage reinforcement learning for visual reasoning in vision\-language models\.arXiv preprint arXiv:2509\.13031\.Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p2.1),[§1](https://arxiv.org/html/2605.07274#S1.p3.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Ding, M\. Chen, Z\. Feng, T\. Xiao, W\. Qu, W\. Shao, and Y\. Fu \(2025\)VTPerception\-r1: enhancing multimodal reasoning via explicit visual and textual perceptual grounding\.arXiv preprint arXiv:2509\.24776\.Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p3.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- H\. Duan, J\. Yang, Y\. Qiao, X\. Fang, L\. Chen, Y\. Liu, X\. Dong, Y\. Zang, P\. Zhang, J\. Wang,et al\.\(2024\)Vlmevalkit: an open\-source toolkit for evaluating large multi\-modality models\.InProceedings of the 32nd ACM International Conference on Multimedia,pp\. 11198–11201\.Cited by:[§B\.2](https://arxiv.org/html/2605.07274#A2.SS2.SSS0.Px1.p1.1),[§B\.2](https://arxiv.org/html/2605.07274#A2.SS2.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1)\.
- H\. Fang, C\. Zhou, J\. Kong, K\. Gao, B\. Chen, T\. Liang, G\. Ma, and S\. Xia \(2026\)Grounding language with vision: a conditional mutual information calibrated decoding strategy for reducing hallucinations in LVLMs\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p2.1)\.
- A\. Favero, L\. Zancato, M\. Trager, S\. Choudhary, P\. Perera, A\. Achille, A\. Swaminathan, and S\. Soatto \(2024\)Multi\-modal hallucination control by visual information grounding\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Cited by:[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1)\.
- Gemini Team, P\. Georgiev, V\. I\. Lei, R\. Burnell, L\. Bai,et al\.\(2024\)Gemini 1\.5: Unlocking multimodal understanding across millions of tokens of context\.arXiv e\-prints\.External Links:2403\.05530Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Ghosh, C\. K\. R\. Evuru, S\. Kumar, U\. Tyagi, O\. Nieto, Z\. Jin, and D\. Manocha \(2025\)Visual description grounding reduces hallucinations and boosts reasoning in LVLMs\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p2.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Guo, L\. Xu, J\. Liu, Y\. Dan, and S\. Qiu \(2026\)Segment policy optimization: effective segment\-level credit assignment in RL for large language models\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1)\.
- Y\. He, C\. Huang, Z\. Li, J\. Huang, and Y\. Yang \(2025\)Visplay: self\-evolving vision\-language models from images\.arXiv preprint arXiv:2511\.15661\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Huang, X\. Qu, Y\. Li, Y\. Luo, Z\. He, D\. Liu, and Y\. Cheng \(2026a\)Spotlight on token perception for multimodal reinforcement learning\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§1](https://arxiv.org/html/2605.07274#S1.p3.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- W\. Huang, B\. Jia, S\. Cao, Z\. Ye, F\. zhao, Z\. Xu, Y\. Hu, and S\. Lin \(2026b\)Vision\-r1: incentivizing reasoning capability in multimodal large language models\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§1](https://arxiv.org/html/2605.07274#S1.p3.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- B\. Li, Z\. Lin, W\. Peng, J\. de Dieu Nyandwi, D\. Jiang, Z\. Ma, S\. Khanuja, R\. Krishna, G\. Neubig, and D\. Ramanan \(2024\)NaturalBench: evaluating vision\-language models on natural adversarial samples\.InThe Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[9th item](https://arxiv.org/html/2605.07274#A2.I1.i9.p1.1),[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1)\.
- Y\. Li, L\. Wei, K\. Zheng, J\. Huang, L\. Kong, L\. Sun, and W\. Huang \(2025\)Vision matters: simple visual perturbations can boost multimodal math reasoning\.arXiv preprint arXiv:2506\.09736\.Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- Z\. Li, W\. Yu, C\. Huang, Z\. Liang, R\. Liu, F\. Liu, J\. Chen, D\. Yu, J\. L\. Boyd\-Graber, H\. Mi, and D\. Yu \(2026\)Vision\-SR1: self\-rewarding vision\-language model via reasoning decomposition and multi\-reward policy optimization\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- X\. Liu, J\. Ni, Z\. Wu, C\. Du, L\. Dou, H\. Wang, T\. Pang, and M\. Q\. Shieh \(2026a\)NoisyRollout: reinforcing visual reasoning with data augmentation\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Liu, T\. Qu, Z\. Zhong, B\. PENG, S\. Liu, B\. Yu, and J\. Jia \(2026b\)VisionReasoner: unified reasoning\-integrated visual perception via reinforcement learning\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2017\)Decoupled weight decay regularization\.arXiv preprint arXiv:1711\.05101\.Cited by:[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px2.p1.1)\.
- J\. Lu, J\. Wu, J\. Li, K\. Huang, S\. Yang, G\. Wang, J\. Wu, X\. Wang, and X\. He \(2026\)Bridging perception and reasoning: token reweighting for rlvr in multimodal llms\.arXiv preprint arXiv:2603\.25077\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1)\.
- P\. Lu, H\. Bansal, T\. Xia, J\. Liu, C\. Li, H\. Hajishirzi, H\. Cheng, K\. Chang, M\. Galley, and J\. Gao \(2024\)MathVista: evaluating mathematical reasoning of foundation models in visual contexts\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[1st item](https://arxiv.org/html/2605.07274#A2.I1.i1.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1)\.
- T\. Luo, A\. Cao, G\. Lee, J\. Johnson, and H\. Lee \(2025\)Probing visual language priors in VLMs\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=bhTBirS0qi)Cited by:[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1)\.
- Z\. Miao, H\. Jia, L\. Li, C\. Qian, Y\. Xiong, W\. Yan, and J\. Shao \(2026\)Seeing with you: perception\-reasoning coevolution for multimodal reasoning\.arXiv preprint arXiv:2603\.28618\.Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§1](https://arxiv.org/html/2605.07274#S1.p3.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- R\. Qiao, Q\. Tan, G\. Dong, M\. Wu, C\. Sun, X\. Song, Z\. GongQue, S\. Lei, Z\. Wei, M\. Zhang,et al\.\(2024\)We\-math: does your large multimodal model achieve human\-like mathematical reasoning?\.arXiv preprint arXiv:2407\.01284\.Cited by:[3rd item](https://arxiv.org/html/2605.07274#A2.I1.i3.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1)\.
- G\. H\. Sarch, S\. Saha, N\. Khandelwal, A\. Jain, M\. J\. Tarr, A\. Kumar, and K\. Fragkiadaki \(2026\)Grounded reinforcement learning for visual reasoning\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\.K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.Vol\.abs/2402\.03300\.Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2605.07274#S3.SS1.p1.3),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- S\. Sharma, S\. Gupta, and S\. Sadbhawna \(2026\)See, think, learn: a self\-taught multimodal reasoner\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p3.1)\.
- A\. Su, H\. Wang, W\. Ren, F\. Lin, and W\. Chen \(2026\)Pixel reasoner: incentivizing pixel space reasoning via curiosity\-driven reinforcement learning\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- W\. Sun, H\. Chen, Y\. Du, Y\. Zheng, and C\. G\. M\. Snoek \(2026\)RegionReasoner: region\-grounded multi\-round visual reasoning\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- H\. Tran, Z\. Yao, and H\. Yu \(2025\)Exploiting tree structure for credit assignment in rl training of llms\.arXiv preprint arXiv:2509\.18314\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Wang, C\. Qu, Z\. Huang, W\. Chu, F\. Lin, and W\. Chen \(2026a\)VL\-rethinker: incentivizing self\-reflection of vision\-language models with reinforcement learning\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§B\.1](https://arxiv.org/html/2605.07274#A2.SS1.SSS0.Px2.p1.2),[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px2.p1.1)\.
- K\. Wang, J\. Pan, W\. Shi, Z\. Lu, H\. Ren, A\. Zhou, M\. Zhan, and H\. Li \(2024\)Measuring multimodal mathematical reasoning with math\-vision dataset\.InThe Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[2nd item](https://arxiv.org/html/2605.07274#A2.I1.i2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2605.07274#S4.SS3.p1.1)\.
- P\. Wang, Y\. Wei, Y\. Peng, X\. Wang, W\. Qiu, W\. Shen, T\. Xie, J\. Pei, J\. Zhang, Y\. Hao,et al\.\(2025\)Skywork r1v2: multimodal hybrid reinforcement learning for reasoning\.arXiv preprint arXiv:2504\.16656\.Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Wang, L\. Yu, C\. Gao, C\. Zheng, S\. Liu, R\. Lu, K\. Dang, X\. Chen, J\. Yang, Z\. Zhang, Y\. Liu, A\. Yang, A\. Zhao, Y\. Yue, S\. Song, B\. Yu, G\. Huang, and J\. Lin \(2026b\)Beyond the 80/20 rule: high\-entropy minority tokens drive effective reinforcement learning for LLM reasoning\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1)\.
- Z\. Wang, F\. Xiong, L\. Lin, X\. Hu, Y\. Wang, Y\. Wang, M\. Zhang, and X\. Chu \(2026c\)Visually\-guided policy optimization for multimodal reasoning\.arXiv preprint arXiv:2604\.09349\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Wang, X\. Guo, S\. Stoica, H\. Xu, H\. WANG, H\. Ha, X\. Chen, Y\. Chen, M\. Yan, F\. Huang, and H\. Ji \(2026d\)Perception\-aware policy optimization for multimodal reasoning\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§B\.1](https://arxiv.org/html/2605.07274#A2.SS1.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2605.07274#S1.p2.1),[§1](https://arxiv.org/html/2605.07274#S1.p3.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- J\. Xia, Y\. Zang, P\. Gao, S\. Li, and K\. Zhou \(2025\)Visionary\-r1: mitigating shortcuts in visual reasoning with reinforcement learning\.arXiv preprint arXiv:2505\.14677\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- T\. Xiao, X\. Xu, Z\. Huang, H\. Gao, Q\. Liu, Q\. Liu, and E\. Chen \(2025\)Perception\-r1: advancing multimodal reasoning capabilities of mllms via visual perception reward\.arXiv preprint arXiv:2506\.07218\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- Y\. Xiao, E\. Sun, T\. Liu, and W\. Wang \(2024\)LogicVista: multimodal llm logical reasoning benchmark in visual contexts\.External Links:2407\.04973Cited by:[8th item](https://arxiv.org/html/2605.07274#A2.I1.i8.p1.1),[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1),[Table 1](https://arxiv.org/html/2605.07274#S3.T1),[Table 1](https://arxiv.org/html/2605.07274#S3.T1.15.2),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1)\.
- H\. Xu, Q\. Zhu, H\. Deng, J\. Li, L\. Hou, Y\. Wang, L\. Shang, R\. Xu, and F\. Mi \(2025\)Kdrl: post\-training reasoning llms via unified knowledge distillation and reinforcement learning\.arXiv preprint arXiv:2506\.02208\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Yang, C\. Qin, Q\. Si, M\. Chen, N\. Gu, D\. Yao, Z\. Lin, W\. Wang, J\. Wang, and N\. Duan \(2026\)Self\-distilled rlvr\.arXiv preprint arXiv:2604\.03128\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Yao, Q\. Yin, J\. Zhang, M\. Yang, Y\. Wang, W\. Wu, F\. Su, L\. Shen, M\. Qiu, D\. Tao,et al\.\(2025\)R1\-sharevl: incentivizing reasoning capabilities of multimodal large language models via share\-grpo\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§1](https://arxiv.org/html/2605.07274#S1.p3.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Ye, Q\. Li, X\. Feng, R\. Chen, Z\. Li, H\. Ren, K\. Chen, D\. Tu, and B\. Qin \(2026\)Not all tokens see equally: perception\-grounded policy optimization for large vision\-language models\.arXiv preprint arXiv:2604\.01840\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2605.07274#S3.SS2.p1.1)\.
- E\. Yu, K\. Lin, L\. Zhao, jisheng yin, Y\. Wei, Y\. Peng, H\. Wei, J\. Sun, C\. Han, Z\. Ge, X\. Zhang, D\. Jiang, J\. Wang, and W\. Tao \(2026a\)Perception\-r1: pioneering perception policy with reinforcement learning\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p3.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px2.p1.1)\.
- Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, YuYue, W\. Dai, T\. Fan, G\. Liu, J\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, R\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, Y\. Wu, and M\. Wang \(2026b\)DAPO: an open\-source LLM reinforcement learning system at scale\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px1.p1.1)\.
- W\. Yu, Z\. Yang, L\. Li, J\. Wang, K\. Lin, Z\. Liu, X\. Wang, and L\. Wang \(2024\)Mm\-vet: evaluating large multimodal models for integrated capabilities\.InInternational conference on machine learning,Cited by:[7th item](https://arxiv.org/html/2605.07274#A2.I1.i7.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1)\.
- X\. Yue, T\. Zheng, Y\. Ni, Y\. Wang, K\. Zhang, S\. Tong, Y\. Sun, B\. Yu, G\. Zhang, H\. Sun, Y\. Su, W\. Chen, and G\. Neubig \(2024\)MMMU\-Pro: A More Robust Multi\-discipline Multimodal Understanding Benchmark\.External Links:2409\.02813Cited by:[6th item](https://arxiv.org/html/2605.07274#A2.I1.i6.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2605.07274#S4.SS3.p1.1)\.
- Y\. Zha, K\. Zhou, Y\. Wu, Y\. Wang, J\. Feng, Z\. Xu, S\. Hao, Z\. Liu, E\. P\. Xing, and Z\. Hu \(2026\)Vision\-g1: towards general reasoning vision\-language models via reinforcement learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Zhang, J\. Huang, H\. Yao, S\. Liu, X\. Zhang, S\. Lu, and D\. Tao \(2025\)R1\-vl: learning to reason with multimodal large language models via step\-wise group relative policy optimization\.arXiv preprint arXiv:2503\.12937\.Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p1.1),[§1](https://arxiv.org/html/2605.07274#S1.p3.1)\.
- R\. Zhang, D\. Jiang, Y\. Zhang, H\. Lin, Z\. Guo, P\. Qiu, A\. Zhou, P\. Lu, K\. Chang, P\. Gao,et al\.\(2024\)MathVerse: does your multi\-modal llm truly see the diagrams in visual math problems?\.arXiv preprint arXiv:2403\.14624\.Cited by:[4th item](https://arxiv.org/html/2605.07274#A2.I1.i4.p1.1),[§1](https://arxiv.org/html/2605.07274#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1)\.
- Z\. Zhang, S\. Jiang, Y\. Shen, Y\. Zhang, D\. Ram, S\. Yang, Z\. Tu, W\. Xia, and S\. Soatto \(2026\)Reinforcement\-aware knowledge distillation for llm reasoning\.arXiv preprint arXiv:2602\.22495\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px3.p1.1)\.
- Y\. Zheng, J\. Lu, S\. Wang, Z\. Feng, D\. Kuang, Y\. Xiong, and R\. Zhang \(2025\)EasyR1: an efficient, scalable, multi\-modality rl training framework\.Cited by:[§B\.1](https://arxiv.org/html/2605.07274#A2.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px2.p1.1)\.
- J\. Zhou, S\. Wang, J\. Dong, K\. Liu, L\. Li, J\. Gao, J\. Jiang, L\. Kong, and C\. Wu \(2025\)PROREASON: multi\-modal proactive reasoning with decoupled eyesight and wisdom\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,Cited by:[§1](https://arxiv.org/html/2605.07274#S1.p3.1)\.
- J\. Zhu, W\. Wang, Z\. Chen, Z\. Liu, S\. Ye, L\. Gu, H\. Tian, Y\. Duan, W\. Su, J\. Shao,et al\.\(2025\)Internvl3: exploring advanced training and test\-time recipes for open\-source multimodal models\.arXiv preprint arXiv:2504\.10479\.Cited by:[§2](https://arxiv.org/html/2605.07274#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Zou, X\. Guo, R\. Yang, J\. Zhang, B\. Hu, and H\. Zhang \(2025\)DynaMath: a dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models\.InThe Thirteenth International Conference on Learning Representations,Cited by:[5th item](https://arxiv.org/html/2605.07274#A2.I1.i5.p1.1),[§4\.1](https://arxiv.org/html/2605.07274#S4.SS1.SSS0.Px3.p1.1)\.

## Appendix

## Appendix ATraining Procedure

For clarity and reproducibility, we provide a step\-by\-step description of the SRPO training procedure in Algorithm[1](https://arxiv.org/html/2605.07274#alg1)\. The algorithm elaborates on the role\-aware credit assignment mechanism introduced in Section[3\.2](https://arxiv.org/html/2605.07274#S3.SS2)\. Each training iteration consists of four main stages\. First, the behavior policy generates a group of on\-policy responses for each image–question pair under the structured perception–reasoning template\. Second, a verifier assigns outcome\-level rewards based on format compliance and final\-answer correctness, from which group\-normalized GRPO advantages are computed\. Third, SRPO parses each response into perception and reasoning spans and performs stop\-gradient rescoring with the behavior policy under controlled context changes: perception tokens are rescored under the original and masked images to estimate visual dependency, while reasoning tokens are rescored with and without the generated perception to estimate perception\-supported grounding consistency\. Finally, the role\-specific scores are centered within each trajectory, converted into bounded positive token\-level modulation weights, and used to form token\-level advantages for the PPO\-style policy update\. The response\-token entropy surrogate is added only in the final optimization objective and does not affect the stop\-gradient credit scores or modulation weights\.

## Appendix BImplementation and Evaluation Details

### B\.1Implementation Details

#### Overall setup\.

Our implementation is built on the EasyR1 framework\[Zhenget al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib36)\]\. We conduct experiments with Qwen2\.5\-VL\-3B and Qwen2\.5\-VL\-7B as base models\. Unless otherwise specified, both model scales use the same training recipe\. All main experiments are trained on 8 NVIDIA H100 GPUs, and the vision tower is unfrozen during training\.

#### RL training configuration\.

We train all models for two epochs on ViRL39K\[Wanget al\.,[2026a](https://arxiv.org/html/2605.07274#bib.bib5)\]\. For each prompt, the policy samples 8 responses, and the group\-relative advantage is computed from verifiable outcome rewards\. The verifier reward combines format compliance and answer correctness\. In our implementation, answer correctness is a binary signal: a response receives accuracy reward 1 if its final answer matches the ground\-truth answer under the equivalence checker, and 0 otherwise\. The rollout batch size is 384, the actor global batch size is 128, and the validation batch size is 512\. The maximum response length is set to 2048 for both Qwen2\.5\-VL\-3B and Qwen2\.5\-VL\-7B\[Baiet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib32)\]\. During rollout, we use top\-ppsampling withp=0\.99p=0\.99\. For policy optimization, we adopt asymmetric PPO clipping with lower and upper clipping ratios of 0\.2 and 0\.28, respectively\. Online filtering is enabled, and no explicit reference\-policy KL loss is used\.

#### Self\-distilled credit assignment\.

For perception\-token credit assignment, SRPO constructs a weak visual branch by masking the image\. Specifically, we use random patch blackening: the image is divided into non\-overlapping14×1414\\times 14patches, and each patch is independently masked with probability 0\.5\[Wanget al\.,[2026d](https://arxiv.org/html/2605.07274#bib.bib11)\]\. The perception\-token credit is computed from the log\-probability contrast between the original\-image branch and the masked\-image branch\. The resulting perception scores are then passed to the unified trajectory\-level modulation step together with the reasoning\-token scores\.

For reasoning\-token credit assignment, SRPO compares each reasoning token under two textual conditioning contexts\. The first is the full rollout context, which includes the image, question, generated perception, and previous reasoning tokens\. The second is a perception\-ablated context, which removes the generated perception while retaining the image, question, and reasoning prefix\. This contrast estimates how much the generated perception supports each reasoning token beyond what is already explained by the image–question pair and the reasoning prefix\. The perception\-ablated scoring branch uses the templateqi\_only\_reasoning\_answer\.jinja\. The resulting residual is modulated by the sign of the trajectory\-level advantage and mapped to a positive token\-level factor through an exponential transformation\. In the unified trajectory\-level modulation, we setλmod=0\.5\\lambda\_\{\\mathrm\{mod\}\}=0\.5and clip the final token weights to\[0\.8,1\.2\]\[0\.8,1\.2\]\.

Algorithm 1SRPO: Structured Role\-aware Policy Optimization0:Policy

πθ\\pi\_\{\\theta\}, dataset

𝒟\\mathcal\{D\}, group size

GG, masking operator

ℳ\\mathcal\{M\}\.

0:Updated policy

πθ\\pi\_\{\\theta\}\.

1:foreach training iterationdo

2:Set the behavior policy

πθold←πθ\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\\leftarrow\\pi\_\{\\theta\}\.

3:Sample a batch of training instances

\(I,q,a\)∼𝒟\(I,q,a\)\\sim\\mathcal\{D\}\.

4:⊳\\trianglerightPhase 1: On\-policy Rollout Generation

5:foreach

\(I,q,a\)\(I,q,a\)do

6:Sample

GGstructured rollouts

\{yi=\(yiperc,yireas\)\}i=1G∼πθold\(⋅∣I,q\)\\\{y\_\{i\}=\(y\_\{i\}^\{\\mathrm\{perc\}\},y\_\{i\}^\{\\mathrm\{reas\}\}\)\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid I,q\)\.

7:Compute verifier rewards

RiR\_\{i\}and group\-relative advantages

A^i\\hat\{A\}\_\{i\}\.

8:endfor

9:⊳\\trianglerightPhase 2: Role\-specific Stop\-gradient Rescoring

10:foreach trajectory

yiy\_\{i\}do

11:Parse perception positions

𝒫i\\mathcal\{P\}\_\{i\}, reasoning positions

ℛi\\mathcal\{R\}\_\{i\}, and valid positions

𝒱​\(yi\)\\mathcal\{V\}\(y\_\{i\}\)\.

12:Construct a masked image

Imask=ℳ​\(I\)I^\{\\mathrm\{mask\}\}=\\mathcal\{M\}\(I\)\.

13:foreach

t∈𝒱​\(yi\)t\\in\\mathcal\{V\}\(y\_\{i\}\)do

14:if

t∈𝒫it\\in\\mathcal\{P\}\_\{i\}then

15:Compute

δi,tperc\\delta\_\{i,t\}^\{\\mathrm\{perc\}\}by Eq\. \([3\.1](https://arxiv.org/html/2605.07274#S3.E1)\) using

πθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\.

16:Compute

ri,tpercr\_\{i,t\}^\{\\mathrm\{perc\}\}by Eq\. \([3\.2](https://arxiv.org/html/2605.07274#S3.E2)\)\.

17:Set

ri,t←ri,tpercr\_\{i,t\}\\leftarrow r\_\{i,t\}^\{\\mathrm\{perc\}\}\.

18:elseif

t∈ℛit\\in\\mathcal\{R\}\_\{i\}then

19:Compute

δi,treas\\delta\_\{i,t\}^\{\\mathrm\{reas\}\}by Eq\. \([3\.3](https://arxiv.org/html/2605.07274#S3.E3)\) using

πθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\.

20:Compute

ri,treasr\_\{i,t\}^\{\\mathrm\{reas\}\}by Eq\. \([3\.4](https://arxiv.org/html/2605.07274#S3.E4)\)\.

21:Set

ri,t←ri,treasr\_\{i,t\}\\leftarrow r\_\{i,t\}^\{\\mathrm\{reas\}\}\.

22:endif

23:endfor

24:endfor

25:⊳\\trianglerightPhase 3: Unified Trajectory\-level Modulation

26:foreach trajectory

yiy\_\{i\}do

27:Compute the response\-level baseline

r¯i=\|𝒱​\(yi\)\|−1​∑t∈𝒱​\(yi\)ri,t\\bar\{r\}\_\{i\}=\|\\mathcal\{V\}\(y\_\{i\}\)\|^\{\-1\}\\sum\_\{t\\in\\mathcal\{V\}\(y\_\{i\}\)\}r\_\{i,t\}\.

28:foreach

t∈𝒱​\(yi\)t\\in\\mathcal\{V\}\(y\_\{i\}\)do

29:Compute

wi,tw\_\{i,t\}by Eq\. \([3\.5](https://arxiv.org/html/2605.07274#S3.E5)\)\.

30:Form the token\-level advantage

A~i,t=A^i​wi,t\\tilde\{A\}\_\{i,t\}=\\hat\{A\}\_\{i\}w\_\{i,t\}\.

31:endfor

32:endfor

33:⊳\\trianglerightPhase 4: Policy Optimization

34:Update

θ\\thetaby maximizing the total objective in Eq\. \([3\.6](https://arxiv.org/html/2605.07274#S3.E6)\)\.

35:endfor

#### Additional actor regularization\.

To improve the stability of actor optimization, we include a lightweight sampled\-token negative\-log\-probability regularizer in the actor loss:

ℒreg=η​𝔼\(st,at\)​\[−log⁡πθ​\(at∣st\)\],\\mathcal\{L\}\_\{\\mathrm\{reg\}\}=\\eta\\,\\mathbb\{E\}\_\{\(s\_\{t\},a\_\{t\}\)\}\\left\[\-\\log\\pi\_\{\\theta\}\(a\_\{t\}\\mid s\_\{t\}\)\\right\],\(B\.1\)where the expectation is taken over valid response tokens sampled during rollout, andη=0\.03\\eta=0\.03\. This term regularizes the current policy along the sampled response trajectory and is applied only during actor optimization\. It is not used to compute the self\-distilled token scores or the modulation weights\. We note that this term is a sampled\-token entropy surrogate rather than a full\-vocabulary Shannon entropy\.

### B\.2Evaluation Details

#### Evaluation framework\.

We evaluate all models using VLMEvalKit\[Duanet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib43)\], which provides standardized benchmark loading, prompting, answer extraction, and metric computation\. For each benchmark, we follow its official VLMEvalKit evaluation protocol and report the corresponding official metric\. For simplicity, we refer to the reported benchmark score as accuracy throughout the paper\.

Table 4:Vision\-language benchmarks evaluated in VLMEvalKit\.
#### Benchmarks\.

Our evaluation covers nine vision\-language benchmarks using VLMEvalKit\[Duanet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib43)\], spanning math\-centric reasoning, multidisciplinary multimodal understanding, logical reasoning, integrated visual reasoning, and robustness evaluation\. Table[4](https://arxiv.org/html/2605.07274#A2.T4)summarizes the dataset sizes and reported metrics, and we briefly describe the evaluation focus of each benchmark below\.

- •MathVista\[Luet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib58)\]is a comprehensive benchmark for visual mathematical reasoning\. It covers diverse visual contexts such as geometry diagrams, charts, tables, and scientific figures, making it suitable for evaluating whether models can solve mathematical problems grounded in visual information\.
- •MathVision\[Wanget al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib41)\]focuses on challenging multimodal mathematical reasoning problems\. Its examples require models to interpret visual content and perform multi\-step mathematical inference, providing a strong testbed for advanced visual\-symbolic reasoning\.
- •We\-Math\[Qiaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib39)\]provides a diagnostic evaluation for multimodal mathematical reasoning\. By organizing problems around mathematical knowledge concepts and reasoning steps, it enables fine\-grained analysis of a model’s strengths and weaknesses beyond aggregate accuracy\.
- •MathVerse\[Zhanget al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib42)\]is designed to examine whether multimodal models genuinely rely on diagrams in visual math problems\. It presents problems under different visual\-textual information distributions, allowing evaluation of a model’s dependence on visual evidence rather than textual shortcuts\.
- •DynaMath\[Zouet al\.,[2025](https://arxiv.org/html/2605.07274#bib.bib44)\]evaluates the robustness and generalization of multimodal mathematical reasoning\. It introduces dynamic variations of visual math problems, testing whether models can maintain consistent reasoning under controlled changes in problem instances\.
- •MMMU\-Pro\[Yueet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib38)\]is a challenging benchmark for multidisciplinary multimodal understanding\. It reduces shortcut solutions from textual clues and requires models to integrate visual and textual information across diverse academic subjects\.
- •MM\-Vet\[Yuet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib46)\]evaluates integrated multimodal capabilities through open\-ended visual question answering\. It tests whether models can jointly use perception, knowledge, reasoning, and answer generation to solve visually grounded tasks\.
- •LogicVista\[Xiaoet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib40)\]focuses on logical reasoning in visual contexts\. It evaluates whether models can perform structured reasoning over diagrams and other visual inputs, covering capabilities beyond purely mathematical problem solving\.
- •NaturalBench\[Liet al\.,[2024](https://arxiv.org/html/2605.07274#bib.bib54)\]evaluates multimodal robustness under natural adversarial examples\. It is particularly relevant for testing whether models rely on genuine visual evidence rather than language priors or superficial image\-text correlations\.

#### Decoding configuration\.

Unless otherwise specified, all models are evaluated with single\-sample greedy decoding\. We set the temperature to 0, top\-ppto 1\.0, and the maximum number of generated tokens to 2048\. The same decoding configuration is applied to all compared models to avoid confounding benchmark performance with sampling variance or test\-time compute\.

#### Prompting protocol\.

We use the official VLMEvalKit prompts and answer extraction rules whenever available\. For each benchmark, the same prompt template and evaluation pipeline are used for all models\. This ensures that performance differences are attributable to the trained policies rather than benchmark\-specific prompt engineering or post\-processing choices\.

#### Metrics and LLM\-as\-a\-judge evaluation\.

For benchmarks with deterministic answer matching, we use the official answer extraction and exact\-match evaluation implemented in VLMEvalKit\. For benchmarks or instances that require model\-based semantic judgment, we use GPT\-4o\-mini as the LLM judge with temperature set to 0\. The judge receives the question, model prediction, and ground\-truth answer, and determines whether the prediction is semantically equivalent to the reference answer\. All judged evaluations use the same fixed judging pipeline across methods\.

### B\.3Prompt Templates

We use a shared structured response template for the main RLVR rollout generation and evaluation\. The template requires the model to explicitly decompose its response into a perception stage, a reasoning stage, and a final answer\. For fair comparison, the same main rollout template is used for GRPO, DAPO, and SRPO\. The GRPO and DAPO baselines are optimized with their original objectives under this shared response format, whereas SRPO further parses the generated perception and reasoning spans for role\-aware token\-level credit modulation\. No ground\-truth rationales or intermediate perception annotations are used\.

Prompt for Main Rollout and Inference\{Question\}Analyze the image/video and solve the problem with an explicit decomposition\.First, write a concise but self\-contained perception block that captures only the visual facts needed for solving the problem, wrapped in<perception\>and</perception\>tags\.Second, write the reasoning process grounded in the image/video and the perception block, wrapped in<reasoning\>and</reasoning\>tags\.Finally, give the answer in\\boxed\{\}\.Required format:<perception\> visual evidence here </perception\><reasoning\> grounded reasoning here </reasoning\>\\boxed\{FINAL ANSWER\}

In addition to the main rollout template, SRPO uses an auxiliary question–image rescoring template to compute the reasoning\-token support signal\. This auxiliary template removes the perception block and keeps only the reasoning span and final\-answer format\. It is used only for stop\-gradient rescoring in the rollout\-versus\-question\-image comparison, and is not used as the primary rollout or evaluation prompt\.

Prompt for Question–Image Rescoring\{Question\}Analyze the image/video and solve the problem\.Write the reasoning process grounded in the image/video, wrapped in<reasoning\>and</reasoning\>tags\.Finally, give the answer in\\boxed\{\}\.Required format:<reasoning\> grounded reasoning here </reasoning\>\\boxed\{FINAL ANSWER\}

All auxiliary rescoring computations are used only as stop\-gradient signals for credit modulation\. They do not provide additional ground\-truth rationales, do not use intermediate perception annotations, and do not introduce an auxiliary supervised loss\.

## Appendix CAblation Results on Qwen2\.5\-VL\-3B

![Refer to caption](https://arxiv.org/html/2605.07274v1/x7.png)Figure 7:Training dynamics of SRPO ablations on Qwen2\.5\-VL\-3B\. The left panel compares component ablations, and the right panel compares entropy\-coefficient variants\. Solid lines indicate running averages with a stepping window size of 20\.Table 5:Ablation studies on Qwen2\.5\-VL\-3B\. The upper block studies role\-aware design components, and the lower block studies the entropy coefficientη\\eta\. The full SRPO setting is highlighted in blue\.Table[C](https://arxiv.org/html/2605.07274#A3)reports the 3B\-scale ablation results\. The upper block shows that removing any role\-aware design component leads to consistent performance degradation compared with the full SRPO variant\. In particular, removing perception credit weakens visual\-evidence modeling, removing reasoning credit reduces perception\-supported inference, and removing unified modulation produces the largest overall degradation, confirming the importance of trajectory\-level token reweighting\. The lower block further evaluates the entropy coefficientη\\eta\. Compared withη=0\\eta=0andη=0\.1\\eta=0\.1, the default settingη=0\.03\\eta=0\.03achieves the best performance across all reported benchmarks, suggesting that a moderate regularization strength provides a better balance between stable optimization and reward\-driven improvement\. Figure[7](https://arxiv.org/html/2605.07274#A3.F7)shows consistent training dynamics: the full SRPO configuration maintains a stronger accuracy trajectory than its component\-ablated variants, whileη=0\.03\\eta=0\.03remains competitive and stable throughout training\.

## Appendix DBroader Impacts

SRPO studies reinforcement\-learning\-based post\-training for improving visually grounded multimodal reasoning\. It does not introduce external reward models, separate teacher models, or additional supervision, but instead refines how existing verifier rewards are assigned to different response tokens\. By separating perception and reasoning roles, SRPO encourages models to extract task\-relevant visual evidence before performing subsequent inference\. This may benefit applications such as educational problem solving, chart understanding, diagram reasoning, and other settings where final answers should be supported by image evidence\.

More broadly, SRPO provides an interpretable optimization perspective for multimodal RLVR\. Rather than treating all tokens in a response as equally responsible for the final outcome, it distinguishes visual evidence extraction from perception\-supported reasoning\. This can help diagnose whether improvements in multimodal reasoning come from genuine visual grounding or from shortcuts based on language priors\. However, SRPO is not a standalone guarantee of factual correctness or safety, and applications in high\-stakes domains would still require careful validation and human oversight\.

## Appendix ELimitations

SRPO introduces additional computation during training\. For each on\-policy rollout, the method performs behavior\-policy rescoring under controlled changes of visual or textual context\. These stop\-gradient rescoring passes are lightweight compared with training separate teachers or reward models, but they still increase the cost relative to vanilla GRPO\. This overhead may become more significant for larger models, longer responses, or larger rollout groups\.

Our empirical evaluation focuses on a set of multimodal reasoning benchmarks involving mathematics, charts, diagrams, logic, and general visual reasoning\. Although these benchmarks cover diverse reasoning patterns, they do not exhaust all real\-world multimodal scenarios\. Future work could evaluate SRPO on longer\-horizon multimodal tasks, interactive visual reasoning, video understanding, and domains where intermediate evidence can be more directly supervised or verified\.

## Appendix FAdditional Case Study

We provide additional qualitative examples across different benchmarks to illustrate how SRPO supports visually grounded reasoning\. Each case includes the original visual question, the ground\-truth answer, and the complete model response\. We highlight perception evidence and reasoning inference spans to show how the model first extracts task\-relevant visual information and then uses it for subsequent reasoning\. These examples complement the quantitative results by showing the perception\-to\-reasoning structure encouraged by SRPO\. Across geometry, chart, table, logic, OCR, and natural\-image tasks, the model typically grounds its final answer in explicitly identified visual evidence rather than relying only on language priors\.

Generated Sample by SRPO on MMMU\-Pro\-V![Refer to caption](https://arxiv.org/html/2605.07274v1/figures/appendix/mmmupro.png)Question:Which structure in this picture is responsible for synthesizing proteins \(assembling proteins\)?Options:A\. I; B\. II; C\. IV; D\. IIIGround truth: C<perception\>The image shows a cell with labeled structures: \-I appears to be the nucleus\.\-II appears to be the endoplasmic reticulum\.\-III appears to be the Golgi apparatus\.\-IV appears to be the ribosomes\.The question asks which structure is responsible for synthesizing proteins \(assembling proteins\)\.</perception\><reasoning\>In a cell, proteins are synthesized onribosomes\. Ribosomes are labeled asIVin the image\. The nucleus \(I\) is responsible for storing genetic information, the endoplasmic reticulum \(II\) is involved in protein transport and modification, and the Golgi apparatus \(III\) is involved in packaging and transporting proteins\. Therefore, the structure responsible for synthesizing proteins is the ribosomes\.</reasoning\>C\\boxed\{C\}Figure 8:MMMU\-Pro\-V case study\. SRPO identifies the labeled cell organelles and grounds the answer in biological knowledge that ribosomes synthesize proteins\.Generated Sample by SRPO on WeMath![Refer to caption](https://arxiv.org/html/2605.07274v1/figures/appendix/wemath.png)Question:As shown in the figure, there is an isosceles trapezoid made of wire\. This wire can also be used to form a right\-angled cube frame, and the volume of the cube frame is shown in the figure\. What is the length of the legs of the isosceles trapezoid in cm?Options:A\. 4; B\. 5; C\. 2; D\. No correct answerGround truth: C<perception\>The image showsan isosceles trapezoidanda right\-angled cube\. The trapezoid has bases of3 cmand5 cm, and the cube has a volume of1 cm3, which means each side of the cube is1 cm\.</perception\><reasoning\>First, we determine the side length of the cube\. Since the volume of the cube is 1 cm3, each side of the cube is 1 cm\.The wire used to form the isosceles trapezoid is the same as the wire used to form the cube\. The cube has12 edges, each of length 1 cm, so the total length of the wire is12 cm\.The isosceles trapezoid has two legs of equal length, sayxx, and the bases are 3 cm and 5 cm\. The perimeter of the trapezoid is the sum of all its sides:2​x\+3\+5=2​x\+8\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}2x\+3\+5=2x\+8\}Since the wire length is 12 cm, we have:2​x\+8=12\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}2x\+8=12\}2​x=42x=4x=2\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}x=2\}So, the length of the legs of the isosceles trapezoid is 2 cm\.</reasoning\>C\\boxed\{C\}Figure 9:WeMath case study\. SRPO connects visual quantities from two geometric objects and converts them into a perimeter constraint\.Generated Sample by SRPO on MathVision MINI![Refer to caption](https://arxiv.org/html/2605.07274v1/figures/appendix/mathvision.png)Question:A circleKKis inscribed in a quarter circle with radius 6 as shown in the figure\. What is the radius of circleKK?Choices:\(A\)6−22\\frac\{6\-\\sqrt\{2\}\}\{2\}; \(B\)3​22\\frac\{3\\sqrt\{2\}\}\{2\}; \(C\) 2\.5; \(D\) 3; \(E\)6​\(2−1\)6\(\\sqrt\{2\}\-1\)\.Ground truth: E<perception\>The image showsa quarter circle with a radius of 6, anda circleKKis inscribed within this quarter circle\. We need to determine the radius of circleKK\.</perception\><reasoning\>To solve for the radius of circleKK, we can use the fact that the circleKKis inscribed in the quarter circle\. Let the radius of circleKKberr\. The center of circleKKis at a distancerrfrom the right and bottom edges of the quarter circle\. The distance from the center of the quarter circle to the center of circleKKis6−r6\-r\. This distance is also the hypotenuse of a right triangle with legs of lengthrreach\. Therefore, we can write the equation:\(6−r\)2=r2\+r2=2​r2\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}\(6\-r\)^\{2\}=r^\{2\}\+r^\{2\}=2r^\{2\}\}Expanding and solving the equation:36−12​r\+r2=2​r236\-12r\+r^\{2\}=2r^\{2\}36−12​r=r236\-12r=r^\{2\}r2\+12​r−36=0r^\{2\}\+12r\-36=0This is a quadratic equation in the formr2\+12​r−36=0r^\{2\}\+12r\-36=0\. We can solve it using the quadratic formular=−b±b2−4​a​c2​ar=\\frac\{\-b\\pm\\sqrt\{b^\{2\}\-4ac\}\}\{2a\}, wherea=1a=1,b=12b=12, andc=−36c=\-36:r\\displaystyle r=−12±122−4⋅1⋅\(−36\)2⋅1=−12±144\+1442\\displaystyle=\\frac\{\-12\\pm\\sqrt\{12^\{2\}\-4\\cdot 1\\cdot\(\-36\)\}\}\{2\\cdot 1\}=\\frac\{\-12\\pm\\sqrt\{144\+144\}\}\{2\}=−12±2882=−12±12​22=−6±6​2\.\\displaystyle=\\frac\{\-12\\pm\\sqrt\{288\}\}\{2\}=\\frac\{\-12\\pm 12\\sqrt\{2\}\}\{2\}=\-6\\pm 6\\sqrt\{2\}\.Sincerrmust be positive, we have:r=−6\+6​2=6​\(2−1\)\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}r=\-6\+6\\sqrt\{2\}=6\(\\sqrt\{2\}\-1\)\}</reasoning\>E\\boxed\{E\}Figure 10:MathVision case study\. SRPO identifies the tangency structure of the inscribed circle and derives the radius through a geometric constraint\.Generated Sample by SRPO on MathVerse MINI Vision\-Only![Refer to caption](https://arxiv.org/html/2605.07274v1/figures/appendix/mathverse.png)Question:According to the question shown in the image, please directly answer the question and provide the correct option letter, e\.g\., A, B, C, D\.Ground truth: C<perception\>The image shows triangleA​B​CABCwith∠​A=50∘\\angle A=50^\{\\circ\}\. The pointDDis theintersectionof the angle bisectorsB​DBDandC​DCDof△​A​B​C\\triangle ABC\. We are asked to determine the measure of∠​D\\angle D\.</perception\><reasoning\>To solve for∠​D\\angle D, we start by determining the measures of∠​A​B​C\\angle ABCand∠​A​C​B\\angle ACB\.Since the sum of the angles in a triangle is180∘180^\{\\circ\},∠​A​B​C\+∠​A​C​B\+∠​A=180∘\\angle ABC\+\\angle ACB\+\\angle A=180^\{\\circ\}∠​A​B​C\+∠​A​C​B\+50∘=180∘\\angle ABC\+\\angle ACB\+50^\{\\circ\}=180^\{\\circ\}∠​A​B​C\+∠​A​C​B=130∘\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}\\angle ABC\+\\angle ACB=130^\{\\circ\}\}SinceDDis the intersection of the angle bisectorsB​DBDandC​DCD,∠​A​B​D=12​∠​A​B​C\\angle ABD=\\frac\{1\}\{2\}\\angle ABCand∠​A​C​D=12​∠​A​C​B\\angle ACD=\\frac\{1\}\{2\}\\angle ACB\.The angle∠​D\\angle Din△​B​D​C\\triangle BDCcan be found using the fact that the sum of the angles in a triangle is180∘180^\{\\circ\}:∠​D\+∠​A​B​D\+∠​A​C​D=180∘\\angle D\+\\angle ABD\+\\angle ACD=180^\{\\circ\}∠​D\+12​∠​A​B​C\+12​∠​A​C​B=180∘\\angle D\+\\frac\{1\}\{2\}\\angle ABC\+\\frac\{1\}\{2\}\\angle ACB=180^\{\\circ\}∠​D\+12​\(130∘\)=180∘\\angle D\+\\frac\{1\}\{2\}\(130^\{\\circ\}\)=180^\{\\circ\}∠​D\+65∘=180∘\\angle D\+65^\{\\circ\}=180^\{\\circ\}∠​D=115∘\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}\\angle D=115^\{\\circ\}\}</reasoning\>C\\boxed\{C\}Figure 11:MathVerse case study\. SRPO extracts the angle\-bisector structure and uses it to derive the target angle\.Generated Sample by SRPO on NaturalBench![Refer to caption](https://arxiv.org/html/2605.07274v1/figures/appendix/naturalbench.jpg)Question:Is there a person walking through the illuminated underground cavern?Ground truth: Yes<perception\>The image showsan illuminated underground cavernwith large, sculpted rock formations\. There isa person visible in the background, walking through the cavern\. The person is small in comparison to the vastness of the cavern, emphasizing the scale of the environment\.</perception\><reasoning\>The image depicts an underground cavern with significant rock formations\. The presence ofa person in the backgroundprovides a scale reference\. The person iswalkingthrough the cavern, moving along a path that is illuminated\. The size of the person relative to the cavern confirms that they are indeed walking through the cavern\.</reasoning\>Yes\\boxed\{\\text\{Yes\}\}Figure 12:NaturalBench case study\. SRPO grounds the yes/no answer in a visually localized object and its relation to the scene\.Generated Sample by SRPO on MMVet![Refer to caption](https://arxiv.org/html/2605.07274v1/figures/appendix/mmvet.png)Question:Which room is bigger, the double garage or the living room?Ground truth: double garage<perception\>The image shows the floor plan with the following dimensions:Double garage:19×21\.2519\\times 21\.25;Living room:21\.5×18\.521\.5\\times 18\.5\.</perception\><reasoning\>To determine which room is bigger, we need to calculate the area of each room\.The area of the double garage is:19×21\.25=403\.75\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}19\\times 21\.25=403\.75\}square feet\.The area of the living room is:21\.5×18\.5=398\.25\{\\color\[rgb\]\{0\.921875,0\.4296875,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.921875,0\.4296875,0\}21\.5\\times 18\.5=398\.25\}square feet\.Comparing the two areas,403\.75 square feetfor the double garage is greater than398\.25 square feetfor the living room\. Therefore, the double garage is bigger\.</reasoning\>double garage\\boxed\{\\text\{double garage\}\}Figure 13:MMVet case study\. SRPO reads room dimensions from a floor plan and compares the corresponding areas to answer a spatial\-mathematical question\.Generated Sample by SRPO on LogicVista![Refer to caption](https://arxiv.org/html/2605.07274v1/figures/appendix/logicvista.png)Question:The Small Silver Watch displays the time as 16:00\. Select from A, B and C\.Choices:\(A\) True \(B\) False \(C\) Insufficient InformationGround truth: B<perception\>The image provides the following information:1\.The Large Silver Watch states the time as 15:50\.2\.The Small Silver Watch displays the same time as the Gold Watch\.3\.The Bronze Watch is small in size\.4\.The Gold Watch is ten minutes slower than the Large Silver Watch\.5\.The Bronze Watch is five minutes faster than the Small Silver Watch\.We are asked to determine if the Small Silver Watch displays the time as16:00\.</perception\><reasoning\>First, we determine the time on the Gold Watch:
\- The Large Silver Watch shows 15:50\.
\- The Gold Watch is ten minutes slower than the Large Silver Watch\.
\- Therefore, the Gold Watch shows15:50−1015\{:\}50\-10minutes = 15:40\.Next, we determine the time on the Small Silver Watch:\- The Small Silver Watch displays the same time as the Gold Watch\.\- Therefore, the Small Silver Watch shows15:40\.However, we are asked if the Small Silver Watch displays 16:00\.\- The Small Silver Watch shows15:40, not 16:00\.Therefore, the statement that the Small Silver Watch displays 16:00 is false\.</reasoning\>B\\boxed\{B\}Figure 14:LogicVista case study\. SRPO extracts textual relations from the visual prompt and uses them for deductive reasoning over watch times\.