Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation

# Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation
Source: [https://arxiv.org/html/2607.23125](https://arxiv.org/html/2607.23125)
Shuai Wang1Daoan Zhang2Zhe Tang3Hao Cheng4Jiaheng Wei1 1The Hong Kong University of Science and Technology \(Guangzhou\)2ByteDance Inc\. 3Zhejiang University of Technology4Hong Kong Baptist University

###### Abstract

Post\-training enables vision\-language models \(VLMs\) to understand human instructions and perform various downstream tasks\. Current post\-training methods usually rely on human\-annotated data, distillation from external models, reinforcement learning with human feedback, or verifiable answers\. This limits their ability to improve without external supervision\. To tackle this, we propose NOPD \(Noisy Student On\-Policy Self\-Distillation\), a simple yet effective self\-distillation approach that improves VLMs without any external models or ground\-truth answers\. Our key insight is that prediction discrepancies between clean and corrupted inputs naturally induce a self\-supervision signal\. In NOPD, the model learns from corrupted inputs while using its own predictions under clean inputs as token\-level supervision\. We show the effectiveness of NOPD on five visual reasoning tasks; it can match and even outperform reinforcement learning approaches or distillation from external models\. Notably, when trained with 2\.1K samples from Geometry3K, NOPD improves Qwen2\.5\-VL\-7B by 20 points on its validation set\. It also shows generalization on out\-of\-distribution test sets and achieves 7\.4 point gains on MathVista\. Furthermore, we demonstrate that NOPD is a general approach to enhance VLMs, achieving improvements across three models on 12 benchmarks\.

## 1Introduction

Vision\-language models \(VLMs\) have made remarkable progress in visual question answering \(VQA\), visual reasoning and chart understanding\(Baiet al\.,[2025b](https://arxiv.org/html/2607.23125#bib.bib8); Honget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib18); Liuet al\.,[2023](https://arxiv.org/html/2607.23125#bib.bib45)\)\. The advancement is primarily achieved in the post\-training stage, which may involve supervised fine\-tuning \(SFT\)\(Liuet al\.,[2023](https://arxiv.org/html/2607.23125#bib.bib45)\), reinforcement learning with human feedback\(Ouyanget al\.,[2022](https://arxiv.org/html/2607.23125#bib.bib28)\)or verifiable rewards \(RLVR\)\(Guoet al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib54); OLMoet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib11); Shaoet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib7)\), and knowledge distillation\(Agarwalet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib4); Guet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib13)\)\.

Although recent research has advanced the foundational capabilities of VLMs, typical post\-training approaches exhibit inherent limitations\. SFT usually needs human annotation or distillation from frontier models\. RLVR typically requires verifiable questions and only provides a sparse reward signal for the whole response\. Off\-policy and on\-policy knowledge distillation provide stronger and denser token\-level supervision from an external model \(usually calledteacher\)\.

Given these limitations and the strong capabilities of modern VLMs, we ask: can a VLM improve itself without any ground\-truth answers or supervision from external models, i\.e\., “self\-boosting”? Recently, some self\-distillation approaches have been proposed to improve models without external models by using environment feedback\(Hübotteret al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib26)\), or using answers as additional context\(Zhaoet al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib24)\)\. They are mainly designed for language models and still rely on external information\. We aim to improve VLMs without any external supervision or feedback\.

We show that supervision can be self\-induced by exploiting prediction discrepancies under asymmetric input distributions\. Specifically, when a model processes clean and corrupted inputs, the resulting difference in output distributions naturally provides a dense learning signal—without requiring labels, rewards, or external teachers\. We instantiate this principle and propose noisy student on\-policy self\-distillation \(NOPD\), a self\-improvement approach for VLMs\. In NOPD, a single model plays both teacher and student roles\. The teacher works with clean inputs, while the student is trained using corrupted ones\. The predictions under clean inputs serve as token\-level supervision for the noisy inputs\. Compared with previous methods, NOPD obviates the need for high\-quality responses and external strong models as shown in Figure[1](https://arxiv.org/html/2607.23125#S1.F1)\.

We conduct extensive experiments to validate the effectiveness of NOPD\. We first conduct preliminary experiments to compare NOPD with different post\-training methods on five visual reasoning benchmarks\. Extensive experiments demonstrate that NOPD consistently improves VLMs across multiple benchmarks, achieving performance comparable to or exceeding methods that rely on external supervision\. Notably, NOPD also leads to implicit reasoning compression, reducing output length while improving accuracy\. These results suggest that effective post\-training can be achieved without any external supervision\. Finally, we validate whether NOPD generalizes across different models and tasks beyond reasoning\.

![Refer to caption](https://arxiv.org/html/2607.23125v1/x1.png)

No GTOn\-policyNo externalToken\-levelLow samplingneededdatamodelslearning signalcostSFT✘✘✔✔✔RLVR✘✔✔✘✘KD✔✘✘✔✔OPD✔✔✘✔✔NOPD \(ours\)✔✔✔✔✔

Figure 1:Comparison of different post\-training approaches for VLMs\.Top: demonstration of different approaches\.qqdenotes text query andxxdenotes image\. We omit the subscripts of loss functions except GRPO for simplicity\. We highlight all sampling operations withred, e\.g\.,y∼πθ\(⋅\|q,x\)y\\sim\\pi\_\{\\theta\}\(\\cdot\|q,x\)\.πθ\\pi\_\{\\theta\}denotes the policy model in \(a\) and \(b\), and the student model for distillation in \(c\), \(d\), and \(e\)\.π∗\\pi^\{\*\}denotes the teacher for distillation methods\. In \(e\),𝒯\\mathcal\{T\}denotes a pre\-defined transformation\. \(e\) gives an overview of our method NOPD\. Different from typical on\-policy distillation, NOPD shares the same parameters between student and teacher\. The teacher supervision signal comes from the model under raw inputs\(q,x\)\(q,x\)\. For the student model, inputs are corrupted images𝒯​\(x\)\\mathcal\{T\}\(x\)and text queryqq\.Bottom: summary of comparison\. GT denotes answers for given input questions, and may include detailed chain\-of\-thought\(Weiet al\.,[2022b](https://arxiv.org/html/2607.23125#bib.bib57)\)traces\.
## 2Preliminaries

In this section, we briefly review existing post\-training paradigms\.

#### Notation\.

We useqqto denote a text query, andxxdenotes an image from the training set𝒮\\mathcal\{S\}\. A vision\-language model parameterized byθ\\thetais defined asπθ\\pi\_\{\\theta\}\. Given an input pair\(q,x\)\\left\(q,x\\right\), its likelihood underπθ\\pi\_\{\\theta\}isπθ​\(y∣q,x\)=∏t=1\|y\|πθ​\(yt∣q,x,y<t\)\\pi\_\{\\theta\}\(y\\mid q,x\)=\\prod\\limits\_\{t=1\}^\{\\lvert y\\rvert\}\\pi\_\{\\theta\}\\left\(y\_\{t\}\\mid q,x,y\_\{<t\}\\right\), where\|y\|\\lvert y\\rvertdenotes the number of tokens inyy\.y∗y^\{\*\}denotes the ground\-truth response for input pair\(q,x\)\\left\(q,x\\right\), which is typically obtained from human labeling or distillation from frontier models\.

#### Supervised fine\-tuning \(SFT\)\.

SFT is widely used in post\-training of modern foundation models that minimizes the following negative log\-likelihood loss

ℒSFT​\(θ\)=−∑t=1\|y∗\|log⁡πθ​\(yt∣q,x,y<t∗\)\.\\mathcal\{L\}\_\{\\text\{SFT\}\}\\left\(\\theta\\right\)=\-\\sum\_\{t=1\}^\{\\lvert y^\{\*\}\\rvert\}\\log\\pi\_\{\\theta\}\\left\(y\_\{t\}\\mid q,x,y^\{\*\}\_\{<t\}\\right\)\.\(1\)Note that ify∗y^\{\*\}is generated by other models rather than human annotation, it is also called sequence\-level knowledge distillation\(Kim and Rush,[2016](https://arxiv.org/html/2607.23125#bib.bib29)\)\. SFT provides token\-level supervision to guide the learning with the help ofy∗y^\{\*\}\.

#### Reinforcement learning with verifiable rewards \(RLVR\)\.

RLVR\(OLMoet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib11)\)is an on\-policy reinforcement learning approach used for specific domains where questions have verifiable answers, such as math and code\. The representative algorithm is GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib7)\), which computes advantages in a relative group instead of using additional critic models\. The GRPO objective is formulated as

ℒGRPO\(θ\)=𝔼\(q,x\)∼𝒮,\{yi\}i=1G∼πθold\(⋅∣q,x\)\[1G∑i=1G1\|yi\|∑t=1\|yi\|min\(wi,t\(θ\)A^i,t,\\displaystyle\\mathcal\{L\}\_\{\\textrm\{GRPO\}\}\\left\(\\theta\\right\)=\\mathbb\{E\}\_\{\(q,x\)\\sim\\mathcal\{S\},\\,\\\{y\_\{i\}\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\theta\_\{\\textrm\{old\}\}\}\\left\(\\cdot\\mid q,x\\right\)\}\\Bigg\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\frac\{1\}\{\\lvert y\_\{i\}\\rvert\}\\sum\_\{t=1\}^\{\\lvert y\_\{i\}\\rvert\}\\min\\Big\(w\_\{i,t\}\\left\(\\theta\\right\)\\widehat\{A\}\_\{i,t\},clip\(wi,t\(θ\),1−ε,1\+ε\)A^i,t\)\],\\displaystyle\\mathrm\{clip\}\\big\(w\_\{i,t\}\\left\(\\theta\\right\),1\-\\varepsilon,1\+\\varepsilon\\big\)\\widehat\{A\}\_\{i,t\}\\Big\)\\Bigg\],\(2\)whereGGis the number of generated responses for each input pair\(q,x\)\\left\(q,x\\right\)\(i\.e\., the group size\) andε\\varepsilonis a hyperparameter to control the clipping range for stabilizing training\. The importance ratiowi,t​\(θ\)w\_\{i,t\}\(\\theta\)and advantageA^i,t\\widehat\{A\}\_\{i,t\}of tokenyi,ty\_\{i,t\}are:

wi,t​\(θ\)=πθ​\(yi,t∣q,x,yi,<t\)πθold​\(yi,t∣q,x,yi,<t\),A^i,t=A^i=r​\(q,x,yi\)−mean​\(\{r​\(q,x,yi\)\}i=1G\)std​\(\{r​\(q,x,yi\)\}i=1G\),\\displaystyle w\_\{i,t\}\\left\(\\theta\\right\)=\\frac\{\\pi\_\{\\theta\}\\left\(y\_\{i,t\}\\mid q,x,y\_\{i,<t\}\\right\)\}\{\\pi\_\{\\theta\_\{\\text\{old\}\}\}\\left\(y\_\{i,t\}\\mid q,x,y\_\{i,<t\}\\right\)\},\\quad\\widehat\{A\}\_\{i,t\}=\\widehat\{A\}\_\{i\}=\\frac\{r\\left\(q,x,y\_\{i\}\\right\)\-\\mathrm\{mean\}\\left\(\\\{r\\left\(q,x,y\_\{i\}\\right\)\\\}\_\{i=1\}^\{G\}\\right\)\}\{\\mathrm\{std\}\\left\(\\\{r\\left\(q,x,y\_\{i\}\)\\right\\\}\_\{i=1\}^\{G\}\\right\)\},\(3\)respectively, where all the tokens inyiy\_\{i\}share the same advantage asA^i\\widehat\{A\}\_\{i\}\.r​\(q,x,yi\)r\\left\(q,x,y\_\{i\}\\right\)denotes the reward for given inputs\(q,x\)\(q,x\)and responsesyygenerated byπθ\\pi\_\{\\theta\}\. RLVR benefits from on\-policy learning while suffering from sparse supervision signals and high sampling cost to rollout during training\.

#### Off\-policy knowledge distillation\.

The goal of knowledge distillation \(KD\)\(Hintonet al\.,[2015](https://arxiv.org/html/2607.23125#bib.bib12)\)is to transfer knowledge from the strong model \(usually namedteacher\) to the weak model \(usually namedstudent\)\. Standard KD can be formulated as:

ℒKD\(θ\)=𝔼\(q,x\)∼𝒮,y∗\[𝒟KL\(π∗\(y∗∣q,x\)∥πθ\(y∗∣q,x\)\)\],\\mathcal\{L\}\_\{\\textrm\{KD\}\}\\left\(\\theta\\right\)=\\mathbb\{E\}\_\{\\left\(q,x\\right\)\\sim\\mathcal\{S\},\\,y^\{\*\}\}\\left\[\\mathcal\{D\}\_\{\\textrm\{KL\}\}\\left\(\\pi^\{\*\}\\left\(y^\{\*\}\\mid q,x\\right\)\\\|\\pi\_\{\\theta\}\\left\(y^\{\*\}\\mid q,x\\right\)\\right\)\\right\],\(4\)whereπ∗\\pi^\{\*\}denotes the teacher model and𝒟KL​\(P∥Q\)\\mathcal\{D\}\_\{\\textrm\{KL\}\}\(P\\\|Q\)denotes the Kullback–Leibler \(KL\) divergence\.

#### On\-policy distillation \(OPD\)\.

OPD\(Agarwalet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib4); Guet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib13)\)trains the student model with self\-generated sequences and learns feedback from the teacher on these sequences\. Specifically, OPD minimizes an objective of the form:

ℒOPD\(θ\)=𝔼\(q,x\)∼𝒮,y∼πθ\(⋅∣q,x\)\[𝒟KL\(πθ\(y∣q,x\)∥π∗\(y∣q,x\)\)\]\.\\mathcal\{L\}\_\{\\text\{OPD\}\}\\left\(\\theta\\right\)=\\mathbb\{E\}\_\{\\left\(q,x\\right\)\\sim\\mathcal\{S\},\\,y\\sim\\pi\_\{\\theta\}\\left\(\\cdot\\mid q,x\\right\)\}\\left\[\\mathcal\{D\}\_\{\\textrm\{KL\}\}\\left\(\\pi\_\{\\theta\}\\left\(y\\mid q,x\\right\)\\\|\\pi^\{\*\}\\left\(y\\mid q,x\\right\)\\right\)\\right\]\.\(5\)Off\-/on\-policy distillation provides dense token\-level supervision without requiring human annotations, but typically relies on a stronger teacher model\. When the teacher is derived from the same model as the student \(e\.g\., sharing parameters\) rather than an external model, we refer to this setting as on\-policy self\-distillation\(Zhaoet al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib24); Hübotteret al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib26)\)\. In this case, privileged information can be transferred through differences in inputs\(Vapnik and Izmailov,[2015](https://arxiv.org/html/2607.23125#bib.bib55)\)\.

Algorithm 1Training Procedure of NOPD1:Training set

𝒮=\{\(qi,xi\)\}i=1\|S\|\\mathcal\{S\}=\\\{\(q\_\{i\},x\_\{i\}\)\\\}\_\{i=1\}^\{\|S\|\}; vision\-language model

πθ\\pi\_\{\\theta\}; batch size

BB; image transformation

𝒯\\mathcal\{T\}; max iterations

TmaxT\_\{\\max\}
2:Trained model

πθ\\pi\_\{\\theta\}
3:Initialize

t←0t\\leftarrow 0
4:while

t<Tmaxt<T\_\{\\max\}and not convergeddo

5:Sample a mini\-batch

\{\(qi,xi\)\}i=1B\\\{\(q\_\{i\},x\_\{i\}\)\\\}\_\{i=1\}^\{B\}from

𝒮\\mathcal\{S\}
6:Sample on\-policy response

yi∼πθ\(⋅∣qi,𝒯\(xi\)\),i=1,…,By\_\{i\}\\sim\\pi\_\{\\theta\}\\left\(\\cdot\\mid q\_\{i\},\\mathcal\{T\}\\left\(x\_\{i\}\\right\)\\right\),\\quad i=1,\\dots,B
7:Compute the loss as Eq\. \([6](https://arxiv.org/html/2607.23125#S3.E6)\)

ℒ\(θ\)=𝔼\(q,x\)∼𝒮,y∼πθ\(⋅\|q,𝒯\(x\)\)\[𝒟KL\(πθ\(y∣q,𝒯\(x\)\)∥πθ\(y∣q,x\)\)\]\\mathcal\{L\}\\left\(\\theta\\right\)=\\mathbb\{E\}\_\{\\left\(q,x\\right\)\\sim\\mathcal\{S\},\\,y\\sim\\pi\_\{\\theta\}\\left\(\\cdot\|q,\\mathcal\{T\}\(x\)\\right\)\}\\left\[\\mathcal\{D\}\_\{\\textrm\{KL\}\}\\left\(\\pi\_\{\\theta\}\\left\(y\\mid q,\\mathcal\{T\}\\left\(x\\right\)\\right\)\\\|\\pi\_\{\\theta\}\\left\(y\\mid q,x\\right\)\\right\)\\right\]
8:Update model parameters

θ\\thetausing

∇θℒ\\nabla\_\{\\theta\}\\mathcal\{L\}\(c\.f\. Eq\. \([8](https://arxiv.org/html/2607.23125#S3.E8)\)\)

9:

t←t\+1t\\leftarrow t\+1
10:endwhile

11:return

πθ\\pi\_\{\\theta\}

![Refer to caption](https://arxiv.org/html/2607.23125v1/figure/image_distort/image_raw.png)

![Refer to caption](https://arxiv.org/html/2607.23125v1/figure/image_distort/image_noise.jpg)

![Refer to caption](https://arxiv.org/html/2607.23125v1/figure/image_distort/image_gamma.jpg)

![Refer to caption](https://arxiv.org/html/2607.23125v1/figure/image_distort/image_downsample.jpg)

Figure 2:Examples of different distorted images\. The types of image distortions \(from left to right\): original images, add noise, gamma transformation, downsample\. We resize the downsampled image for clarity\. The image is from the training set of Geometry3K\.

## 3Method

We propose NOPD, short fornoisy studenton\-policy self\-distillation, a self\-training approach that enables VLMs to improve themselves without any ground\-truth answers or external models\. Algorithm[1](https://arxiv.org/html/2607.23125#alg1)provides an overview of NOPD\. The inputs to the student in the algorithm are corrupted images𝒯​\(x\)\\mathcal\{T\}\(x\)and the corresponding text queryqq\. We first generate an on\-policy responseyyunder corrupted imagesy∼πθ​\(𝒯​\(x\),q\)y\\sim\\pi\_\{\\theta\}\\left\(\\mathcal\{T\}\\left\(x\\right\),q\\right\), where𝒯\\mathcal\{T\}is a pre\-defined transformation for images\. After that, we use the outputs of the same modelπθ\\pi\_\{\\theta\}under clean inputs\(q,x\)\(q,x\)as the “teacher” output to provide dense supervision signals\. Finally, our objective is to minimize the divergence between the student model and the teacher model as follows

ℒNOPD​\(θ\)=𝔼\(q,x\)∼𝒮,y∼πθ\(⋅∣q,𝒯\(x\)\)​\[𝒟KL​\(πθ​\(y∣q,𝒯​\(x\)\)∥πθ​\(y∣q,x\)⏟sg\)\],\\mathcal\{L\}\_\{\\textrm\{NOPD\}\}\\left\(\\theta\\right\)=\\mathbb\{E\}\_\{\\left\(q,x\\right\)\\sim\\mathcal\{S\},\\,y\\sim\\pi\_\{\\theta\}\\left\(\\cdot\\mid q,\\mathcal\{T\}\\left\(x\\right\)\\right\)\}\\left\[\\mathcal\{D\}\_\{\\textrm\{KL\}\}\\left\(\\pi\_\{\\theta\}\\left\(y\\mid q,\\mathcal\{T\}\\left\(x\\right\)\\right\)\\\|\\underbrace\{\\pi\_\{\\theta\}\\left\(y\\mid q,x\\right\)\}\_\{\{\\color\[rgb\]\{0,0,1\}\\texttt\{sg\}\}\}\\right\)\\right\],\(6\)wheresgdenotes stop gradient\. Note that the KL divergence is not symmetric, which means that𝒟KL​\(P∥Q\)≠𝒟KL​\(Q∥P\)\\mathcal\{D\}\_\{\\textrm\{KL\}\}\\left\(P\\\|Q\\right\)\\neq\\mathcal\{D\}\_\{\\textrm\{KL\}\}\\left\(Q\\\|P\\right\), whereP=πθ​\(y∣q,𝒯​\(x\)\)P=\\pi\_\{\\theta\}\\left\(y\\mid q,\\mathcal\{T\}\\left\(x\\right\)\\right\)andQ=πθ​\(y∣q,x\)Q=\\pi\_\{\\theta\}\\left\(y\\mid q,x\\right\)\. The KL divergence in Eq\. \([6](https://arxiv.org/html/2607.23125#S3.E6)\) is called reverse KL\. We consider two other commonly used variants: forward KL𝒟KL​\(Q∥P\)\\mathcal\{D\}\_\{\\textrm\{KL\}\}\\left\(Q\\\|P\\right\)and generalized Jensen–Shannon divergence \(JSD\), which is defined as

𝒟JSD​\(P∥Q\)​\(β\)=β​DKL​\(P∥β​P\+\(1−β\)​Q\)\+\(1−β\)​DKL​\(Q\|β​P\+\(1−β\)​Q\),\\mathcal\{D\}\_\{\\textrm\{JSD\}\}\\left\(P\\\|Q\\right\)\\left\(\\beta\\right\)=\\beta D\_\{\\mathrm\{KL\}\}\\left\(P\\,\\\|\\,\\beta P\+\\left\(1\-\\beta\\right\)Q\\right\)\+\\left\(1\-\\beta\\right\)D\_\{\\mathrm\{KL\}\}\\left\(Q\\,\|\\,\\beta P\+\\left\(1\-\\beta\\right\)Q\\right\),\(7\)whereβ∈\(0,1\)\\beta\\in\(0,1\)is the weighting coefficient\. The gradients of JSD are similar to forward KL whenβ→0\\beta\\rightarrow 0and reverse KL whenβ→1\\beta\\rightarrow 1, respectively\. We verify the effectiveness of different divergences in Sec\.[4\.2](https://arxiv.org/html/2607.23125#S4.SS2)\. We stop the gradient from the “teacher” signal, i\.e\.,πθ​\(y∣q,x\)\\pi\_\{\\theta\}\\left\(y\\mid q,x\\right\), to prevent model collapse111If we do not stop the gradient ofπθ​\(y∣q,x\)\\pi\_\{\\theta\}\\left\(y\\mid q,x\\right\),πθ\\pi\_\{\\theta\}will generate the same output whatever the input to make the loss zero\.and do not backpropagate through the student’s sampling distributionπθ\(⋅∣q,𝒯\(x\)\)\\pi\_\{\\theta\}\(\\cdot\\mid q,\\mathcal\{T\}\(x\)\), which makes the training process more stable and efficient following\(Agarwalet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib4)\)\.

Distinguished from other post\-training approaches, our method NOPD is a self\-training approach to improve VLMs without any external supervision \(models, verifiable answers, and reasoning traces\)\. In addition, NOPD also has low sample cost, is more data efficient, and could reduce output length with performance increase; see Sec\.[4\.2](https://arxiv.org/html/2607.23125#S4.SS2)for verification\. We compare our method with other post\-training approaches in Figure[1](https://arxiv.org/html/2607.23125#S1.F1)and provide more discussion in Sec\.[5](https://arxiv.org/html/2607.23125#S5)\.

Our key contribution lies in introducing asymmetric input distributions by adding permutation𝒯\\mathcal\{T\}for the student vision inputs to enable self\-induced supervision, which makes our method different from other self\-distillation methods\(Zhaoet al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib24); Hübotteret al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib26)\)\. The motivation behind this is that we aim to improve student models by exposing them to difficult environments in terms of noise during learning\. Learning from asymmetric input distributions to enable self\-induced supervision also improves generalization to out\-of\-distribution domains\.

#### Gradient Analysis\.

We can derive the NOPD gradient as follows \(see Sec\.[C](https://arxiv.org/html/2607.23125#A3)for details\):

∇θℒNOPD​\(θ\)=\\displaystyle\\small\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\text\{NOPD\}\}\(\\theta\)=𝔼\(q,x\)∼𝒮,y∼πθ\(⋅∣q,𝒯\(x\)\)\[∑t=1\|y\|𝔼y^t∼πθ\(⋅∣𝒯\(x\),q,y<t\)\\displaystyle\\;\\mathbb\{E\}\_\{\(q,x\)\\sim\\mathcal\{S\},\\;y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid q,\\mathcal\{T\}\(x\)\)\}\\Bigg\[\\sum\_\{t=1\}^\{\|y\\rvert\}\\mathbb\{E\}\_\{\\hat\{y\}\_\{t\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid\\mathcal\{T\}\(x\),q,y\_\{<t\}\)\}\[logπθ​\(y^t∣q,𝒯​\(x\)\)πθ​\(y^t∣q,x\)⋅∇θlogπθ\(y^t∣q,𝒯\(x\),y<t\)\]\],\\displaystyle\\;\\quad\\Bigg\[\\log\\frac\{\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,x\)\}\\cdot\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,\\mathcal\{T\}\\left\(x\\right\),y\_\{<t\}\)\\Bigg\]\\Bigg\],\(8\)wherey^t∈Δ\|V\|−1\\hat\{y\}\_\{t\}\\in\\Delta^\{\\lvert V\\rvert\-1\}is\|V\|\\lvert V\\rvert\-simplex and denotes the model’s predicted probability distribution over the vocabulary at decoding steptt, and\|V\|\\lvert V\\rvertis the vocabulary size\.

The NOPD gradient can be interpreted as a contrastive policy gradient\. The inner term,∇θlog⁡πθ​\(y^t∣q,𝒯​\(x\),y<t\)\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,\\mathcal\{T\}\(x\),y\_\{<t\}\), is the standard policy gradient that propagates the sampled tokeny^t\\hat\{y\}\_\{t\}back to the model parameters\. The log\-ratio termlog⁡πθ​\(y^t∣q,𝒯​\(x\)\)πθ​\(y^t∣q,x\)\\log\\dfrac\{\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,x\)\}acts as a contrastive reward, amplifying updates for tokens whose predicted distributions differ significantly between the transformed input\(q,𝒯​\(x\)\)\\left\(q,\\mathcal\{T\}\\left\(x\\right\)\\right\)and the original input\(q,x\)\\left\(q,x\\right\)\.

#### The choice of image permutation\.

For image transformation𝒯\\mathcal\{T\}, we consider three ways:

∙\\bulletAdditive Gaussian noise\. An augmented imagex′x^\{\\prime\}is generated by adding Gaussian noise to the input image:x′=clip​\(x\+ϵ,0,1\)x^\{\\prime\}=\\text\{clip\}\(x\+\\epsilon,0,1\), whereϵ∼𝒩​\(0,σ2\)\\epsilon\\sim\\mathcal\{N\}\(0,\\sigma^\{2\}\)\. The noise standard deviationσ\\sigmais randomly sampled from\{0\.08,0\.12,0\.18,0\.26,0\.38\}\\\{0\.08,0\.12,0\.18,0\.26,0\.38\\\}, following\(Hendrycks and Dietterich,[2019](https://arxiv.org/html/2607.23125#bib.bib14)\)\.

∙\\bulletLower resolution\. We randomly downsample the input imagexxby a factor of22or44, resulting in a lower\-resolution image\. This process reduces image fidelity and decreases the number of vision tokens, thereby improving training efficiency\.

∙\\bulletGamma transformation\. We apply gamma transformation to adjust the brightness and contrast of given images\. To be specific, the augmented image isx′=xγx^\{\\prime\}=x^\{\\gamma\}and we sampleγ∼U​\[0\.5,2\]\\gamma\\sim U\[0\.5,2\]\.

We provide examples of distorted images in Figure[2](https://arxiv.org/html/2607.23125#S2.F2)\. Note that we choose hyperparameters for image transformation𝒯\\mathcal\{T\}directly widely used in computer vision tasks and do not tune hyperparameters due to the limitation of compute resources\. During training, we randomly select one transformation and apply it to get corrupted imagexx\.

Table 1:Comparison of different approaches\.We highlight thebestresult andperformance degradationcompared with the base model\.Geometry3KMathVistaMathVisionMMMUMMMU\-ProtestvaltestminitestvaloverallQwen3\.5\-2B73\.265\.376\.144\.364\.250\.3SFT73\.266\.074\.842\.863\.849\.4KD74\.065\.775\.443\.364\.950\.8RLVR78\.468\.279\.348\.167\.053\.5OPD77\.267\.878\.746\.668\.155\.3NOPD \(ours\)78\.069\.180\.447\.267\.654\.7Qwen2\.5\-VL\-7B35\.433\.768\.425\.158\.638\.3SFT38\.837\.070\.226\.259\.138\.3KD39\.137\.370\.225\.859\.138\.1RLVR52\.151\.072\.328\.162\.841\.8OPD54\.852\.073\.128\.163\.642\.0NOPD \(ours\)55\.253\.775\.830\.963\.243\.3

## 4Experiments

We conduct extensive experiments to answer the following questions:

∙\\bulletHow does NOPD compare to other post\-training approaches? \(Sec\.[4\.1](https://arxiv.org/html/2607.23125#S4.SS1)\)

∙\\bulletHow do the design choices affect performance? \(Sec\.[4\.2](https://arxiv.org/html/2607.23125#S4.SS2)\)

∙\\bulletIs NOPD a general approach to improve VLMs itself in other domains? \(Sec\.[4\.3](https://arxiv.org/html/2607.23125#S4.SS3)\)

### 4\.1A Preliminary Experiment on Reasoning Tasks

In this section, we conduct a preliminary experiment to verify the effectiveness of our method and compare it with other post\-training approaches including SFT, off\-policy KD, RLVR and OPD\.

Table 2:Ablation studiesof key design choices\. We train Qwen3\.5\-2B with the training set of Geometry3K and report accuracy on MathVista\. Default settings aremarked\.LossAcc\.Forward KL78\.1Reverse KL80\.4JSD \(β=0\.5\\beta=0\.5\)79\.8\(a\)Loss type
Acc\.Full vocab80\.4Sampled token79\.1\(b\)Computation strategy
Acc\.Fix76\.9EMA79\.1Sync80\.4\(c\)Teacher update
Gen lengthAcc\.102479\.4204880\.4409680\.5\(d\)Generation Length

#### Models\.

We use Qwen2\.5\-VL\-7B\(\-Instruct\)\(Baiet al\.,[2025b](https://arxiv.org/html/2607.23125#bib.bib8)\)and Qwen3\.5\-2B222[https://huggingface\.co/Qwen/Qwen3\.5\-2B](https://huggingface.co/Qwen/Qwen3.5-2B)as the base models\. Note that Qwen3\.5\-2B is a hybrid model that supports thinking and no\-thinking modes\. To control for variables, we conduct experiments exclusively in the thinking mode\.

#### Training details & baselines\.

We use the Geometry3K training set\(Luet al\.,[2021](https://arxiv.org/html/2607.23125#bib.bib2)\), which contains 2\.1K image\-text pairs\. For the SFT baseline, we use responses distilled from Qwen3\.5\-27B/Qwen2\.5\-VL\-72B\-Instruct for Qwen3\.5\-2B/Qwen2\.5\-VL\-7B\. For a fair comparison, we keep the same teacher model for off\-policy KD and on\-policy distillation experiments\. For RLVR, we use GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib7)\)as the standard implementation\. We also experiment with other variants, such as DAPO\(Yuet al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib10)\)and GSPO\(Zhenget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib9)\), but observe no significant improvement\. Training hyperparameters are included in Table[7](https://arxiv.org/html/2607.23125#A3.T7)\.

#### Evaluation\.

We first evaluate on the test and val split of Geometry3K\(Luet al\.,[2021](https://arxiv.org/html/2607.23125#bib.bib2)\)as the in\-distribution test set\. In addition, we add MathVista\(Luet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib1)\), MathVision\(Wanget al\.,[2024a](https://arxiv.org/html/2607.23125#bib.bib3)\), MMMU\(Yueet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib5)\)and MMMU\-Pro\(Yueet al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib6)\)as out\-of\-distribution test sets\. For Qwen2\.5\-VL\-7B\(Baiet al\.,[2025b](https://arxiv.org/html/2607.23125#bib.bib8)\), we use greedy decoding with CoT\(Weiet al\.,[2022b](https://arxiv.org/html/2607.23125#bib.bib57)\)and report pass@1 accuracy\. For Qwen3\.5\-2B, we follow the Qwen official guide to use random sampling and report avg@16 for stable performance\. More details are included in Sec\.[B\.1](https://arxiv.org/html/2607.23125#A2.SS1)\.

#### Results\.

Table[1](https://arxiv.org/html/2607.23125#S3.T1)reports results on five benchmarks\. First, NOPD consistently improves the base model acrossallbenchmarks\. Specifically, NOPD improves 4\.4 points on MMMU\-Pro and 4\.8 points on the test set of Geometry3K with Qwen3\.5\-2B\. In addition, NOPD achieves a 20\-point improvement on the val split of Geometry3K and 7\.4 points on MathVista with Qwen2\.5\-VL\-7B\. Trained only with the Geometry3K training set, NOPD also improves performance on out\-of\-distribution test sets, which demonstrates strong out\-of\-domain generalization\.

Furthermore, NOPD can match or even outperform RLVR and OPD on multiple benchmarks\. We also find that performance degradation often occurs with SFT and off\-policy KD, which indicates that they struggle to improve open\-weight models even with the help of teacher models or external labels\. RLVR and OPD consistently improve all benchmarks for both models, but they require external ground\-truth answers and models\. The results show that NOPD is a self\-training method that improves VLMs without any external signals \(ground\-truth answers or external models\) and the effectiveness of NOPD matches RLVR and OPD\.

### 4\.2Analysis

#### Design choices\.

We conduct extensive experiments to study key design choices in our method and present the results in Table[2\(d\)](https://arxiv.org/html/2607.23125#S4.T2.st4)\. First, a key design choice is the loss function\. We compare forward KL, reverse KL, and JSD in Table[2\(a\)](https://arxiv.org/html/2607.23125#S4.T2.st1)\. Reverse KL achieves the best result, while other objectives improve the baselines consistently\.

When computing the divergence between the student and the teacher model, we compute all tokens over the vocabulary\. Another strategy is to compute probabilities only at the tokens sampled by the student\. Table[2\(b\)](https://arxiv.org/html/2607.23125#S4.T2.st2)shows that computing probabilities on all tokens performs better, which demonstrates the benefit of richer supervision and teacher knowledge\. Other reasons also include that only transferring knowledge of sampled tokens inevitably undermines the mathematical equivalence when we compute KL divergence, and preserving full vocabulary logits yields stable gradient estimates\. However, directly transferring logits for a large vocabulary size \(usually \>100K\) becomes expensive\. We only transfer last\-layer teacher hidden states and recompute logits on the fly with the teacher’s head when computing the loss function\. Because the dimension of hidden states \(e\.g\., 6144 for Qwen3\.5\-2B\) is much smaller than the dimension of the full vocabulary \(e\.g\., 248320 for Qwen3\.5\-2B\), the corresponding communication overhead becomes affordable and practical, and only introduces negligible recomputation\. The infrastructure design is from KDFlow\(Zhanget al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib49)\)\.

In our experiments, we update the teacher fully in sync with the student model, i\.e\.,π∗=πθ\\pi^\{\*\}=\\pi\_\{\\theta\}at every training step\. We also consider two other variants: \(a\) fixing the teacher model as the initial weightsπ∗=πθ\|t=0\\pi^\{\*\}=\\pi\_\{\\theta\}\|\_\{t=0\}wherettdenotes training steps, and \(b\) EMA update:π∗←m​π∗\+\(1−m\)​πθ\\pi^\{\*\}\\leftarrow m\\pi^\{\*\}\+\\left\(1\-m\\right\)\\pi\_\{\\theta\}wherem=0\.9m=0\.9controls the update rate\. We find that fully synchronizing the weights between the teacher and the student achieves the best performance in Table[2\(c\)](https://arxiv.org/html/2607.23125#S4.T2.st3)\. We hypothesize that the capability of the student model improves during training and can provide better supervision signals\.

![Refer to caption](https://arxiv.org/html/2607.23125v1/x2.png)Figure 3:The number of output tokens and accuracy on three benchmarks\.To reduce computational overhead, we constrain the generation length to 2048 for the student\. We explore different generation lengths in Table[2\(d\)](https://arxiv.org/html/2607.23125#S4.T2.st4)\. We find that increasing the generation length to 4096 brings only limited gains compared with other baselines\. To balance computation and performance, we set the maximum generation length to 2048\. The intuition behind this is that early tokens are more critical for learning in the autoregressive decoding process\. Errors in early tokens can accumulate and lead to incorrect outputs\. Therefore, providing supervision signals only for early tokens is sufficient for the student’s learning\.

#### Implicit reasoning compression\.

We compare the number of output tokens and accuracy on three datasets using Qwen3\.5\-2B in Figure[3](https://arxiv.org/html/2607.23125#S4.F3)\. Besides the improvement in accuracy, we find that our method simultaneously compresses the thinking process across three benchmarks, which improves the efficiency of reasoning\. To be specific, on MathVista, our method reduces tokens by57\.9%and improves accuracy by 4\.3%\. Although we do not design a specific technique for compressing tokens, NOPD achieves implicit reasoning compression, improving efficiency at test time\.

#### Effectiveness across training datasets\.

To evaluate the effectiveness of our method across different training datasets, we conduct experiments on MMK12\(Menget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib15)\), which contains 15\.6K samples\. All other settings remain unchanged, except that we increase the training steps to 200 due to the larger dataset size compared to Geometry3K\. As shown in Figure[4](https://arxiv.org/html/2607.23125#S4.F4), NOPD consistently improves performance when trained on MMK12, demonstrating strong scalability with increased data and training compute\. For instance, accuracy on MathVision improves by 3\.2 points, from 47\.2% to 50\.4%\.

![Refer to caption](https://arxiv.org/html/2607.23125v1/x3.png)Figure 4:Accuracy on three benchmarks during training with two different training sets\. We train Qwen3\.5\-2B with MMK12 and Geometry3K, respectively\.
#### Corruption type of images\.

We conduct experiments with different image corruption types to verify their importance\. As shown in Table[3](https://arxiv.org/html/2607.23125#S4.T3), all corruptions improve the base model, with adding noise yielding the largest gain \(73\.2→77\.173\.2\\rightarrow 77\.1\) and combining all types performing best\. This highlights the importance of image corruption for student inputs\.

Table 3:Ablation on corruption\. N, G, L denotes noise, gamma transformation, lower resolution, respectively\. We report results on Geometry3K test set with Qwen3\.5\-2B\.NGLGeometry3K✔✔✔78\.0✔77\.1✔74\.4✔76\.873\.6w/o training73\.2

### 4\.3Could NOPD be a General Approach to Improve VLMs?

#### Models\.

To validate whether our method can be used to improve VLMs on different tasks and to assess its scalability with data, we conduct experiments using larger\-scale datasets and evaluate on multiple tasks\. We choose Qwen2\.5\-VL\-7B\(Baiet al\.,[2025b](https://arxiv.org/html/2607.23125#bib.bib8)\)as the instruction model and Qwen3\-VL\-Thinking\(Baiet al\.,[2025a](https://arxiv.org/html/2607.23125#bib.bib47)\)as the thinking model\. To further examine the effectiveness of our method across different models, we include GLM4\.6\-V\-Flash with 9B parameters\(Honget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib18)\)\.

#### Training & data\.

We collect data from multiple sources, including Geometry3K\(Luet al\.,[2021](https://arxiv.org/html/2607.23125#bib.bib2)\), GeomVerse\(Kazemiet al\.,[2023](https://arxiv.org/html/2607.23125#bib.bib30)\), Geo170K\(Gaoet al\.,[2023](https://arxiv.org/html/2607.23125#bib.bib31)\), MMK12\(Menget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib15)\), GeoQA\+\(Cao and Xiao,[2022](https://arxiv.org/html/2607.23125#bib.bib39)\), CoSyn\(Yanget al\.,[2025b](https://arxiv.org/html/2607.23125#bib.bib40)\), VQAv2\(Goyalet al\.,[2017](https://arxiv.org/html/2607.23125#bib.bib41)\), ArxivQA\(Liet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib42)\), ChartQA\(Masryet al\.,[2022](https://arxiv.org/html/2607.23125#bib.bib43)\)and GQA\(Hudson and Manning,[2019](https://arxiv.org/html/2607.23125#bib.bib44)\)\.

We remove similar images or questions to improve the diversity of the training set\. In addition, to ensure vision relevance so that all questions must be answered with images, we prompt Qwen3\-VL\-8B\-Instruct and remove image\-irrelevant questions\. To balance the training data distribution, we classify all questions into three domains: general, reasoning, and chart, and uniformly sample different domains during training\. We get 19K/56K/54K samples for general, chart, and reasoning domains, respectively\. We increase training steps by 1000 and keep other hyperparameters the same\.

#### Evaluation\.

To provide a comprehensive evaluation across diverse tasks, we assess all models on 12 benchmarks covering three categories, including general visual question answering \(VQA\), reasoning, and chart understanding\. Specifically, for general VQA, we adopt MMStar\(Chenet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib33)\), MMBench\-EN\(Liuet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib32)\), BLINK\(Fuet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib34)\), CV\-Bench\(Tonget al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib35)\), and Realworld\-QA\.333[https://huggingface\.co/datasets/xai\-org/RealworldQA](https://huggingface.co/datasets/xai-org/RealworldQA)For reasoning tasks, we utilize MathVista\(Luet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib1)\), MathVision\(Wanget al\.,[2024a](https://arxiv.org/html/2607.23125#bib.bib3)\), MMMU\(Yueet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib5)\), and MMMU\-Pro\(Yueet al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib6)\)\. For chart understanding, we evaluate models on ChartQA\-Pro\(Masryet al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib36)\), ChartMuseum\(Tanget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib37)\), and CharXiv \(reasoning\)\(Wanget al\.,[2024b](https://arxiv.org/html/2607.23125#bib.bib38)\)\.

For different models, we adopt decoding parameters recommended by the official guide \(see Table[6](https://arxiv.org/html/2607.23125#A3.T6)\) and follow the evaluation protocol inlmms\-eval\(Zhanget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib53)\)\.

#### Results\.

We present results in Table[4](https://arxiv.org/html/2607.23125#S4.T4)\. First, it is noticed that NOPD achieves consistent gains across three models\. Specifically, we improve Qwen2\.5\-VL\-7B onallbenchmarks\. For Qwen3\-VL\-8B\-Thinking and GLM\-4\.6V\-Flash, we get improvements on 10 out of 12 benchmarks\. This shows that NOPD could be used as a general self\-improvement approach for VLMs\. Furthermore, NOPD yields substantial improvements across all three domains, which shows the effectiveness of NOPD is not limited to reasoning tasks but extends to other domains such as VQA and chart understanding\.

Table 4:Empirical results on multimodal benchmarks with different models\.\+xand\-xindicate improvement/decline compared with initial models\.Qwen2\.5NOPDQwen3\-VLNOPDGLM\-4\.6VNOPD\-VL\-7B\-7B\-8B\-Thinking\-8B\-Flash\-9BMMStar62\.569\.9\(\+7\.4\)73\.675\.2\(\+1\.6\)74\.376\.8\(\+2\.5\)MMBench\-EN83\.483\.6\(\+0\.2\)90\.690\.4\(\-0\.2\)91\.090\.8\(\-0\.2\)BLINK56\.462\.3\(\+5\.9\)64\.764\.1\(\-0\.6\)65\.566\.4\(\+0\.9\)CV\-Bench80\.084\.3\(\+4\.3\)84\.885\.6\(\+0\.8\)86\.086\.3\(\+0\.3\)General VQARealworld\-QA68\.573\.5\(\+5\.0\)71\.973\.4\(\+1\.5\)76\.976\.9\(\+0\.0\)MathVista68\.678\.4\(\+9\.8\)78\.581\.4\(\+2\.9\)82\.983\.3\(\+0\.4\)MathVision22\.434\.4\(\+12\.0\)59\.662\.1\(\+2\.5\)54\.157\.5\(\+3\.4\)MMMU51\.358\.1\(\+6\.8\)73\.575\.7\(\+2\.2\)71\.274\.2\(\+3\.0\)ReasoningMMMU\-Pro34\.639\.6\(\+5\.0\)60\.962\.6\(\+1\.7\)59\.963\.7\(\+3\.8\)ChartQA\-Pro43\.344\.3\(\+1\.0\)58\.462\.9\(\+4\.5\)62\.662\.4\(\-0\.2\)ChartMuseum26\.831\.4\(\+4\.6\)46\.751\.6\(\+4\.9\)49\.851\.3\(\+1\.5\)ChartCharXiv42\.547\.5\(\+5\.0\)53\.058\.4\(\+5\.4\)59\.663\.5\(\+3\.9\)

## 5Related Work

#### Self\-improvement for foundation models\.

Previous work has advanced the self\-improvement of foundation models by fine\-tuning on self\-generated data\(Zelikmanet al\.,[2022](https://arxiv.org/html/2607.23125#bib.bib22)\), automatically constructing preference data\(Denget al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib19); Donget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib20)\), or using LLMs as judges to select responses or label preference data\(Yuanet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib16); Zhanget al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib21); Gulcehreet al\.,[2023](https://arxiv.org/html/2607.23125#bib.bib17)\)\. Recently, some self\-distillation approaches\(Heet al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib25); Zhaoet al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib24)\), where a single model plays both teacher and student roles in the on\-policy distillation process, have been proposed to improve itself without additional teachers\. However, the above methods are designed for language models and require additional feedback from environments\(Hübotteret al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib26)\)or ground\-truth labels\(Zhaoet al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib24)\)\. Our NOPD also belongs to self\-distillation approaches but does not need additional feedback or answers to improve VLMs\.

#### Vision\-language models\.

VLMs that combine visual encoders such as CLIP\(Radfordet al\.,[2021](https://arxiv.org/html/2607.23125#bib.bib52)\)with large language models have shown strong ability to understand and reason with both visual and text information inputs\(Liuet al\.,[2023](https://arxiv.org/html/2607.23125#bib.bib45); Baiet al\.,[2025b](https://arxiv.org/html/2607.23125#bib.bib8)\)\. Recently, the development of VLMs has largely been driven by a relentless pursuit of scale in terms of model and data size\(Doshi,[2025](https://arxiv.org/html/2607.23125#bib.bib46); Honget al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib18); Baiet al\.,[2025a](https://arxiv.org/html/2607.23125#bib.bib47); Teamet al\.,[2026](https://arxiv.org/html/2607.23125#bib.bib48)\)\. Post\-training with high\-quality SFT data and large\-scale reinforcement learning plays an important role\. However, this leads to a high demand for human labels, distillation from frontier models, and the high sampling cost of reinforcement learning\. To tackle this, we present NOPD, a self\-training approach that eliminates the need for external supervision signals\.

#### Post\-training approaches\.

Post\-training that fine\-tunes the pre\-trained language models aims to make pre\-trained models follow human instructions and give useful and helpful responses\(Ouyanget al\.,[2022](https://arxiv.org/html/2607.23125#bib.bib28)\)\. Typical post\-training includes supervised fine\-tuning\(Weiet al\.,[2022a](https://arxiv.org/html/2607.23125#bib.bib27)\), direct preference optimization\(Rafailovet al\.,[2023](https://arxiv.org/html/2607.23125#bib.bib23)\), reinforcement learning with verifiable rewards\(Shaoet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib7); Liuet al\.,[2025](https://arxiv.org/html/2607.23125#bib.bib56)\)or from human feedback\(Ouyanget al\.,[2022](https://arxiv.org/html/2607.23125#bib.bib28)\), and distillation from stronger models in an off\-policy or on\-policy manner\(Agarwalet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib4); Guet al\.,[2024](https://arxiv.org/html/2607.23125#bib.bib13); Kim and Rush,[2016](https://arxiv.org/html/2607.23125#bib.bib29)\)\.

## 6Conclusion

We introduce Noisy Student On\-Policy Self\-Distillation \(NOPD\), a simple yet effective approach to improve VLMs without requiring ground\-truth answers or external models\. Our key insight is that forcing models to learn in difficult environments, e\.g\., noisy images, improves their capability across different tasks\. Unlike traditional distillation approaches that require teacher models, we use the outputs of the same model under clean inputs as the dense token\-level supervision signal\. Experimental results show that NOPD improves the reasoning ability of VLMs across both in\-distribution and out\-of\-distribution sets\. Furthermore, we demonstrate that NOPD can serve as a general approach to improve VLMs across different models and tasks\.

## References

- \[1\]\(2024\)On\-policy distillation of language models: learning from self\-generated mistakes\.InICLR,Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1),[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px5.p1.1),[§3](https://arxiv.org/html/2607.23125#S3.p1.16),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px3.p1.1)\.
- \[2\]S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge,et al\.\(2025\)Qwen3\-vl technical report\.arXiv preprint arXiv:2511\.21631\.Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px2.p1.1)\.
- \[3\]S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang, H\. Zhong, Y\. Zhu, M\. Yang, Z\. Li, J\. Wan, P\. Wang, W\. Ding, Z\. Fu, Y\. Xu, J\. Ye, X\. Zhang, T\. Xie, Z\. Cheng, H\. Zhang, Z\. Yang, H\. Xu, and J\. Lin\(2025\)Qwen2\.5\-vl technical report\.arXiv preprint arXiv:2502\.13923\.Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1),[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px2.p1.1)\.
- \[4\]J\. Cao and J\. Xiao\(2022\)An augmented benchmark dataset for geometric question answering through dual parallel text encoding\.InProceedings of the 29th international conference on computational linguistics,pp\. 1511–1520\.Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[5\]L\. Chen, J\. Li, X\. Dong, P\. Zhang, Y\. Zang, Z\. Chen, H\. Duan, J\. Wang, Y\. Qiao, D\. Lin, and F\. Zhao\(2024\)Are we on the right way for evaluating large vision\-language models?\.InNeurIPS,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[6\]K\. Cheng, W\. Song, J\. Fan, Z\. Ma, Q\. Sun, F\. Xu, C\. Yan, N\. Chen, J\. Zhang, and J\. Chen\(2025\)Caparena: benchmarking and analyzing detailed image captioning in the llm era\.InACL,Cited by:[Appendix C](https://arxiv.org/html/2607.23125#A3.SS0.SSS0.Px3.p2.1)\.
- \[7\]Y\. Deng, P\. Lu, F\. Yin, Z\. Hu, S\. Shen, Q\. Gu, J\. Zou, K\. Chang, and W\. Wang\(2024\)Enhancing large vision language models with self\-training on image comprehension\.InNeurIPS,Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[8\]Q\. Dong, L\. Dong, X\. Zhang, Z\. Sui, and F\. Wei\(2025\)Self\-boosting large language models with synthetic preference data\.InICLR,Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[9\]R\. Doshi\(2025\-12\)Gemini 3 pro: the frontier of vision ai\.Note:[https://blog\.google/innovation\-and\-ai/technology/developers\-tools/gemini\-3\-pro\-vision/](https://blog.google/innovation-and-ai/technology/developers-tools/gemini-3-pro-vision/)Google BlogCited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px2.p1.1)\.
- \[10\]X\. Fu, Y\. Hu, B\. Li, Y\. Feng, H\. Wang, X\. Lin, D\. Roth, N\. A\. Smith, W\. Ma, and R\. Krishna\(2024\)Blink: multimodal large language models can see but not perceive\.InECCV,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[11\]J\. Gao, R\. Pi, J\. Zhang, J\. Ye, W\. Zhong, Y\. Wang, L\. Hong, J\. Han, H\. Xu, Z\. Li,et al\.\(2023\)G\-llava: solving geometric problem with multi\-modal large language model\.arXiv preprint arXiv:2312\.11370\.Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[12\]Y\. Goyal, T\. Khot, D\. Summers\-Stay, D\. Batra, and D\. Parikh\(2017\)Making the v in vqa matter: elevating the role of image understanding in visual question answering\.InCVPR,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[13\]Y\. Gu, L\. Dong, F\. Wei, and M\. Huang\(2024\)MiniLLM: knowledge distillation of large language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1),[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px5.p1.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px3.p1.1)\.
- \[14\]C\. Gulcehre, T\. L\. Paine, S\. Srinivasan, K\. Konyushkova, L\. Weerts, A\. Sharma, A\. Siddhant, A\. Ahern, M\. Wang, C\. Gu,et al\.\(2023\)Reinforced self\-training \(rest\) for language modeling\.arXiv preprint arXiv:2308\.08998\.Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[15\]D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)DeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1)\.
- \[16\]Y\. He, S\. Kaur, A\. Bhaskar, Y\. Yang, J\. Liu, N\. Ri, L\. Fowl, A\. Panigrahi, D\. Chen, and S\. Arora\(2026\)Self\-distillation zero: self\-revision turns binary rewards into dense supervision\.arXiv preprint arXiv:2604\.12002\.Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[17\]D\. Hendrycks and T\. Dietterich\(2019\)Benchmarking neural network robustness to common corruptions and perturbations\.InICLR,Cited by:[§3](https://arxiv.org/html/2607.23125#S3.SS0.SSS0.Px2.p2.6)\.
- \[18\]G\. Hinton, O\. Vinyals, and J\. Dean\(2015\)Distilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px4.p1.3)\.
- \[19\]W\. Hong, W\. Yu, X\. Gu, G\. Wang, G\. Gan, H\. Tang, J\. Cheng, J\. Qi, J\. Ji, L\. Pan,et al\.\(2025\)Glm\-4\.5 v and glm\-4\.1 v\-thinking: towards versatile multimodal reasoning with scalable reinforcement learning\.arXiv preprint arXiv:2507\.01006\.Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1),[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px2.p1.1)\.
- \[20\]J\. Hübotter, F\. Lübeck, L\. Behric, A\. Baumann, M\. Bagatella, D\. Marta, I\. Hakimi, I\. Shenfeld, T\. K\. Buening, C\. Guestrin,et al\.\(2026\)Reinforcement learning via self\-distillation\.arXiv preprint arXiv:2601\.20802\.Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p3.1),[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px5.p1.2),[§3](https://arxiv.org/html/2607.23125#S3.p3.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[21\]D\. A\. Hudson and C\. D\. Manning\(2019\)GQA: a new dataset for real\-world visual reasoning and compositional question answering\.InCVPR,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[22\]M\. Kazemi, H\. Alvari, A\. Anand, J\. Wu, X\. Chen, and R\. Soricut\(2023\)Geomverse: a systematic evaluation of large models for geometric reasoning\.arXiv preprint arXiv:2312\.12241\.Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[23\]Y\. Kim and A\. M\. Rush\(2016\)Sequence\-level knowledge distillation\.InEMNLP,Cited by:[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px2.p1.2),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px3.p1.1)\.
- \[24\]L\. Li, Y\. Wang, R\. Xu, P\. Wang, X\. Feng, L\. Kong, and Q\. Liu\(2024\)Multimodal arxiv: a dataset for improving scientific comprehension of large vision\-language models\.InACL,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[25\]H\. Liu, C\. Li, Q\. Wu, and Y\. J\. Lee\(2023\)Visual instruction tuning\.InNeurIPS,Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px2.p1.1)\.
- \[26\]X\. Liu, J\. Ni, Z\. Wu, C\. Du, L\. Dou, H\. Wang, T\. Pang, and M\. Q\. Shieh\(2025\)NoisyRollout: reinforcing visual reasoning with data augmentation\.InNeurIPS,Cited by:[Appendix C](https://arxiv.org/html/2607.23125#A3.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px3.p1.1)\.
- \[27\]Y\. Liu, H\. Duan, Y\. Zhang, B\. Li, S\. Zhang, W\. Zhao, Y\. Yuan, J\. Wang, C\. He, Z\. Liu,et al\.\(2024\)Mmbench: is your multi\-modal model an all\-around player?\.InECCV,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[28\]P\. Lu, H\. Bansal, T\. Xia, J\. Liu, C\. Li, H\. Hajishirzi, H\. Cheng, K\. Chang, M\. Galley, and J\. Gao\(2024\)MathVista: evaluating mathematical reasoning of foundation models in visual contexts\.InICLR,Cited by:[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[29\]P\. Lu, R\. Gong, S\. Jiang, L\. Qiu, S\. Huang, X\. Liang, and S\. Zhu\(2021\)Inter\-gps: interpretable geometry problem solving with formal language and symbolic reasoning\.InACL,Cited by:[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[30\]A\. Masry, X\. L\. Do, J\. Q\. Tan, S\. Joty, and E\. Hoque\(2022\)Chartqa: a benchmark for question answering about charts with visual and logical reasoning\.InACL,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[31\]A\. Masry, M\. S\. Islam, M\. Ahmed, A\. Bajaj, F\. Kabir, A\. Kartha, M\. T\. R\. Laskar, M\. Rahman, S\. Rahman, M\. Shahmohammadi,et al\.\(2025\)Chartqapro: a more diverse and challenging benchmark for chart question answering\.InACL,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[32\]F\. Meng, L\. Du, Z\. Liu, Z\. Zhou, Q\. Lu, D\. Fu, T\. Han, B\. Shi, W\. Wang, J\. He,et al\.\(2025\)Mm\-eureka: exploring the frontiers of multimodal reasoning with rule\-based reinforcement learning\.arXiv preprint arXiv:2503\.07365\.Cited by:[§4\.2](https://arxiv.org/html/2607.23125#S4.SS2.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[33\]T\. OLMo, P\. Walsh, L\. Soldaini, D\. Groeneveld, K\. Lo, S\. Arora, A\. Bhagia, Y\. Gu, S\. Huang, M\. Jordan,et al\.\(2024\)2 olmo 2 furious\.arXiv preprint arXiv:2501\.00656\.Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1),[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px3.p1.13)\.
- \[34\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Gray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. Christiano, J\. Leike, and R\. Lowe\(2022\)Training language models to follow instructions with human feedback\.InNeurIPS,Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px3.p1.1)\.
- \[35\]A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark,et al\.\(2021\)Learning transferable visual models from natural language supervision\.InICML,Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px2.p1.1)\.
- \[36\]R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn\(2023\)Direct preference optimization: your language model is secretly a reward model\.InNeurIPS,Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px3.p1.1)\.
- \[37\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§1](https://arxiv.org/html/2607.23125#S1.p1.1),[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px3.p1.13),[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px3.p1.1)\.
- \[38\]L\. Tang, G\. Kim, X\. Zhao, T\. Lake, W\. Ding, F\. Yin, P\. Singhal, M\. Wadhwa, Z\. L\. Liu, Z\. Sprague,et al\.\(2025\)Chartmuseum: testing visual reasoning capabilities of large vision\-language models\.arXiv preprint arXiv:2505\.13444\.Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[39\]K\. Team, T\. Bai, Y\. Bai, Y\. Bao, S\. Cai, Y\. Cao, Y\. Charles, H\. Che, C\. Chen, G\. Chen,et al\.\(2026\)Kimi k2\. 5: visual agentic intelligence\.arXiv preprint arXiv:2602\.02276\.Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px2.p1.1)\.
- \[40\]S\. Tong, E\. L\. B\. II, P\. Wu, S\. Woo, A\. J\. IYER, S\. C\. Akula, S\. Yang, J\. Yang, M\. Middepogu, Z\. Wang, X\. Pan, R\. Fergus, Y\. LeCun, and S\. Xie\(2024\)Cambrian\-1: a fully open, vision\-centric exploration of multimodal LLMs\.InNeurPS,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[41\]V\. Vapnik and R\. Izmailov\(2015\)Learning using privileged information: similarity control and knowledge transfer\.Journal of Machine Learning Research16\(61\),pp\. 2023–2049\.Cited by:[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px5.p1.2)\.
- \[42\]K\. Wang, J\. Pan, W\. Shi, Z\. Lu, H\. Ren, A\. Zhou, M\. Zhan, and H\. Li\(2024\)Measuring multimodal mathematical reasoning with MATH\-vision dataset\.InNeurIPS,Cited by:[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[43\]Z\. Wang, M\. Xia, L\. He, H\. Chen, Y\. Liu, R\. Zhu, K\. Liang, X\. Wu, H\. Liu, S\. Malladi, A\. Chevalier, S\. Arora, and D\. Chen\(2024\)CharXiv: charting gaps in realistic chart understanding in multimodal LLMs\.InNeurIPS,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[44\]J\. Wei, M\. Bosma, V\. Zhao, K\. Guu, A\. W\. Yu, B\. Lester, N\. Du, A\. M\. Dai, and Q\. V\. Le\(2022\)Finetuned language models are zero\-shot learners\.InICLR,Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px3.p1.1)\.
- \[45\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, brian ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou\(2022\)Chain of thought prompting elicits reasoning in large language models\.InNeurIPS,Cited by:[Figure 1](https://arxiv.org/html/2607.23125#S1.F1),[Figure 1](https://arxiv.org/html/2607.23125#S1.F1.19.9.11),[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px3.p1.1)\.
- \[46\]S\. Yang, Y\. Liu, B\. Zhai, X\. Sun, Z\. Liu, E\. Barsoum, M\. Li, and C\. Xu\(2025\)Captionqa: is your caption as useful as the image itself?\.arXiv preprint arXiv:2511\.21025\.Cited by:[Appendix C](https://arxiv.org/html/2607.23125#A3.SS0.SSS0.Px3.p2.1)\.
- \[47\]Y\. Yang, A\. Patel, M\. Deitke, T\. Gupta, L\. Weihs, A\. Head, M\. Yatskar, C\. Callison\-Burch, R\. Krishna, A\. Kembhavi, and C\. Clark\(2025\)Scaling text\-rich image understanding via code\-guided synthetic multimodal data generation\.InACL,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px2.p1.1)\.
- \[48\]Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, YuYue, W\. Dai, T\. Fan, G\. Liu, J\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, R\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, Y\. Wu, and M\. Wang\(2025\)DAPO: an open\-source LLM reinforcement learning system at scale\.InNeurIPS,Cited by:[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px2.p1.1)\.
- \[49\]W\. Yuan, R\. Y\. Pang, K\. Cho, X\. Li, S\. Sukhbaatar, J\. Xu, and J\. E\. Weston\(2024\)Self\-rewarding language models\.InICML,Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[50\]X\. Yue, Y\. Ni, K\. Zhang, T\. Zheng, R\. Liu, G\. Zhang, S\. Stevens, D\. Jiang, W\. Ren, Y\. Sun, C\. Wei, B\. Yu, R\. Yuan, R\. Sun, M\. Yin, B\. Zheng, Z\. Yang, Y\. Liu, W\. Huang, H\. Sun, Y\. Su, and W\. Chen\(2024\)MMMU: a massive multi\-discipline multimodal understanding and reasoning benchmark for expert agi\.InCVPR,Cited by:[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[51\]X\. Yue, T\. Zheng, Y\. Ni, Y\. Wang, K\. Zhang, S\. Tong, Y\. Sun, B\. Yu, G\. Zhang, H\. Sun, Y\. Su, W\. Chen, and G\. Neubig\(2025\)MMMU\-pro: a more robust multi\-discipline multimodal understanding benchmark\.InACL,Cited by:[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p1.1)\.
- \[52\]E\. Zelikman, Y\. Wu, J\. Mu, and N\. Goodman\(2022\)STar: bootstrapping reasoning with reasoning\.InNeurIPS,Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[53\]D\. Zhang, S\. Zhoubian, Z\. Hu, Y\. Yue, Y\. Dong, and J\. Tang\(2024\)ReST\-MCTS\*: LLM self\-training via process reward guided tree search\.InNeurIPS,Cited by:[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[54\]K\. Zhang, B\. Li, P\. Zhang, F\. Pu, J\. A\. Cahyono, K\. Hu, S\. Liu, Y\. Zhang, J\. Yang, C\. Li,et al\.\(2025\)Lmms\-eval: reality check on the evaluation of large multimodal models\.InNAACL,Cited by:[§4\.3](https://arxiv.org/html/2607.23125#S4.SS3.SSS0.Px3.p2.1)\.
- \[55\]S\. Zhang, X\. Zhang, T\. Zhang, B\. Hu, Y\. Chen, and J\. Xu\(2026\)KDFlow: a user\-friendly and efficient knowledge distillation framework for large language models\.arXiv preprint arXiv:2603\.01875\.Cited by:[§4\.2](https://arxiv.org/html/2607.23125#S4.SS2.SSS0.Px1.p2.1)\.
- \[56\]S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. Grover\(2026\)Self\-distilled reasoner: on\-policy self\-distillation for large language models\.arXiv preprint arXiv:2601\.18734\.Cited by:[Appendix C](https://arxiv.org/html/2607.23125#A3.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.23125#S1.p3.1),[§2](https://arxiv.org/html/2607.23125#S2.SS0.SSS0.Px5.p1.2),[§3](https://arxiv.org/html/2607.23125#S3.p3.1),[§5](https://arxiv.org/html/2607.23125#S5.SS0.SSS0.Px1.p1.1)\.
- \[57\]C\. Zheng, S\. Liu, M\. Li, X\. Chen, B\. Yu, C\. Gao, K\. Dang, Y\. Liu, R\. Men, A\. Yang,et al\.\(2025\)Group sequence policy optimization\.arXiv preprint arXiv:2507\.18071\.Cited by:[§4\.1](https://arxiv.org/html/2607.23125#S4.SS1.SSS0.Px2.p1.1)\.

## Appendix ALimitations and Social Impact

### A\.1Limitations\.

We propose NOPD, an effective self\-boosting approach that improves VLMs without external information\. The one limitation of our approach is that it is only used in vision\-language models because it relies on data augmentations for vision inputs\.

### A\.2Social Impact

#### Positive impacts\.

We study post\-training and self\-improvements for VLMs, which improves the ability of foundation models\.

#### Negative impacts\.

This paper does not inherently involve or encourage applications with direct negative societal impacts\. We focus on a general approach to improve VLMs\.

## Appendix BExperiment Details

### B\.1Evaluation Details

#### Decoding & judge\.

For all models, we use vLLM to accelerate inference and follow the official guide to set the decoding parameters and summarize them in Table[6](https://arxiv.org/html/2607.23125#A3.T6)\. For Qwen2\.5\-VL\-7B, we report pass@1 accuracy because we use greedy decoding\. For other models, we report avg@16 to provide stable results\. For tasks that need the language model to extract and judge answers, we use GPT\-OSS\-120B\.444[https://huggingface\.co/openai/gpt\-oss\-120b](https://huggingface.co/openai/gpt-oss-120b)

#### Benchmark\-specific choices\.

In the preliminary experiments of Sec\.[4\.1](https://arxiv.org/html/2607.23125#S4.SS1), we evaluate the test/validation split of Geometry3K in an open\-ended manner, which differs from the multiple\-choice evaluation inlmms\-eval\. For MMMU and MMMU\-Pro with Qwen2\.5\-VL\-7B, we report the maximum performance between the direct answer and the CoT prompt, following the official guide\. For ChartMuseum, we report results on the test set\. For CharXiv, we report results on the validation set for the reasoning split\.

### B\.2Training Details

We summarize the training hyperparameters in Table[7](https://arxiv.org/html/2607.23125#A3.T7)\.

## Appendix CMore Results and Analysis

#### Comparison with OPSD\.

OPSD\[[56](https://arxiv.org/html/2607.23125#bib.bib24)\]also belongs to on\-policy self\-distillation where a single model plays both teacher and student roles\. OPSD uses the ground\-truth solution as additional privileged information input to the teacher model to provide a high\-quality supervision signal\. Formally, we reformat it in training VLM scenarios:

ℒOPSD\(θ\)=𝔼\(q,x\)∼𝒟,y∼πθ\(⋅∣q,x\)\[𝒟KL\(πθ\(y∣q,x\)∥πθ\(y∣q,x,y∗\)\)\]\.\\mathcal\{L\}\_\{\\text\{OPSD\}\}\\left\(\\theta\\right\)=\\mathbb\{E\}\_\{\\left\(q,x\\right\)\\sim\\mathcal\{D\},\\,y\\sim\\pi\_\{\\theta\}\\left\(\\cdot\\mid q,x\\right\)\}\\left\[\\mathcal\{D\}\_\{\\textrm\{KL\}\}\\left\(\\pi\_\{\\theta\}\\left\(y\\mid q,x\\right\)\\\|\\pi\_\{\\theta\}\\left\(y\\mid q,x,y^\{\*\}\\right\)\\right\)\\right\]\.\(9\)Unlike OPSD, our method NOPD does not need ground\-truth solutions to provide additional information\. In addition, OPSD and NOPD are also orthogonal, which means we could combine OPSD and NOPD\. We conduct preliminaries following the section in Sec\.[4\.1](https://arxiv.org/html/2607.23125#S4.SS1)and results are shown as follows\.

Table 5:Comparison with OPSD\.Geometry3KMathVistaMathVisionMMMUMMMU\-ProtestvaltestminitestvaloverallQwen3\.5\-2B73\.265\.376\.144\.364\.250\.3OPSD76\.467\.879\.544\.966\.150\.5NOPD78\.069\.180\.447\.267\.654\.7OPDS\+NOPD78\.570\.480\.247\.568\.855\.6From results of Table[5](https://arxiv.org/html/2607.23125#A3.T5), we can see that NOPD is more effective than OPSD while eliminating the need for ground\-truth solutions\. Furthermore, combining OPSD and NOPD could enhance performance\.

#### Difference with NoisyRollout\.

NOPD and NoisyRollout\[[26](https://arxiv.org/html/2607.23125#bib.bib56)\]share a similar part of adding corruption to images for VLMs\. However, the motivations behind this are totally different\. NoisyRollout is a variant of GRPO that adds noise to images toimprove exploration in reinforcement learning\. NoisyRollout still belongs to RLVR and needs a verifiable answer as we compare in Figure[1](https://arxiv.org/html/2607.23125#S1.F1)\. Our NOPD uses discrepancies between clean and distorted images as a self\-supervision signal to improve VLMs without an external model or answers\.

#### Unleashing the potential of NOPD for open\-ended generation tasks\.

We mainly evaluate NOPD on verifiable tasks, such as reasoning and question answering\. Since on\-policy distillation provides adensetoken\-level supervision signal, it is natural to ask: is NOPD effective on open\-ended generation tasks, such as image captioning? We conduct preliminary experiments on image captioning tasks\. We randomly sample 50K images from ShareGPT\-4o555[https://sharegpt4o\.github\.io/](https://sharegpt4o.github.io/)and train Qwen2\.5\-VL\-7B for 200 steps\.

We evaluate the ability of image captioning on CaptionQA\[[46](https://arxiv.org/html/2607.23125#bib.bib50)\]and CapArena\[[6](https://arxiv.org/html/2607.23125#bib.bib51)\]\. We report accuracy on CaptionQA and win rate against Gemini\-3\.0\-Flash and GPT\-5 on CapArena with 100 samples\. Results on CaptionQA are shown in Table[8](https://arxiv.org/html/2607.23125#A3.T8)\. From Table[8](https://arxiv.org/html/2607.23125#A3.T8), it can be seen that NOPD improves Qwen2\.5\-VL\-7B on all domains\. Specifically, in natural domains, NOPD achieves 13\-point gains\. We also use GPT\-4o as a VLM judge against GPT\-5 and Gemini\-3\-Flash in Figure[5](https://arxiv.org/html/2607.23125#A3.F5)on the CapArena Benchmarks\. With NOPD, a model with 7B parameters can match the performance of frontier models\. The quality of captions on more than 50% of images outperforms and matches proprietary models such as Gemini\-3\-Flash and GPT\-5\.

#### Gradient derivation\.

First, we detail the gradient of the NOPD loss as defined in Eq\. \([6](https://arxiv.org/html/2607.23125#S3.E6)\):

∇θℒNOPD​\(θ\)=∇θ​∑t=1\|y\|∑v=1\|V\|πθ​\(yv,t\|q,𝒯​\(x\)\)​log⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πθ​\(yv,t\|q,x\)\.\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\textrm\{NOPD\}\}\(\\theta\)=\\nabla\_\{\\theta\}\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,x\)\}\.\(10\)Since we stop the gradient ofπθ​\(yv,t\|q,x\)\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,x\), we denoteπref​\(yv,t\|q,x\)=πθ​\(yv,t\|q,x\)\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)=\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,x\)for clarity and ignore the subscript NOPD\.

∇θℒ​\(θ\)\\displaystyle\\nabla\_\{\\theta\}\\mathcal\{L\}\(\\theta\)=∇θ​∑t=1\|y\|∑v=1\|V\|πθ​\(yv,t\|q,𝒯​\(x\)\)​log⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πref​\(yv,t\|q,x\)\\displaystyle=\\nabla\_\{\\theta\}\\sum\_\{t=1\}^\{\\lvert y\\rvert\}\\sum\_\{v=1\}^\{\\lvert V\\rvert\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)\}=∑t=1\|y\|∑v=1\|V\|log⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πref​\(yv,t\|q,x\)​∇θπθ​\(yv,t\|q,𝒯​\(x\)\)\+∑t=1\|y\|∑v=1\|V\|πθ​\(yv,t\|q,𝒯​\(x\)\)​∇θlog⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πref​\(yv,t\|q,x\)\\displaystyle=\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)\}\\nabla\_\{\\theta\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\+\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\\nabla\_\{\\theta\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)\}=∑t=1\|y\|∑v=1\|V\|log⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πref​\(yv,t\|q,x\)​∇θπθ​\(yv,t\|q,𝒯​\(x\)\)\+∑t=1\|y\|∑v=1\|V\|πθ​\(yv,t\|q,𝒯​\(x\)\)​∇θπθ​\(yv,t\|q,𝒯​\(x\)\)πθ​\(yv,t\|q,𝒯​\(x\)\)\\displaystyle=\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)\}\\nabla\_\{\\theta\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\+\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\\frac\{\\nabla\_\{\\theta\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}=∑t=1\|y\|∑v=1\|V\|log⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πref​\(yv,t\|q,x\)​∇θπθ​\(yv,t\|q,𝒯​\(x\)\)\+∑t=1\|y\|∑v=1\|V\|∇θπθ​\(yv,t\|q,𝒯​\(x\)\)\\displaystyle=\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)\}\\nabla\_\{\\theta\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\+\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\nabla\_\{\\theta\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)=∑t=1\|y\|∑v=1\|V\|log⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πref​\(yv,t\|q,x\)​∇θπθ​\(yv,t\|q,𝒯​\(x\)\)\+∑t=1\|y\|∇θ​∑v=1\|V\|πθ​\(yv,t∣q,𝒯​\(x\)\)⏟1\\displaystyle=\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)\}\\nabla\_\{\\theta\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\+\\sum\_\{t=1\}^\{\|y\|\}\\nabla\_\{\\theta\}\\sum\_\{v=1\}^\{\|V\|\}\\underbrace\{\\pi\_\{\\theta\}\(y\_\{v,t\}\\mid q,\\mathcal\{T\}\(x\)\)\}\_\{1\}=∑t=1\|y\|∑v=1\|V\|log⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πref​\(yv,t\|q,x\)​∇θπθ​\(yv,t\|q,𝒯​\(x\)\)\\displaystyle=\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)\}\\nabla\_\{\\theta\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)=∑t=1\|y\|∑v=1\|V\|log⁡πθ​\(yv,t\|q,𝒯​\(x\)\)πref​\(yv,t\|q,x\)​πθ​\(yv,t\|q,𝒯​\(x\)\)​∇θlog⁡πθ​\(yv,t\|q,𝒯​\(x\)\)\\displaystyle=\\sum\_\{t=1\}^\{\|y\|\}\\sum\_\{v=1\}^\{\|V\|\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\textrm\{ref\}\}\(y\_\{v,t\}\|q,x\)\}\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta\}\(y\_\{v,t\}\|q,\\mathcal\{T\}\(x\)\)=∑t=1\|y\|𝔼y^t∼πθ\(⋅∣𝒯\(x\),q,y<t\)\[log⁡πθ​\(y^t∣q,𝒯​\(x\)\)πθ​\(y^t∣q,x\)⋅∇θlog⁡πθ​\(y^t∣q,𝒯​\(x\),y<t\)\]\.\\displaystyle=\\sum\_\{t=1\}^\{\|y\|\}\\mathbb\{E\}\_\{\\hat\{y\}\_\{t\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid\\mathcal\{T\}\(x\),q,y\_\{<t\}\)\}\\quad\\Bigg\[\\log\\frac\{\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,\\mathcal\{T\}\(x\)\)\}\{\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,x\)\}\\cdot\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta\}\(\\hat\{y\}\_\{t\}\\mid q,\\mathcal\{T\}\(x\),y\_\{<t\}\)\\Bigg\]\.\(11\)Finally, we get the gradient ofℒNOPD\\mathcal\{L\_\{\\textrm\{NOPD\}\}\}in Eq\. \([8](https://arxiv.org/html/2607.23125#S3.E8)\)\.

Table 6:Summary of decoding parameters during evaluation\.PresenceRepetitionModelTemperatureTop\-pTop\-kpenaltypenaltyMax tokensQwen2\.5\-VL\-7B01\.011\.01\.02048Qwen3\.5\-2B1\.00\.95201\.51\.032768Qwen3\-VL\-8B\-Thinking1\.00\.95200\.01\.040960GLM\-4\.6V\-Flash0\.80\.621\.01\.116384

Table 7:Training hyperparameters\. “X/Y” denotes that we use X for Qwen2\.5\-VL\-7B and Y for Qwen3\.5\-2B\.SFTOff\-polocy KDRLVROPDNOPDTraining backendFSDP2PrecisionBF16Gradient clip1\.0Attention backendFlashAttention 2OptimizerAdamWLearning rate1e\-6Inference engine\-\-SGLangLRSchedulercosinecosineconstantconstantconstantEpoch / iter3 epoch3 epoch100 steps100 steps100 stepsGlobal batch size64641283232Rollout temperature\-\-1\.01\.01\.0Rollout top\-p\-\-0\.950\.950\.95Rollout number\-\-811Max gen length\-\-2048/1638420482048
#### Does NOPD work by improving diversity?

We found that injecting vision permutation𝒯\\mathcal\{T\}slightly changes the distribution of student models\. This naturally raises the question: does the improvement of NOPD come from the diversity of rollouts? To disentangle the effects of diversity and vision permutation, we conduct experiments under different rollout temperatures\. For each question, we sample 5 responses and score their diversity using an LLM as a judge on a scale from 1 to 10\. Note that we keep the number of rollout response to 1 when computing gradients to align other settings\. Other questions are only used to examine diversity\.

Results are shown in Table[9](https://arxiv.org/html/2607.23125#A3.T9)\. We found that with image transformation𝒯\\mathcal\{T\}, the diversity of rollout outputs does not improve at temperatures of 0\.8 and 1\.0, yet performance increases\. At a temperature of 1\.2, diversity continues to improve but performance drops\. These results indicate that the performance gains of NOPD do not stem from higher diversity\.

Table 8:Results on CaptionQA\.ModelNaturalDocumentE\-commEmbodiedAvg\.GPT588\.1589\.9895\.4485\.3089\.72Gemini\-3\-Flash87\.6684\.5989\.9784\.0386\.56Gemini\-3\.1\-Pro88\.4990\.8290\.3685\.1588\.71Qwen2\.5\-VL\-7B71\.6475\.8585\.3868\.3675\.31\+ NOPD84\.6882\.7087\.6474\.1982\.31![Refer to caption](https://arxiv.org/html/2607.23125v1/x4.png)Figure 5:Win\-rate comparison on CapArena with Gemini\-3\-Flash and GPT\-5\.
#### Visualization examples\.

We provide two example visualizations in Table[10](https://arxiv.org/html/2607.23125#A3.T10)and Table[11](https://arxiv.org/html/2607.23125#A3.T11)to demonstrate that NOPD improves the perception and reasoning abilities compared with the base model\. In particular, Qwen2\.5\-VL\-7B fails to properly analyze the relative relationships between the line segments in the figure \(Table[10](https://arxiv.org/html/2607.23125#A3.T10)\) and to correctly identify which segment the labeled length refers to \(Table[11](https://arxiv.org/html/2607.23125#A3.T11)\)\.

Table 9:Ablation on different rollout temperature\.TemperatureAccuracyDiversityw/o𝒯\\mathcal\{T\}0\.873\.17\.21\.073\.67\.41\.273\.07\.5w/𝒯\\mathcal\{T\}0\.878\.07\.11\.078\.07\.41\.277\.87\.8Table 10:Case study of a visualized example from MathVision with Qwen2\.5\-VL\-7B\. Wehighlight errorsgenerated by Qwen2\.5\-VL\-7B\.![[Uncaptioned image]](https://arxiv.org/html/2607.23125v1/figure/example_images/mathvision_2190.jpg)Question: In△​A​B​C\\triangle ABC, mediansA​D¯\\overline\{AD\}andC​E¯\\overline\{CE\}intersect atPP,P​E=1\.5PE=1\.5,P​D=2PD=2, andD​E=2\.5DE=2\.5\. What is the area ofA​E​D​CAEDC? Answer: 13\.5Qwen2\.5\-VL\-7BTo solve the problem, we need to determine the area of quadrilateralA​E​D​CAEDCin△​A​B​C\\triangle ABCgiven the lengths of the segments formed by the intersection of the medians at pointPP\. 1\. \*\*Identify the properties of the medians and centroid:\*\* \- The medians of a triangle intersect at the centroidPP, which divides each median into a ratio of 2:1\. \- Therefore,A​P:P​D=2:1AP:PD=2:1andC​P:P​E=2:1CP:PE=2:1\. 2\. \*\*Determine the lengths of the segments:\*\* \- GivenP​E=1\.5PE=1\.5andP​D=2PD=2, we can findP​DPDandP​EPEin terms of the total length of the median\. \- SinceP​D=2PD=2andP​E=1\.5PE=1\.5,the total length of the medianC​ECEisP​D\+P​E=2\+1\.5=3\.5PD\+PE=2\+1\.5=3\.5\.\- The centroidPPdividesC​ECEinto segmentsC​PCPandP​EPEin the ratio 2:1\. Therefore,C​P=2×1\.5=3CP=2\\times 1\.5=3andP​E=1\.5PE=1\.5\. 3\. \*\*Calculate the area of△​A​E​D​C\\triangle AEDC:\*\* \- The area of△​A​E​D​C\\triangle AEDCcan be found by considering the areas of the smaller triangles formed by the medians\. \-The centroid divides the triangle into six smaller triangles of equal area\. \- The area of△​A​E​D​C\\triangle AEDCis the sum of the areas of△​A​P​E\\triangle APEand△​C​P​E\\triangle CPE\. 4\. \*\*Use the given lengths to find the area:\*\* \-The area of△​A​E​D​C\\triangle AEDCis the sum of the areas of△​A​P​E\\triangle APEand△​C​P​E\\triangle CPE\. \- The area of△​A​P​E\\triangle APEis13\\frac\{1\}\{3\}of the area of△​A​E​C\\triangle AECand the area of△​C​P​E\\triangle CPEis13\\frac\{1\}\{3\}of the area of△​C​E​D\\triangle CED\. \- The area of△​A​E​D​C\\triangle AEDCis the sum of the areas of△​A​P​E\\triangle APEand△​C​P​E\\triangle CPE, which is23\\frac\{2\}\{3\}of the area of△​A​E​C\\triangle AEC\. 5\. \*\*Calculate the area of△​A​E​D​C\\triangle AEDC:\*\* \- The area of△​A​E​D​C\\triangle AEDCis23\\frac\{2\}\{3\}of the area of△​A​E​C\\triangle AEC\. \- The area of△​A​E​C\\triangle AECis the sum of the areas of△​A​P​E\\triangle APEand△​C​P​E\\triangle CPE, which is23\\frac\{2\}\{3\}of the area of△​A​E​D​C\\triangle AEDC\. 6\. \*\*Final answer:\*\* \- The area of△​A​E​D​C\\triangle AEDCis10\\boxed\{10\}\.✘NOPDTo solve the problem, we aim to find the area of quadrilateralA​E​D​CAEDCin△​A​B​C\\triangle ABCbased on the segment lengths determined by the intersection of the medians at pointPP\.1\.Identify the properties of the medians and centroid:\- The medians of a triangle meet at the centroidPP, which divides each median in a ratio of 2:1 \(measured from the vertex to the midpoint\)\. \- Hence,A​P:P​D=2:1AP:PD=2:1andC​P:P​E=2:1CP:PE=2:1\.2\.Determine the lengths of the segments:\- GivenP​D=2PD=2, and usingP​D=13​A​DPD=\\tfrac\{1\}\{3\}AD, we obtainA​D=3×2=6AD=3\\times 2=6\. \- GivenP​E=1\.5PE=1\.5, and usingP​E=13​C​EPE=\\tfrac\{1\}\{3\}CE, we obtainC​E=3×1\.5=4\.5CE=3\\times 1\.5=4\.5\. \- In addition,D​E=2\.5DE=2\.5\. SinceDDandEEare midpoints,D​EDEis a midsegment, soD​E=12​A​CDE=\\tfrac\{1\}\{2\}AC, which impliesA​C=5AC=5\.3\.Relate the medians to the area of the triangle:\- The three medians of a triangle form a new triangle whose area is34\\tfrac\{3\}\{4\}of the area of the original triangle\. \- In this case, the medians have lengths66,4\.54\.5, and the third median can be found to be7\.57\.5\.4\.Compute the area using the given lengths:\- Applying Heron’s formula to the triangle formed by the medians:s=6\+4\.5\+7\.52=9s=\\frac\{6\+4\.5\+7\.5\}\{2\}=9Area=9​\(9−6\)​\(9−4\.5\)​\(9−7\.5\)=13\.5\\text\{Area\}=\\sqrt\{9\(9\-6\)\(9\-4\.5\)\(9\-7\.5\)\}=13\.5\- Therefore, the area of△​A​B​C\\triangle ABCis:\[A​B​C\]=43×13\.5=18\[ABC\]=\\frac\{4\}\{3\}\\times 13\.5=185\.Calculate the area ofA​E​D​CAEDC:\- TriangleB​E​DBEDis formed by connecting midpoints, so its area is14\\tfrac\{1\}\{4\}of the area of△​A​B​C\\triangle ABC:\[B​E​D\]=14×18=4\.5\[BED\]=\\frac\{1\}\{4\}\\times 18=4\.5\- Thus, the area of quadrilateralA​E​D​CAEDCis:\[A​E​D​C\]=\[A​B​C\]−\[B​E​D\]=18−4\.5=13\.5\[AEDC\]=\[ABC\]\-\[BED\]=18\-4\.5=13\.56\.Final answer:\- The area of△​A​E​D​C\\triangle AEDCis13\.5\\boxed\{13\.5\}\.✔Table 11:Case study of a visualized example from from validation set of Geometry3K with Qwen2\.5\-VL\-7B\. Wehighlight errorsgenerated by Qwen2\.5\-VL\-7B\.![[Uncaptioned image]](https://arxiv.org/html/2607.23125v1/figure/example_images/geo3k_val_293.png)Question: In⊙H\\odot H,P​Q=3​x−4PQ=3x\-4andR​S=14RS=14\. Findxx\. Answer: 6Qwen2\.5\-VL\-7BTo solve forxxin the given problem, we need to use the properties of circles and chords\. Here’s the step\-by\-step reasoning: 1\. \*\*Identify the given information:\*\* \- The circle is centered atHH\. \-P​Q=3​x−4PQ=3x\-4\. \-R​S=14RS=14\. 2\. \*\*Understand the relationship between the chords and the diameter:\*\* \- SinceTTandUUare points on the circle such thatT​H⟂P​QTH\\perp PQandU​H⟂R​SUH\\perp RS, these perpendiculars bisect the chordsP​QPQandR​SRSrespectively\. \- Therefore,P​T=T​Q=P​Q2PT=TQ=\\frac\{PQ\}\{2\}andR​U=U​S=R​S2RU=US=\\frac\{RS\}\{2\}\. 3\. \*\*Calculate the lengths of the segments:\*\* \- GivenP​Q=3​x−4PQ=3x\-4, the length ofP​TPTandT​QTQis:P​T=T​Q=P​Q2=3​x−42PT=TQ=\\frac\{PQ\}\{2\}=\\frac\{3x\-4\}\{2\}\- GivenR​S=14RS=14, the length ofR​URUandU​SUSis:R​U=U​S=R​S2=142=7RU=US=\\frac\{RS\}\{2\}=\\frac\{14\}\{2\}=74\. \*\*Set up the equation using the total length of the chordP​QPQ:\*\* \- SinceP​Q=P​T\+T​QPQ=PT\+TQ:P​Q=3​x−42\+3​x−42=3​x−4PQ=\\frac\{3x\-4\}\{2\}\+\\frac\{3x\-4\}\{2\}=3x\-45\. \*\*Solve forxx:\*\* \- We knowP​Q=3​x−4PQ=3x\-4andfrom the diagram,P​Q=16PQ=16\(sinceTTandUUare midpoints and the total length is twice the segment from the center to the midpoint of the chord\)\. \- Set up the equation:3​x−4=163x\-4=16\- Solve forxx:3​x=203x=20x=203x=\\frac\{20\}\{3\}Therefore, the value ofxxis203\\boxed\{\\frac\{20\}\{3\}\}\.✘NOPDTo solve forxxin the given problem, we need to use the properties of circles and chords\. Here’s the step\-by\-step reasoning:1\.Identify the given information:\- The circle is centered atHH\. \-P​Q=3​x−4PQ=3x\-4\. \-R​S=14RS=14\. \- The perpendicular distances from the center to both chords are equal:H​T=H​U=8HT=HU=8\.2\.Understand the relationship between the chords and the center:\- SinceH​T⟂P​QHT\\perp PQandH​U⟂R​SHU\\perp RS, these perpendiculars bisect the chordsP​QPQandR​SRSrespectively\. \- Therefore,P​T=T​Q=P​Q2PT=TQ=\\frac\{PQ\}\{2\}andR​U=U​S=R​S2RU=US=\\frac\{RS\}\{2\}\.3\.Use the equal distance property of chords:\- In a circle, chords that are equidistant from the center are equal in length\. \- SinceH​T=H​U=8HT=HU=8, chordsP​QPQandR​SRSare equal\. \- Hence,P​Q=R​SPQ=RS\.4\.Set up the equation:3​x−4=143x\-4=145\.Solve forxx:3​x=183x=18x=6x=66\.Final answer:\- The value ofxxis6\\boxed\{6\}\.✔