SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning
Summary
The paper introduces SeePhys Pro, a benchmark to diagnose modality transfer issues in multimodal RL for physics reasoning, revealing that models struggle with representation-invariant reasoning and often rely on residual textual cues rather than visual evidence.
View Cached Full Text
Cached at: 05/13/26, 08:11 AM
Paper page - SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning
Source: https://huggingface.co/papers/2605.09266 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
SeePhys Pro benchmark reveals that current multimodal models struggle with representation-invariant reasoning when information shifts from text to visual formats, and demonstrates that blind training can improve performance through residual textual cues.
We introduce SeePhys Pro, a fine-grainedmodality transferbenchmark that studies whether models preserve the same reasoning capability when critical information is progressively transferred from text to image. Unlike standardvision-essential benchmarksthat evaluate a single input form, SeePhys Pro features four semantically aligned variants for each problem with progressively increasing visual elements. Our evaluation shows that current frontier models are far fromrepresentation-invariant reasoners: performance degrades on average as information moves from language to diagrams, withvisual variable groundingas the most critical bottleneck. Motivated by this inference-time fragility, we further develop large training corpora formultimodal RLVRand useblind trainingas a diagnostic control, finding that RL with all training images masked can still improve performance on unmasked validation sets. To analyze this effect,text-deletion, image-mask-rate, andformat-saturationcontrols suggest that such gains can arise from residual textual and distributional cues rather than valid visual evidence. Our results highlight the need to evaluatemultimodal reasoningnot only by final-answer accuracy, but also by robustness undermodality transferand by diagnostics that test whether improvements rely on task-critical visual evidence.
View arXiv pageView PDFProject pageGitHub9Add to collection
Get this paper in your agent:
hf papers read 2605\.09266
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.09266 in a model README.md to link it from this page.
Datasets citing this paper4
#### Kun-Xiang/Track3-SeePhysPro-Testmini Viewer• Updatedabout 20 hours ago • 830 • 552 #### Kun-Xiang/Track3-SeePhysPro-Test Viewer• Updatedabout 20 hours ago • 3.32k • 147 #### Kun-Xiang/PhysRL Viewer• Updatedabout 2 hours ago • 47k • 15 • 1 #### Kun-Xiang/SeePhysPro Viewer• Updatedabout 2 hours ago • 4.15k • 9
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.09266 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
Bad Seeing or Bad Thinking? Rewarding Perception for Vision-Language Reasoning
This paper introduces a reinforcement learning framework that improves perception-reasoning synergy in vision-language models by explicitly rewarding perceptual fidelity, using a 'blindfolded reasoning' proxy and structured verbal verification to address ambiguity in modality credit assignment.
Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap
This paper introduces CrossMath, a controlled multimodal reasoning benchmark that reveals a critical limitation in current vision-language models: they perform reasoning primarily in textual space rather than genuine vision-grounded reasoning, with visual input often degrading performance compared to text-only baselines. The authors propose fine-tuning approaches to mitigate this modality gap and improve multimodal reasoning capabilities.
The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm
This paper challenges the assumption that current Vision-Language Models faithfully synthesize multimodal data, proposing an information-theoretic Modality Translation Protocol with new metrics (Toll, Curse, Fallacy of Seeing) to evaluate trustworthiness over traditional multimodal gain.
BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs
BilliardPhys-Bench is a new benchmark that tests multimodal LLMs on physical reasoning using synthetic billiards scenarios, requiring predictions of collisions and final ball positions. The paper finds that current models struggle with longer simulations and exhibit a 'stasis bias' of predicting no interaction when uncertain.
Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning
This paper audits multimodal physics evaluation pipelines, revealing issues like train-eval contamination, translation drift, and MCQ saturation. It releases new datasets (PhysCorp-A, PhysR1Corp, PhysOlym-A) and a training recipe (Physics-R1) that significantly improves performance on held-out olympiad problems.