SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning

Hugging Face Daily Papers Papers
multimodal-rl physics-reasoning benchmark modality-transfer blind-training visual-grounding

Summary

The paper introduces SeePhys Pro, a benchmark to diagnose modality transfer issues in multimodal RL for physics reasoning, revealing that models struggle with representation-invariant reasoning and often rely on residual textual cues rather than visual evidence.

We introduce SeePhys Pro, a fine-grained modality transfer benchmark that studies whether models preserve the same reasoning capability when critical information is progressively transferred from text to image. Unlike standard vision-essential benchmarks that evaluate a single input form, SeePhys Pro features four semantically aligned variants for each problem with progressively increasing visual elements. Our evaluation shows that current frontier models are far from representation-invariant reasoners: performance degrades on average as information moves from language to diagrams, with visual variable grounding as the most critical bottleneck. Motivated by this inference-time fragility, we further develop large training corpora for multimodal RLVR and use blind training as a diagnostic control, finding that RL with all training images masked can still improve performance on unmasked validation sets. To analyze this effect, text-deletion, image-mask-rate, and format-saturation controls suggest that such gains can arise from residual textual and distributional cues rather than valid visual evidence. Our results highlight the need to evaluate multimodal reasoning not only by final-answer accuracy, but also by robustness under modality transfer and by diagnostics that test whether improvements rely on task-critical visual evidence.
Original Article
View Cached Full Text

Cached at: 05/13/26, 08:11 AM

Paper page - SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning

Source: https://huggingface.co/papers/2605.09266 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

SeePhys Pro benchmark reveals that current multimodal models struggle with representation-invariant reasoning when information shifts from text to visual formats, and demonstrates that blind training can improve performance through residual textual cues.

We introduce SeePhys Pro, a fine-grainedmodality transferbenchmark that studies whether models preserve the same reasoning capability when critical information is progressively transferred from text to image. Unlike standardvision-essential benchmarksthat evaluate a single input form, SeePhys Pro features four semantically aligned variants for each problem with progressively increasing visual elements. Our evaluation shows that current frontier models are far fromrepresentation-invariant reasoners: performance degrades on average as information moves from language to diagrams, withvisual variable groundingas the most critical bottleneck. Motivated by this inference-time fragility, we further develop large training corpora formultimodal RLVRand useblind trainingas a diagnostic control, finding that RL with all training images masked can still improve performance on unmasked validation sets. To analyze this effect,text-deletion, image-mask-rate, andformat-saturationcontrols suggest that such gains can arise from residual textual and distributional cues rather than valid visual evidence. Our results highlight the need to evaluatemultimodal reasoningnot only by final-answer accuracy, but also by robustness undermodality transferand by diagnostics that test whether improvements rely on task-critical visual evidence.

View arXiv pageView PDFProject pageGitHub9Add to collection

Get this paper in your agent:

hf papers read 2605\.09266

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.09266 in a model README.md to link it from this page.

Datasets citing this paper4

#### Kun-Xiang/Track3-SeePhysPro-Testmini Viewer• Updatedabout 20 hours ago • 830 • 552 #### Kun-Xiang/Track3-SeePhysPro-Test Viewer• Updatedabout 20 hours ago • 3.32k • 147 #### Kun-Xiang/PhysRL Viewer• Updatedabout 2 hours ago • 47k • 15 • 1 #### Kun-Xiang/SeePhysPro Viewer• Updatedabout 2 hours ago • 4.15k • 9

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.09266 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap

arXiv cs.CL

This paper introduces CrossMath, a controlled multimodal reasoning benchmark that reveals a critical limitation in current vision-language models: they perform reasoning primarily in textual space rather than genuine vision-grounded reasoning, with visual input often degrading performance compared to text-only baselines. The authors propose fine-tuning approaches to mitigate this modality gap and improve multimodal reasoning capabilities.

See2Think: Do Multimodal Models Really Use Intermediate Visual States?

Hugging Face Daily Papers

This paper introduces See2Think, a unified evaluation framework to test whether multimodal models genuinely rely on intermediate visual states during reasoning. It finds that visual reasoning is highly model- and environment-dependent, with faithful rendering being the main bottleneck.