Region-Level Policy Optimization for Fine-grained MLLM Perception

Hugging Face Daily Papers Papers
fine-grained-perception multimodal-llm reinforcement-learning region-proposal token-compression visual-encoding

Summary

Vision-RL² is a method for improving fine-grained perception in multimodal large language models by using region-level reinforcement learning to compress visual tokens and enhance performance across multiple benchmarks without full model fine-tuning.

Fine-grained visual perception in MLLMs is commonly improved by raising the resolution, but the added visual tokens inflate vision-encoding and language-model prefilling costs. We show that the two operations underlying fine-grained perception, localizing the region of interest (RoI) and recognizing its content, have different resolution requirements. In a controlled diagnostic, localization tolerates roughly 3 to 4 times stronger token compression than recognition, which motivates localizing from a coarse view and concentrating resolution on the selected evidence. Decoding coordinates with the MLLM can be trained end-to-end from answers, but costs a full model pass per query and depends on grounding ability. A lightweight proposal network distilled from the model's attention is fast, but inherits the noise of its attention targets. The RoI from the proposal network reaches the answer through a discrete region choice, so its faithfulness to the answer cannot supervise the network. We therefore optimize the proposal network with region-level reinforcement learning, which we call Vision-RL2. It treats coherent regions as actions, and a frozen MLLM reader scores each one by how its removal changes the answer likelihood. Complementary subtractive and additive objectives suppress distracting proposals and recover missing evidence, updating only the predictor without region annotations, response sampling, or reasoning trajectories. The refined proposal further enables a sparse encoding that magnifies evidence and excludes background tokens. Across six fine-grained benchmarks and four MLLM backbones, Vision-RL2 improves accuracy over the base model at every token budget and surpasses its largest-budget accuracy with about 4 times fewer visual tokens. Code is available at https://github.com/YuHengsss/VisionRL2 .
Original Article
View Cached Full Text

Cached at: 09/18/26, 03:01 AM

Paper page - Region-Level Policy Optimization for Fine-grained MLLM Perception

Source: https://huggingface.co/papers/2609.19745 Vision-RL²: Region-Level Policy Optimization for Fine-grained MLLM Perception

Fine-grained perception in MLLMs usually means raising the resolution, which inflates visual-token and prefill cost. We show thatlocalizingthe region of interest tolerates roughly 3–4× stronger token compression thanrecognizingits content, so the two should not share one resolution.

Vision-RL² trains a lightweight region proposal network on top of afrozenMLLM with region-level RL: coherent regions are the actions, and a frozen reader scores each one by how much its removal changes the answer likelihood. No region annotations, no response sampling, no reasoning traces. The refined proposal also enables a sparse visual encoding that keeps only foreground tokens.

Across six fine-grained benchmarks (V*, ZoomBench, HR-Bench 4K/8K, MME-RealWorld EN/CN) and four backbones (Qwen3.5-4B/9B, Qwen2.5-VL-7B, and the encoder-free Gemma-4-12B), Vision-RL² improves over the base model at every visual-token budget, and outperforms prior methods that fully fine-tune the model while training an order of magnitude fewer parameters.

Similar Articles

Structured Role-Aware Policy Optimization for Multimodal Reasoning

arXiv cs.AI

This paper introduces Structured Role-Aware Policy Optimization (SRPO), a method that improves multimodal reasoning in Large Vision-Language Models by assigning token-level credit based on distinct perception and reasoning roles within reinforcement learning frameworks.

Reinforcing Multimodal Reasoning Against Visual Degradation

Hugging Face Daily Papers

This paper introduces ROMA, an RL fine-tuning framework that enhances the robustness of multimodal large language models against visual degradations like blur and compression artifacts. It achieves this through a dual-forward-pass strategy and specialized regularization techniques, improving performance on reasoning benchmarks without sacrificing accuracy on clean inputs.

VFA: Empowering Multilingual MLLMs via Vision-Free Adaptation

arXiv cs.CL

This paper proposes Vision-Free Adaptation (VFA), a framework that enhances multilingual capabilities in multimodal large language models by merging multilingual and vision-aligned task vectors without visual data, demonstrating improved performance and data efficiency.