TRON: Targeted Rule-Verifiable Online Environments for Visual Reasoning RL

Hugging Face Daily Papers Papers
reinforcement-learning visual-reasoning online-environments rule-verifiable training-signals multimodal benchmarks

Summary

TRON introduces a scalable online environment for visual reasoning reinforcement learning that generates unlimited diverse training instances with verifiable answers, showing consistent performance improvements across multiple multimodal benchmarks.

Reinforcement learning (RL) for visual reasoning needs scalable, verifiable, and controllable training signals. Existing visual RL post-training trains on static curated datasets, with fixed image-question-answer samples bounded by their collection budget. In this work, we introduce TRON (Targeted, Rule-verifiable Online eNvironments), an online environment substrate: a training rollout is generated on demand by a controllable generator-verifier program that samples a fresh latent visual state, renders an image, asks a question, and exactly verifies the answer. A single run can therefore draw an unbounded stream of fresh instances at the difficulty level required by the current curriculum. The current TRON suite contains 520 environments organized into five ability buckets (spatial, mathematical, diagram, pattern/logic, and counting); the same substrate supports both a single full model trained on all buckets and per-bucket ability-specialist models, with no additional data collection. We also introduce a substrate analysis covering generation reliability, instance and level diversity, cross-environment near-duplicates, and base-model pass rate by difficulty level. RL post-training with METHOD consistently improves performance on ten external multimodal reasoning benchmarks across Qwen3-VL-4B, Qwen2.5-VL-7B, and MiMo-VL-7B-SFT.
Original Article
View Cached Full Text

Cached at: 06/03/26, 07:36 AM

Paper page - TRON: Targeted Rule-Verifiable Online Environments for Visual Reasoning RL

Source: https://huggingface.co/papers/2606.01599

Abstract

TRON enables scalable and controllable reinforcement learning for visual reasoning through an online environment substrate that generates unlimited diverse training instances with verifiable answers.

Reinforcement learning(RL) forvisual reasoningneeds scalable, verifiable, and controllable training signals. Existing visual RL post-training trains on static curated datasets, with fixed image-question-answer samples bounded by their collection budget. In this work, we introduce TRON (Targeted, Rule-verifiable Online eNvironments), anonline environment substrate: a training rollout is generated on demand by acontrollable generator-verifier programthat samples a freshlatent visual state, renders an image, asks a question, and exactly verifies the answer. A single run can therefore draw an unbounded stream of fresh instances at the difficulty level required by the current curriculum. The current TRON suite contains 520 environments organized into fiveability buckets(spatial, mathematical, diagram, pattern/logic, and counting); the same substrate supports both a single full model trained on all buckets and per-bucket ability-specialist models, with no additional data collection. We also introduce a substrate analysis covering generation reliability, instance and level diversity, cross-environment near-duplicates, and base-model pass rate by difficulty level. RL post-training with METHOD consistently improves performance on ten externalmultimodal reasoning benchmarksacross Qwen3-VL-4B, Qwen2.5-VL-7B, and MiMo-VL-7B-SFT.

View arXiv pageView PDFProject pageGitHub1Add to collection

Get this paper in your agent:

hf papers read 2606\.01599

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.01599 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.01599 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.01599 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Tandem Reinforcement Learning with Verifiable Rewards

arXiv cs.AI

Proposes Tandem Reinforcement Learning (TRL), extending the tandem training paradigm to RLVR to improve reasoning compatibility and legibility for weaker models and humans, showing that TRL matches solo performance while enhancing handoff robustness and reducing distributional drift.

Trace: A Taxonomy-Guided Environment for Multidomain Visual Reasoning

Hugging Face Daily Papers

TRACE is a taxonomy-guided environment with 1,000 visual reasoning tasks across 11 domains. Training Qwen2.5-VL-3B and Qwen2.5-VL-7B on 64,000 TRACE instances improves their macro-average performance across 24 external benchmarks by 3.51 and 4.06 percentage points respectively.

Video Models Can Reason with Verifiable Rewards

Hugging Face Daily Papers

VideoRLVR optimizes video diffusion models for verifiable reasoning tasks using reinforcement learning with rule-based rewards, achieving better performance than supervised methods in constraint-satisfying video generation.

Reasoning Arena: Trace Tournaments When Verifiable Rewards Fall Short

Hugging Face Daily Papers

Reasoning Arena improves reinforcement learning with verifiable rewards by using trace tournaments and Bradley-Terry models to generate meaningful gradients from non-diverse reward groups, resulting in faster training and better reasoning performance.