MMSearch-OPD: Easy Made Hard, Hard Made Learnable for Long-Horizon Multimodal Search Agents
Summary
MMSearch-OPD 提出了一种结合可控数据精炼与联合强化学习和在线策略蒸馏的方法，以提升长时程多模态搜索智能体的能力，通过将简单问题扩展为困难且可学习的训练样本取得显著改进。
View Cached Full Text
Cached at: 10/16/26, 07:03 AM
Paper page - MMSearch-OPD: Easy Made Hard, Hard Made Learnable for Long-Horizon Multimodal Search Agents
Source: https://huggingface.co/papers/2610.22240 Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
MMSearch-OPD improves long-horizon multimodal search by refining reasoning data and jointly training with reinforcement learning and on-policy distillation.
Multimodal deep searchis moving from short queries to long-horizon investigations that continually acquire evidence. However, training such agents remains constrained by limited challenging training questions and insufficient process feedback. We introduce MMSearch-OPD, combining controllable data refinement with jointreinforcement learning(RL) andon-policy distillation(OPD). Our Refine pipeline performs targeted expansion ofreasoning chainsderived from existing simple multimodal questions. Parameterized transformations control search depth andcross-modal evidencerequirements while reusing existing image--question pairs. For training, answer rewards and lightweightbehavior penaltiesguide RL, while the reward gap modulatestoken-level teacher guidanceon imperfect student trajectories. Anexponentially decaying distillation coefficientreduces teacher dependence as training proceeds. Experiments show gains from data refinement and further improvements from joint training after SFT, indicating stronger long-horizonmultimodal deep searchcapabilities.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2610\.22240
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.22240 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.22240 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.22240 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration
Introduces SearchOS, a multi-agent framework for robust open-domain information-seeking that externalizes search progress into explicit states via a novel Search-Oriented Context Management (SOCM) system, achieving state-of-the-art results on WideSearch and GISA benchmarks.
ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering
Proposes ProMSA, a progressive multimodal search agent for knowledge-based visual question answering that adaptively selects search strategies and optimizes through sequence-level reinforcement learning, achieving consistent gains on E-VQA and InfoSeek.
MOAE: Multi-Objective Agent Evolution with Pareto-Preserving Search
MOAE proposes a Pareto-preserving evolutionary search method to simultaneously optimize multiple objectives like task performance, trajectory quality, and safety for LLM agents without fixed scalarization during search.
HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents
HyperEyes is a parallel multimodal search agent that uses dual-grained reinforcement learning to optimize inference efficiency, achieving higher accuracy with significantly fewer tool-call rounds compared to existing agents.
OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents
OpenSearch-VL is an open-source framework and paper introducing a recipe for training frontier multimodal search agents using reinforcement learning, featuring specialized data curation and a novel training algorithm.