VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocation

Hugging Face Daily Papers Papers

Summary

VideoSeeker introduces a paradigm for instance-level video understanding that integrates agentic reasoning with visual prompts, achieving superior performance through automated data synthesis and reinforcement learning, outperforming GPT-4o and Gemini-2.5-Pro.

Large Vision-Language Models (LVLMs) have shown significant progress in video understanding, yet they face substantial challenges in tasks requiring precise spatiotemporal localization at the instance level. Existing methods primarily rely on text prompts for human-model interaction, but these prompts struggle to provide precise spatial and temporal references, resulting in poor user experience. Furthermore, current approaches typically decouple visual perception from language reasoning, centering reasoning around language rather than visual content, which limits the model's ability to proactively perceive fine-grained visual evidence. To address these challenges, we propose VideoSeeker, a novel paradigm for instance-level video understanding through visual prompts. VideoSeeker seamlessly integrates agentic reasoning with instance-level video understanding tasks, enabling the model to proactively perceive and retrieve relevant video segments on demand. We construct a four-stage fully automated data synthesis pipeline to efficiently generate large-scale, high-quality instance-level video data. We internalize tool-calling and proactive perception capabilities into the model via cold-start supervision and RL training, building a powerful video understanding model. Experiments demonstrate that our model achieves an average improvement of +13.7% over baselines on instance-level video understanding tasks, surpassing powerful closed-source models such as GPT-4o and Gemini-2.5-Pro, while also showing effective transferability on general video understanding benchmarks. The relevant datasets and code will be released publicly.
Original Article
View Cached Full Text

Cached at: 05/19/26, 10:31 AM

Paper page - VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocation

Source: https://huggingface.co/papers/2605.16079 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

VideoSeeker introduces a novel paradigm for instance-level video understanding by integrating agentic reasoning with visual prompts, achieving superior performance through automated data synthesis and reinforcement learning.

Large Vision-Language Models(LVLMs) have shown significant progress in video understanding, yet they face substantial challenges in tasks requiring precise spatiotemporal localization at the instance level. Existing methods primarily rely on text prompts for human-model interaction, but these prompts struggle to provide precise spatial and temporal references, resulting in poor user experience. Furthermore, current approaches typically decouple visual perception from language reasoning, centering reasoning around language rather than visual content, which limits the model’s ability to proactively perceive fine-grained visual evidence. To address these challenges, we propose VideoSeeker, a novel paradigm forinstance-level video understandingthroughvisual prompts. VideoSeeker seamlessly integratesagentic reasoningwithinstance-level video understandingtasks, enabling the model to proactively perceive and retrieve relevant video segments on demand. We construct a four-stage fullyautomated data synthesispipeline to efficiently generate large-scale, high-quality instance-level video data. We internalize tool-calling and proactive perception capabilities into the model viacold-start supervisionand RL training, building a powerful video understanding model. Experiments demonstrate that our model achieves an average improvement of +13.7% over baselines oninstance-level video understandingtasks, surpassing powerful closed-source models such as GPT-4o and Gemini-2.5-Pro, while also showing effective transferability on general video understanding benchmarks. The relevant datasets and code will be released publicly.

View arXiv pageView PDFProject pageGitHub2Add to collection

Get this paper in your agent:

hf papers read 2605\.16079

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.16079 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.16079 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.16079 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning

Hugging Face Daily Papers

OmniSeek is an agentic framework that turns an Omni-LLM into an active multi-turn reasoning agent with native tool use, dynamically retrieving audio/visual evidence across long contexts. It uses a synthetic OmniTraj-170K trajectory corpus, supervised cold-starting, and two-stage verifiable-reward RL with an Audio-Visual Necessity objective to boost cross-modal reasoning.