From Web to Pixels: Bringing Agentic Search into Visual Perception
Summary
This paper introduces WebEye, a benchmark for object localization requiring external knowledge resolution, and Pixel-Searcher, an agentic approach that connects search results to visual annotations.
View Cached Full Text
Cached at: 05/13/26, 04:11 AM
Paper page - From Web to Pixels: Bringing Agentic Search into Visual Perception
Source: https://huggingface.co/papers/2605.12497
Abstract
Researchers introduce WebEye, a benchmark for object localization requiring external knowledge resolution, and Pixel-Searcher, an agent-based approach that connects hidden target identities to visual annotations through search and reasoning.
Visual perception connects high-level semantic understanding to pixel-level perception, but most existing settings assume that the decisive evidence for identifying a target is already in the image or frozen model knowledge. We study a more practical yet harder open-world case where a visible object must first be resolved from external facts, recent events, long-tail entities, or multi-hop relations before it can be localized. We formalize this challenge asPerception Deep Researchand introduce WebEye, anobject-anchored benchmarkwithverifiable evidence,knowledge-intensive queries,precise box/mask annotations, and three task views:Search-based Grounding,Search-based Segmentation, andSearch-based VQA. WebEyes contains 120 images, 473 annotated object instances, 645 unique QA pairs, and 1,927 task samples. We further proposePixel-Searcher, anagentic search-to-pixel workflowthat resolves hidden target identities and binds them to boxes, masks, or grounded answers. Experiments show thatPixel-Searcherachieves the strongest open-source performance across all three task views, while failures mainly arise from evidence acquisition, identity resolution, andvisual instance binding.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.12497
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.12497 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.12497 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.12497 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking
PixelEyes proposes a multi-turn visual reasoning agent that decouples perception and reasoning using mask-guided search and semantic-region breadth-first search, introducing a new benchmark (Pinpoint-Bench) and dataset (PixelEyes-6K) to improve localization in visual evidence seeking.
Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning
Visual-Seeker proposes a visual-native multimodal deep search agent that actively reasons over fine-grained visual details and synthesizes multimodal evidence, achieving state-of-the-art performance on five challenging multimodal search benchmarks.
SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation
SearchEyes uses a typed knowledge graph as the backbone of a simulated search world to unify training data, search environment, and reward signals. It proposes Perception-Knowledge Chains (PKC) for multi-hop path sampling and Hop-Anchored Policy Optimization (HaPO) for step-level credit assignment, achieving state-of-the-art performance on multimodal knowledge-intensive benchmarks.
@dair_ai: Great paper discussing agentic search vs. vector search.
This paper discusses and compares agentic search with vector search approaches.
WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent
WebWatcher is a multimodal agent for deep research that uses synthetic trajectories and reinforcement learning to achieve superior performance in complex visual and textual information retrieval tasks. The paper also introduces BrowseComp-VL, a new benchmark for evaluating multimodal agents.