Paper page - EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge

Source: https://huggingface.co/papers/2606.13120

Abstract

EvoBrowseComp is an evolving benchmark with 800 contamination-free questions synthesized through a three-agent framework that ensures temporal freshness and prevents parametric memorization in search agent evaluation.

Search Agents--large language modelsaugmented with search tools -- have intensified the need for future-proof evaluation benchmarks. Existing benchmarks such asBrowseComprely on static knowledge, making them vulnerable to test-set contamination andparametric memorization. Consequently, models can achieve high scores through fact recall rather than genuine retrieval, obscuring true browsing competence via reasoning shortcuts. In this paper, we introduce EvoBrowseComp, an evolving benchmark of 400 English and 400 Chinesecontamination-freecomplex questions synthesized vialive-web traversal. To collect these questions, we design a three-agent collaborative framework: (1) aQA synthesis agentthat retrieves fresh knowledge from the live web to synthesize QA pairs; (2) aninformation filtering agentthat filters retrieved knowledge in terms of credibility and popularity to block parametric shortcuts; and (3) ahigh-level guidance agentthat formalizes questions intoreasoning graphsto reduce logical redundancy and shortcuts in synthesized QA pairs. Because the framework supports fullyautomated synthesis, EvoBrowseCompcan be regularly updated to prevent data contamination and maintaintemporal freshness. Extensive experiments confirm its great difficulty, requiring broad horizontal search. It establishes a scalable paradigm for auto-updatable, high-difficulty benchmarking that keeps pace with both evolving world knowledge and advancing agent capabilities.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2606\.13120

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.13120 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.13120 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.13120 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.