browsecomp-plus-mini: 15 of BrowseComp-Plus's 830 queries that rank deep-research agents roughly like the full set, no LLM judge

Reddit r/LocalLLaMA 10/18/26, 02:30 PM Tools

benchmark deep-research evaluation search-agents dataset huggingface

Summary

browsecomp-plus-mini 是一个 15 条查询的轻量子集，能以不到完整基准 2% 的成本，大致复现 BrowseComp-Plus 全部 830 条查询对深度研究智能体的排名（加权 Spearman 一致性 0.88），且无需 LLM 评分，仅用规范化精确匹配打分。

I needed a cheap way to compare search agents on BrowseComp-Plus without burning hours of agent time every run. So I built a 15-query subset of the full 830 that ranks systems roughly the way the full benchmark does. - weighted Spearman rank agreement 0.88 averaged over 30 splits, versus 0.75 for a random 15 - two runs more than 10 points apart on the full benchmark keep their order about 96% of the time; 5 to 10 points apart, about 75% - all answers are 1-4 words, so it grades by normalized exact match, no LLM judge - costs under 2% of a full run https://huggingface.co/datasets/LocalLLaMA/browsecomp-plus-mini

Original Article

Similar Articles

EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge

arXiv cs.CL

This paper introduces EvoBrowseComp, a dynamic benchmark of 400 English and 400 Chinese complex questions that are synthesized via live-web traversal to evaluate search agents without test-set contamination, ensuring robustness against parametric memorization.

Beyond Monolingual Deep Research: Evaluating Agents and Retrievers with Cross-Lingual BrowseComp-Plus

arXiv cs.CL

Introduces XBCP (Cross-lingual BrowseComp-Plus), a benchmark for evaluating deep research agents and retrievers in cross-lingual and multilingual settings. Results show significant performance degradation when evidence is in a different language from the query, highlighting both retrieval failures and agent-side difficulty in integrating language-mismatched evidence.

BrowseComp: a benchmark for browsing agents

OpenAI Blog

OpenAI released BrowseComp, a benchmark of 1,266 challenging problems designed to measure AI agents' ability to locate hard-to-find information across the internet, available in their simple evals GitHub repository.

@zijian42chen: Presenting BrowseComp-Plus at ACL2026, Sunday July 5th, 14:00 - 15:30, Harbor G Oral Session. We are grateful to the ma…

X AI KOLs Following

Announcing BrowseComp-Plus, presented at ACL2026, which establishes a task for designing search for agents. The post thanks contributors from Databricks, Google, and the open-source community for accelerating agent, search, and memory development for Opus 4.5.

Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries

Hugging Face Daily Papers

该论文提出 Librarian Agents 框架，利用强化训练 builder agent 将原始文档重组为可复用的层级化工作区，以服务未来的多样化查询。在 BrowseComp-Plus 上比最强检索与预计算基线高出 33 个百分点，并在 BRIGHT 上零样本泛化提升 31 个百分点。