Paper page - SWE-Explore: Benchmarking How Coding Agents Explore Repositories

Source: https://huggingface.co/papers/2606.07297

Abstract

SWE-Explore introduces a benchmark for evaluating coding agents’ repository exploration capabilities by requiring ranked lists of relevant code regions within line budgets, demonstrating that agentic exploration outperforms traditional retrieval methods.

Repository-level coding benchmarks such asSWE-benchhave driven a rapid surge in the capabilities ofcoding agents. Yet they usually treat coding tasks as a holistic, binary prediction problem (e.g., resolved or unresolved), neglecting fine-grained agent capabilities such asrepository understanding,context retrieval,code localization, andbug diagnosis. In this paper, we introduceSWE-Explore, a benchmark that isolates the evaluation ofrepository exploration, a critical capability ofcoding agents. Given a repository and an issue,SWE-Exploreasks an explorer to return a ranked list of relevant code regions under a fixedline budget.SWE-Explorecovers 848 issues across 10 programming languages and 203 open-source repositories. For each instance, we derive line-level ground truth from independent agent trajectories that successfully solved the same issue, distilling the specific code regions their solution paths actually consulted. We evaluate exploration along coverage,ranking, andcontext-efficiencydimensions, showing that these metrics strongly track downstream repair behavior. Across a broad set ofretrieval methods, generalcoding agents, and specialized localizers, we find thatagentic explorersform a clear tier above classical retrieval. While file-level localization is already strong for modern methods,line-level coverageand efficientrankingremain the key axes differentiating state-of-the-art explorers.

View arXiv pageView PDFProject pageGitHub5Add to collection

Get this paper in your agent:

hf papers read 2606\.07297

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.07297 in a model README.md to link it from this page.

Datasets citing this paper1

#### SWE-Explore-Bench/SWE-Explore-Bench Viewer• Updatedabout 17 hours ago • 848 • 61 • 6

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.07297 in a Space README.md to link it from this page.

Collections including this paper1