SWE-Explore: Benchmarking How Coding Agents Explore Repositories
Summary
SWE-Explore introduces a benchmark for evaluating coding agents' repository exploration capabilities, requiring ranked lists of relevant code regions within line budgets. Experiments show agentic exploration outperforms traditional retrieval, and line-level coverage remains a key differentiator.
View Cached Full Text
Cached at: 06/09/26, 08:43 AM
Paper page - SWE-Explore: Benchmarking How Coding Agents Explore Repositories
Source: https://huggingface.co/papers/2606.07297
Abstract
SWE-Explore introduces a benchmark for evaluating coding agents’ repository exploration capabilities by requiring ranked lists of relevant code regions within line budgets, demonstrating that agentic exploration outperforms traditional retrieval methods.
Repository-level coding benchmarks such asSWE-benchhave driven a rapid surge in the capabilities ofcoding agents. Yet they usually treat coding tasks as a holistic, binary prediction problem (e.g., resolved or unresolved), neglecting fine-grained agent capabilities such asrepository understanding,context retrieval,code localization, andbug diagnosis. In this paper, we introduceSWE-Explore, a benchmark that isolates the evaluation ofrepository exploration, a critical capability ofcoding agents. Given a repository and an issue,SWE-Exploreasks an explorer to return a ranked list of relevant code regions under a fixedline budget.SWE-Explorecovers 848 issues across 10 programming languages and 203 open-source repositories. For each instance, we derive line-level ground truth from independent agent trajectories that successfully solved the same issue, distilling the specific code regions their solution paths actually consulted. We evaluate exploration along coverage,ranking, andcontext-efficiencydimensions, showing that these metrics strongly track downstream repair behavior. Across a broad set ofretrieval methods, generalcoding agents, and specialized localizers, we find thatagentic explorersform a clear tier above classical retrieval. While file-level localization is already strong for modern methods,line-level coverageand efficientrankingremain the key axes differentiating state-of-the-art explorers.
View arXiv pageView PDFProject pageGitHub5Add to collection
Get this paper in your agent:
hf papers read 2606\.07297
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.07297 in a model README.md to link it from this page.
Datasets citing this paper1
#### SWE-Explore-Bench/SWE-Explore-Bench Viewer• Updatedabout 17 hours ago • 848 • 61 • 6
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.07297 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
WideSWE: Can Coding Agents Coordinate Changes Across Repositories?
The paper introduces WideSWE, a benchmark for evaluating coding agents on cross-repository tasks, based on 120 real-world tasks, showing that agent success rates vary and joint execution can improve performance.
Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents
Introduces Agent Retrieval Bench, a file-level benchmark evaluating how well coding agents retrieve relevant repository files during the context-acquisition stage. The benchmark includes 427 samples across 25 repositories and evaluates various retrieval methods, finding no single family dominates.
SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?
SWE-bench Science introduces a repository-level benchmark for evaluating coding agents on scientific software repair tasks, revealing failure mechanisms and mixed effects of scientific guidance.
SWE Context Bench just proved something I think a lot of coding agent users already feel
A new benchmark paper 'SWE Context Bench' tests whether coding agents can reuse knowledge across tasks, highlighting a gap in existing benchmarks that only evaluate isolated problem-solving. The author discusses solutions like external memory and mentions tools such as langmem, mem0, supermemory, and Greplica.
FastContext: Training Efficient Repository Explorer for Coding Agents
FastContext introduces specialized exploration models that separate repository exploration from code solving in LLM agents, reducing token consumption by up to 60% while improving resolution rates on software engineering benchmarks.