swe-bench

SWE-Pruner Pro leverages the coding agent's own internal representations to prune long code context, saving up to 39% of tokens while maintaining or improving task performance on multi-turn benchmarks.

China released Ling-2.6-1T, a 1 trillion parameter open-source model that achieves 72.2% on SWE-bench with a 256K context window, claiming high efficiency and compatibility with Claude Code.

A blog post arguing that using a frontier model only for planning and a cheaper model for execution is not cost-effective because reading—not editing—is the primary cost driver; duplicate reading offsets any savings.

This paper proposes function-aware fill-in-the-middle mid-training for coding agent foundation models, exploiting the structural similarity between function calls and agent action-observation loops. The method improves performance on SWE-Bench benchmarks across multiple model sizes and post-training pipelines.

This paper investigates the minimal context needed for coding agents to edit code, finding that natural-language summaries of code are ineffective and that surrounding context matters little, with compressed context achieving equal results at a third of the tokens. It also reveals a noise floor due to temperature-0 API inference.

ACQUIRE is a QA-driven framework that decouples knowledge acquisition from patch generation for software issue resolution, outperforming pre-repair methods on SWE-bench Verified.

This paper introduces an Agent-Computer Interface (ACI) for autonomous coding agents, replacing raw bash with purpose-built commands for navigation, editing, and feedback, achieving state-of-the-art results on SWE-bench and HumanEvalFix.

TRACE is a new self-improvement approach where an AI agent identifies the missing capabilities behind its own failures and trains itself to address them. TRACE-trained Qwen3.6-27B achieves 73.2% on SWE-bench Verified, outperforming much larger models with fewer training rollouts.

OpenAI describes its audit of SWE-Bench Pro using model-based investigator agents and independent reviews from experienced software engineers to ensure thorough evaluation at scale.

OpenAI audited the SWE-Bench Pro coding benchmark and found approximately 30% of tasks broken, retracting their previous recommendation for its use as a leading coding evaluation.

OpenAI audited SWE-Bench Pro, a coding benchmark, and found approximately 30% of tasks are broken due to issues like overly strict tests and underspecified prompts, advising model developers to carefully examine results.

A roundup of benchmarks including SWE-Bench Pro and 312 real-world workflow tasks, likely evaluating AI performance on software engineering challenges.

This paper introduces SWE-Review, a framework that closes the loop on AI-generated pull requests by enabling iterative agentic review and revision cycles, improving code quality and issue resolution. Experimental results show that it outperforms single-turn review and enables effective test-time scaling.

This paper introduces SEA, an architecture for self-evolving agents that confines self-modification to a steering adapter and versioned harness around a frozen base model, using anytime-valid gates to audit modifications against a fixed error budget. Experiments on SWE-bench Verified with four base models show that the suite provides a +4 to +5% improvement on strong base models while preventing regressions.

Senior SWE-Bench is an open-source benchmark that evaluates AI agents on software engineering tasks requiring senior-level skills.

SWE-rebench leaderboard updated with new models (GLM-5.2, Qwen3.6, Gemma 4 31B, etc.) and an improved UI, showing performance rankings on software engineering tasks.

Tweet highlighting top-performing AI models under 128B parameters on the SWE-bench_pro benchmark, noting Alibaba Qwen 3.6 27B and ornith 35B as leading contenders.

A commentator discusses the performance of a 9B model on coding benchmarks, noting that while it beats its base on SWE-bench (69 vs 53), the advantage narrows on behavioral and long-horizon tests, suggesting limited gains outside benchmark distributions.

GitHub benchmarked its Copilot agentic harness against model-vendor harnesses, finding comparable task resolution with fewer tokens across multiple benchmarks, highlighting Copilot's support for over 20 models.

A fully open-source Python replica of Claude Code has been released on GitHub, supporting multiple models like GPT and Gemini, achieving 58.2% on SWE-bench Verified, and claiming to be 6x cheaper than the original.