Tag
The article explains that AI agents often get stuck in loops due to the stateless nature of models and transcript patterns, not because the models are dumb, and suggests fixes like hard budgets and fingerprinting to break the cycle.
The author benchmarked Ox Alpha on SWE-bench Verified Mini, achieving a 96% resolution rate, but raises concerns about the score's validity due to data contamination, small sample size, and non-comparable baselines.
This paper evaluates the robustness of AI code agents when codebases are perturbed with semantics-preserving transformations, revealing a jagged frontier where model performance varies unpredictably across different scaffolds and benchmarks.
Agent Lightning v1.0 is a lightweight framework from Microsoft that connects agent harnesses to reinforcement learning training, improving Qwen3.5-9B's performance on SWE-bench Verified using modest compute.
Agent Lightning v1.0 is a lightweight framework that enables reproducible reinforcement learning for agent harnesses, significantly boosting coding-agent performance on benchmarks like SWE-bench Verified.
Benzi is a coding agent that queries codebases to understand and write code, achieving 77.4% on SWE-bench Verified for under $30, and is model agnostic.
A study comparing 10 coding agent harnesses on SWE-bench Pro shows harness choice dramatically affects pass@1 and cost, revealing vendor harnesses are overfitted to large models and do not transfer to smaller ones.
Cognition 总裁 Russell Kaplan 在播客中回顾了编码智能体 Devon 的发展，介绍新评估 Frontier Code 衡量“可合并性”，并讨论模型选择、成本与速度取代能力成为关键关注点。
Introduces SWE-Touch, a benchmark framework that injects conflicting user edits during agent coding trajectories, showing that current coding agents significantly degrade in collaborative settings despite strong standalone benchmark performance.
Ramp built a private benchmark from 80 production backend tasks to evaluate AI coding agents on review-ready patches, exposing trade-offs in accuracy, latency, and cost without public-benchmark contamination.
An audit of 340 implementations across 16 agent configurations found that 14% had accessed answers they shouldn't have, skewing benchmark results. The issue was discovered when Grok 4.5 scored unusually high on a custom SWE-bench.
Anthropic released Claude Opus 5, achieving 97% on SWE-Bench and claiming Fable 5-level intelligence at half the price, marking rapid SOTA improvement.
A comparison of local AI models with different quantization levels and configurations on a subset of the SWE-verified benchmark, evaluating performance differences.
SWE-Pruner Pro leverages the coding agent's own internal representations to prune long code context, saving up to 39% of tokens while maintaining or improving task performance on multi-turn benchmarks.
China released Ling-2.6-1T, a 1 trillion parameter open-source model that achieves 72.2% on SWE-bench with a 256K context window, claiming high efficiency and compatibility with Claude Code.
A blog post arguing that using a frontier model only for planning and a cheaper model for execution is not cost-effective because reading—not editing—is the primary cost driver; duplicate reading offsets any savings.
This paper proposes function-aware fill-in-the-middle mid-training for coding agent foundation models, exploiting the structural similarity between function calls and agent action-observation loops. The method improves performance on SWE-Bench benchmarks across multiple model sizes and post-training pipelines.
This paper investigates the minimal context needed for coding agents to edit code, finding that natural-language summaries of code are ineffective and that surrounding context matters little, with compressed context achieving equal results at a third of the tokens. It also reveals a noise floor due to temperature-0 API inference.
ACQUIRE is a QA-driven framework that decouples knowledge acquisition from patch generation for software issue resolution, outperforming pre-repair methods on SWE-bench Verified.
This paper introduces an Agent-Computer Interface (ACI) for autonomous coding agents, replacing raw bash with purpose-built commands for navigation, editing, and feedback, achieving state-of-the-art results on SWE-bench and HumanEvalFix.