Tag
SWEeper-Bench 引入了一个包含200个任务的基准，这些任务来自200个广泛使用的开源Web应用的真实bug修复，用于评估编码agent能否在交互式软件中自主发现依赖多步交互才能暴露的缺陷。分析显示，即使是15个前沿agent中最好的也仅通过59.0%的任务，主要瓶颈在于识别bug行为本身。
Experiments on AI agents in code editing tasks show that agents often over-edit code when bugs are already fixed, with performance varying based on task instructions. A benchmark using real repositories highlights issues with agent behavior in software development.
This article compiles a list of intriguing historical stories and anecdotes from computer science and software development, highlighting notable bugs, failures, and design insights.
A discussion questioning whether AI's ability to find software bugs is a problem or an opportunity for companies like Google and Microsoft to proactively fix vulnerabilities.
Vitalik Buterin argues against the narratives that bugs are inevitable and that AI bug-finding necessitates closed-source, stating that writing secure code has become harder but not impossible.