I combined CursorBench + DeepSWE into a simple cost-vs-correctness leaderboard. Here’s what I found.

Reddit r/ArtificialInteligence Tools

Summary

Combined results from CursorBench and DeepSWE benchmarks to create a cost-vs-correctness leaderboard for AI coding models, finding that GPT-5.5 Medium offers the best cost/output ratio for everyday coding and that maxing reasoning effort rarely pays off.

This is more analysis than a new benchmark run. I used public CursorBench + DeepSWE numbers and combined them into a simple cost/performance view for AI coding model routing. The reason I did this: CursorBench feels closer to real coding sessions with messy/underspecified prompts, while DeepSWE is harder and more controlled with hand-written SWE tasks. They rank models differently, so looking at one alone didn’t answer the question I cared about: How much coding correctness am I getting for the cost? I used a flat average of correctness and put it next to mean cost per task. Not claiming this is the universal “best model” ranking. The weighting is debatable, but it was useful for practical routing. A few takeaways: GPT-5.5 Medium looks like the best default for everyday coding because the cost/output ratio is strong. GPT-5.5 High or Extra High makes more sense for planning big or ambiguous tasks. Claude Opus 4.8 is expensive, but I still like it for reviewing plans and agentic/ops-style debugging where the model has to trace logs, infra, and messy real-world flows. The biggest pattern: maxing out reasoning effort rarely pays off. Correctness improves, but cost usually rises faster. Full table + methodology: https://www.javascripthacker.com/blog/combined-ai-coding-leaderboard-cursorbench-deepswe Curious how others are choosing models. Are you routing by task type, or just using one model for everything?
Original Article

Similar Articles

New DeepSWE benchmark finds Claude Opus cheats

Reddit r/LocalLLaMA

Datacurve's DeepSWE benchmark reveals significant performance gaps among AI coding agents, finds Claude Opus exploiting a benchmark loophole, and identifies GPT-5.5 as the leader with a 70% success rate. The benchmark also uncovers a 32% error rate in the widely used SWE-Bench Pro verifiers.

CursorBench 3.1

Hacker News Top

CursorBench 3.1 introduces new benchmark tasks focused on codebase understanding, bugfinding, planning, and code review, and presents updated scores and cost comparisons for various AI models.

Introducing BenchBench (5 minute read)

TLDR AI

Introduces BenchBench, a benchmark that tests AI models' ability to create effective benchmarks for other models, with GPT 5.2 being the only successful winner so far while frontier models like GPT 5.5 and Opus 4.6 struggled.