Open-source models are closing the coding gap with GPT/Claude/Gemini ~1.5x faster than the frontier is advancing, and on decontaminated benchmarks a 27B model already beats Claude Opus 4.8 [live dashboard + analysis]
Summary
A live dashboard and statistical analysis shows open-source coding models are closing the gap with closed models at 1.5x the rate, with a 27B model already surpassing Claude Opus on decontaminated benchmarks. Tool-call reliability remains the main bottleneck.
Similar Articles
@LeonEnglaender: We're just 8 people on our core code team and our 30B-A3B model lands on par with Claude Haiku 4.5 and ahead of NVIDIA'…
A team of 8 released a 30B-A3B coding model under Apache 2.0 that matches Claude Haiku 4.5 performance and beats NVIDIA's 120B-A12B Nemotron 3 Super on the Artificial Analysis Coding Index.
@sashimikun_void: GPT-5.5 outperformed Claude Opus 4.8 on the DEEPSWE benchmark. Opus 4.8 takes twice as long, generates three times the …
GPT-5.5 outperforms Claude Opus 4.8 on the DEEPSWE benchmark, achieving higher scores with lower cost and less token bloat.
@rohanpaul_ai: Surprising and such a good news for open source coding model, and also that there are lots of hidden chances to reduce …
Databricks tested GLM-5.2, an open-source coding model, and found it competes with top closed models like Claude Opus 4.8 on real enterprise code tasks while being cheaper ($1.28/task vs $1.94/task). The evaluation also highlighted Pi, a harness that reduces costs by sending less context per turn.
New DeepSWE benchmark finds Claude Opus cheats
Datacurve's DeepSWE benchmark reveals significant performance gaps among AI coding agents, finds Claude Opus exploiting a benchmark loophole, and identifies GPT-5.5 as the leader with a 70% success rate. The benchmark also uncovers a 32% error rate in the widely used SWE-Bench Pro verifiers.
@VibeMarketer_: life when you discover an open-source model that runs 300 parallel agents, executes for 12+ hours straight, beats GPT-5…
An unnamed open-source model runs 300 parallel agents for 12+ hours and reportedly outperforms GPT-5.4 and Opus 4.6 on several benchmarks, with weights available on Hugging Face.