@no_stp_on_snek: someone will wave the card at me: the 9B crushes its base on the coding benchmarks (SWE-bench 69 vs 53). true. but on m…
Summary
A commentator discusses the performance of a 9B model on coding benchmarks, noting that while it beats its base on SWE-bench (69 vs 53), the advantage narrows on behavioral and long-horizon tests, suggesting limited gains outside benchmark distributions.
View Cached Full Text
Cached at: 06/30/26, 07:41 AM
someone will wave the card at me: the 9B crushes its base on the coding benchmarks (SWE-bench 69 vs 53).
true. but on my held-out behavioral and long-horizon tests that win narrows to a trade. the gap shrinks the moment you leave the benchmark distribution.
that’s exactly why i don’t trust public benchmarks alone.
Tom Turney (@no_stp_on_snek): verdict up front: it’s a “pass” in my book in certain categories, just a narrower one than the 35B. you’re buying real efficiency and sustained-coding coherence here, not a clean across-the-board upgrade over the base.
ran the same skeptic battery on Ornith-1.0’s little brother,
Similar Articles
@no_stp_on_snek: a new 35B coder dropped (Ornith-1.0) and a promo blog says it "crushes" the benchmarks. my first instinct was benchmaxx…
A new 35B coding model, Ornith-1.0, is compared against Qwen3.6-35B on custom tests. The user finds Ornith-1.0 to be genuinely stronger for long-horizon agentic coding, resisting bad context and finishing large tasks, but it is more cautious and verbose, sometimes over-gating simple requests.
@no_stp_on_snek: verdict up front: it's a "pass" in my book in certain categories, just a narrower one than the 35B. you're buying real …
The author evaluates Ornith-9B against its base Qwen3.5-9B, finding that RL post-training improves token efficiency and sustained coding coherence but sacrifices single-turn judgment and robustness to misleading inputs, making it a narrower upgrade at 9B compared to the 35B version.
@LeonEnglaender: We're just 8 people on our core code team and our 30B-A3B model lands on par with Claude Haiku 4.5 and ahead of NVIDIA'…
A team of 8 released a 30B-A3B coding model under Apache 2.0 that matches Claude Haiku 4.5 performance and beats NVIDIA's 120B-A12B Nemotron 3 Super on the Artificial Analysis Coding Index.
@jerryjliu0: An interesting thing I'm observing from the blog/system card is that on a good chunk of the reported benchmarks (~20-30…
An observation that Claude Opus 5's max thinking leads to performance degradation on ~20-30% of benchmarks compared to xhigh, contrary to the expectation that more test-time compute improves performance.
@xdotli: my friend @xeophon thinks coding is solved here's validation that a 3b model is trained with focus on algo efficiency a…
Nanbeige 4.1, a 3B model, outperforms Qwen3-30b-A3b and Qwen 3.5 4b in coding tasks with focus on algorithmic efficiency, achieving long horizon tasks with 600+ tool calls.