I ran the 35B agentic comparison someone asked for (stock vs Ornith vs KAT-Coder, 120 runs)

Reddit r/LocalLLaMA News

Summary

A detailed bakeoff of 35B coding models (KAT-Coder-V2.5-Dev, Qwen3.5, Ornith, etc.) with 120 runs shows KAT-Coder matching the best stock pass rate with cleaner tool behavior, while Ornith fails due to mechanical issues. Full methodology and results are linked.

Someone in the comments of my 27B post-train bakeoff asked for the 35B version, so I ran it. Same setup as last time: fresh Coder workspaces on my k8s cluster, each driving my own agent (Hermes) headlessly, models on llama.cpp via llama-swap on one 5090, every call traced through an OTel shim into SigNoz, full transcript per run. 4 models, 6 self-grading tasks, 5 reps, 120 runs, MTP on every arm, identical sampling, hypotheses pre-registered. KAT-Coder-V2.5-Dev matched the best stock pass rate (29/30, tied with Qwen3.5-35B) at half the input tokens of either stock and the cleanest tool behavior I've measured (zero malformed tool-call leaks in 30 runs; stock Qwen3.6 leaked 195 on one task). All six analysts (three model families) independently called its efficiency discipline rather than corner-cutting: baseline tests before edits, one targeted patch per bug, deliverables at the right path every rep. Ornith went 25/30, losing to its own base. Its failures were mechanics, not knowledge: format leaks killing runs at 23 seconds, whole-file rewrites corrupting unrelated files, and one research run that invented a llama.cpp release tag while its own reasoning said "I mentioned v4659 in my draft which is fabricated," then shipped the tag anyway. The grader passed it. Stock 3.6 is the strongest raw analyst and the biggest token waster; stock 3.5 is the quiet reliable one. Full writeup with methodology, tables, and all six cited per-task analyses: https://kmarble.dev/posts/35b-coder-bakeoff/. Transcripts were AI-analyst-read with my spot-verification of every consequential claim.
Original Article

Similar Articles

KAT Coder 2.5 dev: Do yourself a favor and try it!

Reddit r/LocalLLaMA

A developer enthusiastically recommends KAT Coder 2.5 dev, claiming it is faster, more accurate, and uses fewer tokens than Qwen 3.6 35b a3b, and outperforms Gemma 4 models on their setup, with a GitHub repo containing detailed benchmarks.

Kwaipilot/KAT-Coder-V2.5-Dev

Hugging Face Models Trending

KAT-Coder-V2.5-Dev is an open-weight MoE coding model with 35B total parameters (3B active), achieving state-of-the-art results on agentic coding benchmarks through SFT and RL training.