A hands-on benchmark of 10 AI coding agents run headless against a local Qwen3.5-4B model across five coding tasks, finding that Pi, goose, and Crush finished fastest while Codex CLI failed entirely due to API incompatibility with local setups.
I tested 10 agents, each with its own isolated settings folder. I ran them headless, with auto-confirmation and turned off telemetry where possible. I launched each agent with adapters/<agent>.sh. The idea was simple: give each agent 5 basic tasks, 10 minutes per task, a fresh copy of the fixture every time, and an automated check script (check/check.sh). The tasks: T1 Read a DB port from YAML (don't confuse it with cache port), write to file T2 Write a stats.py script, run it, save output T3 Fix a bug in slugify so tests pass; hidden examples catch overfitting T4 Count ERROR lines across three log files, find most frequent error T5 Rename a function across three files without changing program output Results 1. Pi (v1.3.0) Total time: 1m36s. Took the smallest amount of time to complete all 5 tasks. The model barely has to think about any tool formats. T1 T2 T3 T4 T5 Total ✅ 10s ✅ 14s ✅ 24s ✅ 19s ✅ 29s 5/5 2. goose (v1.55.0) Total time: 2m51s. Solid and consistent. Completed five tasks without fumbling. T1 T2 T3 T4 T5 Total ✅ 16s ✅ 18s ✅ 26s ✅ 45s ✅ 66s 5/5 3. Crush (v0.98.1) Total time: 3m31s. Clean run. Slightly more verbose in reasoning but got everything done without a hiccup. T1 T2 T3 T4 T5 Total ✅ 24s ✅ 26s ✅ 64s ✅ 38s ✅ 59s 5/5 4. Atomic Agent (v0.6.9) Total time: 4m38s. All five completed. Took longer on T3 and T5 - the model needed extra steps to work through multi-file edits but nailed it every time. Did not have broken tool calls and not one had a false completion. T1 T2 T3 T4 T5 Total ✅ 11s ✅ 22s ✅ 77s ✅ 48s ✅ 104s 5/5 5. Cline (v3.0.73) Total time: 5m56s. Choked for over two minutes on T2 before writing the script correctly. Everything else was fine. T1 T2 T3 T4 T5 Total ✅ 22s ✅ 137s ✅ 51s ✅ 69s ✅ 77s 5/5 6. OpenClaw (v2026.10.5) Total time: 5m44s. Ran in their --local-model-lean mode which is basically a small-model preset. Worked well for what it is, just slower than I'd like on T3 and T4. T1 T2 T3 T4 T5 Total ✅ 29s ✅ 32s ✅ 107s ✅ 116s ✅ 60s 5/5 7. OpenCode (v1.19.3) Total time: 17m43s. Completed everything but took nearly 4 minutes per task. The model was clearly overthinking simple steps. Got the job done though, just not efficient. T1 T2 T3 T4 T5 Total ✅ 241s ✅ 225s ✅ 178s ✅ 289s ✅ 130s 5/5 8. Hermes (v0.22.2) Total time: 4m36s. Failed T3 because it only fixed the visible test case. Hidden example C++ & Rust 2026 turned into c++-&-rust-2026 instead of c-rust-2026. Classic overfitting-to-the-test. T1 T2 T3 T4 T5 Total ✅ 31s ✅ 37s ❌ 120s ✅ 45s ✅ 43s 4/5 9. Qwen Code (v0.25.3) Total time: 6m00s. Counted 40 ERROR lines on T4 instead of 43. Pure model miscount - the most frequent error was identified correctly though. T1 T2 T3 T4 T5 Total ✅ 47s ✅ 32s ✅ 123s ❌ 59s ✅ 99s 4/5 10. Codex CLI (v0.163.0) Never even reached the model. Version 0.163 only speaks Responses API, llama.cpp translates to chat format, and the Qwen3.5 (4B Q4_K_M) template throws a 500 "System message must be at the beginning" before any request gets through. T1 T2 T3 T4 T5 Total ❌ ❌ ❌ ❌ ❌ 0/5 What I actually learned A 4B model is enough for basic agent work in almost any framework. Seven agents completed all five tasks. The bottleneck at this difficulty level isn't the model - it's the harness around it. Codex CLI is basically dead for local setups right now unless they add chat-completions back. That one stung because it used to work. Error recovery matters more than raw capability. The two agents that dropped points (Hermes, Qwen Code) didn't fail on impossible tasks - they failed on edge cases where the model got slightly creative instead of literal. Let me know if you'd like to see anything else in the benchmark (actually has anyone tried this with sub-2B models yet?)
Qwen 3.6 27B is praised as a powerful local AI model that outperforms expectations for general intelligence, suitable for practical tasks like code generation, and runs easily with llama.cpp.
By pairing Qwen3.6-35B with the little-coder agent scaffold, the model hits 78.7% on the Polyglot coding benchmark, placing in the public top 10 and rivaling cloud models.
A user describes their fully local AI stack using multiple hardware devices running Chinese models like GLM, Qwen, and Kimi, claiming 87% cost savings compared to frontier models like GPT-5.5 and Opus 4.8, while noting plans to self-host video generation.
Alibaba's Qwen team has released Qwen3.7-Max, a proprietary agent-foundation model achieving top scores on multiple benchmarks including Terminal-Bench 2.0, SWE-Pro, and GPQA Diamond, with consistent performance across various code environments.
The article asks if there are better small AI models than Qwen3.5 4B for building a fast local assistant, focusing on improving capabilities like conversation, reasoning, multilingual support, and tool calling while maintaining speed.