Replaced Claude with local Qwen3.6-27B in my multi-agent orchestrator for 2 weeks

For two weeks I ran my multi-agent orchestrator entirely on Qwen3.6-27B via Ollama, on a single 3090. The goal: see if a local model could replace Claude as the reasoning layer for the lead/manager/sub-agent loop. Here's where it worked and where it broke. Setup: \- RTX 3090, 24GB VRAM \- Qwen3.6-27B at Q6\_K (\~22GB on-GPU), 32k effective context \- Ollama as the inference engine \- Multi-agent orchestrator with structured-JSON plans, plan-approval modal, auto-review pass after sub-agent completion \- Tested across 47 multi-step coding workflows over two real repos What worked (the reasoning layer): \- Plan generation. Qwen3.6 generated multi-step plans roughly as well as Claude on these tasks. Slightly more conservative (fewer unsolicited "let me also refactor X" steps), but coherent and schema-valid at \~95% after a few prompt tweaks. The remaining 5% were schema fixable with one re-prompt. \- Memory extraction. Mem0-style fact extraction every 6 turns worked fine. Qwen pulled out the same kinds of facts Claude does ("user prefers no comments unless they explain a 'why'") and stored them cleanly in Qdrant. \- Auto-review of sub-agent output. A second Qwen instance reviewing the first one's code caught roughly 60% of the bugs Claude's review caught on the same set. Less savage. Still useful and free. Where it broke: \- Tool-call reliability. Qwen3.6's JSON tool-call output had a \~12% format error rate across the 47 tasks. Claude was \~0.5% on the same workload. The errors weren't malformed JSON they were wrong field names, wrong types, hallucinated tool signatures. Outlines / strict-output mode reduced it but didn't kill it. \- Long-context drift. Past \~14k tokens of accumulated session context, Qwen started misremembering decisions it had made earlier ("you said use Postgres" no, I said the opposite). Hard practical limit \~12k tokens, then aggressive summarize-and-reset. \- Cascade-failure handling. When a sub-agent failed, Claude's planner usually noticed and re-planned. Qwen sometimes just generated downstream steps assuming the sub-agent had succeeded. Three cascading hallucinations in 47 runs. Not catastrophic with plan gating in place. Would be catastrophic without. The contrarian take: Qwen3.6-27B is a viable REASONING layer for local multi-agent systems today. It is NOT a viable execution layer. Run plans through it; gate every tool call. Practical implication: if you're building local-only agents, you need (1) structured-output enforcement at the tool-call boundary (outlines, lm-format-enforcer, or your inference engine's grammar mode), (2) plan-approval gating so the 12% format errors don't reach actual file writes, (3) re-plan-on-failure logic the model itself can't be trusted to do. The 12% tool-call gap is the metric to close. Once Qwen3.6 (or the next local model) hits \~2% on this, the case for cloud reasoning in agent loops gets weaker fast. Disclosure: the orchestrator I tested this on is OpenYabby (openyabby.com). I built it. Tested honestly because I genuinely wanted to know if I could stop paying Anthropic.