Why MOE below A10b feels like im gambling
Summary
Developer reports that small-active-parameter MOE models like qwen3.6-35b-A3b exhibit lower coherence and require more guidance than dense qwen3.5-27b, making them hard to slot into agentic workflows.
Similar Articles
Qwen3.5-27B, Qwen3.5-122B, and Qwen3.6-35B on 4x RTX 3090 — MoEs struggle with strict global rules
A user benchmarks three Qwen models (Qwen3.5-27B dense, Qwen3.5-122B-A10B MoE, Qwen3.6-35B-A3B MoE) on 4x RTX 3090 GPUs under real agentic workloads, finding that MoE models consistently underperform the dense 27B at following strict global rules despite speed advantages, with the Qwen3.6-35B leading in generation throughput.
Qwen-AgentWorld-35B-A3B: a 3B-active MoE trained to simulate MCP, terminal, SWE, Android, web and OS environments
Qwen released Qwen-AgentWorld-35B-A3B, a 35B-parameter MoE model with 3B active parameters, designed as a language world model to simulate environment responses for agent interactions across seven domains including MCP, terminal, SWE, Android, web, and OS.
Qwen3.5 122B is the best?
A user shares their experience comparing several large language models (Qwen, Gemma) on complex tool-calling tasks, finding Qwen3.5 122B the most reliable, while criticizing smaller MoE models for instability.
Qwen 35B-A3B MoE vs 27B dense in local coding tests: ~4× faster, much smaller quality gap than I expected
A local experiment comparing Qwen 35B-A3B MoE and Qwen 27B dense on coding-maintenance tasks, finding the MoE model ~3.9× faster with a smaller quality gap than expected.
Forgive my ignorance but how is a 27B model better than 397B?
User questions how Qwen's 27B dense model can outperform its 397B MoE variant, sparking discussion on MoE efficiency versus dense model quality.