Increasing active parameters per token in MOE (Qwen 35B A4B+) reduce reasoning token by 8.5% - and you don't need to train or finetune!

I want to share a short paper just published exploring a simple but surprisingly effective optimization for sparse MoE reasoning models. The idea: Instead of retraining anything, we just tweak the router at runtime. Specifically, we expand the expert selection budget (N≥KN≥K) only in the late transformer layers, with a linear decay factor applied to the extra experts. Early layers stay untouched. So Qwen 3.6 35B A3B becomes Qwen 3.6 35B A4B+ ! What we found — "Succinct Convergence": When you give the model more expert capacity at the decision-critical final layers, it stops rambling. It reaches the same correct answer via significantly shorter reasoning trajectories. Results on full MMLU-Pro (714 questions, Qwen3.6-35B-A3B): 📉 8.5% reduction in mean reasoning tokens ⚡ 10.9% drop in latency (p=6.5×10−6) 🎯 Accuracy unchanged (84.5% vs 84.0% native, p=0.77 — statistically indistinguishable) 🆓 Zero training cost — pure inference-time routing modification Links: 📄 Paper: https://zenodo.org/records/22255483 there you can also take a look to my github repo (with beta version code) and the detailed json results of MMLU-Pro benchmark. In the future i hope i can make same experimentation with a larger model like DeepSeek V4 Flash Q2.0 I'm a Non-native english speaker, part of this post was generated , for translation reason with the help of AI.