Qwen 3.8 27b is strong even at Q3_xxs

Reddit r/LocalLLaMA Models

Summary

The user finds Qwen 3.8 27b in Q3 quantization highly effective for coding tasks with fast inference speeds, outperforming previous models, despite minor issues in general conversations.

So usually I avoid Q3 quants because I have had bad experiences with it, models were usually too degraded, so the smallest I normally do is Q4, since I only have rtx 4060 ti 16gb. But since there hasn't been a 35b-3ab released yet, I had to try it. I don't use LLMs in agentic workflows, just on Textgen since I'm not a coder so this is not the primary use case of LLMs for me - but sometimes I really need some coding capabilities or help. I'm very impressed how it one shot multiple serious coding tasks, resulting in fully working games or web apps, whereas Qwen 3.6 35b (which I used before) either completely failed in some of these or struggled a lot and needed hours/days of assistance/prompting, feedback to make it work. And it is very fast when fully in VRAM. 30-35t/s, basically the same speed as higher quant 35b offloaded to RAM! Only at long context it goes down to 21-22t/s. Older dense models like Gemma 3 27b, Mistral small 24b are only doing 13-17t/s at best. The only thing I noticed is it sometimes misunderstands things during regular convos or fails at basic sorting or counting few scores, while one shotting serious math/logic tasks. Not sure if this is because it's code-maxxed or because of the low quant (I'd think it's heavily the latter but I'd be interested in your guys' experiences who can run this at higher quants). So far I'm very happy with it, it's way better than the higher Q4-Q5 MoEs I've tried so far.
Original Article

Similar Articles

Qwen3.8-27B Q6 is a beast at agentic coding

Reddit r/LocalLLaMA

User feedback indicates that Qwen3.8-27B Q6 demonstrates high performance in agentic coding tasks, maintaining 60-63 tokens/s over 20 hours on dual NVIDIA GPUs.

Qwen 3.6 35B A3B vs Qwen 3.5 122B A10B

Reddit r/LocalLLaMA

User reports Qwen 3.5 122B significantly outperforms Qwen 3.6 35B on multi-step tasks despite benchmark claims, questioning if quantization or setup issues are to blame.