A practitioner compares Qwen3 27B running INT8 vs INT4 AWQ over two weeks of heavy agent logs, finding INT4 dramatically increases reasoning-loop rates (1.9 vs 4.7 loops/1k calls), cuts task success from 92% to 78%, and the gap widens at long context above 100k tokens. INT8 and even BF16 also loop, so quantization amplifies rather than causes the issue.
I know this some old news, but i got some numbers here from my heavy agent load. In mid-October I switch from INT8 (4×3090) to AWQ INT4 (2×3090). Same sampling, harness(Hermes), and loop detector, so I compared about two weeks of logs from each setup. vLLM, FP8 KV, with DFlash2. temp 1.0 / top_p 0.95 / top_k 20, presence_penalty 0, thinking medium. Two models: int4-awq and int8. Results (INT8 → INT4 AWQ) Model calls: 17.7k → 11.4k Loops per 1k calls: 1.9 → 4.7 Conversations with ≥1 loop: 6% → 12% Per-task success rate: 92% → 78% (success rate is after loop got cut, ask it continue can finish the task) Both setups had similar reasoning lengths, so "more quant thinks longer" doesn't explain the gap. EVERY loop was in the reasoning block, so it cooked when it cooking too hard is the problem as you guys know. Retrying rarely fixes it. After a thinking loop cut, next call ask it continue got looped again 14–22% of the time, compared with under 1% normally. The context is what making loop happens, not about luck in sampling. The INT4 gap grows with context. INT4 looped 1.4x as often as INT8 under ~40k tokens and 3.3x as often at 80–120k content. With speculative decoding, loops show up as a smooth speed spike, loops ran at ~200 tok/s versus ~107 tok/s normally on my setup, because the draft model predicts repeated text very well. That makes a cheap watchdog signal. Mostly the model saying it will act and never doing it ("Let me write it. Wait let me... How about..." 10+ line as one loop), or restating the same case analysis dozens of times. TL;DR: INT8 loops too, and people in the HF discussion report the same 20k+ token thinking spirals on BF16, so this isn't purely a quant problem. INT4 or Q4 just makes it a lot worse, especially at long context above 100k~. btw loops not that bad, just don't push model too hard, I feels on daily assistant loops happens much less. I've saved thoes contexts that looped. When Qwen4 27b drops I can just replay them and post the new results.
A user's detailed comparison of Qwen3.8 and Qwen3.6 models in coding tasks, highlighting improvements in instruction following and tracing for Qwen3.8, but with inefficiencies in reasoning.
This article compares Qwen 3.8 to 3.6, noting that Qwen 3.8 reduces reasoning loops on low settings and includes a preserve_thinking parameter to avoid redundant reasoning.
Reddit post compares quantized Qwen3.6-27B variants (INT4, NVFP4, BF16-INT4) showing trade-offs between memory size and accuracy for different use-cases.
A local LLM user reports that sacrificing context length (96K→75K) to run higher-quality quantizations (IQ4_XS→UD-Q4_K_XS with KV-Q8) on a Qwen 27B model actually improved overall task completion time, cutting an 8-hour coding task to under 6 hours due to fewer error-prone drafts.
The user finds Qwen 3.8 27b in Q3 quantization highly effective for coding tasks with fast inference speeds, outperforming previous models, despite minor issues in general conversations.