AtomicChat/Qwen3.8-Flash-Next-GGUF is Really Good

Reddit r/LocalLLaMA Models
quantization llama-cpp inference benchmarking gguf ai-model

Summary

The AtomicChat quantization of Qwen3.8-Flash-Next-GGUF significantly reduces memory usage from 106GB to 65GB while maintaining good inference performance, making it more efficient for hardware-limited setups.

specs hardware: M4 Max 128GB Studio inference engine: llama.cpp (qwen4exp branch) judge: claude-opus-4-6 AtomicChat/Qwen3.8-Flash-Next-GGUF Qwen3.8-Flash-Next is a great model I benched in my previous post, but it is very tight, since all n-grams / PLE are loaded along with the experts, taking 106GB, leaving very little room for K/V, context, etc. Offloading PLE to SSD currently slows down prefill from 600 t/s to 180 t/s on oMLX. u/erikdhoward suggested to try the Atomic Chat quant which I did not know anything about. I tried it, and it is... really good. AtomicChat's quant uses llama.cpp mmap (through GGUF shard layout vs. in the runtime) and keeps the PLE table (n-grams) pageable backed by a file. Because of this the same model that took 106GB, now takes 65GB (starts from 55GB) in RAM. And since PLE is pageable the prefill is actually not that bad, cold start is about 500 t/s. oMLX "right behind you!" Qwen3.8-Flash-Next just came out, and there are many open PRs in oMLX to address the size and performance, including this one that makes PLE offload SSD cold prefill almost 3 times faster 🎉
Original Article

Similar Articles

orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF

Hugging Face Models Trending

This article releases GGUF quantizations of the uncensored Qwen3.8-Flash-Next model, a Mixture-of-Experts preview of the Qwen4 architecture designed for llama.cpp with vision support, requiring a custom build for compatibility.

Qwen3.8 Flash Quants

Reddit r/LocalLLaMA

The author released mainline-compatible imatrix quants for the Qwen3.8-Flash-Next model, offering 20–30GB smaller sizes with competitive PPL, and includes a separate ROCm build for AMD hardware.