@rohanpaul_ai: atomic[.]chat just released 14 compressed quantized builds of DeepSeek V4 Flash 0731. From lossless BF16 to 1-bit, GGUF…
Summary
atomic.chat released 14 quantized GGUF builds of DeepSeek V4 Flash 0731, from lossless BF16 to 1-bit. They recommend AD-IQ2_M for 128GB hardware, which matches the original's token choice 83.6% of the time.
View Cached Full Text
Cached at: 08/04/26, 08:04 AM
atomic[.]chat just released 14 compressed quantized builds of DeepSeek V4 Flash 0731.
From lossless BF16 to 1-bit, GGUF versions for local inference runtimes.
This release measures KL divergence against the uncompressed weights instead, which asks how far the whole probability distribution drifts at every token.
The verdict is that everything above 3 bits is close to the original, and everything below falls apart fast.
The option it recommends for 128GB hardware is AD-IQ2_M. It matches the original’s token choice 83.6% of the time, measured against all other V4 Flash GGUFs in the community.
@atomic_chat_hq is a desktop app that runs LLMs locally.
atomic.chat (@atomic_chat_hq): Run DeepSeek V4 Flash 0731 locally 🐳
We released 14 quants on Hugging Face, from lossless BF16 to 1-bit
AD-IQ2_M is the best fit for 128GB hardware. It matches the original’s token choice 83.6% of the time, measured against all other V4 Flash GGUFs in the community
Similar Articles
DeepSeek-v4-Flash-Mini 54GB GGUF running at ~20.5 t/s
A community build crushes DeepSeek-V4-Flash down to a 54GB IQ2_XXS GGUF variant with aggressive 2-bit quantization, achieving ~20.5 tokens/s on local hardware while drastically reducing memory footprint.
AtomicChat/Qwen3.8-Flash-Next-GGUF is Really Good
The AtomicChat quantization of Qwen3.8-Flash-Next-GGUF significantly reduces memory usage from 106GB to 65GB while maintaining good inference performance, making it more efficient for hardware-limited setups.
@no_stp_on_snek: Delayed but here it is. DeepSeek-V4-Flash-0731 in GGUF at 2.88 bits per weight, 95 GiB, fits and runs on one 128 GB box…
DeepSeek-V4-Flash-0731 is released in GGUF format at 2.88 bits per weight (95 GiB), making the 284B-parameter MoE runnable on 128 GB machines. Includes a detailed operating guide and a required TurboQuant llama.cpp fork.
Deepseek V4 Flash 2, 3 and 4 bits GGUFs
GGUF quantizations of DeepSeek V4 Flash in 2-bit, 3-bit, and 4-bit precisions, made available on Hugging Face for local inference with tools like llama.cpp and Ollama.
Antirez Deepseek 4.1 flash gguf on HF
Antirez has uploaded the quantized gguf version of Deepseek 4.1 flash model to Hugging Face, with partial availability and questions on usage.