@rohanpaul_ai: atomic[.]chat just released 14 compressed quantized builds of DeepSeek V4 Flash 0731. From lossless BF16 to 1-bit, GGUF…

X AI KOLs Timeline Models
quantization gguf deepseek-v4 local-inference kl-divergence

Summary

atomic.chat released 14 quantized GGUF builds of DeepSeek V4 Flash 0731, from lossless BF16 to 1-bit. They recommend AD-IQ2_M for 128GB hardware, which matches the original's token choice 83.6% of the time.

atomic[.]chat just released 14 compressed quantized builds of DeepSeek V4 Flash 0731. From lossless BF16 to 1-bit, GGUF versions for local inference runtimes. This release measures KL divergence against the uncompressed weights instead, which asks how far the whole probability distribution drifts at every token. The verdict is that everything above 3 bits is close to the original, and everything below falls apart fast. The option it recommends for 128GB hardware is AD-IQ2_M. It matches the original's token choice 83.6% of the time, measured against all other V4 Flash GGUFs in the community. @atomic_chat_hq is a desktop app that runs LLMs locally.
Original Article
View Cached Full Text

Cached at: 08/04/26, 08:04 AM

atomic[.]chat just released 14 compressed quantized builds of DeepSeek V4 Flash 0731.

From lossless BF16 to 1-bit, GGUF versions for local inference runtimes.

This release measures KL divergence against the uncompressed weights instead, which asks how far the whole probability distribution drifts at every token.

The verdict is that everything above 3 bits is close to the original, and everything below falls apart fast.

The option it recommends for 128GB hardware is AD-IQ2_M. It matches the original’s token choice 83.6% of the time, measured against all other V4 Flash GGUFs in the community.

@atomic_chat_hq is a desktop app that runs LLMs locally.

atomic.chat (@atomic_chat_hq): Run DeepSeek V4 Flash 0731 locally 🐳

We released 14 quants on Hugging Face, from lossless BF16 to 1-bit

AD-IQ2_M is the best fit for 128GB hardware. It matches the original’s token choice 83.6% of the time, measured against all other V4 Flash GGUFs in the community

Similar Articles

DeepSeek-v4-Flash-Mini 54GB GGUF running at ~20.5 t/s

Reddit r/LocalLLaMA

A community build crushes DeepSeek-V4-Flash down to a 54GB IQ2_XXS GGUF variant with aggressive 2-bit quantization, achieving ~20.5 tokens/s on local hardware while drastically reducing memory footprint.

Deepseek V4 Flash 2, 3 and 4 bits GGUFs

Reddit r/LocalLLaMA

GGUF quantizations of DeepSeek V4 Flash in 2-bit, 3-bit, and 4-bit precisions, made available on Hugging Face for local inference with tools like llama.cpp and Ollama.