Bartowski has delivered DS4 GGUF
Summary
Bartowski has released a GGUF quantized version of DeepSeek-V4-Flash, inviting comparison with Antirez's version.
Similar Articles
antirez/deepseek-v4-gguf
Antirez released GGUF quantizations of DeepSeek V4 Flash specifically tailored for the DS4 inference engine, providing optimized configurations for different RAM sizes and enabling local execution of the large MoE model.
Deepseek V4 Flash 2, 3 and 4 bits GGUFs
GGUF quantizations of DeepSeek V4 Flash in 2-bit, 3-bit, and 4-bit precisions, made available on Hugging Face for local inference with tools like llama.cpp and Ollama.
@Snixtp: DeepSeek V4 Flash on a single RTX Pro 6000?
DeepSeek V4 Flash GGUF quantizations have been released by antirez, enabling the model to run on single GPUs like the RTX Pro 6000 and Macs with 128GB+ RAM. The quantized files are available on Hugging Face with instructions for the DS4 inference engine.
A few words on DS4
Antirez announces DwarfStar 4 (DS4), a local AI tool that runs DeepSeek v4 Flash with asymmetric 2/8 bit quantization on high-end consumer hardware, achieving near-frontier performance. He discusses the project's rapid popularity, future plans for model updates and distributed inference, and the significance of local AI for serious tasks.
Qwen 3.6 27B AutoRound GGUF, need your feedback
A user shares their GGUF quantized version of Qwen 3.6 27B using AutoRound, claiming it performs better than other quants, and invites feedback.