Bartowski has delivered DS4 GGUF
Summary
Bartowski has released a GGUF quantized version of DeepSeek-V4-Flash, inviting comparison with Antirez's version.
Similar Articles
antirez/deepseek-v4-gguf
Antirez released GGUF quantizations of DeepSeek V4 Flash specifically tailored for the DS4 inference engine, providing optimized configurations for different RAM sizes and enabling local execution of the large MoE model.
@Snixtp: DeepSeek V4 Flash on a single RTX Pro 6000?
DeepSeek V4 Flash GGUF quantizations have been released by antirez, enabling the model to run on single GPUs like the RTX Pro 6000 and Macs with 128GB+ RAM. The quantized files are available on Hugging Face with instructions for the DS4 inference engine.
A few words on DS4
Antirez announces DwarfStar 4 (DS4), a local AI tool that runs DeepSeek v4 Flash with asymmetric 2/8 bit quantization on high-end consumer hardware, achieving near-frontier performance. He discusses the project's rapid popularity, future plans for model updates and distributed inference, and the significance of local AI for serious tasks.
Qwen 3.6 27B AutoRound GGUF, need your feedback
A user shares their GGUF quantized version of Qwen 3.6 27B using AutoRound, claiming it performs better than other quants, and invites feedback.
DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-NEO-CODE-Di-IMatrix-MAX-GGUF
DavidAU releases a custom 40B parameter model based on Qwen 3.6, expanded and fine-tuned with Claude 4.6 Opus distill and Deckard datasets, featuring optimized GGUF quantizations for improved precision and uncensored capabilities.