jabbatheduck/DeepSeek-v4-flash-mini · Hugging Face

Source: https://huggingface.co/jabbatheduck/DeepSeek-v4-flash-mini

https://huggingface.co/jabbatheduck/DeepSeek-v4-flash-mini#deepseek-v4-flash-reap-k160–ggufDeepSeek V4 Flash REAP K160 — GGUF

Quantized GGUF of the REAP expert-pruned DeepSeek-V4-Flash-0731 checkpoint. This is an aggressively compressed variant intended for memory-constrained inference on high-end consumer and workstation GPUs.

Source model:0xSero/DeepSeek\-V4\-Flash\-0731\-REAP Quantization inspiration:antirez/deepseek\-v4\-gguf

https://huggingface.co/jabbatheduck/DeepSeek-v4-flash-mini#fileFile

FileSizeRouted expertsEverything elseDeepSeek\-V4\-Flash\-REAP\-IQ2XXS\-w2Q2K\-AProjQ8\-OutQ8\-chat\-v2\.gguf54 GiBIQ2\_XXS(gate, up) +Q2\_K(down) — 160 of 256 retainedQ8\_0attention / shared experts / output head,F16router + embed + indexer + compressor + HC,F32norms / sinks / bias

https://huggingface.co/jabbatheduck/DeepSeek-v4-flash-mini#quantization-recipeQuantization recipe

Tensor classQuantNotesblk\.\*\.ffn\_gate\_exps,blk\.\*\.ffn\_up\_exps**IQ2\_XXSrouted-expert up/gateblk\.\*\.ffn\_down\_expsQ2\_K**routed-expert down (K-quant for stability)blk\.\*\.ffn\_\{gate,up,down\}\_shexp``Q8\_0shared expertsblk\.\*\.attn\_q\_a,attn\_q\_b,attn\_kv,attn\_output\_a,attn\_output\_b``Q8\_0all attention projectionsoutput\.weight``Q8\_0output headtoken\_embd\.weight``F16input embeddingblk\.\*\.ffn\_gate\_inp``F16learned routerblk\.\*\.exp\_probs\_b,blk\.\*\.attn\_sinks, all\*\_norm\.weight``F32``blk\.\*\.ffn\_gate\_tid2eid``I32hash-routing tables (first 3 layers)blk\.\*\.attn\_compressor\_\*,blk\.\*\.indexer\_\*,blk\.\*\.hc\_\*,blk\.\*\.output\_hc\_\*``F16/F32DeepSeek-V4 auxiliary blocks **Why the asymmetry:**routed experts dominate parameter count but fire on only a fraction of tokens. Aggressive quantization on them costs less in average quality than treating router, projections, or shared experts the same way. Keeping decision-making tensors atQ8\_0preserves behavior; crushing the experts buys the size.

https://huggingface.co/jabbatheduck/DeepSeek-v4-flash-mini#base-model-contextBase-model context

This GGUF was derived from a K160 REAP checkpoint — 160 of 256 routed experts retained per MoE scope, top-6 routing preserved, router and indexer remapped to the retained expert IDs. Structural validation passed on the source checkpoint with 46/46 MoE scopes verified.

https://huggingface.co/jabbatheduck/DeepSeek-v4-flash-mini#requirementsRequirements

Use a DeepSeek-V4-capable runtime. Generictransformerspipelines will not handle the custom MXFP4 / sparse-attention / tokenizer components.

Tested with theds4inference engine:https://github.com/antirez/ds4

https://huggingface.co/jabbatheduck/DeepSeek-v4-flash-mini#licenseLicense

MIT. The base model copyright is held by DeepSeek; this GGUF is redistributed under the base model’s release terms.