https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX… Fyi. Fits on 128GB of ram for metal. GGUF is coming just really slow upload speeds right now. Note I haven’t put too many testing cycles on mlx as I have gguf today. Let me know how it goes.

thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX · Hugging Face

Source: https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX

https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX#deepseek-v4-flash–config-i-mlxDeepSeek-V4-Flash — Config-I (MLX)

284B-parameter MoE (21B active) in 101 GiB — built to fit and run on 128 GB Apple Silicon.Config-I mixed-precision quantization at3.05 bpw effective, standard MLX format.

Config-I quantization ofdeepseek-ai/DeepSeek-V4-Flash-0731— the official 2026-07-31 release, superseding the preview — (43 layers, 256 routed experts top-6, hash routing on the first 3 layers, MLA attention, mHC hyper-connections, DSA indexer). The policy applies aggressive 2-bit compression to expert MLPs (where MoE is most tolerant), protects attention at 4-bit, and shields boundary layers, routing, and embeddings at higher precision. See theConfig-I policy writeupfor the derivation.

https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX#compressionCompression

SizeFP8/FP4 source167 GBBF16 equivalent~570 GB**Config-I (3.05 bpw)**108 GB (101 GiB)

https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX#policyPolicy

Tensor groupPrecisionExpert gate/up (routed + shared)2-bitExpert down3-bitMLA attention + indexer + compressor4-bitBoundary layers (first 2 + last 2): attention8-bitBoundary layers: experts4-bitEmbeddings + head8-bitRouter gates, norms, mHC paramsf16 Group size 64 throughout. The boundary-expert 4-bit choice (vs 8-bit in earlier Config-I releases) is what brings the artifact under the 128 GB unified-memory ceiling with headroom for KV cache and activations.

https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX#status-and-testingStatus and testing

**Note:**this MLX build has had lighter testing than theGGUF sibling. If you want the more thoroughly measured artifact (perplexity, backend traps, behavioral notes), start there.

📖**How to run it well:**theofflabel operating guidecovers sampling, failure modes, and the fixes (GGUF-focused, but the sampling and model-behavior notes carry over).

• **Verified:**loads and generates coherently via python mlx-lm (greedy smoke: correctly completes “The opening line of Moby Dick is” and continues cleanly). The model class and quant config load without changes.
• **MLX-specific quality numbers are pending.**This artifact is a distinct quantization from the GGUF sibling (3.05 bpw affine here vs 2.88 bpw TQ hybrid there), so the sibling’s perplexity does not transfer and is not quoted here. A full MLX-side PPL / MMLU / NIAH pass is queued; numbers land here when done.
• Sibling GGUF(for CUDA / DGX Spark / non-MLX):DeepSeek-V4-Flash-ConfigI-GGUF, which carries measured perplexity, backend notes, and observed behaviors. Note: the fast DSv4 engine on Apple Silicon (DwarfStar / antirez ds4, ~5x decode) runs the GGUF/ds4 format, not MLX; if raw speed matters more than the MLX toolchain, use that path.

https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX#using-it-sampling-that-mattersUsing it (sampling that matters)

• Sampling floor:DeepSeek’s official settings aretemperature 1.0, top_p 0.95(top_p 1.0 outside agentic use). Greedy decoding has a documented looping tendency on this model family; do not evaluate or run it greedy.
• **Give it a real output budget.**Reasoning is verbose by design (the vendor allows very long outputs at higher reasoning effort); a smallmax\_tokenstruncates mid-thought and looks like a failure.
• **Observed on the sibling quant (may transfer, unverified on this one):**long unattended symbolic-math derivations can fall into repetition; anti-repetition sampling fixes it there. Route hard math with supervision until MLX-side confirmation.

https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX#running-itRunning it

• mlx-swift-lm:deepseek\_v4is a registered model type — loads directly.
• python mlx-lm: requires thedeepseek\_v4model class (upstream PR pending). Until it lands, drop the includeddeepseek\_v4\.pyintomlx\_lm/models/.

Peak memory during generation: ~109 GB. On a 128 GB Mac, close other memory-heavy apps; raisingiogpu\.wired\_limit\_mbhelps sustained throughput.

https://huggingface.co/thetom-ai/DeepSeek-V4-Flash-ConfigI-MLX#licenseLicense

Inherits the DeepSeek model license from the base model. Quantization bythetom-ai.