I asked Codex to optimize DeepSeek V4 Flash 8-bit MLX on oMLX. Got ~1.6x prefill and ~3x decode speedup.
Summary
The author used Codex to optimize DeepSeek V4 Flash 8-bit MLX on oMLX, achieving approximately 1.6x prefill and 3x decode speedup.
Similar Articles
How I made DeepSeek V4 Flash 12x faster on an M3 Ultra
The author achieved a 12x speedup for DeepSeek V4 Flash on a Mac Studio M3 Ultra by optimizing kernels and implementing effective caching strategies, reducing chat turn latency from 6-20 seconds to 1.6 seconds.
@malikwas1f: The four-line fix took DeepSeek-V4-Flash prefill from 127 → 312 t/s @10K and 91 → 283 @40K, with a passed recall check …
A four-line fix reportedly boosts DeepSeek-V4-Flash prefill speed from 127 to 312 t/s at 10K context and 91 to 283 t/s at 40K, with decode unchanged. The patch is shared via the club-3090 project for serving LLMs on RTX 3090s.
DeepSeek V4 Flash on a Single AMD MI300X
This repository provides configuration, patches, and tuning to run the DeepSeek V4 Flash 304B checkpoint on a single AMD MI300X in production, achieving 168 tok/s decode without quantization. It includes correctness overlays for vLLM ROCm, AITER tuning tables, and a hybrid KV cache strategy.
DeepSeek v4 Flash 0731 4bit ~50tps prefill, ~1tps decode on M5 Air 32gb
A user shares experiments running a 4-bit quantized DeepSeek v4 Flash on a 32GB M5 MacBook Air, achieving roughly 50 tokens/s prefill and 1 token/s decode using streamed experts and other tricks.
[Deepseek-V4-Flash-0731] Full 1M context on a single RTX5090 + DDR5 Desktop Setup with VLLM CPU/Ram Offloading, ~800 tps pp & 15+ tps decode [Agentic Coding]
Technical write-up on running DeepSeek-V4-Flash-0731 with full 1M context on a single RTX 5090 + 256GB DDR5 desktop using vLLM with CPU/RAM offloading, achieving ~800 tps prefill and ~15 tps decode.