@svpino: DeepSeek-V4-Flash running at 5.71 token/s on a Mac M5 Pro. Every day, we get better models running on consumer hardware…
Summary
The tweet highlights DeepSeek-V4-Flash running at 5.71 tokens per second on a Mac M5 Pro, emphasizing advancements in local AI inference on consumer hardware, with a mention of Tencent's open-source Palm-Infra for Apple Silicon optimization.
View Cached Full Text
Cached at: 08/25/26, 02:02 AM
DeepSeek-V4-Flash running at 5.71 token/s on a Mac M5 Pro.
Every day, we get better models running on consumer hardware.
And we aren’t talking about toy models anymore: we can already run a ton of intelligence without paying for an external GPU.
Tencent Cloud (@tencentcloud): From dense models to 122B and 295B MoE, local inference should work with the hardware you have—not require a datacenter. Palm-Infra is open source. Built by Tencent’s #PalmAI team, it brings Metal, CPU SIMD and SSD expert offload to Apple Silicon.
Similar Articles
DeepSeek-V4-Flash 284B on 5.3GB of memory
A developer showcases Mference, a new inference engine that runs MoE models like DeepSeek-V4-Flash on just ~5.3GB of memory by streaming experts from SSD, with a native Mac app and OpenAI-compatible server.
You can run Deepseek 4 flash on mac (M3 Max, 96gb)
A guide on running DeepSeek 4 flash on a Mac M3 Max with 96GB RAM using Antirez's ds4 engine and SSD streaming, achieving ~12 tokens/second inference speed.
@Saboo_Shubham_: OPEN SOURCE AI is killing it. DeepSeek v4 Flash is a quasi-frontier model with a massive 1M context window. It can LOCA…
The article highlights DeepSeek v4 Flash as a quasi-frontier open-source model with a 1M context window, noting its ability to run locally on a 128GB Mac using 2-bit quantization.
DeepSeek v4 Flash 0731 4bit ~50tps prefill, ~1tps decode on M5 Air 32gb
A user shares experiments running a 4-bit quantized DeepSeek v4 Flash on a 32GB M5 MacBook Air, achieving roughly 50 tokens/s prefill and 1 token/s decode using streamed experts and other tricks.
@danveloper: https://x.com/danveloper/status/2064387956387758206
A developer ran DeepSeek-V4-Flash on a Raspberry Pi 5 by streaming model weights from an NVMe SSD, achieving 1.3 tokens/second at 8 watts, demonstrating the feasibility of frontier-adjacent open-weight models on low-cost, offline hardware.