I run 35B–480B coding models on my 36 GB MacBook by streaming MoE experts from SSD — self-contained app, and I publish the benchmarks that *failed* too

Reddit r/LocalLLaMA Tools
moe mac ssd-streaming llama-cpp coding-models local-ai benchmark

Summary

Slipstream streams MoE expert weights from SSD instead of RAM, enabling large coding models (35B–480B) on 36 GB MacBooks. Benchmarks show ~13–19 tok/s for 35B models and ~2.8 tok/s for 118B, with honest reporting of failed approaches.

I got tired of "your Mac can't run that" so I forked llama.cpp to stream a MoE model's expert weights from SSD instead of forcing the whole thing into RAM. A MoE only fires a few experts per token, so most weights sit idle — Slipstream keeps the always-needed weights resident and streams the routed experts on demand, into a bounded RAM cache that refuses to load anything that would swap your Mac. It ships as a native macOS app with the engine bundled inside — download the .dmg, drag to Applications, open. Nothing to compile. Point Kilo/Cline/Cursor/OpenCode at localhost:8080. (Not notarized yet, so the first launch is right-click → Open.) The evidence-first part, because this sub deserves real numbers: - Qwen3.6-35B-A3B (Q4), streamed from internal NVMe: ~13 tok/s at a 10 GiB cache (78% hit), ~19 tok/s at 14 GiB. The 35B is the interactive workhorse. - Laguna 118B-A8B, which does NOT fit in 36 GB: ~2.8 tok/s. Usable for batch coding, not chat. - The single biggest lever wasn't a clever kernel — I moved ONE file (the streamed experts) onto the faster disk and got 2.7×. Storage placement > everything. - A zero-copy path I'd wrongly written off at a tiny cache turned out to be +13–24% at real cache sizes — now on by default. And the ones that DIDN'T work, because that's the honest part: - Dual-SSD striping: negative on internal-NVMe + slow-USB (shared bus). - Speculative prefetch predictors (static + online): −8%. Prediction can't beat the read cost. - HOT-expert reservation: −1 to −6% (it shrinks the general cache). Full write-up incl. the negatives: BENCHMARKS.md. Repo + self-contained .dmg in the link. It's MIT, runs 100% on-device. Inspired by JustVugg/colibri (which does this for CPU/CUDA); I adapted it for Apple Silicon + Metal. Happy to answer benchmark questions — and if someone with two fast NVMe drives wants to test the dual-SSD path (it should win there), I'd love the numbers.
Original Article

Similar Articles

Is anyone getting real coding work done with Qwen3.6-35B-A3B-UD-Q4_K_M on a 32GB Mac in opencode, claude code or similar?

Reddit r/LocalLLaMA

A user shares their experience running Qwen3-35B-A3B quantized model on an M2 MacBook Pro with 32GB RAM for coding tasks via opencode and llama.cpp, finding that the 32K context window limit causes critical memory loss during compaction, making complex coding tasks impractical. They conclude that meaningful agentic coding with this model likely requires at least 128K context, exceeding what their hardware can support.

I benchmarked 21 local LLMs on a MacBook Air M5 for code quality AND speed

Reddit r/LocalLLaMA

A developer benchmarked 21 local LLMs on MacBook Air M5 using HumanEval+ and found Qwen 3.6 35B-A3B (MoE) leads at 89.6% with 16.9 tok/s, while Qwen 2.5 Coder 7B offers the best RAM-to-performance ratio at 84.2% in 4.5 GB. Notably, Gemma 4 models significantly underperformed expectations (31.1% for 31B), possibly due to Q4_K_M quantization effects.