I got Nemotron Puzzle 75B running smoothly on a 64GB M2 Max
Summary
Successfully ran the 75B Nemotron Puzzle model locally on a 64GB M2 Max Mac, demonstrating large model inference on consumer hardware.
Similar Articles
@NVIDIAAI: You're welcome
NVIDIA AI releases a 75B MoE model (9.3B active) compressed from Nemotron-3-Super-120B using the Iterative Puzzle framework, with 1M token context support.
Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4) on M5 Max 2026 on Kapitsa with KFlash [P]
Kapitsa 0.2.26, an open-source Apple Silicon inference engine, publishes M5 Max benchmarks for Qwen, Nemotron, Flint and Clover models, featuring its KFlash best-first tree verification and SSD offloading that runs a 180B MoE at 61 tok/s on a 48GB MacBook Pro.
OMG! If you have a Mac with 64GB, try Qwen3.8-Flash-Next-oQ4e-mtp with oMLX!
用户惊叹于在 M3 Max 64GB Mac 上通过 oMLX 成功运行 Qwen3.8-Flash-Next-oQ4e-mtp 这个约 100GB 的模型，58GB 分配给 GPU，并支持最高 130k 上下文窗口，展示了本地大模型推理性能的显著提升。
Running local models on an M4 with 24GB memory
A guide on running local AI models like Qwen 3.5-9B on an M4 MacBook with 24GB RAM using tools like LM Studio, Ollama, and pi, including specific configuration tips for optimal performance.
2x 512gb ram M3 Ultra mac studios
A user shares their $25k hardware setup of two 512GB RAM M3 Ultra Mac Studios for running large language models locally, having tested DeepSeek V3 Q8 and GLM 5.1 Q4 via the exo distributed inference backend, while awaiting Kimi 2.6 MLX optimization.