Swift1.5 Qwen3.8 Flash Next - Tailored for the 96GB Mac Studio with M5 Ultra

Reddit r/LocalLLaMA Models
mlx quantization mac-studio apple-silicon local-llm qwen inference

Summary

A community MLX quantization of UkisAI's Swift 1.5 (based on Qwen 3.8 Flash Next) at 4.7bpw, tuned for the 96GB Mac Studio with M5 Ultra, delivering strong prefill/decode speeds and 91% top-1 agreement with the BF16 model.

https://huggingface.co/Dankpaws/Swift1.5-Qwen3.8-Flash-Next-MLX-4.7bpw I've had the 96GB Mac Studio with M5 Ultra for about a week now and wasn't satisfied with the results I was getting from the limited number of models available to me. It was a combination of speed, memory headroom, and/or output quality. This is my best attempt at a calibrated MLX quantization of UkisAI’s Swift 1.5. Hope those of you with the hardware enjoy it! Measurement This pack Swift llama.cpp IQ3_XXS Prefill · 25k prompt 3,191 tok/s 1,427 tok/s Prefill · 95k prompt 2,928 tok/s 1,307 tok/s Decode · after 4k prompt 113.7 tok/s 62.7 tok/s Decode · after 95k prompt 81.1 tok/s 44.7 tok/s Top-1 agreement with Swift BF16 91.0% 84.1% 91% is next-token agreement with BF16 across 680 common held-out positions, not task accuracy. Results above are simply from my own machine. mlx-serve 26.10.1. ~107GB download, text-only, 179,200-token tested context.
Original Article

Similar Articles

M5 Ultra - Qwen3.8 Flash Next vs Laguna S 2.1

Reddit r/LocalLLaMA

A same-harness benchmark on a Mac Studio M5 Ultra 256GB compares Qwen3.8-Flash-Next (182GB oQ8e) vs Laguna-S-2.1 GGUF at up to 262K context. Qwen sustains ~4,200 tok/s linear prefill and stable decode, while Laguna degrades superlinearly; quality is a 4/4 draw with very different answer styles.

Qwen3.8-Flash-Next optimised for Macs

Reddit r/LocalLLaMA

The article details custom optimizations for running the Qwen3.8-Flash-Next AI model on Mac M1 Max hardware, including SSD streaming, custom quantizations, and a sparse attention mechanism to improve performance.

Qwen3.8-Flash-Next (125B) at 12-15 tok/s on a 2021 32GB M1 Max

Reddit r/LocalLLaMA

MoEspresso 3, an inference engine, achieves 12-15 decode tokens per second for the Qwen3.8-Flash-Next model on a 2021 M1 Max with 32GB RAM, using techniques like Cache-Prior to optimize performance and outperforming some hosted solutions in benchmarks.