Tag
A user asks whether Gemma 4 E4B remains the best small multimodal model (under 8B/6GB) for running a local, tool-calling personal assistant on a Mac via MLX, noting failures with Ling 3.0 tiny, LFM models, and Qwen 3.5 4B/9B.
作者认为64GB内存不再是本地LLM的'死区'——在M4 Max上运行Qwen3.8 27B进行agentic coding，KV cache和prompt cache使64GB恰好成为sweet spot，打破了此前认为该容量尴尬的共识。
The open-source project ai-phone lets your Mac take over real phone lines directly. All speech recognition, LLM inference, and speech synthesis run locally, so call data never leaves your machine. It supports real-time interruptions and allows coding agents to orchestrate calls via CLI.
Nace.AI's Kapitsa 0.2 ships KFlash, a best-first tree-based speculative decoding method for Apple Silicon, benchmarked against Splash 1.4.0's DFlash2, mlx-serve, Ollama, llama.cpp and others on Qwen models across M4 Pro and M1 Pro MacBooks, showing top decode throughput (~65-201 tok/s).
Nace.ai 发布开源（Apache 2.0）Apple Silicon 优化 Metal 推理引擎 Kapitsa，专为 Qwen 和 Nemotron 模型定制内核，并在 M1 Pro、M4 Pro、M5 Max 上公布对比 MLX、llama.cpp、Ollama 等引擎的解码基准。
苹果发布新版 MLX 框架，展示如何在单台 Mac Studio 上运行 Muse Spark Open（近万亿参数）等前沿模型，并通过 RDMA over Thunderbolt 驱动在多台 Mac 间实现分布式张量/流水线并行推理，在四台 M5 Ultra 集群上达到每秒 36 tokens 以上的解码速度。
Awni Hannun 展示了两台 M5 Ultra 配合 MLX 所能达到的性能水平，凸显本地 Apple Silicon 推理能力的显著提升。
作者将 Ternary Bonsai 2 27B 量化模型迁移到 Apple Neural Engine，在 16GB M6 Mac mini 上实现约 52 tokens/s 的代码生成速度，整机功耗仅约 12W，显著优于 GPU 方案。项目基于 ANEMLL Forge 提供转换脚本、运行时补丁和 OpenAI 兼容接口，并已适配 Grok Build。
A detailed technical writeup on pairing an M5 Ultra 256 GB for prefill with an M3 Ultra 512 GB for decode, using disaggregated inference over Thunderbolt 5 RDMA with GLM-5.3-Flash on MLX, cutting time-to-first-token on long prompts from ~40s to ~18.5s.
作者系统整理了本地部署模型的加速方案，包括在 macOS 上结合 Turbo、MLX、VAE Q8 和前缀 KV cache 让 Qwen-Image 2.1 提速近 5 倍，以及用 Uzu 的投机解码把 Qwen3.5-9B 从 22 tok/s 提升到 92 tok/s、用 TensorRT 让 Qwen 在 RTX 5090 上比 llama.cpp 快 3 倍等案例。
strata-mlx update: interrupted long conversations no longer get reread — time to first character for a 10K-token session drops from 19.7s to 0.26s (M4 Max), and memory management on 32 GB Macs is optimized to avoid swap. The project is an Apple Silicon MLX inference engine for running 125B MoE models like Qwen3.8-Flash-Next, with throughput significantly better than llama.cpp and LM Studio.
strata-mlx 更新：在低内存 Mac 上提升推理速度但增加内存占用，新增 --check 性能预检命令，并评估了专家预测预取方案的收益（仅 1.35x 上限，未实施）。
用户惊叹于在 M3 Max 64GB Mac 上通过 oMLX 成功运行 Qwen3.8-Flash-Next-oQ4e-mtp 这个约 100GB 的模型，58GB 分配给 GPU，并支持最高 130k 上下文窗口，展示了本地大模型推理性能的显著提升。
yibie announces strata-mlx, an unofficial MLX-based inference engine for Mac that runs the 125B MoE model Qwen3.8-Flash-Next from Strata's GGUF files, achieving 1.7–2.9x faster generation than llama.cpp and enabling lower-RAM Macs to offload experts to SSD.
A community MLX quantization of UkisAI's Swift 1.5 (based on Qwen 3.8 Flash Next) at 4.7bpw, tuned for the 96GB Mac Studio with M5 Ultra, delivering strong prefill/decode speeds and 91% top-1 agreement with the BF16 model.
A hands-on benchmark comparing four local LLM inference engines (oMLX 0.7.0, Rapid-MLX 0.15.5, Splash 1.2.1, MTPLX 2.12.2) on a 36 GB M3 Max MacBook Pro, finding Rapid-MLX fastest on Qwen3.6-35B-A3B (~110 tok/s) and Splash fastest on dense Qwen3.8-27B (~31.7 tok/s via speculative decoding).
DecisionTune 1.0 is a 395M-parameter encoder (ModernBERT-large plus a scoring head) that scores options offline for small agent decisions, running in ~10 ms per decision on MLX with Apache-2.0 weights and backends for PyTorch, MLX, and ONNX.
文章详细介绍了如何基于 Apple MLX 将 Qwen3-4B 微调为输出校准数值的决策模型 Gero-4B，涵盖架构改造、结构化训练与 RL 训练三个阶段的完整代码与实践教训。
A demo video shows the Qwen3.8-Flash-Next-MLX-oQ8-MTP model running on an Apple M3 Ultra via tensorfold at ~80 tokens/s, with the author recommending the highest possible quantization for better quality and fewer errors.
A community poll on which MLX engines people use to run models on Apple silicon is showing its top 15 results so far, with voting still open to help reach 1000 votes.