Tag
A hands-on benchmark comparing four local LLM inference engines (oMLX 0.7.0, Rapid-MLX 0.15.5, Splash 1.2.1, MTPLX 2.12.2) on a 36 GB M3 Max MacBook Pro, finding Rapid-MLX fastest on Qwen3.6-35B-A3B (~110 tok/s) and Splash fastest on dense Qwen3.8-27B (~31.7 tok/s via speculative decoding).
A user shares how upgrading from an M2 Pro MacBook to a 128GB Mac Studio lets them run around a dozen Claude coding sessions concurrently, noting the machine is a worthwhile buy for heavy AI workloads and that they won't return to a MacBook Pro for heavy lifting.
A developer stripped antirez's ds4 (DwarfStar) inference engine down to a Metal-only Qwen3.8 Flash Next build, cutting the codebase from 85k to 45k lines and yielding ~10% faster decode speeds with bit-exact output, plus a reusable tooling repo called StarForge for maintaining such forks.
A demo video shows the Qwen3.8-Flash-Next-MLX-oQ8-MTP model running on an Apple M3 Ultra via tensorfold at ~80 tokens/s, with the author recommending the highest possible quantization for better quality and fewer errors.
TensorFold 0.6.2 is an Apache 2.0 open-source LLM inference engine claiming over 2x faster token throughput than MLX on Apple Silicon (61 vs 27 tok/s on an M3 Ultra) via multi-token-prediction speculative decoding, with parallel concurrent requests and OpenAI-compatible API.
A hobbyist built a llama.cpp fork that streams model layers from a 24 GB MacBook to an iPhone 17 Pro Max over USB-C, boosting Qwen 3.8 27B prefill throughput by 29-44% and offloading up to ~5.7 GB of KV cache to the phone's GPU and Neural Engine for very long contexts. Code and bench scripts are published on GitHub.
Blenny is a free, open-source menu bar organizer for Apple silicon Macs running macOS 27, letting users keep, hide, and reorder menu bar apps with draft/apply/undo support and no pointer movement or Screen Recording permission. It is Apache-2.0 licensed and available via Product Hunt.
A detailed technical writeup on bringing up mainline Linux on the M4 Mac mini, overcoming the newly mandatory SPTM (Secure Page Table Monitor) that broke existing m1n1 hypervisor-based workflows, and getting the SoC booting far enough for serial console debugging. The author credits the Asahi Linux team and documents locked GXF/RVBAR registers and early Linux boot progress.
作者发布 Slipstream——一款针对 Apple Silicon 的 C++ Metal 推理引擎，支持原生 SSD 专家流式加载与投机草稿解码，可在 64GB Mac 上以 41–52 tok/s 运行 95.5 GiB 的 Qwen3.8-Flash-Next 模型，比 llama.cpp 快 1.76 倍，且 130k 长上下文下速度不衰减。
A user shares early impressions of Dwarfstar, a tool offering bespoke quantization techniques for running large language models locally, reporting fast performance for Qwen3-8B on an M3 Ultra 96GB Mac Studio with leftover memory headroom.
A community poll on which MLX engines people use to run models on Apple silicon is showing its top 15 results so far, with voting still open to help reach 1000 votes.
A user reports that the open-source TensorFold inference engine, running a 4-bit Qwen3.8-27B with a DFlash2 drafting model, reaches 40-60 tok/s on an M5 Pro Mac mini, beating MTPLX and rivaling an RTX 3090 Ti. This highlights rapid performance gains for local LLM inference on Apple Silicon.
NoteWorthy is a new AI-powered notes app launched on Product Hunt that runs entirely on-device across iPhone, iPad and Mac, auto-categorizing notes and generating summaries without any data leaving the device, no account, and no subscription.
A developer created a library called apple-llm to make Apple's built-in local LLM on macOS easily usable from Node and Python, focusing on privacy and specific tasks like structured data extraction and summarization.
MoEspresso 3, an inference engine, achieves 12-15 decode tokens per second for the Qwen3.8-Flash-Next model on a 2021 M1 Max with 32GB RAM, using techniques like Cache-Prior to optimize performance and outperforming some hosted solutions in benchmarks.
Splash 1.1.0 has been released with GGUF quantization support, MLX import, and optimized kernels for Apple Silicon, enabling efficient local AI inference at high speeds.
This article details the porting of the Kev-0.8B model, a Qwen3.5 variant, to Core ML, demonstrating substantial improvements in inference speed and memory efficiency on Apple Silicon devices through benchmarks.
The article describes how to use the 'make tune-kernels' tool in Splash to optimize AI model decoding on 40-core M5 Max chips, achieving up to 20% speed improvement by tuning kernel layouts for specific hardware.
WattMate is a Mac menu bar app that shows which app is currently draining your battery in watts and estimates the minutes saved if you quit it, then measures the actual change after quitting. It requires Apple silicon and macOS 14+.
A tweet criticizes the optimizations in MLX, suggesting Apple has a software problem rather than a hardware issue, based on performance tests with M5 Ultra Macs.