Tag
A user modified a HomePod Mini to connect it to a local AI model GLM 5.3 Flash using a Spark cluster and oMLX for fluent AI conversations, mentioning that major AI companies wouldn't support this project.
Benchmarks on a MacBook Pro M5 Max show that disabling thinking mode in Qwen3.8-27B severely degrades output quality, while xhigh thinking mode uses 5.5x more tokens and runs 6x longer.
The author used Codex to optimize DeepSeek V4 Flash 8-bit MLX on oMLX, achieving approximately 1.6x prefill and 3x decode speedup.
The author uses the Qwen3.6-35B-A3B model and oMLX tool on the new local machine for daily tasks, finding that both speed and quality far exceed expectations, even outperforming remote LLMs in PA and coding scenarios, demonstrating a significant improvement in on-device AI capabilities.
A blog post comparing MLX inference engines, concluding oMLX as the top choice, with benchmarks on M5 Max 64GB using Qwen3.6-35B-A3B-4bit.