@jun_song: The new engine for MLX is in its final stages of development. Just ran GLM-5.2 on a single MacBook (116GB) hitting 41.8…

X AI KOLs Following Tools
mlx engine performance macbook glm context-window

Summary

Jun Song announces the final development stage of a new MLX engine, achieving 41.8 tok/s on a MacBook with a 256k context window and only ~4% quality loss, representing a significant performance improvement.

The new engine for MLX is in its final stages of development. Just ran GLM-5.2 on a single MacBook (116GB) hitting 41.8 tok/s with a 256k context window. Quality loss is only around ~4%, which puts it right at the 3-4bit quality level. The tech behind this uses a newly introduced layered architecture. When I first started, I was getting 10 tok/s with Kimi-K2.6 (128GB, 1024 context). Now it is fully at production level. Been grinding on this for months. Feels great to see it finally coming out soon.
Original Article
View Cached Full Text

Cached at: 07/09/26, 05:39 PM

The new engine for MLX is in its final stages of development.

Just ran GLM-5.2 on a single MacBook (116GB) hitting 41.8 tok/s with a 256k context window.

Quality loss is only around ~4%, which puts it right at the 3-4bit quality level. The tech behind this uses a newly introduced layered architecture.

When I first started, I was getting 10 tok/s with Kimi-K2.6 (128GB, 1024 context). Now it is fully at production level.

Been grinding on this for months. Feels great to see it finally coming out soon.

Similar Articles

Gemma4 26b MoE running in MLX with turboquant (and custom kernel)

Reddit r/LocalLLaMA

A developer successfully ran Gemma4 26b MoE on Apple MacBook Air M5 using MLX with turboquant and a custom kernel, achieving faster prompt processing and generation speeds than llama.cpp with lower memory usage. The implementation includes instructions for local deployment.