mlx

Nativ is an open-source macOS app that runs Qwen 3.5 9B and other open models locally on Apple Silicon, offering customizable system prompts, telemetry, and integrations with coding agents, with no cloud or subscription required.

The author wrote a pure-MLX runtime for the vision and audio towers of Nvidia's Nemotron Omni model, enabling the full multimodal model to run locally on Apple Silicon Macs. All 23 tests pass against the PyTorch reference, and it achieves 67-152 tok/s on an M5 Max.

A developer updates MLX-Serve, a fast local inference server for Apple Silicon, to support recent models like LiquidAI 2.6B, MiniMax H3 video generation, and DeepSeek V4 Flash, with AntLing 3.0-flash coming soon.

Simon Willison highlights PipeNetwork/minimax-h3-mlx, a Python package that ports MiniMax-H3 to MLX for Apple Silicon, and demonstrates running it on an M5 Max MacBook Pro to generate a video from a text prompt.

TheTom releases an MLX quantized version of DeepSeek V4 Flash (284B MoE, 21B active) at 3.05 bpw, fitting in 101 GiB to run on 128GB Apple Silicon, with GGUF sibling also available.

Qwen Scribe is an open-source tool for private, on-device transcription and dictation on Apple Silicon Macs, using Qwen3-ASR models via MLX. It supports drag-and-drop audio/video transcoding, language detection, SRT export, and system-wide dictation with a HUD.

Eigen Labs launches an open autoresearch competition called MLX.fast to optimize inference speed of the Laguna XS 2.1 model on consumer Macs, aiming to make it as fast as possible via community contributions.

Rapid-MLX 0.11.0 brings major performance gains with prefix-cache and response caching, supports new model families including HY3 295B MoE and Qwen3-Coder-Next 80B, introduces structured output with guaranteed valid tool calls, and adds seamless integration with MCP servers for autonomous agent workflows.

Apple M5 silicon supports INT8 activations for matrix multiplication, but inference backends like MLX and Llama.cpp currently use 16-bit; custom w8a8 kernels achieve up to 1.4x speedup on Gemma4 prefill tasks.

oMLX 0.5.2 release adds live menu bar activity, a reorganized Models menu, Bonsai low-bit kernels, and improved performance with custom Metal kernels and native speculative decoding, making it the fastest way to run MLX models on Mac.

Nativ is a new macOS desktop app that wraps MLX to run AI models locally, offering a chat interface and API server.

A curated list of recommended AI models (e.g., Qwen3.5, Gemma 4, Mistral Medium) for systems with up to 256GB unified RAM, covering creative, coding, agentic, and general chat use cases.

Nativ is a free, open-source macOS app that lets you run frontier open AI models locally on Apple Silicon, with no accounts or subscriptions.

Nativ is a free, open-source macOS app that lets users run frontier open models locally on Apple Silicon, with no accounts, subscriptions, or cloud dependency.

The author describes implementing TurboQuant KV-cache compression into Apple's mlx-swift-lm, achieving 2.7x compression with quality on par with 8-bit, and 3-4x decode speed improvements via a fused Metal kernel.

Fixed three bugs in a qMLX fork for running Qwen3.5-122B on Mac Studio, reducing prefill time from minutes to sub-seconds for long-context inference; open-sourced the fork and benchmark script.

Jun Song announces the final development stage of a new MLX engine, achieving 41.8 tok/s on a MacBook with a 256k context window and only ~4% quality loss, representing a significant performance improvement.

mlx-dspark brings DeepSeek's DSpark and z-lab's DFlash speculative decoding drafters to Apple Silicon via MLX, enabling lossless speedup (~1.4–1.6×, up to 2× on code/math) and an OpenAI-compatible API for local inference.

MTPLX V2 is released, claiming 72+ tokens per second on Qwen 3.6 27B running on a Macbook Pro M5 Max via MLX.

mlx-vlm v0.6.4 is released with support for 5 new model families, TTS/STT endpoints, and significant performance improvements including TurboQuant and continuous batching.