@Raullen: Rapid-MLX 0.11.0 is out! Making local models on Apple Silicon reliable enough to run your agent workflows, not just dem…

X AI KOLs Following Tools
rapid-mlx apple-silicon local-models performance prefix-cache structured-output tool-calls mlx open-source

Summary

Rapid-MLX 0.11.0 brings major performance gains with prefix-cache and response caching, supports new model families including HY3 295B MoE and Qwen3-Coder-Next 80B, introduces structured output with guaranteed valid tool calls, and adds seamless integration with MCP servers for autonomous agent workflows.

Rapid-MLX 0.11.0 is out! Making local models on Apple Silicon reliable enough to run your agent workflows, not just demo them. Performance • Prefix-cache reuse: 13.1s → 0.51s TTFT (25.6x faster on a repeated 6k-token prefix). Shared system prompts stop paying prefill twice. One session saved 38,032 tokens. • Response cache: 656ms → 2ms (284x faster, byte-identical, zero GPU decode). • Throughput: 152 tok/s on Qwen3.5-4B-4bit. Cold starts in seconds. "rapid-mlx chat" grew into a real agent Point it at any MCP server and it chains tools autonomously (e.g., filesystem server → 14 tools → list_directory → answer), complete with a live tool-activity UI. Thinking budgets force-close <think> at decode time to keep things snappy. 5 New Model Families • HY3 295B MoE (native MTP speculative decoding), Qwen3-Coder-Next 80B, MiniCPM5, LFM2.x, and a 2-bit Ternary Bonsai that actually runs. • Kokoro TTS → Parakeet STT round-trip verified word-perfect. • Qwen3-VL vision, KV cache export/import, and structured output on llguidance. Schema-valid tool calls by construction. If you run coding agents on local models, you know the pain of "almost-JSON" arguments killing your parser. 0.11.0 constrains decoding itself—the model cannot emit a malformed tool call. • Verified across harmony/gpt-oss, qwen3_coder_xml, gemma4, deepseek_v3, plus hermes. • Auto, forced, and required. On by default, zero flags. • The Keystone: It now holds for reasoning models too! <think> tags no longer break forced calls because of grammar offset issues. Reliability & Honest Gaps Every release runs an automated dogfood battery (5 models × 11 checks on real hardware). Invalid schema → honest 400. Unknown model → 404 with available options. No silent fallbacks. Gap: Qwen3.6 native MTP is parked for now, and a same-machine Ollama head-to-head is up next. Now in homebrew-core brew install rapid-mlx rapid-mlx chat qwen3.5-4b-4bit Apache-2.0, MLX-native. 3.4k stars — break it and tell us, issues get answered fast.
Original Article
View Cached Full Text

Cached at: 07/25/26, 08:11 PM

Rapid-MLX 0.11.0 is out! Making local models on Apple Silicon reliable enough to run your agent workflows, not just demo them.

Performance • Prefix-cache reuse: 13.1s → 0.51s TTFT (25.6x faster on a repeated 6k-token prefix). Shared system prompts stop paying prefill twice. One session saved 38,032 tokens. • Response cache: 656ms → 2ms (284x faster, byte-identical, zero GPU decode). • Throughput: 152 tok/s on Qwen3.5-4B-4bit. Cold starts in seconds.

“rapid-mlx chat” grew into a real agent Point it at any MCP server and it chains tools autonomously (e.g., filesystem server → 14 tools → list_directory → answer), complete with a live tool-activity UI. Thinking budgets force-close at decode time to keep things snappy.

5 New Model Families • HY3 295B MoE (native MTP speculative decoding), Qwen3-Coder-Next 80B, MiniCPM5, LFM2.x, and a 2-bit Ternary Bonsai that actually runs. • Kokoro TTS → Parakeet STT round-trip verified word-perfect. • Qwen3-VL vision, KV cache export/import, and structured output on llguidance.

Schema-valid tool calls by construction. If you run coding agents on local models, you know the pain of “almost-JSON” arguments killing your parser. 0.11.0 constrains decoding itself—the model cannot emit a malformed tool call. • Verified across harmony/gpt-oss, qwen3_coder_xml, gemma4, deepseek_v3, plus hermes. • Auto, forced, and required. On by default, zero flags. • The Keystone: It now holds for reasoning models too! tags no longer break forced calls because of grammar offset issues.

Reliability & Honest Gaps Every release runs an automated dogfood battery (5 models × 11 checks on real hardware). Invalid schema → honest 400. Unknown model → 404 with available options. No silent fallbacks. Gap: Qwen3.6 native MTP is parked for now, and a same-machine Ollama head-to-head is up next.

Now in homebrew-core brew install rapid-mlx rapid-mlx chat qwen3.5-4b-4bit

Apache-2.0, MLX-native. 3.4k stars — break it and tell us, issues get answered fast.

Similar Articles

@cevenif: For those running local LLMs on Macs, here's a tool worth watching — Rapid-MLX. It delivers 2-4x faster inference on M-series chips than Ollama, thanks to being built directly on Apple's MLX framework for more thorough utilization of the chip architecture. Key highlights: KV cache pruning plus…

X AI KOLs Timeline

Rapid-MLX is a local LLM inference tool optimized for Apple M-series chips. Built on the MLX framework, it achieves 2 to 4 times faster inference than Ollama, supports multiple models, tool calling, and an OpenAI API-compatible interface.