Tag
Kapitsa 0.2.26, an open-source Apple Silicon inference engine, publishes M5 Max benchmarks for Qwen, Nemotron, Flint and Clover models, featuring its KFlash best-first tree verification and SSD offloading that runs a 180B MoE at 61 tok/s on a 48GB MacBook Pro.
PyTorch 2.15 brings major MPS backend improvements for Mac users, including FP8 support, 5-6.5× faster MoE model decoding, 8× faster softmax, and dramatic memory reductions for video VAE decoding, driven by Hugging Face's PyTorch team and community contributors.
Nace.ai 发布开源（Apache 2.0）Apple Silicon 优化 Metal 推理引擎 Kapitsa，专为 Qwen 和 Nemotron 模型定制内核，并在 M1 Pro、M4 Pro、M5 Max 上公布对比 MLX、llama.cpp、Ollama 等引擎的解码基准。
This work integrates ggml's Metal kernels into the transformers library, allowing GGUF models to run directly with improved compatibility and performance for fast local inference on Mac.
oMLX 0.5.2 release adds live menu bar activity, a reorganized Models menu, Bonsai low-bit kernels, and improved performance with custom Metal kernels and native speculative decoding, making it the fastest way to run MLX models on Mac.
在自定义Rust内核和Swift调度逻辑的优化下，Qwen3.6-35b-A3B模型的预填速度在2k提示下从255 tok/s提升到1058 tok/s，实现了约4倍的加速，解码和困惑度未受影响。
Solo researcher released Spiral, a tool that compresses LLMs to INT3 and KV-cache to INT2 with custom fused Metal kernels for Apple Silicon, currently shipping Qwen-7B preview.
DFlash v0.1.4 releases custom Metal verify kernels for quantized Qwen3 hybrid models with significant peak memory reduction and 2.2x throughput improvements at long context on M5 Max GPUs.