INT3 compression+fused metal kernels [R]
Summary
Solo researcher released Spiral, a tool that compresses LLMs to INT3 and KV-cache to INT2 with custom fused Metal kernels for Apple Silicon, currently shipping Qwen-7B preview.
Similar Articles
I ported EXL3 to run well on Apple Silicon - PonyExl3
Ported the EXL3 LLM codec to run on Apple Silicon via Metal, achieving high prefill and generation speeds on M5 Max (e.g., ~600 tok/s prefill, 17-80 tok/s gen on various models).
Metal-Sci: A Scientific Compute Benchmark for Evolutionary LLM Kernel Search on Apple Silicon
Metal-Sci introduces a 10-task benchmark for optimizing scientific computing kernels on Apple Silicon, paired with an evolutionary search framework driven by large language models. The study evaluates models like Claude Opus 4.7, Gemini 3.1 Pro, and GPT 5.5, demonstrating significant speedups while using out-of-distribution testing to catch silent performance regressions.
Qwen3.6-35B-A3B-Abliterated-Heretic-MLX-4bit
The user reviews a quantized and fine-tuned version of the Qwen3.6-35B model optimized for Apple Silicon via MLX, praising its speed, intelligence, and lack of safety disclaimers.
Automatic1111 for Apple metal, 40% speed up sd1.5
The article introduces a fine-tuned Automatic1111 fork for Apple Silicon that adds Metal optimizations (e.g., Metal Flash Attention) to speed up Stable Diffusion 1.5 generation, cutting times from 8-10s to 3-7s on an M3 Pro and from 13-20s to 8-10s on an M1 Mac Mini.
Qwen3.8-27B is now up to ~3× faster on Apple Silicon with mlx-dspark
mlx-dspark v0.10.0 adds support for Qwen3.8-27B on Apple Silicon, providing up to 3x faster inference through speculative decoding with lossless verification.