@LinusEkenstam: Quite a big deal. 1.35x faster inference than MLX-LM 1.23x faster at prefill having the hybrid option to pick from insi…
Summary
Perplexity has open-sourced Lily, a local inference engine optimized for Qwen3.6-35B-A3B on Apple Silicon, achieving 1.35x faster inference than MLX-LM.
View Cached Full Text
Cached at: 09/03/26, 12:09 PM
Quite a big deal.
1.35x faster inference than MLX-LM 1.23x faster at prefill
having the hybrid option to pick from inside perplexity is awesome. seeing Lily get open sourced is even better.
Anyone can leverage this, and OS can push this even further.
Perplexity (@perplexity_ai): Today we’re open-sourcing Lily, the local inference engine we built for hybrid compute in Perplexity Computer.
Lily is specialized for Qwen3.6-35B-A3B on Apple silicon, built so on-device compute doesn’t bottleneck Computer tasks.
Read more:
Similar Articles
Optimizing On-Device Inference for Apple Silicon (20 minute read)
Apple's Lily engine optimizes on-device LLM inference for Apple silicon by leveraging unified memory and hardware, outperforming MLX-LM, and is tuned for the Qwen3.6-35B-A3B model's architecture.
Qwen3.8-27B is now up to ~3× faster on Apple Silicon with mlx-dspark
mlx-dspark v0.10.0 adds support for Qwen3.8-27B on Apple Silicon, providing up to 3x faster inference through speculative decoding with lossless verification.
Perplexity open-sourced their Mac inference server for Qwen 3.6
Perplexity has open-sourced a Mac inference server optimized for the Qwen 3.6 model to achieve best performance on Apple Silicon.
~ 2x Speed Boost for Qwen3.8 27B on Apple Silicon
MTPLX framework achieves a 2x speed boost for Qwen3.8 27B and 1.5x for Qwen3.6 35B models on Apple Silicon, featuring auto-tuning and base conversion for MLX models.
@akshay_pachaar: This inference engine runs LLMs 4x faster than llama.cpp: (Qwen3.5-9B at 92 tokens/s on 16GB Mac) I ran Qwen3.5 9B on a…
Benchmark thread showing Uzu, an open-source inference engine from Mirai, running Qwen3.5-9B at 92.1 tokens/s on a 16GB M5 MacBook Pro — about 4x faster than llama.cpp (22 tok/s) and MLX (25.1 tok/s) — by using tree-based speculative decoding with a Weaver drafter to break the memory-bandwidth ceiling of ordinary local decoding.