throughput

Open-dLLM adapts Qwen3.6 to use diffusion-based generation, achieving over 3,000 tok/s on an RTX 5090 for short sequences, with code released on GitHub.

Kimi K2.6 achieves 56 tokens per second on a single MI300X GPU; user plans further throughput benchmarking.

K2.6 successfully downloaded and deployed the Qwen3.5-0.8B model locally on a Mac, using the niche Zig language to implement and optimize inference, demonstrating the new model’s generalization ability. After 4,000+ tool calls and 12+ hours of continuous operation, K2.6 iterated 14 times, boosting throughput from ~15 tokens/s to ~193 tokens/s, ultimately achieving 20% faster inference than LM Studio.

The article introduces core concepts from the book "Systems Performance" regarding latency, throughput, cache hierarchies, etc., and references latency numbers from experts like Jeff Dean, emphasizing the importance of hands-on practice for performance engineering.