@HotAisle: Kimi K2.6 + DFlash: 508 tok/s on 8x MI300X 5.6x throughput improvement over baseline autoregressive serving 90 tok/s → …
Summary
Kimi K2.6 paired with DFlash inference system achieves 508 tokens/s on 8×AMD MI300X, a 5.6× throughput jump from 90 tokens/s baseline with zero quality loss.
Similar Articles
@QuixiAI: @Kimi_Moonshot K2.6 running on my mi300x, 56 tps (single request). I will run a throughput test
Kimi K2.6 achieves 56 tokens per second on a single MI300X GPU; user plans further throughput benchmarking.
@jun_song: Working on fitting Kimi-K2.6 (1T) on 128GB Mac. Trying to get 40tok/s, and minimize the quality loss.
A developer is optimizing the Kimi-K2.6 (1T) model to run efficiently on a 128GB Mac, targeting 40 tokens per second while minimizing quality loss.
First Kimi K3 results on home lab ~ 4t/s
User shares first home-lab results running Kimi K3 on 768GB DDR5 and 2x5090 using a llama.cpp fork and Q2_K quant, reporting prefill speeds of 50-70 tps and decoding tps that increases over time.
@JiaZhihao: Excited to share Lithos’ serving stack for Kimi K2.7 Code, a 1T-parameter frontier coding model. On a single 8×B200 nod…
Lithos announces its inference engine serving Kimi K2.7 Code, achieving over 1,000 tokens/sec per user on a single 8×B200 node at native precision, 3.4–5.7× faster than major providers.
VRAM disk cache of MoE makes 340 pp/s 9.6 tg/s for Kimi 2.7 on a single dgx spark
A detailed strategy using VRAM as disk cache with unified memory and llama.cpp settings achieves 340 pp/s and 9.6 tg/s for Kimi K2.7 on a single DGX Spark.