Kimi K3 full model running on 16x GB10 cluster at 20+tps
Summary
Kimi K3 full model runs on a 16x GB10 cluster at 20+ tokens per second average, with plans to publish the vllm image and instructions.
Similar Articles
@ciprianveg: Great speed-up: Full Kimi K3 — the 2.8T-parameter model running on 16× NVIDIA GB10 (Spark) - V5 coming by tomorrow, cca…
Kimi K3, a 2.8T-parameter AI model, is running on a cluster of 16 NVIDIA GB10s with impressive speed metrics, and a V5 update is expected tomorrow with about 20% faster performance and improved concurrency.
I got Kimi-k3 running.....
User successfully runs the Kimi-k3 model using llama.cpp on high-end hardware, achieving low tokens per second (0.41 prompt eval, 0.23 generation).
@UnTalNixon_exe: Forget about GPUs and million-dollar clusters. They just made the world's largest open model (Kimi K3 – 2.78 trillion p…
A new tool called kimi-k3-in-c runs the 2.78T-parameter Kimi K3 open model on a single CPU with as little as 8.24 GB RAM, streaming experts from disk and achieving deterministic output at 10-32 seconds per token.
I hosted Kimi K3 (2.8T parameters) using 8 B300s. 92 tok/s, $190 per million tokens
The article describes hosting the Kimi K3 AI model with 2.8 trillion parameters using 8 B300 GPUs, achieving 92 tokens per second and costing $190 per million tokens, while comparing it with Unsloth's dynamic GGUF quantization method.
First Kimi K3 results on home lab ~ 4t/s
User shares first home-lab results running Kimi K3 on 768GB DDR5 and 2x5090 using a llama.cpp fork and Q2_K quant, reporting prefill speeds of 50-70 tps and decoding tps that increases over time.