@thealexker: underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it…
Summary
Kimi.ai released Kimi K3, a 2.8 trillion parameter multimodal model with 1 million context, featuring novel Delta Attention and Attention Residuals, and a self-optimizing stack including MiniTriton compiler. The model achieves up to 6.3x faster decoding and ~25% higher training efficiency.
View Cached Full Text
Cached at: 07/16/26, 10:23 PM
underrated gems in Kimi-K3 release:
an early K3 wrote the majority of the kernels in the late development stages
it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile then it designed a chip, by a model, for a model, in one 48-hour autonomous run
the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights.
we’re watching software build its own hardware
Kimi.ai (@Kimi_Moonshot): Introducing Kimi K3: Open Frontier Intelligence
🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal 🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts 🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional
Similar Articles
Kimi-K3 Technical Report [pdf]
MoonshotAI releases Kimi-K3, a 2.8T-parameter open-weight multimodal agentic model with a 1M-token context window, built on new Kimi Delta Attention and Attention Residuals architecture, achieving significant scaling improvements.
Kimi K3: Open Frontier Intelligence
Kimi introduces Kimi K3, a 2.8 trillion parameter open model with native vision and 1M context, built on Kimi Delta Attention and Attention Residuals. It achieves frontier-level performance in coding and reasoning, with full weights to be released by July 2026.
Releasing the model weights and technical report of Kimi K3 (2 minute read)
Kimi Moonshot released Kimi K3, a 2.8-trillion-parameter multimodal model with a 1M context window and architectural innovations like Kimi Delta Attention and Attention Residuals, claiming significant efficiency gains and outperforming Claude Opus 4.8 and GPT-5.5 on internal benchmarks.
@businessbarista: Best explainer on Kimi K3 i've read. It walks you through how the model works & the elegant innovation behind it: - K3 …
The tweet highlights an explainer of Moonshot AI's Kimi K3, a 2.8-trillion-parameter open model using novel Kimi Delta Attention to cut memory growth and speed up long-context inference, with strong performance on agentic and long-horizon tasks at low caching cost.
Kimi K3 Architecture Overview and Notes
Sebastian Raschka provides an architectural overview of the open-weight Kimi K3 model, highlighting its scaling from 48B to 2.8T parameters, new LatentMoE and attention residual components, removal of RoPE in favor of NoPE, and native multimodal support. The model emphasizes inference efficiency and matches frontier performance.