@eliebakouch: Kimi K3 (2.8T total parameters) is an open weight model competing with fable and gpt 5.6 sol while being much cheaper, …
Summary
Kimi K3 is an open-weight 2.8 trillion parameter LLM with innovations like linear attention, latent MoE, and new activation functions, offering competitive performance at lower cost.
View Cached Full Text
Cached at: 07/17/26, 12:24 AM
Kimi K3 (2.8T total parameters) is an open weight model competing with fable and gpt 5.6 sol while being much cheaper, this is just insane
i don’t think we realize how impressive it is to scale to this parameter count and ship a banger model. they scaled more than 2x from their previous model while keeping research bets such as linear attention (KDA), quantile load balancing, attention residual..
the arch uses (stable?) latent moe from nvidia with 16 experts out of 896 (same expert sparsity as DSv4), per head muon (like in glm 5), new activation function called “Sigmoid Tanh Unit (SiTU)”, QB load balancing by the goat @Jianlin_S
waiting to see the tech report (they mentioned they will release one ) more test time compute curve and test it my own research tasks, but overall this looks really good and this is only the beginning
the gap between K2.5 and K2.7 was impressive, it will likely be the same for K3 and K3.2!!
Kimi.ai (@Kimi_Moonshot): Introducing Kimi K3: Open Frontier Intelligence
🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal 🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts 🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional
Similar Articles
@akshay_pachaar: Kimi K2.6 raises the bar for open-source models. Moonshot released it yesterday, and for the first time, an open-weight…
Moonshot's open-weight Kimi K2.6 matches Claude Opus 4.6 on key agentic benchmarks while costing significantly less.
Kimi k3 is 2.8t! Will need to have an aggressive iQ2_XXS or IQ1.8!
Kimi has released a new 2.8 trillion parameter model, k3, which is larger than expected and will require aggressive quantization to run on consumer hardware.
@thealexker: underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it…
Kimi.ai released Kimi K3, a 2.8 trillion parameter multimodal model with 1 million context, featuring novel Delta Attention and Attention Residuals, and a self-optimizing stack including MiniTriton compiler. The model achieves up to 6.3x faster decoding and ~25% higher training efficiency.
Kimi K3, and what we can still learn from the pelican benchmark
Chinese AI lab Moonshot AI announced Kimi K3, a 2.8 trillion parameter open-weights model, claiming it is the first open 3T-class model and beating several leading models on benchmarks. The article also discusses the model's pricing and a fun pelican SVG benchmark test.
Kimi K3 Shows Open-Weight Models Are About to Overtake the Frontier
Kimi K3, an open-weight model, demonstrates that open-weight models are nearing or surpassing proprietary frontier models in performance.