Kimi K3: Open Frontier Intelligence

Hugging Face Daily Papers Papers
open-source mixture-of-experts large-language-model vision long-context reinforcement-learning frontier-intelligence

Summary

Kimi K3 is a 2.8 trillion parameter Mixture-of-Experts model with 104 billion active parameters, native vision, and a 1-million-token context window, achieving frontier-level performance across multiple domains and released as open weights.

We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.
Original Article
View Cached Full Text

Cached at: 07/28/26, 06:33 AM

Paper page - Kimi K3: Open Frontier Intelligence

Source: https://huggingface.co/papers/2607.24653 Published on Jul 27

#1 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

WeintroduceKimiK3,a2.8TparameterMixture-of-Expertsmodelwith104billionactivatedparameters,nativevisioncapabilities,anda1-million-tokencontextwindow.KimiK3isbuiltonKimiDeltaAttentionandAttentionResiduals,whichimproveinformationflowacrosssequencelengthandmodeldepth.TogetherwithStableLatentMoE,whicheffectivelyactivates16of896routedexpertspertoken,andrefinedtraininganddatarecipes,theseadvancesyieldanapproximately2.5ximprovementinoverallscalingefficiencyoverKimiK2.Post-traininghighlightsreinforcementlearningacrossgeneral,agentic,andcodingdomainsandmultiplereasoning-effortlevels,enablingcompositionalgeneralizationandrobustlong-horizonexecution.At2.8Tscale,KimiK3issupportedbyinfrastructureadvancesinmultipleareas:algorithm-systemco-designforKDA,perfectlybalancedexpert-paralleltrainingwithefficientmemorymanagement,million-tokenagenticRLwithpersistentrolloutandsandboxstates,anddeploymentinnovations.ExtensiveevaluationsshowthatKimiK3achievesfrontier-levelperformanceacrosslong-horizoncoding,agentic,knowledge,reasoning,andvisiontasks.Whileitsoverallperformancestilltrailsthemostpowerfulproprietarymodels,namelyClaudeFable5andGPT-5.6Sol,KimiK3consistentlyoutperformsotheropenandproprietarymodelsevaluatedinoursuite.WereleasethefullKimiK3modelweightstofacilitatefutureresearchandacceleratethebroaderdeploymentandadoptionoffrontierintelligence.

View arXiv pageView PDFProject pageAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.24653 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.24653 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.24653 in a Space README.md to link it from this page.

Collections including this paper2

Similar Articles

Kimi K3: Open Frontier Intelligence

Reddit r/LocalLLaMA

Kimi introduces Kimi K3, a 2.8 trillion parameter open model with native vision and 1M context, built on Kimi Delta Attention and Attention Residuals. It achieves frontier-level performance in coding and reasoning, with full weights to be released by July 2026.

Kimi-K3 is published on HuggingFace

Reddit r/artificial

Moonshot AI has released Kimi-K3, a 2.8T-parameter mixture-of-experts model with 1M token context window, available on HuggingFace under a permissive license with commercial limitations.

reteetzad/Kimi-K3

Hugging Face Models Trending

Kimi K3 is an upcoming open frontier model from Moonshot AI, featuring mixture-of-experts architecture, native agentic capabilities, and extended context for code understanding, scheduled for release on July 27, 2026.

unsloth/Kimi-K3

Hugging Face Models Trending

Kimi K3 is an open-weight 2.8T-parameter native multimodal agentic model with a novel architecture (KDA, AttnRes), 1M token context, and open-source MoE framework.

Kimi K3 Architecture Overview and Notes

Hacker News Top

Sebastian Raschka provides an architectural overview of the open-weight Kimi K3 model, highlighting its scaling from 48B to 2.8T parameters, new LatentMoE and attention residual components, removal of RoPE in favor of NoPE, and native multimodal support. The model emphasizes inference efficiency and matches frontier performance.