@businessbarista: Best explainer on Kimi K3 i've read. It walks you through how the model works & the elegant innovation behind it: - K3 …

Best explainer on Kimi K3 i've read. It walks you through how the model works & the elegant innovation behind it: - K3 is the biggest open model anyone has released: 2.8 trillion parameters total, though only 104 billion of them do the work on any given word. - The problem Moonshot went after is memory. Normally a model keeps notes on every word it has read, and that pile grows with every token, which is why long conversations get slow and expensive. - K3 mostly stops the memory pile from growing. Three out of every four layers use a new mechanism called Kimi Delta Attention, which keeps a fixed-size working memory and edits it as it goes, overwriting what's stale instead of hoarding everything. The fourth layer keeps a compressed record of each word so exact details are still recoverable when they matter. - They proved this on a small model first. A 48B test version used up to 75% less memory and ran about 4× faster at long context, while matching or beating the conventional design on the benchmarks they reported. - Training leaned hard on long jobs including coding, browsing, research, visual work, agent sessions running hundreds or thousands of tool calls in a row. - 1 Million cached input tokens costs $0.30. An agent can pull the same repo, the same docs, the same tool definitions back into context over and over without the bill getting stupid. Moonshot credits that to the memory design working alongside their serving stack. - The biggest takeaway: context window size is the main character, but what matters is what a model compresses, what it forgets, and how it gets exact information back.