@MaximeRivest: Only 6B active parameters are needed to beat Opus 4.5. This means 6B parameters can work out the right next token and s…

X AI KOLs Timeline News

Summary

Maxime Rivest argues that only 6B active parameters are needed to beat Opus 4.5, suggesting that mixture-of-experts routing means frontier-level performance is achievable with under ~24B parameters, and encouraging more confidence in fine-tuning open models to compete with large labs.

Only 6B active parameters are needed to beat Opus 4.5. This means 6B parameters can work out the right next token and still perform above Opus 4.5. The catch is that, with current LLM architectures, each token is routed to a different set of experts, so the model jumps between experts constantly. I believe this suggests that any task can be done at frontier level with under ~24B parameters. Of course, I have no proof and no certainty, but there's enough here that we should all believe a lot more strongly that we can fine-tune models and beat the giant labs in our own domain, task, or business.
Original Article
View Cached Full Text

Cached at: 10/02/26, 10:46 PM

Only 6B active parameters are needed to beat Opus 4.5.

This means 6B parameters can work out the right next token and still perform above Opus 4.5. The catch is that, with current LLM architectures, each token is routed to a different set of experts, so the model jumps between experts constantly. I believe this suggests that any task can be done at frontier level with under ~24B parameters. Of course, I have no proof and no certainty, but there’s enough here that we should all believe a lot more strongly that we can fine-tune models and beat the giant labs in our own domain, task, or business.

Similar Articles

The idea: on a CPU the decode speed depends on the active params per token, not the total. My objective is trying to run a 10B at 100tok/s on a mid level PC (No GPU).

Reddit r/LocalLLaMA

Presents the insight that CPU decode speed depends on active parameters per token, not total parameters, and proposes a 10B-parameter model using ternary weights and granular MoE to achieve high token rates on mid-level PCs. Reports sandbox measurements showing a speedup from 176 to 848 tok/s on an 8.3M model with minimal quality loss.