@MaximeRivest: Only 6B active parameters are needed to beat Opus 4.5. This means 6B parameters can work out the right next token and s…
Summary
Maxime Rivest argues that only 6B active parameters are needed to beat Opus 4.5, suggesting that mixture-of-experts routing means frontier-level performance is achievable with under ~24B parameters, and encouraging more confidence in fine-tuning open models to compete with large labs.
View Cached Full Text
Cached at: 10/02/26, 10:46 PM
Only 6B active parameters are needed to beat Opus 4.5.
This means 6B parameters can work out the right next token and still perform above Opus 4.5. The catch is that, with current LLM architectures, each token is routed to a different set of experts, so the model jumps between experts constantly. I believe this suggests that any task can be done at frontier level with under ~24B parameters. Of course, I have no proof and no certainty, but there’s enough here that we should all believe a lot more strongly that we can fine-tune models and beat the giant labs in our own domain, task, or business.
Similar Articles
@MaximeRivest: 4B at a time, we could get to 10T-parameter models: a true mixture of experts. Jev showed us we can route pretty decent…
Maxime Rivest argues that 4B-parameter expert models could be combined into a true mixture-of-experts reaching 10T total parameters, enabling frontier-level performance at negligible cost on edge hardware, as small models fine-tuned on few synthetic examples already rival frontier systems.
@MaximeRivest: 4 Million parameter! 10 000 examples can get even BERT tiny to beat opus and kimi!
A study or model with only 4 million parameters, fine-tuned on 10,000 examples, achieves performance surpassing opus and kimi in certain benchmarks.
@AlmustyFX: Mixture-of-Experts lightweight solutions are becoming increasingly mature. Ling-3.0-tiny has a total scale of 7.9B para…
Ling-3.0-tiny uses a Mixture-of-Experts architecture with 7.9B total parameters but only activates 1.3B per run, reducing computational overhead to enable smoother local execution on Mac.
The idea: on a CPU the decode speed depends on the active params per token, not the total. My objective is trying to run a 10B at 100tok/s on a mid level PC (No GPU).
Presents the insight that CPU decode speed depends on active parameters per token, not total parameters, and proposes a 10B-parameter model using ternary weights and granular MoE to achieve high token rates on mid-level PCs. Reports sandbox measurements showing a speedup from 176 to 848 tok/s on an 8.3M model with minimal quality loss.
@savipww: a 744B parameter model just booted on a laptop with 25 gigs of ram and i read the repo twice before i believed it repo …
A 744B parameter mixture-of-experts model boots on a laptop with 25GB RAM by storing expert weights on SSD and only loading the active ~40B parameters per token, enabling local inference despite the model's size.