Why there is a lack of new 100B-120B models?
Summary
Analysis of the trend in AI model sizes, noting a gap in the 100-120B parameter range with recent releases focusing on smaller (25-35B) or larger (200B+) models.
Similar Articles
A 4b model is now beating 30b ones at web research and the reason is not size
A 4 billion parameter open model from the Apodex family outperforms 30 billion parameter models on web research benchmarks, attributed to careful training data and self-verification techniques rather than raw scale, suggesting a more democratic trajectory for AI capability.
We need a 80-160B model urgently. The unified memory device market needs more Models.
The author argues that there is an urgent need for AI models in the 80-160B parameter range to support users with unified memory devices (e.g., high-RAM Apple/AMD systems), as recent models are either too small or too large for their hardware.
@LottoLabs: There’s so much demand for a good small model, look at top downloaded qwen models All < 9b
Observation that there is high demand for small AI models, as seen in the top downloads of Qwen models under 9B parameters.
@ChrisGPotts: We take for granted that larger models are better than smaller ones, but why is this so? Our new paper, led by Jing Hua…
This paper investigates why larger models outperform smaller ones, attributing it to data-induced competition for neural resources through formal analysis and experiments.
AMD & Intel, now onwards it's your turn to release your own models
NVIDIA has released a 550B parameter model (Nemotron-3-Ultra-550B), prompting commentary that AMD and Intel should similarly release their own AI models as model development becomes a commodity for hardware companies.