Model Size Scaling in 2023-2031 (21 minute read)
Summary
An analysis of AI model size scaling trends from 2023 to 2031, published on LessWrong.
View Cached Full Text
Cached at: 06/23/26, 01:43 PM
Similar Articles
Scaling Laws, Carefully (25 minute read)
A comprehensive overview of scaling laws in deep learning, tracing their theoretical roots and empirical findings, and explaining how loss decreases predictably with model size, data, and compute.
Why there is a lack of new 100B-120B models?
Analysis of the trend in AI model sizes, noting a gap in the 100-120B parameter range with recent releases focusing on smaller (25-35B) or larger (200B+) models.
How far behind are open models? (17 minute read)
An analysis from LessWrong examining the performance gap between open-source and proprietary AI models.
Scaling How We Build and Test Our Most Advanced AI
The article discusses the growing importance of reliability, security, and user protections as AI models become more capable and personalized.
Scaling laws for neural language models
Foundational empirical study demonstrating power-law scaling relationships between language model performance and model size, dataset size, and compute budget, with implications for optimal training allocation and sample efficiency.