Are there good closed vs open LLM rankings? Also, are 70B–350B models actually worth it?
Summary
A discussion about the existence of trustworthy rankings comparing closed and open large language models, and whether models in the 70B–350B parameter range are worth the cost.
Similar Articles
Have we reached the point where open-source LLMs are “just good enough”?
A discussion on whether open-source LLMs are now 'just good enough' for most use cases, questioning the added value of proprietary models and the cost-benefit tradeoffs.
@cjzafir: VLMs (Vertical Language Models) are beating top LLMs. These small 7B to 15B niche-focused models are beating SoTA model…
The author demonstrates that small vertical language models (6B-15B) can outperform top LLMs on niche benchmarks through cost-effective fine-tuning using open-source models and Codex orchestration, achieving results with a $300 dataset.
Estimating worst case frontier risks of open weight LLMs
OpenAI researchers study worst-case frontier risks of releasing open-weight LLMs through malicious fine-tuning (MFT) in biology and cybersecurity domains, finding that open-weight models underperform frontier closed-weight models and don't substantially advance harmful capabilities.
Open and closed models are on different exponentials (8 minute read)
The article analyzes the economic divergence between open and closed AI models, arguing that premium closed models will maintain high margins through superior intelligence (especially for coding agents), while open models follow a different trajectory of commoditization and efficiency.
The gap between open weights LLMs and closed source LLMs
Analyzes the gap between open weights and closed source LLMs using the Artificial Analysis Intelligence Index and other benchmarks, finding that the gap is shrinking on some metrics but stable on others.