Are super tiny LLMs any good?
Summary
Explores whether very small language models can handle casual conversations adequately, and what training factors differentiate the better ones.
Similar Articles
Does size really matter? (LLMs vs. SLMs)
Discusses the trade-offs between large language models (LLMs) and small language models (SLMs), questioning whether larger models are always necessary for production use cases and exploring the future of AI deployment.
@cjzafir: VLMs (Vertical Language Models) are beating top LLMs. These small 7B to 15B niche-focused models are beating SoTA model…
The author demonstrates that small vertical language models (6B-15B) can outperform top LLMs on niche benchmarks through cost-effective fine-tuning using open-source models and Codex orchestration, achieving results with a $300 dataset.
Small LLMs: Pruning vs. Training from Scratch
This paper empirically compares pruning vs. training small language models from scratch, finding that pruning provides a strong advantage under limited token budgets but that the advantage diminishes as training scales, especially with coarse pruning.
What would optimal use of LLMs even look like?
Explores the speculative idea of optimizing human interaction with LLMs by conforming to their native communication patterns, such as using neuralese, rather than forcing them to adapt to human language.
The death of SLMs?
The author reflects on whether small language models under 27B are being overshadowed by larger models like Qwen 3.5 and Gemma 4, and asks the community for capable SLMs for agentic coding tasks.