I find it funny that a flash model is now 512GB
Summary
The author comments on the evolution of language model sizes, noting that 100GB models were once considered large, but now 512GB models are common, leading to smaller models being called 'tiny'.
Similar Articles
@TheAhmadOsman: Everyone is talking about small and specialized models finally Tweet below is from 17 months ago
A tweet highlights Jina AI's ReaderLM-v2, a small 4GB model that achieves high accuracy in extracting information from messy DOM elements, exemplifying the trend toward specialized small language models.
@NFTCPS: 4GB VRAM running 70B large model? It actually works! AirLLM did a clever trick — layered inference, not loading the whole model into VRAM at once, but layer by layer, compute and discard, squeezing the giant into a small GPU. The best part: 100% open source, freebie warning https://github.com/0xSo…
AirLLM is a fully open-source tool that uses layered inference (loading and releasing VRAM layer by layer) to enable 70B large language models to run on GPUs with only 4GB VRAM, without quantization, distillation, or pruning. It already supports running Llama3.1 405B on 8GB VRAM.
@TheAhmadOsman: HOLYYYY 27B model under 6GBs and 4GBs Local AI will be the default P.S. We are gonna get this optimized in ODS by @Osma…
Ternary Bonsai 27B, a large language model, is demonstrated running locally on an NVIDIA RTX 5090 GPU, requiring under 6GB of memory and enabling end-to-end agentic workflows on consumer hardware.
Even Google still believes in small models for coding.
A news article discussing Google's continued commitment to small AI models for code generation, despite the industry trend toward larger models.
Me train LLM on 8GB from Scratch. Me happy
Built a repository to train a tiny language model (25M parameters) from scratch on 8GB VRAM, with support for MTP but noting limitations of mHC and BitNet.