@dunik_7: he made 405-billion-parameter models run on a single 8GB gaming GPU. instead of turning it into a paid service, he open…
Summary
AirLLM is an open-source tool that enables running 405-billion-parameter models on a single 8GB gaming GPU by streaming layers, priced at free and Apache 2.0 licensed.
View Cached Full Text
Cached at: 07/16/26, 04:21 PM
he made 405-billion-parameter models run on a single 8GB gaming GPU.
instead of turning it into a paid service, he open-sourced the whole thing.
Gavin Li Founder and CEO of Anima AI in the Bay Area.
‘airllm’ - it keeps one layer on the GPU at a time and streams the rest, so a 70B model fits in 4GB and DeepSeek’s full 671B fits in about 12GB
22,700 stars. Apache 2.0.
→ https://github.com/lyogavin
/ he has 808 followers. / his repo has 22,700 stars.
bookmark it. that’s what shipping something impossible looks like.
dunik (@dunik_7): running a 405-billion-parameter model used to mean a $30,000 GPU rack.
one developer made it run on a single 8GB card the kind that’s been sitting in gaming laptops for years.
it’s called AirLLM, and the trick is almost stupidly simple: instead of loading the whole model, it
Similar Articles
@tom_doerr: Runs 70B LLMs on single 4GB GPU https://github.com/lyogavin/airllm
AirLLM is an open-source tool that optimizes inference memory usage, enabling 70B LLMs to run on a single 4GB GPU without quantization, and supports 405B models on 8GB VRAM.
@NFTCPS: 4GB VRAM running 70B large model? It actually works! AirLLM did a clever trick — layered inference, not loading the whole model into VRAM at once, but layer by layer, compute and discard, squeezing the giant into a small GPU. The best part: 100% open source, freebie warning https://github.com/0xSo…
AirLLM is a fully open-source tool that uses layered inference (loading and releasing VRAM layer by layer) to enable 70B large language models to run on GPUs with only 4GB VRAM, without quantization, distillation, or pruning. It already supports running Llama3.1 405B on 8GB VRAM.
lyogavin/airllm
AirLLM is an open-source library that enables running large language models (up to 405B) on a single 4GB GPU without quantization, distillation, or pruning, significantly lowering the hardware barrier for local LLM inference.
@LottoLabs: A very cool model for the GPU poor bros Trained on an ungodly amount of tokens for a 8b a1b model Gonna be super fast e…
LottoLabs announces LiquidAI's LFM2.5-8B-A1B-GGUF model, an 8B parameter model trained on a massive token count and optimized for fast inference on limited GPU hardware, with support for llama.cpp, Ollama, vLLM, and more.
@heyrobinai: THE ENTIRE AI INDUSTRY JUST GOT HUMILIATED a tiny model trained in just a few hours on a single graphics card is planni…
Yann LeCun's team releases LeWorldModel, a tiny 15M-parameter physics model trained on a single GPU in hours that outperforms billion-dollar foundation models in planning speed and physical plausibility, challenging the dominant scaling paradigm.