@sudoingX: anyone running a 16gb card, stop scrolling. @pupposandro and @davideciffa got qwen 35b-a3b down to 13.3gb, measured on …
Summary
A technique called luce spark allows Qwen 35B-a3B MoE model to run on a 16GB GPU (like RTX 3090) by learning which experts are frequently used and streaming the rest from RAM, achieving ~100 tok/s without VRAM bottleneck.
View Cached Full Text
Cached at: 06/11/26, 05:43 PM
anyone running a 16gb card, stop scrolling. @pupposandro and @davideciffa got qwen 35b-a3b down to 13.3gb, measured on a 3090 gpu.
which means a model you literally could not load before now fits, running around 100 tok/s, near what you’d get with every expert resident on a 24gb card.
the clever part is the thing everyone gets wrong about moe. it only touches ~3b of its 35b params per token, routes to about 8 of 256 experts, but you still pay full vram to keep all of them around in case they’re next.
luce spark learns which experts your traffic actually hits, pins those hot, and streams the rest from ram hidden under the matmuls so there’s no speed cliff. one flag, and it tunes itself warmer every restart.
this is the kind of work that quietly drops the whole local inference tier down a card. don’t let it scroll past.
Similar Articles
Running Qwen 3.8 next on 16vram+32ram - A useful/fun post for the gpu poors
A Reddit user shares how they successfully ran the Qwen 3.8 Next MoE model on a system with 16GB VRAM and 32GB RAM using aggressive quantization and specific llama.cpp settings, achieving usable performance for large models on limited hardware.
Qwen 35B-A3B is very usable with 12GB of VRAM
A user benchmarks Qwen 35B-A3B (a 35B MoE model) on a 12GB RTX 3060, finding that 12GB VRAM is a practical sweet spot for running the model with 32k context, achieving ~47 t/s generation.
@sudoingX: save this one. it answers a question every 24gb GPU owner asks and almost nobody gets right. how much context can i act…
A user shares detailed VRAM usage measurements for running Qwen 3.6 27b on a 24GB GPU, showing context window sizes and headroom, and notes that a used RTX 3090 performs identically to newer cards.
@rohanpaul_ai: Beautiful visual of somebody running, qwen 3.8 27B locally on a RTX 5090 32 GB VRAM system with 115 tokens/sec note, Qw…
Tweet highlights running the Qwen 3.8 27B model locally on an RTX 5090 system with 32GB VRAM, achieving 115 tokens/sec, and notes the official BF16 checkpoint is 55.6GB.
Luce Spark: a 35B MoE on a 16 GB GPU, without the offload tax
Luce Spark is an open-source tool that enables running 35B MoE models on 16GB GPUs by intelligently caching hot experts on the GPU while keeping the rest in system RAM, using a calibrated placement and bounded async cache to maintain high throughput without the usual offload speed cliff.