Ran gemma 4 12b on my 3090 yesterday and I think the local model game just changed
Summary
A user reports running Google's Gemma 4 12B model locally on a single RTX 3090 via GGUF quantization, finding strong performance including real 256k context, multimodal capabilities, and function calling that outperforms larger 70B models for coding tasks.
Similar Articles
@DogukanUrker: Gemma 4 12B on a single RTX 3060: the full 262,144 context at ~100 tok/s. (config below) dense model -> MTP speculative…
DogukanUrker demonstrates running Gemma 4 12B with full 262,144 context at ~100 tok/s on a single RTX 3060 using speculative decoding and KV cache splitting, achieving nearly full GPU utilization without CPU offload.
RTX5090, gemma-4-31B-it-Q6_K.gguf. Context: before - 35k, after - 80k!
Running the quantized Gemma-4-31B model on an RTX 5090 increases context length from 35k to 80k, showcasing significant performance improvement.
New Google Gemma 4 12B Claims Near-26B Performance - We Tested Both!
Google's new Gemma 4 12B model claims near-26B performance. In a local test on RTX 4090, the 26B-A4B model was faster and better but the 12B used less VRAM, making it suitable for laptops.
@analogalok: i just ran Google's brand new Unsloth Gemma4 12B dense GGUF on my RTX 4060 using llama.cpp + CUDA 13.2 21 tokens per se…
Google's new Gemma 4 12B is a single decoder-only transformer with encoder-free multimodal input, achieving strong benchmarks while being small enough to run locally on a budget GPU. It is released under Apache 2.0 license.
@leopardracer: GEMMA 4 26B ON AN RTX 4060 WITH A 248K TOKEN CONTEXT WINDOW 20 tokens per second and a context window so large you can …
Gemma 4 26B runs on an RTX 4060 with 248K token context at 20 tokens per second using llama.cpp and Q4_K_XL quantization, enabling local processing of entire codebases on consumer hardware.