@iluciddreaming: Played with local LLMs for two months. Extensively tested various open-source models using Windows 11 + llama.cpp + llama-swap. Here is my final report card: Hardware: i7-13700 + 64GB RAM + RTX 4070. The best combination currently is gemm…
Summary
After two months of local LLM testing, the author finds that the combination of gemma-4-12B-it-QAT and MTP assistance performs best in speed and usability, with hardware i7-13700 + 64GB RAM + RTX 4070.
View Cached Full Text
Cached at: 06/15/26, 05:07 PM
I’ve been playing with local LLMs for two months.
Using Windows 11 + llama.cpp + llama-swap, I extensively tested various open-source models. Here’s my final report card:
Hardware: i7-13700 + 64GB RAM + RTX 4070
The best-performing combination so far is gemma-4-12B-it-qat-UD-Q4_K_XL + MTP support.
I’ve also run other models, but this one offers the best balance of speed and practical usability.
Next, I’m looking forward to DiffusionGemma being integrated into llama.cpp so I can continue experimenting.
Similar Articles
@mylifcc: I'm already running Gemma-4-12b on my Mac. Tech stack: llama.cpp + GGUF Q4_K_M + Metal 32K context, local OpenAI-compatible API. Measured about 36 tok/s, resident RSS about…
User shares their experience using llama.cpp with the GGUF Q4_K_M quantized version of Gemma-4-12b on a Mac, achieving local inference speed of about 36 tok/s and memory usage of about 10GB.
@TraffAlex: Best Local LLMs for Consumer GPUs — llama.cpp Guide (June 2026) What I actually run on consumer hardware right now. Eve…
A guide to the best local LLMs for consumer GPUs as of June 2026, using llama.cpp to run models like Gemma 4-12B, Qwen3.6-27B, and Nex-N2-Mini on 8-32GB VRAM, with setup and launch commands.
Local LLM autocomplete + agentic coding on a single 16GB GPU + 64GB RAM
A technical guide on setting up local LLM autocomplete (Qwen2.5-Coder-7B) and agentic coding (Qwen3.6-35B-A3B) on a single 16GB GPU with 64GB+ RAM using llama.cpp, including commands and performance benchmarks.
Tested how OpenCode Works with SelfHosted LLMS: Qwen 3.5, 3.6, Gemma 4, Nemotron 3, GLM-4.7 Flash - v2
A developer benchmarked multiple self-hosted LLMs (Qwen 3.5/3.6, Gemma 4, Nemotron 3, GLM-4.7) with OpenCode on two coding tasks, revealing speed and quality trade-offs on RTX 4080 hardware.
@analogalok: my 8 GB VRAM gaming laptop is absolutely going to hate me for this. but I still did it. ran a 31b dense model (Gemma 4 …
User runs Gemma 4 31B dense model on 8GB VRAM gaming laptop at ~3 tokens/sec using llama.cpp with MTP speculative decoding, demonstrating feasibility of running a 31B dense model on consumer hardware and proposing agentic workflows where a fast MoE model routes to this slower dense model for hard tasks.