Gemma4_31b_fp8 keeping up with Sonnet_4.6_medium in my harness.
Summary
A user reports that Gemma4_31b in FP8 matches or keeps up with Sonnet_4.6_medium in a custom harness across tasks like Cypher query generation, entity extraction, agentic tool calling, code writing, and multi-vector retrieval synthesis.
Similar Articles
Gemma 4 31B's competence surprised me
A user shares anecdotal findings that Gemma 4 31B outperforms Qwen 3.6 models and matches Opus 4.7 in understanding and refactoring messy academic code, highlighting a benchmark (SciCode) where Gemma excels.
Gemma 4 12B is my new main squeeze
The author shares their experience switching from Qwen 3.6 to Gemma 4 12B (Unsloth Q5_K_XL) for local coding, praising its plug-and-play setup, better syntax accuracy, and manageable VRAM usage despite a slight speed trade-off.
Tested (the updated) Gemma 4 locally on coding with OpenCode
Tested the updated Gemma 4 locally using llama.cpp on an M5 Pro, achieving 60 tokens/s for coding tasks with OpenCode; good for backend but poor for UI/UX.
yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF
A fine-tuned version of Gemma-4-12B, optimized for local coding and agentic tasks, achieving ~3.5x improvement over the base model on the tau2-bench telecom benchmark.
@leopardracer: GEMMA 4 26B ON AN RTX 4060 WITH A 248K TOKEN CONTEXT WINDOW 20 tokens per second and a context window so large you can …
Gemma 4 26B runs on an RTX 4060 with 248K token context at 20 tokens per second using llama.cpp and Q4_K_XL quantization, enabling local processing of entire codebases on consumer hardware.