Benchmarked Dense gemma-4-31b-it vs MoE gemma-4-26b-a4b-it to see if the cost reduction holds up in practice
Summary
A practical benchmark comparing Gemma dense (31B) and MoE (26B) models shows MoE is 25.5% faster and 20% cheaper per query with identical quality, validating theoretical cost savings.
Similar Articles
I ran an experiment on the 30b class of gemma4 and qwen3.5 models to try to learn about energy cost and performance tradeoffs. In other words, which models use more energy to give the same answer quality?
Empirical study on four 30B-class dense and MoE models showing Gemma-4 26B MoE delivers equal accuracy at 1.9–15 Wh while dense and larger MoE variants consume up to 34 Wh for the same reasoning tasks.
Personal Eval follow-up: Gemma4 26B MoE (Q8) vs Qwen3.5 27B Dense vs Gemma4 31B Dense Compared
Personal benchmark shows Qwen3.5-27B Dense and Gemma4-31B Dense fix 100 % of 37 test failures, outperforming Gemma4-26B MoE even at 8-bit quantization, while using fewer tokens and less wall-clock time.
Gemma 4 MTP vs DFlash on 1x H100: dense vs MoE results
This benchmark compares Gemma 4's Multi-Token Prediction (MTP) and z-lab's DFlash speculative decoding methods on a single H100 GPU, showing MTP faster for dense models and DFlash faster for MoE models.
Mutating Gemma 4 31B Dense in to a native Gemma 4 additive-MoE model
A proof-of-concept training script enables converting Gemma 4 31B dense model into an additive-MoE architecture, inspired by a Korean-specific finetune from JDONE-Research.
Gemma 4 31B's competence surprised me
A user shares anecdotal findings that Gemma 4 31B outperforms Qwen 3.6 models and matches Opus 4.7 in understanding and refactoring messy academic code, highlighting a benchmark (SciCode) where Gemma excels.