Benchmarked Dense gemma-4-31b-it vs MoE gemma-4-26b-a4b-it to see if the cost reduction holds up in practice

Reddit r/AI_Agents News

Summary

A practical benchmark comparing Gemma dense (31B) and MoE (26B) models shows MoE is 25.5% faster and 20% cheaper per query with identical quality, validating theoretical cost savings.

Switched the same 100 prompts from Dense to MoE. 20% cheaper. 25.5% faster. Token output unchanged. Everyone says MoE models are cheaper. Fewer parameters activated per token, lower inference cost. But theory is not a production decision. So ran the numbers. The test 100 identical prompts. Two Gemma models. 200 live API calls via OpenRouter. No simulations, no estimates. Dense: gemma-4-31b-it MoE: gemma-4-26b-a4b-it What MoE won on 25.5% faster average latency. 20% cheaper per query. Identical token output on both sides, the architecture change did not touch quality. Verdict: it holds up. Where the gap narrows At median load (P50) the advantage is 27.3%. Under stress (P95) it drops to 12.9%. Both models hit the same infrastructure ceiling under peak load. If your SLA lives at the tail, check your own P95 before committing. At scale That 20% cost difference becomes $2,970/month at 100M daily queries. $30/month at 1M. Depends entirely on your volume. The benchmark was built and run by Neo, You can run it on your own prompts.
Original Article

Similar Articles

Gemma 4 MTP vs DFlash on 1x H100: dense vs MoE results

Reddit r/LocalLLaMA

This benchmark compares Gemma 4's Multi-Token Prediction (MTP) and z-lab's DFlash speculative decoding methods on a single H100 GPU, showing MTP faster for dense models and DFlash faster for MoE models.

Gemma 4 31B's competence surprised me

Reddit r/LocalLLaMA

A user shares anecdotal findings that Gemma 4 31B outperforms Qwen 3.6 models and matches Opus 4.7 in understanding and refactoring messy academic code, highlighting a benchmark (SciCode) where Gemma excels.