Tag
A community post laying out feature requests for Google's upcoming Gemma 5, including looped transformers for better reasoning, DeepSeek's Engram memory, cheaper KV cache via Compressed Sparse Attention, a potential ~120B MoE variant, drafter improvements, and keeping Gemma well-rounded beyond coding benchmarks.