Anyone tried using the new (ish) Gemma diffusion model as a speculative model?
Summary
Explores using Google's Gemma diffusion model as a speculative model for efficient large language model inference.
Similar Articles
DiffusionGemma
Google released DiffusionGemma, an open-weight text generation model (26B parameters, 4B active) under Apache 2 license, demonstrating high inference speeds via NVIDIA's NIM cloud API.
google/diffusiongemma-26B-A4B-it
Google DeepMind releases DiffusionGemma, a 26B-parameter Mixture-of-Experts model that uses discrete diffusion for faster text generation, supporting multimodal inputs and a 256K token context.
Google's latest DiffusionGemma open AI model comes with a 4x speed boost
Google released DiffusionGemma, an experimental open-source diffusion model for text generation that achieves 4x speed boost over autoregressive models, optimized for local processing.
@volokuleshov: Congratulations to Google on open-sourcing Gemma Diffusion! I want to give a shout-out to a group of really talented Co…
Google has open-sourced DiffusionGemma, a novel diffusion-based text generation model that uses block diffusion and efficient encoder-decoder techniques, with contributions from Cornell University researchers.
DiffusionGemma: The Developer Guide- Google Developers Blog
DiffusionGemma is a new experimental model from Google DeepMind that uses parallel generation on a 256-token canvas, achieving up to 4x faster token generation on GPUs. This developer guide explains its architecture, bidirectional context, and includes a fine-tuning recipe for solving Sudoku.