Tag
Introduces a training-free Semantic-Aware Kernel Entropy (SAKE) guidance method for text diffusion models, using order-2 Rényi entropy over a kernel Gram matrix to balance fidelity and diversity during sampling. Experiments show improved Pareto frontier and multi-sample performance on reasoning-intensive tasks.
DeepMind researcher Brendan O'Donoghue provides an in-depth introduction to text diffusion models, which generate text through iterative denoising. Compared to autoregressive models, they offer lower latency but limited throughput, and demonstrate unique advantages such as self-correction and dynamic computation.
Google DeepMind released DiffusionGemma, an open experimental model that generates text in blocks rather than word-by-word, enabling self-correction and faster output.
This paper introduces the Safety-Aware Denoiser (SAD), a framework for integrating safety constraints into text diffusion models during the denoising process. It aims to reduce unsafe generations while preserving quality, addressing a gap in safety research for non-autoregressive models.
The Fast Byte Latent Transformer (BLT-D) has been accepted to ICML 2026, introducing a text diffusion method for parallel byte-level decoding to overcome the speed limitations of traditional byte-level language models.