Tag
A new post highlights a drawback of diffusion LLMs: bidirectional attention causes keys and values to drift across steps, breaking KV caching. However, generation quality is robust to slight KV drift, and research has focused on maximizing stale KV reuse without quality degradation.
ELF proposes a continuous diffusion model for language that uses embedding space and flow matching, outperforming existing discrete and continuous diffusion language models with fewer sampling steps.