Tag
The paper introduces Consistency Forcing (CForce), a distillation technique for diffusion large language models that improves parallel decoding by aligning early-stage predictions with later stages, enhancing speed-quality trade-offs.
Proposes Ripple-Pivot Search, a training-free decoding method for diffusion large language models that proactively commits mid-entropy pivot positions to reduce uncertainty and accelerate parallel decoding, achieving 4-10x speedup.
SLIM-RL introduces a risk-budgeted random-masking reinforcement learning method for diffusion LLMs that avoids trajectory slicing, achieving state-of-the-art results on math and code benchmarks with significantly fewer training samples.
A new post highlights a drawback of diffusion LLMs: bidirectional attention causes keys and values to drift across steps, breaking KV caching. However, generation quality is robust to slight KV drift, and research has focused on maximizing stale KV reuse without quality degradation.
WaveFilter proposes a training-free, wavelet-guided KV cache filtering framework for diffusion large language models that enhances long-context capability by precisely identifying key tokens and constructing sparse caches, improving performance on complex long-context tasks.
This paper introduces Temporal-Spatial Parallel Decoding (TSPD) and Confidence Extrapolation (CE) to accelerate inference in diffusion-based large language models by dynamically deciding when tokens have converged and forecasting logit trends, reducing unnecessary denoising steps while preserving output quality.
This paper introduces WINO and WINO+, methods that enable revokable parallel decoding in diffusion LLMs and distill efficient denoising trajectories, significantly improving the quality-speed trade-off.
This paper introduces DARE, a method for improving the inference efficiency of Diffusion Large Language Models by reusing cached key-value and output activations to reduce computational redundancy with negligible quality loss.