Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding
Summary
Set Diffusion introduces a new class of language models that interpolates between autoregressive and diffusion models by factorizing token generation over flexible-position, flexible-length token sets. This enables faster decoding and flexible token ordering, achieving better speed-quality tradeoffs on reasoning, summarization, and unconditional generation tasks.
View Cached Full Text
Cached at: 07/03/26, 05:43 AM
# Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding Source: [https://arxiv.org/abs/2607.01775](https://arxiv.org/abs/2607.01775) [View PDF](https://arxiv.org/pdf/2607.01775) > Abstract:Discrete diffusion models have steadily improved in quality relative to autoregressive \(AR\) models\. However, these models are normally constrained to fixed\-length generation and do not support key\-value \(KV\) caching\. Block diffusion partially bridges diffusion and AR by generating token blocks left\-to\-right, but its fixed\-size sequential blocks limit decoding flexibility and parallelism\. Here, we present a new class of language models, set diffusion, comprised of \(i\) a likelihood parameterization that factorizes over flexible\-position, flexible\-length token sets and \(ii\) a set\-causal diffusion architecture that supports KV cache updates after every inference step\. By factorizing over token sets instead of fixed\-size blocks, tokens can be decoded in arbitrarily\-ordered sets, including sliding\-window sets, enabling faster inference and support for any\-order decoding\. Set diffusion achieves better speed\-quality tradeoffs on mathematical reasoning, summarization, and unconditional generation compared to prior diffusion language models while offering stronger infilling performance than block diffusion\. We provide the code, along with the model weights and blog post on the project page:[this https URL](https://m-arriola.com/setdlms/) ## Submission history From: Marianne Arriola \[[view email](https://arxiv.org/show-email/ed38dc4f/2607.01775)\] **\[v1\]**Thu, 2 Jul 2026 06:45:43 UTC \(552 KB\)
Similar Articles
Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
The paper introduces Nemotron-Labs-Diffusion, a tri-mode language model that unifies autoregressive, diffusion, and self-speculation decoding, achieving superior throughput and efficiency compared to existing models.
Discrete Diffusion Models: A Unified Framework from Tokenization to Generation
This paper introduces a unified conceptual framework for discrete diffusion models, analyzing their design space through tokenization, state space construction, and highlighting trade-offs in training, inference, and scaling.
Token Time Continuous Diffusion for Language Modeling
Introduces Token Time Continuous Diffusion (TTCD), a new diffusion language model that operates in continuous space with per-token times, outperforming discrete models at high speedups in conditional generation and Sudoku solving.
DiffRetriever: Parallel Representative Tokens for Retrieval with Diffusion Language Models
This paper introduces DiffRetriever, a method that uses diffusion language models to generate multiple representative tokens in parallel for efficient information retrieval, outperforming autoregressive baselines in speed and accuracy.
Supportive Token Revealing for Fast Diffusion Language Model Decoding
This paper proposes AXON, a training-free module that improves the quality-latency trade-off of discrete diffusion language model decoding by intelligently selecting 'anchor' tokens to reveal first, using attention, uncertainty, and confidence signals to support subsequent denoising steps. Experiments on reasoning and code-generation benchmarks show AXON reduces function evaluations while maintaining or improving accuracy.