continuous diffusion for code generation in one step
Summary
The paper presents a method for one-step code generation by making language continuous, using diffusion models, and distilling the trajectory, with accompanying code release.
Similar Articles
Constrained Code Generation with Discrete Diffusion
This paper introduces Constrained Diffusion for Code (CDC), a training-free neurosymbolic inference framework that integrates constraint satisfaction directly into the reverse denoising process of discrete diffusion models for code generation. CDC consistently improves constraint satisfaction in functional correctness, security, and syntax across benchmarks, outperforming existing diffusion and autoregressive baselines.
Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space
This paper reformulates language generation as a stochastic optimal control problem, addressing limitations of autoregressive and diffusion models, and proposes a closed-loop diffusion method in latent control space using Flow Matching, achieving high-fidelity generation and efficient parallel sampling.
Continuous Diffusion Language Models (CDLM's)
The article discusses the resurgence of continuous diffusion models for language generation, highlighting recent research and historical context that challenges the dominance of autoregressive language models.
LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
LangFlow presents the first continuous diffusion language model that rivals discrete diffusion approaches, challenging the long-held belief that continuous diffusion is inferior for language modeling. The work introduces key ingredients like optimal Gumbel-based noise scheduling and demonstrates competitive perplexity and transfer learning performance compared to discrete diffusion baselines.
BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion
This paper introduces BitLM, a language model that uses bitwise continuous diffusion to generate multiple tokens in parallel, aiming to overcome the sequential bottleneck of traditional autoregressive generation while preserving causal structure.