FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
Summary
FlowLM introduces a flow matching language model derived from pre-trained diffusion models via efficient fine-tuning, enabling high-quality few-step text generation that rivals 2,000-step diffusion sampling with far fewer training epochs.
View Cached Full Text
Cached at: 05/21/26, 06:31 AM
# FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation Source: [https://arxiv.org/abs/2605.20199](https://arxiv.org/abs/2605.20199) [View PDF](https://arxiv.org/pdf/2605.20199) > Abstract:We present FlowLM, a flow matching language model transformed from pre\-trained diffusion language models via efficient fine\-tuning\. By re\-aligning the curved sampling trajectories of diffusion models into straight\-line flows, FlowLM enables high quality few\-step generation that rivals or even outperforms the quality of 2,000\-step diffusion sampling with very few training epochs\. Remarkably, finetuned FlowLM reaches performance saturation with only half as many training epochs as training from scratch, both approaches greatly outperforming the original diffusion model, thereby validating our method\. Furthermore, we validate a more effective training objective for flow matching: predicting clean data to consistently guide the sampling process towards the true data distribution\. Empirical results demonstrate that our approach is highly effective for high\-quality, few\-step text generation\. ## Submission history From: Runzhe Zhang \[[view email](https://arxiv.org/show-email/b36be57c/2605.20199)\] **\[v1\]**Mon, 6 Apr 2026 10:36:22 UTC \(3,537 KB\)
Similar Articles
LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
LangFlow presents the first continuous diffusion language model that rivals discrete diffusion approaches, challenging the long-held belief that continuous diffusion is inferior for language modeling. The work introduces key ingredients like optimal Gumbel-based noise scheduling and demonstrates competitive perplexity and transfer learning performance compared to discrete diffusion baselines.
Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space
This paper reformulates language generation as a stochastic optimal control problem, addressing limitations of autoregressive and diffusion models, and proposes a closed-loop diffusion method in latent control space using Flow Matching, achieving high-fidelity generation and efficient parallel sampling.
Language Modeling with Hyperspherical Flows
This paper introduces S-FLM, a novel flow-based language model that operates in a hyperspherical latent space to address the computational costs and semantic limitations of existing discrete diffusion and continuous flow models.
TextLDM: Language Modeling with Continuous Latent Diffusion
This paper introduces TextLDM, a method that adapts visual latent diffusion transformers for language modeling by mapping discrete tokens to continuous latents. It demonstrates that this approach, enhanced by representation alignment, matches GPT-2 performance and unifies visual and text generation architectures.
Continuous Latent Diffusion Language Model
Cola DLM is a hierarchical latent diffusion language model that uses text-to-latent mapping and conditional decoding to achieve efficient, non-autoregressive text generation.