ELF: Embedded Language Flows
Summary
ELF proposes a continuous diffusion model for language that uses embedding space and flow matching, outperforming existing discrete and continuous diffusion language models with fewer sampling steps.
View Cached Full Text
Cached at: 05/12/26, 02:52 PM
Paper page - ELF: Embedded Language Flows
Source: https://huggingface.co/papers/2605.10938
Abstract
Continuous diffusion models for language processing can achieve superior generation quality by operating in embedding space rather than discrete token space, enabling efficient sampling and adaptation of image-domain techniques.
Diffusion andflow-based modelshave become the de facto approaches for generating continuous data, e.g., in domains such as images and videos. Their success has attracted growing interest in applying them tolanguage modeling. Unlike their image-domain counterparts, today’s leading diffusion language models (DLMs) primarily operate overdiscrete tokens. In this paper, we show that continuous DLMs can be made effective with minimal adaptation to the discrete domain. We propose Embedded Language Flows (ELF), a class ofdiffusion modelsincontinuous embedding spacebased oncontinuous-time Flow Matching. Unlike existing DLMs, ELF predominantly stays within thecontinuous embedding spaceuntil the final time step, where it maps todiscrete tokensusing a shared-weight network. This formulation makes it straightforward to adapt established techniques from image-domaindiffusion models, e.g.,classifier-free guidance(CFG). Experiments show that ELF substantially outperforms leading discrete and continuous DLMs, achieving bettergeneration qualitywith fewersampling steps. These results suggest that ELF offers a promising path toward effective continuous DLMs.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.10938
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.10938 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.10938 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.10938 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
@0xLogicrw: MIT's Kai-Ming He team has released a new language model, ELF (Embedded Language Flows). They bypass current autoregressive architectures by directly applying their expertise in diffusion models from the visual domain to text generation. Specifically: the entire generation process is embedded in a continuous vector space, converting it back to...
MIT's Kai-Ming He team released ELF, a new language model that uses diffusion processes in continuous vector space for text generation, bypassing standard autoregressive architectures and significantly reducing data requirements.
LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
LangFlow presents the first continuous diffusion language model that rivals discrete diffusion approaches, challenging the long-held belief that continuous diffusion is inferior for language modeling. The work introduces key ingredients like optimal Gumbel-based noise scheduling and demonstrates competitive perplexity and transfer learning performance compared to discrete diffusion baselines.
ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
ELF-REG scales continuous diffusion language models to reasoning tasks by using representation alignment and entanglement, achieving improved performance on benchmarks like GSM8K and HumanEval compared to other diffusion models.
Rethinking the Roles of Diffusion and Decoding: Predictive-State Flows for Parallel Language Generation
The paper proposes Predictive-State Flows (PSF), a continuous diffusion language model design where a rectified-flow prior generates block-level predictive states and a block-causal decoder generates tokens in parallel, significantly reducing diffusion states and sequential depth while improving efficiency and generative quality over models like ELF-B.
FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
FlowLM introduces a flow matching language model derived from pre-trained diffusion models via efficient fine-tuning, enabling high-quality few-step text generation that rivals 2,000-step diffusion sampling with far fewer training epochs.