ELF: Embedded Language Flows

Hugging Face Daily Papers Papers
diffusion-models flow-matching language-modeling continuous-embeddings generation-quality

Summary

ELF proposes a continuous diffusion model for language that uses embedding space and flow matching, outperforming existing discrete and continuous diffusion language models with fewer sampling steps.

Diffusion and flow-based models have become the de facto approaches for generating continuous data, e.g., in domains such as images and videos. Their success has attracted growing interest in applying them to language modeling. Unlike their image-domain counterparts, today's leading diffusion language models (DLMs) primarily operate over discrete tokens. In this paper, we show that continuous DLMs can be made effective with minimal adaptation to the discrete domain. We propose Embedded Language Flows (ELF), a class of diffusion models in continuous embedding space based on continuous-time Flow Matching. Unlike existing DLMs, ELF predominantly stays within the continuous embedding space until the final time step, where it maps to discrete tokens using a shared-weight network. This formulation makes it straightforward to adapt established techniques from image-domain diffusion models, e.g., classifier-free guidance (CFG). Experiments show that ELF substantially outperforms leading discrete and continuous DLMs, achieving better generation quality with fewer sampling steps. These results suggest that ELF offers a promising path toward effective continuous DLMs.
Original Article
View Cached Full Text

Cached at: 05/12/26, 02:52 PM

Paper page - ELF: Embedded Language Flows

Source: https://huggingface.co/papers/2605.10938

Abstract

Continuous diffusion models for language processing can achieve superior generation quality by operating in embedding space rather than discrete token space, enabling efficient sampling and adaptation of image-domain techniques.

Diffusion andflow-based modelshave become the de facto approaches for generating continuous data, e.g., in domains such as images and videos. Their success has attracted growing interest in applying them tolanguage modeling. Unlike their image-domain counterparts, today’s leading diffusion language models (DLMs) primarily operate overdiscrete tokens. In this paper, we show that continuous DLMs can be made effective with minimal adaptation to the discrete domain. We propose Embedded Language Flows (ELF), a class ofdiffusion modelsincontinuous embedding spacebased oncontinuous-time Flow Matching. Unlike existing DLMs, ELF predominantly stays within thecontinuous embedding spaceuntil the final time step, where it maps todiscrete tokensusing a shared-weight network. This formulation makes it straightforward to adapt established techniques from image-domaindiffusion models, e.g.,classifier-free guidance(CFG). Experiments show that ELF substantially outperforms leading discrete and continuous DLMs, achieving bettergeneration qualitywith fewersampling steps. These results suggest that ELF offers a promising path toward effective continuous DLMs.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.10938

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.10938 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.10938 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.10938 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

@0xLogicrw: MIT's Kai-Ming He team has released a new language model, ELF (Embedded Language Flows). They bypass current autoregressive architectures by directly applying their expertise in diffusion models from the visual domain to text generation. Specifically: the entire generation process is embedded in a continuous vector space, converting it back to...

X AI KOLs Timeline

MIT's Kai-Ming He team released ELF, a new language model that uses diffusion processes in continuous vector space for text generation, bypassing standard autoregressive architectures and significantly reducing data requirements.

LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling

Hugging Face Daily Papers

LangFlow presents the first continuous diffusion language model that rivals discrete diffusion approaches, challenging the long-held belief that continuous diffusion is inferior for language modeling. The work introduces key ingredients like optimal Gumbel-based noise scheduling and demonstrates competitive perplexity and transfer learning performance compared to discrete diffusion baselines.

Masked Language Flow Models

arXiv cs.CL

This paper introduces Masked Language Flow Models (MLFMs), which incorporate masking into flow-based language models to enable continuous flow for conditional generation and allow pretrained Masked Diffusion Models to be converted. The authors propose a novel sampler that alternates continuous denoising with discrete unmasking, demonstrating for the first time that flow-based language models can scale to downstream reasoning and instruction-following tasks.

Language Modeling with Hyperspherical Flows

arXiv cs.LG

This paper introduces S-FLM, a novel flow-based language model that operates in a hyperspherical latent space to address the computational costs and semantic limitations of existing discrete diffusion and continuous flow models.