Large Flow Language Models

Hugging Face Daily Papers Papers
flow-models language-models parallel-decoding diffusion inference-efficiency multi-token-prediction

Summary

The paper presents Lafa, an 8B-parameter blockwise continuous flow language model trained with under 30B tokens that matches or surpasses autoregressive quality while achieving 2.54x inference throughput via self-speculative decoding and blockwise parallel decoding.

Continuous flow language models offer a path to parallel generation with tools for control and acceleration unavailable to discrete diffusion, yet effective techniques for building them at the billion-parameter scale remain less established. We present Lafa (Large flow language model), a fast, high-performing flow language model scaled to 8B parameters in a data-efficient manner. Lafa is a blockwise flow model with causal attention on the prefix and bidirectional, asynchronous flow within each block. This design enables efficient training by adapting pretrained autoregressive (AR) models and fast inference via prefix caching and blockwise parallel decoding. We train Lafa at 0.5B, 7B, and 8B scales with fewer than 30B tokens, more efficient than previous diffusion models, and evaluate it on instruction following, reasoning, and coding, where the learned flow matches or surpasses the quality of AR and can be distilled into flow maps for one- and few-step generation. Notably, flow training preserves or even improves pretrained AR capabilities via a multi-token prediction effect, which we leverage in a self-speculative decoding scheme that verifies flow-based drafts autoregressively within a single model. This yields 2.54x the throughput of an AR model at comparable quality, outperforming multi-token prediction and discrete diffusion methods. Our findings show that continuous flows can serve as a scalable foundation for language modeling.
Original Article
View Cached Full Text

Cached at: 10/21/26, 07:01 AM

Paper page - Large Flow Language Models

Source: https://huggingface.co/papers/2610.28559

Abstract

A large-scale continuous flow language model achieves efficient training, high-quality generation, and faster inference through blockwise flow design, self-speculative decoding, and multi-token prediction.

Continuous flow language modelsoffer a path to parallel generation with tools for control and acceleration unavailable todiscrete diffusion, yet effective techniques for building them at the billion-parameter scale remain less established. We present Lafa (Large flow language model), a fast, high-performing flow language model scaled to 8B parameters in a data-efficient manner. Lafa is ablockwise flow modelwithcausal attentionon the prefix and bidirectional, asynchronous flow within each block. This design enables efficient training by adapting pretrained autoregressive (AR) models and fast inference viaprefix cachingandblockwise parallel decoding. We train Lafa at 0.5B, 7B, and 8B scales with fewer than 30B tokens, more efficient than previous diffusion models, and evaluate it on instruction following, reasoning, and coding, where the learned flow matches or surpasses the quality of AR and can be distilled intoflow mapsfor one- and few-step generation. Notably, flow training preserves or even improves pretrained AR capabilities via amulti-token predictioneffect, which we leverage in aself-speculative decodingscheme that verifies flow-based drafts autoregressively within a single model. This yields 2.54x the throughput of an AR model at comparable quality, outperformingmulti-token predictionanddiscrete diffusionmethods. Our findings show that continuous flows can serve as a scalable foundation for language modeling.

View arXiv pageView PDFGitHub3Add to collection

Get this paper in your agent:

hf papers read 2610\.28559

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper3

#### lafa-hf/lafa-0.5B Text Generation• 0.7B• Updatedabout 4 hours ago • 7 #### lafa-hf/lafa-7B Text Generation• 8B• Updatedabout 4 hours ago • 6 #### lafa-hf/lafa-8B Text Generation• 8B• Updatedabout 4 hours ago • 6

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.28559 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.28559 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling

Hugging Face Daily Papers

LangFlow presents the first continuous diffusion language model that rivals discrete diffusion approaches, challenging the long-held belief that continuous diffusion is inferior for language modeling. The work introduces key ingredients like optimal Gumbel-based noise scheduling and demonstrates competitive perplexity and transfer learning performance compared to discrete diffusion baselines.

Masked Language Flow Models

arXiv cs.CL

This paper introduces Masked Language Flow Models (MLFMs), which incorporate masking into flow-based language models to enable continuous flow for conditional generation and allow pretrained Masked Diffusion Models to be converted. The authors propose a novel sampler that alternates continuous denoising with discrete unmasking, demonstrating for the first time that flow-based language models can scale to downstream reasoning and instruction-following tasks.

Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces

Hugging Face Daily Papers

Multimodal Flow presents a fully continuous generative architecture for language and vision using a shared chunk-causal flow backbone over continuous embedding hyperchunks, with MF-1 pretrained at 0.6B to 1.6B scales that competitively performs on GenEval, DPG-Bench, VQAv2, MMBench, and POPE using only 150B tokens.