@alec_helbling: A growing body of work uses continuous flow models for discrete problems like language generation. A simple recipe: rep…
Summary
The tweet discusses the growing use of continuous flow models for discrete tasks like language generation, proposing a method to represent vocabulary as one-hot vectors and learn flows in continuous space.
View Cached Full Text
Cached at: 09/19/26, 02:59 PM
A growing body of work uses continuous flow models for discrete problems like language generation.
A simple recipe: represent vocabulary items as one-hot vectors, learn a flow in that continuous space, then decode at the simplex corners. https://t.co/4jI1rcmUiZ
Similar Articles
LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
LangFlow presents the first continuous diffusion language model that rivals discrete diffusion approaches, challenging the long-held belief that continuous diffusion is inferior for language modeling. The work introduces key ingredients like optimal Gumbel-based noise scheduling and demonstrates competitive perplexity and transfer learning performance compared to discrete diffusion baselines.
@mihirp98: Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, gene…
Researchers propose Latent Thought Flows, compressing 256 text tokens into 8 continuous latents for one-step generation, achieving a better Pareto frontier than autoregressive baselines.
Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces
Multimodal Flow presents a fully continuous generative architecture for language and vision using a shared chunk-causal flow backbone over continuous embedding hyperchunks, with MF-1 pretrained at 0.6B to 1.6B scales that competitively performs on GenEval, DPG-Bench, VQAv2, MMBench, and POPE using only 150B tokens.
@r_de_santi: Generative models can’t discover what they can’t reach. We’re excited to introduce ActFlow: a continued pre-training sc…
The article introduces ActFlow, a continued pre-training scheme that expands the valid design space for flow and diffusion models, enabling out-of-distribution generative modeling and evolvable search spaces in scientific discovery.
Masked Language Flow Models
This paper introduces Masked Language Flow Models (MLFMs), which incorporate masking into flow-based language models to enable continuous flow for conditional generation and allow pretrained Masked Diffusion Models to be converted. The authors propose a novel sampler that alternates continuous denoising with discrete unmasking, demonstrating for the first time that flow-based language models can scale to downstream reasoning and instruction-following tasks.