Token Time Continuous Diffusion for Language Modeling
Summary
Introduces Token Time Continuous Diffusion (TTCD), a new diffusion language model that operates in continuous space with per-token times, outperforming discrete models at high speedups in conditional generation and Sudoku solving.
View Cached Full Text
Cached at: 07/20/26, 09:41 AM
Paper page - Token Time Continuous Diffusion for Language Modeling
Source: https://huggingface.co/papers/2607.14106
Abstract
Inthispaperweintroducetokentimecontinuousdiffusion(TTCD),anewdiffusionlanguagemodelwhich(a)operatesincontinuousspace,deterministicallymappingGaussiannoisetoafinaltokencanvaswithnofurthersampling,andcrucially(b)incorporatesanewnotionofper-tokentimes,withsometokensproceedingfromnoisetotokenatafasterratethanothers.ContinuousspacemodelinghelpsTTCDavoidtheparallelsamplingofmultipletokens,whichisakeysourceofinaccuracyathighspeedupsformodelsthatiteratepurelyindiscretespace.Thenotionofper-tokentimeshelpsTTCDtobettermodelconditionalgeneration,allowsformoresuretokenstoproceedatafasterrate,andallowsfordifferentiatedinter-tokeninfluencesduringrefinement.TTCDoutperformsdiscretemodelsathighspeedups.Wetraina160MparameterTTCDmodelonOpenWebText,andthenself-distillit;wefindthatathighspeedupswearecomparableinunconditionalgenerationquality,andoutperforminconditionalgeneration,severalexistingmodelsofsimilarsizetrained,onthesamedata,andself-distilled.WeachievesimilargainsinSudokusolvingaswell.
View arXiv pageView PDFAdd to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.14106 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.14106 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.14106 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
LangFlow presents the first continuous diffusion language model that rivals discrete diffusion approaches, challenging the long-held belief that continuous diffusion is inferior for language modeling. The work introduces key ingredients like optimal Gumbel-based noise scheduling and demonstrates competitive perplexity and transfer learning performance compared to discrete diffusion baselines.
Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding
Set Diffusion introduces a new class of language models that interpolates between autoregressive and diffusion models by factorizing token generation over flexible-position, flexible-length token sets. This enables faster decoding and flexible token ordering, achieving better speed-quality tradeoffs on reasoning, summarization, and unconditional generation tasks.
TextLDM: Language Modeling with Continuous Latent Diffusion
This paper introduces TextLDM, a method that adapts visual latent diffusion transformers for language modeling by mapping discrete tokens to continuous latents. It demonstrates that this approach, enhanced by representation alignment, matches GPT-2 performance and unifies visual and text generation architectures.
Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion
This paper introduces a diffusion language model that treats text as a continuous process over binary bitstreams, using entropy-gated stochastic sampling to close the performance gap with autoregressive models. It achieves state-of-the-art results on LM1B and OWT benchmarks while reducing memory footprint.
BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion
This paper introduces BitLM, a language model that uses bitwise continuous diffusion to generate multiple tokens in parallel, aiming to overcome the sequential bottleneck of traditional autoregressive generation while preserving causal structure.