@JulieKallini: Fast Byte Latent Transformer is accepted to ICML 2026! Byte-level LMs promise to free us from subword tokenizers, but d…
Summary
The Fast Byte Latent Transformer (BLT-D) has been accepted to ICML 2026, introducing a text diffusion method for parallel byte-level decoding to overcome the speed limitations of traditional byte-level language models.
View Cached Full Text
Cached at: 05/11/26, 08:42 PM
Fast Byte Latent Transformer is accepted to ICML 2026! ⚡🥪 Byte-level LMs promise to free us from subword tokenizers, but decoding one byte at a time is super slow. We make BLT generation more efficient with BLT-D: text diffusion for parallel byte decoding. 1/ https://t.co/ZIvUgavXvt
Similar Articles
Fast Byte Latent Transformer
This paper introduces BLT Diffusion and speculative decoding techniques for byte-level language models to significantly reduce generation latency and memory bandwidth costs while maintaining quality.
TextLDM: Language Modeling with Continuous Latent Diffusion
This paper introduces TextLDM, a method that adapts visual latent diffusion transformers for language modeling by mapping discrete tokens to continuous latents. It demonstrates that this approach, enhanced by representation alignment, matches GPT-2 performance and unifies visual and text generation architectures.
Cross-Tokenizer LLM Distillation through a Byte-Level Interface
This paper proposes Byte-Level Distillation (BLD), a simple method for cross-tokenizer knowledge transfer in language models by operating at a shared byte-level interface, achieving competitive or superior performance compared to more complex existing approaches across 1B-8B parameter models.
@Pavel_Izmailov: New paper: Latent Context Language Models (LCLMs)! Idea: encode 16 tokens as 1 latent token, and have the LLM work on t…
Introduces Latent Context Language Models (LCLMs), which encode 16 tokens as 1 latent token to improve performance, speed, and memory usage.
Nature paper retrofits LLMs to run on raw bytes, not tokens
A Nature paper introduces 'byteification', a method converting subword language models into byte-level ones with under 1% of a typical pretraining budget, producing models like Bolmo 7B that outperform prior byte-level baselines while retaining subword-level performance.