autoregressive

This paper proposes HERO (History-Enriched Rollout Training), a method that augments standard trajectory supervision for autoregressive neural operators with relative supervision from the model's own optimization history, improving long-horizon accuracy and stability on PDE benchmarks.

PreDiff-LM proposes a hybrid attention mechanism that preserves causal attention for prompt tokens and bidirectional attention for masked target tokens, enabling adaptation of pretrained autoregressive models for discrete masked diffusion language modeling, achieving improvements in perplexity and downstream tasks over prior diffusion baselines.

The lab behind LLaDA2.2 released a diffusion model benchmarked against its own autoregressive model, showing diffusion lags on general knowledge and coding but wins on speed and agent tasks, providing a clean tradeoff data point.

CEDAR proposes a constraint-based method for lagged causal edge discovery in sparse autoregressive time series using AR(1)-residualized distance correlation and targeted conditional-independence tests, achieving efficient edge-level interpretability with O(d²) tests after screening.

This paper introduces a tractable method to control autoregressive large language model generation to satisfy LR(k) context-free grammars in polynomial time, improving over exponential previous methods. It demonstrates that current LLMs often fail to generate sequences satisfying simple nested constraints, motivating the need for efficient constrained generation.

WorldWeaver (W²) introduces cross-agent world state registers to multi-agent video diffusion models, enabling shared world state persistence across agents and views, improving logical consistency in two-agent Minecraft video generation.

Proposes Self Gradient Forcing (SGF), a two-pass training strategy for autoregressive video diffusion models that provides missing supervision for writing useful context memory, enabling strong long-video extrapolation even from short training windows.

Introduces DiffARFNO, a two-stage framework combining autoregressive Fourier-MIONet with a conditional DDIM corrector for long-horizon droplet evolution prediction in inkjet printing, achieving state-of-the-art performance on ANSYS Fluent datasets.

AlayaWorld is a 15B-parameter interactive video world model that generates 24-fps video at 540p and 720p, using autoregressive latent chunk generation with bounded visual context and distillation to reduce inference steps. It achieves state-of-the-art performance on the iWorld-Bench for long-horizon generation.

Inkling is a 975B-parameter sparse mixture-of-experts multimodal model accepting text, image and audio inputs and generating text outputs. Released with open weights for research, fine-tuning, and integration.

NVIDIA introduces TwoTower, a method that decouples context representation and denoising in diffusion language models, achieving 2.42x throughput while retaining 98.7% of autoregressive quality on a 30B MoE backbone.

Flex-Forcing introduces a unified framework for video diffusion that supports both autoregressive and bidirectional generation modes, offering flexible control for video generation tasks.

OPSD-V improves few-step autoregressive video diffusion models by using real long-video data as temporal context during training, providing dense trajectory-level supervision that enhances visual quality and motion dynamics without altering inference mechanisms.

UniSE is a unified, prompt-free, autoregressive speech enhancement model based on a decoder-only language model, supporting multiple tasks like speech restoration, target speaker extraction, and speech separation in a single model.

The paper introduces Nemotron-Labs-Diffusion, a tri-mode language model that unifies autoregressive, diffusion, and self-speculation decoding, achieving superior throughput and efficiency compared to existing models.

Set Diffusion introduces a new class of language models that interpolates between autoregressive and diffusion models by factorizing token generation over flexible-position, flexible-length token sets. This enables faster decoding and flexible token ordering, achieving better speed-quality tradeoffs on reasoning, summarization, and unconditional generation tasks.

Introduces Flex-Forcing, a unified training and inference framework that allows video diffusion models to operate under both bidirectional and autoregressive regimes via a flexible chunking mechanism over temporal and denoising steps, achieving better video quality, long-video stability, and faster inference.

GEAR proposes a method to jointly train a vector-quantized tokenizer and autoregressive generator end-to-end via representation alignment, achieving up to 10x faster convergence on ImageNet gFID compared to strong baselines.

Introduces Speculative Refinement (SpecRef), a training-free hybrid decoding strategy that warm-starts a masked diffusion language model from an autoregressive draft using entropy-guided selective masking. Evaluated across six benchmarks, it reveals that code benchmarks conflate structural discovery with logical correctness, identifies a refinement tension phenomenon, and shows that evaluation protocols can produce different model rankings.

MultiHashFormer is a hash-based generative language model that represents each token as a unique hash signature, enabling parameter-efficient autoregression. It outperforms standard Transformer LMs at 100M, 1B, and 3B scales and supports multilingual vocabulary expansion without increasing parameters.