autoregressive

BYOD presents a LoRA-only recipe for efficiently converting autoregressive language models into diffusion language models, achieving conversion of Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B in about 3-4 hours on a single Colab GPU while keeping all base parameters frozen.

This paper benchmarks the rollout reliability of seven PDE foundation models across seven 2D time-dependent PDE datasets, showing that while fine-tuned foundation models often beat scratch-trained baselines at short horizons, full-horizon rollouts suffer substantial error amplification (median 2.04x), arguing for horizon-resolved reporting in future benchmarks.

提出「先知后画」的混合学习范式，用自回归预训练获取语义知识、扩散后训练进行高保真渲染，并建立计算最优扩展律，仅用6,600 H100 GPU小时训出3B文生图模型，在GenEval和DPG-Bench上取得86/87的成绩。

ORBIT 提出了一种统一的深度时间序列生成框架，通过频率层次化 VQ-VAE 和频率带自回归 Transformer 实现从低频到高频的下一频带预测，从而先构建全局时间结构再逐步补充细节，在多个真实基准上达到最先进的性能与更高的推理效率。

论文提出 Causal Edit Serialization (CES) 框架，通过特殊指针令牌与双索引 RoPE，在标准因果注意力掩码下实现插入、删除、替换、复制和移动等局部化文本编辑，并在文法纠错任务上验证了监督与强化学习后训练的可行性。

Speculative Forcing 通过将中间 clean estimates 视为草稿并提前接受 token、跳过不必要的去噪步骤，加速少步自回归视频生成，在 LongLive 上以约 1.7 步/ token 匹配四步生成器质量，速度提升超过 1.5 倍。

Glance Forcing 提出一种轻量级蒸馏框架，将双向扩散模型的去噪轨迹解耦为慢速与快速两个专家，仅需单条 PF-ODE 轨迹即可超越全训练 SOTA 基线（VBench 总分从 84.04 提升至 84.22），并将轨迹采集成本从数千 GPU 小时降至约 1 个 A100 GPU 小时。

该论文将扩散模型、自回归及其混合体统一到一个“腐化格”（corruption lattice）框架中，证明调度的代价等于其与精确采样的散度，并指出零成本调度的最少步数由数据几何（如图的 treedepth）决定，还提出可在解码前预测调度优劣的核估计方法，并在 dLLM、图像与视频生成上验证了预测。

A step-by-step, first-principles explainer on how prefill and decode actually work inside LLM inference, starting from how ChatGPT-style chats process prompts via tokenization and autoregressive token generation.

SemanTok introduces a flexible-length video tokenizer that feeds frozen DINO features into its encoder and reconstructs them from each retained token prefix, enabling predictable coarse-to-fine semantic tokens for autoregressive video generation. A 201M SemanTok AR model matches or beats a VideoFlexTok model 3.4x its size, with strong semantic alignment and fidelity across model scales and noise levels.

FlashForward introduces a method for faster autoregressive video diffusion by reusing in-flight KV cache and using clean anchors, achieving speedups up to 2.92x while maintaining quality.

Augment Code switched its coding-agent backend to Stefano Ermon's Mercury 2.5 diffusion model, achieving 82% latency reduction and 90% cost cut in production. The article highlights the performance advantages of diffusion models and the need for independent AI benchmarking tools.

The author created Jevgpt, an autoregressive chatbot using a small token vocabulary and spec decoding, shared as a personal project on Hacker News.

Uranus is a next-generation simulation infrastructure for embodied AI that uses a joint-trajectory-conditioned autoregressive diffusion model to enable scalable, low-latency generation of robot simulations with streaming rollout and extensible control.

Reviser is a novel decoder-only Transformer model that enables revision-capable text generation via autoregressive cursor actions, achieving competitive performance with lower inference compute compared to baselines.

This article details building a world model for Super Mario Land using a transformer to predict next game frames from tokenized inputs, and demonstrates autoregressive 'dreaming' to generate future frames.

Zarya is a hybrid language model that jointly optimizes autoregressive and masked diffusion objectives for flexible training and dual-mode inference, with publicly released models in sizes 0.6B, 1.7B, and 4B.

Fraglingo introduces an attachment-aware autoregressive model for molecular design that generates molecules by jointly predicting fragment identity and attachment in a continuous latent space, enhancing property control and flexibility.

This paper analyzes how image tokenizer design affects joint text-image modeling in multimodal models using a controlled autoregressive testbed, showing distinct scaling behaviors and correlations with downstream performance.

Mask Forcing mitigates mode collapse in autoregressive video diffusion distillation by injecting masked cleaner signals during self-rollout, improving visual quality without extra training data.