autoregressive

Augment Code switched its coding-agent backend to Stefano Ermon's Mercury 2.5 diffusion model, achieving 82% latency reduction and 90% cost cut in production. The article highlights the performance advantages of diffusion models and the need for independent AI benchmarking tools.

The author created Jevgpt, an autoregressive chatbot using a small token vocabulary and spec decoding, shared as a personal project on Hacker News.

Uranus is a next-generation simulation infrastructure for embodied AI that uses a joint-trajectory-conditioned autoregressive diffusion model to enable scalable, low-latency generation of robot simulations with streaming rollout and extensible control.

Reviser is a novel decoder-only Transformer model that enables revision-capable text generation via autoregressive cursor actions, achieving competitive performance with lower inference compute compared to baselines.

This article details building a world model for Super Mario Land using a transformer to predict next game frames from tokenized inputs, and demonstrates autoregressive 'dreaming' to generate future frames.

Zarya is a hybrid language model that jointly optimizes autoregressive and masked diffusion objectives for flexible training and dual-mode inference, with publicly released models in sizes 0.6B, 1.7B, and 4B.

Fraglingo introduces an attachment-aware autoregressive model for molecular design that generates molecules by jointly predicting fragment identity and attachment in a continuous latent space, enhancing property control and flexibility.

This paper analyzes how image tokenizer design affects joint text-image modeling in multimodal models using a controlled autoregressive testbed, showing distinct scaling behaviors and correlations with downstream performance.

Mask Forcing mitigates mode collapse in autoregressive video diffusion distillation by injecting masked cleaner signals during self-rollout, improving visual quality without extra training data.

A paper from the DUSt3R team proposes sparse auto-regressive modeling for 3D scene generation from multi-view images, using a voxel-aligned 3D latent space and an occupancy-aware masked autoregressive transformer.

The article discusses the resurgence of continuous diffusion models for language generation, highlighting recent research and historical context that challenges the dominance of autoregressive language models.

MegaParts introduces a scalable framework for part-aware 3D object generation using token-efficient vector-quantized tokens and autoregressive modeling, enabling generation of objects with up to 300 parts.

ChronoSSM introduces an autoregressive State Space Model that jointly models events and timestamps, showing that joint training improves temporal recoverability without degrading content generation quality.

A best-paper research from NVIDIA and collaborators introduces WorldTrace, a training-free framework that keeps compressed memory addressable in autoregressive video world models by assigning fixed slot-rank positions, enabling coherent long rollouts and long-range recall beyond the training horizon.

A new transformer variant, the full-bandwidth transformer, feeds back top-layer hidden states through a gated linear unit to improve reasoning and efficiency without altering the core architecture. Trained up to 400B tokens, it matches standard transformers trained with 1.5x more data while producing shorter reasoning traces.

A researcher muses on the unstable equilibrium where language uses autoregressive models while other modalities use diffusion, and speculates that a unified multimodal architecture depends on the order each modality reveals information. He seals a bet on this idea.

This paper studies a self-supervised task for generating single-cell gene expression vectors using an autoregressive transformer with a quantized VAE tokenizer. It reports scaling laws and a compute-optimal frontier for single-cell foundation models, with potential fine-tuning for perturbation prediction.

This paper presents a multimodal auto-regressive transformer surrogate that models variable well operations and geological uncertainty for geological carbon storage, achieving accurate predictions and enabling uncertainty quantification via MCMC data assimilation.

A tweet argues that masked language modeling was unnecessary and that autoregressive models would have sufficed, with a nod to BERT.

JoyAI-Video-Edit is a 16B-parameter autoregressive diffusion framework for real-time, open-ended video editing, achieving 720p editing at ~30 FPS on a single NVIDIA B200 GPU.