autoregressive-models

This paper investigates whether tabular foundation models (TFMs) like TabPFN, TabICL, TabDPT, and TabFM produce predictions consistent with any joint distribution. It demonstrates that all evaluated TFMs violate both marginalization and factorization consistency for classification and regression, questioning their Bayesian inference claims.

This paper introduces a training-free method to improve revisit consistency in autoregressive generative rendering by using temporal and spatial correspondences from the 3D engine to maintain consistent appearance when the camera revisits locations.

Introduces Surrogate Latent Policy Optimization (SLPO) to apply outcome-reward RL to autoregressive latent reasoners, enabling test-time scaling and variable-horizon policies that improve accuracy on harder instances.

This paper develops the QQ equality from cognitive science into an audit criterion for LLMs, characterizes theoretical mechanism classes, and empirically tests on an open-weight instruction-tuned model, finding that saturation (near-deterministic responses) prevents distribution-level audits.

Introduces the Member vs Generated Inference (MGI) task to distinguish training members from generated outputs in generative models, and proposes Data Circuit Breaker (DCB), a three-stage method combining autoencoder and latent generator signals, which outperforms existing methods across autoregressive and diffusion models.

A benchmark comparison of local open-weight LLMs on a single H100 (FP8) shows DiffusionGemma is 4x faster but makes 6x more mistakes than Gemma4 26B A4B, highlighting trade-offs between speed and accuracy in diffusion versus autoregressive models.

The paper introduces OPDLM, a method that transforms autoregressive language models into diffusion language models via on-policy distillation, requiring 15x to 7000x fewer training tokens while retaining knowledge from the original model.

Researchers propose a lightweight autoregressive framework for graph generation that uses structure-guided topological ordering to achieve near log-linear complexity, addressing scalability and novelty limitations of existing diffusion and autoregressive methods. The approach supports both LSTM and Mamba-style backbones and shows improved novelty while maintaining validity and uniqueness on molecular and non-molecular benchmarks.

This paper analyzes why LLM safety alignment is fragile, attributing it to 'autoregressive consistency'—the tendency of next-token prediction to extend the current response trajectory—which concentrates alignment updates on early tokens. The authors introduce a 'random insertion attack' exploiting this property and propose an adversarial safety alignment framework to address it.

This paper proposes a history-bootstrapped autoregressive flow matching method for reconstructing full spatiotemporal fields (velocity and temperature) from partial observations of boiling dynamics, addressing the ill-posed inverse problem with non-Markovian posterior.

LongLive-RAG formulates long video generation as a retrieval-augmented generation problem, using a dynamic memory of previously generated latents to reduce error accumulation and identity drift, achieving improved quality across multiple autoregressive backbones.

Introduces TUBE, a variational upper bound on log-likelihood for discrete diffusion language models, enabling better evaluation and revealing that masked diffusion models still underperform autoregressive models.

This paper introduces the Attribution Contract, a specification for feature-attribution claims in generative language models, addressing ambiguities in what constitutes a feature and how attribution methods should be evaluated. It uses autoregressive and diffusion models as case studies to show when attribution is informative or misleading.

KV cache stores previously computed key and value vectors during autoregressive generation, allowing models to avoid recomputing the entire sequence at each step, significantly speeding up inference at the cost of increased memory usage.

Q-ARVD is a novel quantization framework to reduce inference costs of autoregressive video diffusion models by addressing frame-wise sensitivity imbalance and weight outlier patterns.

dLLM is an open-source library that converts any autoregressive LLM into a diffusion LLM, enabling parallel decoding and faster text generation.

dLLM is an open-source Python library that allows converting any autoregressive language model into a diffusion language model with minimal compute, unifying training and evaluation.

KVPO introduces an ODE-native online GRPO framework that aligns streaming autoregressive video generators with human preferences using causal-semantic KV cache exploration and a velocity-field surrogate policy, achieving consistent improvements in visual quality and alignment.

This paper proves impossibility theorems showing that primacy effects, anchoring, and order-dependence are architecturally necessary biases in autoregressive language models due to causal masking constraints. The authors validate these theoretical bounds across 12 frontier LLMs and confirm related predictions through pre-registered human experiments involving working memory loads.

CausalCine is a new academic framework for real-time, interactive multi-shot video generation that uses causal modeling and dynamic memory routing to improve cross-shot coherence in autoregressive models.