Tag
Next-Concept Modeling (NCM) 提出一种 coarse-to-fine 两阶段框架，利用多尺度残差 VQ-VAE 将文本编码为层级离散码并逐层并行生成，在 LM1B、OpenWebText 与 XSum 上以 10-31 倍加速达到或超越最强扩散语言模型基线，并在 Sudoku 推理任务上显著优于自回归与扩散基线。
This paper identifies within-group self-selection bias in prioritized experience replay and proposes sibling-aware methods to correct outcome distribution distortion, improving learning efficiency in reinforcement learning.
SupraLabs released Supra-A2A-Nano-Exp, a small any-to-any autoregressive model that unifies text and image tokenization into a single Transformer, serving as an educational prototype rather than a production-ready system.
SimPersona learns discrete buyer personas from raw clickstreams using a VQ-VAE and maps them to persona tokens for LLM-based web agents, achieving high conversion-rate alignment across many live storefronts.
This paper addresses the issue of dimensional collapse in VQ-VAEs, showing that representations often occupy a low-dimensional subspace. It proposes an 'AE Warm-Up' strategy that trains the model as an unquantized autoencoder first, which improves reconstruction quality and increases effective latent dimensionality.
An educational blog post explaining the Vector Quantized Variational Autoencoder (VQ-VAE) architecture, a key component of OpenAI's DALL-E image generation model.
OpenAI's Jukebox is a generative model that produces music as raw audio, including vocals and instruments, using a VQ-VAE for compression and hierarchical Sparse Transformer priors to handle long-range musical structure. It represents a significant step beyond symbolic music generation by operating directly in the raw audio domain.