Tag
This paper introduces MeLISA, a latent-free autoregressive generative surrogate for forecasting high-dimensional physical dynamics that uses pixel-space MeanFlow to achieve efficient one-step generation. It demonstrates superior long-horizon statistical accuracy and inference speed compared to neural operators on turbulent flow benchmarks.
OpenAI introduces the Sparse Transformer, a deep neural network that improves the attention mechanism from O(N²) to O(N√N) complexity, enabling modeling of sequences 30x longer than previously possible across text, images, and audio. The model uses sparse attention patterns and checkpoint-based memory optimization to train networks up to 128 layers deep, achieving state-of-the-art performance across multiple domains.
OpenAI presents implicit generation and generalization methods for energy-based models (EBMs) that use Langevin dynamics for iterative refinement to generate samples without explicit generator networks. The approach offers advantages including adaptive computation time, flexibility in learning disconnected data modes, and built-in compositionality through product of experts.