Attention Is All You Need (to Avoid Spurious Oscillations)
Summary
This paper proposes an attention-based finite-volume scheme for shock transport, using a CFL-conditioned flux to enable efficient and accurate simulations with larger time steps.
View Cached Full Text
Cached at: 09/15/26, 08:45 AM
# Attention Is All You Need (to Avoid Spurious Oscillations) Source: [https://arxiv.org/abs/2609.13531](https://arxiv.org/abs/2609.13531) [View PDF](https://arxiv.org/pdf/2609.13531) > Abstract:Can attention move a shock across several cells in one update without breaking it? We develop a conservative, fixed grid finite\-volume scheme in which a CFL\-conditioned attention flux selects upstream information according to the transport required by the current time step\. One\-dimensional inviscid Burgers transport is used as the central mechanism test: the same learned flux remains reliable in the conventional small\-step regime and, with a time step four times larger, preserves sharp shocks while using one stage per update\. A standard fifth\-order WENO scheme with third\-order strong\-stability\-preserving Runge\-Kutta time integration \(WENO\-5\+SSP\-RK3\) is included alongside controlled Forward Euler comparisons to separate flux selection from time integration\. The learned attention shifts upstream with the local transport reach and becomes more selective near shocks; inference\-time interventions and retrained ablations show that transport\-scale information and state\-dependent selection contribute directly to performance\. Directional two\-dimensional scalar Burgers transport and the one\-dimensional shallow\-water system then test whether the conservation\-scale\-selection principle transfers beyond the original scalar setting\. The results support attention as a learnable information stencil for conservative large\-step shock transport, while identifying finite candidate reach and problem\-dependent robustness as the present limits\. ## Submission history From: Jinyoung Jeong \[[view email](https://arxiv.org/show-email/c14c7406/2609.13531)\] **\[v1\]**Fri, 11 Sep 2026 20:59:26 UTC \(1,536 KB\)
Similar Articles
Attention is All You Need was published 9 years ago...
This article commemorates the 9th anniversary of the 'Attention is All You Need' paper and highlights how transformer-based generative AI systems have solved the Navier-Stokes existence and smoothness problem, demonstrating rapid progress in AI within a decade.
Learning to Discretize: Diffusion-Based Adaptive Mesh with Spectral Guidance
This paper proposes a diffusion-based framework for learning adaptive mesh discretization conditioned on observed PDE dynamics, using spectral guidance and physics constraints to allocate resolution where needed. The method achieves competitive or superior performance across five PDE regimes.
Temporal Attention for Adaptive Control of Euler-Lagrange Systems with Unobservable Memory
This paper proposes a meta-control architecture using temporal self-attention for adaptive control of Euler-Lagrange systems with unobservable memory states. It demonstrates improved tracking performance over baseline methods on a 2-DOF manipulator while identifying failure modes in long-memory regimes.
Time Without Timesteps: Simulating Coupled Dynamical Systems via Self-Consistency
This paper presents a novel approach to simulating coupled dynamical systems by using neural surrogates and self-consistency to solve fixed-point problems over full trajectories, replacing traditional timestep-by-timestep integration. It demonstrates the method on van der Pol oscillators and Hodgkin–Huxley neuron networks, highlighting efficiency gains and unique gradient properties.
The Attention Within: Consensus Dynamics in Selective State Space Models
This paper investigates whether the recurrence in selective state space models drives tokens to consensus similar to attention in transformers, using dynamical systems theory to analyze stability and attraction domains for time-varying weight matrices.