Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting?
Summary
This paper investigates whether complex transformer-based attention is necessary for global spatial information extraction in traffic forecasting, finding that simple global aggregation operators achieve comparable performance with lower computational complexity.
View Cached Full Text
Cached at: 07/15/26, 04:20 AM
# Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting? Source: [https://arxiv.org/abs/2607.12462](https://arxiv.org/abs/2607.12462) [View PDF](https://arxiv.org/pdf/2607.12462) > Abstract:Existing traffic forecasting models commonly focus on extracting spatial dependencies, particularly global spatial information, which characterizes the representations obtained through interactions between each individual node and all nodes across the traffic network\. However, the underlying mechanism by which such global information is modeled and extracted remains insufficiently investigated\. Whether global information must be extracted by high\-degree\-of\-freedom adaptive attention or can be captured by a simple global aggregation operator remains unclear\. For this purpose, we design a controlled ablation framework that replaces only the spatial mixing module to test attention\-based global interaction\. Across six traffic benchmarks, uniform full\-range mixing and standard spatial attention each achieve lower MAE on three datasets, with only a 0\.14% difference in mean MAE, while the former reduces node\-scale spatial mixing complexity from O\(N2\) to O\(N\)\. Mechanism analysis further decomposes spatial attention into a row\-uniform global background and a non\-uniform residual\. The residual shows dataset\-dependent marginal value, suggesting that spatial attention should be justified by stable gains beyond a row\-uniform global background\. The corresponding source code is publicly available at:[this https URL](https://github.com/uuesti/U-Trans) ## Submission history From: Siyao Zhang \[[view email](https://arxiv.org/show-email/b754a013/2607.12462)\] **\[v1\]**Tue, 14 Jul 2026 07:44:01 UTC \(813 KB\)
Similar Articles
PatchSTG: Scalable Spatiotemporal Graph Transformers for Traffic Forecasting on Irregular Sensor Networks
PatchSTG introduces a patch-based spatiotemporal graph Transformer for traffic forecasting on irregular sensor networks, achieving near-linear complexity while maintaining competitive performance.
A Global-Local Graph Attention Network for Traffic Forecasting
Proposes a Global-Local Graph Attention Network (GLGAT) with pairwise encoding and event-based adjacency matrix for traffic forecasting, effectively capturing spatio-temporal correlations and achieving competitive performance on real-world datasets.
EMAGN: Efficient Multi-Attention Graph Network via Learned Clustering for Scalable Traffic Forecasting
EMAGN is a new efficient multi-attention graph network for traffic forecasting that linearizes self-attention via learned clustering, reducing complexity from quadratic to linear while maintaining accuracy close to full-attention models, with significant reductions in training time, inference time, and GPU memory.
@antoniolupetti: "Transformers" by Daniel Jurafsky and James H. Martin is one of the clearest and most mathematically grounded introduct…
A tweet highlights the Transformer architecture chapter from Jurafsky and Martin's textbook, praising its clear and mathematically grounded explanation of self-attention, multi-head attention, and related mechanisms.
Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers
This paper introduces a two-stage token selection framework for visual geometry transformers that reduces computational costs by restricting key/value tokens during global attention, achieving over 85% acceleration on scenes with 500 images while maintaining baseline performance.