Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting?
Summary
This paper investigates whether complex transformer-based attention is necessary for global spatial information extraction in traffic forecasting, finding that simple global aggregation operators achieve comparable performance with lower computational complexity.
View Cached Full Text
Cached at: 07/15/26, 04:20 AM
# Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting? Source: [https://arxiv.org/abs/2607.12462](https://arxiv.org/abs/2607.12462) [View PDF](https://arxiv.org/pdf/2607.12462) > Abstract:Existing traffic forecasting models commonly focus on extracting spatial dependencies, particularly global spatial information, which characterizes the representations obtained through interactions between each individual node and all nodes across the traffic network\. However, the underlying mechanism by which such global information is modeled and extracted remains insufficiently investigated\. Whether global information must be extracted by high\-degree\-of\-freedom adaptive attention or can be captured by a simple global aggregation operator remains unclear\. For this purpose, we design a controlled ablation framework that replaces only the spatial mixing module to test attention\-based global interaction\. Across six traffic benchmarks, uniform full\-range mixing and standard spatial attention each achieve lower MAE on three datasets, with only a 0\.14% difference in mean MAE, while the former reduces node\-scale spatial mixing complexity from O\(N2\) to O\(N\)\. Mechanism analysis further decomposes spatial attention into a row\-uniform global background and a non\-uniform residual\. The residual shows dataset\-dependent marginal value, suggesting that spatial attention should be justified by stable gains beyond a row\-uniform global background\. The corresponding source code is publicly available at:[this https URL](https://github.com/uuesti/U-Trans) ## Submission history From: Siyao Zhang \[[view email](https://arxiv.org/show-email/b754a013/2607.12462)\] **\[v1\]**Tue, 14 Jul 2026 07:44:01 UTC \(813 KB\)
Similar Articles
Spatiotemporal Graph Transformer for Traffic Intelligence in Edge Computing
This paper proposes a spatiotemporal graph Transformer framework for traffic forecasting in edge computing, combining graph neural networks for spatial correlations and Transformer self-attention for long-range temporal dependencies. Experiments on real-world cellular data show it outperforms recurrent graph-based baselines like GCN-LSTM and GCN-GRU.
PatchSTG: Scalable Spatiotemporal Graph Transformers for Traffic Forecasting on Irregular Sensor Networks
PatchSTG introduces a patch-based spatiotemporal graph Transformer for traffic forecasting on irregular sensor networks, achieving near-linear complexity while maintaining competitive performance.
A Global-Local Graph Attention Network for Traffic Forecasting
Proposes a Global-Local Graph Attention Network (GLGAT) with pairwise encoding and event-based adjacency matrix for traffic forecasting, effectively capturing spatio-temporal correlations and achieving competitive performance on real-world datasets.
EMAGN: Efficient Multi-Attention Graph Network via Learned Clustering for Scalable Traffic Forecasting
EMAGN is a new efficient multi-attention graph network for traffic forecasting that linearizes self-attention via learned clustering, reducing complexity from quadratic to linear while maintaining accuracy close to full-attention models, with significant reductions in training time, inference time, and GPU memory.
Ring-based Spatial Transformer: Learning Non-linear Spatial Interactions between Building Distribution and Pedestrian Flow
This study proposes a ring-based Spatial Transformer to model non-linear spatial interactions between building distribution and pedestrian flow around railway stations in Tokyo, outperforming Geographically Weighted Regression and challenging compact city assumptions in urban planning.