STAGformer: A Spatio-temporal Agent Graph Transformer for Micro Mobility Demand Forecasting

# A Spatio-temporal Agent Graph Transformer for Micro Mobility Demand Forecasting
Source: [https://arxiv.org/html/2607.06614](https://arxiv.org/html/2607.06614)
###### Abstract

Accurate station\-level demand forecasting is essential for the efficient operation of bike\-sharing systems, yet it remains challenging due to complex spatio\-temporal dependencies and the large scale of urban networks\. This paper presents STAGformer, a Spatio\-Temporal Agent Graph Transformer that achieves efficient global modeling with linear computational complexity\. The model introduces a two\-step agent attention mechanism, where a small set of learnable spatial and temporal agent tokens first aggregate global information and then broadcast it back to individual stations and time steps, effectively capturing long\-range interactions while reducing the quadratic cost of standard self\-attention to𝒪​\(N​T\)\\mathcal\{O\}\(NT\)\. STAGformer integrates four core modules: a spatio\-temporal encoder that fuses dynamic node features with external contextual factors \(weather, time, points of interest\), a graph propagation module for spatial neighbor aggregation, a temporal convolution module for local pattern extraction, and the agent attention module for global dependency modeling\. Extensive experiments on two real\-world datasets—NYC Citi\-Bike and Chicago Divvy\-Bike—demonstrate that STAGformer consistently outperforms state\-of\-the\-art baselines across multiple prediction horizons, achieving significant improvements in both RMSE and MAE\. Ablation studies validate the contribution of each component, with the agent attention mechanism proving critical for modeling global spatio\-temporal dependencies\. By combining the expressive power of softmax attention with linear efficiency, STAGformer provides a scalable and accurate solution for micro\-mobility demand forecasting in large\-scale urban systems\.

Keywords:Bike\-sharing, Graph Neural Network, Traffic forecasting, Demand modelling

## 1Introduction

As a green and flexible mode of micro\-mobility, bike\-sharing has been widely adopted in numerous cities worldwide, effectively alleviating traffic congestion and promoting low\-carbon travel\[[37](https://arxiv.org/html/2607.06614#bib.bib37),[28](https://arxiv.org/html/2607.06614#bib.bib28)\]\. However, bike\-sharing systems are inherently dynamic and spatio\-temporally heterogeneous: rental and return demands at stations exhibit periodic fluctuations over time, while spatially they are influenced by road network structures, the distribution of points of interest \(POIs\), and real\-time weather conditions \(as illustrated in Fig\.[1](https://arxiv.org/html/2607.06614#S1.F1)\)\. Therefore, accurate station\-level demand forecasting is crucial for vehicle rebalancing, operational optimization, and enhancing user experience\[[30](https://arxiv.org/html/2607.06614#bib.bib30),[32](https://arxiv.org/html/2607.06614#bib.bib32)\]\.

In recent years, deep learning techniques, particularly Graph Neural Networks \(GNNs\) and Transformer architectures, have significantly advanced spatio\-temporal forecasting\[[9](https://arxiv.org/html/2607.06614#bib.bib9),[10](https://arxiv.org/html/2607.06614#bib.bib10),[16](https://arxiv.org/html/2607.06614#bib.bib16)\]\. GNNs effectively model spatial dependencies among stations\[[14](https://arxiv.org/html/2607.06614#bib.bib14),[15](https://arxiv.org/html/2607.06614#bib.bib15)\], while Transformers excel at capturing long\-range temporal correlations\[[17](https://arxiv.org/html/2607.06614#bib.bib17),[26](https://arxiv.org/html/2607.06614#bib.bib26)\]\. Nevertheless, existing methods face two major challenges when applied to large\-scale bike\-sharing networks: \(i\) the computational complexity of standard self\-attention scales quadratically with the product of the number of stationsNNand time stepsTT, i\.e\.,𝒪​\(\(N​T\)2\)\\mathcal\{O\}\(\(NT\)^\{2\}\), rendering it impractical for city\-scale systems\[[16](https://arxiv.org/html/2607.06614#bib.bib16),[18](https://arxiv.org/html/2607.06614#bib.bib18)\]; \(ii\) most models fail to effectively integrate multi\-source external factors \(e\.g\., weather, POIs\) that influence demand, leading to limited prediction accuracy\[[29](https://arxiv.org/html/2607.06614#bib.bib29),[34](https://arxiv.org/html/2607.06614#bib.bib34)\]\.

To address these challenges, we propose a Spatio\-Temporal Agent Graph Transformer \(STAGformer\) based on an agent attention mechanism\. By introducing a small set of learnable “agent” tokens, STAGformer reduces the complexity of self\-attention to linear in both the number of stations and time steps, thereby efficiently capturing global spatio\-temporal dependencies\[[31](https://arxiv.org/html/2607.06614#bib.bib31)\]\. Specifically, STAGformer comprises four core modules: a spatio\-temporal feature encoder that projects historical station features and external factors into a high\-dimensional representation and injects positional information; a graph propagation module that explicitly aggregates information from spatial neighbors using the graph structure; a temporal convolution module that extracts local temporal patterns for each station; and an agent attention module that models long\-range interactions between stations and time steps with linear complexity through a two\-step aggregation and broadcasting mechanism\. Finally, the fused features are passed through an output layer to generate multi\-step predictions of rental and return demands\.

The main contributions of this paper are as follows:

1. 1\.We design an encoding scheme that fuses dynamic node features with global external features, and enhance both global and local pattern extraction through graph propagation and temporal convolution\.
2. 2\.We propose STAGformer, which introduces the agent attention mechanism to the task of bike\-sharing demand forecasting, achieving linear computational complexity while maintaining global modeling capability\.
3. 3\.Extensive experiments on two real\-world datasets demonstrate that STAGformer outperforms state\-of\-the\-art baselines in terms of RMSE and MAE, while offering higher computational efficiency\.

The remainder of this paper is organized as follows: Section 2 reviews related work; Section 3 formulates the problem and provides preliminaries; Section 4 details the STAGformer architecture; Section 5 describes the experimental setup and results analysis; Section 6 concludes the paper and discusses future work\.

![Refer to caption](https://arxiv.org/html/2607.06614v1/chicago.png)Figure 1:Spatial imbalance of bike\-sharing demand and returns in Chicago on Wednesday, September 10, 2025\. Red areas indicate high demand or returns, blue areas low activity\.
## 2Related Work

### 2\.1Spatio\-Temporal Forecasting: Evolution and Challenges

Station\-level demand forecasting in micro\-mobility systems, such as bike\-sharing, is inherently challenging due to complex spatial dependencies, temporal dynamics, and the large scale of urban networks\. Early approaches primarily relied on time series models, e\.g\., ARIMA and its seasonal variants SARIMA, which capture periodicities and trends but fail to incorporate spatial correlations and external factors\[[2](https://arxiv.org/html/2607.06614#bib.bib2),[1](https://arxiv.org/html/2607.06614#bib.bib1)\]\. Subsequently, machine learning models like Support Vector Regression \(SVR\), Random Forests, and Gradient Boosting Trees were adopted, improving accuracy by manually engineering features \(e\.g\., weather, holidays\)\[[5](https://arxiv.org/html/2607.06614#bib.bib5),[4](https://arxiv.org/html/2607.06614#bib.bib4)\]\. However, these methods cannot automatically mine spatial dependencies among stations and have limited capacity for long\-range temporal modeling\.

The advent of deep learning revolutionized spatio\-temporal forecasting\. Recurrent Neural Networks \(RNNs\) and their variants \(LSTM, GRU\) were widely applied to model time series for each station independently, effectively capturing non\-linear temporal dependencies\[[6](https://arxiv.org/html/2607.06614#bib.bib6),[7](https://arxiv.org/html/2607.06614#bib.bib7)\]\. To simultaneously model spatial correlations, researchers turned to Convolutional Neural Networks \(CNNs\) applied to grid\-based urban representations\[[8](https://arxiv.org/html/2607.06614#bib.bib8)\], but grid partitioning fails to adapt to non\-Euclidean road network structures\. The emergence of Graph Neural Networks \(GNNs\) addressed this issue\. The Diffusion Convolutional Recurrent Network \(DCRNN\)\[[9](https://arxiv.org/html/2607.06614#bib.bib9)\]combined diffusion convolution with a sequence\-to\-sequence architecture, achieving the first spatio\-temporal prediction on graphs\. The Spatio\-Temporal Graph Convolutional Network \(STGCN\)\[[10](https://arxiv.org/html/2607.06614#bib.bib10)\]utilized alternating stacks of 1D temporal convolutions and graph convolutions, significantly improving computational efficiency\.

Subsequent studies further explored dynamic graph modeling: ASTGCN introduced attention mechanisms to adaptively adjust spatial weights\[[11](https://arxiv.org/html/2607.06614#bib.bib11)\]; GMAN designed an encoder\-decoder structure with spatio\-temporal attention modules to handle long\-range dependencies\[[12](https://arxiv.org/html/2607.06614#bib.bib12)\]\. For large\-scale dynamic graphs, TrafficStream proposed a streaming GNN framework with continual learning\[[13](https://arxiv.org/html/2607.06614#bib.bib13)\]\. More recent works such as Graph WaveNet\[[14](https://arxiv.org/html/2607.06614#bib.bib14)\]and AGCRN\[[15](https://arxiv.org/html/2607.06614#bib.bib15)\]incorporate adaptive graph learning to capture hidden spatial dependencies without relying on a predefined graph structure\. These methods excel in modeling local spatio\-temporal dependencies but are often limited by fixed or static graph structures and still struggle with capturing global long\-range interactions efficiently\.

### 2\.2Attention Mechanisms and Transformer\-based Models

The attention mechanism, first achieving breakthroughs in natural language processing\[[16](https://arxiv.org/html/2607.06614#bib.bib16)\], was soon introduced to spatio\-temporal forecasting tasks\. Transformers, with their self\-attention capability to directly model dependencies between any two positions, have shown strong potential in traffic flow prediction\[[17](https://arxiv.org/html/2607.06614#bib.bib17)\]\. However, the standard self\-attention mechanism incurs a computational complexity of𝒪​\(\(N​T\)2\)\\mathcal\{O\}\(\(NT\)^\{2\}\), whereNNis the number of nodes andTTis the number of time steps, making it prohibitive for city\-scale networks\.

To mitigate this, several efficient attention variants have been proposed:

- •Sparse attentionrestricts each query to attend to only a subset of keys, reducing complexity but potentially sacrificing global context\[[18](https://arxiv.org/html/2607.06614#bib.bib18)\]\.
- •Separated attentionperforms spatial and temporal attention separately, e\.g\., first attending among nodes per time step \(𝒪​\(T​N2\)\\mathcal\{O\}\(TN^\{2\}\)\) and then along time per node \(𝒪​\(N​T2\)\\mathcal\{O\}\(NT^\{2\}\)\)\. While reducing cost compared to joint attention, it still involves quadratic terms inNNorTT, which remains problematic when either dimension is large\[[12](https://arxiv.org/html/2607.06614#bib.bib12),[11](https://arxiv.org/html/2607.06614#bib.bib11)\]\.
- •Linear attentionreplaces the Softmax similarity with kernel functions to achieve linear complexity, but often suffers from degraded expressiveness and difficulty in capturing sharp attention patterns\[[19](https://arxiv.org/html/2607.06614#bib.bib19),[21](https://arxiv.org/html/2607.06614#bib.bib21)\]\.

![Refer to caption](https://arxiv.org/html/2607.06614v1/params.png)Figure 2:Comparison of model performance and efficiency \(NYC, September 2025, MAE; bubble size indicates parameter count\)\.In parallel, Graph Attention Networks \(GATs\) have achieved remarkable success on graph\-structured data by assigning different weights to neighboring nodes via attention\[[22](https://arxiv.org/html/2607.06614#bib.bib22),[23](https://arxiv.org/html/2607.06614#bib.bib23),[24](https://arxiv.org/html/2607.06614#bib.bib24)\]\. In spatio\-temporal forecasting, many studies combine GAT with recurrent or convolutional networks; ASTGCN\[[11](https://arxiv.org/html/2607.06614#bib.bib11)\]and GMAN\[[12](https://arxiv.org/html/2607.06614#bib.bib12)\]utilize attention to enhance spatial dependency modeling, while MTGNN\[[25](https://arxiv.org/html/2607.06614#bib.bib25)\]captures multi\-scale spatio\-temporal correlations through graph attention layers\. These methods improve modeling of dynamic spatial relationships but still suffer from high computational complexity, limiting their scalability to large station networks\.

More recently, Transformer variants designed for long sequences, such as Informer\[[26](https://arxiv.org/html/2607.06614#bib.bib26)\], have been applied to time series forecasting\. However, they primarily focus on temporal dimension efficiency and do not address the joint spatio\-temporal scalability challenge\. Set Transformers\[[27](https://arxiv.org/html/2607.06614#bib.bib27)\]introduce inducing points to reduce complexity in set\-structured data, which shares conceptual similarity with our agent\-based approach, but they are not tailored to spatio\-temporal graphs and do not incorporate explicit spatial graph propagation\.

### 2\.3Design Rationale and Advantages of STAGformer

Despite the significant progress made by the aforementioned methods, existing models still struggle to balance computational efficiency and global modeling capability for large\-scale bike\-sharing networks\. STAGformer addresses these gaps by leveraging the agent attention mechanism\[[31](https://arxiv.org/html/2607.06614#bib.bib31)\]in the spatio\-temporal domain and integrating it with complementary modules\. The key design features of our approach are:

- •Linear complexity with global receptive field:The agent attention mechanism\[[31](https://arxiv.org/html/2607.06614#bib.bib31)\]replaces the quadratic self\-attention with a two\-step aggregation\-broadcasting process mediated by a small set of learnable agent tokens, reducing complexity from𝒪​\(\(N​T\)2\)\\mathcal\{O\}\(\(NT\)^\{2\}\)to𝒪​\(N​T\)\\mathcal\{O\}\(NT\)\. We extend this mechanism to spatio\-temporal data by introducing separate spatial and temporal agent tokens that independently capture cross\-station and cross\-time interactions, while preserving the Softmax attention’s ability to model arbitrary long\-range dependencies—unlike sparse or kernel\-based linear attention methods that may compromise global information exchange\.
- •Preservation of expressiveness:Unlike linear attention variants that replace Softmax with kernel functions and often sacrifice representational capacity, STAGformer inherits the full Softmax attention in both agent\-attention steps and employs a depthwise convolution \(DWC\) residual branch\[[31](https://arxiv.org/html/2607.06614#bib.bib31)\]to compensate for potential feature diversity loss\. This design ensures that the efficiency gains do not come at the cost of predictive accuracy\.
- •Integration of local and global modules:While agent attention captures global dependencies, STAGformer also incorporates a graph propagation module for explicit spatial neighbor aggregation and a temporal convolution module for local temporal pattern extraction\. This hybrid design complements the global modeling with fine\-grained local information, which is often missing in pure Transformer architectures\.
- •Multi\-source feature fusion:STAGformer seamlessly integrates dynamic node features with external factors \(weather, time, POIs\) through a dedicated encoding scheme, enhancing its ability to capture contextual influences ignored by many existing models\.

In summary, STAGformer provides a scalable and accurate solution for micro\-mobility demand forecasting by combining the strengths of graph\-based local modeling and attention\-based global modeling, all with linear computational complexity\. The experimental results on real\-world datasets demonstrate its superiority over a wide range of baseline methods, confirming the effectiveness of the proposed approach\.

## 3Preliminary

### 3\.1Bike\-sharing System

An urban bike\-sharing network can be modeled as a graph𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\), where𝒱=\{v1,v2,…,vN\}\\mathcal\{V\}=\\\{v\_\{1\},v\_\{2\},\\dots,v\_\{N\}\\\}denotes the set ofNNrental stations andℰ\\mathcal\{E\}represents the spatial connections between stations\. At each time steptt, every stationvi∈𝒱v\_\{i\}\\in\\mathcal\{V\}records multi\-dimensional status information, such as the number of rentals, returns, and available bikes, forming a node feature vector𝐱i,t∈ℝFn\\mathbf\{x\}\_\{i,t\}\\in\\mathbb\{R\}^\{F\_\{n\}\}\. Aggregating features of all stations at timettyields a matrix𝐗t∈ℝN×Fn\\mathbf\{X\}\_\{t\}\\in\\mathbb\{R\}^\{N\\times F\_\{n\}\}, and the complete historical observation sequence can be represented as a 4D tensor𝒳∈ℝB×T×N×Fn\\mathcal\{X\}\\in\\mathbb\{R\}^\{B\\times T\\times N\\times F\_\{n\}\}, whereBBis the batch size andTTis the length of the historical time window\.

In addition to station status, bike\-sharing demand is influenced by various external factors, such as weather conditions \(temperature, precipitation, wind speed\) and temporal information \(hour of the day, day of the week, holidays\)\. Let𝐠t∈ℝFg\\mathbf\{g\}\_\{t\}\\in\\mathbb\{R\}^\{F\_\{g\}\}denote the global feature vector at time steptt; then the sequence of external factors can be expressed as𝒢∈ℝB×T×Fg\\mathcal\{G\}\\in\\mathbb\{R\}^\{B\\times T\\times F\_\{g\}\}\.

### 3\.2Problem Definition

We adopt a sliding window prediction paradigm\. Given a historical observation window of lengthTTand a prediction horizonHH, the goal is to learn a mapping functionffthat projects historical observations onto future station\-level demands overHHtime steps\. For an arbitrary batch, the problem is formalized as:

𝒴^t\+1:t\+H=f​\(𝒳t−T\+1:t,𝒢t−T\+1:t,𝐀ext\),\\hat\{\\mathcal\{Y\}\}\_\{t\+1:t\+H\}=f\\left\(\\mathcal\{X\}\_\{t\-T\+1:t\},\\mathcal\{G\}\_\{t\-T\+1:t\},\\mathbf\{A\}\_\{\\text\{ext\}\}\\right\),\(1\)
where:𝒴^t\+1:t\+H∈ℝB×H×N×D\\hat\{\\mathcal\{Y\}\}\_\{t\+1:t\+H\}\\in\\mathbb\{R\}^\{B\\times H\\times N\\times D\}denotes the predictedDD\-dimensional targets \(e\.g\., rental and return counts\) for each station over the futureHHsteps;𝒳t−T\+1:t∈ℝB×T×N×Fn\\mathcal\{X\}\_\{t\-T\+1:t\}\\in\\mathbb\{R\}^\{B\\times T\\times N\\times F\_\{n\}\}is the historical node features within the observation window;𝒢t−T\+1:t∈ℝB×T×Fg\\mathcal\{G\}\_\{t\-T\+1:t\}\\in\\mathbb\{R\}^\{B\\times T\\times F\_\{g\}\}is the global features \(including weather and temporal information\) within the observation window;𝐀ext∈ℝN×N\\mathbf\{A\}\_\{\\text\{ext\}\}\\in\\mathbb\{R\}^\{N\\times N\}is an external adjacency matrix \(e\.g\., road network distance matrix\)\.

## 4Methodology

![Refer to caption](https://arxiv.org/html/2607.06614v1/heng.png)Figure 3:Overall architecture ofSpatio\-TemporalAgentGraph Transformer\(STAGformer\)\.To improve the accuracy and robustness of station\-level bike\-sharing demand forecasting, we propose a Spatio\-Temporal Agent Graph Transformer \(STAGformer\) based on agent attention\. The model constructs multi\-level feature extraction modules to capture temporal dependencies, spatial correlations, and global spatio\-temporal interactions, while reducing the quadratic complexity of standard self\-attention to linear through the agent attention mechanism\. The overall framework, illustrated in Fig\.[3](https://arxiv.org/html/2607.06614#S4.F3), comprises four core modules: a spatio\-temporal feature encoder that projects raw inputs into high\-dimensional representations and injects positional information; a graph propagation module that explicitly aggregates influences from spatial neighbors using the graph structure; a temporal convolution module that models local temporal dependencies; and an agent attention module that efficiently captures global spatio\-temporal correlations with linear complexity\. Finally, the fused features are passed through an output layer to generate future station\-level demand predictions\.

### 4\.1Spatio\-Temporal Feature Encoding

The model input consists of two types of sequential data: dynamic node features𝒳∈ℝB×T×N×Fn\\mathcal\{X\}\\in\\mathbb\{R\}^\{B\\times T\\times N\\times F\_\{n\}\}and global features𝒢∈ℝB×T×Fg\\mathcal\{G\}\\in\\mathbb\{R\}^\{B\\times T\\times F\_\{g\}\}\. The feature encoding process is as follows:

1. 1\.Linear projection: Apply linear transformations to node features and global features separately, mapping them to a hidden dimensiondd: 𝐇node=𝒳​𝐖node\+𝐛node∈ℝB×T×N×d,\\mathbf\{H\}\_\{\\text\{node\}\}=\\mathcal\{X\}\\mathbf\{W\}\_\{\\text\{node\}\}\+\\mathbf\{b\}\_\{\\text\{node\}\}\\in\\mathbb\{R\}^\{B\\times T\\times N\\times d\},\(2\)𝐇global=𝒢​𝐖global\+𝐛global∈ℝB×T×d\.\\mathbf\{H\}\_\{\\text\{global\}\}=\\mathcal\{G\}\\mathbf\{W\}\_\{\\text\{global\}\}\+\\mathbf\{b\}\_\{\\text\{global\}\}\\in\\mathbb\{R\}^\{B\\times T\\times d\}\.\(3\)
2. 2\.Global feature expansion and concatenation: Replicate𝐇global\\mathbf\{H\}\_\{\\text\{global\}\}along the node dimensionNNtimes to obtain𝐇globalexp∈ℝB×T×N×d\\mathbf\{H\}\_\{\\text\{global\}\}^\{\\text\{exp\}\}\\in\\mathbb\{R\}^\{B\\times T\\times N\\times d\}, then concatenate it with the node features along the feature dimension: 𝐇=\[𝐇node∥𝐇globalexp\]∈ℝB×T×N×2​d,\\mathbf\{H\}=\[\\mathbf\{H\}\_\{\\text\{node\}\}\\parallel\\mathbf\{H\}\_\{\\text\{global\}\}^\{\\text\{exp\}\}\]\\in\\mathbb\{R\}^\{B\\times T\\times N\\times 2d\},\(4\)where∥\\paralleldenotes concatenation\. For brevity, we denoteC=2​dC=2das the unified feature dimension\.
3. 3\.Positional encoding: To incorporate temporal and spatial order information, we introduce learnable temporal positional encodings𝐏time∈ℝ1×T×1×C\\mathbf\{P\}\_\{\\text\{time\}\}\\in\\mathbb\{R\}^\{1\\times T\\times 1\\times C\}and spatial positional encodings𝐏space∈ℝ1×1×N×C\\mathbf\{P\}\_\{\\text\{space\}\}\\in\\mathbb\{R\}^\{1\\times 1\\times N\\times C\}\. These are added via broadcasting: 𝐇=𝐇\+𝐏time\+𝐏space\.\\mathbf\{H\}=\\mathbf\{H\}\+\\mathbf\{P\}\_\{\\text\{time\}\}\+\\mathbf\{P\}\_\{\\text\{space\}\}\.\(5\)The positional encoding parameters are initialized to zero and optimized jointly with the model\.
4. 4\.Dimension rearrangement: The tensor shape is kept asℝB×T×N×C\\mathbb\{R\}^\{B\\times T\\times N\\times C\}to match the input format of subsequent modules\.

Thus, the raw input is encoded into a feature tensor𝐇\\mathbf\{H\}containing spatio\-temporal information, ready for the following modules\.

### 4\.2Graph Propagating and Temporal Convolution

#### 4\.2\.1Graph Propagating

To explicitly leverage graph structural information, we introduce a graph propagation module\. The graph structure can be given by an external adjacency matrix𝐀ext∈ℝN×N\\mathbf\{A\}\_\{\\text\{ext\}\}\\in\\mathbb\{R\}^\{N\\times N\}or learned adaptively\. Adaptive graph learning uses learnable node embeddings𝐄∈ℝN×da\\mathbf\{E\}\\in\\mathbb\{R\}^\{N\\times d\_\{a\}\}to compute similarities via inner product, followed by ReLU activation, division by a learnable temperature parameterτ\\tau, and Softmax normalization:

𝐀^=Softmax​\(ReLU​\(𝐄𝐄⊤\)τ\)\.\\hat\{\\mathbf\{A\}\}=\\text\{Softmax\}\\left\(\\frac\{\\text\{ReLU\}\(\\mathbf\{E\}\\mathbf\{E\}^\{\\top\}\)\}\{\\tau\}\\right\)\.\(6\)
Given the normalized adjacency matrix, we performKK\-step graph propagation\. For the feature tensor𝐇∈ℝB×T×N×C\\mathbf\{H\}\\in\\mathbb\{R\}^\{B\\times T\\times N\\times C\}, propagation is applied along the node dimension:

𝐇\(k\)=𝐀^k​𝐇,k=0,1,…,K,\\mathbf\{H\}^\{\(k\)\}=\\hat\{\\mathbf\{A\}\}^\{k\}\\mathbf\{H\},\\quad k=0,1,\\dots,K,\(7\)where𝐀^0=𝐈\\hat\{\\mathbf\{A\}\}^\{0\}=\\mathbf\{I\}and𝐀^k\\hat\{\\mathbf\{A\}\}^\{k\}denotes the adjacency matrix afterkkpropagation steps \(implemented via matrix multiplication\)\. In practice,𝐀^k​𝐇\\hat\{\\mathbf\{A\}\}^\{k\}\\mathbf\{H\}is realized by repeatedly applying𝐀^\\hat\{\\mathbf\{A\}\}to𝐇\\mathbf\{H\}along the node dimension\.

Higher\-order features \(k≥1k\\geq 1\) are fused using learnable weights\{αk\}k=1K\\\{\\alpha\_\{k\}\\\}\_\{k=1\}^\{K\}, normalized by Softmax:

𝐇prop=∑k=1Kαk​𝐇\(k\)\.\\mathbf\{H\}\_\{\\text\{prop\}\}=\\sum\_\{k=1\}^\{K\}\\alpha\_\{k\}\\mathbf\{H\}^\{\(k\)\}\.\(8\)

#### 4\.2\.2Temporal Convolution Module

To capture local temporal dependencies for each station, we introduce a lightweight temporal convolution module\. This module applies 1D convolutions along the time dimension to extract short\-term variation patterns, and employs residual connections to enhance training stability\.

Given the feature tensor𝐇∈ℝB×T×N×C\\mathbf\{H\}\\in\\mathbb\{R\}^\{B\\times T\\times N\\times C\}, the temporal convolution module first reshapes it to\(B​N\)×C×T\(BN\)\\times C\\times T, enabling parallel processing of all stations’ time series along the merged batch\-station dimension\. A 1D convolution \(either standard or depthwise separable\) is then applied:

𝐇conv=Conv1Dkt,groups=g​\(𝐇reshaped\),\\mathbf\{H\}\_\{\\text\{conv\}\}=\\text\{Conv1D\}\_\{k\_\{t\},\\text\{groups\}=g\}\\left\(\\mathbf\{H\}\_\{\\text\{reshaped\}\}\\right\),\(9\)where the kernel sizektk\_\{t\}controls the local receptive field; wheng=Cg=C, this becomes a depthwise separable convolution\. The convolution output is sequentially passed through batch normalization, GELU activation, and Dropout:

𝐇conv=Dropout​\(GELU​\(BN​\(𝐇conv\)\)\)\.\\mathbf\{H\}\_\{\\text\{conv\}\}=\\text\{Dropout\}\\left\(\\text\{GELU\}\\left\(\\text\{BN\}\(\\mathbf\{H\}\_\{\\text\{conv\}\}\)\\right\)\\right\)\.\(10\)
This module can be stacked for multiple layers to progressively capture local temporal features at different scales\. The final output tensor retains the full spatio\-temporal structure and can be directly fed into the subsequent agent attention module\.

### 4\.3Spatio\-Temporal Agent Attention

In spatio\-temporal forecasting tasks, traditional self\-attention faces a trade\-off between computational complexity and long\-range modeling capability\. For traffic forecasting, the input typically comprisesTTtime steps andNNspatial nodes\. Directly applying global self\-attention to allT×NT\\times Ntokens incurs a complexity of𝒪​\(\(T​N\)2​C\)\\mathcal\{O\}\(\(TN\)^\{2\}C\), which is infeasible for large\-scale graphs\. Even with separated spatial attention \(self\-attention amongNNnodes at each time step\) and temporal attention \(self\-attention along time for each node\), the total complexity remains𝒪​\(T​N2​C\+N​T2​C\)\\mathcal\{O\}\(TN^\{2\}C\+NT^\{2\}C\), which is still costly whenNNorTTis large\. To address this, we employ the agent attention mechanism\[[31](https://arxiv.org/html/2607.06614#bib.bib31)\]in the spatio\-temporal setting\. Specifically, we introduce separate sets of spatial and temporal agent tokens, and decompose the agent attention into two symmetric branches that respectively model cross\-station and cross\-time dependencies\. This design achieves efficient global modeling with linear complexity while preserving the expressiveness of Softmax attention\.

#### 4\.3\.1General Self\-Attention Module

Existing Transformer\-based spatio\-temporal forecasting methods typically adopt two attention paradigms\. One is joint spatio\-temporal attention, which flattens the spatio\-temporal positions intoN×TN\\times Ttokens and computes self\-attention among all tokens\. Here,𝐐,𝐊,𝐕\\mathbf\{Q\},\\mathbf\{K\},\\mathbf\{V\}are obtained by linearly projecting𝒳\\mathcal\{X\}\. The complexity of this operation is𝒪​\(\(N​T\)2​d\)\\mathcal\{O\}\(\(NT\)^\{2\}d\), and memory consumption grows sharply with the number of stations or the time window length\.

The second paradigm is separated spatio\-temporal attention, which performs spatial and temporal attention independently\. For spatial self\-attention, the batch and time dimensions are merged into\(B⋅T,N,C\)\(B\\cdot T,N,C\), yielding a complexity of𝒪​\(N2​T​d\)\\mathcal\{O\}\(N^\{2\}Td\)\. Subsequently, temporal self\-attention is applied independently for each node, resulting in𝒪​\(T2​N​d\)\\mathcal\{O\}\(T^\{2\}Nd\)\. The total complexity is𝒪​\(N2​T​d\+T2​N​d\)\\mathcal\{O\}\(N^\{2\}Td\+T^\{2\}Nd\)\.

Compared to the first approach, separated attention effectively models spatio\-temporal dependencies separately, avoiding the high computational cost and memory footprint of joint attention\. However, it still encounters a computational bottleneck whenNNorTTis large\.

Attn​\(𝒳\)=Softmax​\(𝐐𝐊⊤dk\)​𝐕,\\text\{Attn\}\(\\mathcal\{X\}\)=\\text\{Softmax\}\\left\(\\frac\{\\mathbf\{Q\}\\mathbf\{K\}^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\}\\right\)\\mathbf\{V\},\(11\)

#### 4\.3\.2Basic Form of Spatio\-Temporal Agent Attention

For an input feature tensorX∈ℝB×T×N×CX\\in\\mathbb\{R\}^\{B\\times T\\times N\\times C\}, we first view it asB×TB\\times Tindependent spatial graphs, each containingNNnodes\. Linear projections yield queriesQQ, keysKK, and valuesV∈ℝB×T×N×CV\\in\\mathbb\{R\}^\{B\\times T\\times N\\times C\}\. To reduce computation, we introduce a small set of learnable spatial agent tokensPs∈ℝ1×ns×CP\_\{s\}\\in\\mathbb\{R\}^\{1\\times n\_\{s\}\\times C\}\(withns≪Nn\_\{s\}\\ll N\), which are broadcast to match the batch dimension\. The spatial agent attention is computed in two steps:

1. 1\.Agent aggregation: The agent tokensPsP\_\{s\}serve as queries, performing Softmax attention with keysKKand valuesVVto aggregate global information into agent contextVPV\_\{P\}: VP=Softmax​\(\(Ps\)​KT/d\+Bagent\)​V,V\_\{P\}=\\text\{Softmax\}\\big\(\(P\_\{s\}\)K^\{T\}/\\sqrt\{d\}\+B\_\{\\text\{agent\}\}\\big\)V,\(12\)whereBagent∈ℝ1×H×ns×1B\_\{\\text\{agent\}\}\\in\\mathbb\{R\}^\{1\\times H\\times n\_\{s\}\\times 1\}is an agent bias \(HHis the number of attention heads\) andd=C/Hd=C/His the dimension per head\.
2. 2\.Agent broadcasting: The original queriesQQattend to the agent tokensPsP\_\{s\}\(now serving as keys\) and the agent contextVPV\_\{P\}in a second Softmax attention, broadcasting the aggregated global information back to each node: Os=Softmax​\(Q​\(Ps\)T/d\+Bnode\)​VP,O\_\{s\}=\\text\{Softmax\}\\big\(Q\(P\_\{s\}\)^\{T\}/\\sqrt\{d\}\+B\_\{\\text\{node\}\}\\big\)V\_\{P\},\(13\)whereBnode∈ℝ1×H×1×nsB\_\{\\text\{node\}\}\\in\\mathbb\{R\}^\{1\\times H\\times 1\\times n\_\{s\}\}is another set of agent biases\.

To preserve feature diversity, we introduce a depthwise convolution \(DWC\) module applied to the valuesVV, and add it as a residual branch to the output:

Os=Os\+DWC⁡\(V\),O\_\{s\}=O\_\{s\}\+\\operatorname\{DWC\}\(V\),\(14\)where DWC employs a depthwise separable 1D convolution with kernel size 3 \(Conv1d,groups = C\) sliding along the node dimension, effectively recovering feature diversity that may be lost in the two\-step attention\.

Similarly, for the temporal dimension, we introducentn\_\{t\}learnable temporal agent tokensPt∈ℝ1×nt×CP\_\{t\}\\in\\mathbb\{R\}^\{1\\times n\_\{t\}\\times C\}and transpose the input tensor to\(B,N,T,C\)\(B,N,T,C\), yieldingB×NB\\times Ntime series of lengthTT\. The temporal agent attention process is symmetric to the spatial version, using independent temporal agent biasesBt,agentB\_\{t,\\text\{agent\}\}andBt,nodeB\_\{t,\\text\{node\}\}to obtain the temporal attention outputOtO\_\{t\}\. DWC is not used in the temporal branch because the feature dimension is typically shorter and the loss of feature diversity is less significant\.

#### 4\.3\.3Overall Module Structure

In the GraphAgentAttention module, when the input is a 4D tensor\(B,T,N,C\)\(B,T,N,C\)and temporal attention is enabled, the forward process is:

1. 1\.Apply spatial agent attention to the input, yieldingXspatial=SpatialAttn​\(X\)X\_\{\\text\{spatial\}\}=\\text\{SpatialAttn\}\(X\)\.
2. 2\.Apply temporal agent attention toXspatialX\_\{\\text\{spatial\}\}, yieldingXtemporal=TemporalAttn​\(Xspatial\)X\_\{\\text\{temporal\}\}=\\text\{TemporalAttn\}\(X\_\{\\text\{spatial\}\}\)\.
3. 3\.The final output is the residual sum:Xout=Xspatial\+XtemporalX\_\{\\text\{out\}\}=X\_\{\\text\{spatial\}\}\+X\_\{\\text\{temporal\}\}\.

This module can be stacked for multiple layers, with residual connections and layer normalization to stabilize training\.

#### 4\.3\.4Complexity Analysis

In spatial agent attention, the two attention steps each have a complexity of𝒪​\(B​T​N​ns​C\)\\mathcal\{O\}\(BTNn\_\{s\}C\)\. Including the projections and DWC, the total complexity is:

Ω=4​B​T​N​C2⏟projections\+B​T​N​C⏟agent acquisition\+2​B​T​N​ns​C⏟agent attention\+k2​B​T​N​C⏟DWC,\\Omega=\\underbrace\{4BTNC^\{2\}\}\_\{\\text\{projections\}\}\+\\underbrace\{BTNC\}\_\{\\text\{agent acquisition\}\}\+\\underbrace\{2BTNn\_\{s\}C\}\_\{\\text\{agent attention\}\}\+\\underbrace\{k^\{2\}BTNC\}\_\{\\text\{DWC\}\},\(15\)wherek=3k=3is the DWC kernel size\. Temporal agent attention has a similar complexity of𝒪​\(B​N​T​nt​C\)\\mathcal\{O\}\(BNTn\_\{t\}C\)\. Sincensn\_\{s\}andntn\_\{t\}are constants much smaller thanNNandTT\(e\.g\.,ns=32n\_\{s\}=32,nt=32n\_\{t\}=32\), the overall complexity is linear in bothNNandTT, i\.e\.,𝒪​\(N​T\)\\mathcal\{O\}\(NT\)\. In contrast, standard spatio\-temporal self\-attention \(global or separated\) is at least𝒪​\(N2​T\+N​T2\)\\mathcal\{O\}\(N^\{2\}T\+NT^\{2\}\), which becomes prohibitively expensive for largeNNorTT\.

#### 4\.3\.5Comparison with Standard Spatio\-Temporal Attention

Standard spatio\-temporal attention typically adopts two paradigms:

- •Global attention: Flattens the spatio\-temporal positions intoN​TNTtokens and computes fully\-connected self\-attention, with complexity𝒪​\(\(N​T\)2​C\)\\mathcal\{O\}\(\(NT\)^\{2\}C\)\. It maintains a full receptive field but is infeasible for large\-scale data\.
- •Separated attention: First computes spatial attention independently for each time step \(𝒪​\(T​N2​C\)\\mathcal\{O\}\(TN^\{2\}C\)\), then temporal attention independently for each node \(𝒪​\(N​T2​C\)\\mathcal\{O\}\(NT^\{2\}C\)\)\. Although complexity is reduced to quadratic, it still involvesN2N^\{2\}andT2T^\{2\}terms\.

In contrast, spatio\-temporal agent attention reduces the quadratic complexity to linear by introducing a small number of learnable agent tokens\. Specifically, the spatial and temporal branches have complexities of𝒪​\(B​T​N​ns​C\)\\mathcal\{O\}\(BTNn\_\{s\}C\)and𝒪​\(B​N​T​nt​C\)\\mathcal\{O\}\(BNTn\_\{t\}C\), respectively, wherens,nt≪N,Tn\_\{s\},n\_\{t\}\\ll N,Tare constants\. Thus, the overall complexity is linear inNNandTT, i\.e\.,𝒪​\(N​T\)\\mathcal\{O\}\(NT\), significantly outperforming the quadratic complexity of standard attention\. Moreover, the two\-step Softmax mechanism preserves global information exchange; the agent biases allow flexible encoding of relative positional relationships between nodes and time steps; and the DWC module compensates for the inherent feature diversity loss in linear attention, striking a favorable balance between computational efficiency and expressiveness\.

## 5Experiment

In this section, we present the implementation details, dataset descriptions, evaluation metrics, and baseline methods for comparison\. We then conduct experiments to assess the performance of STAGformer, including comparisons with baseline methods, ablation studies, and case studies under different scenarios\.

### 5\.1Data Description

To evaluate our model, we collected and integrated data from two cities, combining bike\-sharing operational records with meteorological data, points of interest \(POIs\), and road network information\. These data collectively provide the necessary input features for model training, as summarized in Table[1](https://arxiv.org/html/2607.06614#S5.T1)\.

Bike\-sharing trip data:This dataset contains operational records from the NYC Citi\-Bike111[https://ride\.citibikenyc\.com/system\-data](https://ride.citibikenyc.com/system-data)and Chicago Divvy\-Bike222[https://divvybikes\.com/system\-data](https://divvybikes.com/system-data)systems\. The dataset primarily consists of two parts: \(i\) rental and return trip records for each station, and \(ii\) station\-level details such as spatial and temporal information\.

Meteorological feature data:Weather data for both cities were collected from the NOAA National Weather Service database333[https://www\.noaa\.gov/](https://www.noaa.gov/), providing historical daily weather records\. Each record describes basic weather conditions and categorizes daily weather into four main types: sunny, cloudy, rainy/snowy, and extreme weather\.

Road network and POI data:POI data were obtained from OpenStreetMap \(OSM\)444[https://www\.openstreetmap\.org/](https://www.openstreetmap.org/), offering detailed information around stations\. Table[1](https://arxiv.org/html/2607.06614#S5.T1)lists the main POI types and their counts for each city\. We adopted POIs within a 150\-meter radius of each station as input features for the model\. Additionally, road network and city boundary data were collected from the open data platforms of New York City555[https://opendata\.cityofnewyork\.us/](https://opendata.cityofnewyork.us/)and Chicago666[https://data\.cityofchicago\.org/](https://data.cityofchicago.org/)for supplementary analysis\.

Table 1:POIs Data from OpenStreetMap
### 5\.2Implementation

Preparation\.We preprocessed the bike\-sharing trip data for New York City and Chicago, including determining station spatio\-temporal information, data cleaning, missing value imputation, outlier detection, and normalization\. Weather data were aligned with trip data at an hourly temporal resolution\. Road network and POI data for each city were merged to form an external adjacency matrix for the graph propagation module\. Finally, all data were consolidated into a unified dataset\.

Device\.Experiments were conducted on the AutoDL platform, with an Intel\(R\) Xeon\(R\) Platinum 8260 CPU @ 2\.40GHz, an RTX 4080 SUPER GPU \(32GB VRAM\), and 60GB of RAM\. The software environment included PyTorch 2\.8\.0, Python 3\.12, CUDA 12\.8, and Ubuntu 22\.04\. The model was trained using the AdamW optimizer with an initial learning rate of1×10−31\\times 10^\{\-3\}, batch size of 8, for 300 epochs, employing early stopping to prevent overfitting\.

### 5\.3Evaluation Metrics

Accuracy metrics\.We evaluate model performance using Root Mean Square Error \(RMSE\) and Mean Absolute Error \(MAE\)\. These metrics collectively provide a robust framework to assess the model’s capability to capture subtle variations in the data\.

RMSE=1n​∑i=1n\(Yi−Yi^\)2\\text\{RMSE\}=\\sqrt\{\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\(Y\_\{i\}\-\\hat\{Y\_\{i\}\}\)^\{2\}\}\(16\)MAE=1n​∑i=1n\|Yi−Yi^\|\.\\text\{MAE\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\|Y\_\{i\}\-\\hat\{Y\_\{i\}\}\|\.\(17\)wherey^i\\hat\{y\}\_\{i\}is the predicted value, andyiy\_\{i\}is the ground truth\.NNdenotes the number of stations\. The ground truth for stationiiis denoted asyi=\[yirented,yireturned\]y\_\{i\}=\[y\_\{i\}^\{\\text\{rented\}\},y\_\{i\}^\{\\text\{returned\}\}\], and the predicted value isy^i=\[y^irented,y^ireturned\]\\hat\{y\}\_\{i\}=\[\\hat\{y\}\_\{i\}^\{\\text\{rented\}\},\\hat\{y\}\_\{i\}^\{\\text\{returned\}\}\]\.

Computational efficiency metrics\.In addition to prediction accuracy, we evaluate computational efficiency using two widely adopted metrics: the number of parameters \(Params\) and the number of floating\-point operations \(FLOPs\)\. Params reflect model complexity and storage requirements, while FLOPs measure the computational cost during inference\. We report the total multiply\-accumulate operations \(MACs\) as computed by thethoplibrary, where 1 MAC is considered as one FLOP in our context\. For a fair comparison, all models are evaluated under the same input shape\(B,T,N,Fn\)\(B,T,N,F\_\{n\}\)and on the same hardware\. These metrics demonstrate the scalability and practical deployability of the model, especially in large\-scale micro\-mobility networks\.

Table 2:Performance of each model on two city Sharing\-bike DatasetsUnderlined values indicate the best results among compared models; bold values represent our model\.

### 5\.4Comparison to Baselines

We selected nine representative methods for comparison in this study, described as follows:

- •Linear Regression\[[20](https://arxiv.org/html/2607.06614#bib.bib20)\]: Predicts bike demand using weather and spatial features with neighborhood\-level aggregation, serving as a simple yet interpretable baseline\.
- •Spatial Regression\[[37](https://arxiv.org/html/2607.06614#bib.bib37)\]: Models spatial lag and error structures to capture spatio\-temporal dependencies in bike\-sharing demand, accounting for interactions among neighboring stations\.
- •GRU\[[7](https://arxiv.org/html/2607.06614#bib.bib7)\]: A gated recurrent neural network that captures temporal dependencies in sequential data for each station independently, widely used for time series forecasting\.
- •Transformer\[[16](https://arxiv.org/html/2607.06614#bib.bib16)\]: A self\-attention based architecture that models long\-range dependencies across all stations and time steps jointly, offering strong expressiveness but high computational cost\.
- •GAT\[[22](https://arxiv.org/html/2607.06614#bib.bib22)\]: Applies attention over graph neighborhoods to aggregate spatial information with learnable importance weights, enhancing the modeling of dynamic spatial relationships\.
- •BikeMAN\[[38](https://arxiv.org/html/2607.06614#bib.bib38)\]: A multi\-level spatio\-temporal attention neural network for station\-level bike traffic prediction\. It employs an encoder\-decoder architecture with spatial attention on input features and temporal attention on hidden states, integrating weather and POI data to capture complex correlations\.
- •STAEformer\[[40](https://arxiv.org/html/2607.06614#bib.bib40)\]: A Transformer variant with spatio\-temporal adaptive embeddings that enhances traffic forecasting performance by learning node\-specific temporal patterns without complex architectural modifications\.
- •T\-STAR\[[43](https://arxiv.org/html/2607.06614#bib.bib43)\]: A context\-aware Transformer framework for short\-term probabilistic demand forecasting in dock\-based shared micro\-mobility systems, integrating external factors and uncertainty estimation\.
- •BGM\[[33](https://arxiv.org/html/2607.06614#bib.bib33)\]: A dynamic graph modeling approach for demand prediction in expanding bike\-sharing networks, adapting to new stations and evolving mobility patterns through graph structure learning\.

### 5\.5Ablation Study

To thoroughly validate the effectiveness of each core module in STAGformer, we conducted a series of ablation experiments on both the New York and Chicago datasets, examining the change in model performance after removing specific modules\. The ablation variants are defined as follows:

- •w/o E \(without External Features\):Removes global external features \(weather, time, POIs, etc\.\), using only historical station trip data as input\.
- •w/o G \(without Graph Propagation\):Removes the graph propagation module, i\.e\., no explicit aggregation of spatial neighbor information using the graph structure, retaining only temporal convolution and agent attention\.
- •w/o TC \(without Time Convolution\):Removes the temporal convolution module, relying solely on graph propagation and agent attention to capture temporal dependencies\.
- •w/o Attn \(without Agent Attention\):Removes the agent attention module, retaining only graph propagation and temporal convolution; thus, the model cannot model global long\-range spatio\-temporal interactions\.
- •STAGformer:The full model incorporating all modules\.

![Refer to caption](https://arxiv.org/html/2607.06614v1/arrival.png)

![Refer to caption](https://arxiv.org/html/2607.06614v1/departure.png)

Figure 4:Arrival and departure patterns of bike\-sharing stations in New York City\.![Refer to caption](https://arxiv.org/html/2607.06614v1/ablation_study.png)Figure 5:Ablation study results on New York and Chicago datasets\. Bars represent error metrics \(MAE, MSE, RMSE\) and the red line shows the R2score\.The experimental results, as shown in the figures, lead to the following conclusions:

1. 1\.Importance of external features:In the New York dataset, w/o External increased RMSE/MAE by 32\.9% / 130\.7% compared to the full model; in Chicago, the increases were 25\.2% / 83\.2%\. This indicates that external factors such as weather, time, and POIs significantly influence bike\-sharing demand, and fusing multi\-source features effectively improves prediction accuracy\.
2. 2\.Role of graph propagation module:w/o Graph led to performance degradation in both datasets \(New York: 0\.8% / 2\.2%; Chicago: 0\.7% / 5\.3%\)\. This demonstrates that explicitly aggregating spatial neighbor information using the graph structure helps capture local spatial correlations, especially in areas with dense road network topology\.
3. 3\.Necessity of temporal convolution module:The error increases for w/o TConv \(New York: 7\.0% / 4\.6%; Chicago: 2\.8% / 6\.6%\) confirm the effectiveness of lightweight temporal convolution in extracting short\-term temporal patterns for each station\. Relying solely on attention mechanisms may overlook local details\.
4. 4\.Contribution of agent attention module:The most significant performance drop occurred for w/o Agent \(New York: 37\.9% / 47\.9%; Chicago: 8\.9% / 18\.4%\)\. This validates the critical role of agent attention in modeling global spatio\-temporal dependencies, such as long\-distance station coordination and periodic patterns\. Removing this module reduces the model to only local graph propagation and temporal convolution, making it unable to capture complex global interactions\.

In summary, the four modules of STAGformer are complementary and indispensable\. Global external features, graph propagation, temporal convolution, and agent attention characterize the spatio\-temporal properties of bike\-sharing demand from different dimensions, collectively ensuring high accuracy and robustness\.

### 5\.6Case Study

In addition to the Chicago case presented in the introduction, we further validate the visualization of STAGformer on the New York City dataset\. To intuitively illustrate the spatio\-temporal heterogeneity of bike\-sharing systems and verify STAGformer’s ability to capture complex patterns, we plotted the departure and arrival distributions at 8:00 AM on a weekday in September 2025 for the NYC Citi\-Bike system, as shown in Fig\.[4](https://arxiv.org/html/2607.06614#S5.F4)\. Since displaying all approximately 2000 stations simultaneously would make the image too dense to interpret, we randomly sampled 300 stations and compared the model’s predictions with the actual trip demands\. The upper subplot shows arrival distribution, and the lower subplot shows departure distribution\. We also forecast the morning peak and evening peak, as shown in Fig\.[6](https://arxiv.org/html/2607.06614#S5.F6); the color depth represents the magnitude of rentals/returns, with red indicating high demand and blue indicating low demand\. The following typical patterns can be observed:

1. 1\.High\-intensity activity in core areas:Stations in Midtown and Lower Manhattan exhibit extremely high departure and arrival volumes, reflecting commuter demand in the central business district during morning and evening peaks\. These stations show a surge in departures during the morning peak and a surge in arrivals during the evening peak, exhibiting a pronounced tidal pattern\.
2. 2\.Spatial imbalance:In stark contrast to Manhattan, stations in Brooklyn and Queens \(scattered points around Manhattan\) have overall lower activity levels \(mostly blue or light red\), with relatively balanced departure and arrival distributions\. This spatial heterogeneity stems from functional zoning: residential areas are dominated by departures in the morning peak, while commercial areas are dominated by arrivals; internal flows in peripheral residential areas are minimal\.
3. 3\.Local coordination patterns:Within Manhattan, neighboring stations often exhibit similar rental/return patterns\. For example, stations near subway hubs maintain high turnover throughout the day, while stations in surrounding office areas show more pronounced morning and evening peaks\. This local spatial correlation underscores the necessity of the graph propagation module\.
4. 4\.Long\-range dependencies between distant stations:Although Manhattan and Brooklyn are geographically distant, stations at opposite ends of certain commuter corridors \(e\.g\., Manhattan business district and Brooklyn residential areas\) exhibit complementary departure and arrival patterns: high arrivals in Manhattan and high departures in Brooklyn during the morning peak, and the reverse during the evening peak\. Such long\-range dependencies exceed the receptive field of local graph convolution and require a global attention mechanism\.

![Refer to caption](https://arxiv.org/html/2607.06614v1/nyc_morning.png)\(a\)Morning peak
![Refer to caption](https://arxiv.org/html/2607.06614v1/nyc_evening.png)\(b\)Evening peak

Figure 6:Forecast morning peak and evening peak departure and arrival heatmaps in New York City\.STAGformer effectively captures the above patterns through the following design elements:

- •Graph propagation moduleaggregates spatial neighbor information, strengthening local correlation modeling\.
- •Agent attention modulemodels dependencies between distant stations at linear cost through spatial and temporal agents\.
- •External feature fusionencodes weather, time, and POI information, helping the model distinguish demand patterns in different functional zones\.

The experimental results show that STAGformer achieves significantly lower prediction errors than baseline models in highly dynamic areas, demonstrating its capability to model complex spatio\-temporal patterns\. This case study not only visualizes the intrinsic structure of the data but also provides intuitive evidence for the effectiveness of the model design\.

## 6Conclusion

This paper proposed STAGformer, a spatio\-temporal forecasting architecture that integrates the agent attention mechanism\[[31](https://arxiv.org/html/2607.06614#bib.bib31)\]with graph propagation and temporal convolution for station\-level bike\-sharing demand forecasting\. By introducing separate spatial and temporal agent tokens and a two\-step agent attention mechanism, the model reduces the quadratic complexity of standard self\-attention to linear, while effectively capturing long\-range dependencies across both spatial and temporal dimensions\. The architecture integrates four key components: a spatio\-temporal feature encoder that fuses dynamic node features with external global factors, a graph propagation module for aggregating spatial neighbor information, a temporal convolution module for modeling local temporal patterns, and a dedicated agent attention module that models global interactions via spatial and temporal agents\. Extensive experiments on two real\-world datasets \(NYC Citi\-Bike and Chicago Divvy\-Bike\) demonstrate that STAGformer consistently outperforms a wide range of baseline methods, achieving superior accuracy in terms of RMSE and MAE across multiple prediction horizons\. Notably, the agent attention mechanism yields linear complexity with respect to the number of stations and time steps, enabling efficient deployment in large\-scale urban networks without sacrificing the expressive power of softmax attention\.

Ablation studies further validate the contribution of each module\. Removing external features leads to significant performance drops, confirming the importance of integrating multi\-source contextual information\. The graph propagation and temporal convolution modules are shown to be essential for capturing local spatial correlations and short\-term temporal dynamics, respectively\. Most importantly, the agent attention module proves critical for modeling global spatio\-temporal dependencies; its removal causes the largest error increase, underscoring its role in enabling long\-range interactions among distant stations and capturing periodic patterns\. Together, these components form a robust and efficient framework for micro\-mobility demand forecasting\.

Despite its promising results, STAGformer has certain limitations that open avenues for future research\. First, the current model relies on a fixed graph structure derived from road network distances; incorporating dynamic graph learning that adapts to evolving mobility patterns could further improve accuracy\. Second, while the agent attention mechanism reduces computational cost, its performance may be sensitive to the number of agent tokens, which requires empirical tuning\. Future work could explore adaptive strategies for agent selection\. Additionally, extending the model to other micro\-mobility systems such as e\-scooters or ride\-hailing services, and incorporating richer external data \(e\.g\., real\-time events, social media trends\), would test its generalizability\. Finally, integrating STAGformer into a real\-time decision\-support system for bike\-sharing operators could translate predictive gains into tangible operational benefits, such as optimized rebalancing and improved user satisfaction\.

## Acknowledgement

This research is not supported by any organizations\.

## References

- \[1\]J\. Kim, H\. Kim, H\. Kim, D\. Lee, and S\. Yoon, “A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges,”Artif\. Intell\. Rev\., vol\. 58, no\. 7, p\. 216, 2025\.
- \[2\]C\. Chatfield and H\. Xing,The Analysis of Time Series: An Introduction with R, 7th ed\. Boca Raton, FL, USA: Chapman and Hall/CRC, 2019\.
- \[3\]D\. D\. Lee, P\. Pham, Y\. Largman, and A\. Ng, “Advances in neural information processing systems 22,”Neural Inf\. Process\. Syst\., vol\. 1, no\. 1, pp\. 1–11, 2009\.
- \[4\]Z\.\-H\. Zhou,Machine Learning\. Springer Nature, 2021\.
- \[5\]J\. H\. Friedman, “Greedy function approximation: a gradient boosting machine,”Ann\. Statist\., pp\. 1189–1232, 2001\.
- \[6\]S\. Hochreiter and J\. Schmidhuber, “Long short\-term memory,”Neural Comput\., vol\. 9, no\. 8, pp\. 1735–1780, 1997\.
- \[7\]K\. Cho et al\., “Learning phrase representations using RNN encoder–decoder for statistical machine translation,” inProc\. Conf\. Empir\. Methods Nat\. Lang\. Process\. \(EMNLP\), 2014, pp\. 1724–1734\.
- \[8\]J\. Zhang, Y\. Zheng, and D\. Qi, “Deep spatio\-temporal residual networks for citywide crowd flows prediction,” inProc\. AAAI Conf\. Artif\. Intell\., vol\. 31, no\. 1, 2017\.
- \[9\]Y\. Li, R\. Yu, C\. Shahabi, and Y\. Liu, “Diffusion convolutional recurrent neural network: Data\-driven traffic forecasting,”arXiv preprint arXiv:1707\.01926, 2017\.
- \[10\]B\. Yu, H\. Yin, and Z\. Zhu, “Spatio\-temporal graph convolutional networks: A deep learning framework for traffic forecasting,”arXiv preprint arXiv:1709\.04875, 2017\.
- \[11\]S\. Guo, Y\. Lin, N\. Feng, C\. Song, and H\. Wan, “Attention based spatial\-temporal graph convolutional networks for traffic flow forecasting,” inProc\. AAAI Conf\. Artif\. Intell\., vol\. 33, no\. 1, 2019, pp\. 922–929\.
- \[12\]C\. Zheng, X\. Fan, C\. Wang, and J\. Qi, “GMAN: A graph multi\-attention network for traffic prediction,” inProc\. AAAI Conf\. Artif\. Intell\., vol\. 34, no\. 1, 2020, pp\. 1234–1241\.
- \[13\]X\. Chen, J\. Wang, and K\. Xie, “TrafficStream: A streaming traffic flow forecasting framework based on graph neural networks and continual learning,”arXiv preprint arXiv:2106\.06273, 2021\.
- \[14\]Z\. Wu, S\. Pan, G\. Long, J\. Jiang, and C\. Zhang, “Graph WaveNet for deep spatial\-temporal graph modeling,”arXiv preprint arXiv:1906\.00121, 2019\.
- \[15\]L\. Bai, L\. Yao, C\. Li, X\. Wang, and C\. Wang, “Adaptive graph convolutional recurrent network for traffic forecasting,” inAdv\. Neural Inf\. Process\. Syst\., vol\. 33, 2020, pp\. 17804–17815\.
- \[16\]A\. Vaswani et al\., “Attention is all you need,” inAdv\. Neural Inf\. Process\. Syst\., vol\. 30, 2017\.
- \[17\]M\. Xu et al\., “Spatial\-temporal transformer networks for traffic flow forecasting,”arXiv preprint arXiv:2001\.02908, 2020\.
- \[18\]R\. Child, S\. Gray, A\. Radford, and I\. Sutskever, “Generating long sequences with sparse transformers,”arXiv preprint arXiv:1904\.10509, 2019\.
- \[19\]A\. Katharopoulos, A\. Vyas, N\. Pappas, and F\. Fleuret, “Transformers are RNNs: Fast autoregressive transformers with linear attention,” inProc\. Int\. Conf\. Mach\. Learn\. \(ICML\), 2020, pp\. 5156–5165\.
- \[20\]D\. Singhvi, S\. Singhvi, P\. I\. Frazier, S\. G\. Henderson, E\. O’Mahony, D\. B\. Shmoys, and D\. B\. Woodard, “Predicting bike usage for New York City’s bike sharing system,” inProc\. AAAI Workshop on Computational Sustainability, Austin, TX, USA, 2015\.
- \[21\]K\. Choromanski et al\., “Rethinking attention with performers,”arXiv preprint arXiv:2009\.14794, 2020\.
- \[22\]P\. Veličković, G\. Cucurull, A\. Casanova, A\. Romero, P\. Liò, and Y\. Bengio, “Graph attention networks,”arXiv preprint arXiv:1710\.10903, 2017\.
- \[23\]S\. Brody, U\. Alon, and E\. Yahav, “How attentive are graph attention networks?”arXiv preprint arXiv:2105\.14491, 2021\.
- \[24\]J\. Zhang, X\. Shi, J\. Xie, H\. Ma, I\. King, and D\.\-Y\. Yeung, “GaAN: Gated attention networks for learning on large and spatiotemporal graphs,”arXiv preprint arXiv:1803\.07294, 2018\.
- \[25\]Z\. Wu, S\. Pan, G\. Long, J\. Jiang, X\. Chang, and C\. Zhang, “Connecting the dots: Multivariate time series forecasting with graph neural networks,” inProc\. ACM SIGKDD Int\. Conf\. Knowl\. Discov\. Data Min\., 2020, pp\. 753–763\.
- \[26\]H\. Zhou et al\., “Informer: Beyond efficient transformer for long sequence time\-series forecasting,” inProc\. AAAI Conf\. Artif\. Intell\., vol\. 35, no\. 12, 2021, pp\. 11106–11115\.
- \[27\]J\. Lee, Y\. Lee, J\. Kim, A\. Kosiorek, S\. Choi, and Y\. W\. Teh, “Set transformer: A framework for attention\-based permutation\-invariant neural networks,” inProc\. Int\. Conf\. Mach\. Learn\. \(ICML\), 2019, pp\. 3744–3753\.
- \[28\]A\. Behroozi and A\. Edrisi, “Predicting travel demand of a bike sharing system using graph convolutional neural networks,”Public Transp\., vol\. 17, no\. 1, pp\. 281–317, 2025\.
- \[29\]R\. Rochas, A\. Furno, and N\.\-E\. El Faouzi, “Contextual data integration for bike\-sharing demand prediction with graph neural networks in degraded weather conditions,” inProc\. IEEE Int\. Conf\. Intell\. Transp\. Syst\. \(ITSC\), 2023, pp\. 5436–5441\.
- \[30\]J\. Wang, T\. Miwa, and T\. Morikawa, “A demand truncation and migration poisson model for real demand inference in free\-floating bike\-sharing system,”IEEE Trans\. Intell\. Transp\. Syst\., vol\. 24, no\. 10, pp\. 10525–10536, Oct\. 2023\.
- \[31\]D\. Han et al\., “Agent attention: On the integration of softmax and linear attention,” inProc\. Eur\. Conf\. Comput\. Vis\. \(ECCV\), 2024, pp\. 124–140\.
- \[32\]J\. Feng and H\. Liu, “An adaptive spatial\-temporal method capturing for short\-term bike\-sharing prediction,”IEEE Trans\. Intell\. Transp\. Syst\., vol\. 25, no\. 11, pp\. 16761–16774, Nov\. 2024\.
- \[33\]Y\. Zhao, H\. Wen, X\. Zhang, and M\. Luo, “BGM: Demand prediction for expanding bike\-sharing systems with dynamic graph modeling,” inProc\. Int\. Joint Conf\. Artif\. Intell\. \(IJCAI\), 2025, pp\. 10008–10016\.
- \[34\]Z\. Xiang, F\. Zeng, L\. Liu, J\. Wu, S\. Mumtaz, and V\. C\. M\. Leung, “Bike\-sharing demand prediction based on dynamic time warping and spatio\-temporal graph attention network,”IEEE Trans\. Intell\. Transp\. Syst\., 2025, to be published\.
- \[35\]S\. Jiang, Z\. Strout, B\. He, D\. Peng, P\. B\. Shull, and B\. P\. L\. Lo, “Dual stream meta learning for road surface classification and riding event detection on shared bikes,”IEEE Trans\. Syst\. Man Cybern\. Syst\., vol\. 53, no\. 11, pp\. 7188–7200, Nov\. 2023\.
- \[36\]N\. Wiedemann, T\. Uscidda, and M\. Raubal, “GeOT: a spatially explicit framework for evaluating spatio\-temporal predictions,”Int\. J\. Geogr\. Inf\. Sci\., vol\. 39, no\. 10, pp\. 2236–2266, 2025\.
- \[37\]A\. Faghih\-Imani and N\. Eluru, “Incorporating the impact of spatio\-temporal interactions on bicycle sharing system demand: A case study of New York CitiBike system,”J\. Transp\. Geogr\., vol\. 54, pp\. 218–227, 2016\.
- \[38\]X\. Yang, J\. Wang, S\. Han, and S\. He, “Micromobility flow prediction: A bike sharing station\-level study via multi\-level spatial\-temporal attention neural network,”arXiv preprint arXiv:2507\.16020, 2025\.
- \[39\]A\. Tziorvas, G\. S\. Theodoropoulos, and Y\. Theodoridis, “MoDE\-Boost: Boosting shared mobility demand with edge\-ready prediction models,”arXiv preprint arXiv:2602\.16573, 2026\.
- \[40\]H\. Liu et al\., “Spatio\-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting,” inProc\. ACM Int\. Conf\. Inf\. Knowl\. Manag\. \(CIKM\), 2023, pp\. 4125–4129\.
- \[41\]H\. Wang, J\. Chen, T\. Pan, Z\. Dong, L\. Zhang, R\. Jiang, and X\. Song, “STGformer: Efficient spatiotemporal graph transformer for traffic forecasting,”arXiv preprint arXiv:2410\.00385, 2024\.
- \[42\]Y\. Liu and Z\. Zhang, “STGFormer: Spatio\-temporal GraphFormer for 3D human pose estimation in video,”Pattern Recognit\., p\. 112239, 2025\.
- \[43\]J\. Cheng, G\. H\. de Almeida Correia, O\. Cats, and S\. S\. Azadeh, “T\-STAR: A context\-aware transformer framework for short\-term probabilistic demand forecasting in dock\-based shared micro\-mobility,”arXiv preprint arXiv:2602\.06866, 2026\.