Learning Spatio-Temporal Foundation Models from Pure Synthetic Data

# Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
Source: [https://arxiv.org/html/2607.16251](https://arxiv.org/html/2607.16251)
Yutong Feng12, Shiyuan Piao3, Yutong Xia2, Xu Liu2, Wenqi Fan4, Fugee Tsung3, See\-Kiong Ng2, Yuxuan Liang1 1Hong Kong University of Science and Technology \(Guangzhou\) 2National University of Singapore 3Hong Kong University of Science and Technology 4Hong Kong Polytechnic University ytfeng\.caspian@163\.com; shiyuan\.piao@connect\.ust\.hk yuxliang@outlook\.com Work done at Hong Kong University of Science and Technology \(Guangzhou\)\.Y\. Liang is the corresponding author\. Email: yuxliang@outlook\.com

###### Abstract

Spatio\-Temporal Foundation Models \(STFMs\) aim to learn generalizable representations of complex dynamical systems across space and time\. However, existing approaches suffer from distributional bias in real\-world pre\-training data, structural bottlenecks of autoregressive or diffusion\-based paradigms, and objectives that overemphasize point\-wise reconstruction in noisy observation space\. We proposeNeoST, the first spatio\-temporal foundation model pre\-trained solely on procedurally generated synthetic systems\. NeoST introduces a scalable synthetic pre\-training corpus to mitigate real\-world bias, a latent\-space reasoning architecture that generates and iteratively refines multiple future trajectories without sequential error accumulation, and latent\-space objectives that emphasize structural dynamics and enable inference\-time correction under distribution shifts\. Extensive experiments across diverse real\-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real\-world spatio\-temporal systems, achieves superior long\-horizon stability and inference efficiency\.

## 1Introduction

Spatio\-Temporal Foundation Models \(STFMs\)\[[29](https://arxiv.org/html/2607.16251#bib.bib1),[10](https://arxiv.org/html/2607.16251#bib.bib3),[14](https://arxiv.org/html/2607.16251#bib.bib2)\]are large\-scale pre\-trained models that learn generalizable representations of spatio–temporal dependencies from diverse and heterogeneous spatio\-temporal data, enabling a single unified backbone to support multiple downstream tasks across domains through zero\-shot or lightweight adaptation\. Unlike conventional task\-specific spatio\-temporal models\[[19](https://arxiv.org/html/2607.16251#bib.bib4)\]that are trained independently for each dataset and objective, STFMs aim to distill universal structural priors of space and time into a scalable parameterization, shifting the paradigm from one\-model\-per\-task engineering to foundation\-level representation learning\. STFMs unify fragmented modeling paradigms, enabling label\-efficient cross\-domain transfer and robust generalization under distribution shifts\. Furthermore, by distilling the structural evolution of complex dynamic systems\[[11](https://arxiv.org/html/2607.16251#bib.bib5)\], STFMs establish a critical foundation for data\-driven World Models\[[41](https://arxiv.org/html/2607.16251#bib.bib7),[59](https://arxiv.org/html/2607.16251#bib.bib6)\]\.

Despite their promise, the development of STFMs faces fundamental challenges across the entire modeling pipeline, beginning with the quality and diversity of available training data\[[10](https://arxiv.org/html/2607.16251#bib.bib3)\]\. Models pre\-trained on specific real\-world collections often overfit to the idiosyncratic topology and temporal rhythms of those specific sensor deployments, hindering their generalizability to unseen scenarios\. This imbalance induces strong distributional bias during pre\-training, leading models to overfit domain\-specific patterns and underperform when transferred to less\-represented systems\. Additionally, real\-world spatio\-temporal data are often plagued by sensor noise, irregular sampling, and extensive missing values\[[46](https://arxiv.org/html/2607.16251#bib.bib12)\]\. Such imperfections not only degrade representation quality but also entangle intrinsic system dynamics with measurement artifacts, making it difficult for STFMs to distill clean and universal structural priors of space and time\.

Beyond data limitations, current STFMs are largely built upon either autoregressive\[[28](https://arxiv.org/html/2607.16251#bib.bib8),[5](https://arxiv.org/html/2607.16251#bib.bib13),[31](https://arxiv.org/html/2607.16251#bib.bib14)\]or diffusion\-based paradigms\[[54](https://arxiv.org/html/2607.16251#bib.bib15),[53](https://arxiv.org/html/2607.16251#bib.bib16),[48](https://arxiv.org/html/2607.16251#bib.bib17),[17](https://arxiv.org/html/2607.16251#bib.bib18)\], each of which introduces structural bottlenecks\. Autoregressive models generate future states sequentially, conditioning each prediction on previously generated outputs\. While conceptually straightforward, this paradigm is prone to error accumulation: early prediction mistakes propagate and amplify over long horizons, hindering stable long\-term forecasting and limiting the model’s capacity for global reasoning and revision\. Diffusion\-based models alleviate some issues of sequential dependency by modeling complex distributions through iterative denoising, but they typically require dozens to hundreds of sampling steps during inference\. Such computational overhead is prohibitive in time\-sensitive scenarios, such as disaster response or real\-time urban management, where rapid and reliable predictions are essential\.

Challenges further arise in training objectives and inference paradigms\[[10](https://arxiv.org/html/2607.16251#bib.bib3)\]\. Spatio\-temporal signals often exhibit sparse or weak semantics in the raw time domain, where point\-wise losses\[[18](https://arxiv.org/html/2607.16251#bib.bib19)\]encourage models to focus on local fluctuations while overlooking global structures such as trends, periodicities, and latent dynamical mechanisms\[[7](https://arxiv.org/html/2607.16251#bib.bib20),[22](https://arxiv.org/html/2607.16251#bib.bib21)\]\. Optimizing directly in the observation space may therefore bias the model toward short\-term fidelity rather than structural understanding\. In addition, most existing STFMs lack effective test\-time adaptation capabilities\. When confronted with distribution shifts, such as stemming from seasonal changes, policy interventions, or sensor redeployments, these models typically treat the observed input sequence as fixed context, without explicitly modeling or correcting for the shift\. The inability to leverage incoming observations to recalibrate predictions at inference time significantly constrains robustness and generalization in open\-world settings\.

![Refer to caption](https://arxiv.org/html/2607.16251v1/x1.png)Figure 1:Comparison of Paradigms: NeoST vs\. Recent STFM\.To tackle these challenges, we proposeNeoST, the first spatio\-temporal foundation modelpre\-trained solely on synthetic spatio\-temporal data\. Our main contributions can be summarized as the following three aspects:

- •On thedataside, we depart from conventional reliance on large but biased real\-world datasets and instead construct a synthetic pre\-training corpus with virtually unlimited scale and controllable diversity\. By procedurally generating spatio\-temporal systems that span a broad range of dynamics, noise levels, and structural patterns, we mitigate distributional bias and avoid the detrimental effects of missing values and measurement noise in low\-quality pre\-training data\.
- •On thearchitectureside, we propose a latent\-space reasoning paradigm for STFMs, replacing the dominant autoregressive and diffusion\-based formulations\. Our model generates multiple candidate future trajectories directly in latent space and performs iterative refinement through patch\-level and trajectory\-level correction mechanisms\. By decoupling reasoning from step\-wise generation, NeoST is capable of deeper deliberation over alternative futures under a fixed computational budget\.
- •On thetraining & inferenceside, NeoST compute losses in latent space rather than directly in the raw time domain\. This approach encourages the model to learn and reason about the underlying evolutionary dynamics and structural patterns of spatio\-temporal systems, rather than merely fitting superficial value\-level details\. Moreover, the latent reasoning process naturally enables inference\-time correction based on observed inputs, enhancing robustness under distribution shifts\.

## 2Related Works

The success of large\-scale pre\-training in NLP has inspired analogous efforts in time series and spatio\-temporal data\[[20](https://arxiv.org/html/2607.16251#bib.bib36),[39](https://arxiv.org/html/2607.16251#bib.bib54)\]\. Time Series Foundation Models \(TSFMs\)\[[28](https://arxiv.org/html/2607.16251#bib.bib8)\]such as TimesFM\[[8](https://arxiv.org/html/2607.16251#bib.bib37)\], Chronos\[[1](https://arxiv.org/html/2607.16251#bib.bib38)\], Moirai\[[50](https://arxiv.org/html/2607.16251#bib.bib39),[35](https://arxiv.org/html/2607.16251#bib.bib40),[32](https://arxiv.org/html/2607.16251#bib.bib41)\], UniTime\[[34](https://arxiv.org/html/2607.16251#bib.bib42)\], MOMENT\[[15](https://arxiv.org/html/2607.16251#bib.bib43)\], and Time\-MOE\[[44](https://arxiv.org/html/2607.16251#bib.bib44)\]consolidate multiple forecasting tasks within a single pre\-trained backbone, demonstrating the viability of foundation\-scale temporal representation learning\. However, these models ignore the rich spatial structure present in real\-world systems\. Emerging STFMs extend this paradigm to spatially resolved domains\[[29](https://arxiv.org/html/2607.16251#bib.bib1),[14](https://arxiv.org/html/2607.16251#bib.bib2)\]\. OpenCity\[[27](https://arxiv.org/html/2607.16251#bib.bib46)\]integrates Transformer\-GNN hybrids pre\-trained on heterogeneous traffic datasets to enable zero\-shot urban forecasting\. UniST\[[56](https://arxiv.org/html/2607.16251#bib.bib45)\]introduces knowledge\-guided prompts for unified urban spatio\-temporal prediction across few\-shot and zero\-shot settings\. UrbanGPT\[[26](https://arxiv.org/html/2607.16251#bib.bib48)\]couples spatio\-temporal encoders with LLM prompting for instruction\-based spatial reasoning under limited supervision\. In climate science\[[30](https://arxiv.org/html/2607.16251#bib.bib10),[12](https://arxiv.org/html/2607.16251#bib.bib11)\], models such as Pangu\-Weather\[[6](https://arxiv.org/html/2607.16251#bib.bib51)\]and ClimaX\[[38](https://arxiv.org/html/2607.16251#bib.bib52)\]apply large Transformer architectures to global reanalysis data, achieving spatio\-temporal generality across physical domains\. Though promising, existing STFMs share common limitations: they rely on real\-world data that is domain\-biased and noisy, adopt either error\-prone autoregressive decoding or computationally intensive diffusion sampling, and optimize directly in the observation space without structural abstraction\.An expanded version of the related works can be found in the Appendix[A](https://arxiv.org/html/2607.16251#A1)\.

## 3Synthetic Spatio\-Temporal Data

We construct our synthetic data engine by explicitly parameterizing a family of stochastic graph dynamical systems defined on a graph𝒢=\(𝒱,ℰ\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{E\}\)with\|𝒱\|=N\|\\mathcal\{V\}\|=N\. Let𝐀∈ℝN×N\\mathbf\{A\}\\in\\mathbb\{R\}^\{N\\times N\}denote the adjacency matrix and𝐋\\mathbf\{L\}the corresponding normalized graph Laplacian\. At each time steptt, the latent dynamical state is represented as𝐇t∈ℝN×dh\\mathbf\{H\}\_\{t\}\\in\\mathbb\{R\}^\{N\\times d\_\{h\}\}, wheredhd\_\{h\}denotes the latent channel dimension\. The observable signal𝐱t∈ℝN\\mathbf\{x\}\_\{t\}\\in\\mathbb\{R\}^\{N\}is obtained through a random linear projection of𝐇t\\mathbf\{H\}\_\{t\}, while the latent state itself remains fully accessible during training\. Further analysis is provided in Appendix[B](https://arxiv.org/html/2607.16251#A2)\.

![Refer to caption](https://arxiv.org/html/2607.16251v1/x2.png)Figure 2:Synthetic data pipeline\.Graph Sampling & Operator Construction\.To promote structural diversity, the graph topology is sampled on\-the\-fly from random graph families, including Erdős–Rényi and Poisson degree models\. Self\-loops are added to stabilize the spectrum\. Given𝐀\\mathbf\{A\}, we compute the normalized Laplacian𝐋=𝐈−𝐃−1/2​𝐀𝐃−1/2,\\mathbf\{L\}=\\mathbf\{I\}\-\\mathbf\{D\}^\{\-1/2\}\\mathbf\{A\}\\mathbf\{D\}^\{\-1/2\},where𝐃\\mathbf\{D\}is the degree matrix\. To model spatial interactions in a principled manner, we leverage Chebyshev polynomials of the rescaled Laplacian\. Let𝐋^\\widehat\{\\mathbf\{L\}\}denote the spectrum\-normalized Laplacian, and define𝐓1=𝐋^,𝐓2=2​𝐋^2−𝐈\.\\mathbf\{T\}\_\{1\}=\\widehat\{\\mathbf\{L\}\},\\mathbf\{T\}\_\{2\}=2\\widehat\{\\mathbf\{L\}\}^\{2\}\-\\mathbf\{I\}\.These operators provide localized yet expressive spectral filters without requiring eigen\-decomposition during sequence generation\. To ensure scale generalization, we dynamically sample system configurations from broad distributions, including graph sizeNN, latent dimensiondd, and horizons\(T,L\)\(T,L\), exposing the model to highly heterogeneous spatial scales and temporal regimes\.

Stochastic Latent Dynamics\.The temporal evolution follows a stochastic recurrence:

𝐇t\+1=a​𝐇t\+𝝁​\(𝐇t,𝐇t−1,𝐀\)\+𝝈​\(𝐇t,𝐇t−1,𝐀\)⊙ϵt\+𝐛,\\mathbf\{H\}\_\{t\+1\}=a\\mathbf\{H\}\_\{t\}\+\\boldsymbol\{\\mu\}\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\mathbf\{A\}\)\+\\boldsymbol\{\\sigma\}\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\mathbf\{A\}\)\\odot\\boldsymbol\{\\epsilon\}\_\{t\}\+\\mathbf\{b\},\(1\)wherea∈\(0,1\)a\\in\(0,1\)controls system dissipation,ϵt∼𝒩​\(𝟎,𝐈\)\\boldsymbol\{\\epsilon\}\_\{t\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\mathbf\{I\}\)introduces stochastic excitation,𝐛\\mathbf\{b\}is a sample\-wise bias term, and⊙\\odotdenotes element\-wise multiplication\. The drift term𝝁\\boldsymbol\{\\mu\}and diffusion term𝝈\\boldsymbol\{\\sigma\}are parameterized as

𝝁​\(⋅\)=tanh⁡\(\[𝐓1​𝐇t∥𝐓2​𝐇t−1\]​𝐖1\)​𝐖2,𝝈​\(⋅\)=sigmoid​\(\[𝐓1​𝐇t∥𝐓2​𝐇t−1\]​𝐖3\),\\boldsymbol\{\\mu\}\(\\cdot\)=\\tanh\\\!\\Big\(\[\\mathbf\{T\}\_\{1\}\\mathbf\{H\}\_\{t\}\\;\\\|\\;\\mathbf\{T\}\_\{2\}\\mathbf\{H\}\_\{t\-1\}\]\\mathbf\{W\}\_\{1\}\\Big\)\\mathbf\{W\}\_\{2\},\\quad\\boldsymbol\{\\sigma\}\(\\cdot\)=\\mathrm\{sigmoid\}\\\!\\Big\(\[\\mathbf\{T\}\_\{1\}\\mathbf\{H\}\_\{t\}\\;\\\|\\;\\mathbf\{T\}\_\{2\}\\mathbf\{H\}\_\{t\-1\}\]\\mathbf\{W\}\_\{3\}\\Big\),\(2\)where𝐖1,𝐖2,𝐖3∈ℝdh×dh\\mathbf\{W\}\_\{1\},\\mathbf\{W\}\_\{2\},\\mathbf\{W\}\_\{3\}\\in\\mathbb\{R\}^\{d\_\{h\}\\times d\_\{h\}\}are learnable parameters of the generator and\[⋅∥⋅\]\[\\cdot\\\|\\cdot\]denotes channel\-wise concatenation\. This formulation couples first\- and second\-order spectral responses with nonlinear transformations, producing rich spatio\-temporal behaviors including diffusion\-like smoothing, oscillatory propagation, and stochastic perturbations\. Notably, the generator parameters are randomly initialized for each sample, yielding a distribution over dynamical systems rather than a single fixed simulator\. Consequently, the pre\-training corpus spans a broad manifold of graph\-coupled stochastic processes with heterogeneous spectral characteristics and stability regimes\.

Observation Generation\.Observable signals are obtained through a random linear projection𝐱t=𝐇t​𝐰,\\mathbf\{x\}\_\{t\}=\\mathbf\{H\}\_\{t\}\\mathbf\{w\},where𝐰∈ℝd\\mathbf\{w\}\\in\\mathbb\{R\}^\{d\}is sampled per instance\. This projection induces partial observability, mimicking real\-world sensing processes that capture low\-dimensional measurements of higher\-dimensional latent states\. To approximate realistic data imperfections while maintaining full control of the generative mechanism, we randomly mask entries in𝐗1:T\\mathbf\{X\}\_\{1:T\}with a controllable ratio\. Since these corruptions are externally imposed and independent of the underlying dynamics, the model can in principle disentangle intrinsic evolution patterns from observational artifacts\.

Output Augmentation\.To bridge the gap between ideal synthetic signals and imperfect real\-world measurements, we randomly mask a proportion of the generated observable sequence with missing values\. This simulates the missing data ubiquitous in real\-world sensor networks\. Crucially, while missing data in real\-world datasets often exhibits fixed topological biases \(e\.g\., broken sensors at specific locations\), our random masking strategy enforces an unstructured, unbiased distribution of missingness across both space and time\. This prevents the model from relying on superficial, dataset\-specific failure topologies and instead forces it to learn robust, position\-invariant spatiotemporal dependencies, significantly enhancing its zero\-shot transferability to unseen downstream systems\.

## 4Model Architecture

A Spatio\-Temporal Foundation Model \(STFM\) is defined as a parametric functionfθ:\(𝐗1:T,𝐀\)↦𝐘^T\+1:T\+L,f\_\{\\theta\}:\(\\mathbf\{X\}\_\{1:T\},\\mathbf\{A\}\)\\mapsto\\widehat\{\\mathbf\{Y\}\}\_\{T\+1:T\+L\},whereθ\\thetadenotes the model parameters,𝐗1:T∈ℝT×N\\mathbf\{X\}\_\{1:T\}\\in\\mathbb\{R\}^\{T\\times N\}is the historical observation,𝐀∈ℝN×N\\mathbf\{A\}\\in\\mathbb\{R\}^\{N\\times N\}encodes spatial dependencies, and𝐘^T\+1:T\+L∈ℝL×N\\widehat\{\\mathbf\{Y\}\}\_\{T\+1:T\+L\}\\in\\mathbb\{R\}^\{L\\times N\}is the predicted future sequence\. Formally, an STFM should support arbitrary input lengthsT∈ℕ\+T\\in\\mathbb\{N\}^\{\+\}, arbitrary graph sizesN∈ℕ\+N\\in\\mathbb\{N\}^\{\+\}, and arbitrary forecasting horizonsL∈ℕ\+L\\in\\mathbb\{N\}^\{\+\}\. NeoST follows a latent\-space spatio\-temporal modeling paradigm that decouples representation learning, global reasoning, and output adaptation within a unified foundation backbone\. Figure[3](https://arxiv.org/html/2607.16251#S4.F3)illustrates the overall architecture of our model\. The model first transforms raw observations into structured patch\-level embeddings, producing a compact latent representation that captures local temporal continuity and node\-level semantics\. These patch embeddings are then processed by a latentSpatio\-Temporal Encoderthat jointly models temporal dependencies and graph\-aware spatial interactions, yielding context\-aware latent states\. On top of the encoded representations, aHierarchical Latent Reasonergenerates future trajectory embeddings by performing multi\-step global reasoning directly in latent space, enabling the model to construct and refine candidate futures without relying on strictly autoregressive decoding\. Finally, aOutput Adaptermaps latent predictions back to the observation space to produce𝐘^T\+1:T\+L\\widehat\{\\mathbf\{Y\}\}\_\{T\+1:T\+L\}, supporting both pre\-training and tuning\. All major components are cascaded through the Hyper\-connection\[[61](https://arxiv.org/html/2607.16251#bib.bib65)\], which preserves information flow across modules while maintaining stable optimization\.

![Refer to caption](https://arxiv.org/html/2607.16251v1/x3.png)Figure 3:Overall architecture of NeoST\.### 4\.1Preprocessing

Given an input sequence𝐗1:T∈ℝT×N\\mathbf\{X\}\_\{1:T\}\\in\\mathbb\{R\}^\{T\\times N\}, we first apply instance normalization\[[21](https://arxiv.org/html/2607.16251#bib.bib66)\]along the temporal dimension to reduce scale heterogeneity\. For each nodeviv\_\{i\}, we compute the temporal meanμ\(i\)\\mu^\{\(i\)\}and standard deviationσ\(i\)\\sigma^\{\(i\)\}, and normalize the signal viax^t\(i\)=\(xt\(i\)−μ\(i\)\)/σ~\(i\)\\hat\{x\}\_\{t\}^\{\(i\)\}=\(x\_\{t\}^\{\(i\)\}\-\\mu^\{\(i\)\}\)/\\tilde\{\\sigma\}^\{\(i\)\}, whereσ~\(i\)=max⁡\(σ\(i\),1\)\\tilde\{\\sigma\}^\{\(i\)\}=\\max\(\\sigma^\{\(i\)\},1\)prevents small\-variance sequences from being over\-amplified\. Each normalized scalar observation is then lifted to adpd\_\{p\}\-dimensional vector through a fixed random linear projection𝐳t\(i\)=x^t\(i\)​𝐰\+𝐛\\mathbf\{z\}\_\{t\}^\{\(i\)\}=\\hat\{x\}\_\{t\}^\{\(i\)\}\\mathbf\{w\}\+\\mathbf\{b\}, where𝐰\\mathbf\{w\}and𝐛\\mathbf\{b\}are randomly initialized and kept non\-trainable to stabilize optimization, and masked values are mapped to zero vectors\. The resulting embeddings are segmented into temporal patches of lengthppwith stridess, optionally allowing overlap, and each flattened patch is projected to the hidden dimensionddto form patch tokens\. An optional trend decomposition branch computes first\-order differences between adjacent patches and injects the encoded residual signal into the patch representation, enabling sensitivity to short\-range dynamics\.

### 4\.2Spatio\-Temporal Encoder

The spatio\-temporal encoder transforms the patch\-level representation𝐙1:PT∈ℝPT×N×d\\mathbf\{Z\}\_\{1:P\_\{T\}\}\\in\\mathbb\{R\}^\{P\_\{T\}\\times N\\times d\}, wherePTP\_\{T\}denotes the number of patches obtained from patchifying the input sequence, into a contextual latent memory\. This is accomplished through alternating between two complementary mechanisms: intra\-node temporal attention, which captures temporal evolution within individual nodes, and inter\-node spatial propagation, which disseminates information across nodes according to learned relational affinities\. The encoder stacksLencL\_\{\\mathrm\{enc\}\}such layers, with each layer jointly implementing both mechanisms in a unified computation block\. Critically, the two pathways are designed to be mutually exclusive in their information sources, ensuring that each mechanism contributes a distinct representation\.

Intra\-node Temporal Attention\.For each nodeviv\_\{i\}, the patch sequence𝐙1:PT\(i\)∈ℝPT×d\\mathbf\{Z\}\_\{1:P\_\{T\}\}^\{\(i\)\}\\in\\mathbb\{R\}^\{P\_\{T\}\\times d\}is processed independently by a causal gate self\-attention module\[[40](https://arxiv.org/html/2607.16251#bib.bib67)\]equipped with Rotary Position Embeddings \(RoPE\)\[[45](https://arxiv.org/html/2607.16251#bib.bib68)\], which encodes relative temporal positions without relying on additive positional encodings\. This operation produces two complementary value representations per node𝐕res\\mathbf\{V\}^\{\\mathrm\{res\}\}and𝐕dis\\mathbf\{V\}^\{\\mathrm\{dis\}\}\.𝐕res∈ℝPT×N×d\\mathbf\{V\}^\{\\mathrm\{res\}\}\\in\\mathbb\{R\}^\{P\_\{T\}\\times N\\times d\}captures the self\-contained temporal dynamics of each node and serves as a residual pathway that preserves node\-specific history\.𝐕dis∈ℝPT×N×d\\mathbf\{V\}^\{\\mathrm\{dis\}\}\\in\\mathbb\{R\}^\{P\_\{T\}\\times N\\times d\}produces temporal features that are subsequently made available for spatial distribution to neighboring nodes\. The temporal pathway operates independently per node while the spatial pathway is explicitly prevented from collapsing into identity by masking self\-connections, ensuring that each mechanism contributes a distinct and complementary inductive bias to the final representation\.

Inter\-node Spatial Propagation\.Given the temporally encoded features, the encoder performs spatial information routing through a graph attention mechanism operating across nodes at each patch position\. Queries𝐐∈ℝPT×N×d\\mathbf\{Q\}\\in\\mathbb\{R\}^\{P\_\{T\}\\times N\\times d\}and keys𝐊∈ℝPT×N×d\\mathbf\{K\}\\in\\mathbb\{R\}^\{P\_\{T\}\\times N\\times d\}are computed from the current patch representations, and a node\-to\-node attention map is derived as𝐀p=softmax​\(a​𝐐p​𝐊p⊤\+𝐌\)∈ℝN×N,\\mathbf\{A\}\_\{p\}=\\mathrm\{softmax\}\\left\(a\\mathbf\{Q\}\_\{p\}\\mathbf\{K\}\_\{p\}^\{\\top\}\+\\mathbf\{M\}\\right\)\\in\\mathbb\{R\}^\{N\\times N\},wherea=1/d/Ha=1/\\sqrt\{d/H\}is the scalar factor,HHis the number of attention heads,𝐌\\mathbf\{M\}is a masking matrix with𝐌i​i=−∞\\mathbf\{M\}\_\{ii\}=\-\\inftyand𝐌i​j=0\\mathbf\{M\}\_\{ij\}=0fori≠ji\\neq j\. The diagonal masking explicitly prohibits each node from attending to itself during spatial aggregation, ensuring that the spatial pathway carries exclusively cross\-node information\. The attention\-weighted aggregation of𝐕dis\\mathbf\{V\}^\{\\mathrm\{dis\}\}yields a spatially\-propagated representation𝐕agg=𝐀p​𝐕pdis\\mathbf\{V\}^\{\\mathrm\{agg\}\}=\\mathbf\{A\}\_\{p\}\\mathbf\{V\}^\{\\mathrm\{dis\}\}\_\{p\}, which is then modulated by a learned sigmoid gate𝐠=σ​\(𝐖g​𝐙\)\\mathbf\{g\}=\\sigma\(\\mathbf\{W\}\_\{g\}\\mathbf\{Z\}\)to adaptively control how much spatial context is incorporated at each position\.

Output Combination\.The outputs of both pathways are concatenated and linearly projected to produce the updated representation, followed by a position\-wise feed\-forward network:

𝐙′=𝐖o​\[𝐕res∥𝐕agg⊙𝐠\]\+𝐙,𝐙′′=FFN​\(𝐙′\)\+𝐙′\.\\mathbf\{Z\}^\{\\prime\}=\\mathbf\{W\}\_\{o\}\\left\[\\mathbf\{V\}^\{\\mathrm\{res\}\}\\,\\\|\\,\\mathbf\{V\}^\{\\mathrm\{agg\}\}\\odot\\mathbf\{g\}\\right\]\+\\mathbf\{Z\},\\qquad\\mathbf\{Z\}^\{\\prime\\prime\}=\\mathrm\{FFN\}\(\\mathbf\{Z\}^\{\\prime\}\)\+\\mathbf\{Z\}^\{\\prime\}\.\(3\)This formulation ensures that each node’s output integrates both its own temporal trajectory and a gated summary of information propagated from the rest of the graph\. The final encoder output𝐙1:PT∈ℝPT×N×d\\mathbf\{Z\}\_\{1:P\_\{T\}\}\\in\\mathbb\{R\}^\{P\_\{T\}\\times N\\times d\}serves as the contextual memory that conditions all subsequent latent reasoning\. Furthermore, the dense residual pathways embedded within this integration step facilitate stable gradient flow, guaranteeing robust representation learning during the end\-to\-end pre\-training phase\.

### 4\.3Hierarchical Reasoner

Given the contextual patch memory𝐙1:PT∈ℝPT×N×d\\mathbf\{Z\}\_\{1:P\_\{T\}\}\\in\\mathbb\{R\}^\{P\_\{T\}\\times N\\times d\}produced by the spatio\-temporal encoder, the hierarchical reasoner is responsible for generating future trajectory embeddings directly in latent space\. Rather than decoding predictions step\-by\-step in the observation domain, the reasoner operates entirely over patch tokens, producing𝐙^1:PL∈ℝPL×N×d\\hat\{\\mathbf\{Z\}\}\_\{1:P\_\{L\}\}\\in\\mathbb\{R\}^\{P\_\{L\}\\times N\\times d\}, wherePLP\_\{L\}is the number of output patch tokens corresponding to the target horizonLL\. The overall reasoning process follows a coarse\-to\-fine pipeline: An ensemble of multi\-scale predictors first generates candidate trajectory hypotheses at different temporal resolutions in latent space\. These candidates are then consolidated and refined at both the token level and the trajectory level, enabling the model to reconcile multi\-scale information and align its latent predictions with the encoded historical context\.

Multi\-Scale Latent Prediction\.To capture future dynamics at multiple temporal resolutions, we instantiate a set ofSSindependent predictors\{𝒫s\}s=1S\\\{\\mathcal\{P\}\_\{s\}\\\}\_\{s=1\}^\{S\}, each operating at a distinct temporal strideΔs\\Delta\_\{s\}patch steps\. Predictor𝒫s\\mathcal\{P\}\_\{s\}maps a global context summary𝐳¯∈ℝN×d\\bar\{\\mathbf\{z\}\}\\in\\mathbb\{R\}^\{N\\times d\}to a sequence of latent states at its native stride and truncated toPLP\_\{L\}steps, which can be written as𝐙^1:PL\(s\)=𝒫s​\(𝐳¯,PL\)∈ℝPL×N×d\.\\hat\{\\mathbf\{Z\}\}^\{\(s\)\}\_\{1:P\_\{L\}\}=\\mathcal\{P\}\_\{s\}\\\!\\left\(\\bar\{\\mathbf\{z\}\},\\;P\_\{L\}\\right\)\\in\\mathbb\{R\}^\{P\_\{L\}\\times N\\times d\}\.Here,𝐳¯=\(1/P\)​∑p=1P𝐙p\\bar\{\\mathbf\{z\}\}=\(1/P\)\\sum\_\{p=1\}^\{P\}\\mathbf\{Z\}\_\{p\}condenses the average observed historical context into a compact initialization signal\. TheSSresulting trajectories are concatenated along the feature dimension to form a multi\-scale composite representation𝐙^1:PLcat=\[𝐙^\(1\)​‖𝐙^\(2\)‖​⋯∥𝐙^\(S\)\]∈ℝPL×N×S​d,\\hat\{\\mathbf\{Z\}\}^\{\\mathrm\{cat\}\}\_\{1:P\_\{L\}\}=\\left\[\\hat\{\\mathbf\{Z\}\}^\{\(1\)\}\\,\\\|\\,\\hat\{\\mathbf\{Z\}\}^\{\(2\)\}\\,\\\|\\,\\cdots\\,\\\|\\,\\hat\{\\mathbf\{Z\}\}^\{\(S\)\}\\right\]\\in\\mathbb\{R\}^\{P\_\{L\}\\times N\\times Sd\},to which a learnable scale embedding𝐞ch∈ℝS​d\\mathbf\{e\}\_\{\\mathrm\{ch\}\}\\in\\mathbb\{R\}^\{Sd\}is added to distinguish each scale’s contribution prior to fusion\. In our implementation, each predictor𝒫s\\mathcal\{P\}\_\{s\}is realized as a ChebConv\-LSTM\[[42](https://arxiv.org/html/2607.16251#bib.bib69)\]with polynomial orderKK, which propagate spatial dependencies across nodes at every step\.

Token\-Level Refinement via Conditional Temporal Convolution\.The multi\-scale composite is subsequently fused into a unifieddd\-dimensional token sequence\. Critically, the appropriate fusion strategy is not universal but depends on the specific input context\. We therefore formulate token\-level refinement as a context\-conditioned operation, where the fusion weights are dynamically generated from the observed context rather than fixed as shared parameters\. A lightweight meta\-networkϕmeta\\phi\_\{\\mathrm\{meta\}\}takes the global context summary𝐳¯\\bar\{\\mathbf\{z\}\}and produces an instance\-specific convolutional kernel𝐖dyn=ϕmeta​\(𝐳¯\)∈ℝd×\(S​d\+d\)×k,\\mathbf\{W\}^\{\\mathrm\{dyn\}\}=\\phi\_\{\\mathrm\{meta\}\}\(\\bar\{\\mathbf\{z\}\}\)\\in\\mathbb\{R\}^\{d\\times\(Sd\+d\)\\times k\},wherekkis the kernel width\. This kernel is then applied along the temporal dimension to a augmented input\[𝐙^cat\+𝐞ch∥𝐄pos\]\\left\[\\hat\{\\mathbf\{Z\}\}^\{\\mathrm\{cat\}\}\+\\mathbf\{e\}\_\{\\mathrm\{ch\}\}\\;\\\|\\;\\mathbf\{E\}^\{\\mathrm\{pos\}\}\\right\], which concatenates the channel embedding𝐞ch∈ℝS​d\\mathbf\{e\}\_\{\\mathrm\{ch\}\}\\in\\mathbb\{R\}^\{Sd\}with sinusoidal position embeddings𝐄pos∈ℝPL×d\\mathbf\{E\}^\{\\mathrm\{pos\}\}\\in\\mathbb\{R\}^\{P\_\{L\}\\times d\}to provide positional awareness, yielding𝐙^fused∈ℝPL×N×d\\hat\{\\mathbf\{Z\}\}^\{\\mathrm\{fused\}\}\\in\\mathbb\{R\}^\{P\_\{L\}\\times N\\times d\}\. The convolution is applied independently for each node, ensuring that the token\-level merging of multi\-scale signals is modulated by the observed context\.

Trajectory\-Level Refinement via Encoder\-Decoder Reasoning\.While token\-level fusion reconciles scale inconsistencies locally, the resulting trajectory may still exhibit global structural incoherence or misalignment with the historical context\. To address this, we apply a two\-stage refinement that reasons over the full trajectory sequence jointly\. A trajectory encoderℛenc\\mathcal\{R\}\_\{\\mathrm\{enc\}\}first processes𝐙^fused\\hat\{\\mathbf\{Z\}\}^\{\\mathrm\{fused\}\}to propagate long\-range temporal dependencies across future tokens, after which a trajectory decoderℛdec\\mathcal\{R\}\_\{\\mathrm\{dec\}\}grounds the refined trajectory in the observed history𝐙1:PT\\mathbf\{Z\}\_\{1:P\_\{T\}\}through cross\-attention\. Bothℛenc\\mathcal\{R\}\_\{\\mathrm\{enc\}\}andℛdec\\mathcal\{R\}\_\{\\mathrm\{dec\}\}are instantiated as stacks of transformer layers\.ℛenc\\mathcal\{R\}\_\{\\mathrm\{enc\}\}employs causal gate self\-attention with RoPE to model temporal ordering within the predicted trajectory, whileℛdec\\mathcal\{R\}\_\{\\mathrm\{dec\}\}performs cross gate attention between the refined trajectory queries and𝐙1:PT\\mathbf\{Z\}\_\{1:P\_\{T\}\}\.

## 5Training & Inference

Pre\-training Phase\.Beyond providing ground\-truth future values𝐘T\+1:T\+L\\mathbf\{Y\}\_\{T\+1:T\+L\}, the engine exposes the underlying latent state vectors\{𝐇t\}t=T\+1T\+L\\\{\\mathbf\{H\}\_\{t\}\\\}\_\{t=T\+1\}^\{T\+L\}that directly govern the generative dynamics of each synthetic system\. This privileged supervisory signal, unavailable from real\-world datasets, forms the cornerstone of our pre\-training objective\. Concretely, we flatten𝐙^out\\hat\{\\mathbf\{Z\}\}^\{\\mathrm\{out\}\}to𝐙^flat∈ℝ\(PL​N\)×d\\hat\{\\mathbf\{Z\}\}^\{\\mathrm\{flat\}\}\\in\\mathbb\{R\}^\{\(P\_\{L\}N\)\\times d\}and apply the Principal Component Analysis \(PCA\)\[[49](https://arxiv.org/html/2607.16251#bib.bib70)\]to project it into thedhd\_\{h\}\-dimensional space of the synthetic latent matrix𝐇∈ℝ\(PL​N\)×dh\\mathbf\{H\}\\in\\mathbb\{R\}^\{\(P\_\{L\}N\)\\times d\_\{h\}\}\. The resulting pre\-training objective, which we termLatent Alignment Loss \(LAL\), interpolates between a latent alignment term and a standard observation\-space reconstruction term via a scalar hyperparameterα∈\[0,1\]\\alpha\\in\[0,1\]:

ℒLAL=\(1−α\)​‖𝐇−PCAdh​\(𝐙^flat\)‖F2\+α​‖𝐘T\+1:T\+L−𝐘^T\+1:T\+L‖F2\.\\mathcal\{L\}\_\{\\mathrm\{LAL\}\}=\(1\-\\alpha\)\\,\\bigl\\\|\\mathbf\{H\}\-\\mathrm\{PCA\}\_\{d\_\{h\}\}\\\!\\bigl\(\\hat\{\\mathbf\{Z\}\}^\{\\mathrm\{flat\}\}\\bigr\)\\bigr\\\|\_\{F\}^\{2\}\+\\alpha\\,\\bigl\\\|\\mathbf\{Y\}\_\{T\+1:T\+L\}\-\\hat\{\\mathbf\{Y\}\}\_\{T\+1:T\+L\}\\bigr\\\|\_\{F\}^\{2\}\.\(4\)By placing the primary supervisory weight on the latent space,ℒLAL\\mathcal\{L\}\_\{\\mathrm\{LAL\}\}steers NeoST toward internalizing the structural and dynamical properties of spatio\-temporal systems rather than overfitting to the superficial statistical patterns of any particular observation domain\.

Test\-Time Adaptation\.Let𝐙~enc∈ℝT×N×ds\\tilde\{\\mathbf\{Z\}\}^\{\\mathrm\{enc\}\}\\in\\mathbb\{R\}^\{T\\times N\\times d\_\{s\}\}denote the time\-resolved encoder embeddings obtained by depatchifying the encoder output, whereds=d/pd\_\{s\}=d/pis the per\-timestep embedding dimension andppis the patch length\. Collecting all spatiotemporal tokens that are not masked or missing, we form the feature matrix𝐙obs∈ℝM×ds\\mathbf\{Z\}\_\{\\mathrm\{obs\}\}\\in\\mathbb\{R\}^\{M\\times d\_\{s\}\}and the corresponding observation vector𝐯obs∈ℝM\\mathbf\{v\}\_\{\\mathrm\{obs\}\}\\in\\mathbb\{R\}^\{M\}, whereMMis the number of valid observations\. The adapter solves the ridge regression problem by utilizing the following equation:𝐰∗,b∗=arg⁡min𝐰,b‖𝐙obs​𝐰\+b​1−𝐯obs‖2\+λ​‖𝐰‖2\.\\mathbf\{w\}^\{\*\},b^\{\*\}=\\mathop\{\\arg\\min\}\_\{\\mathbf\{w\},\\,b\}\\;\\bigl\\\|\\mathbf\{Z\}\_\{\\mathrm\{obs\}\}\\mathbf\{w\}\+b\\,\\mathbf\{1\}\-\\mathbf\{v\}\_\{\\mathrm\{obs\}\}\\bigr\\\|^\{2\}\+\\lambda\\\|\\mathbf\{w\}\\\|^\{2\}\.The equation admits the closed\-form solution𝐰∗=\(𝐙obs⊤​𝐙obs\+λ​𝐈\)−1​𝐙obs⊤​𝐯~obs\\mathbf\{w\}^\{\*\}=\(\\mathbf\{Z\}\_\{\\mathrm\{obs\}\}^\{\\top\}\\mathbf\{Z\}\_\{\\mathrm\{obs\}\}\+\\lambda\\mathbf\{I\}\)^\{\-1\}\\mathbf\{Z\}\_\{\\mathrm\{obs\}\}^\{\\top\}\\tilde\{\\mathbf\{v\}\}\_\{\\mathrm\{obs\}\}, where𝐯~obs\\tilde\{\\mathbf\{v\}\}\_\{\\mathrm\{obs\}\}denotes the bias\-corrected target\. Denoting the depatchified future embeddings as𝐙~out∈ℝL×N×ds\\tilde\{\\mathbf\{Z\}\}^\{\\mathrm\{out\}\}\\in\\mathbb\{R\}^\{L\\times N\\times d\_\{s\}\}, the predicted future sequence is obtained by applying the fitted mapping𝐘^T\+1:T\+L=𝐙~flatout​𝐰∗\+b∗,\\hat\{\\mathbf\{Y\}\}\_\{T\+1:T\+L\}=\\tilde\{\\mathbf\{Z\}\}^\{\\mathrm\{out\}\}\_\{\\mathrm\{flat\}\}\\,\\mathbf\{w\}^\{\*\}\+b^\{\*\},where𝐙~flatout∈ℝ\(L​N\)×ds\\tilde\{\\mathbf\{Z\}\}^\{\\mathrm\{out\}\}\_\{\\mathrm\{flat\}\}\\in\\mathbb\{R\}^\{\(LN\)\\times d\_\{s\}\}is the reshaped future embedding matrix\.

## 6Experiments

In our experiments, we aim to address the following research questions \(RQs\):

- •\(RQ1\): How effectively does NeoST transfer to diverse unseen real\-world systems?
- •\(RQ2\): How does each component and hyperparameter in NeoST contribute to model performance?
- •\(RQ3\): How efficient is NeoST compared to other foundation models?
- •\(RQ4\): How can synthetic spatio\-temporal data enhance the generalization ability of models?

### 6\.1Experimental Setups

Datsets & Baselines\.NeoST’s pre\-training utilizes the synthetic data introduced in Section[3](https://arxiv.org/html/2607.16251#S3)\. For testing, we selected six classic spatio\-temporal datasets for evaluation: PEMS03/04/07/08\[[16](https://arxiv.org/html/2607.16251#bib.bib27)\], PEMS\-BAY & METR\-LA\[[25](https://arxiv.org/html/2607.16251#bib.bib28)\]\. We compare NeoST with the following models:

- •Spatio\-Temporal Foundation Models \(STFMs\): FactoST\[[60](https://arxiv.org/html/2607.16251#bib.bib49)\], OpenCity\[[27](https://arxiv.org/html/2607.16251#bib.bib46)\], UniST\[[56](https://arxiv.org/html/2607.16251#bib.bib45)\]
- •Time Series Foundation Models \(TSFMs\): ROSE\[[47](https://arxiv.org/html/2607.16251#bib.bib72)\], TimesFM\[[8](https://arxiv.org/html/2607.16251#bib.bib37)\], Moirai\[[32](https://arxiv.org/html/2607.16251#bib.bib41)\];

We adopted Mean Absolute Error \(MAE\) and Root Mean Square Error \(RMSE\) as evaluation metrics\.

Implementation Details\.All experiments are conducted on a single NVIDIA A100 80GB GPU paired with an Intel Xeon Platinum 8358P CPU \(32C @ 2\.60GHz\)\. During the synthetic data generation process, the number of nodes and the latent dimension for each spatio\-temporal graph are uniformly sampled from the ranges\[64,512\]\[64,512\]and\[4,16\]\[4,16\], respectively, producing 1,000 distinct samples per epoch\. The core model architecture is configured with a hidden dimension of 96 and 4 attention heads\. The spatio\-temporal encoder stacks 4 layers, while the trajectory refinement module is structured with a 2\-layer encoder and a 2\-layer decoder\. The hyper\-connections throughout the network maintain an expansion rate of 4\. We pre\-train the model using AdamW\[[37](https://arxiv.org/html/2607.16251#bib.bib71)\]with a batch size of 8 for 15 epochs\. The other experimental settings are the same as those inZhonget al\.\[[60](https://arxiv.org/html/2607.16251#bib.bib49)\]\.

### 6\.2Model Comparison \(RQ1\)

We evaluate the models on six established spatio\-temporal datasets in the zero\-shot scenario, with the results summarized in Table[1](https://arxiv.org/html/2607.16251#S6.T1)\. NeoST consistently surpasses both spatio\-temporal and time series foundation models, achieving the lowest MAE and RMSE in ten out of the twelve evaluated settings\. Our model demonstrates particularly substantial improvements in challenging long\-horizon scenarios of 96 steps, lowering the MAE by significant margins on datasets such as PEMS\-03, PEMS\-04, PEMS\-07, and METR\-LA compared to the strongest baselines\. This sustained long\-term accuracy validates the effectiveness of our latent reasoning paradigm in mitigating the error accumulation typically observed in conventional autoregressive architectures\. Although FactoST secures a marginal lead on the short horizon of PEMS\-Bay and TimesFM performs slightly better on the long horizon of PEMS\-08, NeoST remains highly competitive in these instances while dominating the overall benchmark\. Crucially, these strong zero\-shot results confirm that pre\-training exclusively on diverse synthetic data successfully circumvents the distributional biases of real\-world datasets\.

Table 1:Quantitative comparison of zero\-shot forecasting performance\. Results are reported as MAE / RMSE, where lower values indicate better predictive performance\. The best and second\-best results are highlighted inredandblue, respectively\. Datasets marked with an asterisk \(\*\) were included in the pre\-training corpora of specific baselines and are therefore excluded from the ranking\.
### 6\.3Ablation and Hyperparameter Analysis \(RQ2\)

To rigorously validate the individual contributions of NeoST’s core components, we conduct a comprehensive ablation study\. Specifically, we evaluate the zero\-shot forecasting Mean Absolute Error \(MAE\) under both short\-term \(12 steps\) and long\-term \(96 steps\) horizons on the PEMS03 and PEMS07 datasets\. This analysis systematically ablates critical architectural modules, examines the sensitivity of the latent alignment objective by varying the trade\-off hyperparameterα∈\[0,1\]\\alpha\\in\[0,1\]\(Figure[4](https://arxiv.org/html/2607.16251#S6.F4)\), and isolates the empirical efficacy of the test\-time adaptation mechanism \(Figure[5](https://arxiv.org/html/2607.16251#S6.F5)\)\.

![Refer to caption](https://arxiv.org/html/2607.16251v1/x4.png)Figure 4:Ablation and hyperparameter analysis\.- •Architectural Contributions:The full NeoST framework attains the lowest MAE and exhibits superior predictive stability across most evaluated scenarios\. Specifically, disabling multi\-scale prediction significantly impairs forecasting robustness over long\-term horizons, whereas omitting the trajectory refinement module results in a substantial increase in cumulative structural errors\.
- •Objective Trade\-off \(α\\alpha\):Model predictive performance demonstrates high sensitivity to the balance between latent and observation\-space objectives\. We find that relying exclusively on latent alignment \(α=0\.0\\alpha=0\.0\) results in suboptimal error rates, while pure observation\-space optimization \(α=1\.0\\alpha=1\.0\) restricts robust generalization under distributional shifts\. The optimal balance consistently resides withinα∈\[0\.2,0\.6\]\\alpha\\in\[0\.2,0\.6\], validating our proposed synergistic dual\-objective design\.
- •Test\-Time Adaptation \(TTA\):As shown in Figure[5](https://arxiv.org/html/2607.16251#S6.F5), disabling the TTA module consistently degrades forecasting accuracy across most scenarios\. The performance drop is particularly pronounced in both short and long horizons on PEMS07 and PEMS08\. This confirms that leveraging inference\-time observation context is crucial for mitigating distribution shifts in zero\-shot transfer\.

![Refer to caption](https://arxiv.org/html/2607.16251v1/x5.png)Figure 5:Impact of Test\-Time Adaptation \(TTA\)\.
### 6\.4Inference Efficiency \(RQ3\)

![Refer to caption](https://arxiv.org/html/2607.16251v1/x6.png)Figure 6:Efficiency comparison\.To answer RQ3, we evaluate the computational efficiency of NeoST by comparing its inference time, parameter scale, and predictive performance \(MAE\) against representative baselines on PEMS03 dataset, as illustrated in Figure[6](https://arxiv.org/html/2607.16251#S6.F6)\. NeoST achieves the lowest MAE among all evaluated models, demonstrating superior predictive accuracy\. While lightweight models like UniST \(2\.1s\) and FactoST \(8\.1s\) exhibit faster inference speeds, they suffer from significantly higher prediction errors\. Conversely, massive models such as TimesFM \(200\.0M params, 470\.1s\) incur prohibitive computational costs without outperforming our approach\. Operating with 89\.26M parameters and a 24\.0s inference time, NeoST maintains a computational footprint comparable to MOIRAI \(22\.2s\) and OpenCity \(25\.3s\) but delivers substantially better forecasting accuracy\. These results establish that NeoST achieves a favorable trade\-off between reasoning efficiency and spatio\-temporal modeling performance\.

### 6\.5Manifold Coverage of Synthetic Data \(RQ4\)

![Refer to caption](https://arxiv.org/html/2607.16251v1/x7.png)Figure 7:t\-SNE visualization\.To address RQ4, we visualize the representational manifold of our synthetic corpus against six real\-world benchmarks using t\-SNE\. After applying instance normalization to fixed\-length temporal sequences \(12 steps\) from all datasets to isolate intrinsic temporal dynamics, we projected these features into a 2D space\. As illustrated in Figure[7](https://arxiv.org/html/2607.16251#S6.F7), the synthetic data forms an expansive, continuous manifold that comprehensively envelopes the localized clusters of all real\-world test sets\. This empirical evidence validates our core hypothesis: pre\-training exclusively on this highly diverse synthetic manifold equips the foundation model with universal structural priors, mitigating real\-world data biases and enabling robust zero\-shot generalization\.

## 7Conclusion & Future Works

In this work, we introduced NeoST, a novel Spatio\-Temporal Foundation Model that overcomes the limitations of biased real\-world datasets and computationally expensive generation paradigms\. By pre\-training exclusively on a procedurally generated synthetic corpus, NeoST effectively mitigates distributional biases and measurement artifacts\. Extensive experiments across diverse benchmarks demonstrate that NeoST achieves superior generalization, long\-horizon stability, and inference efficiency in diverse real\-world spatio\-temporal systems\. For future work, we plan to scale the synthetic generation engine to incorporate more complex physical laws and non\-stationary dynamics\.

## References

- \[1\]A\. F\. Ansari, L\. Stella, C\. Turkmen, X\. Zhang, P\. Mercado, H\. Shen, O\. Shchur, S\. S\. Rangapuram, S\. P\. Arango, S\. Kapoor,et al\.\(2024\)Chronos: learning the language of time series\.arXiv preprint arXiv:2403\.07815\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[2\]\(2025\)Synthetic datasets for machine learning on spatio\-temporal graphs using pdes\.arXiv preprint arXiv:2502\.04140\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.
- \[3\]M\. Assran, Q\. Duval, I\. Misra, P\. Bojanowski, P\. Vincent, M\. Rabbat, Y\. LeCun, and N\. Ballas\(2023\)Self\-supervised learning from images with a joint\-embedding predictive architecture\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 15619–15629\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1)\.
- \[4\]R\. Balestriero and Y\. LeCun\(2025\)Lejepa: provable and scalable self\-supervised learning without the heuristics\.arXiv preprint arXiv:2511\.08544\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1)\.
- \[5\]K\. Benidis, S\. S\. Rangapuram, V\. Flunkert, Y\. Wang, D\. Maddix, C\. Turkmen, J\. Gasthaus, M\. Bohlke\-Schneider, D\. Salinas, L\. Stella,et al\.\(2022\)Deep learning for time series forecasting: tutorial and literature survey\.ACM Computing Surveys55\(6\),pp\. 1–36\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p3.1)\.
- \[6\]K\. Bi, L\. Xie, H\. Zhang, X\. Chen, X\. Gu, and Q\. Tian\(2023\)Accurate medium\-range global weather forecasting with 3d neural networks\.Nature619\(7970\),pp\. 533–538\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[7\]M\. Cuturi and M\. Blondel\(2017\)Soft\-dtw: a differentiable loss function for time\-series\.InInternational conference on machine learning,pp\. 894–903\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.16251#S1.p4.1)\.
- \[8\]A\. Das, W\. Kong, R\. Sen, and Y\. Zhou\(2024\)A decoder\-only foundation model for time\-series forecasting\.ICML\.Note:Pretrained TimesFM on 100B time points from Google Trends, Wikipedia, synthetic and real\-world dataCited by:[1st item](https://arxiv.org/html/2607.16251#A3.I1.i1.p1.1),[§2](https://arxiv.org/html/2607.16251#S2.p1.1),[2nd item](https://arxiv.org/html/2607.16251#S6.I2.i2.p1.1)\.
- \[9\]T\. D\. Edwards, J\. Alvey, J\. Alsing, N\. H\. Nguyen, and B\. D\. Wandelt\(2024\)Scaling\-laws for large time\-series models\.arXiv preprint arXiv:2405\.13867\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1)\.
- \[10\]Y\. Fang, H\. Miao, Y\. Liang, L\. Deng, Y\. Cui, X\. Zeng, Y\. Xia, Y\. Zhao, T\. B\. Pedersen, C\. S\. Jensen,et al\.\(2026\)Unraveling spatio\-temporal foundation models via the pipeline lens: a comprehensive review\.IEEE Transactions on Knowledge and Data Engineering\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p1.1),[§1](https://arxiv.org/html/2607.16251#S1.p2.1),[§1](https://arxiv.org/html/2607.16251#S1.p4.1)\.
- \[11\]Y\. Feng, X\. Liu, Y\. Xia, and Y\. Liang\(2025\)FlowNet: modeling dynamic spatio\-temporal systems via flow propagation\.arXiv preprint arXiv:2511\.05595\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p1.1)\.
- \[12\]Y\. Feng, Q\. Wang, Y\. Xia, J\. Huang, S\. Zhong, and Y\. Liang\(2024\-08\)Spatio\-temporal field neural networks for air quality inference\.InProceedings of the Thirty\-Third International Joint Conference on Artificial Intelligence, IJCAI\-24,K\. Larson \(Ed\.\),pp\. 7260–7268\.Note:AI for GoodExternal Links:[Document](https://dx.doi.org/10.24963/ijcai.2024/803),[Link](https://doi.org/10.24963/ijcai.2024/803)Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[13\]N\. Gao, H\. Xue, W\. Shao, S\. Zhao, K\. K\. Qin, A\. Prabowo, M\. S\. Rahaman, and F\. D\. Salim\(2022\)Generative adversarial networks for spatio\-temporal data: a survey\.ACM Transactions on Intelligent Systems and Technology \(TIST\)13\(2\),pp\. 1–25\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.
- \[14\]A\. Goodge, W\. S\. Ng, B\. Hooi, and S\. K\. Ng\(2025\)Spatio\-temporal foundation models: vision, challenges, and opportunities\.arXiv preprint arXiv:2501\.09045\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p1.1),[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[15\]M\. Goswami, K\. Szafer, A\. Choudhry, Y\. Cai, S\. Li, and A\. Dubrawski\(2024\)Moment: a family of open time\-series foundation models\.arXiv preprint arXiv:2402\.03885\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[16\]S\. Guo, Y\. Lin, H\. Wan, X\. Li, and G\. Cong\(2021\)Learning dynamics and heterogeneity of spatial\-temporal graph data for traffic forecasting\.IEEE Transactions on Knowledge and Data Engineering34\(11\),pp\. 5415–5428\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1),[§6\.1](https://arxiv.org/html/2607.16251#S6.SS1.p1.1)\.
- \[17\]J\. Hu, X\. Liu, Z\. Fan, Y\. Liang, and R\. Zimmermann\(2024\)Towards unifying diffusion models for probabilistic spatio\-temporal graph learning\.InProceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems,pp\. 135–146\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p3.1)\.
- \[18\]A\. Jadon, A\. Patil, and S\. Jadon\(2024\)A comprehensive survey of regression\-based loss functions for time series forecasting\.InInternational Conference on Data Management, Analytics & Innovation,pp\. 117–147\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.16251#S1.p4.1)\.
- \[19\]G\. Jin, Y\. Liang, Y\. Fang, Z\. Shao, J\. Huang, J\. Zhang, and Y\. Zheng\(2023\)Spatio\-temporal graph neural networks for predictive learning in urban computing: a survey\.IEEE transactions on knowledge and data engineering36\(10\),pp\. 5388–5408\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p1.1)\.
- \[20\]M\. Jin, Q\. Wen, Y\. Liang, C\. Zhang, S\. Xue, X\. Wang, J\. Zhang, Y\. Wang, H\. Chen, X\. Li,et al\.\(2023\)Large models for time series and spatio\-temporal data: a survey and outlook\.arXiv preprint arXiv:2310\.10196\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[21\]T\. Kim, J\. Kim, Y\. Tae, C\. Park, J\. Choi, and J\. Choo\(2021\)Reversible instance normalization for accurate time\-series forecasting against distribution shift\.InInternational conference on learning representations,Cited by:[§4\.1](https://arxiv.org/html/2607.16251#S4.SS1.p1.13)\.
- \[22\]V\. Le Guen and N\. Thome\(2019\)Shape and time distortion loss for training deep time series forecasting models\.Advances in neural information processing systems32\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.16251#S1.p4.1)\.
- \[23\]L\. Li, H\. Xue, S\. Ao, Y\. Song, and F\. Salim\(2025\)HiT\-jepa: a hierarchical self\-supervised trajectory embedding framework for similarity computation\.arXiv preprint arXiv:2507\.00028\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1)\.
- \[24\]L\. Li, H\. Xue, Y\. Song, and F\. Salim\(2024\)T\-jepa: a joint\-embedding predictive architecture for trajectory similarity computation\.InProceedings of the 32nd ACM international conference on advances in geographic information systems,pp\. 569–572\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1)\.
- \[25\]Y\. Li, R\. Yu, C\. Shahabi, and Y\. Liu\(2018\)Diffusion convolutional recurrent neural network: data\-driven traffic forecasting\.InInternational Conference on Learning Representations \(ICLR ’18\),Cited by:[§6\.1](https://arxiv.org/html/2607.16251#S6.SS1.p1.1)\.
- \[26\]Z\. Li, L\. Xia, J\. Tang, Y\. Xu, L\. Shi, L\. Xia, D\. Yin, and C\. Huang\(2024\)Urbangpt: spatio\-temporal large language models\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 5351–5362\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[27\]Z\. Li, L\. Xia, L\. Shi, Y\. Xu, D\. Yin, and C\. Huang\(2024\)Opencity: open spatio\-temporal foundation models for traffic prediction\.arXiv preprint arXiv:2408\.10269\.Cited by:[2nd item](https://arxiv.org/html/2607.16251#A3.I2.i2.p1.1),[§2](https://arxiv.org/html/2607.16251#S2.p1.1),[1st item](https://arxiv.org/html/2607.16251#S6.I2.i1.p1.1)\.
- \[28\]Y\. Liang, H\. Wen, Y\. Nie, Y\. Jiang, M\. Jin, D\. Song, S\. Pan, and Q\. Wen\(2024\)Foundation models for time series analysis: a tutorial and survey\.InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining,pp\. 6555–6565\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p3.1),[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[29\]Y\. Liang, H\. Wen, Y\. Xia, M\. Jin, B\. Yang, F\. Salim, Q\. Wen, S\. Pan, and G\. Cong\(2025\)Foundation models for spatio\-temporal data science: a tutorial and survey\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 2,pp\. 6063–6073\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p1.1),[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[30\]Y\. Liang, Y\. Xia, S\. Ke, Y\. Wang, Q\. Wen, J\. Zhang, Y\. Zheng, and R\. Zimmermann\(2023\)Airformer: predicting nationwide air quality in china with transformers\.InProceedings of the AAAI conference on artificial intelligence,Vol\.37,pp\. 14329–14337\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[31\]B\. Lim and S\. Zohren\(2021\)Time\-series forecasting with deep learning: a survey\.Philosophical transactions of the royal society a: mathematical, physical and engineering sciences379\(2194\)\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p3.1)\.
- \[32\]C\. Liu, T\. Aksu, J\. Liu, X\. Liu, H\. Yan, Q\. Pham, S\. Savarese, D\. Sahoo, C\. Xiong, and J\. Li\(2025\)Moirai 2\.0: when less is more for time series forecasting\.arXiv preprint arXiv:2511\.11698\.Cited by:[2nd item](https://arxiv.org/html/2607.16251#A3.I1.i2.p1.1),[§2](https://arxiv.org/html/2607.16251#S2.p1.1),[2nd item](https://arxiv.org/html/2607.16251#S6.I2.i2.p1.1)\.
- \[33\]X\. Liu, T\. Aksu, J\. Liu, Q\. Wen, Y\. Liang, C\. Xiong, S\. Savarese, D\. Sahoo, J\. Li, and C\. Liu\(2025\)Empowering time series analysis with synthetic data: a survey and outlook in the era of foundation models\.arXiv preprint arXiv:2503\.11411\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.
- \[34\]X\. Liu, J\. Hu, Y\. Li, S\. Diao, Y\. Liang, B\. Hooi, and R\. Zimmermann\(2024\)Unitime: a language\-empowered unified model for cross\-domain time series forecasting\.InProceedings of the ACM Web Conference 2024,pp\. 4095–4106\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[35\]X\. Liu, J\. Liu, G\. Woo, T\. Aksu, Y\. Liang, R\. Zimmermann, C\. Liu, S\. Savarese, C\. Xiong, and D\. Sahoo\(2024\)Moirai\-moe: empowering time series foundation models with sparse mixture of experts\.arXiv preprint arXiv:2410\.10469\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[36\]X\. Liu, Y\. Xia, Y\. Liang, J\. Hu, Y\. Wang, L\. Bai, C\. Huang, Z\. Liu, B\. Hooi, and R\. Zimmermann\(2023\)Largest: a benchmark dataset for large\-scale traffic forecasting\.Advances in Neural Information Processing Systems36,pp\. 75354–75371\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.
- \[37\]I\. Loshchilov and F\. Hutter\(2017\)Decoupled weight decay regularization\.arXiv preprint arXiv:1711\.05101\.Cited by:[§6\.1](https://arxiv.org/html/2607.16251#S6.SS1.p2.2)\.
- \[38\]T\. Nguyen, J\. Brandstetter, A\. Kapoor, J\. K\. Gupta, and A\. Grover\(2023\)Climax: a foundation model for weather and climate\.arXiv preprint arXiv:2301\.10343\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[39\]J\. Ni, S\. Wang, M\. Jin, Q\. He, and W\. Jin\(2026\)STReasoner: empowering llms for spatio\-temporal reasoning in time series via spatial\-aware reinforcement learning\.arXiv preprint arXiv:2601\.03248\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[40\]Z\. Qiu, Z\. Wang, B\. Zheng, Z\. Huang, K\. Wen, S\. Yang, R\. Men, L\. Yu, F\. Huang, S\. Huang,et al\.\(2025\)Gated attention for large language models: non\-linearity, sparsity, and attention\-sink\-free\.arXiv preprint arXiv:2505\.06708\.Cited by:[§4\.2](https://arxiv.org/html/2607.16251#S4.SS2.p2.6)\.
- \[41\]H\. Qu, R\. Morel, M\. McCabe, A\. Bietti, F\. Lanusse, S\. Ho, and Y\. LeCun\(2026\)Representation learning for spatiotemporal physical systems\.arXiv preprint arXiv:2603\.13227\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p1.1)\.
- \[42\]Y\. Seo, M\. Defferrard, P\. Vandergheynst, and X\. Bresson\(2016\)Structured sequence modeling with graph convolutional recurrent networks\.External Links:1612\.07659,[Link](https://arxiv.org/abs/1612.07659)Cited by:[§4\.3](https://arxiv.org/html/2607.16251#S4.SS3.p2.13)\.
- \[43\]J\. Shi, Q\. Ma, H\. Ma, and L\. Li\(2024\)Scaling law for time series forecasting\.Advances in Neural Information Processing Systems37,pp\. 83314–83344\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1)\.
- \[44\]X\. Shi, S\. Wang, Y\. Nie, D\. Li, Z\. Ye, Q\. Wen, and M\. Jin\(2024\)Time\-moe: billion\-scale time series foundation models with mixture of experts\.arXiv preprint arXiv:2409\.16040\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[45\]J\. Su, M\. Ahmed, Y\. Lu, S\. Pan, W\. Bo, and Y\. Liu\(2024\)Roformer: enhanced transformer with rotary position embedding\.Neurocomputing568,pp\. 127063\.Cited by:[§4\.2](https://arxiv.org/html/2607.16251#S4.SS2.p2.6)\.
- \[46\]X\. Wang, H\. Zhang, P\. Wang, Y\. Zhang, B\. Wang, Z\. Zhou, and Y\. Wang\(2023\)An observed value consistent diffusion model for imputing missing values in multivariate time series\.InProceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining,pp\. 2409–2418\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p2.1)\.
- \[47\]Y\. Wang, Y\. Qiu, P\. Chen, K\. Zhao, Y\. Shu, Z\. Rao, L\. Pan, B\. Yang, and C\. Guo\(2024\)ROSE: register\-assisted general time series forecasting with decomposed frequency learning\.Cited by:[2nd item](https://arxiv.org/html/2607.16251#S6.I2.i2.p1.1)\.
- \[48\]H\. Wen, Y\. Lin, Y\. Xia, H\. Wan, Q\. Wen, R\. Zimmermann, and Y\. Liang\(2023\)Diffstg: probabilistic spatio\-temporal graph forecasting with denoising diffusion models\.InProceedings of the 31st ACM international conference on advances in geographic information systems,pp\. 1–12\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p3.1)\.
- \[49\]S\. Wold, K\. Esbensen, and P\. Geladi\(1987\)Principal component analysis\.Chemometrics and intelligent laboratory systems2\(1\-3\),pp\. 37–52\.Cited by:[§5](https://arxiv.org/html/2607.16251#S5.p1.7)\.
- \[50\]G\. Woo, C\. Liu, A\. Kumar, C\. Xiong, S\. Savarese, and D\. Sahoo\(2024\)Unified training of universal time series forecasting transformers\.Cited by:[§2](https://arxiv.org/html/2607.16251#S2.p1.1)\.
- \[51\]L\. Wu, H\. Wen, H\. Hu, X\. Mao, Y\. Xia, E\. Shan, J\. Zheng, J\. Lou, Y\. Liang, L\. Yang,et al\.\(2024\)LaDe: the first comprehensive last\-mile express dataset from industry\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 5991–6002\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.
- \[52\]Y\. Xia, C\. Xu, Y\. Liang, Q\. Wen, R\. Zimmermann, and J\. Bian\(2025\)Causal time series generation via diffusion models\.arXiv preprint arXiv:2509\.20846\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.
- \[53\]Y\. Yang, M\. Jin, H\. Wen, C\. Zhang, Y\. Liang, L\. Ma, Y\. Wang, C\. Liu, B\. Yang, Z\. Xu, J\. Bian, S\. Pan, and Q\. Wen\(2024\)A survey on diffusion models for time series and spatio\-temporal data\.External Links:2404\.18886Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p3.1)\.
- \[54\]Y\. Yang, M\. Jin, H\. Wen, C\. Zhang, Y\. Liang, L\. Ma, Y\. Wang, C\. Liu, B\. Yang, Z\. Xu, S\. Pan, and Q\. Wen\(2026\-02\)A survey on diffusion models for time series and spatio\-temporal data\.ACM Comput\. Surv\.58\(8\)\.External Links:ISSN 0360\-0300,[Link](https://doi.org/10.1145/3783986),[Document](https://dx.doi.org/10.1145/3783986)Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p3.1)\.
- \[55\]Q\. Yao, C\. H\. Yang, R\. Jiang, Y\. Liang, M\. Jin, and S\. Pan\(2024\)Towards neural scaling laws for time series foundation models\.arXiv preprint arXiv:2410\.12360\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1)\.
- \[56\]Y\. Yuan, J\. Ding, J\. Feng, D\. Jin, and Y\. Li\(2024\)Unist: a prompt\-empowered universal model for urban spatio\-temporal prediction\.Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 4095–4106\.Cited by:[1st item](https://arxiv.org/html/2607.16251#A3.I2.i1.p1.1),[§2](https://arxiv.org/html/2607.16251#S2.p1.1),[1st item](https://arxiv.org/html/2607.16251#S6.I2.i1.p1.1)\.
- \[57\]H\. Zhang, Y\. Xia, S\. Zhong, K\. Wang, Z\. Tong, Q\. Wen, R\. Zimmermann, and Y\. Liang\(2024\)Predicting parking availability in singapore with cross\-domain data: a new dataset and a data\-driven approach\.arXiv preprint arXiv:2405\.18910\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.
- \[58\]K\. Zhang, Q\. Wen, C\. Zhang, R\. Cai, M\. Jin, Y\. Liu, J\. Y\. Zhang, Y\. Liang, G\. Pang, D\. Song,et al\.\(2024\)Self\-supervised learning for time series analysis: taxonomy, progress, and prospects\.IEEE transactions on pattern analysis and machine intelligence46\(10\),pp\. 6775–6794\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px2.p1.1)\.
- \[59\]W\. Zhang, B\. Terver, A\. Zholus, S\. Chitnis, H\. Sutaria, M\. Assran, R\. Balestriero, A\. Bar, A\. Bardes, Y\. LeCun,et al\.\(2026\)Hierarchical planning with latent world models\.arXiv preprint arXiv:2604\.03208\.Cited by:[§1](https://arxiv.org/html/2607.16251#S1.p1.1)\.
- \[60\]S\. Zhong, J\. Qiu, Y\. Wu, X\. Zou, Z\. Rao, B\. Yang, C\. Guo, H\. Xu, and Y\. LiangLearning to factorize spatio\-temporal foundation models\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[1st item](https://arxiv.org/html/2607.16251#S6.I2.i1.p1.1),[§6\.1](https://arxiv.org/html/2607.16251#S6.SS1.p2.2)\.
- \[61\]D\. Zhu, H\. Huang, Z\. Huang, Y\. Zeng, Y\. Mao, B\. Wu, Q\. Min, and X\. Zhou\(2024\)Hyper\-connections\.arXiv preprint arXiv:2409\.19606\.Cited by:[§4](https://arxiv.org/html/2607.16251#S4.p1.9)\.
- \[62\]Y\. Zhu, Y\. Ye, S\. Zhang, X\. Zhao, and J\. Yu\(2023\)Difftraj: generating gps trajectory with diffusion probabilistic model\.Advances in Neural Information Processing Systems36,pp\. 65168–65188\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.
- \[63\]Y\. Zhu, J\. J\. Yu, X\. Zhao, Q\. Liu, Y\. Ye, W\. Chen, Z\. Zhang, X\. Wei, and Y\. Liang\(2024\)Controltraj: controllable trajectory generation with topology\-constrained diffusion model\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 4676–4687\.Cited by:[Appendix A](https://arxiv.org/html/2607.16251#A1.SS0.SSS0.Px1.p1.1)\.

## Appendix ARelated Works

##### Synthetic Spatio\-Temporal Data\.

The use of synthetic data for time series and spatio\-temporal modeling has gained increasing attention, particularly in the era of foundation models\. A comprehensive survey is provided byLiuet al\.\[[33](https://arxiv.org/html/2607.16251#bib.bib23)\], which highlights procedural and simulation\-based generation as scalable alternatives to real\-world data collection\. In contrast, most existing spatio\-temporal models are trained on a limited set of real\-world benchmarks, predominantly from transportation and urban domains, such as PEMS\[[16](https://arxiv.org/html/2607.16251#bib.bib27)\], LargeST\[[36](https://arxiv.org/html/2607.16251#bib.bib29)\], SINPA\[[57](https://arxiv.org/html/2607.16251#bib.bib30)\], and LaDe\[[51](https://arxiv.org/html/2607.16251#bib.bib31)\]\. While influential, these datasets share similar sensing infrastructures and domain biases, restricting distributional diversity\. Extending synthesis to the spatio\-temporal domain introduces additional complexity, as models must jointly capture spatial dependencies and temporal dynamics\. GAN\-based approaches\[[13](https://arxiv.org/html/2607.16251#bib.bib24)\]generate realistic mobility traces or sensor readings but are typically domain\-specific\. PDE\-based simulators\[[2](https://arxiv.org/html/2607.16251#bib.bib25)\]construct synthetic spatio\-temporal graphs governed by physical dynamics, offering improved controllability\. Diffusion models for trajectory\[[62](https://arxiv.org/html/2607.16251#bib.bib32),[63](https://arxiv.org/html/2607.16251#bib.bib33)\]and time series\[[52](https://arxiv.org/html/2607.16251#bib.bib35)\]generation have also been explored\. Despite these efforts, synthetic data is rarely used as theprimarysource for STFM pre\-training\. Most existing STFMs still rely heavily on aggregated real\-world datasets, inevitably inheriting their inherent domain biases and persistent data quality limitations\.

##### Training Paradigms and Objectives for Foundation Models\.

Traditional point\-wise regression losses in the observation space often struggle to capture global structural similarities or temporal dynamics, prompting the use of specialized alignment metrics\[[18](https://arxiv.org/html/2607.16251#bib.bib19),[7](https://arxiv.org/html/2607.16251#bib.bib20),[22](https://arxiv.org/html/2607.16251#bib.bib21)\]\. In parallel, Self\-Supervised Learning \(SSL\)\[[58](https://arxiv.org/html/2607.16251#bib.bib59)\]has transformed representation learning\. A particularly promising evolution is the Joint\-Embedding Predictive Architecture \(JEPA\)\[[3](https://arxiv.org/html/2607.16251#bib.bib55),[4](https://arxiv.org/html/2607.16251#bib.bib56)\]\. By forcing models to predict missing representations entirely within an abstract latent space rather than reconstructing raw values, JEPA inherently discards high\-frequency noise and focuses on semantic features\. This paradigm has recently achieved notable success in spatio\-temporal tasks, such as trajectory representation learning via T\-JEPA\[[24](https://arxiv.org/html/2607.16251#bib.bib57)\]and HiT\-JEPA\[[23](https://arxiv.org/html/2607.16251#bib.bib58)\]\. These developments conceptually echo our proposed Latent Alignment Loss \(LAL\) in NeoST, reinforcing the necessity of latent\-space reasoning for complex dynamics\. Furthermore, the shift toward foundation\-level modeling is fundamentally intertwined with neural scaling laws\. Recent empirical studies demonstrate that TSFMs adhere to predictable scaling behaviors, where performance consistently improves alongside increases in model parameters and dataset size\[[55](https://arxiv.org/html/2607.16251#bib.bib62),[43](https://arxiv.org/html/2607.16251#bib.bib63),[9](https://arxiv.org/html/2607.16251#bib.bib64)\]\. By coupling a highly scalable synthetic data engine with efficient latent\-space reasoning, NeoST is inherently positioned to exploit these scaling principles, circumventing the finite scale and inherent biases of real\-world spatio\-temporal datasets\.

## Appendix BTheoretical Properties of Synthetic Data

This appendix characterizes the dynamical properties induced by the proposed synthetic data generator\. Rather than instantiating a single fixed dynamical law, the generator samples graph topologies, generator parameters, biases, observation maps, and masking patterns, thereby defining a distribution over stochastic graph\-coupled dynamical systems\. Consequently, the synthetic corpus can contain a broad range of spatio\-temporal regimes, including spatial dependence, temporal autocorrelation, delayed feedback, diffusion\-like smoothing, wave\-like propagation, quasi\-periodic responses, nonlinear saturation, heteroscedastic stochasticity, and partial observability\.

LetΘ\\Thetadenote the sampled system configuration for one synthetic instance\. Unless otherwise stated, all statements below are conditioned on a fixedΘ\\Theta\.

##### Distribution over dynamical systems\.

Since each synthetic instance samples its own graph, generator weights, bias, and observation projection, the overall data distribution can be written as

p​\(𝐗\)=∫pΘ​\(𝐗\)​𝑑ν​\(Θ\),p\(\\mathbf\{X\}\)=\\int p\_\{\\Theta\}\(\\mathbf\{X\}\)\\,d\\nu\(\\Theta\),\(5\)whereν\\nuis the sampling distribution over system configurations\. This mixture interpretation is central to the design of the generator: different sampled systems may exhibit different spatial scales, temporal responses, stochastic intensities, nonlinear regimes, and observation patterns\. Thus, the corpus covers a heterogeneous family of structured spatio\-temporal processes rather than trajectories from a single simulator\.

###### Proposition B\.1\(Conditional second\-order Markov structure\)\.

For a fixed sampled configurationΘ\\Theta, the latent process satisfies

p​\(𝐇t\+1∣𝐇t,𝐇t−1,…,𝐇0,Θ\)=p​\(𝐇t\+1∣𝐇t,𝐇t−1,Θ\)\.p\(\\mathbf\{H\}\_\{t\+1\}\\mid\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\ldots,\\mathbf\{H\}\_\{0\},\\Theta\)=p\(\\mathbf\{H\}\_\{t\+1\}\\mid\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\Theta\)\.\(6\)Equivalently,

𝐇~t=\[𝐇t𝐇t−1\]\\widetilde\{\\mathbf\{H\}\}\_\{t\}=\\begin\{bmatrix\}\\mathbf\{H\}\_\{t\}\\\\ \\mathbf\{H\}\_\{t\-1\}\\end\{bmatrix\}\(7\)forms a first\-order time\-homogeneous Markov chain conditioned onΘ\\Theta\.

###### Proof\.

Conditioned onΘ\\Theta, the one\-step update depends on the past only through\(𝐇t,𝐇t−1\)\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\}\)and the fresh innovationϵt\\boldsymbol\{\\epsilon\}\_\{t\}, which is independent of previous states\. The augmented\-state representation then gives a first\-order Markov chain\. ∎

The conditioning onΘ\\Thetais essential\. Since generator parameters are sampled per instance, the marginal process obtained after integrating outΘ\\Thetais a mixture of Markov processes and need not itself be second\-order Markov\.

###### Proposition B\.2\(Conditionally Gaussian heteroscedastic transition\)\.

For fixedΘ\\Theta, the transition distribution satisfies

vec​\(𝐇t\+1\)∣𝐇t,𝐇t−1,Θ∼𝒩​\(𝐦t,𝚺t\),\\mathrm\{vec\}\(\\mathbf\{H\}\_\{t\+1\}\)\\mid\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\Theta\\sim\\mathcal\{N\}\(\\mathbf\{m\}\_\{t\},\\mathbf\{\\Sigma\}\_\{t\}\),\(8\)where

𝐦t=vec​\(a​𝐇t\+𝝁​\(𝐇t,𝐇t−1,𝐀\)\+𝐛\),\\mathbf\{m\}\_\{t\}=\\mathrm\{vec\}\\\!\\left\(a\\mathbf\{H\}\_\{t\}\+\\boldsymbol\{\\mu\}\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\mathbf\{A\}\)\+\\mathbf\{b\}\\right\),\(9\)and

𝚺t=diag⁡\(vec​\(𝝈​\(𝐇t,𝐇t−1,𝐀\)2\)\)\.\\mathbf\{\\Sigma\}\_\{t\}=\\operatorname\{diag\}\\left\(\\mathrm\{vec\}\\left\(\\boldsymbol\{\\sigma\}\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\mathbf\{A\}\)^\{2\}\\right\)\\right\)\.\(10\)Thus, the latent dynamics are conditionally Gaussian with state\-dependent covariance\.

###### Proof\.

Given\(𝐇t,𝐇t−1\)\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\}\)andΘ\\Theta, the only random component in the update is

𝝈​\(𝐇t,𝐇t−1,𝐀\)⊙ϵt\.\\boldsymbol\{\\sigma\}\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\mathbf\{A\}\)\\odot\\boldsymbol\{\\epsilon\}\_\{t\}\.Sinceϵt\\boldsymbol\{\\epsilon\}\_\{t\}has independent standard Gaussian entries, element\-wise scaling yields a diagonal Gaussian covariance\. ∎

Although each one\-step transition is conditionally Gaussian, the process is generally not marginally Gaussian, because both its conditional mean and conditional variance depend nonlinearly on graph\-filtered latent states\. This state\-dependent variance introduces heteroscedastic stochasticity, a common feature of real spatio\-temporal sensing systems\.

###### Theorem B\.3\(Spectrally bounded graph filtering\)\.

Assume the sampled graph is undirected with nonnegative weights and nonzero degrees\. Then

spec⁡\(𝐋^\)⊂\[−1,1\],\\operatorname\{spec\}\(\\widehat\{\\mathbf\{L\}\}\)\\subset\[\-1,1\],\(11\)and for any Chebyshev polynomialTkT\_\{k\},

‖Tk​\(𝐋^\)‖2≤1\.\\\|T\_\{k\}\(\\widehat\{\\mathbf\{L\}\}\)\\\|\_\{2\}\\leq 1\.\(12\)In particular,

‖𝐓1‖2≤1,‖𝐓2‖2≤1\.\\\|\\mathbf\{T\}\_\{1\}\\\|\_\{2\}\\leq 1,\\qquad\\\|\\mathbf\{T\}\_\{2\}\\\|\_\{2\}\\leq 1\.\(13\)

###### Proof\.

For an undirected graph with nonnegative weights, the normalized Laplacian has spectrum in\[0,2\]\[0,2\]\. Hence𝐋^=𝐋−𝐈\\widehat\{\\mathbf\{L\}\}=\\mathbf\{L\}\-\\mathbf\{I\}has spectrum in\[−1,1\]\[\-1,1\]\. Since\|Tk​\(x\)\|≤1\|T\_\{k\}\(x\)\|\\leq 1forx∈\[−1,1\]x\\in\[\-1,1\], symmetry of𝐋^\\widehat\{\\mathbf\{L\}\}gives

‖Tk​\(𝐋^\)‖2=maxλ∈spec⁡\(𝐋^\)⁡\|Tk​\(λ\)\|≤1\.\\\|T\_\{k\}\(\\widehat\{\\mathbf\{L\}\}\)\\\|\_\{2\}=\\max\_\{\\lambda\\in\\operatorname\{spec\}\(\\widehat\{\\mathbf\{L\}\}\)\}\|T\_\{k\}\(\\lambda\)\|\\leq 1\.∎

Theorem[B\.3](https://arxiv.org/html/2607.16251#A2.Thmtheorem3)concerns the stability of the graph filtering operators\. At the same time,𝐓1​𝐇t\\mathbf\{T\}\_\{1\}\\mathbf\{H\}\_\{t\}and𝐓2​𝐇t−1\\mathbf\{T\}\_\{2\}\\mathbf\{H\}\_\{t\-1\}explicitly couple neighboring or spectrally related nodes\. Therefore, spatial dependence, graph smoothness, and localized propagation are structural properties induced by the generator\.

###### Theorem B\.4\(Dissipative moment control\)\.

Fix a sampled configurationΘ\\Theta\. If the initial latent states have finite second moments, then there existsCΘ<∞C\_\{\\Theta\}<\\inftysuch that

supt≥0𝔼Θ​‖𝐇t‖F2≤CΘ\.\\sup\_\{t\\geq 0\}\\mathbb\{E\}\_\{\\Theta\}\\\|\\mathbf\{H\}\_\{t\}\\\|\_\{F\}^\{2\}\\leq C\_\{\\Theta\}\.\(14\)

###### Proof\.

Sincetanh⁡\(⋅\)\\tanh\(\\cdot\)is bounded and the sampled weights are finite\-dimensional, there existsMμ,Θ<∞M\_\{\\mu,\\Theta\}<\\inftysuch that

‖𝝁​\(𝐇t,𝐇t−1,𝐀\)‖F≤Mμ,Θ\\\|\\boldsymbol\{\\mu\}\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\mathbf\{A\}\)\\\|\_\{F\}\\leq M\_\{\\mu,\\Theta\}for all states\. Moreover, becausesigmoid​\(z\)∈\(0,1\)\\mathrm\{sigmoid\}\(z\)\\in\(0,1\),

𝔼Θ​\[‖𝝈​\(𝐇t,𝐇t−1,𝐀\)⊙ϵt‖F2∣ℱt\]≤N​dh\.\\mathbb\{E\}\_\{\\Theta\}\\left\[\\\|\\boldsymbol\{\\sigma\}\(\\mathbf\{H\}\_\{t\},\\mathbf\{H\}\_\{t\-1\},\\mathbf\{A\}\)\\odot\\boldsymbol\{\\epsilon\}\_\{t\}\\\|\_\{F\}^\{2\}\\mid\\mathcal\{F\}\_\{t\}\\right\]\\leq Nd\_\{h\}\.\(15\)Chooseη\>0\\eta\>0such thatα=\(1\+η\)​a2<1\\alpha=\(1\+\\eta\)a^\{2\}<1\. Then

𝔼Θ​\[‖𝐇t\+1‖F2∣ℱt\]≤α​‖𝐇t‖F2\+C0,Θ,\\mathbb\{E\}\_\{\\Theta\}\\left\[\\\|\\mathbf\{H\}\_\{t\+1\}\\\|\_\{F\}^\{2\}\\mid\\mathcal\{F\}\_\{t\}\\right\]\\leq\\alpha\\\|\\mathbf\{H\}\_\{t\}\\\|\_\{F\}^\{2\}\+C\_\{0,\\Theta\},\(16\)for someC0,Θ<∞C\_\{0,\\Theta\}<\\infty\. Iterating this inequality gives

𝔼Θ​‖𝐇t‖F2≤αt​𝔼Θ​‖𝐇0‖F2\+1−αt1−α​C0,Θ,\\mathbb\{E\}\_\{\\Theta\}\\\|\\mathbf\{H\}\_\{t\}\\\|\_\{F\}^\{2\}\\leq\\alpha^\{t\}\\mathbb\{E\}\_\{\\Theta\}\\\|\\mathbf\{H\}\_\{0\}\\\|\_\{F\}^\{2\}\+\\frac\{1\-\\alpha^\{t\}\}\{1\-\\alpha\}C\_\{0,\\Theta\},which proves the claim\. ∎

This result establishes moment control of the generated trajectories\. It should not be interpreted as a universal geometric\-ergodicity or total\-variation mixing\-time guarantee, which would require additional irreducibility and minorization assumptions\. For synthetic data generation, the main implication is that long trajectories can be generated while keeping latent magnitudes controlled\.

##### Temporal memory and oscillatory responses\.

The dependence on both𝐇t\\mathbf\{H\}\_\{t\}and𝐇t−1\\mathbf\{H\}\_\{t\-1\}introduces delayed feedback\. This second\-order structure enables temporal autocorrelation, phase\-lagged responses, and oscillatory patterns\. Around a reference state𝐳¯\\bar\{\\mathbf\{z\}\}, the conditional mean dynamics admit the local linearization

δ​𝐳t\+1≈𝐉​\(𝐳¯\)​δ​𝐳t\+𝐆​\(𝐳¯\)​ϵt,𝐳t=\[vec​\(𝐇t\)vec​\(𝐇t−1\)\],\\delta\\mathbf\{z\}\_\{t\+1\}\\approx\\mathbf\{J\}\(\\bar\{\\mathbf\{z\}\}\)\\delta\\mathbf\{z\}\_\{t\}\+\\mathbf\{G\}\(\\bar\{\\mathbf\{z\}\}\)\\boldsymbol\{\\epsilon\}\_\{t\},\\qquad\\mathbf\{z\}\_\{t\}=\\begin\{bmatrix\}\\mathrm\{vec\}\(\\mathbf\{H\}\_\{t\}\)\\\\ \\mathrm\{vec\}\(\\mathbf\{H\}\_\{t\-1\}\)\\end\{bmatrix\},\(17\)where

𝐉​\(𝐳¯\)=\[a​𝐈\+D1​𝝁​\(𝐳¯\)D2​𝝁​\(𝐳¯\)𝐈𝟎\],\\mathbf\{J\}\(\\bar\{\\mathbf\{z\}\}\)=\\begin\{bmatrix\}a\\mathbf\{I\}\+D\_\{1\}\\boldsymbol\{\\mu\}\(\\bar\{\\mathbf\{z\}\}\)&D\_\{2\}\\boldsymbol\{\\mu\}\(\\bar\{\\mathbf\{z\}\}\)\\\\ \\mathbf\{I\}&\\mathbf\{0\}\\end\{bmatrix\},\(18\)and𝐆​\(𝐳¯\)\\mathbf\{G\}\(\\bar\{\\mathbf\{z\}\}\)denotes the diffusion amplitude evaluated at the same reference state\. HereD1​𝝁D\_\{1\}\\boldsymbol\{\\mu\}andD2​𝝁D\_\{2\}\\boldsymbol\{\\mu\}are the Jacobian blocks of the vectorized drift with respect tovec​\(𝐇t\)\\mathrm\{vec\}\(\\mathbf\{H\}\_\{t\}\)andvec​\(𝐇t−1\)\\mathrm\{vec\}\(\\mathbf\{H\}\_\{t\-1\}\)\.

###### Proposition B\.5\(Local mechanism for quasi\-periodic responses\)\.

Suppose𝐉​\(𝐳¯\)\\mathbf\{J\}\(\\bar\{\\mathbf\{z\}\}\)has a stable complex conjugate eigenvalue pair

λ1,2=r​e±i​θ,0<r<1,θ∈\(0,π\)\.\\lambda\_\{1,2\}=re^\{\\pm i\\theta\},\\qquad 0<r<1,\\qquad\\theta\\in\(0,\\pi\)\.If the corresponding mode is excited by the stochastic innovation and observed by the output projection, then the observed sequence has an enhanced resonant response near

fosc=θ2​π\.f\_\{\\mathrm\{osc\}\}=\\frac\{\\theta\}\{2\\pi\}\.\(19\)

###### Proof sketch\.

For the frozen\-diffusion linear system, the spectral response of a linear readout is governed by

\(𝐈−e−i​ω​𝐉​\(𝐳¯\)\)−1\.\(\\mathbf\{I\}\-e^\{\-i\\omega\}\\mathbf\{J\}\(\\bar\{\\mathbf\{z\}\}\)\)^\{\-1\}\.A complex eigenvalue pairr​e±i​θre^\{\\pm i\\theta\}enhances the response near angular frequencies±θ\\pm\\theta\. If this mode is both excited and observed, the output spectrum contains a resonant component nearθ/\(2​π\)\\theta/\(2\\pi\)\. ∎

This proposition identifies a local mechanism for quasi\-periodic behavior rather than exact periodicity\. With Gaussian excitation, damped oscillatory modes can be repeatedly stimulated and appear as oscillatory autocorrelation, spectral concentration, or phase\-lagged propagation\.

##### Graph\-spectral diversity\.

The graph filters couple temporal feedback with graph\-spectral structure\. If\(𝐪i,si\)\(\\mathbf\{q\}\_\{i\},s\_\{i\}\)is an eigenpair of𝐋^\\widehat\{\\mathbf\{L\}\}, then

𝐓1​𝐪i=si​𝐪i,𝐓2​𝐪i=\(2​si2−1\)​𝐪i\.\\mathbf\{T\}\_\{1\}\\mathbf\{q\}\_\{i\}=s\_\{i\}\\mathbf\{q\}\_\{i\},\\qquad\\mathbf\{T\}\_\{2\}\\mathbf\{q\}\_\{i\}=\(2s\_\{i\}^\{2\}\-1\)\\mathbf\{q\}\_\{i\}\.\(20\)Thus, different graph frequencies receive different first\- and second\-order feedback\. Since2​si2−12s\_\{i\}^\{2\}\-1changes sign at\|si\|=1/2\|s\_\{i\}\|=1/\\sqrt\{2\}, some spectral components may behave like smoothing or diffusion modes, while others may behave like delayed\-feedback or propagation modes\. Although the nonlinear transformations and channel mixing prevent a global decomposition into independent graph modes, this view explains why diffusion\-like, wave\-like, and oscillatory patterns can arise in the synthetic data\.

##### Nonlinear and irregular regimes\.

The bounded nonlinearitytanh⁡\(⋅\)\\tanh\(\\cdot\)allows the generator to interpolate between near\-linear and saturated regimes\. Small effective weights lead to mean\-reverting stochastic autoregressive behavior, whereas larger effective weights can push the drift into saturation, producing threshold\-like transitions, bursts, plateaus, and intermittent dynamics\.

The deterministic skeleton can also be used to diagnose chaotic\-like finite\-time sensitivity\. Removing the Gaussian excitation yields

𝐳t\+1=𝐟Θ​\(𝐳t\)\.\\mathbf\{z\}\_\{t\+1\}=\\mathbf\{f\}\_\{\\Theta\}\(\\mathbf\{z\}\_\{t\}\)\.Let

𝚽T​\(𝐳0\)=D​𝐟Θ​\(𝐳T−1\)​⋯​D​𝐟Θ​\(𝐳0\)\.\\mathbf\{\\Phi\}\_\{T\}\(\\mathbf\{z\}\_\{0\}\)=D\\mathbf\{f\}\_\{\\Theta\}\(\\mathbf\{z\}\_\{T\-1\}\)\\cdots D\\mathbf\{f\}\_\{\\Theta\}\(\\mathbf\{z\}\_\{0\}\)\.The finite\-time Lyapunov diagnostic is

λT​\(𝐳0\)=1T​log⁡‖𝚽T​\(𝐳0\)‖2\.\\lambda\_\{T\}\(\\mathbf\{z\}\_\{0\}\)=\\frac\{1\}\{T\}\\log\\left\\\|\\mathbf\{\\Phi\}\_\{T\}\(\\mathbf\{z\}\_\{0\}\)\\right\\\|\_\{2\}\.\(21\)A positiveλT\\lambda\_\{T\}indicates local exponential separation of nearby initial conditions over the finite window\. Such behavior is not universal across sampled systems, but it can occur in high\-gain nonlinear regimes\. Since the drift is bounded anda∈\(0,1\)a\\in\(0,1\), local expansion can coexist with global moment control, yielding bounded but irregular chaotic\-like trajectories\.

##### Observation, masking, and corpus\-level coverage\.

The observed sequence is obtained through a random projection of the latent state, so latent modes may be amplified, attenuated, or hidden by the observation map\. Random masking further introduces external observational corruption without changing the latent transition law, mimicking realistic settings where structured dynamics are only partially observed\.

The randomization over graphs, weights, noise amplitudes, and observation maps provides corpus\-level regime diversity\. Ifℛ\\mathcal\{R\}is a set of configurations inducing a particular spatio\-temporal regime andν​\(ℛ\)=pℛ\>0\\nu\(\\mathcal\{R\}\)=p\_\{\\mathcal\{R\}\}\>0, then a corpus ofMMindependently sampled systems contains at least one system fromℛ\\mathcal\{R\}with probability

1−\(1−pℛ\)M\.1\-\(1\-p\_\{\\mathcal\{R\}\}\)^\{M\}\.\(22\)Thus, any regime with nonzero probability under the sampling distribution appears in sufficiently large synthetic corpora with high probability\.

##### Summary\.

In summary, the generator produces a controlled yet diverse family of graph\-coupled stochastic processes: \(i\) conditioned on each sampled system, the latent dynamics are second\-order Markov with conditionally Gaussian and heteroscedastic transitions; \(ii\) Chebyshev graph filtering induces spectrally controlled spatial dependence and localized propagation; \(iii\) the dissipative coefficient and bounded nonlinearities provide moment\-controlled long trajectories; \(iv\) the second\-order recurrence enables temporal memory, delayed feedback, and quasi\-periodic responses; \(v\) graph\-spectral filtering supports diffusion\-like, wave\-like, and oscillatory patterns across different spectral components; \(vi\) nonlinear saturation and high\-gain regimes can yield intermittent or chaotic\-like behavior; and \(vii\) random projection and masking induce partial and imperfect observation\. These properties align the synthetic corpus with the structural characteristics of real\-world spatio\-temporal forecasting tasks while keeping the generation process controlled and scalable\.

## Appendix CSupplementary Experimental Details

### C\.1Evaluation Datasets

To evaluate downstream spatio\-temporal \(ST\) forecasting, we curate a suite of real\-world benchmarks spanning traffic flow, speed, electricity consumption, and meteorology, as summarized in Table[2](https://arxiv.org/html/2607.16251#A3.T2)\. These datasets are characterized by significant heterogeneity in spatial granularity \(e\.g\., urban regions vs\. individual sensors\), temporal resolutions \(ranging from 5\-minute to hourly intervals\), and diverse prediction targets\. Such multifaceted diversity provides a rigorous testbed to validate the cross\-task generalization and adaptability of FactoST across both short\- and long\-term forecasting horizons\.

Table 2:List of evaluation spatio\-temporal datasets\.
### C\.2Evaluation Metrics

To evaluate the prediction performance, we employ two widely used regression metrics: Mean Absolute Error \(MAE\) and Root Mean Squared Error \(RMSE\)\. Let𝐘=\{Y1,…,YN\}\\mathbf\{Y\}=\\\{Y\_\{1\},\\dots,Y\_\{N\}\\\}denote the ground truth of the spatio\-temporal data and𝐘^=\{Y^1,…,Y^N\}\\hat\{\\mathbf\{Y\}\}=\\\{\\hat\{Y\}\_\{1\},\\dots,\\hat\{Y\}\_\{N\}\\\}represent the values predicted by the model, whereNNis the total number of test samples\. These metrics are formally defined as follows:

RMSE​\(𝐘,𝐘^\)=1N​∑iN\(Yi−Y^i\)2,MAE​\(𝐘,𝐘^\)=1N​∑iN\|Yi−Y^i\|,\\text\{RMSE\}\(\\mathbf\{Y\},\\hat\{\\mathbf\{Y\}\}\)=\\sqrt\{\\frac\{1\}\{N\}\\sum\_\{i\}^\{N\}\\left\(Y\_\{i\}\-\\hat\{Y\}\_\{i\}\\right\)^\{2\}\},\\quad\\text\{MAE\}\(\\mathbf\{Y\},\\hat\{\\mathbf\{Y\}\}\)=\\frac\{1\}\{N\}\\sum\_\{i\}^\{N\}\\left\|Y\_\{i\}\-\\hat\{Y\}\_\{i\}\\right\|,\(23\)

### C\.3Baselines and Implementation

To ensure fair comparisons, we evaluate all baseline models within a unified framework using standardized metrics \(MAE and RMSE\)\. Model hyperparameters are either retained at their default settings, as reported in the original literature, or optimized via grid search on the validation set\. The specific implementation strategies for each baseline category are outlined below\.

#### C\.3\.1Time Series Foundation Models \(TSFMs\)

- •TimesFM\[[8](https://arxiv.org/html/2607.16251#bib.bib37)\]: Developed by Google Research, this is a large\-scale, pretrained, decoder\-only time series foundation model capable of high\-accuracy univariate forecasting across various domains and frequencies\. We utilize the official implementation \([https://github\.com/google\-research/timesfm](https://github.com/google-research/timesfm)\)\. The pretrained weights are sourced from[https://huggingface\.co/google/timesfm\-1\.0\-200m](https://huggingface.co/google/timesfm-1.0-200m)\.
- •

#### C\.3\.2Spatio\-Temporal Foundation Models \(STFMs\)

- •UniST\[[56](https://arxiv.org/html/2607.16251#bib.bib45)\]: A universal STFM driven by prompt learning, pretrained across multiple urban scenarios to achieve robust generalization\. As the official codebase \([https://github\.com/tsinghua\-fib\-lab/UniST](https://github.com/tsinghua-fib-lab/UniST)\) inherently supports only fixed 6\-step prediction horizons, we retrain the model on 13 datasets from its original release to fit our 12\- and 96\-step forecasting scenarios\.
- •OpenCity\[[27](https://arxiv.org/html/2607.16251#bib.bib46)\]: A versatile STFM supporting both zero\-shot and few\-shot forecasting across a wide range of city\-level applications\. We integrate this model into our pipeline using theOpencity\-plus\.pthcheckpoint\.

## Appendix DFurther Discussion

### D\.1Limitations & Future Improvements

While NeoST demonstrates strong zero\-shot generalization and inference efficiency across diverse real\-world benchmarks, we acknowledge several methodological and architectural boundaries in the current framework that provide important avenues for future research\.

Sim\-to\-Real Gap in Topology and Noise Distributions\.Our synthetic data generation engine relies on Erdős\-Rényi and Poisson degree models to sample spatial graph topologies\. Furthermore, the temporal evolution utilizes a standard Gaussian noise formulation,ϵt∼𝒩​\(0,I\)\\epsilon\_\{t\}\\sim\\mathcal\{N\}\(0,I\), for stochastic excitation\. However, real\-world complex networks \(e\.g\., urban traffic infrastructure, epidemiological contact networks\) frequently exhibit scale\-free or small\-world topological properties\. Similarly, exogenous shocks in physical systems often manifest as heavy\-tailed distributions rather than ideal Gaussian perturbations\. These idealized assumptions may introduce a “sim\-to\-real” distribution gap, potentially constraining the model’s robustness against extreme anomalies or black\-swan events\. Future iterations of the generative engine should incorporate scale\-free network generation algorithms \(e\.g\., the Barabási\-Albert model\) and non\-Gaussian jump processes \(e\.g\., Lévy flights\) to more accurately reflect real\-world macroscopic statistics\.

Computational Scalability of Test\-Time Adaptation\.To mitigate distribution shifts during inference, NeoST employs a Ridge Regression\-based Test\-Time Adaptation \(TTA\) mechanism\. This requires computing the closed\-form solutionw∗=\(Zo​b​s⊤​Zo​b​s\+λ​I\)−1​Zo​b​s⊤​v~o​b​sw^\{\*\}=\(Z\_\{obs\}^\{\\top\}Z\_\{obs\}\+\\lambda I\)^\{\-1\}Z\_\{obs\}^\{\\top\}\\tilde\{v\}\_\{obs\}\. While highly effective for the moderately sized spatial graphs evaluated in our benchmarks, the matrix inversion step incurs a cubic computational complexity\. For ultra\-large\-scale spatial networks \(e\.g\., nation\-wide power grids or comprehensive city\-level traffic systems with tens of thousands of nodes\), this online adaptation step may introduce latency bottlenecks, potentially offsetting the high inference efficiency inherent to our latent\-space reasoning paradigm\. Future work could explore iterative solvers, gradient\-based meta\-learning, or low\-rank approximations to accelerate this process at scale\.

Lack of Uncertainty Quantification\.The current NeoST architecture is designed exclusively for deterministic point forecasting, optimizing for reconstruction fidelity using the Frobenius norm in the objective function\. In many critical spatio\-temporal applications, such as extreme weather forecasting, epidemiological modeling, or autonomous fleet routing, point predictions are insufficient; decision\-makers require reliable uncertainty quantification and confidence intervals to assess risk\. The absence of native probabilistic forecasting capabilities limits the model’s immediate utility in risk\-sensitive scenarios\. Future work will extend the Output Adapter to predict the parameters of continuous probability distributions or integrate quantile regression mechanisms, thereby enabling robust and calibrated uncertainty estimation for downstream tasks\.

### D\.2Broader Impacts

The development of NeoST, a Spatio\-Temporal Foundation Model pre\-trained exclusively on synthetic data, offers significant societal benefits by mitigating the demographic, geographic, and socioeconomic biases typically encoded in real\-world sensor deployments\. Furthermore, its efficient architecture promotes environmentally sustainable deployment compared to massive foundation models\. However, deploying such models in safety\-critical domains such as urban traffic management and disaster response, carries inherent risks\. The "sim\-to\-real" distribution gap may cause the model to behave unpredictably during extreme, heavy\-tailed anomalies or black\-swan events, potentially endangering public safety or infrastructure security\. Additionally, while our pre\-training data involves no human subjects, the model’s capacity to analyze sequential mobility data could be misused for mass surveillance or discriminatory practices if fine\-tuned on sensitive, unanonymized tracking data\. To ensure safe and equitable use, we urge practitioners to integrate robust uncertainty quantification prior to deployment, strictly adhere to human rights and data privacy regulations, and validate the model rigorously against domain\-specific, real\-world conditions\.