How does a 102M-parameter transformer forecast multivariate time series?

I recently worked through the architecture of t0-alpha, a 101.6M-parameter foundation model for time-series forecasting. The design choice I found most interesting is that it separates two kinds of reasoning: Time attention learns how each variable evolves across time. Group attention allows related variables to exchange information. The rest of the architecture, briefly: inputs are split into patches of 32 time steps; each patch is embedded into a 512-dimensional representation; the model uses 24 transformer blocks: 16 time-attention and 8 group-attention; it uses time-aware rotary embeddings, RMSNorm and SwiGLU; it predicts nine quantiles for probabilistic forecasting; it supports a context window of up to 1,024 time steps. Its reported aggregate CRPS on GIFT-Eval is 0.4941, roughly in the same range as TimesFM 2.5 and Chronos-2, despite having only around 102M parameters. I wrote a visual, from-first-principles walkthrough here: https://towardsdatascience.com/time-series-llms-explained-with-t0-alpha/ I would be interested in other views on two questions: Does separating temporal attention from cross-variable attention provide a useful inductive bias? Can smaller, specialised foundation models remain competitive with much larger forecasting models? I am also running an iso-parameter GIFT-Eval comparison against rival foundation models and classical baselines, which I plan to write up next.