KITE：一种用于时间序列事件预测的知识注入式时序编码器

# KITE: A Knowledge-Injected Temporal Encoder for Time-Series Event Forecasting
Source: [https://arxiv.org/html/2610.12738](https://arxiv.org/html/2610.12738)
\\workshoptitle

Foundation Models for Temporal Systems: From Forecasting to World Modeling

Pratik JawaharAffiliation:Forgis AGRiccardo MaggioniAffiliation:ETH ZurichGian\-Alessandro LombardiAffiliation:Forgis AGMatteo Cozzi Marek Gajewski Federico Martelli Camilla MazzoleniAffiliation:ETH ZurichAffiliation:Forgis AGPhilipp Christian PetersenAffiliation:University of Vienna

###### Abstract

Forecasting rare, high\-cost events in multivariate time\-series systems is hard because their precursors are seldom observed, so a purely data\-driven model has little to learn from\. The engineers who run these systems, however, hold knowledge that precedes such events, information absent from the scarce labels\. We call this a*semantic prior*and inject it into a self\-supervised event forecaster through two switchable mechanisms, a covariate mechanism and an attention\-bias mechanism, each the identity at its null, so one architecture serves every prior type with no per\-dataset architecture search\. A prior helps when it supplies information the observed window does not already contain: in a short context an informative prior beats no prior in\-domain \(pooled h\-AUROC over the no\-prior baseline: a periodicity clock\+14%\+14\\%,p<0\.001p<0\.001; a conservation residual\+47%\+47\\%at a0\.250\.25label fraction\), and a working prior transfers out\-of\-domain \(\+53%\+53\\%\), substituting for feature transfer, while priors the window already recovers stay inert and the attention\-bias mechanism shows no measurable gain\. KITE is a switchable, attributable route to contextual conditioning, and a substrate for scaling toward cross\-domain, pretrained time\-series foundation*world models*that carry their governing structure explicitly\.

## 1Introduction

Multivariate time\-series systems, building\-management arrays, water networks, air\-quality monitors, power infrastructure, are watched by dense arrays of timestamped variables, and the events that matter most are rare and expensive to label, so a purely data\-driven model has little to learn from on exactly the stratum that matters\. A domain engineer, by contrast, holds knowledge that precedes any data, which variable is the high\-voltage load, that a reading above a threshold is a warning, that the daily cycle repeats every14401440minutes, that a pump drives a downstream temperature\. This a\-priori knowledge, which we term a*semantic prior*, is exactly the information the scarce labels do not carry\.

We use*semantic prior*for engineering or domain knowledge, available before any data is seen, about the variables of a system and their relations: a variable’s*identity*, a*bound*, a*periodicity*, a governing*conservation*law, or a*causal edge*\. That domain knowledge can act as a prior is an old idea, from classical feature engineering to Bayesian priors and inductive bias\[[1](https://arxiv.org/html/2610.12738#bib.bib1)\]; the question we take up is how to inject such heterogeneous knowledge into a self\-supervised event forecaster in a way that is general across prior types, simple to apply, and honest about attribution\.

Existing approaches address only slices of this problem: each injects a single prior type through an always\-on pathway entangled with the backbone \(Section[2](https://arxiv.org/html/2610.12738#S2)\), so a new prior or a new system means a new architecture, and none unifies the covariate and relational routes across the range of priors, bounds, conservation laws, periodicity, that engineers rely on\.

KITE \(Knowledge\-Injected Temporal Encoder\) addresses this gap by treating a prior as a*switchable input*to a single self\-supervised backbone rather than a change to its architecture\. Built on a joint\-embedding predictive architecture \(JEPA\) for multivariate time series\[[2](https://arxiv.org/html/2610.12738#bib.bib2),[3](https://arxiv.org/html/2610.12738#bib.bib3)\], it routes prior types through two mechanisms: mechanism \(I\), a*covariate mechanism*, admits per\-variable and derived priors as synthetic variables \(a static descriptor to the encoder, a future\-known signal to the horizon\-conditioned predictor, a past\-only derived law such as a conservation residual to a covariate\); mechanism \(II\), an*attention\-bias mechanism*, biases encoder attention with a relational prior such as a causal graph\. Relative to the JEPA backbone, only the future\-known covariate path and the whole of mechanism \(II\) are new; the rest is routing\. Because a prior enters as an input, one architecture serves every prior type with no per\-dataset architecture search, and each prior\-versus\-no\-prior comparison is a clean ablation\. The prior’s weights are still learned: a static or past\-known prior enters Stage\-1 pretraining, so a new one needs a new Stage\-1 run, while a future\-known prior enters only the predictor at Stage 2\. Our contributions are: \(1\) a taxonomy of twelve semantic\-prior categories mapped to these two mechanisms, of which four are implemented and evaluated here and the rest chart the design space; \(2\) the KITE architecture, built on a permutation\-invariant, cardinality\-free encoder so one model runs across systems with different variables; and \(3\) an empirical study organised by a single principle,*a prior helps when it supplies information the observed window does not already contain*: in a short context an informative prior beats no prior in\-domain \(Occupancy clock\+0\.104\+0\.104,p<0\.001p<0\.001; BATADAL conservation residual\+0\.741\+0\.741\) and transfers out\-of\-domain \(BATADAL→\\toSYNTH\-CON\+0\.32\+0\.32\) by substituting for feature transfer, and an account of where priors are inert \(redundant static injection, the graph bias, long context\)\.

## 2Related work

The dominant context for KITE is the wave of pretrained time\-series foundation models, from Chronos\-2\[[4](https://arxiv.org/html/2610.12738#bib.bib4)\], TimesFM\[[5](https://arxiv.org/html/2610.12738#bib.bib5)\], and Moirai, whose any\-variate attention accepts arbitrary variable counts\[[6](https://arxiv.org/html/2610.12738#bib.bib6)\], to the variate\-token designs PatchTST\[[7](https://arxiv.org/html/2610.12738#bib.bib7)\]and iTransformer\[[8](https://arxiv.org/html/2610.12738#bib.bib8)\], the growing push to pair series with structured and textual context\[[9](https://arxiv.org/html/2610.12738#bib.bib9)\], and industrial pretraining corpora that map heterogeneous machines onto a shared variable schema\[[10](https://arxiv.org/html/2610.12738#bib.bib10)\]: these transfer with little retraining but forecast future*values*and expose no switchable pathway for a\-priori domain knowledge\. Injecting a prior is itself not new, but each line handles one type through an always\-on pathway trained into the weights: covariate\-conditioned forecasters\[[11](https://arxiv.org/html/2610.12738#bib.bib11),[12](https://arxiv.org/html/2610.12738#bib.bib12),[13](https://arxiv.org/html/2610.12738#bib.bib13)\], graph\-conditioned models\[[14](https://arxiv.org/html/2610.12738#bib.bib14),[15](https://arxiv.org/html/2610.12738#bib.bib15),[16](https://arxiv.org/html/2610.12738#bib.bib16),[17](https://arxiv.org/html/2610.12738#bib.bib17)\], physics\-informed losses\[[18](https://arxiv.org/html/2610.12738#bib.bib18)\], architecture\-baked periodicity\[[19](https://arxiv.org/html/2610.12738#bib.bib19),[20](https://arxiv.org/html/2610.12738#bib.bib20)\], and text\-conditioned JEPA encoders that treat descriptions as always\-on identifiers\[[21](https://arxiv.org/html/2610.12738#bib.bib21)\]\. KITE builds on the JEPA family\[[2](https://arxiv.org/html/2610.12738#bib.bib2),[3](https://arxiv.org/html/2610.12738#bib.bib3)\]and unifies covariate, relational, conservation, and periodicity priors as switchable inputs to one backbone, each the identity at its null, so a prior’s contribution is separable from a pathway’s capacity\.

## 3Method

Figure 1:Two\-stage KITE architecture\.Stage 1 \(pretraining\):a funnel encoder with causal\-graph attention bias ingests the context; static/past\-known covariates enter the encoder while future\-known covariates enter the horizon\-conditioned predictor, and a shared\-weight target encoder \(not EMA\) supplies the latent\-prediction target\. The causal\-graph bias acts only in Stage\-1 pretraining \(the encoder is frozen at Stage 2\); its graph is*per\-dataset*, and in the reported runs its gate stays at the nullβ=0\\beta=0\(Section[3\.3](https://arxiv.org/html/2610.12738#S3.SS3), Appendix[G](https://arxiv.org/html/2610.12738#A7)\)\.Stage 2 \(finetuning\):a compact head over the frozen encoder consumes a past/future by variable\-type input grid and emits a monotone event\-probability surfacep⁡\(t,Δ​t\)p\(t,\\Delta t\)\.### 3\.1The semantic\-prior taxonomy

The organising idea is that domain knowledge can be sorted by*when it is knowable*and*whether it is per\-variable or relational*, which determines where the prior enters\. The twelve categories are listed in Appendix[D](https://arxiv.org/html/2610.12738#A4); we implement and evaluate four, periodicity \(\#5\), conservation \(\#10\), a structural descriptor \(\#1\), and causal\-graph attention bias \(\#9\)\. Static per\-variable priors were tested and found inert, motivating the routing view over static injection\.

### 3\.2Backbone: encoder, predictor, and event head

KITE’s contribution is the semantic\-prior conditioning; the backbone is a horizon\-conditioned event JEPA\[[3](https://arxiv.org/html/2610.12738#bib.bib3)\]\. A variate\-token encoderfθf\_\{\\theta\}maps a window to a summary𝐡t\\mathbf\{h\}\_\{t\}, patchified into\(C×N\)\(C\\times N\)tokens with*no per\-variable slot*\(RoPE\-ordered patches, variables an unordered set\), so it is permutation\-invariant and cardinality\-free \(Ctest≠CtrainC\_\{\\text\{test\}\}\\neq C\_\{\\text\{train\}\}; Appendix[F\.2](https://arxiv.org/html/2610.12738#A6.SS2)\)\. A horizon\-conditioned predictorgϕ​\(𝐡t,Δ​t\)g\_\{\\phi\}\(\\mathbf\{h\}\_\{t\},\\Delta t\)and a discrete\-survival head compose per\-horizon hazardsλk\\lambda\_\{k\}\[[22](https://arxiv.org/html/2610.12738#bib.bib22),[23](https://arxiv.org/html/2610.12738#bib.bib23)\]into a monotone event\-probability surfacep⁡\(t,Δ​t\)=1−∏j≤k\(1−λj\)p\(t,\\Delta t\)=1\-\\prod\_\{j\\leq k\}\(1\-\\lambda\_\{j\}\)\. Stage 1 pretrains encoder and predictor self\-supervised \(shared\-weight target,L1L\_\{1\}latent loss with a VICReg\[[24](https://arxiv.org/html/2610.12738#bib.bib24)\]collapse term\); Stage 2 freezes the encoder and trains the head on labels\. We evaluate by the rank\-based horizon\-AUROC \(h\-AUROC\)\.

### 3\.3Semantic\-prior mechanisms

KITE injects knowledge through two mechanisms, each the identity at its null, so at initialization the conditioned model computes exactly the unconditioned one \(Figure[1](https://arxiv.org/html/2610.12738#S3.F1); full per\-path detail, equations, and the graph\-bias schematic are in Appendix[A](https://arxiv.org/html/2610.12738#A1)\)\.Mechanism \(I\), covariates, routes per\-variable and derived priors as synthetic variables along three paths, by*when*the signal is knowable: a*static*descriptor to the encoder \(a zero\-initialized fusionout=x\+Wδ​δ\\mathrm\{out\}=x\+W\_\{\\delta\}\\delta, present in pretraining\), a*future\-known*covariate to the horizon\-conditioned predictor \(a periodicity clock evaluated att\+Δ​tt\{\+\}\\Delta t, so it anticipates rather than detects\), and a*past\-only derived*law to a covariate \(a conservation residual∑i​n−∑o​u​t−dL/dt\\sum\_\{in\}\-\\sum\_\{out\}\-dL/dt, feature engineering over an adaptable model\)\.Mechanism \(II\), attention bias, adds a scalar\-gated driver\-adjacencyβ​A\\beta Ato the encoder’s attention logits, identity atβ=0\\beta\{=\}0; it does not measurably help here \(bounded to≈0\.006\\approx 0\.006h\-AUROC, Appendix[G](https://arxiv.org/html/2610.12738#A7)\)\.

## 4Results and Discussion

We test this principle, that a prior helps when it supplies information the window lacks, in two regimes, in\-domain short context \(Claim A\) and out\-of\-domain transfer \(Claim B\), by h\-AUROC over55to1010seeds against zeroed\- and shuffled\-prior controls, so a gain cannot be an artefact of the extra input\. Identity at the null holds only at fixed weights, and training with a prior also adapts the rest of the network, so every reported null is a separately trained model with the prior zeroed or absent, never the prior model with its input removed at test time\.

Table 1:Semantic priors against external baselines \(h\-AUROC, mean±\\pmsd over55seeds unless noted; higher is better; “det\.” = training\-free deterministic detector\)\.No prioris the prior\-free control under the same recipe \(zeroed on the BATADAL rows, absent elsewhere; the two agree on Occupancy,0\.7670\.767vs\.0\.7620\.762\); the external forecasters have no semantic\-prior pathway\.The estimand differs by row and is comparable only within a row\(stratum\-restricted h\-AUROC on the onset/ongoing rows, overall on the0\.250\.25\-label row\), so*No prior*values below0\.50\.5are not errors: on the hard stratum the prior\-free backbone anti\-ranks the rare\-event windows\. In the transfer row,N/Amarks supervised baselines, which have no transfer mode; Chronos\-2 is scored zero\-shot on the same25072507windows \(225225onset\), window\-bootstrap95%95\\%CI\[0\.718,0\.822\]\[0\.718,0\.822\]\. The two KITE transfer cells share one Stage\-1 encoder; the prior arm scores0\.9220\.922whether it was pretrained on the target, the source, or not at all\.Dataset \(estimand\)Prior \(\#\)KITE \(prior\)No priorChronos\-2PatchTSTiTransformer*Short\-context, in\-domain \(Claim A\)*Occupancy \(silent\-onset\)Periodicity clock \(\#5\)0\.8660\.7620\.733±0\.0030\.733\{\\pm\}0\.0030\.702±0\.0370\.702\{\\pm\}0\.0370\.754±0\.0250\.754\{\\pm\}0\.025BATADAL \(ongoing\)Conservation resid\. \(\#10\)0\.8770\.1360\.9470\.947\(det\.\)0\.732±0\.0730\.732\{\\pm\}0\.0730\.672±0\.0550\.672\{\\pm\}0\.055BATADAL \(overall,0\.250\.25lab\.\)Conservation \(\#10\)0\.6440\.6440\.4370\.6420\.642\(det\.\)0\.656±0\.0240\.656\{\\pm\}0\.0240\.534±0\.0360\.534\{\\pm\}0\.036SYNTH\-ROUTE\-v2 \(onset\)Structural descr\. \(\#1, synth\.\)0\.8920\.8920\.6740\.909±0\.0070\.909\{\\pm\}0\.0070\.710±0\.0650\.710\{\\pm\}0\.0650\.916±0\.0110\.916\{\\pm\}0\.011*Out\-of\-domain transfer \(Claim B\)*BATADAL→\\toSYNTH\-CON \(onset\)Conservation resid\. \(\#10, OOD\)0\.9220\.6010\.7740\.774N/AN/A

### 4\.1Claim A: an informative prior beats no prior in short context

Table[1](https://arxiv.org/html/2610.12738#S4.T1)carries Claim A \(bars: Figure[3](https://arxiv.org/html/2610.12738#A3.F3)\)\. On the flagship Occupancy silent\-onset stratum under a shortw=360w=360context, the true\-phase periodicity clock lifts the prior\-free baseline from0\.7620\.762to0\.8660\.866; with the clock on both paths, a zeroed clock stays at baseline \(0\.7670\.767\) and a shuffled one falls0\.3460\.346below the true phase, so the gain is the phase itself\. Where the clock enters matters: alone, as a time\-of\-day sine, it scores0\.7210\.721; as ordinary encoder input channels, the way a generic covariate\-capable forecaster takes it,0\.7970\.797; only at the predictor’s horizon0\.8660\.866\(Appendix[I](https://arxiv.org/html/2610.12738#A9)\)\. On BATADAL atw=512w=512the prior\-free null sits below chance, and the conservation residual lifts it to0\.8770\.877\(p=1\.2×10−5p=1\.2\\times 10^\{\-5\},n=5n=5\), still helping at a0\.250\.25label fraction \(\+0\.207\+0\.207\)\. A synthetic structural descriptor \(\#1\) confirms the static path works when a genuine structural prior exists \(p=0\.030p=0\.030\)\.

Against the external baselines of Table[1](https://arxiv.org/html/2610.12738#S4.T1), none of2020arms beats KITE at BH\-FDR0\.050\.05; they sit at or below No prior on Occupancy and well above it elsewhere, some with point estimates above KITE’s\. Our claim is that the prior lifts a backbone above its own null at no extra training cost, not that KITE beats foundation models\.

#### Claim B: transfer by substitution, not amplification\.

A domain\-invariant prior \(a calendar clock, a mass\-balance residual\) survives a domain shift that scrambles learned features: across ten cells it lifts the forecasting stratum by\+0\.12\+0\.12to\+0\.39\+0\.39where it applies, while a static per\-variable prior does not \(\+0\.006\+0\.006\)\. The sharper prediction that it should help*more*out of domain than in does not survive \(OOD−\-ID=−0\.002=\-0\.002\); instead the data show*substitution*\(Appendix[C](https://arxiv.org/html/2610.12738#A3)\): a prior with no pretraining beats transfer with no prior in eight of ten cells, and zero\-shot Chronos\-2 \(0\.7740\.774\) sits between KITE with the prior \(0\.9220\.922\) and without it \(0\.6010\.601\)\.

### 4\.2Where priors do not help

The boundaries are as informative as the wins \(Appendix[B](https://arxiv.org/html/2610.12738#A2)\): a prior is inert when the window already carries the information \(static injection0/190/19under BH\-FDR; the clock null once the context spans a full period; the graph bias bounded to≈0\.006\\approx 0\.006\), when it is misaligned \(a wrong period reverses the Occupancy win,−0\.022\-0\.022\), and when a present\-state signal is asked to anticipate \(the residual detects but does not forecast onset, and on the ongoing stratum a zero\-parameter threshold on it alone,0\.9210\.921, outranks KITE: that win is the residual’s; Appendix[I](https://arxiv.org/html/2610.12738#A9)\); the pre\-registered Weather control passes as a falsification check \(Δ=−0\.017\\Delta=\-0\.017,p=0\.69p=0\.69\)\. The evidence is scoped to short context and two priors, several taxonomy categories remain conceptual, mechanism \(II\) is bounded but undetermined, and the strongest transfer target is synthetic; Appendix[H](https://arxiv.org/html/2610.12738#A8)verifies the shipped configuration\.

## 5Conclusion

KITE injects domain knowledge into a self\-supervised event forecaster as a switchable input\. A prior helps when it supplies information the observed window lacks: in short context it beats no prior, and out of domain it substitutes for feature transfer, while redundant static injection and the graph bias do not\. Never entangling the backbone, a switchable prior is a natural substrate for pretrained, cross\-domain time\-series foundation*world models*that carry their governing structure explicitly\.

## References

- \[1\]Laura von Rueden, Sebastian Mayer, Katharina Beckh, Bogdan Georgiev, Sven Giesselbach, Raoul Heese, Birgit Kirsch, Julius Pfrommer, Annika Pick, Rajkumar Ramamurthy, Michal Walczak, Jochen Garcke, Christian Bauckhage, and Jannis Schuecker\.Informed machine learning – a taxonomy and survey of integrating prior knowledge into learning systems\.*IEEE Transactions on Knowledge and Data Engineering*, 35\(1\):614–633, 2023\.doi:10\.1109/TKDE\.2021\.3079836\.
- \[2\]Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas\.Self\-supervised learning from images with a joint\-embedding predictive architecture\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 15619–15629, 2023\.
- \[3\]Jonas Petersen, Gian\-Alessandro Lombardi, Riccardo Maggioni, Camilla Mazzoleni, Federico Martelli, and Philipp Petersen\.HEPA: A self\-supervised horizon\-conditioned event predictive architecture for time series\.arXiv preprint arXiv:2605\.11130, 2026\.Spotlight at the Foundation Models for Structured Data \(FMSD\) Workshop, ICML 2026\.
- \[4\]Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C\. Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erickson, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, and Michael Bohlke\-Schneider\.Chronos\-2: From univariate to universal forecasting\.arXiv preprint arXiv:2510\.15821, 2025\.
- \[5\]Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou\.A decoder\-only foundation model for time\-series forecasting\.In*Proceedings of the 41st International Conference on Machine Learning \(ICML\)*, volume 235 of*Proceedings of Machine Learning Research*, pages 10148–10167\. PMLR, 2024\.
- \[6\]Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo\.Unified training of universal time series forecasting transformers\.In*Proceedings of the 41st International Conference on Machine Learning \(ICML\)*, volume 235 of*Proceedings of Machine Learning Research*, pages 53140–53164\. PMLR, 2024\.
- \[7\]Yuqi Nie, Nam H\. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam\.A time series is worth 64 words: Long\-term forecasting with Transformers\.In*International Conference on Learning Representations \(ICLR\)*, 2023\.
- \[8\]Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long\.iTransformer: Inverted transformers are effective for time series forecasting\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- \[9\]Haoxin Liu, Yichen Zhou, Rajat Sen, B\. Aditya Prakash, and Abhimanyu Das\.Rethinking multimodal time\-series forecasting evaluation\.In*Proceedings of the 43rd International Conference on Machine Learning \(ICML\)*, volume 306 of*Proceedings of Machine Learning Research*, pages 80183–80283\. PMLR, 2026\.
- \[10\]Karim Othman, Jonas Petersen, Matei Ignuta\-Ciuncanu, Camilla Mazzoleni, Federico Martelli, Alessandro Lombardi, Riccardo Maggioni, and Philipp Petersen\.FactoryNet: A large\-scale dataset toward industrial time\-series foundation models\.arXiv preprint arXiv:2605\.09081, 2026\.Accepted at the AI4Physics and FMSD Workshops, ICML 2026\.
- \[11\]Bryan Lim, Sercan Ö\. Arık, Nicolas Loeff, and Tomas Pfister\.Temporal fusion transformers for interpretable multi\-horizon time series forecasting\.*International Journal of Forecasting*, 37\(4\):1748–1764, 2021\.doi:10\.1016/j\.ijforecast\.2021\.03\.012\.
- \[12\]Lu Han, Yu Liu, Lan Li, Qiwen Deng, Jian Jiang, Yinbo Sun, Zhe Yu, Binfeng Wang, Xingyu Lu, Lintao Ma, Han\-Jia Ye, and De\-Chuan Zhan\.UniCA: Unified covariate adaptation for time series foundation model\.In*International Conference on Learning Representations \(ICLR\)*, 2026\.
- \[13\]Yosuke Yamaguchi, Issei Suemitsu, Yuki Kajihara, and Wenpeng Wei\.CITRAS\-FM: Tiny time series foundation model for covariate\-informed zero\-shot forecasting\.arXiv preprint arXiv:2606\.10798, 2026\.Accepted at the European Signal Processing Conference \(EUSIPCO\) 2026\.
- \[14\]Hang Zhao, Yujing Wang, Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang\.Multivariate time\-series anomaly detection via graph attention network\.In*IEEE International Conference on Data Mining \(ICDM\)*, pages 841–850, 2020\.doi:10\.1109/ICDM50108\.2020\.00093\.
- \[15\]Ailin Deng and Bryan Hooi\.Graph neural network\-based anomaly detection in multivariate time series\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 35, pages 4027–4035, 2021\.doi:10\.1609/aaai\.v35i5\.16523\.
- \[16\]Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu\.Diffusion convolutional recurrent neural network: Data\-driven traffic forecasting\.In*International Conference on Learning Representations \(ICLR\)*, 2018\.
- \[17\]Bing Yu, Haoteng Yin, and Zhanxing Zhu\.Spatio\-temporal graph convolutional networks: A deep learning framework for traffic forecasting\.In*Proceedings of the Twenty\-Seventh International Joint Conference on Artificial Intelligence \(IJCAI\)*, pages 3634–3640, 2018\.doi:10\.24963/ijcai\.2018/505\.
- \[18\]Maziar Raissi, Paris Perdikaris, and George E\. Karniadakis\.Physics\-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations\.*Journal of Computational Physics*, 378:686–707, 2019\.doi:10\.1016/j\.jcp\.2018\.10\.045\.
- \[19\]Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long\.Autoformer: Decomposition transformers with Auto\-Correlation for long\-term series forecasting\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 34, pages 22419–22430, 2021\.
- \[20\]Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long\.TimesNet: Temporal 2D\-variation modeling for general time series analysis\.In*International Conference on Learning Representations \(ICLR\)*, 2023\.
- \[21\]Utsav Dutta, Gerardo Pastrana, Sina Khoshfetrat Pakazad, and Henrik Ohlsson\.Giving sensors a voice: Multimodal JEPA for semantic time\-series embeddings\.In*Proceedings of the 43rd International Conference on Machine Learning \(ICML\)*, volume 306 of*Proceedings of Machine Learning Research*, pages 27330–27378\. PMLR, 2026\.
- \[22\]Michael F\. Gensheimer and Balasubramanian Narasimhan\.A scalable discrete\-time survival model for neural networks\.*PeerJ*, 7:e6257, 2019\.doi:10\.7717/peerj\.6257\.
- \[23\]Håvard Kvamme and Ørnulf Borgan\.Continuous and discrete\-time survival prediction with neural networks\.*Lifetime Data Analysis*, 27\(4\):710–736, 2021\.doi:10\.1007/s10985\-021\-09532\-6\.
- \[24\]Adrien Bardes, Jean Ponce, and Yann LeCun\.VICReg: Variance\-invariance\-covariance regularization for self\-supervised learning\.In*International Conference on Learning Representations \(ICLR\)*, 2022\.
- \[25\]J\. J\. Downs and E\. F\. Vogel\.A plant\-wide industrial process control problem\.*Computers & Chemical Engineering*, 17\(3\):245–255, 1993\.doi:10\.1016/0098\-1354\(93\)80018\-I\.
- \[26\]Cory A\. Rieth, Ben D\. Amsel, Randy Tran, and Maia B\. Cook\.Additional Tennessee Eastman process simulation data for anomaly detection evaluation\.Harvard Dataverse, 2017\.URL[https://doi\.org/10\.7910/DVN/6C3JR1](https://doi.org/10.7910/DVN/6C3JR1)\.
- \[27\]Randall Balestriero and Yann LeCun\.LeJEPA: Provable and scalable self\-supervised learning without the heuristics\.arXiv preprint arXiv:2511\.08544, 2025\.
- \[28\]Luis M\. Candanedo and Véronique Feldheim\.Accurate occupancy detection of an office room from light, temperature, humidity and CO2measurements using statistical learning models\.*Energy and Buildings*, 112:28–39, 2016\.doi:10\.1016/j\.enbuild\.2015\.11\.071\.
- \[29\]Luis Candanedo\.Occupancy detection\.UCI Machine Learning Repository, 2016\.URL[https://doi\.org/10\.24432/C5X01N](https://doi.org/10.24432/C5X01N)\.
- \[30\]Riccardo Taormina, Stefano Galelli, Nils Ole Tippenhauer, Elad Salomons, Avi Ostfeld, Demetrios G\. Eliades, et al\.Battle of the attack detection algorithms: Disclosing cyber attacks on water distribution networks\.*Journal of Water Resources Planning and Management*, 144\(8\):04018048, 2018\.doi:10\.1061/\(ASCE\)WR\.1943\-5452\.0000969\.
- \[31\]Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang\.Informer: Beyond efficient transformer for long sequence time\-series forecasting\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 35, pages 11106–11115, 2021\.doi:10\.1609/aaai\.v35i12\.17325\.
- \[32\]Shuyi Zhang, Bin Guo, Anlan Dong, Jing He, Ziping Xu, and Song Xi Chen\.Cautionary tales on air\-quality improvement in Beijing\.*Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences*, 473\(2205\):20170457, 2017\.doi:10\.1098/rspa\.2017\.0457\.
- \[33\]Song Chen\.Beijing multi\-site air quality\.UCI Machine Learning Repository, 2017\.URL[https://doi\.org/10\.24432/C5RK5G](https://doi.org/10.24432/C5RK5G)\.
- \[34\]Iurii D\. Katser and Vyacheslav O\. Kozitsin\.Skoltech Anomaly Benchmark \(SKAB\)\.Kaggle, 2020\.URL[https://doi\.org/10\.34740/KAGGLE/DSV/1693952](https://doi.org/10.34740/KAGGLE/DSV/1693952)\.
- \[35\]Steffen Moritz, Frederik Rehbach, Sowmya Chandrasekaran, Margarita Rebolledo, and Thomas Bartz\-Beielstein\.GECCO industrial challenge 2018 dataset: A water quality dataset for the ‘Internet of Things: Online anomaly detection for drinking water quality’ competition at the Genetic and Evolutionary Computation Conference 2018, Kyoto, Japan\.Zenodo, 2018\.URL[https://doi\.org/10\.5281/zenodo\.3884398](https://doi.org/10.5281/zenodo.3884398)\.
- \[36\]Abhinav Saxena, Kai Goebel, Don Simon, and Neil Eklund\.Damage propagation modeling for aircraft engine run\-to\-failure simulation\.In*2008 International Conference on Prognostics and Health Management*, pages 1–9\. IEEE, 2008\.doi:10\.1109/PHM\.2008\.4711414\.

## Appendix AMechanism details

Each mechanism is the identity at its null: with the prior input zeroed the pathway adds nothing, so at initialization the conditioned model is exactly the unconditioned one\. After training the remaining weights have adapted to the prior, so the no\-prior comparisons of Section[4](https://arxiv.org/html/2610.12738#S4)use separately trained arms rather than removing the prior at test time\.

#### Static path \(encoder\)\.

A static per\-variable descriptor \(categories \#1–\#4, \#6–\#8, \#12\) is fused through a pathway that is identity at initialization,

out=x\+Wδ​δ,Wδ​initialized to​0,\\mathrm\{out\}=x\+W\_\{\\delta\}\\,\\delta,\\quad W\_\{\\delta\}\\text\{ initialized to \}0,\(1\)present during Stage\-1 pretraining so the frozen encoder learns to use it\. Identity comes from the*weights*, not a scalar gate \(atanh⁡\(g\)\\tanh\(g\)gate opens slowly and costs much of every flagship effect, Appendix[H](https://arxiv.org/html/2610.12738#A8)\)\. Generic descriptors are inert where the window already carries what they encode; the gain appears on a synthetic benchmark built to*hide*identity\.

#### Future\-known path \(predictor\)\.

A future\-known prior enters asc⁡\(t\+Δ​t\)∈ℝmc\(t\+\\Delta t\)\\in\\mathbb\{R\}^\{m\}concatenated into the predictor input\[𝐡t;Δ​t;c⁡\(t\+Δ​t\)\]\[\\mathbf\{h\}\_\{t\};\\Delta t;c\(t\+\\Delta t\)\]and evaluated at the horizon, so it anticipates rather than detects\. The tested instance is a periodicity clock,

c⁡\(t\+Δ​t\)=\[sin,cos\]​\(2​π​\(t\+Δ​t\)modPP\),c\(t\+\\Delta t\)=\[\\sin,\\cos\]\\\!\\left\(2\\pi\\,\\frac\{\(t\+\\Delta t\)\\bmod P\}\{P\}\\right\),\(2\)identity at its nullc⁡\(⋅\)≡0c\(\\cdot\)\\equiv 0\.

#### Past\-only derived path \(covariate\)\.

A governing law whose instantaneous residual is computable from the observed past, e\.g\. the conservation residual∑i​n−∑o​u​t−dL/dt\\sum\_\{in\}\-\\sum\_\{out\}\-dL/dt, enters as an ordinary past\-known covariate; it detects a present\-state violation but does not anticipate onset\.

#### Attention bias\.

A causal graph fixes a driver\-adjacencyA∈\{0,1\}C×CA\\in\\\{0,1\\\}^\{C\\times C\}\(A⁡\[a,b\]=1A\[a,b\]\{=\}1iffbbdrivesaa, zero diagonal\), added to the attention logits,

logit⁡\(qa,kb\)\+=β⋅A⁡\[a,b\],\\mathrm\{logit\}\(q\_\{a\},k\_\{b\}\)\\mathrel\{\+\}=\\beta\\cdot A\[a,b\],\(3\)identity atβ=0\\beta=0\(Figure[2](https://arxiv.org/html/2610.12738#A1.F2), Appendix[F\.3](https://arxiv.org/html/2610.12738#A6.SS3)\)\. One design point remains open: the predictor’s future\-known input is not yet cardinality\-compatible across datasets with differing covariate counts \(Appendix[E](https://arxiv.org/html/2610.12738#A5)\)\.

Figure 2:Causal\-graph attention bias\. A causal graph over variables \(v2,v3v\_\{2\},v\_\{3\}driven byv1v\_\{1\}; a\) gives the driver\-adjacencyAA\(b\)\. Since a token is a \(variable, time\-patch\) pair,AAis tiled over all causally\-allowed token pairs and added to the query–key logit asβ​A\\beta A\(c; future keys masked\)\. Atβ=0\\beta\{=\}0the bias is the identity\.

## Appendix BDataset–prior alignment

Table[2](https://arxiv.org/html/2610.12738#A2.T2)gives the full mapping between datasets and the semantic\-prior categories they carry, referenced from Section[4\.2](https://arxiv.org/html/2610.12738#S4.SS2)\.

Table 2:Dataset–prior alignment\. Category numbers refer to Table[3](https://arxiv.org/html/2610.12738#A4.T3)\.#### Where priors do not help \(full evidence\)\.

The four causes summarised in Section[4\.2](https://arxiv.org/html/2610.12738#S4.SS2)rest on the following runs\.*Information the window already carries*: static per\-variable injections are inert across the static\-injection ablations \(0/190/19survive BH\-FDR; the real prior is indistinguishable from a shuffled prior even when isolated\), consistent with a FiLM\-style identity prior being provably absorbable into the following linear map; the periodicity clock is null once the context spans a full period \(a full\-context ETTm1 sweep:0/30/3cells survive FDR\); and the causal\-graph structural prior is null once the bias is switched on at all \(375375runs over five datasets with no positive cell in1919; the earlier graph\-correctness ablations are void, their arms having been byte\-identical\)\.*A prior misaligned to the events*: a wrong FFT\-estimated period reverses the Occupancy win to−0\.022\-0\.022\(its phase precesses across the split\), and the pre\-registered Weather negative control is null \(−0\.017\-0\.017,p=0\.69p=0\.69\)\.*Present state rather than anticipation*: the conservation residual detects an ongoing violation \(\+0\.741\+0\.741on the BATADAL ongoing stratum\) but does not forecast onset before it begins \(the onset\-stratum runs: BATADAL onset−0\.053\-0\.053; GECCO onset−0\.136\-0\.136\)\.*Capacity rather than content*: the static\-identity FiLM prior is absorbable by the next linear layer, and in the cross\-domain transfer experiment gives only\+0\.006\+0\.006\(p=0\.56p=0\.56, real≈\\approxshuffled\)\.

## Appendix CCross\-domain transfer: full results

Figure 3:An informative prior beats no prior\(h\-AUROC; chance0\.50\.5, rank\-based; each bar pairs the prior against the prior\-free control of Table[1](https://arxiv.org/html/2610.12738#S4.T1)\)\.\(a\)In\-domain, short context: each prior clears its null on the stratum where it applies \(clock on silent\-onset*forecasting*; conservation residual on ongoing\-violation*detection*and at a0\.250\.25label fraction; a structural descriptor on synthetic SYNTH\-ROUTE\-v2\)\.\(b\)Out\-of\-domain transfer, frozen encoder \(BATADAL→\\toSYNTH\-CON\): a working prior transfers and*substitutes*for feature transfer, while a static\-identity prior does not\.This appendix expands the Claim\-B result summarised in Section[4](https://arxiv.org/html/2610.12738#S4)\. Every run holds the target fixed and varies only where the encoder was pretrained \(id, the target itself;ood, a source pool;scratch, no pretraining\), with the prior delivered to the predictor at Stage 2; because the prior never touches the encoder, the arms in a cell are paired exactly off one cached Stage\-1 checkpoint\. Two pre\-registered gates decide whether a cell is interpretable: G1 \(mechanism\-arrival\) asks whether the prior helps in\-domain at all, and G2 \(transfer\-substrate\) asks whether source pretraining beats scratch\. Both gates pass in exactly one cell, BATADAL→\\toSYNTH\-CON under a frozen encoder, where the out\-of\-domain lift equals the in\-domain lift \(OOD−\-ID=−0\.002=\-0\.002\[−0\.024\-0\.024,\+0\.020\+0\.020\],p=0\.82p=0\.82\): a clean null for the amplification prediction\.

The substitution reading is that a prior with no pretraining beats transfer with no prior in eight of ten cells, by\+0\.027\+0\.027to\+0\.381\+0\.381, and is null exactly where the prior is known not to work \(Weather,\+0\.003\+0\.003,p=0\.72p=0\.72\)\. The competence\-gradient signal appears under the supervised source recipe on BATADAL→\\toSYNTH\-CON, where the prior’s lift rises as encoder competence falls \(in\-domain\+0\.135\+0\.135, out\-of\-domain\+0\.195\+0\.195, random encoder\+0\.343\+0\.343;scratch−\-id=\+0\.208=\+0\.208\[\+0\.040\+0\.040,\+0\.377\+0\.377\],p=0\.027p=0\.027, with G2 passing atp=0\.05p=0\.05\), but that cell fails G1 by a ceiling effect \(the in\-domain model already scores0\.7880\.788without the prior\), so the pre\-registration marks it non\-interpretable and we report it as suggestive only\. The largest external\-validity gap is that the primary conservation target is synthetic, because a second real mass\-balance network \(WADI, SWaT\) was not licensable in the available time; the BATADAL source is real\.

## Appendix DExtended semantic\-prior taxonomy

Table[3](https://arxiv.org/html/2610.12738#A4.T3)lists the twelve categories with an added implementation\-status column\.

Table 3:Extended taxonomy with the finer per\-category mechanism and implementation status\.E: implemented and evaluated here;I: mechanism implemented;N: tested and found inert;C: conceptual scaffold\. Static per\-variable priors \(\#1–\#4, \#6–\#8, \#12 via static injection\) were found inert in the static\-injection ablations; \#1 is evaluated via a structural per\-variable descriptor on synthetic data\.
## Appendix ECovariate mechanism illustration

Figure[4](https://arxiv.org/html/2610.12738#A5.F4)makes the routing decision of Section[3\.3](https://arxiv.org/html/2610.12738#S3.SS3)concrete on two datasets\. ETTm1’s channel identity \(a static, per\-channel fact: which channel is this transformer’s high\-voltage active load\) and Weather’s units and smoothness descriptors \(both past\-known per\-channel facts\) travel the static path of mechanism \(I\)\. They reach the encoder through the zero\-initialized identity\-at\-null connection of Equation[1](https://arxiv.org/html/2610.12738#A1.E1), so at initialization they contribute nothing and the encoder reduces exactly to the unconditioned baseline\. The Occupancy periodicity clock \(P=1440P=1440, Equation[2](https://arxiv.org/html/2610.12738#A1.E2)\) takes the future\-known path of mechanism \(I\) instead\. It skips the encoder entirely, is evaluated at the forecast horizonΔ​t\\Delta t, and is concatenated straight into the predictor’s input, telling the model the future phase of the daily cycle rather than leaving it to infer that phase from a context window that may not span a full period\.

What separates the two routes is what the prior actually conditions\. A static or past\-known descriptor shapes the encoder’s representation of the observed window, while a future\-known covariate shapes the predictor’s forecast at a given horizon\. Keeping them on separate pathways, rather than folding every covariate into one stream, is what lets Section[4\.2](https://arxiv.org/html/2610.12738#S4.SS2)attribute a measured gain to the mechanism that carried it, static injection or future\-known phase, instead of to a generic increase in input information\.

Figure 4:Covariate mechanism\. Static and past\-known priors \(ETTm1 identity; Weather units and smoothness\) enter the encoder through the zero\-initialized identity\-at\-null pathway; the future\-known periodicity clock \(P=1440P=1440\) enters the predictor at the forecast horizon\.
## Appendix FProofs

### F\.1KITE tokenization

Figure[5](https://arxiv.org/html/2610.12738#A6.F5)shows the tokenization procedure discussed in Section[3\.2](https://arxiv.org/html/2610.12738#S3.SS2)\.

Figure 5:KITE tokenization\. AT×CT\\times Cwindow is cut intoN=T/PN=T/Ptemporal patches; a sharedLinear⁡\(P→d\)\\mathrm\{Linear\}\(P\\\!\\to\\\!d\)maps each \(variable, patch\) to a token, yielding a\(C×N\)\(C\\times N\)grid fed to the encoderfθf\_\{\\theta\}\. Temporal patches are ordered \(RoPE\); variables form an unordered set with no per\-variable parameter, making KITE permutation\-invariant and cardinality\-free \(Ctest≠CtrainC\_\{\\text\{test\}\}\\neq C\_\{\\text\{train\}\}\)\.
### F\.2Permutation invariance and cardinality freedom

We establish the two properties of the KITE backbone \(Figure[1](https://arxiv.org/html/2610.12738#S3.F1)\): the encoder output is unchanged when the variables are reordered, and the encoder works for any variable countCC\. Letπ\\pibe any reordering of variables andx≤tπx\_\{\\leq t\}^\{\\pi\}the input with variables reordered byπ\\pi\.

#### Embedding\.

The window is cut into patches and every \(variable, patch\) is embedded by the same mapLinear⁡\(P→d\)\\mathrm\{Linear\}\(P\\to d\):zc,i=W​xi\(c\)z\_\{c,i\}=Wx\_\{i\}^\{\(c\)\}, whereW∈ℝd×PW\\in\\mathbb\{R\}^\{d\\times P\}is the shared embedding matrix, identical for all\(c,i\)\(c,i\)\. BecauseWWhas no variable index, reordering the variables leaves the token set unchanged and only changes its order along the token axis,Z⁡\(x≤tπ\)=π⋅Z⁡\(x≤t\)Z\(x\_\{\\leq t\}^\{\\pi\}\)=\\pi\\cdot Z\(x\_\{\\leq t\}\)\.

#### Attention\.

The encoder forms queries, keys, and values from tokens alone, and its causal mask restricts attention by temporal patch index, never by variable\. Attention therefore depends only on token content, so reordering the input tokens produces the identically reordered output:

Attn⁡\(π⋅Z\)=π⋅Attn⁡\(Z\)\.\\mathrm\{Attn\}\(\\pi\\cdot Z\)=\\pi\\cdot\\mathrm\{Attn\}\(Z\)\.\(4\)

#### Pooling\.

The final representation𝐡t\\mathbf\{h\}\_\{t\}is a symmetric aggregation over the token axis, and a symmetric function ignores the order of a set:

agg⁡\(π⋅V\)=agg⁡\(V\)\.\\mathrm\{agg\}\(\\pi\\cdot V\)=\\mathrm\{agg\}\(V\)\.\(5\)

#### Composition\.

Chaining the three stages,

𝐡t​\(x≤tπ\)=agg⁡\(Attn⁡\(π⋅Z\)\)=agg⁡\(π⋅Attn⁡\(Z\)\)=agg⁡\(Attn⁡\(Z\)\)=𝐡t​\(x≤t\),\\mathbf\{h\}\_\{t\}\(x\_\{\\leq t\}^\{\\pi\}\)=\\mathrm\{agg\}\\big\(\\mathrm\{Attn\}\(\\pi\\cdot Z\)\\big\)=\\mathrm\{agg\}\\big\(\\pi\\cdot\\mathrm\{Attn\}\(Z\)\\big\)=\\mathrm\{agg\}\\big\(\\mathrm\{Attn\}\(Z\)\\big\)=\\mathbf\{h\}\_\{t\}\(x\_\{\\leq t\}\),\(6\)so the reordering that enters at embedding disappears at pooling and𝐡t\\mathbf\{h\}\_\{t\}is independent of variable order\. Every parameter is applied per token and none has a dimension equal toCC: changingCCchanges the number of token rows, not the shape of any parameter, so the trainedθ\\thetaruns atCtest≠CtrainC\_\{\\text\{test\}\}\\neq C\_\{\\text\{train\}\}with nothing modified\.

### F\.3Identity atβ=0\\beta=0

The mechanism \(Figure[2](https://arxiv.org/html/2610.12738#A1.F2)\) adds the scaled driver\-adjacency to the attention logits,logit⁡\(qa,kb\)\+=β​A​\[a,b\]\\mathrm\{logit\}\(q\_\{a\},k\_\{b\}\)\\mathrel\{\+\}=\\beta\\,A\[a,b\], whereA∈\{0,1\}C×CA\\in\\\{0,1\\\}^\{C\\times C\}is indexed by \(query variableaa, key variablebb\) andA⁡\[a,b\]=1A\[a,b\]=1iffbbis a causal driver ofaa\(a direct graph parent within its control loop\) and00otherwise, including the entire diagonal and every driver\-less row\. Atβ=0\\beta=0the added term is0⋅A⁡\[a,b\]=00\\cdot A\[a,b\]=0for every pair, so the attention logits, and with them the whole forward pass, are identical to the unconditioned encoder at the same weights\. As for the covariate paths, a trained comparison still needs a separately trained arm\. The adjacency introduces no learned weights of its own, so the entire prior enters through the scalarβ\\beta\.

## Appendix GGraph mechanism: correctness, correction, and the bounded null

The graph attention\-bias mechanism assumes by default that the input causal graph is correct; here we characterise how an inaccurate graph behaves\. The mechanism’s only contact with the graph is the driver\-adjacency matrixA∈\{0,1\}C×CA\\in\\\{0,1\\\}^\{C\\times C\}, so a graph change that does not changeAAdoes not change the model, and graph correctness is therefore defined onAA, equivalently on the set of \(target, driver\) pairs it flags\. LetA∗A^\{\*\}be the driver\-adjacency of the ground\-truth graph andA^\\hat\{A\}that of the graph under test, with flagged pair setsE∗=\{\(a,b\):A∗​\[a,b\]=1\}E^\{\*\}=\\\{\(a,b\):A^\{\*\}\[a,b\]=1\\\}andE^=\{\(a,b\):A^​\[a,b\]=1\}\\hat\{E\}=\\\{\(a,b\):\\hat\{A\}\[a,b\]=1\\\}\. Graph correctness is the overlap

γ=\|E∗∩E^\|\|E∗∪E^\|∈\[0,1\],\\gamma=\\frac\{\|E^\{\*\}\\cap\\hat\{E\}\|\}\{\|E^\{\*\}\\cup\\hat\{E\}\|\}\\in\[0,1\],\(7\)whereγ=1\\gamma=1flags exactly the right driver pairs andγ=0\\gamma=0flags none correctly\. The Jaccard form ignores the manyC2C^\{2\}entries both graphs correctly leave unflagged, which would otherwise make almost any graph look correct, and scores only the pairs the bias acts on\.

#### The corrected null and its bound\.

An early ablation compared graphs at the releasedβ=0\\beta=0, where the additive bias vanishes identically, so correct, shuffled, random, and dense adjacencies produce byte\-identical activations; those arms are therefore void\. Fixingβ=2\\beta=2switches the mechanism on for the first time, and it demonstrably acts: the share of cross\-variable attention mass landing on graph edges rises from a3\.8%3\.8\\%chance level to19\.1%19\.1\\%on Weather and from1\.6%1\.6\\%to8\.7%8\.7\\%on the Tennessee Eastman process \(TEP\)\[[25](https://arxiv.org/html/2610.12738#bib.bib25),[26](https://arxiv.org/html/2610.12738#bib.bib26)\]\. Even so it does not help\. Across seven adjacency variants on Weather every paired difference in h\-AUROC falls within±0\.0013\\pm 0\.0013, against a seed standard deviation of0\.00380\.0038, and closing every cross\-variable path inside the encoder costs only−0\.0011\-0\.0011h\-AUROC \(95%95\\%CI\[−0\.0063,\+0\.0040\]\[\-0\.0063,\+0\.0040\],n=5n=5\), bounding what any variable\-relational prior could carry here at about0\.0060\.006\. This is a bound, not a refutation, because the positive control is under\-powered: on a synthetic benchmark whose event shifts two variables’ joint distribution while both marginals stay fixed, a linear probe handed all4545pairwise correlations reaches only0\.5590\.559against a variable\-blind floor of0\.4540\.454, with KITE at0\.4780\.478–0\.5190\.519inside that corridor\. We therefore report mechanism \(II\) as undetermined with a small upper bound rather than refuted\.

## Appendix HImplementation alignment

The results in Section[4](https://arxiv.org/html/2610.12738#S4)were produced by an implementation that differed from the design described in Section[3\.2](https://arxiv.org/html/2610.12738#S3.SS2)and Section[3\.3](https://arxiv.org/html/2610.12738#S3.SS3)in four places\. A companion study re\-ran both flagships under the idealised form of each, separately and together, on the same data, splits, seeds and metric, changing one thing at a time\. Every arm below is paired by seed against its own identity\-at\-null control \(the covariate present but zeroed\),1010seeds on Occupancy and55on BATADAL, with Benjamini–Hochberg control within each flagship\.

#### Two changes have no detectable cost\.

Removing the per\-variable slot and supplying the future\-known covariate during Stage\-1 pretraining each leave every flagship without a detectable change \(allp≥0\.08p\\geq 0\.08, Table[4](https://arxiv.org/html/2610.12738#A8.T4)\); at these sample sizes this is a failure to reject, not demonstrated equivalence\. This paper describes the slot\-free encoder; the Stage\-1 covariate variant is not adopted, for the reason given below\.

#### One costs accuracy, and one gives no reason to switch\.

A scalar\-gated form of the fusion in Equation \([1](https://arxiv.org/html/2610.12738#A1.E1)\),out=x\+tanh⁡\(g\)​δ\\mathrm\{out\}=x\+\\tanh\(g\)\\,\\delta, costs accuracy on every flagship \(Table[4](https://arxiv.org/html/2610.12738#A8.T4)\), so this paper describes the zero\-initialized fusion\. Substituting SIGReg for VICReg is neither better nor detectably worse on any flagship \(below\), so the VICReg collapse term is kept\.

#### A distinction the study had to make\.

Equation \([1](https://arxiv.org/html/2610.12738#A1.E1)\)’s “present during pretraining” refers to the*static descriptor entering the encoder*, and that is what the implementation does: the descriptor is concatenated to the Stage\-1 input alongside the observed variables, so the encoder is trained with it and is frozen afterwards with it\. The*future\-known*covariate of the predictor path is a separate mechanism and is deliberately not present in Stage 1: the predictor is retrained at Stage 2 anyway, and keeping the covariate out of pretraining keeps the encoder’s Stage\-1 data identical in the prior and no\-prior arms\. The row below measures the variant that supplies it during pretraining and finds no detectable cost, so nothing is lost by leaving it out\.

Table 4:Prior\-versus\-nullΔ\\Deltah\-AUROC under each idealised change, against the identity\-at\-null \(zeroed\-covariate\) control\. The*shipped*row is this study’s paired re\-run of the configuration behind Table[1](https://arxiv.org/html/2610.12738#S4.T1), reproducing each flagship effect up to seed noise\.Costis the paired per\-seed difference in the prior arm’s h\-AUROC from that shipped configuration \(so it need not equal the change inΔ\\Delta, whose null arm can also move\)\.
#### The gated fusion is the expensive change\.

Implemented faithfully,ggends training at a magnitude of0\.0290\.029on Occupancy and0\.0070\.007on BATADAL: the pathway is gated rather than dead \(ggis never exactly zero\) but it admits only a few percent of the descriptor\. The mechanism is visible across and within flagships: the flagship whose gate opens less loses proportionally more \(77%77\\%against53%53\\%\), and within each flagship the seeds whose gate opened wider score higher \(Pearsonr=\+0\.89r=\+0\.89and\+0\.81\+0\.81,n=10n=10; correlational across seeds, so supporting evidence rather than a causal demonstration\)\. Nothing in the objective rewards opening the gate, because atg=0g=0the gradient reachingggis proportional to how useful the descriptor would be if it were already admitted\. The zero\-initialized fusion avoids this: the descriptor enters at full scale from the first step and only its weights are learned\.

#### SIGReg is not adopted\.

Against the otherwise\-identical configuration, neither SIGReg\[[27](https://arxiv.org/html/2610.12738#bib.bib27)\]nor a Sliced\-Wasserstein variant is better on any flagship \(allp≥0\.08p\\geq 0\.08\), so the pre\-registered adoption rule \(non\-inferior everywhere and strictly better somewhere\) returns*keep VICReg*\. The collapse diagnostics nonetheless move: both sketching objectives raise the per\-feature standard deviation from0\.0890\.089to0\.520\.52–0\.640\.64against VICReg’s hinge target of11, and the Sliced\-Wasserstein variant additionally reduces mean pairwise cosine from0\.9560\.956to0\.7340\.734, which the Epps–Pulley statistic cannot do because centering each one\-dimensional sketch makes it invariant to a shared mean direction\. Representation quality and task score are therefore decoupled on these benchmarks\.

#### Permutation invariance and cardinality freedom, measured\.

With the slot removed, the encoder summary is invariant to variable reordering to the floating\-point rounding floor \(maxπ⁡∥𝐡t​\(xπ\)−𝐡t​\(x\)∥∞≈2×10−7\\max\_\{\\pi\}\\lVert\\mathbf\{h\}\_\{t\}\(x^\{\\pi\}\)\-\\mathbf\{h\}\_\{t\}\(x\)\\rVert\_\{\\infty\}\\approx 2\\times 10^\{\-7\}over2020permutations, both readouts, with and without a padding mask\); with the slot present the same quantity is10−210^\{\-2\}to2×10−12\\times 10^\{\-1\}\. On a*trained*model, however, the downstream cost of that violation is negligible: over5050permutations of the test set the slot\-free model moves by exactly00and the slot model by at most8×10−48\\times 10^\{\-4\}h\-AUROC, an order of magnitude below the0\.0090\.009run\-to\-run floor this pipeline produces from initialization alone\. One slot\-free encoder also runs unmodified at every variable count tested \(Ctest∈\{2,…,10\}C\_\{\\text\{test\}\}\\in\\\{2,\\dots,10\\\}againstCtrain=5C\_\{\\text\{train\}\}=5, h\-AUROC0\.8220\.822to0\.9080\.908\), whereas the slot encoder cannot be evaluated at anyC≠CtrainC\\neq C\_\{\\text\{train\}\}at all\.

#### A bound on the BATADAL row\.

Scored through the same evaluation code, on the same windows, strata, and re\-run as this study’s shipped row, a zero\-parameter threshold on the conservation residual itself reaches0\.9210\.921on the ongoing stratum, above the model’s own0\.8730\.873, and0\.4890\.489\(chance\) on onset\. That row is therefore a detection result on which a training\-free detector suffices, consistent with the deterministic\-detector column of Table[1](https://arxiv.org/html/2610.12738#S4.T1), and it carries no evidence about anticipation in either direction\.

## Appendix IPrior\-only and generic\-routing baselines

A prior\-versus\-no\-prior contrast holds the architecture fixed\. Table[5](https://arxiv.org/html/2610.12738#A9.T5)asks the converse: how far does the prior get on its own, and does it matter where it enters? Every scorer is evaluated on the same test windows and stratum as the corresponding KITE result\.

Table 5:Prior\-only and generic\-routing baselines \(h\-AUROC\)\. Prior\-only scorers use the prior and nothing else: no sensors, no encoder, no pretraining\. “Input channels” feeds the clock’s past phase to the encoder as extra variables, the way any covariate\-capable forecaster would take it\. Occupancy KITE arms are means over1010seeds, BATADAL over55\.Row \(estimand\)ScorerUsesh\-AUROCOccupancy \(silent\-onset\)no prior \(KITE backbone\)sensors0\.7620\.762clock only, time\-of\-day sineprior0\.7210\.721clock only, logistic fit on the calendarprior0\.0780\.078KITE, clock as encoder input channelssensors \+ prior0\.7970\.797KITE, clock to the predictor at the horizonsensors \+ prior0\.866\\mathbf\{0\.866\}BATADAL \(ongoing\)no prior \(zeroed\)sensors0\.1360\.136residual only, threshold \(0 parameters\)prior0\.921\\mathbf\{0\.921\}residual only, logistic regressionprior0\.8160\.816KITE, residual as covariatesensors \+ prior0\.8770\.877BATADAL \(overall, full labels\)residual only, thresholdprior0\.6310\.631KITE, residual as covariatesensors \+ prior0\.6370\.637The two flagships answer differently\. On Occupancy the clock alone is weaker than no prior at all, and a fitted calendar model inverts because the finetuning split spans under two days, so the calendar’s phase does not carry across periods\. What converts phase into a hazard is the architecture, and where the clock enters matters: as ordinary input channels it adds\+0\.035\+0\.035, at the horizon\+0\.104\+0\.104\. On BATADAL the residual alone is sufficient: a threshold on it outranks the trained pipeline on the ongoing stratum and matches it overall\. That row therefore shows the value of the residual as a detection feature, not of the architecture, as Appendix[H](https://arxiv.org/html/2610.12738#A8)notes\. We did not run the external forecasters of Table[1](https://arxiv.org/html/2610.12738#S4.T1)with the prior as an extra input; the encoder\-input arm above is the within\-backbone version of that baseline\.

## Appendix JReproducibility

All experiments use public benchmarks\. The real datasets are Occupancy Detection\[[28](https://arxiv.org/html/2610.12738#bib.bib28),[29](https://arxiv.org/html/2610.12738#bib.bib29)\], the BATADAL water\-distribution challenge data\[[30](https://arxiv.org/html/2610.12738#bib.bib30)\], ETTm1\[[31](https://arxiv.org/html/2610.12738#bib.bib31)\], the Beijing multi\-site air\-quality data\[[32](https://arxiv.org/html/2610.12738#bib.bib32),[33](https://arxiv.org/html/2610.12738#bib.bib33)\], SKAB\[[34](https://arxiv.org/html/2610.12738#bib.bib34)\], GECCO water\-quality\[[35](https://arxiv.org/html/2610.12738#bib.bib35)\], the Weather benchmark \(the Jena weather\-station record,[https://www\.bgc\-jena\.mpg\.de/wetter/](https://www.bgc-jena.mpg.de/wetter/), as released with\[[19](https://arxiv.org/html/2610.12738#bib.bib19)\]\), CMAPSS FD001\[[36](https://arxiv.org/html/2610.12738#bib.bib36)\], and the Tennessee Eastman process\[[25](https://arxiv.org/html/2610.12738#bib.bib25),[26](https://arxiv.org/html/2610.12738#bib.bib26)\], all obtained from their standard public releases \(the UCI Machine Learning Repository for Occupancy and Beijing air\-quality; the BATADAL challenge, GECCO challenge, and SKAB distributions; the long\-horizon forecasting benchmark collection for ETTm1 and Weather; the NASA prognostics repository for CMAPSS; and Harvard Dataverse for TEP\)\. The synthetic datasets \(SYNTH\-ROUTE\-v2 and SYNTH\-CON\) are generated by the released code from fixed seeds; their generators, together with the typed semantic priors and per\-dataset causal graphs, will be released with the code\. Typed priors are provided for BATADAL, Beijing air\-quality, ETTm1, GECCO, SKAB, and Weather, and causal graphs for all2222datasets\.

The primary metric is h\-AUROC, the AUROC of the event\-probability surface pooled and averaged over forecast horizons1,…,K1,\\dots,K\(Section[3\.2](https://arxiv.org/html/2610.12738#S3.SS2)\)\. Every prior\-versus\-null comparison is paired by seed under an otherwise identical recipe, so the only intended difference between arms is the prior input; each arm is pretrained and finetuned independently, except in the transfer study \(Appendix[C](https://arxiv.org/html/2610.12738#A3)\), where the prior is delivered predictor\-side and the arms share one cached Stage\-1 checkpoint\. In\-domain effects are reported over55to1010seeds and transfer effects over55seeds, with a zeroed\-prior and a shuffled\-prior control ladder; significance uses paired tests with Benjamini–Hochberg FDR control at0\.050\.05across the multiple\-comparison families noted in each result\. The external baselines \(Chronos\-2, and a PatchTST/iTransformer backbone\) are run over55seeds on the same splits and strata, with the deterministic\-detector arms run once\. The complete configuration files, checkpoints manifest, and per\-seed result tables will be released together with the code\.