Out-of-distribution Neural Inference in Dynamical Ising Models

# Out-of-distribution Neural Inference in Dynamical Ising Models
Source: [https://arxiv.org/html/2607.03039](https://arxiv.org/html/2607.03039)
Yuan\-Bin ZhuCenter for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing 10048, ChinaShuang Qiao[qiaos@cnu\.edu\.cn](https://arxiv.org/html/2607.03039v1/mailto:[email protected])Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing 10048, ChinaShi\-Ju Ran[sjran@cnu\.edu\.cn](https://arxiv.org/html/2607.03039v1/mailto:[email protected])Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing 10048, China

###### Abstract

Neural networks are increasingly used to infer hidden physical structure from dynamical observations, yet it remains unclear whether their out\-of\-distribution performance reflects transferable physical rule learning\. We address this question in a controlled inverse problem: reconstructing interaction graphs of a kinetic Ising model from Glauber magnetization trajectories\. Across convolutional, graph, Transformer, and hybrid architectures, we find that data\-driven training produces distinct and reproducible statistical strategies under topology and temperature shifts\. Edge\-population diagnostics reveal that Transformer\-based models tend to preserve the link density of the training ensemble, whereas convolutional models can collapse toward sparse\- or no\-link predictions that appear out\-of\-distribution stable by exploiting the majority no\-link class\. Thus, high in\-distribution accuracy and apparent out\-of\-distribution robustness do not necessarily imply a learned dynamics\-to\-structure rule\. Instead, neural reconstruction can be governed by architecture\-dependent statistical priors\. Our results identify a concrete failure mode of standard data\-driven learning in physical inverse problems and motivate rule\-guided principles for machine\-learning\-assisted scientific discovery\.

## Introduction\.—

Artificial intelligence \(AI\) and machine learning \(ML\) have become increasingly important tools in, e\.g\., physicsCarleo and Troyer \([2017](https://arxiv.org/html/2607.03039#bib.bib1)\); He \([2024](https://arxiv.org/html/2607.03039#bib.bib2)\); Bracco and others \([2025](https://arxiv.org/html/2607.03039#bib.bib3)\), biologyJumper and others \([2021](https://arxiv.org/html/2607.03039#bib.bib4)\); Hwang and others \([2024](https://arxiv.org/html/2607.03039#bib.bib5)\); Maket al\.\([2024](https://arxiv.org/html/2607.03039#bib.bib6)\), materials scienceLi and others \([2025b](https://arxiv.org/html/2607.03039#bib.bib7)\); Xie and Grossman \([2018](https://arxiv.org/html/2607.03039#bib.bib8)\), and climate researchBracco and others \([2025](https://arxiv.org/html/2607.03039#bib.bib3)\); Schneider and others \([2022](https://arxiv.org/html/2607.03039#bib.bib10)\); Reichstein and others \([2019](https://arxiv.org/html/2607.03039#bib.bib11)\)\. Beyond accelerating interpolation within existing data, a central ambition of AI for Science is to assist scientific discovery by predicting structures or mechanisms, in regimes that are not represented in the training dataLi and others \([2025b](https://arxiv.org/html/2607.03039#bib.bib7)\); Caro and others \([2023](https://arxiv.org/html/2607.03039#bib.bib16)\); Li and others \([2025a](https://arxiv.org/html/2607.03039#bib.bib17)\)\. This ambition makes out\-of\-distribution \(OOD\) prediction a necessary benchmark for scientific ML: discovery requires extrapolation beyond the statistical ensemble from which the model has learned, not merely accurate prediction on new samples drawn from the same distributionLi and others \([2025b](https://arxiv.org/html/2607.03039#bib.bib7)\); Omee and others \([2024](https://arxiv.org/html/2607.03039#bib.bib21)\); Muckley and others \([2023](https://arxiv.org/html/2607.03039#bib.bib52)\); Segal and others \([2025](https://arxiv.org/html/2607.03039#bib.bib53)\); Ursu and others \([2025](https://arxiv.org/html/2607.03039#bib.bib54)\)\.

This requirement is in tension with the standard data\-driven learning paradigm\. Most supervised ML models are trained by minimizing empirical prediction error on finite datasets and are validated under an independently and identically distributed assumption\. Under this setting, high in\-distribution \(ID\) accuracy primarily demonstrates interpolation within the training ensemble\. It does not, by itself, establish that the model has learned a transferable physical rule\. In scientific applications, however, the relevant target often lies outside the training ensemble: a new material composition, a new phase, a new interaction topology, or a new dynamical regime\. The key question is therefore not only whether neural networks predict accurately, but what kind of inference they perform when the test system violates the statistical ID assumptions of the training data\.

The distinction between statistical interpolation and transferable physical inference has been widely discussed in the context of distribution shift\. Studies in images, language, and benchmark learning tasks have shown that models with high ID performance can fail when the test distribution changesKoh and others \([2021](https://arxiv.org/html/2607.03039#bib.bib18)\); Hendrycks and others \([2021](https://arxiv.org/html/2607.03039#bib.bib19)\); Ovadia and others \([2019](https://arxiv.org/html/2607.03039#bib.bib20)\); Omee and others \([2024](https://arxiv.org/html/2607.03039#bib.bib21)\); Koch and others \([2024](https://arxiv.org/html/2607.03039#bib.bib22)\)\. Similar concerns arise in physical applications, including materials property predictionOmee and others \([2024](https://arxiv.org/html/2607.03039#bib.bib21)\), quantum dynamics learningCaro and others \([2023](https://arxiv.org/html/2607.03039#bib.bib16)\), and complex\-flow modelingRabeh and others \([2025](https://arxiv.org/html/2607.03039#bib.bib27)\), where distribution shifts may be induced by changes in structure, control parameters, or regions of state spaceOmee and others \([2024](https://arxiv.org/html/2607.03039#bib.bib21)\); Rabeh and others \([2025](https://arxiv.org/html/2607.03039#bib.bib27)\); Caro and others \([2023](https://arxiv.org/html/2607.03039#bib.bib16)\); Vasiliauskaite and others \([2024](https://arxiv.org/html/2607.03039#bib.bib26)\)\. For inverse problems, this issue is even sharper: the model must infer hidden physical structures or parameters from finite observations, and an accurate prediction may result either from a learned physical relation or from a statistical regularity inherited from the training ensembleNguyenet al\.\([2017](https://arxiv.org/html/2607.03039#bib.bib32)\); Karnakov and others \([2024](https://arxiv.org/html/2607.03039#bib.bib33)\); Bingham and others \([2024](https://arxiv.org/html/2607.03039#bib.bib34)\); Patel and others \([2022](https://arxiv.org/html/2607.03039#bib.bib35)\)\.

Here we address this issue using the kinetic Ising model with Glauber dynamics as a controlled physical testbedGlauber \([1963](https://arxiv.org/html/2607.03039#bib.bib50)\)\. The forward problem maps an interaction topology, represented by an adjacency matrix, to time\-dependent local magnetization trajectories\. The inverse problem considered here is to reconstruct the underlying interaction topology from these trajectories \(Fig\.[1](https://arxiv.org/html/2607.03039#S0.F1)\)\. This allows us to separate three key notions that are often conflated in scientific ML: ID fitting, OOD prediction, and transferable dynamics\-to\-structure inference\. The training and ID test data are generated from the same ensemble of lattice topologies and temperatures\. We consider OOD generalization under two physically distinct distribution shifts: the topology shift with the lattice structures absent from training, and the temperature shift where the trajectories are generated at unseen temperatures\. We compare convolutional neural networks \(CNNs\), graph neural networks \(GNNs\), Transformers, and hybrid architectures, which encode different inductive biases for processing spatiotemporal magnetization data\.

Our results reveal that data\-driven neural reconstruction in this physical inverse problem is governed by architecture\-dependent inference strategies under distribution shift\. Although different architectures usually achieve high in\-distribution accuracy, which is a widely\-recognized fact, they respond differently when the graph topology or dynamical temperature is changed\. Edge\-population diagnostics show that Transformer\-based models tend to preserve the link density of the training ensemble, whereas convolutional models can collapse toward sparse\-link or no\-link predictions that exploit the majority no\-link class\. These behaviors show that different neural architectures do not merely differ in accuracy; they implement distinct statistical strategies for mapping dynamics to structure\. This positive diagnostic finding leads to an important caution: apparent OOD robustness need not imply a transferable physical rule, but may instead arise from architecture\-dependent statistical priors\. The results identify a concrete failure mode of standard supervised learning in physical inverse problems and motivate mechanism\-aware, rule\-guided approaches for machine\-learning\-assisted scientific discovery\.

![Refer to caption](https://arxiv.org/html/2607.03039v1/Fig1.png)Figure 1:Schematic illustration of topology reconstruction in the kinetic Ising model using neural\-network models\. Given a time\-dependent local magnetization trajectory, the model predicts the underlying adjacency matrix of the interaction graph\.
## Kinetic Ising model as a testbed for generalization\.—

We consider a high\-dimensional Ising model with interaction topology encoded by an adjacency matrix𝑨\\bm\{A\},

H=−J​∑i,jAi​j​si​sj\.H=\-J\\sum\_\{i,j\}A\_\{ij\}s\_\{i\}s\_\{j\}\.\(1\)The local magnetization dynamics governed by the mean\-field Glauber equation

d​mid​t=−mi\+tanh⁡\(β​J​∑jAi​j​mj\),\\frac\{dm\_\{i\}\}\{dt\}=\-m\_\{i\}\+\\tanh\\Big\(\\beta J\\sum\_\{j\}A\_\{ij\}m\_\{j\}\\Big\),\(2\)wheremi​\(t\)=⟨si​\(t\)⟩m\_\{i\}\(t\)=\\langle s\_\{i\}\(t\)\\rangle\(note we take the Boltzmann constant askB=1k\_\{B\}=1\)\. For each topology, we numerically solve Eq\. \(2\) from sampled initial conditions and use the resulting magnetization trajectories as neural\-network inputs\. The prediction target is the upper\-triangular part of𝑨\\bm\{A\}, corresponding to all candidate undirected links\.

We evaluate the ID and two OOD test sets\. The ID set uses the same topology ensemble and temperature as training, with independently sampled initial conditions\. The topology\-shift OOD set uses unseen lattice structures at fixed temperature, whereas the temperature\-shift OOD set uses training topologies but generates trajectories at unseen temperatures\. These settings separately probe structural and thermodynamic distribution shifts\.

We compare two\-layer CNN \(CNN\-2\), a deeper three\-layer CNN \(CNN\-3\), GNN, Transformer, and hybrid CNN\-Transformer architectures, which encode convolutional localityD’Ascoli and others \([2021](https://arxiv.org/html/2607.03039#bib.bib37)\); Alzubaidi and others \([2021](https://arxiv.org/html/2607.03039#bib.bib41)\), message\-passing relational structureBattaglia and others \([2018](https://arxiv.org/html/2607.03039#bib.bib42)\), attention\-based long\-range dependenceD’Ascoli and others \([2021](https://arxiv.org/html/2607.03039#bib.bib37)\); Vaswani and others \([2017](https://arxiv.org/html/2607.03039#bib.bib44)\), and hybrid local–global processingD’Ascoli and others \([2021](https://arxiv.org/html/2607.03039#bib.bib37)\); Lu and others \([2022](https://arxiv.org/html/2607.03039#bib.bib46)\), respectively\. The details for different architectures are given in the Supplemental Material[1](https://arxiv.org/html/2607.03039#bib.bib56)\. All models are trained forL=12L=12spins, givingL​\(L−1\)/2=66L\(L\-1\)/2=66candidate links\. The main setting usesNc=25N\_\{c\}=25true links, for which a no\-link predictor already reaches41/66=62\.1%41/66=62\.1\\%\. We therefore report both reconstruction accuracyγ\\gammaand the average predicted link numberN^c\\widehat\{N\}\_\{c\}\. A balancedNc=33N\_\{c\}=33with temperature shift is explored in the Supplemental Material[1](https://arxiv.org/html/2607.03039#bib.bib56)\. Training uses Adam optimizerKingma and Ba \([2014](https://arxiv.org/html/2607.03039#bib.bib51)\)and binary cross\-entropy loss,

ℒ=−1N​∑i=1N\[yi​log⁡\(y^i\)\+\(1−yi\)​log⁡\(1−y^i\)\]\.\\mathcal\{L\}=\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\[y\_\{i\}\\log\(\\widehat\{y\}\_\{i\}\)\+\(1\-y\_\{i\}\)\\log\(1\-\\widehat\{y\}\_\{i\}\)\\right\]\.\(3\)
![Refer to caption](https://arxiv.org/html/2607.03039v1/Fig2.png)Figure 2:\(a\) Training accuracyγIDtra\\gamma\_\{\\mathrm\{ID\}\}^\{\\mathrm\{tra\}\}, \(b\) ID test accuracyγIDtest\\gamma\_\{\\mathrm\{ID\}\}^\{\\mathrm\{test\}\}, and \(c\) topology\-shift OOD accuracyγOODtop\\gamma\_\{\\mathrm\{OOD\}\}^\{\\mathrm\{top\}\}of the five models as a function of the number of training lattice topologiesNLN\_\{L\}\. The dashed line labels the accuracy of the no\-link baseline,41/66=62\.1%41/66=62\.1\\%\.
## OOD generalization under topology shift\.—

We first evaluate under a topology shift\. The training set containsN=35000N=35000trajectories generated atT=1T=1fromNLN\_\{L\}lattice topologies, withN/NLN/N\_\{L\}trajectories sampled from each topology\. The ID test set uses the same topology ensemble and temperature but independently sampled initial conditions, whereas the OOD set replaces the training topologies by unseen lattices at the same temperature\.

Figures[2](https://arxiv.org/html/2607.03039#S0.F2)\(a\) and[2](https://arxiv.org/html/2607.03039#S0.F2)\(b\) show that all five neural networks reach high training and ID test accuracies, with similar trends asNLN\_\{L\}varies\. Generally, the ranking is Transformer, Hybrid, GNN, CNN\-2, and CNN\-3, indicating that Transformer\-based models extract the most predictive information within the ID setting\. The decrease of ID accuracies with increasingNLN\_\{L\}is consistent with the fixed sample budget: increasingNLN\_\{L\}both reduces the samples per topology and broadens the set of topology\-dependent dynamical responses, thereby making the inverse problem harder\.

The same models behave qualitatively differently under topology shift\. As shown in Fig\.[2](https://arxiv.org/html/2607.03039#S0.F2)\(c\), all accuracies drop substantially; forNL=10N\_\{L\}=10, they are only around56%56\\%\. The ranking is also reversed: CNN\-2 and CNN\-3 outperform the Transformer\. Moreover, the OOD accuracy increases withNLN\_\{L\}, with CNN\-3 rising from about56%56\\%atNL=10N\_\{L\}=10to about61%61\\%atNL=50N\_\{L\}=50, opposite to the ID trend\.

These results show that ID and topology\-OOD tests probe different aspects of the learned inverse map\. ID accuracy primarily measures trajectory\-level interpolation: the model is evaluated on unseen dynamical realizations generated from the same topology ensemble used for training\. In this regime, increasingNLN\_\{L\}makes the reconstruction task harder, and the accuracy decreases toward, but remains above, the no\-link baseline\.

By contrast, the topology\-OOD test probes topology\-level extrapolation, requiring the same learned map to reconstruct interaction graphs that are absent from the training ensemble\. The opposite dependence onNLN\_\{L\}therefore indicates that the inductive biases favoring ID interpolation do not necessarily favor transferable dynamics\-to\-topology inference\. More importantly, it shows that OOD accuracy alone is not a direct measure of physical rule learning: it must be interpreted together with reconstruction diagnostics that reveal what type of graph the model actually predicts\. This motivates a complementary diagnostic beyond edge\-wise accuracy\.

## OOD generalization under temperature shift\.—

We next test temperature shift by training all models atT=1T=1withN=35000N=35000trajectories andNL=50N\_\{L\}=50topologies, and evaluating them on the same topologies at unseen temperatures ranging fromT=0\.05T=0\.05to100100\. This setting keeps the interaction graphs fixed while changing the dynamical trajectories generated by the Glauber equation\.

Figure[3](https://arxiv.org/html/2607.03039#S0.F3)\(a\) shows an asymmetric response to low\- and high\-temperature shifts\. Near the training temperatureT=1T=1, all models retain high accuracy and preserve the ID ranking\. In the low\-temperature regime,T=0\.05T=0\.05–0\.20\.2, the accuracy decreases moderately while the ranking remains unchanged, with the Transformer dropping from about90%90\\%atT=1T=1to about75%75\\%atT=0\.05T=0\.05, whereas CNN\-3 remains near63%63\\%\. The high\-temperature regime is more disruptive\. AsTTincreases, all models degrade, and a crossover occurs aroundT≃5T\\simeq 5–1010, after which the ranking becomes CNN\-3, CNN\-2, Hybrid, GNN, and Transformer\. In this regime, the accuracies fall below the no\-link baseline, indicating that temperature shift can destroy link\-specific predictive information rather than merely perturb the input distribution\.

The stronger degradation under high\-temperature shifts than under low\-temperature shifts has a direct physical origin\. LoweringTTenhances magnetic ordering and shifts the trajectory distribution, but topology\-dependent correlations remain partly visible in the ordered dynamics\. RaisingTT, by contrast, suppresses magnetic ordering and weakens topology\-dependent dynamical responses, thereby reducing the information available for link reconstruction\. Temperature shift therefore probes more than robustness to a changed input distribution: it tests whether the learned inverse map can still extract topology\-relevant signals when the dynamical signatures of the interaction topology are weakened by thermal fluctuations\.

![Refer to caption](https://arxiv.org/html/2607.03039v1/Fig3.png)Figure 3:\(a\) Temperature\-shift OOD accuracyγOODtem\\gamma\_\{\\mathrm\{OOD\}\}^\{\\mathrm\{tem\}\}of the five models as a function ofTT\. The dashed line labels the accuracy of the no\-link baseline,41/66=62\.1%41/66=62\.1\\%\. \(b\) Average number of predicted linksN^c\\widehat\{N\}\_\{c\}in the reconstructed adjacency matrices on the ID test set as a function ofNLN\_\{L\}\. \(c\) Average number of predicted linksN^c\\widehat\{N\}\_\{c\}on the temperature\-shift OOD test set as a function ofTTin theNc=25N\_\{c\}=25setting\. The dash\-dotted lines indicate the true link numberNc=25N\_\{c\}=25\.
## Prediction strategies behind apparent OOD robustness\.—

The preceding OOD tests reveal a central ambiguity: similar accuracy values can arise from qualitatively different reconstruction strategies\. A model may achieve high accuracy by identifying the correct links, by preserving the typical link density of the training ensemble, or by exploiting the majority no\-link class\. This ambiguity is especially important in theNc=25N\_\{c\}=25setting, where 41 of the 66 candidate pairs are unlinked and a naive no\-link predictor already reaches41/66=62\.1%41/66=62\.1\\%\. Thus, the nearly flat OOD accuracy of CNN\-3 and the strong degradation of the Transformer cannot be interpreted from accuracy alone\. This calls for a diagnostic of what type of graph each model actually predicts\.

We use the average predicted number of links,N^c\\widehat\{N\}\_\{c\}, as such a diagnostic \(Figs\.[3](https://arxiv.org/html/2607.03039#S0.F3)\(b\) and[3](https://arxiv.org/html/2607.03039#S0.F3)\(c\)\)\. This quantity separates models that preserve the training graph density from those that collapse toward sparse\-link or no\-link predictions\. On the ID test set, the Transformer largely preserves the graph density, withN^c\\widehat\{N\}\_\{c\}decreasing only from about 25 atNL=10N\_\{L\}=10to about 23 atNL=50N\_\{L\}=50\. CNN\-3 behaves differently:N^c\\widehat\{N\}\_\{c\}drops from about 18 to about 3 over the same range\. This contrast persists under temperature shift atNL=50N\_\{L\}=50\. OverT=0\.05T=0\.05–100100, CNN\-3 remains nearN^c≃3\\widehat\{N\}\_\{c\}\\simeq 3, whereas the Transformer continues to predict many links, withN^c≃21\\widehat\{N\}\_\{c\}\\simeq 21atT=0\.05T=0\.05,N^c≃23\\widehat\{N\}\_\{c\}\\simeq 23atT=1T=1, andN^c≃18\\widehat\{N\}\_\{c\}\\simeq 18forT≳10T\\gtrsim 10\.

These diagnostics reveal two distinct prediction strategies\. The Transformer follows a density\-preserving strategy: it predicts a graph density close to that of the training ensemble, which supports high ID accuracy when the learned correlations also localize links correctly\. Under OOD shifts, however, this strategy becomes fragile because many predicted links are placed at incorrect positions\. CNN\-3 instead moves toward a conservative no\-link majority strategy in the imbalanced task\. Its relatively flat OOD accuracy mainly reflects correct classification of many no\-link entries, not reliable recovery of true links\.

This interpretation also consistently clarifies the topology\-shift results in Fig\.[2](https://arxiv.org/html/2607.03039#S0.F2)\(c\)\. IncreasingNLN\_\{L\}can reduce overfitting to a small set of graph realizations, but it also makes some models more conservative, as indicated by the decreasingN^c\\widehat\{N\}\_\{c\}\. In the imbalancedNc=25N\_\{c\}=25setting, such conservativeness can raise apparent accuracy by suppressing false positives on the majority no\-link class\. Since the OOD accuracy remains close to or below the no\-link baseline, however, this improvement should not be interpreted as strong knowledge transfer\. A balanced\-density control withNc=33N\_\{c\}=33, reported in the Supplemental Material[1](https://arxiv.org/html/2607.03039#bib.bib56), confirms that the low\-link behavior of CNN\-3 is shaped by the interplay between architectural bias and the class prior\.

These results sharpen the interpretation of OOD robustness in physical inverse problems\. In the kinetic Ising reconstruction task, stable OOD accuracy can arise either from transferable dynamics\-to\-structure inference or from statistical strategies such as preserving the training graph density and exploiting the majority no\-link class\. The predicted link population,N^c\\widehat\{N\}\_\{c\}, exposes this distinction and shows that different architectures implement different inference strategies under the same physical shift\. Thus, OOD accuracy alone is insufficient evidence for physical rule learning, and it must be accompanied by diagnostics of the predicted graph structure, consistent with broader observations that deep networks can exploit shortcut rules and that standard accuracy can obscure qualitatively different decision strategiesGeirhos and others \([2020](https://arxiv.org/html/2607.03039#bib.bib47)\); Lapuschkin and others \([2019](https://arxiv.org/html/2607.03039#bib.bib48)\); Megahed and others \([2021](https://arxiv.org/html/2607.03039#bib.bib49)\)\.

## Conclusion\.—

We have studied out\-of\-distribution neural reconstruction in a controlled physical inverse problem: inferring kinetic Ising interaction graphs from Glauber magnetization trajectories\. Although all architectures achieve high in\-distribution accuracy, topology and temperature shifts reveal that their OOD behavior is governed by distinct data\-driven inference strategies\. Edge\-population diagnostics show that Transformer\-based models tend to preserve the link density of the training ensemble, whereas convolutional models can collapse toward sparse\-link or no\-link predictions that exploit class imbalance\. Thus, apparent OOD robustness does not necessarily imply transferable dynamics\-to\-structure rule learning\. These results point to a broader open challenge for AI\-assisted physics: how to combine data, architectures, and training objectives so that neural models move beyond dataset\-specific statistical shortcuts toward rule\-guided, knowledge\-driven physical prediction\.

###### Acknowledgements\.

This work was supported by the National Natural Science Foundation of China \(Grant No\. 12404092\) and Beijing Natural Science Foundation \(Grant No\. QY25387\)\. The numerical simulations were partially performed on the robotic AI\-Scientist platform of Chinese Academy of Sciences\.

## References

- \[1\]Note:Supplemental material containing neural\-network architecture details and the balanced\-link control withNc=33N\_\{c\}=33\.Cited by:[Kinetic Ising model as a testbed for generalization\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px2.p3.7),[Prediction strategies behind apparent OOD robustness\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px5.p4.4)\.
- Review of deep learning: concepts, cnn architectures, challenges, applications, future directions\.J\. Big Data8,pp\. 53\.Cited by:[Kinetic Ising model as a testbed for generalization\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px2.p3.7)\.
- P\. W\. Battagliaet al\.\(2018\)Relational inductive biases, deep learning, and graph networks\.arXiv preprint\.External Links:1806\.01261Cited by:[Kinetic Ising model as a testbed for generalization\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px2.p3.7)\.
- D\. Binghamet al\.\(2024\)Inverse problems for physics\-based process models\.Annu\. Rev\. Stat\. Appl\.11,pp\. 461–482\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- A\. Braccoet al\.\(2025\)Machine learning for the physics of climate\.Nat\. Rev\. Phys\.7,pp\. 6–20\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- G\. Carleo and M\. Troyer \(2017\)Solving the quantum many\-body problem with artificial neural networks\.Science355,pp\. 602–606\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- M\. C\. Caroet al\.\(2023\)Out\-of\-distribution generalization for learning quantum dynamics\.Nat\. Commun\.14,pp\. 3751\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1),[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- S\. D’Ascoliet al\.\(2021\)ConViT: improving vision transformers with soft convolutional inductive biases\.InProceedings of the 38th International Conference on Machine Learning,PMLR, Vol\.139\.Cited by:[Kinetic Ising model as a testbed for generalization\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px2.p3.7)\.
- R\. Geirhoset al\.\(2020\)Shortcut learning in deep neural networks\.Nat\. Mach\. Intell\.2,pp\. 665–673\.Cited by:[Prediction strategies behind apparent OOD robustness\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px5.p5.1)\.
- R\. J\. Glauber \(1963\)Time\-dependent statistics of the ising model\.J\. Math\. Phys\.4,pp\. 294\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p4.1)\.
- Y\.\-H\. He \(2024\)AI\-driven research in pure mathematics and theoretical physics\.Nat\. Rev\. Phys\.6,pp\. 546–553\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- D\. Hendryckset al\.\(2021\)The many faces of robustness: a critical analysis of out\-of\-distribution generalization\.InProceedings of the IEEE/CVF International Conference on Computer Vision,Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- H\. Hwanget al\.\(2024\)Big data and deep learning for rna biology\.Exp\. Mol\. Med\.56,pp\. 1293–1321\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- J\. Jumperet al\.\(2021\)Highly accurate protein structure prediction with alphafold\.Nature596,pp\. 583–589\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- P\. Karnakovet al\.\(2024\)Solving inverse problems in physics by optimizing a discrete loss: fast and accurate learning without neural networks\.PNAS Nexus3,pp\. pgae005\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- D\. P\. Kingma and J\. Ba \(2014\)Adam: a method for stochastic optimization\.arXiv preprint\.External Links:1412\.6980Cited by:[Kinetic Ising model as a testbed for generalization\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px2.p3.7)\.
- L\. M\. Kochet al\.\(2024\)Distribution shift detection for the postmarket surveillance of medical ai algorithms: a retrospective simulation study\.npj Digit\. Med\.7,pp\. 120\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- P\. W\. Kohet al\.\(2021\)WILDS: a benchmark of in\-the\-wild distribution shifts\.InProceedings of the International Conference on Machine Learning,pp\. 5637–5664\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- S\. Lapuschkinet al\.\(2019\)Unmasking clever hans predictors and assessing what machines really learn\.Nat\. Commun\.10,pp\. 1096\.Cited by:[Prediction strategies behind apparent OOD robustness\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px5.p5.1)\.
- H\. Liet al\.\(2025a\)Out\-of\-distribution generalization on graphs: a survey\.IEEE Trans\. Pattern Anal\. Mach\. Intell\.47,pp\. 10490–10512\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- K\. Liet al\.\(2025b\)Probing out\-of\-distribution generalization in machine learning for materials\.Commun\. Mater\.6,pp\. 9\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- Z\. Luet al\.\(2022\)Bridging the gap between vision transformers and convolutional neural networks on small datasets\.InAdvances in Neural Information Processing Systems,Cited by:[Kinetic Ising model as a testbed for generalization\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px2.p3.7)\.
- K\.\-K\. Mak, Y\.\-H\. Wong, and M\. R\. Pichika \(2024\)Artificial intelligence in drug discovery and development\.InDrug Discovery and Evaluation: Safety and Pharmacokinetic Assays,F\. J\. Hock and M\. K\. Pugsley \(Eds\.\),External Links:[Document](https://dx.doi.org/10.1007/978-3-031-35529-5%5F92)Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- F\. M\. Megahedet al\.\(2021\)The class imbalance problem\.Nat\. Methods18,pp\. 1270–1272\.Cited by:[Prediction strategies behind apparent OOD robustness\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px5.p5.1)\.
- E\. S\. Muckleyet al\.\(2023\)Interpretable models for extrapolation in scientific machine learning\.Digital Discovery2,pp\. 1425–1435\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- H\. C\. Nguyen, R\. Zecchina, and J\. Berg \(2017\)Inverse statistical problems: from the inverse ising problem to data science\.Adv\. Phys\.66,pp\. 197–261\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- S\. S\. Omeeet al\.\(2024\)Structure\-based out\-of\-distribution materials property prediction: a benchmark study\.npj Comput\. Mater\.10,pp\. 144\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1),[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- Y\. Ovadiaet al\.\(2019\)Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift\.Adv\. Neural Inf\. Process\. Syst\.32,pp\. 13991\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- D\. V\. Patelet al\.\(2022\)Solution of physics\-based bayesian inverse problems with deep generative priors\.Comput\. Methods Appl\. Mech\. Eng\.400,pp\. 115428\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- A\. Rabehet al\.\(2025\)Benchmarking scientific machine\-learning approaches for flow prediction around complex geometries\.Commun\. Eng\.4,pp\. 182\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- M\. Reichsteinet al\.\(2019\)Deep learning and process understanding for data\-driven earth system science\.Nature566,pp\. 195–204\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- T\. Schneideret al\.\(2022\)Climate goals and computing the future of clouds\.Nature604,pp\. 463–468\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- N\. Segalet al\.\(2025\)Known unknowns: out\-of\-distribution property prediction in materials and molecules\.npj Comput\. Mater\.11,pp\. 345\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- E\. Ursuet al\.\(2025\)Training data composition determines machine learning generalization and biological rule discovery\.Nat\. Mach\. Intell\.7,pp\. 1206–1219\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.
- V\. Vasiliauskaiteet al\.\(2024\)Generalization of neural network models for complex network dynamics\.Commun\. Phys\.7,pp\. 348\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p3.1)\.
- A\. Vaswaniet al\.\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,Cited by:[Kinetic Ising model as a testbed for generalization\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px2.p3.7)\.
- T\. Xie and J\. C\. Grossman \(2018\)Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties\.Phys\. Rev\. Lett\.120,pp\. 145301\.Cited by:[Introduction\.—](https://arxiv.org/html/2607.03039#S0.SS0.SSS0.Px1.p1.1)\.