Physics-Informed Neural Networks with Learnable Loss Balancing and Transfer Learning

# Physics-Informed Neural Networks with Learnable Loss Balancing and Transfer Learning
Source: [https://arxiv.org/html/2605.05217](https://arxiv.org/html/2605.05217)
\\patchcmd

Physics\-Informed Neural Networks with Learnable Loss Balancing and Transfer Learning

###### Abstract

We propose a self\-supervised physics\-informed neural network \(PINN\) framework that adaptively balances physics\-based and data\-driven supervision for scientific machine learning under data scarcity\. Unlike prior PINNs that rely on fixed or heuristic weighting of physics residuals and data loss, our approach introduces a learnable blending neuron that dynamically adjusts the relative contribution of each term based on their uncertainties\. This mechanism enables stable training and improved generalization without manual tuning\. To further enhance efficiency, we integrate a transfer learning strategy that reuses representations from related domains and adapts them to new physical systems with limited data\. We validate the framework for the prediction of heat transfer in liquid\-metal miniature heat sinks using only 87 CFD datapoints, where the adaptive PINN achieves an error<8%<8\\%, outperforming shallow neural networks, kernel methods, and physics\-only baselines\. Our framework provides a general recipe for embedding physics adaptively into neural networks, offering a robust and reproducible approach for data\-scarce problems across various scientific domains, including fluid dynamics and material modeling\.

## 1Introduction

Scientific machine learning is increasingly solving problems where only limited data is available, where incorporating underlying physical principles is essential, such as space missionsPirayeshshirazinezhad and Fathi \([2025](https://arxiv.org/html/2605.05217#bib.bib26)\); Pirayeshshirazinezhadet al\.\([2022](https://arxiv.org/html/2605.05217#bib.bib25)\); Pirayeshshirazinezhad \([2022](https://arxiv.org/html/2605.05217#bib.bib28)\), swarm of robotsPirayeshshirazinezhad and Fathi \([2025](https://arxiv.org/html/2605.05217#bib.bib26)\), and particle acceleratorsDiaz Cruzet al\.\([2022](https://arxiv.org/html/2605.05217#bib.bib27)\); Pirayeshshirazinezhad \([2022](https://arxiv.org/html/2605.05217#bib.bib28)\); Cruzet al\.\([2019](https://arxiv.org/html/2605.05217#bib.bib29)\); Pirayeshet al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib30)\)\. Traditional data\-driven neural networks excel when large datasets are present, but their accuracy and stability degrade when observations are scarce or noisy\. In such cases, the incorporation of governing physics into the learning pipeline has shown a promising approachSharmaet al\.\([2023](https://arxiv.org/html/2605.05217#bib.bib6)\); Guastoniet al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib5)\)\. Physics\-informed neural networks \(PINNs\) and related frameworks integrate physical residuals directly into the loss function, constraining models to respect conservation laws and differential equations\.

A key challenge in PINNs is balancing the contributions of data\-driven and physics\-based losses\. Fixed or manually tuned weights can lead to poor convergence or biased predictions\. Prior studies have investigated adaptive strategies, including self\-adaptive PINNsMcClenny and Braga\-Neto \([2020](https://arxiv.org/html/2605.05217#bib.bib22)\), gradient\-normalization methodsWanget al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib23)\), and uncertainty\-driven weighting in Bayesian PINNsYanget al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib24)\)\. Despite these advances, most approaches require either heuristic rules, Bayesian overhead, or remain sensitive to hyperparameter choices\. This motivates the need for a lightweight, self\-supervised mechanism that dynamically adjusts the physics–data trade\-off during training\.

In parallel, transfer learning has gained prominence in both general MLZhuanget al\.\([2020](https://arxiv.org/html/2605.05217#bib.bib9)\); Liuet al\.\([2019](https://arxiv.org/html/2605.05217#bib.bib10)\)and scientific domainsJeonet al\.\([2022a](https://arxiv.org/html/2605.05217#bib.bib8);[b](https://arxiv.org/html/2605.05217#bib.bib7)\), enabling knowledge reuse across related tasks\. For thermal\-fluid applications, transfer learning has been used to accelerate CFD surrogatesBaghbanet al\.\([2019](https://arxiv.org/html/2605.05217#bib.bib1)\); Pirayeshshirazinezhad \([2025](https://arxiv.org/html/2605.05217#bib.bib12)\), but its integration with PINNs remains underexplored\. Combining adaptive physics–data weighting with transfer learning offers the potential for robust learning even in data\-scarce regimes\.

In this work, we propose a self\-supervised PINN framework with alearnable blending neuronthat dynamically balances physics residuals and data losses based on their uncertainties\. We further incorporate a transfer learning scheme that reuses hidden\-layer representations from related domains to accelerate training in new physical systems\. To evaluate the method, we consider the prediction of convective heat transfer in sodium\-cooled miniature heat sinks using only 87 CFD datapoints, a regime where high\-fidelity simulation is prohibitively costly\. Our contributions are threefold:

- •We introduce a simple yet effective self\-supervised mechanism for adaptive loss balancing in PINNs, removing the need for manual tuning or Bayesian complexity\.
- •We use transfer learning, demonstrating its effectiveness in scientific ML tasks with scarce data\.
- •We validate the framework on a challenging liquid\-metal heat transfer problem and benchmark against shallow neural networks, kernel methods, and physics\-only baselines\.

This framework provides a general recipe for embedding physics adaptively into neural networks, with implications for a wide range of domains including heat transfer, materials science, and aerospace engineering\.

## 2Related Work

#### Machine learning for thermal–fluid systems\.

Data\-driven models have been widely applied in fluid mechanics and thermal sciences, particularly for predicting heat transfer coefficients and fluid properties when experimental or CFD data are limited\. Examples include neural\-network and kernel\-based surrogates for nanofluid flowsBaghbanet al\.\([2019](https://arxiv.org/html/2605.05217#bib.bib1)\); Tafarrojet al\.\([2017](https://arxiv.org/html/2605.05217#bib.bib14)\); Kurt and Kayfeci \([2009](https://arxiv.org/html/2605.05217#bib.bib3)\); Yousefiet al\.\([2012](https://arxiv.org/html/2605.05217#bib.bib4)\), convective heat transfer prediction in coils and microchannelsBaghbanet al\.\([2016](https://arxiv.org/html/2605.05217#bib.bib2)\); Bhattacharyaet al\.\([2022](https://arxiv.org/html/2605.05217#bib.bib13)\), and convolutional\-network approaches for wall\-bounded turbulenceGuastoniet al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib5)\)\. More recent work has combined ML with high\-fidelity CFD solvers to accelerate simulationsJeonet al\.\([2022b](https://arxiv.org/html/2605.05217#bib.bib7)\); Pirayeshshirazinezhad \([2025](https://arxiv.org/html/2605.05217#bib.bib12)\)\. Comprehensive reviews of physics\-informed ML in fluid mechanics emphasize the potential of integrating physics into learning pipelinesSharmaet al\.\([2023](https://arxiv.org/html/2605.05217#bib.bib6)\)\.

#### Physics\-informed neural networks\.

PINNs incorporate governing equations as soft constraints, improving generalization under data scarcity\. However, balancing the relative contributions of physics and data remains challenging\. Several adaptive schemes have been proposed: self\-adaptive PINNs using gradient informationMcClenny and Braga\-Neto \([2020](https://arxiv.org/html/2605.05217#bib.bib22)\), NTK\-based analyses of PINN training pathologiesWanget al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib23)\), and Bayesian approaches that weight losses according to uncertaintyYanget al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib24)\)\. These methods improve training stability but often require heuristic tuning, additional complexity, or Bayesian overhead\. Our approach differs by introducing a simple*learnable blending neuron*that dynamically adjusts physics–data weighting in a fully self\-supervised manner\.

#### Transfer learning in scientific ML\.

Transfer learning has achieved success across domains from computer vision to geoscienceZhuanget al\.\([2020](https://arxiv.org/html/2605.05217#bib.bib9)\); Liuet al\.\([2019](https://arxiv.org/html/2605.05217#bib.bib10)\)\. In thermal–fluid contexts, transfer strategies have been applied to accelerate unsteady CFD simulationsJeonet al\.\([2022a](https://arxiv.org/html/2605.05217#bib.bib8)\)and other flow\-physics surrogates\. Nevertheless, integration of transfer learning into PINNs remains underexplored\. Our framework bridges this gap by combining transfer learning with adaptive physics–data balancing, enabling PINNs to efficiently adapt knowledge across related physical systems\.

#### Positioning of this work\.

In summary, while prior work has investigated adaptive weighting in PINNsMcClenny and Braga\-Neto \([2020](https://arxiv.org/html/2605.05217#bib.bib22)\); Wanget al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib23)\); Yanget al\.\([2021](https://arxiv.org/html/2605.05217#bib.bib24)\)and physics\-informed transfer learning for fluid simulationsJeonet al\.\([2022a](https://arxiv.org/html/2605.05217#bib.bib8)\), our contribution unifies these directions\. We present a*self\-supervised adaptive PINN with transfer learning*, validated on a challenging small\-data case of sodium\-cooled miniature heat sinks\. This combination yields a robust and lightweight framework for scientific ML under data scarcity, complementing and extending prior approaches\.

## 3Methodology

### 3\.1CFD Simulations

Computational Fluid Dynamics \(CFD\) simulations were conducted to generate a dataset of 87 data points\. These simulations follow the numerical framework described inPirayeshshirazinezhad \([2025](https://arxiv.org/html/2605.05217#bib.bib12)\)and were executed usingAnsys Fluent\. The objective was to obtain precise Nusselt numbers for both laminar and turbulent flow of liquid sodium in stainless steel \(SS\-316\) rectangular miniature heat sinks under varying physical conditions\.

The input parameters spanned a wide range, including heat sink width, aspect ratio, hydraulic diameter, and Peclet number of the sodium coolant\. The governing equations included the fundamental equations of incompressible, steady\-state flow: the continuity equation, the Navier–Stokes momentum equations, and the energy conservation equation\. Additionally, heat conduction within the solid substrate was modeled with a temperature\-dependent conductivity\. These equations are summarized as follows:

#### Continuity equation \(incompressible flow\):

∇⋅\(ρ​𝐮\)=0,\\nabla\\cdot\(\\rho\\mathbf\{u\}\)=0,\(1\)whereρ\\rhois the fluid density and𝐮\\mathbf\{u\}is the velocity vector\.

#### Navier–Stokes momentum equation:

∇⋅\(ρ​𝐮𝐮\)=−∇P\+∇⋅\(μ​\(∇𝐮\+∇T𝐮\)\)\+ρ​𝐠,\\nabla\\cdot\(\\rho\\mathbf\{u\}\\mathbf\{u\}\)=\-\\nabla P\+\\nabla\\cdot\\big\(\\mu\(\\nabla\\mathbf\{u\}\+\\nabla^\{T\}\\mathbf\{u\}\)\\big\)\+\\rho\\mathbf\{g\},\(2\)wherePPis pressure,μ\\muis dynamic viscosity, and𝐠\\mathbf\{g\}is the gravitational acceleration\.

#### Energy equation \(fluid\):

∇⋅\(ρ​cp​𝐮​T\)=∇⋅\(kf​∇T\),\\nabla\\cdot\(\\rho c\_\{p\}\\mathbf\{u\}T\)=\\nabla\\cdot\(k\_\{f\}\\nabla T\),\(3\)wherecpc\_\{p\}is the specific heat capacity,TTis the temperature field, andkfk\_\{f\}is the thermal conductivity of the fluid\.

#### Heat conduction equation \(solid substrate\):

∇⋅\(ks​∇T\)=0,\\nabla\\cdot\(k\_\{s\}\\nabla T\)=0,\(4\)whereksk\_\{s\}is the thermal conductivity of the solid material\.

The study employed steady\-state numerical simulations with a no\-slip boundary condition at the solid–fluid interfaces of the miniature heat sinks\. The coolant was introduced at uniform velocity and constant inlet temperature\.

### 3\.2Self\-Supervised Adaptive PINN Framework

The machine learning framework is based on physics\-informed neural networks \(PINNs\), where the governing partial differential equations \(PDEs\) of fluid flow and heat transfer are embedded as soft constraints in the training objective\. Letθ\\thetadenote the network parameters\. The standard PINN loss is a weighted sum of data\-driven and physics\-driven terms:

ℒ​\(θ\)=λd​ℒdata​\(θ\)\+λp​ℒphysics​\(θ\),\\mathcal\{L\}\(\\theta\)=\\lambda\_\{d\}\\,\\mathcal\{L\}\_\{\\text\{data\}\}\(\\theta\)\+\\lambda\_\{p\}\\,\\mathcal\{L\}\_\{\\text\{physics\}\}\(\\theta\),\(5\)whereℒdata\\mathcal\{L\}\_\{\\text\{data\}\}represents the discrepancy between network predictions and available CFD data, andℒphysics\\mathcal\{L\}\_\{\\text\{physics\}\}measures PDE residuals from Equations[1](https://arxiv.org/html/2605.05217#S3.E1)–[4](https://arxiv.org/html/2605.05217#S3.E4)\. The coefficientsλd\\lambda\_\{d\}andλp\\lambda\_\{p\}balance the contributions of the two terms\.

#### Adaptive blending neuron\.

Instead of fixingλd\\lambda\_\{d\}andλp\\lambda\_\{p\}manually, we introduce a learnable*blending neuron*that adaptively adjusts their relative contributions during training\. Specifically,

λd=σ​\(α\),λp=1−σ​\(α\),\\lambda\_\{d\}=\\sigma\(\\alpha\),\\qquad\\lambda\_\{p\}=1\-\\sigma\(\\alpha\),\(6\)whereσ​\(⋅\)\\sigma\(\\cdot\)is the sigmoid function andα\\alphais a trainable scalar parameter\. This formulation ensures0<λd,λp<10<\\lambda\_\{d\},\\lambda\_\{p\}<1and allows the model to automatically discover the optimal balance between physics and data supervision\. During training,α\\alphais updated by backpropagation along withθ\\theta, making the weighting self\-supervised\.

#### Data\-driven loss\.

The data loss is defined as the mean squared error \(MSE\) between predicted and CFD\-computed Nusselt numbers:

ℒdata​\(θ\)=1Nd​∑i=1Nd\(y^i​\(θ\)−yi\)2,\\mathcal\{L\}\_\{\\text\{data\}\}\(\\theta\)=\\frac\{1\}\{N\_\{d\}\}\\sum\_\{i=1\}^\{N\_\{d\}\}\\big\(\\hat\{y\}\_\{i\}\(\\theta\)\-y\_\{i\}\\big\)^\{2\},\(7\)whereyiy\_\{i\}are ground\-truth CFD values andy^i​\(θ\)\\hat\{y\}\_\{i\}\(\\theta\)are PINN predictions at the same inputs\.

#### Physics residual loss\.

The physics loss is constructed from PDE residuals evaluated at collocation points\{𝐱j\}j=1Np\\\{\\mathbf\{x\}\_\{j\}\\\}\_\{j=1\}^\{N\_\{p\}\}:

ℒphysics​\(θ\)=1Np​∑j=1Np\(ℛ​\(𝐱j;θ\)\)2,\\mathcal\{L\}\_\{\\text\{physics\}\}\(\\theta\)=\\frac\{1\}\{N\_\{p\}\}\\sum\_\{j=1\}^\{N\_\{p\}\}\\left\(\\mathcal\{R\}\(\\mathbf\{x\}\_\{j\};\\theta\)\\right\)^\{2\},\(8\)whereℛ\\mathcal\{R\}denotes the residual of the governing equations \(Equations[1](https://arxiv.org/html/2605.05217#S3.E1)–[4](https://arxiv.org/html/2605.05217#S3.E4)\) computed with PINN\-predicted velocity, pressure, and temperature fields\.

### 3\.3Transfer Learning for Data\-Scarce Regimes

To further enhance learning efficiency, we incorporate a transfer learning \(TL\) strategy\. A base PINN is first trained on a source dataset \(e\.g\., water\-cooled microchannels\), where larger training data are available\. The network parametersθ∗\\theta^\{\*\}from the source task are then used to initialize the target PINN for sodium\-cooled miniature heat sinks\. Specifically,

θtarget\(0\)←θsource∗\.\\theta\_\{\\text\{target\}\}^\{\(0\)\}\\leftarrow\\theta^\{\*\}\_\{\\text\{source\}\}\.\(9\)During fine\-tuning, only the last few layers and the blending neuron parameterα\\alphaare updated, while early layers retain transferable low\-level representations\. This approach reduces training time and improves convergence stability under extremely small target datasets \(87 CFD points\)\.

### 3\.4Training Procedure

The overall training process alternates between minimizingℒdata\\mathcal\{L\}\_\{\\text\{data\}\}andℒphysics\\mathcal\{L\}\_\{\\text\{physics\}\}, with weights governed by the adaptive blending neuron\. A schematic of the framework is shown in Figure[1](https://arxiv.org/html/2605.05217#S3.F1)\. The Adam optimizer was employed with learning rate scheduling, and early stopping was used to prevent overfitting\. Monte Carlo cross\-validationShan \([2022](https://arxiv.org/html/2605.05217#bib.bib15)\); Elmessiryet al\.\([2017](https://arxiv.org/html/2605.05217#bib.bib16)\)was applied to quantify generalization performance and statistical robustness\.

![Refer to caption](https://arxiv.org/html/2605.05217v1/PINN.png)Figure 1:Schematic of the proposed self\-supervised adaptive PINN with transfer learning\. The blending neuron learns to weight physics residuals and data loss\.

## 4Results

### 4\.1Gaussian Process and Support Vector Regression

Table[1](https://arxiv.org/html/2605.05217#S4.T1)compares Gaussian Process \(GP\) and Support Vector Regression \(SVR\) models using a radial basis function \(RBF\) kernel\. SVR consistently outperforms GP due to its ability to exploit data distribution more effectively and tune additional hyperparameters for extrapolation\. Among SVR models, randomized search \(RS\) optimization achieves a minimum mean absolute percentage error \(MAPE\) of2\.72%2\.72\\%after four iterations, while Bayesian optimization converges faster, reaching a lower MAPE of1\.25%1\.25\\%within 17 iterations\.

Table 1:Performance comparison of GP and SVR models using RBF kernel\.The corresponding SVR hyperparameters are reported in Table[2](https://arxiv.org/html/2605.05217#S4.T2), where Bayesian optimization selects a higher regularization parameter \(C=27\.23C=27\.23\) compared to RS \(C=10C=10\), further contributing to its superior generalization\.

Table 2:Optimal hyperparameters for SVR under RS and Bayesian optimization\.
### 4\.2Transfer Learning for Neural Networks

Transfer learning \(TL\) further improves neural network performance\. Table[3](https://arxiv.org/html/2605.05217#S4.T3)shows that transferring the first hidden layer from a water\-trained network reduces MAPE from0\.00280\.0028\(no transfer\) to0\.00200\.0020\. Genetic Algorithm \(GA\)\-based optimization selected an optimal architecture of two hidden layers, with three neurons transferred from the source network and eight randomly initialized neurons in the second layer\. Figure[2](https://arxiv.org/html/2605.05217#S4.F2)confirms that transferring earlier layers captures generalizable features, while transferring layers closer to the output degrades accuracy due to domain\-specific representations\.

Table 3:Impact of transfer learning on neural network performance\.![Refer to caption](https://arxiv.org/html/2605.05217v1/TR_Layer.png)Figure 2:Effect of transfer learning from different layers\. The first\-layer transfer achieves the lowest error, while transferring deeper layers closer to the output increases error\.The statistical difference between water and sodium datasets is illustrated in Figure[3](https://arxiv.org/html/2605.05217#S4.F3)\.

Kernel Density Estimates \(Figure[3](https://arxiv.org/html/2605.05217#S4.F3)\) confirm a broader variance for sodium data\. A Mann–Whitney U test rejects the null hypothesis \(p≪0\.05p\\ll 0\.05\), supporting the suitability of TL from water to sodium\.

![Refer to caption](https://arxiv.org/html/2605.05217v1/KDE_TR.png)Figure 3:KDE comparison of water and sodium Nusselt numbers\. Sodium exhibits higher variance than water, supporting transfer learning across domains\.
### 4\.3Self\-Supervised PINN Performance

Bayesian optimization determined the optimal architecture of the self\-supervised PINN to be two hidden layers of 20 neurons and one hidden layer of 12 neurons, with Adam optimizer at learning rate 0\.34\. The 10\-fold cross\-validated MAPE was0\.01850\.0185after 100 optimization iterations\. Figure[4](https://arxiv.org/html/2605.05217#S4.F4)shows the distribution of the physics coefficient neuronλp\\lambda\_\{p\}, centered around 0\.5, confirming balanced contributions of physics and data\.

![Refer to caption](https://arxiv.org/html/2605.05217v1/Dist.png)Figure 4:Distribution of the learned physics coefficient neuron in the self\-supervised PINN, centered near 0\.5\.
### 4\.4Benchmarking and Validation

Table[4](https://arxiv.org/html/2605.05217#S4.T4)reports benchmarking results across all methods\. SVR–Bayesian achieves the lowest error among classical ML methods \(MAPE = 0\.0125\), while the adaptive NN with transfer learning achieves the overall lowest error \(MAPE = 0\.0020\)\. The self\-supervised PINN achieves competitive performance \(MAPE = 0\.0185\) but demonstrates superior robustness\.

Table 4:MAPE comparison across ML methods for Nusselt number prediction\.Monte Carlo simulations \(500 trials\) were further used to analyze robustness \(Table[5](https://arxiv.org/html/2605.05217#S4.T5)\)\. While the PINN exhibited higher variance in MAPE during training, its prediction variance on the holdout dataset was lower than both baseline NN \(no transfer\) and kernel methods\. This indicates improved robustness to hyperparameter and initialization randomness\.

Table 5:Monte Carlo validation metrics across ML methods\. TR\_NN = transfer learning NN\.Figures[5](https://arxiv.org/html/2605.05217#S4.F5)–[6](https://arxiv.org/html/2605.05217#S4.F6)visualize predictions on holdout sets\. Kernel\-based methods achieve reasonable accuracy but tend to underfit, whereas NN\-based methods capture more variance\. The self\-supervised PINN remains consistently within the±8%\\pm 8\\%error margin, validating its robustness for Nusselt number prediction in sodium heat sinks\.

![Refer to caption](https://arxiv.org/html/2605.05217v1/GP.png)Figure 5:GP and SVR predictions on holdout dataset\. SVR–Bayesian achieves better fit than GP, though both remain within±8%\\pm 8\\%error margin\.![Refer to caption](https://arxiv.org/html/2605.05217v1/NN.png)Figure 6:Neural network predictions on holdout dataset\. The self\-supervised PINN provides the most robust estimations within±8%\\pm 8\\%margin\.

## 5Discussion

The results demonstrate that classical kernel\-based methods such as GP and SVR provide competitive baselines, with SVR–Bayesian achieving a MAPE of0\.01250\.0125\. However, their performance is limited by sensitivity to kernel choice and hyperparameter tuning\. In contrast, neural\-network approaches, particularly those incorporating transfer learning, achieve substantially lower errors \(0\.00200\.0020\) by reusing generalizable representations from water\-cooled datasets\. This highlights the potential of cross\-domain transfer in scientific ML, where related physical systems often share underlying structural features\.

The self\-supervised PINN achieves a higher MAPE \(0\.01850\.0185\) compared to the best NN and SVR models, yet exhibits superior robustness during Monte Carlo validation\. Specifically, it maintains lower prediction variance on holdout datasets despite larger variance during training\. This robustness arises from the adaptive blending neuron, which balances physics\-based and data\-driven supervision without requiring manual tuning\. The learned coefficient distribution \(centered around 0\.5\) confirms that the network adaptively exploits both sources of information\.

An important insight is that the adaptive PINN, although not the lowest in raw error, provides a more reliable and interpretable framework for deployment in data\-scarce regimes\. When CFD data availability is limited, the physics component stabilizes learning, reducing overfitting and improving generalization\. The trade\-off between raw accuracy and robustness is particularly relevant in safety\-critical applications such as thermal management of liquid\-metal systems, aerospace vehicles, and biomedical devices\.

## 6Conclusion

We introduced a self\-supervised PINN framework that adaptively balances physics residuals and data\-driven errors through a learnable blending neuron, and we combined this with transfer learning to enhance performance under extreme data scarcity\. Our main findings are:

- •SVR with Bayesian optimization provides strong baselines but requires extensive hyperparameter tuning\.
- •Neural networks benefit significantly from transfer learning, with first\-layer transfer achieving the lowest MAPE \(0\.00200\.0020\)\.
- •The self\-supervised PINN achieves competitive accuracy \(0\.01850\.0185MAPE\) while demonstrating superior robustness to hyperparameter and initialization randomness\.
- •Statistical analysis of Nusselt number distributions confirms the validity of transferring representations from water to sodium domains\.

Overall, our framework provides a robust approach toward small\-data scientific ML, offering both accuracy and reliability\. While demonstrated on sodium\-cooled miniature heat sinks, the approach generalizes to other domains where governing equations are available but data are limited\. Future work includes extending the blending mechanism to multi\-physics scenarios, incorporating uncertainty quantification, and scaling the framework to large\-scale 3D CFD simulations\.

## References

- A\. Baghban, M\. Bahadori, J\. Rozyn, M\. Lee, A\. Abbas, A\. Bahadori, and A\. Rahimali \(2016\)Estimation of air dew point temperature using computational intelligence schemes\.Applied Thermal Engineering93,pp\. 1043–1052\.Cited by:[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Baghban, M\. Kahani, M\.A\. Nazari, M\.H\. Ahmadi, and W\.\-M\. Yan \(2019\)Sensitivity analysis and application of machine learning methods to predict the heat transfer performance of cnt/water nanofluid flows through coils\.International Journal of Heat and Mass Transfer128,pp\. 825–835\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p3.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Bhattacharya, M\.K\. Verma, and A\. Bhattacharya \(2022\)Predictions of reynolds and nusselt numbers in turbulent convection using machine learning models\.Physics of Fluids34\(2\)\.Cited by:[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- J\. A\. D\. Cruz, S\. Biedron, M\. Martinez\-Ramon, S\. Sosa, and R\. Pirayesh \(2019\)Studies in applying machine learning to llrf and resonance control in superconducting rf cavities\.arXiv preprint arXiv:1910\.07648\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p1.1)\.
- J\. Diaz Cruz, S\. Biedron, R\. Pirayesh, and S\. Sosa \(2022\)Machine learning\-based tuning of control parameters for llrf system of superconducting cavities\.Technical reportSLAC National Accelerator Laboratory \(SLAC\), Menlo Park, CA \(United States\)\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p1.1)\.
- A\. Elmessiry, W\.O\. Cooper, T\.F\. Catron, J\. Karrass, Z\. Zhang, and M\.P\. Singh \(2017\)Triaging patient complaints: monte carlo cross\-validation of six machine learning classifiers\.JMIR Medical Informatics5\(3\),pp\. e7140\.Cited by:[§3\.4](https://arxiv.org/html/2605.05217#S3.SS4.p1.2)\.
- L\. Guastoni, A\. Güemes, A\. Ianiro, S\. Discetti, P\. Schlatter, H\. Azizpour, and R\. Vinuesa \(2021\)Convolutional\-network models to predict wall\-bounded turbulence from wall quantities\.Journal of Fluid Mechanics928,pp\. A27\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p1.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Jeon, J\. Lee, H\. Eivazi, R\. Vinuesa, and S\.J\. Kim \(2022a\)Physics\-informed transfer learning strategy to accelerate unsteady fluid flow simulations\.External Links:2206\.06817Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p3.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px4.p1.1)\.
- J\. Jeon, J\. Lee, and S\.J\. Kim \(2022b\)Finite volume method network for the acceleration of unsteady computational fluid dynamics: non\-reacting and reacting flows\.International Journal of Energy Research46\(8\),pp\. 10770–10795\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p3.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- H\. Kurt and M\. Kayfeci \(2009\)Prediction of thermal conductivity of ethylene glycol–water solutions by using artificial neural networks\.Applied Energy86\(10\),pp\. 2244–2248\.Cited by:[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Liu, K\. Chen, G\. Xu, X\. Sun, M\. Yan, W\. Diao, and H\. Han \(2019\)Convolutional neural network\-based transfer learning for optical aerial images change detection\.IEEE Geoscience and Remote Sensing Letters17\(1\),pp\. 127–131\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p3.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px3.p1.1)\.
- L\. McClenny and U\. Braga\-Neto \(2020\)Self\-adaptive physics\-informed neural networks\.External Links:2009\.04544Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p2.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px4.p1.1)\.
- R\. Pirayesh, S\. Biedron, J\. Diaz Cruz, M\. Martínez\-Ramón,et al\.\(2021\)Hierarchical intelligent real\-time optimal control for llrf using time series machine learning methods and transfer learning\.In12th International Particle Accelerator Conference \(IPAC’21\), Campinas, SP, Brazil, 24\-28 May 2021,pp\. 4329–4331\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p1.1)\.
- R\. Pirayeshshirazinezhad, S\. G\. Biedroń, J\. A\. D\. Cruz, S\. S\. Güitrón, and M\. Martinez\-Ramon \(2022\)Designing monte carlo simulation and an optimal machine learning to optimize and model space missions\.IEEe Access10,pp\. 45643–45662\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p1.1)\.
- R\. Pirayeshshirazinezhad and N\. Fathi \(2025\)Explainable ai\-enhanced supervisory control for robust multi\-agent robotic systems\.arXiv preprint arXiv:2509\.15491\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p1.1)\.
- R\. Pirayeshshirazinezhad \(2022\)Artificial intelligence, controls, and sensor fusion for optimization and modeling of space missions and particle accelerators\.Ph\.D\. Thesis,The University of New Mexico\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p1.1)\.
- R\. Pirayeshshirazinezhad \(2025\)SPINN: an optimal self\-supervised physics\-informed neural network framework\.arXiv preprint arXiv:2509\.05886\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p3.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2605.05217#S3.SS1.p1.1)\.
- G\. Shan \(2022\)Monte carlo cross\-validation for a study with binary outcome and limited sample size\.BMC Medical Informatics and Decision Making22\(1\),pp\. 1–15\.Cited by:[§3\.4](https://arxiv.org/html/2605.05217#S3.SS4.p1.2)\.
- P\. Sharma, W\. T\. Chung, B\. Akoush, and M\. Ihme \(2023\)A review of physics\-informed machine learning in fluid mechanics\.Energies16\(5\),pp\. 2343\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p1.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- M\.M\. Tafarroj, O\. Mahian, A\. Kasaeian, K\. Sakamatapan, A\.S\. Dalkilic, and S\. Wongwises \(2017\)Artificial neural network modeling of nanofluid flow in a microchannel heat sink using experimental data\.International Communications in Heat and Mass Transfer86,pp\. 25–31\.Cited by:[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Wang, Y\. Teng, and P\. Perdikaris \(2021\)When and why pinns fail to train: a neural tangent kernel perspective\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p2.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px4.p1.1)\.
- L\. Yang, X\. Meng, and G\. E\. Karniadakis \(2021\)B\-pinns: bayesian physics\-informed neural networks for forward and inverse pde problems with noisy data\.External Links:2102\.06754Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p2.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px4.p1.1)\.
- F\. Yousefi, H\. Karimi, and M\.M\. Papari \(2012\)Modeling viscosity of nanofluids using diffusional neural networks\.Journal of Molecular Liquids175,pp\. 85–90\.Cited by:[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px1.p1.1)\.
- F\. Zhuang, Z\. Qi, K\. Duan, D\. Xi, Y\. Zhu, H\. Zhu, H\. Xiong, and Q\. He \(2020\)A comprehensive survey on transfer learning\.Proceedings of the IEEE109\(1\),pp\. 43–76\.Cited by:[§1](https://arxiv.org/html/2605.05217#S1.p3.1),[§2](https://arxiv.org/html/2605.05217#S2.SS0.SSS0.Px3.p1.1)\.