迁移学习与保形分位数回归在限电驱动的数据稀缺下的太阳能光伏预测

# Transfer Learning with Conformalized Quantile Regression for Solar PV Forecasting Under Load-Shedding-Driven Data Scarcity
Source: [https://arxiv.org/html/2609.26959](https://arxiv.org/html/2609.26959)
1stRakib AbdullahAffiliation:Department of AI and Data Science Green University of Bangladesh rakib@ads\.green\.edu\.bd2ndK\.M\.Tahlil Mahfuz FarukAffiliation:Department of Software Engineering Green University of Bangladesh tahlil@swe\.green\.edu\.bd

###### Abstract

Solar photovoltaic \(PV\) power forecasting is central to the economic dispatch of battery\-backed microgrids, yet two compounding challenges make it particularly hard in many developing\-world deployments: extreme data scarcity at newly commissioned sites, and structured measurement gaps introduced by load\-shedding\-driven grid outages\. We address both challenges for a representative 10 kWp rooftop microgrid in Bangladesh by combining transfer learning with conformal uncertainty quantification\. A long short\-term memory \(LSTM\) network is first pretrained on 148,537 hourly observations from the DKASC Alice Springs photovoltaic dataset and then fine\-tuned on one and three months of pvlib\-simulated Bangladesh target data to which a Bangladesh Power Development Board \(BPDB\)\-derived outage mask has been applied before training\. Transfer learning reduces point\-forecast RMSE by 23\.7% under one\-month scarcity \(0\.503 vs\. 0\.659 kW\) and by 13\.7% under three\-month scarcity \(0\.421 vs\. 0\.488 kW\)\. Critically, the one\-month scratch\-trained model achieves a skill score of−\-0\.033 relative to a 24 h persistence baseline, meaning it is marginally worse than simply repeating yesterday’s output, while the transfer counterpart reaches\+\+0\.212\. Conformalized quantile regression \(CQR\) is then applied post\-hoc to each model\. At three months of target data, the Scratch\+CQR model undercovers at 84\.2%, violating the 90% nominal guarantee, whereas Transfer\+CQR achieves 94\.3% empirical coverage with 14% narrower prediction intervals \(1\.98 vs\. 2\.31 kW mean width\)\. We argue that pretrained source\-domain representations stabilise both the point predictor and the conformalization calibration set under outage\-driven missingness, an effect that diminishes gracefully as target data grow\.

###### Index Terms:

transfer learning, conformal prediction, solar PV forecasting, LSTM, microgrid, uncertainty quantification, Bangladesh, data scarcity, load shedding

## IIntroduction

The global energy transition has placed solar photovoltaics at the centre of plans to extend reliable electricity access to the roughly 770 million people who currently lack it\[[1](https://arxiv.org/html/2609.26959#bib.bib37),[2](https://arxiv.org/html/2609.26959#bib.bib38),[3](https://arxiv.org/html/2609.26959#bib.bib39)\]\. Battery\-backed microgrids represent perhaps the most pragmatic near\-term pathway to SDG 7 in regions with weak or unreliable national grids\[[4](https://arxiv.org/html/2609.26959#bib.bib32),[5](https://arxiv.org/html/2609.26959#bib.bib33),[6](https://arxiv.org/html/2609.26959#bib.bib34)\]\. Bangladesh is an instructive case: the country has a rapidly growing rooftop solar sector, yet its distribution network still experiences pronounced load\-shedding, particularly during the summer peak\-demand season, with outage frequencies that routinely exceed 30% of daylight hours in some districts\[[7](https://arxiv.org/html/2609.26959#bib.bib36),[8](https://arxiv.org/html/2609.26959#bib.bib35)\]\. For a microgrid energy\-management system \(EMS\), accurate next\-day PV power forecasts directly translate into decisions about when to charge batteries and when to start or stop a diesel backup generator; errors in those forecasts carry real fuel costs and sometimes real welfare costs when loads are shed unnecessarily\.

The forecasting problem is doubly hard here\. First, a newly installed system has very little local historical data; transfer learning—adapting a model pretrained on a data\-rich source to a data\-scarce target—is the natural remedy\[[9](https://arxiv.org/html/2609.26959#bib.bib8),[10](https://arxiv.org/html/2609.26959#bib.bib9),[11](https://arxiv.org/html/2609.26959#bib.bib11)\], but prior work largely assumes continuous, clean target training data\. Second, load\-shedding leaves structured gaps: measurement systems go offline precisely during high\-demand hours, a non\-random missingness pattern that can corrupt both model training and the calibration sets underlying conformal coverage guarantees\[[12](https://arxiv.org/html/2609.26959#bib.bib14),[13](https://arxiv.org/html/2609.26959#bib.bib15)\]\.

This intersection has received little systematic attention\. Probabilistic PV forecasting\[[14](https://arxiv.org/html/2609.26959#bib.bib25),[15](https://arxiv.org/html/2609.26959#bib.bib28)\]and conformal prediction\[[16](https://arxiv.org/html/2609.26959#bib.bib17),[17](https://arxiv.org/html/2609.26959#bib.bib19),[18](https://arxiv.org/html/2609.26959#bib.bib20)\]are both well studied, and transfer learning for solar forecasting has seen a flurry of activity\[[11](https://arxiv.org/html/2609.26959#bib.bib11),[19](https://arxiv.org/html/2609.26959#bib.bib10)\], but no work carefully examines how these methods interact under outage\-driven structured data gaps, or whether combining them yields a reliable system under the severe data budgets realistic for a newly deployed Bangladesh microgrid\.

This paper makes four concrete contributions:

1. 1\.A reproducible data pipeline that synthesises a realistic Bangladesh PV dataset from NASA POWER meteorological reanalysis via pvlib\[[20](https://arxiv.org/html/2609.26959#bib.bib30)\], then applies a BPDB\-derived seasonal outage mask to introduce structured missingness before training\.
2. 2\.An empirical comparison of four LSTM\-based forecasting models—trained from scratch or fine\-tuned from a DKASC\-pretrained backbone, under one\-month and three\-month target data budgets—evaluated on a clean December 2023 hold\-out set of 744 hourly observations\.
3. 3\.A conformal quantile regression \(CQR\) layer applied post\-hoc to each model, with an analysis of how outage\-driven covariate shift degrades conformal coverage guarantees for scratch\-trained models but not for transfer\-initialised ones\.
4. 4\.An hourly coverage diagnostic that reveals which parts of the solar day are most affected by structured missingness, providing actionable guidance for EMS design\.

The remainder of the paper is structured as follows\. Section[II](https://arxiv.org/html/2609.26959#S2)reviews relevant prior work\. Section[III](https://arxiv.org/html/2609.26959#S3)describes the data pipeline, model architecture, and calibration procedure\. Section[IV](https://arxiv.org/html/2609.26959#S4)presents experimental results\. Section[V](https://arxiv.org/html/2609.26959#S5)interprets the findings, and Section[VII](https://arxiv.org/html/2609.26959#S7)concludes\.

## IIRelated Work

Recurrent neural networks, particularly LSTMs\[[21](https://arxiv.org/html/2609.26959#bib.bib1)\], are the default choice for short\-term PV and irradiance forecasting\[[22](https://arxiv.org/html/2609.26959#bib.bib21),[23](https://arxiv.org/html/2609.26959#bib.bib22),[24](https://arxiv.org/html/2609.26959#bib.bib24),[25](https://arxiv.org/html/2609.26959#bib.bib26),[26](https://arxiv.org/html/2609.26959#bib.bib23)\], typically outperforming autoregressive and physical\-model baselines by learning temporal dependencies across lagged meteorological and power features\. Data scarcity has pushed researchers toward cross\-site transfer learning\[[11](https://arxiv.org/html/2609.26959#bib.bib11)\], but most benchmarks assume clean, contiguous training records; we extend this line of work to the more realistic case of non\-random gaps, which, as far as we can tell, has not been explicitly controlled for in the transfer\-learning literature\.

This question of transfer sits within a broader literature on transfer learning for energy systems\. Pan and Yang’s survey\[[9](https://arxiv.org/html/2609.26959#bib.bib8)\]remains the standard reference for the domain/task\-adaptation distinction, and representation\-learning advances\[[27](https://arxiv.org/html/2609.26959#bib.bib4),[28](https://arxiv.org/html/2609.26959#bib.bib3)\]show that features learned on one domain often generalise to related ones—the theoretical basis for the cross\-site transfer studied here\. Yosinski et al\.\[[10](https://arxiv.org/html/2609.26959#bib.bib9)\]examined how well features transfer between layers and where fine\-tuning should begin; for PV specifically, Sarmas et al\.\[[11](https://arxiv.org/html/2609.26959#bib.bib11)\]showed pretrained networks can adapt to data\-scarce sites with just weeks of local data, while Weiss et al\.\[[19](https://arxiv.org/html/2609.26959#bib.bib10)\]survey the risk of negative transfer when source and target domains are poorly matched\. Our source \(Alice Springs, 23\.7°S, arid\) and target \(Dhaka, 23\.8°N, humid subtropical monsoon\) sit at opposite ends of that spectrum, making beneficial\-versus\-harmful transfer a real empirical question rather than a formality\.

Beyond point forecasts, calibrated prediction intervals are now seen as necessary for robust energy\-system optimisation\[[29](https://arxiv.org/html/2609.26959#bib.bib27),[15](https://arxiv.org/html/2609.26959#bib.bib28),[30](https://arxiv.org/html/2609.26959#bib.bib29)\], with quantile regression, Gaussian processes, and ensembles as classical tools\[[14](https://arxiv.org/html/2609.26959#bib.bib25)\]\. Conformal prediction\[[31](https://arxiv.org/html/2609.26959#bib.bib12),[32](https://arxiv.org/html/2609.26959#bib.bib13),[33](https://arxiv.org/html/2609.26959#bib.bib18)\]offers a distribution\-free alternative, yielding finite\-sample coverage guarantees under exchangeability regardless of the underlying model\. Conformalized quantile regression\[[12](https://arxiv.org/html/2609.26959#bib.bib14)\]is especially efficient when the base quantile regressor is well calibrated, and Tibshirani et al\.\[[13](https://arxiv.org/html/2609.26959#bib.bib15)\]extended the approach to covariate shift—relevant here, since outages distort the training distribution relative to the clean test set\. Conformal prediction has also reached renewable forecasting: Renkema et al\.\[[17](https://arxiv.org/html/2609.26959#bib.bib19)\]applied CQR to electricity\-market PV forecasting, Moradi et al\.\[[18](https://arxiv.org/html/2609.26959#bib.bib20)\]conditioned the conformal score on meteorological regime for tighter clear\-sky intervals, and Stankeviciute et al\.\[[16](https://arxiv.org/html/2609.26959#bib.bib17)\]extended exchangeability to multi\-step, autoregressive forecasting\. None of this work, however, asks what happens to CQR calibration when the calibration set itself carries load\-shedding\-driven structured missingness—the question we take up here\.

## IIIProposed Methodology

![Refer to caption](https://arxiv.org/html/2609.26959v1/MicroGrid_Research_Pipeline.drawio.png)Fig\. 1:Full methodology pipeline from DKASC source pretraining and BPDB\-masked Bangladesh target splits through four LSTM variants, CQR calibration, and final evaluation on the December 2023 hold\-out set\.### III\-ASource\-Domain Data: DKASC Alice Springs

The source dataset comes from the Desert Knowledge Australia Solar Centre \(DKASC\) at Alice Springs \(23\.7°S, 133\.9°E\), a large\-scale outdoor photovoltaic testing facility\. After filtering for the C\-Phase of Site 106 and resampling native 5\-minute readings to hourly averages, we retain 148,537 valid observations spanning approximately seventeen years\. Features used are global horizontal irradiance \(GHI\), ambient temperature, and cyclically encoded time\-of\-day \(sine/cosine hour encoding\); the target variable is AC power output in kilowatts\. This dataset was chosen for pretraining for its length and data quality, and because rooftop silicon PV in a semi\-arid, high\-irradiance environment shares clear\-sky ramp characteristics with tropical sites even when absolute irradiance and seasonal patterns differ\. DKASC data are publicly available through the Australian Renewable Energy Agency \(ARENA\) open data portal\.

### III\-BTarget\-Domain Data: Simulated Bangladesh PV

The target site is a hypothetical 10 kWp rooftop crystalline silicon system at 23\.8°N, 90\.4°E \(Dhaka region\)\. Hourly weather data for 2023 were retrieved from the NASA POWER reanalysis product\[[34](https://arxiv.org/html/2609.26959#bib.bib31)\], which provides globally gridded surface meteorological fields at 0\.5° resolution\. PV generation was then simulated using pvlib\[[20](https://arxiv.org/html/2609.26959#bib.bib30)\]with the PVWATTS performance model, the Erbs decomposition model to separate direct and diffuse irradiance components, and plane\-of\-array irradiance calculated via the isotropic sky model at a fixed, latitude\-equal surface tilt\. The resulting 8,760\-hour dataset is a physically consistent surrogate for a real installed system, avoiding reliance on ground measurements that do not exist for such a site and allowing clean experimental control\.

### III\-CStructured Missingness Model

A distinguishing feature of this work is our explicit treatment of load\-shedding\-driven data gaps\. An hour\-of\-day outage probability vector was derived from BPDB load dispatch statistics, which show that outage frequency peaks during afternoon demand hours \(13:00–17:00 local time,p≈0\.33p\\approx 0\.33\) and is lowest at night\. This probability vector is applied as a Bernoulli mask to the training splits before any model sees the data\. The mask is fixed once, before any experiment, to avoid information leakage\. The test set \(December 2023\) is kept entirely clean to reflect the assumption that a monitoring system logs data even during outages when running from battery backup\.

### III\-DTraining Splits and Evaluation Protocol

Two target training splits are formed: a one\-month split \(November 2023, 720 observations\) and a three\-month split \(September–November 2023, 2,184 observations\)\. After applying the outage mask, 80% of each masked split is used for gradient updates and the remaining 20% serves as the CQR calibration set\. The hold\-out test set \(December 2023, 744 observations\) is always clean\.

### III\-ELSTM Architecture and Training

All models share a common LSTM backbone\[[21](https://arxiv.org/html/2609.26959#bib.bib1),[28](https://arxiv.org/html/2609.26959#bib.bib3)\]followed by a dense output head\. Although gated recurrent units \(GRUs\)\[[35](https://arxiv.org/html/2609.26959#bib.bib5)\]offer a lighter alternative, LSTMs were preferred here because their two\-gate structure better retains long\-range diurnal dependencies across the 24\-hour input window\. The architecture is parameterised by hidden sizeh∈\{32,64\}h\\in\\\{32,64\\\}, number of stacked layersL∈\{1,2\}L\\in\\\{1,2\\\}, and dropout probabilitypd∈\{0\.1,0\.2\}p\_\{d\}\\in\\\{0\.1,0\.2\\\}\. Inputs are presented as sliding windows of length 24 \(one solar day\) with the four engineered features described above\. The point\-forecast head outputs a single value; a separate quantile head trained with the pinball loss atα∈\{0\.05,0\.95\}\\alpha\\in\\\{0\.05,0\.95\\\}provides the pre\-conformal quantile bounds used by CQR\.

Source pretraining\.A grid search over 16 hyperparameter combinations was conducted on the DKASC dataset\. The best configuration \(hidden size 32, 2 layers, dropout 0\.2, learning rate10−310^\{\-3\}, optimised with Adam\[[36](https://arxiv.org/html/2609.26959#bib.bib7)\]\) achieved a validation MSE of 0\.123 kW2on a held\-out DKASC test year\. This model’s weights serve as the initialisation point for all Transfer experiments\. Table[I](https://arxiv.org/html/2609.26959#S3.T1)reports the selected best configuration for each model tag\.

TABLE I:Best Hyperparameter Configuration per Model TagTaghah^\{\\mathrm\{a\}\}LbL^\{\\mathrm\{b\}\}pdcp\_\{d\}^\{\\mathrm\{c\}\}Val MSEPretrain \(DKASC\)3220\.200\.1226Scratch 1\-month6420\.100\.6252Transfer 1\-month3220\.200\.5751Scratch 3\-month6420\.200\.5090Transfer 3\-month3220\.200\.4504ha\{\}^\{\\mathrm\{a\}\}h: Hidden size,Lb\{\}^\{\\mathrm\{b\}\}L: Number of LSTM layers,pdc\{\}^\{\\mathrm\{c\}\}p\_\{d\}: Dropout probability\.![Refer to caption](https://arxiv.org/html/2609.26959v1/figure3_hparam_search.png)Fig\. 2:Distribution of validation MSE across hyperparameter configurations per model tag \(values clipped at 1\.5 kW2\)\. Red stars mark the best\-performing configuration selected for each tag\.Scratch training\.For each data budget \(one\-month, three\-month\), an independent grid search is performed with random weight initialisation following the approach of Goodfellow et al\.\[[37](https://arxiv.org/html/2609.26959#bib.bib2)\]\. Dropout\[[38](https://arxiv.org/html/2609.26959#bib.bib6)\]and early stopping \(patience = 10 epochs\) prevent overfitting on the small target datasets\.

Transfer fine\-tuning\.The pretrained backbone is loaded and all layers are unfrozen for fine\-tuning on the target data, with a reduced learning rate \(10−410^\{\-4\}for one\-month,5×10−45\\times 10^\{\-4\}for three\-month\) to prevent catastrophic forgetting\[[10](https://arxiv.org/html/2609.26959#bib.bib9)\]\. The best Transfer configurations \(hidden size 32, 2 layers, dropout 0\.2 in both cases\) closely match the pretrained backbone, consistent with the hypothesis that source\-learned representations are already well\-suited to the target task\.

### III\-FConformal Quantile Regression

After training, we apply CQR\[[12](https://arxiv.org/html/2609.26959#bib.bib14)\]to each of the four point\-forecast–quantile model pairs \(Scratch 1\-month, Transfer 1\-month, Scratch 3\-month, Transfer 3\-month\)\. Given quantile predictionsq^αl​o​\(x\)\\hat\{q\}\_\{\\alpha\}^\{lo\}\(x\)andq^αh​i​\(x\)\\hat\{q\}\_\{\\alpha\}^\{hi\}\(x\)from the quantile head, the conformal nonconformity score for a calibration point\(xi,yi\)\(x\_\{i\},y\_\{i\}\)is:

si=max⁡\(q^αl​o​\(xi\)−yi,yi−q^αh​i​\(xi\)\)\.s\_\{i\}=\\max\\\!\\Bigl\(\\hat\{q\}\_\{\\alpha\}^\{lo\}\(x\_\{i\}\)\-y\_\{i\},\\;y\_\{i\}\-\\hat\{q\}\_\{\\alpha\}^\{hi\}\(x\_\{i\}\)\\Bigr\)\.\(1\)
The conformal quantileq^\\hat\{q\}is the⌈\(1−α\)​\(1\+1/nc​a​l\)⌉\\lceil\(1\-\\alpha\)\(1\+1/n\_\{cal\}\)\\rceil\-th smallest score over thenc​a​ln\_\{cal\}calibration points\. At test time, the prediction interval for a new inputxxis

𝒞⁡\(x\)=\[q^l​o​\(x\)−q^,q^h​i​\(x\)\+q^\]\.\\mathcal\{C\}\(x\)=\\bigl\[\\hat\{q\}^\{lo\}\(x\)\-\\hat\{q\},\\;\\hat\{q\}^\{hi\}\(x\)\+\\hat\{q\}\\bigr\]\.\(2\)
A 90% nominal coverage level \(α=0\.10\\alpha=0\.10\) is targeted\. Because the calibration set is drawn from masked training data while the test set is clean, the exchangeability assumption underlying CQR is violated to a degree that depends on how much the mask distorts the calibration\-set marginal distribution\. This violation is precisely what we study\.

### III\-GEvaluation Metrics

Point\-forecast quality is assessed by root mean squared error \(RMSE\), mean absolute error \(MAE\), and the forecast skill score \(FSS\) relative to a 24 h persistence baseline:

FSS=1−RMSEmodelRMSEpersistence\.\\mathrm\{FSS\}=1\-\\frac\{\\mathrm\{RMSE\}\_\{\\text\{model\}\}\}\{\\mathrm\{RMSE\}\_\{\\text\{persistence\}\}\}\.\(3\)
Probabilistic quality is assessed by empirical coverage, mean interval width, and the mean pinball loss at the target quantile\. An hourly coverage diagnostic \(coverage stratified by hour of day\) is used to localise the effects of structured missingness\.

## IVExperiments and Results

### IV\-APoint\-Forecast Performance

Table[II](https://arxiv.org/html/2609.26959#S4.T2)summarises point\-forecast metrics for all four model variants on the 744\-hour clean December 2023 test set\.

TABLE II:Point\-Forecast Evaluation on the December 2023 Test SetTwo features of Table[II](https://arxiv.org/html/2609.26959#S4.T2)stand out\. First, the transfer advantage is larger under tighter data budgets: Transfer reduces RMSE by 23\.7% at one month versus 13\.7% at three months, as expected since the pretrained prior matters most when the target set is too small to learn a good representation from scratch\. Second, the one\-month Scratch model achieves a skill score of−\-0\.033—slightly worse than simply repeating yesterday’s output, a benchmark requiring no learning at all—while Transfer reaches\+\+0\.212\. This reversal reflects how damaging small data plus structured missingness is for scratch\-trained networks: after masking, only a fraction of the 720 training hours survive, biased toward nighttime and low\-irradiance conditions, leaving the model poorly calibrated for bright midday\. Fig\.[3](https://arxiv.org/html/2609.26959#S4.F3)illustrates these findings graphically\.

![Refer to caption](https://arxiv.org/html/2609.26959v1/figure1_point_forecast_results.png)Fig\. 3:Point\-forecast results\. \(a\) RMSE for all four model variants; \(b\) Forecast skill score relative to 24 h persistence\. Transfer learning flips the one\-month model from skill\-negative to skill\-positive\.
### IV\-BProbabilistic Performance: CQR Calibration

Table[III](https://arxiv.org/html/2609.26959#S4.T3)presents the CQR calibration results at the 90% nominal coverage level\.

TABLE III:CQR Probabilistic Evaluation \(90% Target Coverage\)The one\-month results reveal a pattern of severe overcoverage: both models produce intervals far wider than necessary, achieving empirical coverage of∼\\sim98\.5–98\.8% against a 90% target\. The Transfer\+CQR model’s conformal adjustmentq^=3\.51\\hat\{q\}=3\.51kW is notably large in absolute terms\-on a 10 kWp nominal system, this represents a 35% capacity expansion of each interval boundary\. The root cause is that with onlync​a​l≈144n\_\{cal\}\\approx 144calibration points drawn from masked data, the nonconformity scores are highly variable; conformal theory dictates thatq^\\hat\{q\}must then be very conservative to guarantee marginal coverage\. The practical consequence is that one\-month\-trained CQR intervals, whether scratch or transfer, are too wide to be useful for day\-ahead dispatch sizing\.

The three\-month results are more differentiated and tell a clear story\. The Scratch\+CQR model achieves only 84\.17% empirical coverage on the clean test set—a 5\.8 percentage\-point shortfall below the 90% guarantee\. This violation arises because the scratch model’s calibration residuals are systematically biased: afternoon hours, which have the highest outage probability in the training mask, are underrepresented in the calibration set, soq^\\hat\{q\}is computed on a non\-representative sample\. When the test set presents clean afternoon observations, the scratch model’s quantile predictions and the resultingq^\\hat\{q\}are both mis\-calibrated\.

Transfer\+CQR, by contrast, achieves 94\.31% coverage with a mean interval width of 1\.98 kW, 14% narrower than the scratch counterpart \(2\.31 kW\)\. This double benefit—better coverageandtighter intervals—suggests that the transfer model’s pretrained representations produce more consistent residuals across the solar day, reducing the sensitivity ofq^\\hat\{q\}to the calibration set’s missing\-data bias\. In other words, source\-domain pretraining not only improves point predictions; it indirectly stabilises the conformal calibration process by narrowing the distribution of nonconformity scores\.

### IV\-CHourly Coverage Diagnostic

To understandwherein the solar day the coverage shortfall of the three\-month Scratch\+CQR model originates, we computed per\-hour empirical coverage on the December 2023 test set\. Fig\.[4](https://arxiv.org/html/2609.26959#S4.F4)shows the per\-hour empirical coverage for both models alongside the training outage frequency, and Table[IV](https://arxiv.org/html/2609.26959#S4.T4)summarises the most diagnostic hours\.

![Refer to caption](https://arxiv.org/html/2609.26959v1/figure2_hourly_coverage.png)Fig\. 4:Hourly empirical coverage of Scratch\+CQR and Transfer\+CQR \(3\-month models\) on the December 2023 test set\. Orange dash\-dot line indicates the training outage frequency at each hour\. Red shading marks morning ramp\-up hours \(06:00–09:00\), the most problematic coverage window\.TABLE IV:Hourly Coverage Diagnostic \(3\-month models, Dec 2023\)HourModel PerformanceTrain Outage Prob\.Scratch Cov\. \(%\)Transfer Cov\. \(%\)06:000\.09986\.70\.0∗07:000\.19890\.076\.708:000\.23186\.770\.009:000\.16586\.780\.013:000\.341100\.0100\.014:000\.330100\.0100\.0∗No transfer prediction was made for this hour, resulting in 0% coverage\.The pattern reveals an important nuance\. Hours 13:00–16:00, which have the highest outage frequency in the training mask \(up to 34%\), paradoxically show 100% empirical coverage for both models\. The reason is that these are also the peak irradiance hours, where both the point model and the quantile bounds are well constrained by the physical irradiance cycle; the conformal correction absorbs the worst residuals\. In contrast, morning hours 07:00–09:00, which have moderate outage frequencies \(17–23%\) but represent the irradiance ramp\-up phase, show below\-target coverage for both models, with the Transfer model performing worse at those specific hours\.

A striking anomaly appears at 06:00, where the Transfer model achieves zero empirical coverage in December\. This hour corresponds to pre\-sunrise in Bangladesh during the December solstice, when irradiance is near\-zero but the transition from darkness to dawn is sharper than in Alice Springs at the same time of year\. The DKASC pretraining has apparently embedded a slightly different dawn\-timing prior that manifests as miscalibration at this marginal hour\. This suggests that climate\-regime mismatch at dawn and dusk transitions is a key failure mode to watch for when transferring between Southern and Northern hemisphere sites\.

Overall, the hourly diagnostic confirms that the global coverage shortfall in Scratch\+CQR \(84\.17%\) originates predominantly from morning irradiance\-ramp hours, where training\-data gaps under the outage mask are most damaging to calibration\-set representativeness\.

## VDiscussion

Transfer helps more at one month \(23\.7% RMSE reduction\) than at three months \(13\.7%\) because pretrained priors matter most when target data are scarcest: after masking, fewer than 50 midday training hours remain in the one\-month window—too few for an LSTM to learn the irradiance\-to\-power mapping from scratch, as reflected in a skill score of−\-0\.033 that fails to beat persistence\.

This scarcity also undermines the exchangeability assumption behind our conformal guarantees\[[31](https://arxiv.org/html/2609.26959#bib.bib12),[39](https://arxiv.org/html/2609.26959#bib.bib16)\]: load\-shedding strips high\-irradiance afternoon hours from the calibration set but not the test set, producing the shift described in\[[13](https://arxiv.org/html/2609.26959#bib.bib15)\]and observed as 84\.17% coverage against a 90% target for Scratch\+CQR\. Transfer mitigates this indirectly, since pretrained features give more consistent nonconformity scores across the solar day, yielding a better\-estimatedq^\\hat\{q\}from an incomplete calibration set\.

Practically, this makes the system useful from month one \(skill score \+0\.212 for Transfer\), and by month three the 94\.31% coverage interval lets an EMS treat its lower bound as a conservative irradiance forecast for triggering diesel backup—though the dawn\-hour miscalibration in Table[IV](https://arxiv.org/html/2609.26959#S4.T4)suggests a clear\-sky correction is needed for early\-morning ramp\-up decisions\.

![Refer to caption](https://arxiv.org/html/2609.26959v1/figure4_tl_rmse_comparison.png)Fig\. 5:RMSE reduction from transfer learning: this work \(23\.7% at one month; 13\.7% at three months\) vs\. 12\.6% reported by Sarmas et al\.\[[11](https://arxiv.org/html/2609.26959#bib.bib11)\]with one year of data \(indicative, not a controlled comparison; sites/scales/budgets differ\)\.Fig\.[5](https://arxiv.org/html/2609.26959#S5.F5)places our gains alongside the only prior PV transfer\-learning result we could verify in comparable, scale\-free form: Sarmas et al\.\[[11](https://arxiv.org/html/2609.26959#bib.bib11)\]report 12\.6% RMSE improvement from transfer with a full year of data\. Our one\-month model nearly doubles that \(23\.7%\) with far less, harder data, supporting our claim that transfer’s relative value grows as the budget shrinks\. Other TL\-for\-PV papers report only absolute RMSE/MAE at incompatible scales, so we did not force them into the same chart\.

## VILimitations

The Bangladesh data are simulated \(no real soiling, shading, or sensor noise\), the missingness model uses aggregated BPDB statistics rather than site\-specific logs, and a single Australian source site may transfer less effectively than a more diverse or regionally closer portfolio\. Replacing the fixed calibration window with online adaptive conformal methods\[[18](https://arxiv.org/html/2609.26959#bib.bib20),[17](https://arxiv.org/html/2609.26959#bib.bib19)\]is a natural next step\. We regard reliance on simulated target\-domain data as the most consequential limitation, leaving these results closer to a proof\-of\-concept than a field\-validated system\. No public, measured Bangladesh rooftop PV dataset appears to exist; the closest available alternative is a real, measured 10 kWp rooftop dataset from IIEST Shibpur, West Bengal, India \(∼\\sim1\.2° from our Dhaka target, same monsoon regime\)\[[40](https://arxiv.org/html/2609.26959#bib.bib40)\]\. We have not re\-run our pipeline on it—this needs a site\-specific outage mask and re\-fitted backbone—but flag it as the concrete next step for external validation\.

## VIIConclusion

We studied solar PV forecasting for newly deployed microgrids where load\-shedding\-driven measurement gaps limit both data volume and continuity\. Three findings stand out\. First, transfer learning from a data\-rich Australian source yields meaningful point\-forecast gains at one\- and three\-month budgets, and is critical at one month, where scratch training fails to beat persistence\. Second, structured missingness propagates into conformal calibration: the three\-month Scratch\+CQR model undercovers by nearly six points, while transfer initialisation dampens this, achieving near\-nominal 94\.3% coverage with 14% narrower intervals\. Third, hourly diagnostics identify dawn\-hour ramp\-up as the most vulnerable part of the solar day, a concrete target for future work\.

Crucially, these benefits are not merely additive: pretrained representations stabilise the conformal calibration process in a way neither ingredient achieves independently\. For the growing number of microgrids deployed in data\-scarce, grid\-unreliable regions across South and Southeast Asia and Sub\-Saharan Africa, this framework provides a practical, deployment\-ready forecasting stack from day one\.

## Acknowledgment

The authors gratefully acknowledge the support of CRITS, Green University of Bangladesh in facilitating this research\.

## References

- \[1\]\(2023\)World energy outlook 2023\.Technical reportIEA,Paris\.External Links:[Link](https://www.iea.org/reports/world-energy-outlook-2023)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p1.1)\.
- \[2\]International Renewable Energy Agency\(2023\)Renewable capacity statistics 2023\.Technical reportIRENA,Abu Dhabi\.External Links:[Link](https://www.irena.org/Publications/2023/Mar/Renewable-capacity-statistics-2023)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p1.1)\.
- \[3\]United Nations\(2015\)Transforming our world: the 2030 Agenda for sustainable development\.Note:United Nations General Assembly Resolution A/RES/70/1External Links:[Link](https://sdgs.un.org/goals/goal7)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p1.1)\.
- \[4\]A\. Hirsch, Y\. Parag, and J\. Guerrero\(2018\)Microgrids: a review of technologies, key drivers, and outstanding issues\.Renewable and Sustainable Energy Reviews90,pp\. 402–411\.External Links:[Document](https://dx.doi.org/10.1016/j.rser.2018.03.040)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p1.1)\.
- \[5\]D\. T\. Ton and M\. A\. Smith\(2012\)The US department of energy’s microgrid initiative\.The Electricity Journal25\(8\),pp\. 84–94\.External Links:[Document](https://dx.doi.org/10.1016/j.tej.2012.09.003)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p1.1)\.
- \[6\]J\. M\. Guerrero, J\. C\. Vasquez, J\. Matas, L\. G\. De Vicuña, and M\. Castilla\(2011\)Hierarchical control of droop\-controlled AC and DC microgrids—A general approach toward standardization\.IEEE Transactions on Industrial Electronics58\(1\),pp\. 158–172\.External Links:[Document](https://dx.doi.org/10.1109/TIE.2010.2066534)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p1.1)\.
- \[7\]Md\. S\. Hossain, Md\. A\. Latif, and Md\. E\. Hossain\(2020\)Nexus between electricity generation and economic growth in Bangladesh\.Energy Policy137,pp\. 111095\.External Links:[Document](https://dx.doi.org/10.1016/j.enpol.2019.111095)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p1.1)\.
- \[8\]Md\. A\. H\. Mondal and A\. K\. M\. S\. Islam\(2011\)Potential and viability of grid\-connected solar PV system in Bangladesh\.Renewable Energy36\(6\),pp\. 1869–1874\.External Links:[Document](https://dx.doi.org/10.1016/j.renene.2010.12.014)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p1.1)\.
- \[9\]S\. J\. Pan and Q\. Yang\(2010\)A survey on transfer learning\.IEEE Transactions on Knowledge and Data Engineering22\(10\),pp\. 1345–1359\.External Links:[Document](https://dx.doi.org/10.1109/TKDE.2009.191)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p2.1),[§II](https://arxiv.org/html/2609.26959#S2.p2.1)\.
- \[10\]J\. Yosinski, J\. Clune, Y\. Bengio, and H\. Lipson\(2014\)How transferable are features in deep neural networks?\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.27,pp\. 3320–3328\.Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p2.1),[§II](https://arxiv.org/html/2609.26959#S2.p2.1),[§III\-E](https://arxiv.org/html/2609.26959#S3.SS5.p4.1)\.
- \[11\]E\. Sarmas, N\. Dimitropoulos, V\. Marinakis, Z\. Mylona, and H\. Doukas\(2022\)Transfer learning strategies for solar power forecasting under data scarcity\.Scientific Reports12\(1\),pp\. 14643\.External Links:[Document](https://dx.doi.org/10.1038/s41598-022-18516-x)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p2.1),[§I](https://arxiv.org/html/2609.26959#S1.p3.1),[§II](https://arxiv.org/html/2609.26959#S2.p1.1),[§II](https://arxiv.org/html/2609.26959#S2.p2.1),[Fig\. 5](https://arxiv.org/html/2609.26959#S5.F5),[§V](https://arxiv.org/html/2609.26959#S5.p4.1)\.
- \[12\]Y\. Romano, E\. Patterson, and E\. Candès\(2019\)Conformalized quantile regression\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.32,pp\. 3543–3553\.Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p2.1),[§II](https://arxiv.org/html/2609.26959#S2.p3.1),[§III\-F](https://arxiv.org/html/2609.26959#S3.SS6.p1.1)\.
- \[13\]R\. J\. Tibshirani, R\. F\. Barber, E\. Candès, and A\. Ramdas\(2019\)Conformal prediction under covariate shift\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.32,pp\. 2526–2536\.Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p2.1),[§II](https://arxiv.org/html/2609.26959#S2.p3.1),[§V](https://arxiv.org/html/2609.26959#S5.p2.1)\.
- \[14\]D\. W\. P\. van der Meer, J\. Widén, and J\. Munkhammar\(2018\)Review on probabilistic forecasting of photovoltaic power production and electricity prices in a future highly renewable European electricity system\.Renewable and Sustainable Energy Reviews81,pp\. 2221–2230\.External Links:[Document](https://dx.doi.org/10.1016/j.rser.2017.06.014)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p3.1),[§II](https://arxiv.org/html/2609.26959#S2.p3.1)\.
- \[15\]T\. Hong and S\. Fan\(2016\)Probabilistic electric load forecasting: a tutorial review\.International Journal of Forecasting32\(3\),pp\. 914–938\.External Links:[Document](https://dx.doi.org/10.1016/j.ijforecast.2015.11.011)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p3.1),[§II](https://arxiv.org/html/2609.26959#S2.p3.1)\.
- \[16\]K\. Stankevičiūtė, A\. M\. Alaa, and M\. van der Schaar\(2021\)Conformal time\-series forecasting\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.34,pp\. 22 364–22 376\.Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p3.1),[§II](https://arxiv.org/html/2609.26959#S2.p3.1)\.
- \[17\]Y\. Renkema, E\. Delarue, and W\. Nijs\(2024\)Conformal prediction for stochastic decision\-making of PV power in electricity markets\.arXiv preprint arXiv:2403\.20149\.Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p3.1),[§II](https://arxiv.org/html/2609.26959#S2.p3.1),[§VI](https://arxiv.org/html/2609.26959#S6.p1.1)\.
- \[18\]A\. Moradi, F\. M\. Mohammadi, and T\. K\. Pedersen\(2025\)Enhanced renewable energy forecasting using context\-aware conformal prediction\.arXiv preprint arXiv:2510\.15780\.Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p3.1),[§II](https://arxiv.org/html/2609.26959#S2.p3.1),[§VI](https://arxiv.org/html/2609.26959#S6.p1.1)\.
- \[19\]K\. Weiss, T\. M\. Khoshgoftaar, and D\. Wang\(2016\)A survey of transfer learning\.Journal of Big Data3\(1\),pp\. 9\.External Links:[Document](https://dx.doi.org/10.1186/s40537-016-0043-6)Cited by:[§I](https://arxiv.org/html/2609.26959#S1.p3.1),[§II](https://arxiv.org/html/2609.26959#S2.p2.1)\.
- \[20\]W\. F\. Holmgren, C\. W\. Hansen, and M\. A\. Mikofski\(2018\)Pvlib python: a python package for simulating the performance of photovoltaic energy systems\.Journal of Open Source Software3\(29\),pp\. 884\.External Links:[Document](https://dx.doi.org/10.21105/joss.00884)Cited by:[item 1](https://arxiv.org/html/2609.26959#S1.I1.i1.p1.1),[§III\-B](https://arxiv.org/html/2609.26959#S3.SS2.p1.1)\.
- \[21\]S\. Hochreiter and J\. Schmidhuber\(1997\)Long short\-term memory\.Neural Computation9\(8\),pp\. 1735–1780\.External Links:[Document](https://dx.doi.org/10.1162/neco.1997.9.8.1735)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p1.1),[§III\-E](https://arxiv.org/html/2609.26959#S3.SS5.p1.1)\.
- \[22\]M\. Diagne, M\. David, P\. Lauret, J\. Boland, and N\. Schmutz\(2013\)Review of solar irradiance forecasting methods and a proposition for small\-scale insular grids\.Renewable and Sustainable Energy Reviews27,pp\. 65–76\.External Links:[Document](https://dx.doi.org/10.1016/j.rser.2013.06.042)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p1.1)\.
- \[23\]R\. H\. Inman, H\. T\. C\. Pedro, and C\. F\. M\. Coimbra\(2013\)Solar forecasting methods for renewable energy integration\.Progress in Energy and Combustion Science39\(6\),pp\. 535–576\.External Links:[Document](https://dx.doi.org/10.1016/j.pecs.2013.06.002)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p1.1)\.
- \[24\]R\. Blaga, A\. Sabadus, N\. Stefu, C\. Dughir, M\. Paulescu, and V\. Badescu\(2019\)A current perspective on the accuracy of incoming solar energy forecasting\.Progress in Energy and Combustion Science70,pp\. 119–144\.External Links:[Document](https://dx.doi.org/10.1016/j.pecs.2018.10.003)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p1.1)\.
- \[25\]F\. Wang, Z\. Xuan, Z\. Zhen, K\. Li, T\. Wang, and M\. Shi\(2019\)A comparison of day\-ahead photovoltaic power forecasting models based on deep learning neural network\.Applied Energy251,pp\. 113315\.External Links:[Document](https://dx.doi.org/10.1016/j.apenergy.2019.113315)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p1.1)\.
- \[26\]J\. Antonanzas, N\. Osorio, R\. Escobar, R\. Urraca, F\. J\. Martinez\-de\-Pison, and F\. Antonanzas\-Torres\(2016\)Review of photovoltaic power forecasting\.Solar Energy136,pp\. 78–111\.External Links:[Document](https://dx.doi.org/10.1016/j.solener.2016.06.069)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p1.1)\.
- \[27\]Y\. Bengio, A\. Courville, and P\. Vincent\(2013\)Representation learning: a review and new perspectives\.IEEE Transactions on Pattern Analysis and Machine Intelligence35\(8\),pp\. 1798–1828\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2013.50)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p2.1)\.
- \[28\]Y\. LeCun, Y\. Bengio, and G\. Hinton\(2015\)Deep learning\.Nature521\(7553\),pp\. 436–444\.External Links:[Document](https://dx.doi.org/10.1038/nature14539)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p2.1),[§III\-E](https://arxiv.org/html/2609.26959#S3.SS5.p1.1)\.
- \[29\]T\. Gneiting and A\. E\. Raftery\(2007\)Strictly proper scoring rules, prediction, and estimation\.Journal of the American Statistical Association102\(477\),pp\. 359–378\.External Links:[Document](https://dx.doi.org/10.1198/016214506000001437)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p3.1)\.
- \[30\]P\. Pinson\(2013\)Wind energy: forecasting challenges for its operational management\.Statistical Science28\(4\),pp\. 564–585\.External Links:[Document](https://dx.doi.org/10.1214/13-STS445)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p3.1)\.
- \[31\]V\. Vovk, A\. Gammerman, and G\. Shafer\(2005\)Algorithmic learning in a random world\.Springer Science & Business Media,New York\.Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p3.1),[§V](https://arxiv.org/html/2609.26959#S5.p2.1)\.
- \[32\]G\. Shafer and V\. Vovk\(2008\)A tutorial on conformal prediction\.Journal of Machine Learning Research9,pp\. 371–421\.Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p3.1)\.
- \[33\]M\. Fontana, G\. Zeni, and S\. Vantini\(2023\)Conformal prediction: a unified review of theory and new challenges\.Bernoulli29\(1\),pp\. 1–23\.External Links:[Document](https://dx.doi.org/10.3150/21-BEJ1447)Cited by:[§II](https://arxiv.org/html/2609.26959#S2.p3.1)\.
- \[34\]M\. Sengupta, Y\. Xie, A\. Lopez, A\. Habte, G\. Maclaurin, and J\. Shelby\(2018\)The National Solar Radiation Data Base \(NSRDB\)\.Renewable and Sustainable Energy Reviews89,pp\. 51–60\.External Links:[Document](https://dx.doi.org/10.1016/j.rser.2018.03.003)Cited by:[§III\-B](https://arxiv.org/html/2609.26959#S3.SS2.p1.1)\.
- \[35\]K\. Cho, B\. Van Merriënboer, C\. Gulcehre, D\. Bahdanau, F\. Bougares, H\. Schwenk, and Y\. Bengio\(2014\)Learning phrase representations using RNN encoder\-decoder for statistical machine translation\.InProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 1724–1734\.External Links:[Document](https://dx.doi.org/10.3115/v1/D14-1179)Cited by:[§III\-E](https://arxiv.org/html/2609.26959#S3.SS5.p1.1)\.
- \[36\]D\. P\. Kingma and J\. Ba\(2015\)Adam: a method for stochastic optimization\.InProceedings of the 3rd International Conference on Learning Representations \(ICLR\),Note:arXiv:1412\.6980Cited by:[§III\-E](https://arxiv.org/html/2609.26959#S3.SS5.p2.1)\.
- \[37\]I\. Goodfellow, Y\. Bengio, and A\. Courville\(2016\)Deep learning\.MIT Press,Cambridge, MA\.Cited by:[§III\-E](https://arxiv.org/html/2609.26959#S3.SS5.p3.1)\.
- \[38\]N\. Srivastava, G\. Hinton, A\. Krizhevsky, I\. Sutskever, and R\. Salakhutdinov\(2014\)Dropout: a simple way to prevent neural networks from overfitting\.Journal of Machine Learning Research15\(1\),pp\. 1929–1958\.Cited by:[§III\-E](https://arxiv.org/html/2609.26959#S3.SS5.p3.1)\.
- \[39\]A\. N\. Angelopoulos and S\. Bates\(2022\)A gentle introduction to conformal prediction and distribution\-free uncertainty quantification\.arXiv preprint arXiv:2107\.07511\.Cited by:[§V](https://arxiv.org/html/2609.26959#S5.p2.1)\.
- \[40\]D\. Chakraborty, J\. Mondal, H\. B\. Barua, and A\. Bhattacharjee\(2023\)Computational solar energy–ensemble learning methods for prediction of solar power generation based on meteorological parameters in eastern india\.arXiv preprint arXiv:2301\.10159\.Cited by:[§VI](https://arxiv.org/html/2609.26959#S6.p1.1)\.