Mind the Residual Gap: Probabilistic Downscaling under Real-World Bias

# Probabilistic Downscaling under Real-World Bias
Source: [https://arxiv.org/html/2606.30821](https://arxiv.org/html/2606.30821)
## Mind the Residual Gap: Probabilistic Downscaling under Real\-World Bias

Yujin Kim Department of Computer Science Cornell University Ithaca, NY, 14853 yk826@cornell\.eduNidhi Soma Department of Computer Science Cornell University Ithaca, NY 14853 ns848@cornell\.edu Sarah Dean Department of Computer Science Cornell University Ithaca, NY 14853 sdean@cornell\.edu

###### Abstract

Probabilistic downscaling is the task of modeling the conditional distribution of high\-resolution fields given coarse inputs, and is a central challenge to atmospheric science, climate modeling, and other multiscale physical systems\. A widely used paradigm decomposes the problem into a deterministic mean predictor followed by a stochastic residual generator\. While effective in idealized settings, this mean–residual approach frequently produces biased and under\-dispersive ensembles in real\-world applications\. Is this merely generic predictive uncertainty miscalibration? We show that the root cause is more fundamental:residual target misspecification, the residual distribution induced during training differs systematically from the one required at test time due to downscaling bias\. To close this gap, we introduceReMatch\(Residual Distribution Matching\)\. ReMatch aligns the training residual distribution toward the test\-time regime via optimal transport in a low\-dimensional PCA space\. This preserves the statistical benefits of the mean–residual framework while reducing the train–test mismatch in the residual targets seen by the stochastic generator\. On a controlled synthetic benchmark with varying bias levels and a real\-world HRRR–ERA5 wind field downscaling task, ReMatch substantially reduces under\-dispersion, improves calibration \(SSR and CRPS\), and outperforms strong baselines, including the standard mean–residual model and its variants, as well as state\-of\-the\-art super\-resolution models\. Our code is available at[https://github\.com/sdean\-group/ReMatch\.git](https://github.com/sdean-group/ReMatch.git)

## 1Introduction

Probabilistic downscaling seeks to model the conditional distribution of high\-resolution fields given coarse\-resolution inputs\. This problem is a common challenge in atmospheric science, climate modeling, and other multiscale physical systems\. Fine\-scale dynamics are essential for downstream tasks such as hazard assessment, local forecasting, trajectory planning, and uncertainty\-aware decision making, but directly simulating them is often computationally expensive\. Downscaling is inherently an ill\-posed inverse problem due to unresolved subgrid variability, even in idealized settings where the low\-resolution input is obtained by simple downsampling of the high\-resolution field\. In real\-world applications, however, the challenge is substantially greater\. Systematic biases and mismatches between coarse\- and fine\-resolution data sources—collectively referred to as*downscaling bias*—introduce additional, often domain\- or source\-dependent discrepanciesMaraun and Widmann \([2018](https://arxiv.org/html/2606.30821#bib.bib1)\)\. These biases distort the low\-to\-high resolution relationship far beyond simple interpolation, rendering the required high\-resolution correction both stochastic and systematically more difficult to infer\.

A widely adopted strategy for such tasks is the two\-stage mean–residual decomposition: a deterministic regressor first predicts a coarse approximation of the target, after which a stochastic generative model produces a residual correction that provides fine\-scale detail and probabilistic spread\. This design has been successfully applied across probabilistic downscalingMardaniet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib2)\); Fotiadiset al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib3)\); Rampalet al\.\([2024](https://arxiv.org/html/2606.30821#bib.bib4)\), climate forecastingYuet al\.\([2024](https://arxiv.org/html/2606.30821#bib.bib11)\); Nguyenet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib9)\), speech synthesisChenet al\.\([2022](https://arxiv.org/html/2606.30821#bib.bib8)\), and medical imaging[Zhanget al\.](https://arxiv.org/html/2606.30821#bib.bib7)\. In atmospheric downscaling, CorrDiffMardaniet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib2)\)is a prominent representative: it trains a regression model on variables such as temperature, wind, and radar reflectivity, then employs a conditional diffusion model to generate stochastic residual corrections\. However, CorrDiff and subsequent work consistently report that the resulting ensembles are under\-dispersive, with ensemble spread significantly smaller than the ensemble mean errorMardaniet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib2)\); Fotiadiset al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib3)\)\.

In this paper, we argue that this under\-dispersion is not merely a generic diffusion calibration\. Rather, it stems from a fundamental issue that we term*residual target misspecification*\. In the mean–residual framework, the residual is not an intrinsic stochastic component of the high\-resolution field; it is an*induced*correction target defined relative to the upstream mean predictorμϕ​\(x\)\\mu\_\{\\phi\}\(x\)\. When the low\- and high\-resolution fields exhibit significant downscaling bias \(as is typical in real\-world cross\-source or cross\-resolution settings\), the mean predictor overfits training\-specific biases and artifacts\. As a result, the residual distribution observed during training becomes systematically different from the correction distribution required at test time\. We formalize this train–test mismatch in the residual target itself and show—both theoretically and empirically—that its severity increases directly with the strength of low\-resolution \(LR\) – high\-resolution \(HR\) bias\.

To address this, we proposeReMatch\(Residual distribution Matching\)\. Using a calibration set as a proxy for the test regime, ReMatch explicitly aligns the training residual distribution with the test\-time residual distribution via optimal transportWanet al\.\([2023](https://arxiv.org/html/2606.30821#bib.bib15)\)\. This preserves the statistical benefits of the mean–residual decomposition while ensuring that the stochastic residual generator is trained on residual targets whose distribution is more closely matched to those encountered at inference\.

Our main contribution is threefold: \(i\) a rigorous explanation of why mean—residual pipelines are prone to under\-dispersion in biased downscaling settings, \(ii\) ReMatch, a practical solution that benefits both accuracy and ensemble calibration, and \(iii\) a comprehensive empirical evaluation on a synthetic benchmarkChunget al\.\([2023](https://arxiv.org/html/2606.30821#bib.bib40)\)with systematically varied LR—HR bias and a real\-world wind field downscaling task using HRRRDowellet al\.\([2022](https://arxiv.org/html/2606.30821#bib.bib42)\)and ERA5Hersbachet al\.\([2020](https://arxiv.org/html/2606.30821#bib.bib41)\)\. Across these settings, ReMatch consistently outperforms strong baselines—including the standard mean—residual pipeline, its classifier\-free guidance variantsHo and Salimans \([2022](https://arxiv.org/html/2606.30821#bib.bib44)\), uncertainty\-conditioned residual diffusion, and state\-of\-the\-art deterministic super\-resolution modelsLianget al\.\([2021](https://arxiv.org/html/2606.30821#bib.bib45)\); Wenet al\.\([2022](https://arxiv.org/html/2606.30821#bib.bib46)\)—confirming that residual target misspecification is the primary cause of under\-dispersion in biased downscaling\.

## 2Background and Setup

### 2\.1Probabilistic Downscaling

We consider probabilistic downscaling of high\-resolution geophysical fieldsy∈ℝcout×H×Wy\\in\\mathbb\{R\}^\{c\_\{\\mathrm\{out\}\}\\times H\\times W\}from low\-resolution inputsx∈ℝcin×h×w,x\\in\\mathbb\{R\}^\{c\_\{\\mathrm\{in\}\}\\times h\\times w\},whereH\>hH\>handW\>wW\>w\. Here,cinc\_\{\\mathrm\{in\}\}andcoutc\_\{\\mathrm\{out\}\}denote the numbers of input and output channels, respectively, such as velocity, temperature or other physical variables, while\(h,w\)\(h,w\)and\(H,W\)\(H,W\)denote the low\- and high\-resolution spatial grid sizes\. The task is to model the conditional distribution ofyygivenxx\.

In many physical downscaling problems, this mapping is fundamentally non\-deterministic\. A useful abstraction is to assume that the low\-resolution observation is obtained from the high\-resolution state through an effective degradation processx=D​\(y,s\),x=D\(y,s\),whereDDmay include not only resolution reduction, but also regridding, projection, source mismatch, simulator bias, and other systematic discrepancies, andssdenotes latent source\- or pipeline\-dependent factors\. For example, an ERA5\-to\-HRRR wind downscaling task combines differences in spatial resolution, numerical model formulation, data assimilation, pressure\-level definition, and grid projection within the same effective degradation processDD\. In idealized super\-resolution benchmarks,DDis often close to a simple downsampling operator, whereas in real\-world settings it may reflect substantial bias and cross\-source mismatch\. In either case,DDis many\-to\-one, so the inverse problem is ill\-posed\. Hence the natural learning target is not a unique deterministic inverse, but the conditional distributiony∼p​\(y∣x\)\.y\\sim p\(y\\mid x\)\.This probabilistic viewpoint is important not only because the inverse problem is many\-to\-one, but also because downstream applications often require uncertainty\-aware predictions\. In multiscale physical systems, unresolved fine\-scale variability can affect forecasting, decision making, and motion planning of agents moving through the predicted field\. Successful downscaling should therefore model both a central prediction and the spread of physically plausible high\-resolution realizations\.

### 2\.2Mean–Residual Decomposition

We study a class of pipelines in which an upstream mean predictor

μϕ​\(x\)∈ℝcout×H×W\\mu\_\{\\phi\}\(x\)\\in\\mathbb\{R\}^\{c\_\{\\mathrm\{out\}\}\\times H\\times W\}first produces a deterministic approximation of the target, and a residual generator then models the remaining correctionMardaniet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib2)\);[Liet al\.](https://arxiv.org/html/2606.30821#bib.bib12); Yuet al\.\([2024](https://arxiv.org/html/2606.30821#bib.bib11)\); Nguyenet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib9)\); Chenet al\.\([2022](https://arxiv.org/html/2606.30821#bib.bib8)\); Liet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib14)\); Laiet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib13)\)\. The target is decomposed as

y=μϕ​\(x\)\+rϕ,rϕ:=y−μϕ​\(x\)\.y=\\mu\_\{\\phi\}\(x\)\+r\_\{\\phi\},\\qquad r\_\{\\phi\}:=y\-\\mu\_\{\\phi\}\(x\)\.A residual generator

qθ​\(r∣x,μϕ​\(x\)\)q\_\{\\theta\}\(r\\mid x,\\mu\_\{\\phi\}\(x\)\)then produces stochastic corrections

r^∼qθ​\(r∣x,μϕ​\(x\)\),\\hat\{r\}\\sim q\_\{\\theta\}\(r\\mid x,\\mu\_\{\\phi\}\(x\)\),leading tokknumber of ensemble predictions

y^\(i\)=μϕ​\(x\)\+r^\(i\),i=1,…,k\.\\hat\{y\}^\{\(i\)\}=\\mu\_\{\\phi\}\(x\)\+\\hat\{r\}^\{\(i\)\},\\qquad i=1,\\dots,k\.
CorrDiffMardaniet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib2)\)is a prominent example of this design in atmospheric downscaling: it first predicts a deterministic high\-resolution field and then applies a diffusion model to generate a residual correction\. This decomposition is motivated by the idea that the residual distribution is statistically easier to model than the full target distribution, since the upstream mean predictor removes a substantial fraction of the large\-scale variance\.

Crucially, onceμϕ\\mu\_\{\\phi\}is fixed, the residual stage is trained on a predictor\-induced target rather than an intrinsic component of the data distribution\. This distinction is central to our analysis: the residual target can change when the reliability ofμϕ\\mu\_\{\\phi\}changes across domains, even if the underlying downscaling task remains the same\.

### 2\.3Domains and Splits

For training and test splits, let\(X,Y\)∼Pd\(X,Y\)\\sim P\_\{d\},d∈\{tr,te\}d\\in\\\{\\mathrm\{tr\},\\mathrm\{te\}\\\}, denote the joint input–target distribution\. Although the two splits are assumed to be i\.i\.d\. samples from the same population, the fitted upstream predictor is not symmetric with respect to them: it is optimized on the training split and evaluated on held\-out test samples\. For a fixedμϕ\\mu\_\{\\phi\}, each split therefore induces its own empirical residual random variable

Rdϕ​\(Xd,Yd\):=Yd−μϕ​\(Xd\),\(Xd,Yd\)∼Pd\.R\_\{d\}^\{\\phi\}\(X\_\{d\},Y\_\{d\}\):=Y\_\{d\}\-\\mu\_\{\\phi\}\(X\_\{d\}\),\\qquad\(X\_\{d\},Y\_\{d\}\)\\sim P\_\{d\}\.As a result, its errors, and therefore the induced residualsRtrϕR\_\{\\mathrm\{tr\}\}^\{\\phi\}andRteϕR\_\{\\mathrm\{te\}\}^\{\\phi\}, can have systematically different distributions\. This train–test residual mismatch is the object we analyze in the next section\.

## 3Residual Target Misspecification

We formalize the induced shift in the residual distribution asresidual target misspecification\. Unlike an intrinsic stochastic component of the high\-resolution field, the residual target is defined dependent to an upstream mean predictorμϕ\\mu\_\{\\phi\}\. In real\-world downscaling, the effective degradation process may combine resolution reduction, regridding, source mismatch, and model bias, soμϕ\\mu\_\{\\phi\}may learn a training\-domain LR–HR correspondence rather than a globally valid one\. When this learned relation changes in reliability or structure across test regimes, the induced residual target also shifts\. We refer to this predictor\-induced shift as residual target misspecification\.

### 3\.1Residual Distribution Shift Induced by Mean\-Error Gap

For each domaind∈\{tr,te\}d\\in\\\{\\mathrm\{tr\},\\mathrm\{te\}\\\}, define the expected squared residual magnitude

Md​\(ϕ\):=𝔼\(X,Y\)∼Pd​\[‖Y−μϕ​\(X\)‖22\]=𝔼R∼Rdϕ​\[‖R‖22\]\.M\_\{d\}\(\\phi\):=\\mathbb\{E\}\_\{\(X,Y\)\\sim P\_\{d\}\}\\bigl\[\\\|Y\-\\mu\_\{\\phi\}\(X\)\\\|\_\{2\}^\{2\}\\bigr\]=\\mathbb\{E\}\_\{R\\sim R^\{\\phi\}\_\{d\}\}\\bigl\[\\\|R\\\|\_\{2\}^\{2\}\\bigr\]\.This quantity measures the typical scale of the correction that the residual generator must model on domaindd\.

###### Proposition 1\(Residual magnitude gap implies residual distribution shift\)\.

For a fixed mean predictorμϕ\\mu\_\{\\phi\}, the 2\-Wasserstein distance between the induced residual distributions satisfy

W2​\(Rtrϕ,Rteϕ\)≥\|Mtr​\(ϕ\)−Mte​\(ϕ\)\|\.W\_\{2\}\(R^\{\\phi\}\_\{\\mathrm\{tr\}\},R^\{\\phi\}\_\{\\mathrm\{te\}\}\)\\geq\\left\|\\sqrt\{M\_\{\\mathrm\{tr\}\}\(\\phi\)\}\-\\sqrt\{M\_\{\\mathrm\{te\}\}\(\\phi\)\}\\right\|\.

###### Proof\.

For any domaindd,W2​\(Rdϕ,δ0\)=Md​\(ϕ\)W\_\{2\}\(R^\{\\phi\}\_\{d\},\\delta\_\{0\}\)=\\sqrt\{M\_\{d\}\(\\phi\)\}whereδ0\\delta\_\{0\}is the Dirac measure at the origin\. The claim then follows from the reverse triangle inequality forW2W\_\{2\}\. ∎

Although the lower bound in Proposition 1 is not always tight, it establishes a simple measurable certificate of residual distribution shift: a residual\-energy gap necessarily implies a distributional gap\.

### 3\.2Why Misspecification is Amplified in Residual Space

Even under an i\.i\.d\. train–test split,μϕ\\mu\_\{\\phi\}is optimized on the training samples and therefore tends to achieve lower empirical residual energy on the training split than on held\-out samples\. In practice, the most common direction isMtr​\(ϕ\)<Mte​\(ϕ\)\.M\_\{\\mathrm\{tr\}\}\(\\phi\)<M\_\{\\mathrm\{te\}\}\(\\phi\)\.In biased downscaling settings, this gap can easily be amplified, leaving small residuals during training but requiring larger corrections at test time\. Residualization can also make small mismatches in the GT and mean prediction more visible\. To see this, suppose the train and test residuals areRtrϕ,Rteϕ\.R\_\{\\mathrm\{tr\}\}^\{\\phi\},R\_\{\\mathrm\{te\}\}^\{\\phi\}\.Then their difference can be decomposed asRteϕ−Rtrϕ=\(Yte−Ytr\)−\(μϕ​\(Xte\)−μϕ​\(Xtr\)\)\.R\_\{\\mathrm\{te\}\}^\{\\phi\}\-R\_\{\\mathrm\{tr\}\}^\{\\phi\}=\\left\(Y\_\{\\mathrm\{te\}\}\-Y\_\{\\mathrm\{tr\}\}\\right\)\-\\left\(\\mu\_\{\\phi\}\(X\_\{\\mathrm\{te\}\}\)\-\\mu\_\{\\phi\}\(X\_\{\\mathrm\{tr\}\}\)\\right\)\.Thus, residual\-space mismatch depends not only on the shift in the target field, but also on how the mean predictor shifts across domains\.

### 3\.3Empirical Characterization

We empirically verify that residual target misspecification is concentrated in the residual space\. Using the mean–residual modelMardaniet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib2)\)trained on the real\-world wind\-field downscaling benchmark described in Section[5\.2](https://arxiv.org/html/2606.30821#S5.SS2), we compare the train and test distributions of the mean predictor output and the corresponding residual target\. For each field sample, we compute its samplewise RMS magnitude, specifically, the normalizedℓ2\\ell\_\{2\}norm over all channels and spatial locations\. We report the mean and standard deviation of these samplewise magnitude distributions, followed by train–test discrepancy metrics:W2W\_\{2\}, normalizedW2W\_\{2\}111Because the residual and mean predictor output have vastly different scales, raw Wasserstein distances can be misleading\. We therefore additionally report a*normalized Wasserstein distance*, defined asW2W\_\{2\}divided by the average RMS scale of the two distributions\., Jensen–Shannon \(JS\) divergence, and Kolmogorov–Smirnov \(KS\) statistic\.

Table 1:Train–test distribution shift comparison between the predictor\-defined residual distribution and the upstream mean predictor output\. Mean and standard deviation are computed from the samplewise RMS magnitude over all channels and spatial locations\.Table[1](https://arxiv.org/html/2606.30821#S3.T1)shows that the train–test shift is much more pronounced in the residual space than in the mean\-prediction space\. The residual distribution has large discrepancies in normalizedW2W\_\{2\}, JS divergence, and KS statistic, whereas the mean predictor output changes only mildly across domains\. This supports our claim that the predictor\-induced residual target can shift sharply even when the upstream mean output remains relatively stable\.

## 4ReMatch: Residual Distribution Matching

We introduce Residual Distribution Matching \(ReMatch\), a probabilistic downscaling framework that combines deterministic mean prediction, residual target distribution alignment, and conditional diffusion\-based residual generation\. Its core idea is to preserve the useful mean prior provided by a deterministic high\-resolution mean predictor while explicitly adapting the training residual targets toward a regime that better matches the correction statistics required at test time\.

ReMatch uses a held\-out calibration split as an observable proxy for the test\-time target residual distribution\. Accordingly, we extend the domain notation tod∈\{tr,cal,te\},d\\in\\\{\\mathrm\{tr\},\\mathrm\{cal\},\\mathrm\{te\}\\\},which denote the training, calibration, and test splits, respectively\. The training split is used to fit the upstream mean predictor, and the calibration split is used for residual distribution matching\. The residual generator is trained on both training and calibration split, while the test split is reserved for final evaluation\.

### 4\.1Stage 1: High\-Resolution Mean Field Estimation

Let\(x,y\)∼Ptr\(x,y\)\\sim P\_\{\\mathrm\{tr\}\}denote a training input–target pair\. We train an upstream mean predictor using the pointwise regression objective

μϕ​\(x\)∈ℝcout×H×W,ℒreg​\(ϕ\)=𝔼\(x,y\)∼Ptr​\[‖y−μϕ​\(x\)‖pp\]\.\\mu\_\{\\phi\}\(x\)\\in\\mathbb\{R\}^\{c\_\{\\mathrm\{out\}\}\\times H\\times W\},\\quad\\mathcal\{L\}\_\{\\mathrm\{reg\}\}\(\\phi\)=\\mathbb\{E\}\_\{\(x,y\)\\sim P\_\{\\mathrm\{tr\}\}\}\\bigl\[\\\|y\-\\mu\_\{\\phi\}\(x\)\\\|\_\{p\}^\{p\}\\bigr\]\.This stage produces a structured deterministic approximation of the target field that serves as a strong prior for subsequent stochastic refinement\. For each training sample, it induces a residual

rϕ​\(x,y\)=y−μϕ​\(x\)\.r\_\{\\phi\}\(x,y\)=y\-\\mu\_\{\\phi\}\(x\)\.

### 4\.2Stage 2: Residual Target Alignment via Transport

We address the domain shiftRtrϕ≠RteϕR\_\{\\mathrm\{tr\}\}^\{\\phi\}\\neq R\_\{\\mathrm\{te\}\}^\{\\phi\}by transporting the training residual distribution toward the calibration residual distribution, which serves as a proxy for the test\-time correction regime\. This is naturally formulated using optimal transport, which provides a framework for distribution matching by transporting mass from a source distribution to a target distributionVillani and others \([2009](https://arxiv.org/html/2606.30821#bib.bib19)\); Peyré and Cuturi \([2019](https://arxiv.org/html/2606.30821#bib.bib20)\)\.

At a high level, ReMatch replaces each training residual with an OT\-weighted combination of calibration residuals\. The transport cost combines residual similarity with low\-resolution conditioning consistency, preventing residuals from being transported to calibration samples with incompatible LR inputs\. This shifts the training residual distribution toward the calibration regime while preserving compatibility among the LR input, pseudo\-mean, and transported residual\. However, applying optimal transport directly in pixel space is computationally expensive and statistically unstable in high\-dimensional residual fieldsChoiet al\.\([2023](https://arxiv.org/html/2606.30821#bib.bib21)\); Balajiet al\.\([2020](https://arxiv.org/html/2606.30821#bib.bib30)\)\. To make transport over high\-dimensional residual fields tractable and less sensitive to grid\-scale noise, we compute the transport cost in compact PCA representationsAgustssonet al\.\([2017](https://arxiv.org/html/2606.30821#bib.bib37)\); Yuanet al\.\([2024](https://arxiv.org/html/2606.30821#bib.bib33)\); Liuet al\.\([2018](https://arxiv.org/html/2606.30821#bib.bib36)\)\. We fit separate PCA bases to the concatenated training and calibration residuals and to the corresponding low\-resolution inputs\. Letzitr,zjcalz\_\{i\}^\{\\mathrm\{tr\}\},z\_\{j\}^\{\\mathrm\{cal\}\}denote residual PCA representations, and letℓitr,ℓjcal\\ell\_\{i\}^\{\\mathrm\{tr\}\},\\ell\_\{j\}^\{\\mathrm\{cal\}\}denote LR\-conditioning PCA representations\. We define transport cost

ci​j=α​‖zitr−zjcal‖22\+λcond​‖ℓitr−ℓjcal‖22\.c\_\{ij\}=\\alpha\\\|z\_\{i\}^\{\\mathrm\{tr\}\}\-z\_\{j\}^\{\\mathrm\{cal\}\}\\\|\_\{2\}^\{2\}\+\\lambda\_\{\\mathrm\{cond\}\}\\\|\\ell\_\{i\}^\{\\mathrm\{tr\}\}\-\\ell\_\{j\}^\{\\mathrm\{cal\}\}\\\|\_\{2\}^\{2\}\.Here,ci​jc\_\{ij\}is the cost of matching training sampleiito calibration samplejj, withα\\alphaandλcond\\lambda\_\{\\mathrm\{cond\}\}controlling the residual and LR\-conditioning terms\. The low\-resolution conditioning term in the cost function is crucial, as it constrains transport to occur between samples with similar large\-scale inputs, thereby reducing the risk of generating incompatible pseudo\-means\. We solve an optimal transport problem using the costci​jc\_\{ij\}and obtain a transport planπ\\pi\. The transported residual is then reconstructed as

z~i=∑jπi​j​zjcal∑jπi​j,r~i=UK​z~i\+r¯,μ~i=yi−r~i,\\tilde\{z\}\_\{i\}=\\frac\{\\sum\_\{j\}\\pi\_\{ij\}z\_\{j\}^\{\\mathrm\{cal\}\}\}\{\\sum\_\{j\}\\pi\_\{ij\}\},\\qquad\\tilde\{r\}\_\{i\}=U\_\{K\}\\tilde\{z\}\_\{i\}\+\\bar\{r\},\\qquad\\tilde\{\\mu\}\_\{i\}=y\_\{i\}\-\\tilde\{r\}\_\{i\},whereπi​j\\pi\_\{ij\}is the transport weight from training sampleiito calibration samplejj\. Additional implementation details are provided in Appendix[A\.3](https://arxiv.org/html/2606.30821#A1.SS3)\.

### 4\.3Stage 3: Residual Generation with Diffusion

In the final stage, we train a conditional diffusion modelqθ​\(r∣x,μ\)q\_\{\\theta\}\(r\\mid x,\\mu\)to generate stochastic residual corrections\. The model is trained on the union of \(i\) the transported training samples\{\(xi,μ~i,r~i\)\}i=1ntr\\\{\(x\_\{i\},\\tilde\{\\mu\}\_\{i\},\\tilde\{r\}\_\{i\}\)\\\}\_\{i=1\}^\{n\_\{\\mathrm\{tr\}\}\}and \(ii\) the original calibration samples\{\(xj,μϕ\(xj\),rj\)\}j=1ncal\}\\\{\(x\_\{j\},\\mu\_\{\\phi\}\(x\_\{j\}\),r\_\{j\}\)\\\}\_\{j=1\}^\{n\_\{\\mathrm\{cal\}\}\}\\\}\.

This design preserves the computational advantages of the mean–residual decomposition while explicitly correcting the train–test mismatch in the residual distribution\. Algorithm[A\.2](https://arxiv.org/html/2606.30821#A1.SS2)provides an overview of the three\-stage ReMatch training pipeline\.

Implementation Note\.ReMatch is modular: the mean predictor and stochastic residual generator can be instantiated with different architectures; implementation details are for are provided in Appendix[A\.6](https://arxiv.org/html/2606.30821#A1.SS6)\.

## 5Experiments and Results

We evaluate ReMatch on a controlled synthetic benchmark and a real\-world atmospheric downscaling task\. The synthetic experiment validates the mechanism of bias\-induced residual target misspecification by systematically varying the LR–HR bias level\. The real\-world experiment tests ReMatch under realistic ERA5\-to\-HRRR wind\-field distribution shifts\. We evaluate downscaling fidelity and probabilistic calibration, and further assess downstream impact through a particle\-advection experiment, where ensemble trajectories reveal how calibration affects predicted motion through the flow field\.

Baselines and Variants\.We use two versions of ReMatch: ReMatchUwith a UNet mean predictor and ReMatchSwith a SwinIRLianget al\.\([2021](https://arxiv.org/html/2606.30821#bib.bib45)\)mean predictor\. We compare ReMatch against deterministic baselines, including UNetSonget al\.\([2020](https://arxiv.org/html/2606.30821#bib.bib48)\), Conv\-FNOWenet al\.\([2022](https://arxiv.org/html/2606.30821#bib.bib46)\), and SwinIRLianget al\.\([2021](https://arxiv.org/html/2606.30821#bib.bib45)\), and probabilistic CorrDiff\-based baselines\. The latter include the original CorrDiffMardaniet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib2)\), a reduced\-capacity mean\-predictor variant \(CorrDiffm\), classifier\-free guidance on residual diffusion \(CFGHo and Salimans \([2022](https://arxiv.org/html/2606.30821#bib.bib44)\)\), and uncertainty\-conditioned variants \(UCq\\mathrm\{UC\}\_\{q\}andUCμ\\mathrm\{UC\}\_\{\\mu\}\)\.

Except for SwinLR and Conv\-FNO, all regression models are UNet\-basedSonget al\.\([2020](https://arxiv.org/html/2606.30821#bib.bib48)\), and all diffusion models use an EDM\-style residual diffusion formulationKarraset al\.\([2022](https://arxiv.org/html/2606.30821#bib.bib47)\)with a UNet\-based score network\. We implement conditioning by channel\-wise concatenation\. Details of each baseline and implementation are provided in Appendix[A\.6](https://arxiv.org/html/2606.30821#A1.SS6)\.

Evaluation Metrics\.We report RMSE, Spread–Skill Ratio \(SSR\), and Continuous Ranked Probability Score \(CRPS\) as primary metrics\. RMSE measures reconstruction accuracy, while SSR and CRPS evaluate probabilistic quality; SSR is particularly useful for diagnosing ensemble under\-dispersion\. We report LPIPSZhanget al\.\([2018](https://arxiv.org/html/2606.30821#bib.bib53)\)as an auxiliary perceptual metric, following prior use in atmospheric\-field predictionYuet al\.\([2024](https://arxiv.org/html/2606.30821#bib.bib11)\); Hoksonet al\.\([2026](https://arxiv.org/html/2606.30821#bib.bib52)\)\. We also report ACCNaiet al\.\([2025](https://arxiv.org/html/2606.30821#bib.bib50)\), computed between predicted and ground\-truth \(GT\) deviations from the train set GT mean for each channel and time\.

Metrics in Table[2](https://arxiv.org/html/2606.30821#S5.T2)are computed using 600 random test samples\. Visual metrics are normalized channelwise with quantile extrema, and stochastic\-model LPIPS is averaged over ensembles\.

### 5\.1Bias\-Controlled Synthetic Benchmark

To validate our theoretical analysis, we conduct a controlled experiment on the BLASTNet datasetChunget al\.\([2023](https://arxiv.org/html/2606.30821#bib.bib40)\)\. We artificially inject structured downscaling bias of increasing severity into the low\-resolution inputs while keeping the high\-resolution targets fixed \(details and bias generation procedure in Appendix[A\.4](https://arxiv.org/html/2606.30821#A1.SS4)\)\. We use CorrDiff as a baseline method and ReMatchUas our proposed method\. In all cases, 12 ensemble predictions are generated to calculate spreads and RMSE\.

Dataset\.We use the BLASTNet Momentum128 3D super\-resolution datasetChunget al\.\([2023](https://arxiv.org/html/2606.30821#bib.bib40)\)as a controlled testbed\. The original dataset provides clean paired low\- and high\-resolution fields with 4 channels\. We use the8×8\\timessuper\-resolution setting, with16×1616\\times 16low\-resolution inputs and128×128128\\times 128high\-resolution targets\. We define four bias levels \(Level 0 to 3\), where Level 0 uses the original clean interpolated low\-resolution field and Levels 1–3 introduce progressively stronger structured mismatch by mixing 2D fields along thezzaxis with Gaussian weights \(full procedure in Appendix[A\.4](https://arxiv.org/html/2606.30821#A1.SS4)\)\. Figure[2](https://arxiv.org/html/2606.30821#S5.F2)illustrates how the input changes across the four bias levels under same HR target\.

#### 5\.1\.1Results

Does residual misspecification explain under\-dispersion?Figure[1](https://arxiv.org/html/2606.30821#S5.F1)shows the RMSE–SSR scatter across increasing LR–HR bias levels\. As the bias level increases, the baseline exhibits a clear degradation: RMSE increases while SSR decreases\. This trend indicates that stronger LR–HR mismatch induces a more difficult residual correction regime and leads to increasingly under\-dispersive ensembles\.

Does ReMatch mitigate residual target misspecification?ReMatch improves RMSE in all biased settings, with the exception of Level 0, where the LR input is not biased\. It also consistently improves SSR, and the improvement becomes more pronounced at higher bias levels\. These results support our central hypothesis: when paired low\- and high\-resolution data contain structured bias, the mean–residual decomposition can induce residual target misspecification, and residual distribution matching can mitigate this effect\. Channel\-wise RMSE and SSR results are provided in Appendix[A\.4](https://arxiv.org/html/2606.30821#A1.SS4)\.

![[Uncaptioned image]](https://arxiv.org/html/2606.30821v1/x1.png)

Figure 1:Performance under increasing LR input bias\.ReMatch consistently improves SSR and reduces RMSE in biased settings, with the performance gap becoming more pronounced as the bias level increases\.\(SSR→1\\rightarrow 1, RMSE↓\\downarrow\)
![[Uncaptioned image]](https://arxiv.org/html/2606.30821v1/x2.png)

Figure 2:Qualitative comparison across LR input bias levels\.Rows correspond to LR bias levels\. Columns show the GT HR field, LR input, ReMatch and CorrDiff prediction\.

### 5\.2Real\-World Wind Field Downscaling Benchmark

Our main experiment evaluates ReMatch on a real\-world atmospheric downscaling task with realistic distribution shift\. Deterministic baselines \(UNet, ConvFNO, and SwinIR\) produce a single prediction, whereas stochastic methods \(CDM, CorrDiffm, CorrDiff, CFG, UCμ\\mu, UCq, ReMatchU, and ReMatchS\) generate 12 ensemble members\. Further details on each model are provided in Appendix[A\.6](https://arxiv.org/html/2606.30821#A1.SS6)\.

Dataset\.We use ERA5Hersbachet al\.\([2020](https://arxiv.org/html/2606.30821#bib.bib41)\)as the coarse\-resolution input and HRRRDowellet al\.\([2022](https://arxiv.org/html/2606.30821#bib.bib42)\)as the high\-resolution target over the northeast CONUS region\. The input containsuu\- andvv\-wind fields at five stratospheric pressure levels, giving 10 channels in total and 4 temporal channels for sinusoidal embeddings of sample day and month on a21×2121\\times 21grid\. The target consists of the same 10 wind\-component channels on a168×168168\\times 168grid\. The dataset contains hourly fields from 2018 to 2021\. We use 2018–2020 for training and 2021 for testing; for methods requiring calibration data, we hold out 2020 as a calibration set\. Further details are in Appendix[A\.5](https://arxiv.org/html/2606.30821#A1.SS5)\.

#### 5\.2\.1Results

![Refer to caption](https://arxiv.org/html/2606.30821v1/x3.png)Figure 3:Qualitative comparison of downscaled 50 hPavv\-wind fields on 2021\-06\-05 10:00:00\.The top\-left panel shows the upsampled ERA5 input, and the bottom\-left panel shows the corresponding HRRR GT\. Deterministic baselines produce a single prediction, whereas diffusion\-based methods generate 12 ensemble members; here we show the 7th ensemble member for each stochastic method\.What does realistic LR–HR mismatch look like?The first column of Fig\.[3](https://arxiv.org/html/2606.30821#S5.F3)illustrates LR–HR mismatch: even for the same atmospheric scene, the LR input and the corresponding HR target differ substantially\. In the appendix, Fig\.[7](https://arxiv.org/html/2606.30821#A1.F7)shows more examples where RMSE can quantify this mismatch, and Table[3](https://arxiv.org/html/2606.30821#A1.T3)shows that the amount of bias realistically depends on channel\. The cross\-source mismatch makes real\-world downscaling more challenging than idealized super\-resolution\.

Table 2:Quantitative comparison on HRRR\-ERA5 pressure\-level super\-resolution over all channels\.Notes: For deterministic models, CRPS is replaced with MAE and SSR is omitted\. Best results are shown in bold, and second\-best results are underlined\.

Do the proposed methods improve downscaling performance?Figure[3](https://arxiv.org/html/2606.30821#S5.F3)shows that UNet remains biased in overall scale\. In contrast, mean–residual methods using UNet as the mean predictor, including CorrDiff, CFG, UC, and ReMatchU, produce predictions that are closer to the HR target in both spatial structure and magnitude, suggesting that residual generation helps correct errors in the deterministic mean estimate\. A similar trend is observed for ReMatchS, which uses SwinIR as the mean predictor\. Overall, ReMatchUand ReMatchSproduce visually competitive reconstructions while better recovering the large\-scale magnitude and small\-scale spatial patterns of the HR field\. Table[2](https://arxiv.org/html/2606.30821#S5.T2)shows that ReMatch performs strongly across both deterministic and probabilistic metrics\. ReMatchSand ReMatchUrank highly on the main metrics of interest showing that the proposed residual distribution matching improves both reconstruction accuracy and stochastic calibration\. ReMatchUachieves the best LPIPS, consistent with the qualitative improvements in Figure[3](https://arxiv.org/html/2606.30821#S5.F3)\. ReMatch also remains competitive on ACC, indicating that its gains extend beyond pointwise accuracy\. Overall, these results show that ReMatch improves the accuracy–calibration tradeoff in real\-world wind\-field downscaling\.

Is making the residual target more generic sufficient?If train–test residual target shift is the core problem, one possible strategy is to make the residual target itself less specialized to the training domain\. However, our results show that this is not sufficient: while CorrDiffmand CDM improve SSR relative to CorrDiff, they do so at the cost of reduced accuracy and fidelity\.

How do CorrDiff variants compare?CFG and uncertainty\-conditioned CorrDiff \(UC\) provide two alternative strategies for improving residual generation\. CFG is commonly used to sharpen diffusion samples, but in this setting it does not improve calibration because it does not directly address residual target misspecification\. Uncertainty conditioning improves SSR by providing the residual generator with additional information about the reliability of the mean predictor, but it depends on the quality of the learned uncertainty estimate and requires an additional training stage\.

How much does the deterministic mean predictor matter?Among deterministic baselines, SwinIR achieves the strongest performance, showing that a stronger mean predictor can improve reconstruction accuracy\. However, purely deterministic estimators, including UNet, ConvFNO, and SwinIR cannot quantify uncertainty\. This limits their use in downstream tasks where calibrated ensemble spread is essential\. ReMatchS, which adds residual generation on top of the SwinIR mean predictor, further improves both RMSE and CRPS/MAE, suggesting that ReMatch can benefit from stronger deterministic backbones while still providing calibrated stochastic predictions\.

Does ensemble calibration matter for downstream tasks?Predicted physical fields are often used as environments for downstream decision making and agent motion planning\. To examine how residual calibration may affect such downstream use, we advect particles through 48 consecutive hourly wind fields\. Figure[4](https://arxiv.org/html/2606.30821#S5.F4)compares trajectories from the HR ground\-truth field, CorrDiff ensembles, and ReMatchUensembles\. CorrDiff trajectories deviate from the ground\-truth path as the particle moves away from the initial position, whereas ReMatchUremains better centered around the truth while preserving ensemble spread\.

![Refer to caption](https://arxiv.org/html/2606.30821v1/x4.png)Figure 4:Particle trajectory comparison using ensemble wind\-field predictions over 48 consecutive hourly fields\.The red line denotes the trajectory advected by the HR ground\-truth field, orange lines denote CorrDiff ensemble trajectories, and blue lines denote ReMatchUensemble trajectories\.While this example alone does not establish a general downstream advantage, it suggests that better\-calibrated spread can improve the usefulness of ensemble forecasts for downstream uncertainty\-aware tasks\.

## 6Discussion

Role of calibration data\.ReMatch relies on a held\-out calibration set because residual target adaptation is only meaningful when the calibration residuals reflect the correction regime expected at test time\. This is a weaker and more practical assumption than requiring access to test labels, but it is still an important one: ReMatch is not intended to solve arbitrary distribution shift without representative held\-out data\. In our experiments, we use a train–calibration–test split of approximately2:1:12\{:\}1\{:\}1, which provides enough calibration samples to estimate a stable residual distribution while leaving sufficient data for fitting the mean predictor and residual generator\. The optimal calibration size is likely task\-dependent\. A larger calibration set can better represent the test\-time residual regime and improve transport stability, but it also reduces the amount of data available for training the mean predictor if the total dataset size is fixed\. Studying this tradeoff more systematically is an important direction for future work, especially in settings where labeled high\-resolution calibration data are expensive\.

Residual target misspecification is broader than a mean\-error gap\.Our theoretical analysis uses the residual\-energy gapMtr​\(ϕ\)≠Mte​\(ϕ\)M\_\{\\mathrm\{tr\}\}\(\\phi\)\\neq M\_\{\\mathrm\{te\}\}\(\\phi\)as a simple certificate of residual distribution shift, but this gap is not the only form that misspecification can take\. Even when train and test residuals have similar overall magnitude, they may differ in spatial structure, tail behavior, extremes, or conditional dependence on the input\. Thus, residual target misspecification should be understood more generally as a predictor\-induced target shift: once the upstream mean predictor is fixed, the residual generator is trained on a correction target whose statistics depend on the reliability and structure of that predictor\. The mean\-error gap is therefore one measurable manifestation of a broader mismatch between the residual distribution seen during training and the residual distribution required at inference\.

Limitations and future directions\.ReMatch performs residual distribution matching in a low\-dimensional PCA space, which makes optimal transport more tractable than pixel\-space matching\. However, because PCA is a linear global representation, it may not fully preserve localized extremes, nonlinear structures, or rare high\-frequency residual modes\. Future work could replace PCA with learned or physics\-aware latent representations that better capture dynamically important corrections\. This direction is aligned with the broader use of optimal transport for distribution alignment and domain adaptationCourtyet al\.\([2017b](https://arxiv.org/html/2606.30821#bib.bib59)\), as well as Wasserstein\-based objectives for learning latent generative representationsTolstikhinet al\.\([2018](https://arxiv.org/html/2606.30821#bib.bib27)\); Kolouriet al\.\([2019](https://arxiv.org/html/2606.30821#bib.bib28)\)\. Scalability is another limitation: our current implementation computes transport over the full training and calibration sets, which may become infeasible for much larger datasets\. This could be addressed with stochastic optimization methods for large\-scale OTGenevayet al\.\([2016](https://arxiv.org/html/2606.30821#bib.bib32)\), regularized OT solversCuturi \([2013](https://arxiv.org/html/2606.30821#bib.bib17)\), or learned neural transport maps for out\-of\-sample residual adaptationKorotinet al\.\([2022](https://arxiv.org/html/2606.30821#bib.bib31)\)\.

ConclusionWe studied probabilistic downscaling under real\-world bias through the lens of residual target misspecification, where a fixed mean predictor induces residual targets whose training distribution differs from the correction regime required at test time\. We introduced ReMatch, which adapts training residuals toward a calibration residual regime using optimal transport while preserving the deterministic mean predictor as a useful prior\. Across controlled bias experiments, real\-world ERA5\-to\-HRRR wind downscaling, and a downstream particle\-advection example, ReMatch improves the accuracy–calibration tradeoff over strong baselines\.

More broadly, our results suggest that in residual generative pipelines, calibration failures should not be viewed only as deficiencies of the stochastic sampler; they can originate from the target construction itself\. Designing reliable probabilistic downscaling systems therefore requires matching not only model outputs, but also the predictor\-induced targets on which generative models are trained\.

## Acknowledgements

We thank Preston Culbertson for helpful discussions on diffusion underdispersion in residual generative downscaling\. This work was supported in part by the NVIDIA Academic Grant Program\. This work was partly funded by NSF CCF 2312774, NSF OAC\-2311521, NSF IIS\-2442137, NSF IIS2505098, a gift to the LinkedIn\-Cornell Bowers CIS Strategic Partnership, and an AI2050 Early Career Fellowship program at Schmidt Science\.

## References

- \[1\]\(2017\)Optimal transport maps for distribution preserving operations on latent spaces of generative models\.arXiv preprint arXiv:1711\.01970\.Cited by:[§A\.3](https://arxiv.org/html/2606.30821#A1.SS3.p1.1),[§4\.2](https://arxiv.org/html/2606.30821#S4.SS2.p2.2)\.
- \[2\]A\. N\. Angelopoulos, A\. P\. Kohli, S\. Bates, M\. Jordan, J\. Malik, T\. Alshaabi, S\. Upadhyayula, and Y\. Romano\(2022\)Image\-to\-image regression with distribution\-free uncertainty quantification and applications in imaging\.InInternational Conference on Machine Learning,pp\. 717–730\.Cited by:[§A\.11](https://arxiv.org/html/2606.30821#A1.SS11.p1.3)\.
- \[3\]Y\. Balaji, R\. Chellappa, and S\. Feizi\(2020\)Robust optimal transport with applications in generative modeling and domain adaptation\.Advances in Neural Information Processing Systems33,pp\. 12934–12944\.Cited by:[§4\.2](https://arxiv.org/html/2606.30821#S4.SS2.p2.2)\.
- \[4\]S\. Ben\-David, J\. Blitzer, K\. Crammer, A\. Kulesza, F\. Pereira, and J\. W\. Vaughan\(2010\)A theory of learning from different domains\.Machine learning79\(1\),pp\. 151–175\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px4.p1.1)\.
- \[5\]N\. Bonneel, J\. Rabin, G\. Peyré, and H\. Pfister\(2015\)Sliced and radon wasserstein barycenters of measures\.Journal of Mathematical Imaging and Vision51\(1\),pp\. 22–45\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1)\.
- \[6\]Z\. Chen, Y\. Wu, Y\. Leng, J\. Chen, H\. Liu, X\. Tan, Y\. Cui, K\. Wang, L\. He, S\. Zhao,et al\.\(2022\)Resgrad: residual denoising diffusion probabilistic models for text to speech\.arXiv preprint arXiv:2212\.14518\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2606.30821#S1.p2.1),[§2\.2](https://arxiv.org/html/2606.30821#S2.SS2.p1.3)\.
- \[7\]J\. Choi, J\. Choi, and M\. Kang\(2023\)Generative modeling through the semi\-dual formulation of unbalanced optimal transport\.Advances in Neural Information Processing Systems36,pp\. 42433–42455\.Cited by:[§4\.2](https://arxiv.org/html/2606.30821#S4.SS2.p2.2)\.
- \[8\]W\. T\. Chung, B\. Akoush, P\. Sharma, A\. Tamkin, K\. S\. Jung, J\. Chen, J\. Guo, D\. Brouzet, M\. Talei, B\. Savard,et al\.\(2023\)Turbulence in focus: benchmarking scaling behavior of 3d volumetric super\-resolution with blastnet 2\.0 data\.Advances in Neural Information Processing Systems36,pp\. 77430–77484\.Cited by:[§A\.3](https://arxiv.org/html/2606.30821#A1.SS3.p2.1),[§A\.4](https://arxiv.org/html/2606.30821#A1.SS4.p1.4),[§1](https://arxiv.org/html/2606.30821#S1.p5.1),[§5\.1](https://arxiv.org/html/2606.30821#S5.SS1.p1.1),[§5\.1](https://arxiv.org/html/2606.30821#S5.SS1.p2.4)\.
- \[9\]N\. Courty, R\. Flamary, A\. Habrard, and A\. Rakotomamonjy\(2017\)Joint distribution optimal transportation for domain adaptation\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px4.p1.1)\.
- \[10\]N\. Courty, R\. Flamary, D\. Tuia, and A\. Rakotomamonjy\(2017\)Optimal transport for domain adaptation\.IEEE Transactions on Pattern Analysis and Machine Intelligence39\(9\),pp\. 1853–1865\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2016.2615921)Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px4.p1.1),[§6](https://arxiv.org/html/2606.30821#S6.p3.1)\.
- \[11\]M\. Cuturi\(2013\)Sinkhorn distances: lightspeed computation of optimal transport\.Advances in neural information processing systems26\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1),[§A\.3](https://arxiv.org/html/2606.30821#A1.SS3.p5.4),[§6](https://arxiv.org/html/2606.30821#S6.p3.1)\.
- \[12\]I\. Deshpande, Y\. Hu, R\. Sun, A\. Pyrros, N\. Siddiqui, S\. Koyejo, Z\. Zhao, D\. Forsyth, and A\. G\. Schwing\(2019\)Max\-sliced wasserstein distance and its use for gans\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 10648–10656\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1)\.
- \[13\]D\. C\. Dowell, C\. R\. Alexander, E\. P\. James, S\. S\. Weygandt, S\. G\. Benjamin, G\. S\. Manikin, B\. T\. Blake, J\. M\. Brown, J\. B\. Olson, M\. Hu,et al\.\(2022\)The high\-resolution rapid refresh \(hrrr\): an hourly updating convection\-allowing forecast model\. part i: motivation and system description\.Weather and Forecasting37\(8\),pp\. 1371–1395\.Cited by:[§A\.3](https://arxiv.org/html/2606.30821#A1.SS3.p2.1),[§A\.5](https://arxiv.org/html/2606.30821#A1.SS5.SSS0.Px2),[§1](https://arxiv.org/html/2606.30821#S1.p5.1),[§5\.2](https://arxiv.org/html/2606.30821#S5.SS2.p2.4)\.
- \[14\]S\. Fotiadis, N\. D\. Brenowitz, T\. Geffner, Y\. Cohen, M\. Pritchard, A\. Vahdat, and M\. Mardani\(2025\)Adaptive flow matching for resolving small\-scale physics\.InForty\-second International Conference on Machine Learning,Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px1.p1.1),[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px2.p1.1),[§A\.8](https://arxiv.org/html/2606.30821#A1.SS8.p2.1),[§1](https://arxiv.org/html/2606.30821#S1.p2.1)\.
- \[15\]N\. Fournier and A\. Guillin\(2015\)On the rate of convergence in wasserstein distance of the empirical measure\.Probability Theory and Related Fields162\(3–4\),pp\. 707–738\.External Links:[Document](https://dx.doi.org/10.1007/s00440-014-0583-7)Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1)\.
- \[16\]A\. Genevay, M\. Cuturi, G\. Peyré, and F\. Bach\(2016\)Stochastic optimization for large\-scale optimal transport\.Advances in neural information processing systems29\.Cited by:[§6](https://arxiv.org/html/2606.30821#S6.p3.1)\.
- \[17\]J\. Gong, L\. Bai, P\. Ye, W\. Xu, N\. Liu, J\. Dai, X\. Yang, and W\. Ouyang\(2024\)Cascast: skillful high\-resolution precipitation nowcasting via cascaded modelling\.arXiv preprint arXiv:2402\.04290\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px2.p1.1)\.
- \[18\]H\. Hersbach, B\. Bell, P\. Berrisford, S\. Hirahara, A\. Horányi, J\. Muñoz\-Sabater, J\. Nicolas, C\. Peubey, R\. Radu, D\. Schepers,et al\.\(2020\)The era5 global reanalysis\.Quarterly journal of the royal meteorological society146\(730\),pp\. 1999–2049\.Cited by:[§A\.3](https://arxiv.org/html/2606.30821#A1.SS3.p2.1),[§A\.5](https://arxiv.org/html/2606.30821#A1.SS5.SSS0.Px1),[§1](https://arxiv.org/html/2606.30821#S1.p5.1),[§5\.2](https://arxiv.org/html/2606.30821#S5.SS2.p2.4)\.
- \[19\]J\. Ho and T\. Salimans\(2022\)Classifier\-free diffusion guidance\.arXiv preprint arXiv:2207\.12598\.Cited by:[§1](https://arxiv.org/html/2606.30821#S1.p5.1),[§5](https://arxiv.org/html/2606.30821#S5.p2.2)\.
- \[20\]J\. A\. Hokson, S\. Kanae, and Y\. Hiraga\(2026\)Predicting tropical cyclone rainfall through perceptual\-based identification of similar atmospheric fields\.Weather and Climate Extremes,pp\. 100896\.Cited by:[§5](https://arxiv.org/html/2606.30821#S5.p4.1)\.
- \[21\]T\. Karras, M\. Aittala, T\. Aila, and S\. Laine\(2022\)Elucidating the design space of diffusion\-based generative models\.Advances in neural information processing systems35,pp\. 26565–26577\.Cited by:[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.p1.6),[§5](https://arxiv.org/html/2606.30821#S5.p3.1)\.
- \[22\]A\. Kendall and Y\. Gal\(2017\)What uncertainties do we need in bayesian deep learning for computer vision?\.Advances in neural information processing systems30\.Cited by:[§A\.11](https://arxiv.org/html/2606.30821#A1.SS11.p1.3),[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.SSS0.Px5)\.
- \[23\]S\. Kolouri, P\. E\. Pope, C\. E\. Martin, and G\. K\. Rohde\(2019\)Sliced\-wasserstein autoencoder: an embarrassingly simple generative model\.InInternational Conference on Learning Representations,Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1),[§6](https://arxiv.org/html/2606.30821#S6.p3.1)\.
- \[24\]A\. Korotin, D\. Selikhanovych, and E\. Burnaev\(2022\)Neural optimal transport\.arXiv preprint arXiv:2201\.12220\.Cited by:[§6](https://arxiv.org/html/2606.30821#S6.p3.1)\.
- \[25\]C\. Lai, Y\. Ning, and D\. S\. Boning\(2025\)RDIT: residual\-based diffusion implicit models for probabilistic time series forecasting\.arXiv preprint arXiv:2509\.02341\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px3.p1.1),[§2\.2](https://arxiv.org/html/2606.30821#S2.SS2.p1.3)\.
- \[26\]G\. Li, Y\. Liu, S\. Cao, M\. Chen, X\. Liu, X\. Wang, J\. Zheng, Y\. Li, Z\. Ye, H\. Liang,et al\.ResCast: enhancing global medium\-range precipitation forecasting with residual diffusion model\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px2.p1.1),[§2\.2](https://arxiv.org/html/2606.30821#S2.SS2.p1.3)\.
- \[27\]Q\. Li, Z\. Zhang, L\. Yao, Z\. Li, T\. Zhong, and Y\. Zhang\(2025\)Diffusion\-based decoupled deterministic and uncertain framework for probabilistic multivariate time series forecasting\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§2\.2](https://arxiv.org/html/2606.30821#S2.SS2.p1.3)\.
- \[28\]J\. Liang, J\. Cao, G\. Sun, K\. Zhang, L\. Van Gool, and R\. Timofte\(2021\)Swinir: image restoration using swin transformer\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 1833–1844\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px1.p1.1),[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.SSS0.Px6),[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.p1.6),[§1](https://arxiv.org/html/2606.30821#S1.p5.1),[§5](https://arxiv.org/html/2606.30821#S5.p2.2)\.
- \[29\]T\. Lin, C\. Fan, N\. Ho, M\. Cuturi, and M\. I\. Jordan\(2020\)Projection robust wasserstein distance and riemannian optimization\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9383–9397\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1)\.
- \[30\]H\. Liu, Y\. Guo, N\. Lei, Z\. Shu, S\. Yau, D\. Samaras, and X\. Gu\(2018\)Latent space optimal transport for generative models\.arXiv preprint arXiv:1809\.05964\.Cited by:[§A\.3](https://arxiv.org/html/2606.30821#A1.SS3.p1.1),[§4\.2](https://arxiv.org/html/2606.30821#S4.SS2.p2.2)\.
- \[31\]D\. Maraun and M\. Widmann\(2018\)Statistical downscaling and bias correction for climate research\.Cambridge University Press\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.30821#S1.p1.1)\.
- \[32\]M\. Mardani, N\. Brenowitz, Y\. Cohen, J\. Pathak, C\. Chen, C\. Liu, A\. Vahdat, M\. A\. Nabian, T\. Ge, A\. Subramaniam,et al\.\(2025\)Residual corrective diffusion modeling for km\-scale atmospheric downscaling\.Communications Earth & Environment6\(1\),pp\. 124\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px2.p1.1),[§A\.10](https://arxiv.org/html/2606.30821#A1.SS10.p4.1),[§A\.4](https://arxiv.org/html/2606.30821#A1.SS4.p3.2),[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.SSS0.Px1),[§A\.8](https://arxiv.org/html/2606.30821#A1.SS8.p2.1),[§1](https://arxiv.org/html/2606.30821#S1.p2.1),[§2\.2](https://arxiv.org/html/2606.30821#S2.SS2.p1.3),[§2\.2](https://arxiv.org/html/2606.30821#S2.SS2.p2.1),[§3\.3](https://arxiv.org/html/2606.30821#S3.SS3.p1.3),[§5](https://arxiv.org/html/2606.30821#S5.p2.2)\.
- \[33\]C\. Nai, X\. Chen, S\. Yang, Z\. Xiao, and B\. Pan\(2025\)Boosting weather forecast via generative superensemble\.npj Climate and Atmospheric Science8\(1\),pp\. 377\.Cited by:[§5](https://arxiv.org/html/2606.30821#S5.p4.1)\.
- \[34\]T\. Nguyen, J\. Ma, F\. S\. Khan, S\. B\. Taieb, and S\. Khan\(2025\)RainDiff: end\-to\-end precipitation nowcasting via token\-wise attention diffusion\.arXiv preprint arXiv:2510\.14962\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2606.30821#S1.p2.1),[§2\.2](https://arxiv.org/html/2606.30821#S2.SS2.p1.3)\.
- \[35\]S\. J\. Pan and Q\. Yang\(2010\)A survey on transfer learning\.IEEE Transactions on Knowledge and Data Engineering22\(10\),pp\. 1345–1359\.External Links:[Document](https://dx.doi.org/10.1109/TKDE.2009.191)Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px4.p1.1)\.
- \[36\]F\. Paty and M\. Cuturi\(2019\)Subspace robust wasserstein distances\.InProceedings of the 36th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.97,pp\. 5072–5081\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1)\.
- \[37\]G\. Peyré and M\. Cuturi\(2019\)Computational optimal transport: with applications to data science\.Now Foundations and Trends\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1),[§4\.2](https://arxiv.org/html/2606.30821#S4.SS2.p1.1)\.
- \[38\]N\. Rampal, S\. Hobeichi, P\. B\. Gibson, J\. Baño\-Medina, G\. Abramowitz, T\. Beucler, J\. González\-Abad, W\. Chapman, P\. Harder, and J\. M\. Gutiérrez\(2024\)Enhancing regional climate downscaling through advances in machine learning\.Artificial Intelligence for the Earth Systems3\(2\),pp\. 230066\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.30821#S1.p2.1)\.
- \[39\]I\. Redko, N\. Courty, R\. Flamary, and D\. Tuia\(2019\)Optimal transport for multi\-source domain adaptation under target shift\.InProceedings of the Twenty\-Second International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.89,pp\. 849–858\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px4.p1.1)\.
- \[40\]H\. Shimodaira\(2000\)Improving predictive inference under covariate shift by weighting the log\-likelihood function\.Journal of Statistical Planning and Inference90\(2\),pp\. 227–244\.External Links:[Document](https://dx.doi.org/10.1016/S0378-3758%2800%2900115-4)Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px4.p1.1)\.
- \[41\]Y\. Song, J\. Sohl\-Dickstein, D\. P\. Kingma, A\. Kumar, S\. Ermon, and B\. Poole\(2020\)Score\-based generative modeling through stochastic differential equations\.arXiv preprint arXiv:2011\.13456\.Cited by:[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.SSS0.Px2),[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.p1.6),[§5](https://arxiv.org/html/2606.30821#S5.p2.2),[§5](https://arxiv.org/html/2606.30821#S5.p3.1)\.
- \[42\]M\. Springenberg, N\. Otero, Y\. Xue, and J\. Ma\(2026\)Diffscale: continuous downscaling and bias correction of subseasonal wind speed forecasts using diffusion models\.Journal of Advances in Modeling Earth Systems18\(3\),pp\. e2025MS005282\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px1.p1.1),[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.SSS0.Px4)\.
- \[43\]M\. Sugiyama and M\. Kawanabe\(2012\)Machine learning in non\-stationary environments: introduction to covariate shift adaptation\.MIT Press\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px4.p1.1)\.
- \[44\]I\. Tolstikhin, O\. Bousquet, S\. Gelly, and B\. Schölkopf\(2018\)Wasserstein auto\-encoders\.InInternational Conference on Learning Representations,Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1),[§6](https://arxiv.org/html/2606.30821#S6.p3.1)\.
- \[45\]C\. Villaniet al\.\(2009\)Optimal transport: old and new\.Vol\.338,Springer\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1),[§4\.2](https://arxiv.org/html/2606.30821#S4.SS2.p1.1)\.
- \[46\]Z\. Y\. Wan, R\. Baptista, A\. Boral, Y\. Chen, J\. Anderson, F\. Sha, and L\. Zepeda\-Núñez\(2023\)Debias coarsely, sample conditionally: statistical downscaling through optimal transport and probabilistic diffusion models\.Advances in Neural Information Processing Systems36,pp\. 47749–47763\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px1.p1.1),[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2606.30821#S1.p4.1)\.
- \[47\]J\. Wang, M\. Chen, N\. Karaev, A\. Vedaldi, C\. Rupprecht, and D\. Novotny\(2025\)Vggt: visual geometry grounded transformer\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 5294–5306\.Cited by:[§A\.11](https://arxiv.org/html/2606.30821#A1.SS11.p1.3)\.
- \[48\]J\. Weed and F\. Bach\(2019\)Sharp asymptotic and finite\-sample rates of convergence of empirical measures in wasserstein distance\.InBernoulli,Vol\.25,pp\. 2620–2648\.External Links:[Document](https://dx.doi.org/10.3150/18-BEJ1065)Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px5.p1.1)\.
- \[49\]G\. Wen, Z\. Li, K\. Azizzadenesheli, A\. Anandkumar, and S\. M\. Benson\(2022\)U\-fno—an enhanced fourier neural operator\-based deep\-learning model for multiphase flow\.Advances in Water Resources163,pp\. 104180\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px1.p1.1),[§A\.6](https://arxiv.org/html/2606.30821#A1.SS6.SSS0.Px7),[§1](https://arxiv.org/html/2606.30821#S1.p5.1),[§5](https://arxiv.org/html/2606.30821#S5.p2.2)\.
- \[50\]D\. Yu, X\. Li, Y\. Ye, B\. Zhang, C\. Luo, K\. Dai, R\. Wang, and X\. Chen\(2024\)Diffcast: a unified framework via residual diffusion for precipitation nowcasting\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 27758–27767\.Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2606.30821#S1.p2.1),[§2\.2](https://arxiv.org/html/2606.30821#S2.SS2.p1.3),[§5](https://arxiv.org/html/2606.30821#S5.p4.1)\.
- \[51\]Y\. Yuan, B\. Shahbaba, N\. Fortin, K\. Cooper, Q\. Nie, and A\. Qu\(2024\)Optimal transport for latent integration with an application to heterogeneous neuronal activity data\.arXiv preprint arXiv:2407\.00099\.Cited by:[§A\.3](https://arxiv.org/html/2606.30821#A1.SS3.p1.1),[§4\.2](https://arxiv.org/html/2606.30821#S4.SS2.p2.2)\.
- \[52\]R\. Zhang, P\. Isola, A\. A\. Efros, E\. Shechtman, and O\. Wang\(2018\)The unreasonable effectiveness of deep features as a perceptual metric\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 586–595\.Cited by:[§5](https://arxiv.org/html/2606.30821#S5.p4.1)\.
- \[53\]Z\. Zhang, M\. Hiller, K\. A\. Ehinger, and T\. DrummondPixel\-level residual diffusion transformer: scalable 3d ct volume generation\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§A\.1](https://arxiv.org/html/2606.30821#A1.SS1.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2606.30821#S1.p2.1)\.

## Appendix AAppendix

### A\.1Related Work

##### Statistical and deep learning downscaling\.

Statistical downscaling models high\-resolution fields from coarse inputs using learned relationships between resolutions\[[31](https://arxiv.org/html/2606.30821#bib.bib1)\]\. Classical approaches such as bias correction and quantile mapping are effective for marginal statistics, but often struggle to preserve spatial coherence and multivariate dependencies\. Deep neural networks have improved deterministic downscaling accuracy through convolutional, transformer\-based, and operator\-learning architectures, including SwinIR\[[28](https://arxiv.org/html/2606.30821#bib.bib45)\]and U\-FNO\[[49](https://arxiv.org/html/2606.30821#bib.bib46)\]\. However, deterministic predictors produce only a single estimate and therefore cannot represent the spread of physically plausible high\-resolution states\. This has motivated probabilistic downscaling methods based on modern generative models\[[38](https://arxiv.org/html/2606.30821#bib.bib4)\], including flow matching\[[14](https://arxiv.org/html/2606.30821#bib.bib3)\]and diffusion\-based approaches\[[46](https://arxiv.org/html/2606.30821#bib.bib15),[42](https://arxiv.org/html/2606.30821#bib.bib43)\], which aim to generate calibrated ensembles for uncertainty\-aware downstream applications\.

##### Diffusion\-based mean–residual downscaling\.

Diffusion models have recently become a strong framework for probabilistic atmospheric downscaling because they can model complex high\-dimensional conditional distributions\. CorrDiff\[[32](https://arxiv.org/html/2606.30821#bib.bib2)\]is a representative mean–residual approach: a deterministic regression model first predicts a high\-resolution mean field, and a conditional diffusion model then generates stochastic residual corrections\. Similar residual or two\-stage formulations appear in precipitation nowcasting and forecasting models such as DiffCast\[[50](https://arxiv.org/html/2606.30821#bib.bib11)\], RainDiff\[[34](https://arxiv.org/html/2606.30821#bib.bib9)\], ResCast\[[26](https://arxiv.org/html/2606.30821#bib.bib12)\], and CasCast\[[17](https://arxiv.org/html/2606.30821#bib.bib10)\]\. While this decomposition reduces the variance that the diffusion model must learn, prior work reports persistent under\-dispersion in generated ensembles\[[32](https://arxiv.org/html/2606.30821#bib.bib2),[14](https://arxiv.org/html/2606.30821#bib.bib3)\]\. Our work studies this failure mode directly and argues that it arises from predictor\-induced residual target misspecification, rather than only from limitations of the stochastic sampler\.

##### Residual generative pipelines beyond atmospheric downscaling\.

Mean–residual decompositions have also been adopted outside geophysical downscaling\. In speech synthesis, ResGrad\[[6](https://arxiv.org/html/2606.30821#bib.bib8)\]uses residual diffusion to refine a deterministic mel\-spectrogram prediction, while residual diffusion models have also been used for medical imaging\[[53](https://arxiv.org/html/2606.30821#bib.bib7)\]and probabilistic time\-series forecasting\[[25](https://arxiv.org/html/2606.30821#bib.bib13)\]\. These methods share the same structural assumption: a deterministic predictor provides a useful prior, and a stochastic model learns the remaining correction\. ReMatch focuses on a failure mode that can arise in such pipelines: the residual target learned by the stochastic generator remains dependent on the reliability of the upstream predictor, even when inherent input–target bias is present\.

##### Distribution matching and domain adaptation\.

Our approach is related to domain adaptation, where source and target distributions are aligned to improve transfer under distribution shift\[[4](https://arxiv.org/html/2606.30821#bib.bib38),[35](https://arxiv.org/html/2606.30821#bib.bib39)\]\. Classical covariate\-shift and importance\-weighting methods reweight source samples to match the target input distribution under the assumption that the conditional target remains unchanged\[[40](https://arxiv.org/html/2606.30821#bib.bib58),[43](https://arxiv.org/html/2606.30821#bib.bib57)\], while OT\-based domain adaptation aligns source and target feature\-label distributions through a learned transport plan\[[10](https://arxiv.org/html/2606.30821#bib.bib59),[9](https://arxiv.org/html/2606.30821#bib.bib60),[39](https://arxiv.org/html/2606.30821#bib.bib61)\]\. In downscaling, prior work uses OT to debias the coarse input before conditional diffusion sampling\[[46](https://arxiv.org/html/2606.30821#bib.bib15)\]\. ReMatch differs in that it does not primarily debias the input distribution\. Instead, it aligns the predictor\-induced residual target distribution that the stochastic generator is trained to model\.

##### Distribution matching with projected transport\.

Optimal transport provides a principled framework for comparing and aligning probability distributions\[[45](https://arxiv.org/html/2606.30821#bib.bib19),[37](https://arxiv.org/html/2606.30821#bib.bib20)\]\. Entropic regularization improves the computational scalability of empirical OT\[[11](https://arxiv.org/html/2606.30821#bib.bib17)\]\. However, estimating Wasserstein distances in high\-dimensional spaces is statistically challenging, with convergence rates that deteriorate with dimension\[[15](https://arxiv.org/html/2606.30821#bib.bib35),[48](https://arxiv.org/html/2606.30821#bib.bib34)\]\. A common remedy is to compare distributions through lower\-dimensional projections or compact representations, as in sliced Wasserstein distances\[[5](https://arxiv.org/html/2606.30821#bib.bib23)\], max\-sliced and subspace\-robust variants\[[12](https://arxiv.org/html/2606.30821#bib.bib24),[36](https://arxiv.org/html/2606.30821#bib.bib25),[29](https://arxiv.org/html/2606.30821#bib.bib26)\], and latent\-space transport or regularization methods\[[44](https://arxiv.org/html/2606.30821#bib.bib27),[23](https://arxiv.org/html/2606.30821#bib.bib28)\]\. ReMatch follows this projection\-based perspective by computing transport costs in a latent representation of residuals and conditioning inputs, rather than directly in the full residual field\. This provides a tractable way to match residual correction targets, while suggesting that richer physics\-aware representations or scalable projected distribution\-matching objectives could further improve residual alignment in future work\.

### A\.2Algorithm Overview

Algorithm 1ReMatch training pipeline1:Input:

𝒟tr=\{\(xi,yi\)\}i=1ntr\\mathcal\{D\}\_\{\\mathrm\{tr\}\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\_\{\\mathrm\{tr\}\}\},

𝒟cal=\{\(xj,yj\)\}j=1ncal\\mathcal\{D\}\_\{\\mathrm\{cal\}\}=\\\{\(x\_\{j\},y\_\{j\}\)\\\}\_\{j=1\}^\{n\_\{\\mathrm\{cal\}\}\}
2:Stage 1\. Mean training:

3:

ϕ⋆←arg⁡minϕ⁡ℒreg​\(ϕ;𝒟tr\)\\phi^\{\\star\}\\leftarrow\\arg\\min\_\{\\phi\}\\mathcal\{L\}\_\{\\mathrm\{reg\}\}\(\\phi;\\mathcal\{D\}\_\{\\mathrm\{tr\}\}\)
4:Stage 2\. Residual target alignment:

5:Compute residuals

ritr=yi−μϕ⋆​\(xi\)r\_\{i\}^\{\\mathrm\{tr\}\}=y\_\{i\}\-\\mu\_\{\\phi^\{\\star\}\}\(x\_\{i\}\)and

rjcal=yj−μϕ⋆​\(xj\)r\_\{j\}^\{\\mathrm\{cal\}\}=y\_\{j\}\-\\mu\_\{\\phi^\{\\star\}\}\(x\_\{j\}\)
6:Project onto PCA:

zitr=UK⊤​ritrz\_\{i\}^\{\\mathrm\{tr\}\}=U\_\{K\}^\{\\top\}r\_\{i\}^\{\\mathrm\{tr\}\},

zjcal=UK⊤​rjcalz\_\{j\}^\{\\mathrm\{cal\}\}=U\_\{K\}^\{\\top\}r\_\{j\}^\{\\mathrm\{cal\}\},

ℓitr=V⊤​xi\\ell\_\{i\}^\{\\mathrm\{tr\}\}=V^\{\\top\}x\_\{i\},

ℓjcal=V⊤​xj\\ell\_\{j\}^\{\\mathrm\{cal\}\}=V^\{\\top\}x\_\{j\}
7:Define cost

ci​j=α​‖zitr−zjcal‖22\+λcond​‖ℓitr−ℓjcal‖22c\_\{ij\}=\\alpha\\\|z\_\{i\}^\{\\mathrm\{tr\}\}\-z\_\{j\}^\{\\mathrm\{cal\}\}\\\|\_\{2\}^\{2\}\+\\lambda\_\{\\mathrm\{cond\}\}\\\|\\ell\_\{i\}^\{\\mathrm\{tr\}\}\-\\ell\_\{j\}^\{\\mathrm\{cal\}\}\\\|\_\{2\}^\{2\}
8:Compute sparse transport plan

π\\pi
9:Transport:

z~itr=∑jπi​j​zjcal/∑jπi​j\\tilde\{z\}\_\{i\}^\{\\mathrm\{tr\}\}=\\sum\_\{j\}\\pi\_\{ij\}z\_\{j\}^\{\\mathrm\{cal\}\}/\\sum\_\{j\}\\pi\_\{ij\},

r~i=UK​z~itr\\tilde\{r\}\_\{i\}=U\_\{K\}\\tilde\{z\}\_\{i\}^\{\\mathrm\{tr\}\},

μ~i=yi−r~i\\tilde\{\\mu\}\_\{i\}=y\_\{i\}\-\\tilde\{r\}\_\{i\}
10:Define residual\-learning dataset

𝒟res=\{\(xi,μ~i,r~i\)\}i=1ntr∪\{\(xj,μϕ⋆​\(xj\),rjcal\)\}j=1ncal\.\\mathcal\{D\}\_\{\\mathrm\{res\}\}=\\\{\(x\_\{i\},\\tilde\{\\mu\}\_\{i\},\\tilde\{r\}\_\{i\}\)\\\}\_\{i=1\}^\{n\_\{\\mathrm\{tr\}\}\}\\cup\\\{\(x\_\{j\},\\mu\_\{\\phi^\{\\star\}\}\(x\_\{j\}\),r\_\{j\}^\{\\mathrm\{cal\}\}\)\\\}\_\{j=1\}^\{n\_\{\\mathrm\{cal\}\}\}\.
11:Stage 3\. Residual learning:

12:

θ⋆←arg⁡minθ⁡ℒres​\(θ;𝒟res\)\\theta^\{\\star\}\\leftarrow\\arg\\min\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{res\}\}\(\\theta;\\mathcal\{D\}\_\{\\mathrm\{res\}\}\)
13:Return:

μϕ⋆,qθ⋆\\mu\_\{\\phi^\{\\star\}\},\\;q\_\{\\theta^\{\\star\}\}

### A\.3Residual Distribution Matching via Optimal Transport

Directly computing optimal transport over full residual fields is computationally expensive and can be overly sensitive to grid\-scale noise\. We therefore compute the transport cost in compact PCA representations of the residual targets and low\-resolution conditioning inputs\[[1](https://arxiv.org/html/2606.30821#bib.bib37),[51](https://arxiv.org/html/2606.30821#bib.bib33),[30](https://arxiv.org/html/2606.30821#bib.bib36)\]\. The PCA bases are fitted using the concatenated training and calibration samples, and each field is projected onto the leading components before transport\.

For PCA truncation, we choose the number of modes so that the reconstruction fidelity exceeds99%99\\%for all channels\. For the BlastNet dataset\[[8](https://arxiv.org/html/2606.30821#bib.bib40)\], we use 200 modes for the high\-resolution residual field and 70 modes for the low\-resolution conditioning field\. For the HRRR dataset\[[13](https://arxiv.org/html/2606.30821#bib.bib42)\], we use 100 residual modes and for ERA5\[[18](https://arxiv.org/html/2606.30821#bib.bib41)\], we use 30 modes\. BlastNet requires more modes because it contains a wider variety of spatial patterns induced by varying PDE coefficients and flow conditions\.

For a residual samplerϕr\_\{\\phi\}, let

z=UK⊤​\(rϕ−r¯\)∈ℝKz=U\_\{K\}^\{\\top\}\(r\_\{\\phi\}\-\\bar\{r\}\)\\in\\mathbb\{R\}^\{K\}denote its residual PCA representation, whereUKU\_\{K\}is the residual PCA basis andr¯\\bar\{r\}is the residual mean used for centering\. Similarly, for a low\-resolution inputxx, let

ℓ=VKcond⊤​\(x−x¯\)∈ℝKcond\\ell=V\_\{K\_\{\\mathrm\{cond\}\}\}^\{\\top\}\(x\-\\bar\{x\}\)\\in\\mathbb\{R\}^\{K\_\{\\mathrm\{cond\}\}\}denote its conditioning representation, whereVKcondV\_\{K\_\{\\mathrm\{cond\}\}\}is the PCA basis for the low\-resolution input\. Let

\{zitr,ℓitr\}i=1ntr​and​\{zjcal,ℓjcal\}j=1ncal\\\{z\_\{i\}^\{\\mathrm\{tr\}\},\\ell\_\{i\}^\{\\mathrm\{tr\}\}\\\}\_\{i=1\}^\{n\_\{\\mathrm\{tr\}\}\}\\quad\\text\{and\}\\quad\\\{z\_\{j\}^\{\\mathrm\{cal\}\},\\ell\_\{j\}^\{\\mathrm\{cal\}\}\\\}\_\{j=1\}^\{n\_\{\\mathrm\{cal\}\}\}denote the resulting low\-dimensional representations for the training and calibration sets, respectively\.

We compute transport separately for each output channel\. For each channel, we define the pairwise cost between training sampleiiand calibration samplejjas

ci​j=α​‖zitr−zjcal‖22\+λcond​‖ℓitr−ℓjcal‖22\.c\_\{ij\}=\\alpha\\left\\\|z\_\{i\}^\{\\mathrm\{tr\}\}\-z\_\{j\}^\{\\mathrm\{cal\}\}\\right\\\|\_\{2\}^\{2\}\+\\lambda\_\{\\mathrm\{cond\}\}\\left\\\|\\ell\_\{i\}^\{\\mathrm\{tr\}\}\-\\ell\_\{j\}^\{\\mathrm\{cal\}\}\\right\\\|\_\{2\}^\{2\}\.The first term encourages residual similarity, while the second term preserves consistency with the low\-resolution conditioning input\. This prevents the transported residual from being matched to a calibration residual whose associated low\-resolution field is incompatible with the source input\. In all experiments, we setα=1\.0\\alpha=1\.0andλcond=1\.0\\lambda\_\{\\mathrm\{cond\}\}=1\.0\.

To make the transport computation scalable, we construct a sparsekk\-nearest\-neighbor graph in the augmented PCA feature space and solve a balanced entropic Sinkhorn problem only over the resulting candidate pairs\[[11](https://arxiv.org/html/2606.30821#bib.bib17)\]\. Specifically, we usek=32k=32, entropic regularization0\.10\.1, and 1500 Sinkhorn iterations\. The sparse transport planπ\\piis computed with uniform source and target marginals\. When the number of training and calibration samples differs, the target marginal simply assigns a larger mass to each calibration sample\.

Because a sparsekk\-nearest\-neighbor graph can leave some calibration samples with too few incoming source candidates, we augment the graph before Sinkhorn normalization\. In particular, we enforce a minimum target in\-degree proportional to the source\-to\-target sample ratio by adding nearest reverse\-neighbor edges for under\-covered calibration samples\. This improves feasibility of the balanced transport constraints while keeping the sparse graph size fixed\.

After computing the sparse plan, we construct transported residual coefficients using a weighted top\-mmselection rather than averaging over allkkneighbors\. For each training sample, we select the topm=3m=3calibration candidates according to the transport weights, with a capacity constraint limiting how often each calibration sample can be reused\. This reduces collapse onto a small subset of calibration residuals while avoiding overly diffuse barycentric averages\. The selected weights are renormalized within the top\-mmset\.

For each training sampleii, the transported residual representation is therefore computed as

z~itr=∑j∈𝒯iπ~i​j​zjcal,\\tilde\{z\}\_\{i\}^\{\\mathrm\{tr\}\}=\\sum\_\{j\\in\\mathcal\{T\}\_\{i\}\}\\tilde\{\\pi\}\_\{ij\}z\_\{j\}^\{\\mathrm\{cal\}\},where𝒯i\\mathcal\{T\}\_\{i\}denotes the selected top\-mmcalibration candidates andπ~i​j\\tilde\{\\pi\}\_\{ij\}denotes the corresponding normalized transport weight\. We then reconstruct the transported residual in pixel space as

r~i=UK​z~itr\+r¯\.\\tilde\{r\}\_\{i\}=U\_\{K\}\\tilde\{z\}\_\{i\}^\{\\mathrm\{tr\}\}\+\\bar\{r\}\.
Finally, we define the corresponding pseudo\-mean by enforcing exact consistency with the original high\-resolution target:

μ~i:=yi−r~i\.\\tilde\{\\mu\}\_\{i\}:=y\_\{i\}\-\\tilde\{r\}\_\{i\}\.This yields adapted pseudo training samples

\(xi,μ~i,r~i\)\(x\_\{i\},\\tilde\{\\mu\}\_\{i\},\\tilde\{r\}\_\{i\}\)that preserve the mean–residual decomposition

yi=μ~i\+r~iy\_\{i\}=\\tilde\{\\mu\}\_\{i\}\+\\tilde\{r\}\_\{i\}by construction\. Thus, ReMatch shifts the residual target distribution toward the calibration residual regime while maintaining sample\-wise consistency among the input, pseudo\-mean, residual, and ground\-truth target\.

### A\.4Synthetic Bias\-Controlled Benchmark Details

![Refer to caption](https://arxiv.org/html/2606.30821v1/x5.png)Figure 5:Synthetic LR bias levels used in the BlastNet benchmark\. Level 0 uses the original interpolated LR input\. Levels 1–3 introduce progressively stronger structured mismatch by mixing slices along thezz\-axis with Gaussian weights\.We use the BLASTNet Momentum128 3D super\-resolution dataset\[[8](https://arxiv.org/html/2606.30821#bib.bib40)\]as a controlled synthetic benchmark\. The dataset provides clean paired low\- and high\-resolution turbulent flow fields, where the low\-resolution inputs are generated from the corresponding high\-resolution DNS fields through filtering and downsampling\. Each high\-resolution sample is a three\-dimensional field of size128×128×128128\\times 128\\times 128, with corresponding low\-resolution fields provided at multiple downsampling scales\. Both the input and target fields contain four physical channels: densityRHOand the three velocity componentsUX,UY, andUZ\. We use clusters 4, 11, 14, and 17, and take the8×8\\timesdownsampled inputs \(spatial resolution16×1616\\times 16\)\. Since our implementation operates on 2D inputs, we extract slices along thezz\-axis\. This also enables controlled bias injection along the depth direction\.

We define four bias levels\. Level 0 uses the original interpolated low\-resolution field \(clean interpolation\)\. For levels 1–3, we replace each low\-resolution slice with a weighted mixture of slices along thezz\-axis using Gaussian weights

w​\(z\)=exp⁡\(−12​\(z−cσ\)2\),w\(z\)=\\exp\\\!\\left\(\-\\frac\{1\}\{2\}\\left\(\\frac\{z\-c\}\{\\sigma\}\\right\)^\{2\}\\right\),where the centerccand bandwidthσ\\sigmaare chosen to produce progressively stronger structured mismatch: local mixture \(σ=2\\sigma=2\) for level 1, broad averaging for level 2, and sharp distant centering \(σ=1\\sigma=1\) for level 3\.

Figure[5](https://arxiv.org/html/2606.30821#A1.F5)illustrates the four bias levels and the corresponding weighting patterns\. The baseline method\[[32](https://arxiv.org/html/2606.30821#bib.bib2)\]is trained using the entire training set of 406 samples\. For ReMatch, we partition this training set into a training subset \(n=270n=270\) and a calibration set \(n=136n=136\)\. The test set contains 52 samples\.

#### A\.4\.1Controlled Synthetic Study of Bias\-induced Residual Misspecification

Figure[6](https://arxiv.org/html/2606.30821#A1.F6)shows channel\-wise RMSE and SSR across four bias levels\. Across all channels, ReMatch yields increasingly larger SSR improvements as the bias level increases, with only minor RMSE degradation or, in some cases, RMSE improvement\.

![Refer to caption](https://arxiv.org/html/2606.30821v1/x6.png)Figure 6:Channel\-wise RMSE and SSR ratio over different bias levels\.

### A\.5ERA5\-HRRR Dataset Details

We use theuu\- andvv\-components of wind at five lower\-stratospheric pressure levels: 50, 75, 100, 125, and 150 hPa\. The experiments are conducted over a500​km×500​km500\\,\\mathrm\{km\}\\times 500\\,\\mathrm\{km\}region in the northeastern CONUS, defined bylat∈\[36\.0795,41\.8914\]∘\\mathrm\{lat\}\\in\[36\.0795,41\.8914\]^\{\\circ\}andlon∈\[−82\.5501,−75\.2087\]∘\\mathrm\{lon\}\\in\[\-82\.5501,\-75\.2087\]^\{\\circ\}\.

##### ERA5\[[18](https://arxiv.org/html/2606.30821#bib.bib41)\]

We downloaded25×25​km25\\times 25\\,\\mathrm\{km\}scale ERA5 reanalysis data for every hour from 2018\-2021\. We note that ERA5 only had 70 hPa instead of 75 hPa pressure level data, but we consider them as the same channel since it does not detract from our setting of input/output mismatch\. To create the low\-resolution input for our experiments, we used bilinear interpolation to upsample data to the3×3​km3\\times 3\\,\\mathrm\{km\}resolution that HRRR uses\. We used 2018\-2020 as our train set and 2021 as our test set\. For methods that need a calibration set \(UC, ReMatch\), we used 2018\-2019 as the train set, 2020 as the calibration set, and 2021 as the test set\.

##### HRRR\[[13](https://arxiv.org/html/2606.30821#bib.bib42)\]

We used3×3​km3\\times 3\\,\\mathrm\{km\}scale HRRR pressure data for every hour from 2018\-2021 using lead times and forecasts of up to 6 hours to fill gaps\.

![Refer to caption](https://arxiv.org/html/2606.30821v1/x7.png)Figure 7:Downscaling bias in ERA5 and HRRR has a positive relationship with RMSE\.We show three samples of upsampled ERA5 data and HRRR 100hPauu\-wind from our 2018\-2020 training set\. The left sample with high RMSE shows a clear mismatch in the center while the right sample with low RMSE generally looks the same, indicating that RMSE is a good estimator of input/output mismatch\.To create the combined dataset, we also added four sinusoidal\-embedding temporal input channels for the day and month\. Fig\.[7](https://arxiv.org/html/2606.30821#A1.F7)indicates the main challenge of the real\-world downscaling task with atmospheric data\. There is an inherent mismatch between the LR and HR data over the same region since they come from two different sources\. Table[3](https://arxiv.org/html/2606.30821#A1.T3)shows higher bias levels for lower pressure levels, especially in the v\-wind channel\. We hypothesize this is due to lower pressure levels \(bigger hPa\) being closer to the jet stream \(250 hPa\) which has strong wind magnitudes that could cause more sensor inaccuracies\.

Table 3:Channel\-wise statistics of downscaling bias\.

### A\.6Implementation Details

ReMatch is not tied to a specific architecture\. It only assumes a deterministic mean predictor followed by a stochastic residual generator, so the mean predictor can be instantiated with any suitable pointwise estimator and the residual stage with any stochastic or ensemble\-based generative model\. In our experiments, we use U\-Net regression and SwinIR\[[28](https://arxiv.org/html/2606.30821#bib.bib45)\]as deterministic mean predictors, trained withℓ2\\ell\_\{2\}andℓ1\\ell\_\{1\}losses, respectively\. For stochastic residual prediction, we use EDM\-style diffusion preconditioning and sampling\[[21](https://arxiv.org/html/2606.30821#bib.bib47)\]with a U\-Net\-based score network\[[41](https://arxiv.org/html/2606.30821#bib.bib48)\]\. The low\-resolution inputxxand mean fieldμ\\mu, either transported pseudo\-mean from training setμ~i\\tilde\{\\mu\}\_\{i\}or mean predictor output in calibration setμϕ​\(xj\)\\mu\_\{\\phi\}\(x\_\{j\}\), are concatenated channel\-wise as the conditioning input\. Other stochastic residual generators can be used in the same framework\.

Unless otherwise noted, all UNet\-based regression and diffusion models are trained under the same implementation settings to ensure a fair comparison\. We use data\-parallel training on four NVIDIA A100 GPUs with a per\-GPU batch size of 32 for experiments, in which training takes about 8\-10 hours\. All regression UNets and diffusion models are trained for 8M processed samples, following the training\-duration convention used in the CorrDiff implementation at[https://github\.com/NVIDIA/physicsnemo/blob/main/examples/weather/corrdiff/README\.md](https://github.com/NVIDIA/physicsnemo/blob/main/examples/weather/corrdiff/README.md)\.

For the CorrDiff diffusion backbone, we instantiateEDMPrecondSuperResolutionwith aSongUNetPosEmbddenoising network\. Theminimodel\-size preset controls the capacity of the underlying U\-Net through three parameters:model\_channels,channel\_mult, andattn\_resolutions\. Specifically,model\_channels=64 sets the base channel width;channel\_mult=\[1,2,2\]\[1,2,2\]defines a three\-level encoder–decoder hierarchy with feature widths64,128,12864,128,128; andattn\_resolutions=\[16\]\[16\]inserts self\-attention blocks at the16×1616\\times 16feature\-map resolution\. All other architectural choices follow the default PhysicsNeMo CorrDiff settings\.

##### CorrDiff\[[32](https://arxiv.org/html/2606.30821#bib.bib2)\]

A two\-stage mean–residual baseline consisting of a regression mean predictor followed by a diffusion model for stochastic residual prediction\. We also evaluateCorrDiffm, which uses a reduced\-capacity mean predictor to mitigate residual target distribution shift\. We setmodel\_channels=1616,channel\_mult=\[1,2\]\[1,2\], andattn\_resolutions=\[4\]\[4\]and 3M total processed samples\.

##### UNet\[[41](https://arxiv.org/html/2606.30821#bib.bib48)\]

A deterministic regression baseline to map the LR input directly to the HR output\.

##### Conditional Diffusion Model \(CDM\)

A diffusion model conditioned on the LR input without an upstream mean predictor\.

##### CFG\[[42](https://arxiv.org/html/2606.30821#bib.bib43)\]

Classifier\-free guidance applied only to the residual diffusion model in CorrDiff to sharpen residual samples and mitigate small\-scale underdispersion\. We implement classifier\-free guidance \(CFG\) on top of the CorrDiff architecture as a baseline\. CFG allows conditional sampling without training an additional classifier\. During training for diffusion, we zero out the conditioning on the LR input 10% of the time so the model learns both conditional and unconditional inputs\. During inference, we use 18 denoising steps in the stochastic sampler with a cosine guidance weighting schedule fromw=w=0 to 8\. Comparing this to fixed weights from 0 to 8, we found that scheduling increasing guidance helped maintain RMSE skill while improving spread\. The sample at each time step is then a linear combination of the conditional and unconditional output:J\(xt\|∅\]\)\+w\[J\(xt\|yl​r\)−J\(xt\|∅\)\]J\(x\_\{t\}\|\\emptyset\]\)\+w\[J\(x\_\{t\}\|y\_\{lr\}\)\-J\(x\_\{t\}\|\\emptyset\)\]This encourages the model to learn features induced specifically by the conditioning\.

##### UC\[[22](https://arxiv.org/html/2606.30821#bib.bib54)\]

A CorrDiff variant that conditions the residual diffusion model on an uncertainty estimate of the regression mean, predicted by a lightweight UNet\. We consider two variants:UCμ\\mathrm\{UC\}\_\{\\mu\}, which conditions on the predicted MAE, andUCq\\mathrm\{UC\}\_\{q\}, which conditions on the predicted lower and upper error quantiles,q5q\_\{5\}andq95q\_\{95\}\. To construct these baselines, we first train a deterministic mean predictor on the 2018–2019 training set\. We then train a separate uncertainty estimator on the 2020 calibration set to predict either the MAE or theq5/q95q\_\{5\}/q\_\{95\}error quantiles of the mean predictor\. The uncertainty estimator uses the same UNet to the regression mean and is trained for 8M samples\. Using the calibration set as a proxy for the test regime allows the uncertainty network to learn held\-out error statistics, rather than relying on training\-time errors that may be overconfident\.

- •UCμ\\mathrm\{UC\}\_\{\\mu\}: The network predicts a scalar RMSE value of the residuals between the ground truth and the mean predictor\. Uses L2 loss\.
- •UCq\\mathrm\{UC\}\_\{q\}: The network predicts scalar quantile values for Q5 and Q95\. Uses Pinball loss\.

After training the mean predictor and uncertainty estimator sequentially, we train the residual diffusion model conditioned on the LR input, the predicted mean, and an uncertainty measure\. During diffusion training, we use the ground\-truth uncertainty measure computed from the mean\-prediction error, while at test time we replace it with the output of the uncertainty estimator\. This choice stabilizes diffusion training by providing accurate uncertainty conditioning during optimization, while preserving a fully predictive pipeline at inference\. The residual diffusion model is trained on the combined 2018–2020 data, including both the training and calibration splits\.

##### SwinIR\[[28](https://arxiv.org/html/2606.30821#bib.bib45)\]

A deterministic transformer\-based super\-resolution baseline, which directly maps the LR input to the HR field\. We implement a SwinIR\-based super\-resolution baseline following the network configuration used in the original SwinIR model at[https://github\.com/JingyunLiang/SwinIR](https://github.com/JingyunLiang/SwinIR)\. For the network architecture, we use the main SwinIR configuration from the original paper: the embedding dimension is set to 180, the model contains six Residual Swin Transformer Blocks \(RSTBs\), and each RSTB contains six Swin Transformer layers\. The number of attention heads is set to 6 in each RSTB, the MLP ratio is set to 2, and the residual connection inside each RSTB uses a single3×33\\times 3convolution\. We also use the standard multi\-step PixelShuffle reconstruction head with an intermediate feature dimension of 64\. The only modification from the original SwinIR parameter setting is the window size\. While the original model uses a window size of 8 for super\-resolution, we set the window size to 7 so that the self\-attention windows exactly divide the low\-resolution input size of21×2121\\times 21\. The model is trained using aℓ1\\ell\_\{1\}loss between the predicted and ground\-truth high\-resolution fields\. We train the SwinIR baseline for 100k processed samples, at which point the validation loss is observed to have converged\.

##### Conv\-FNO\[[49](https://arxiv.org/html/2606.30821#bib.bib46)\]

Our main modifications were replacing FNO’s decoder with an Upsampler block and adding a ResidualBlock to the spectral layers in FNO2DEncoder\. We use 34 features, 34 latent channels, 32 spectral and residual layers, kernel size of 3, 2 modes, a padding of 6, and a 8x upscaling factor\. These parameters were chosen to match the implementation in BlastNet to remain similar to an established super\-resolution baseline\. The model was trained using MSE loss between the predicted and ground\-truth high\-resolution fields\. We train the Conv\-FNO baseline for 8 million processed samples, similar to the CorrDiff regression capacity\.

### A\.7Additional Analysis on Real\-World Wind Field Downscaling Task

#### A\.7\.1Method Comparisons

We provide additional qualitative examples of downscaling results for all baseline models and the proposed method\.

![Refer to caption](https://arxiv.org/html/2606.30821v1/x8.png)\(a\)Qualitative comparison of downscaled 125 hPauu\-wind fields on 2021\-07\-20 00:00:00\.
![Refer to caption](https://arxiv.org/html/2606.30821v1/x9.png)\(b\)Qualitative comparison of downscaled 75 hPavv\-wind fields on 2021\-09\-14 01:00:00\.
![Refer to caption](https://arxiv.org/html/2606.30821v1/x10.png)\(c\)Qualitative comparison of downscaled 125 hPauu\-wind fields on 2021\-07\-06 19:00:00\.
![Refer to caption](https://arxiv.org/html/2606.30821v1/x11.png)\(d\)Qualitative comparison of downscaled 150 hPauu\-wind fields on 2021\-10\-19 7:00:00\.

Figure 8:Qualitative comparison of all baseline and proposed methods across randomly selected channels and dates\.

### A\.8Ensemble Samples Comparison

We further analyze individual ensemble samples\. Figure[3](https://arxiv.org/html/2606.30821#S5.F3)shows downscaling results for the 50 hPavv\-wind field on 2021\-09\-05 at 10:00:00\. In Figure[9](https://arxiv.org/html/2606.30821#A1.F9), we show all 12 ensemble samples of the final predictions,μϕ​\(x\)\+rθ​\(x,μϕ\)\\mu\_\{\\phi\}\(x\)\+r\_\{\\theta\}\(x,\\mu\_\{\\phi\}\), together with their predicted residualsrθ​\(x,μϕ\)r\_\{\\theta\}\(x,\\mu\_\{\\phi\}\)\.

![Refer to caption](https://arxiv.org/html/2606.30821v1/x12.png)\(a\)CorrDiff all prediction ensemble samples\.
![Refer to caption](https://arxiv.org/html/2606.30821v1/x13.png)\(b\)ReMatchUall prediction ensemble samples\.
![Refer to caption](https://arxiv.org/html/2606.30821v1/x14.png)\(c\)CorrDiff all residual ensemble samples\.
![Refer to caption](https://arxiv.org/html/2606.30821v1/x15.png)\(d\)ReMatchUall residual ensemble samples\.

Figure 9:Ensemble predictions from Fig\.[3](https://arxiv.org/html/2606.30821#S5.F3)\.Consistent with prior observations\[[32](https://arxiv.org/html/2606.30821#bib.bib2),[14](https://arxiv.org/html/2606.30821#bib.bib3)\]and our quantitative results, CorrDiff produces underdispersed predictions with a low SSR\. This behavior is visible in Figure[9](https://arxiv.org/html/2606.30821#A1.F9): CorrDiff ensemble samples remain highly similar across members, as shown in Figure[9\(a\)](https://arxiv.org/html/2606.30821#A1.F9.sf1), and their residual magnitudes are relatively small, as shown in Figure[9\(c\)](https://arxiv.org/html/2606.30821#A1.F9.sf3), compared with ReMatch residuals in Figure[9\(d\)](https://arxiv.org/html/2606.30821#A1.F9.sf4)\. This suggests that CorrDiff fails to generate sufficiently corrective residuals under residual misspecification\. Given the error\-prone mean prediction shown by the UNet result in Figure[3](https://arxiv.org/html/2606.30821#S5.F3), the residual model should correct large\-scale mean errors rather than merely add high\-frequency details\. However, CorrDiff largely fails to do so, whereas ReMatch produces more diverse and corrective residuals\.

### A\.9ACC Score analysis

![Refer to caption](https://arxiv.org/html/2606.30821v1/x16.png)Figure 10:Comparison of ReMatchUACC and input/output bias per month\.ACC score per month calculated for 2021 and average bias \(RMSE\) per month calculated over 2018\-2021\.With the ACC Score calculated across 2021, we can compare how spatially similar ReMatchUanomalies and HRRR anomalies are with the estimated climatological mean from 2018\-2020 HRRR data\. Anomalies are when the atmospheric variable deviates from the expected historical value at a certain time and location\. We note that having a longer range of years to estimate the climatological mean would likely lead to more accurate estimates of anomalies\. Fig\.[10](https://arxiv.org/html/2606.30821#A1.F10)shows a monthly breakdown of downscaling bias and captures how our model adapts to the bias gap between ERA5 and HRRR\. In winter months the bias is higher, but our model maintains high ACC scores, so ReMatchUis able to capture anomaly structure\.

### A\.10Kinetic Energy Analysis

![Refer to caption](https://arxiv.org/html/2606.30821v1/x17.png)Figure 11:Power spectra for kinetic energy in all pressure levelsFigure[11](https://arxiv.org/html/2606.30821#A1.F11)compares the 1D kinetic energy spectra of the ground\-truth HRRR field, the ERA5 input, CorrDiff predictions, and ReMatch predictions\. This diagnostic measures how kinetic energy is distributed across spatial scales: low wavenumbers correspond to large\-scale structure, whereas high wavenumbers correspond to finer\-scale variability\. A successful downscaling model should therefore not only reduce pointwise error, but also recover the target spectrum over the relevant range of wavenumbers\.

To compute this quantity, we form the horizontal kinetic energy from the wind components and evaluate its one\-dimensional Fourier spectrum\. For each sample, we compute the Fourier coefficients of the horizontal wind componentsuuandvv, and form

E​\(k\)∝12​\(\|u^​\(k\)\|2\+\|v^​\(k\)\|2\)\.E\(k\)\\propto\\frac\{1\}\{2\}\\Big\(\|\\hat\{u\}\(k\)\|^\{2\}\+\|\\hat\{v\}\(k\)\|^\{2\}\\Big\)\.We average over the remaining spatial dimension and over samples\. For stochastic methods, the spectrum is computed separately for each ensemble member and then averaged across ensemble members\.

The figure shows that ERA5 overestimates kinetic energy at all wavenumbers relative to HRRR, indicating a scale\-dependent bias\. ReMatch removes a substantially larger fraction of this excess power and tracks the HRRR spectrum more closely over a broad range of wavenumbers\. The gap between CorrDiff and ReMatch is most visible at higher wavenumbers, where CorrDiff preserves more of the excessive energy inherited from ERA5, whereas ReMatch remains closer to the HRRR target\.

This pattern differs from surface\-level wind fields shown in\[[32](https://arxiv.org/html/2606.30821#bib.bib2)\], where HRRR often contains much stronger high\-frequency variability due to land–sea contrast, topography, and other surface effects\. At pressure levels, the flow is smoother, so even the HRRR target has relatively weak high\-frequency energy\. In this regime, the main issue is therefore not only missing fine\-scale structure, but a broader mismatch in scale\-dependent energy\.

### A\.11Uncertainty Conditioned Experiments

Previous work on image regression\[[2](https://arxiv.org/html/2606.30821#bib.bib55),[22](https://arxiv.org/html/2606.30821#bib.bib54)\]often learns pixelwise epistemic or aleatoric uncertainty jointly with the regression predictor\. In our setting, we adapt this idea to the mean–residual CorrDiff architecture by training an auxiliary model to predict uncertainty about the upstream mean and using that quantity as an additional conditioning signal for residual diffusion\. Empirically, however, dense pixelwise uncertainty maps did not learn meaningful or predictive structure, whereas low\-dimensional scalar summaries of the regression error—such as RMSE,Q5Q\_\{5\},Q90Q\_\{90\}, andQ95Q\_\{95\}—were much easier to predict and matched the ground\-truth values more reliably on the test set \(Fig\.[12](https://arxiv.org/html/2606.30821#A1.F12)\)\. We explored two other spatially focused methods called Moran’s I \(a local indicator of spatial autocorrelation\) and VGGT\[[47](https://arxiv.org/html/2606.30821#bib.bib56)\]inspired confidence maps\. While the trained networks provided more meaningful feature maps[13](https://arxiv.org/html/2606.30821#A1.F13), they ultimately did not perform as well as conditioning to diffusion\.

We interpret this through the lens of residual target misspecification\. Under downscaling bias, the regression errorY−μϕ​\(X\)Y\-\\mu\_\{\\phi\}\(X\)is itself a predictor\-defined correction target whose distribution shifts across regimes\. Learning a full pixelwise uncertainty field therefore amounts to inferring a high\-dimensional summary of a misspecified residual target from limited observable information, which is substantially more difficult than predicting a coarse scalar descriptor of its magnitude\. This explains why scalar uncertainty signals can still provide useful information to the diffusion model, as reflected in the UC baselines, while dense pixelwise uncertainty prediction remains unreliable\. More broadly, these results suggest that uncertainty prediction alone is not sufficient to resolve residual misspecification: compressing the error into low\-dimensional summaries may help, but recovering its full spatial structure requires a more direct treatment of the shifted residual target itself\.

![Refer to caption](https://arxiv.org/html/2606.30821v1/x18.png)Figure 12:Comparison of scalar and pixelwise uncertainty estimators\.Both networks predict Q5 and Q95 of the residuals per channel on the test set\. The top plot is a scalar prediction while the bottom plot displays a pixelwise prediction at 2021\-01\-02 11:00:00 at 50 hPauu\-wind field\.![Refer to caption](https://arxiv.org/html/2606.30821v1/x19.png)Figure 13:Comparison of VGGT inspired \(left\) and Moran’s I \(right\) pixelwise uncertainty networks\.Both samples are also from 2021\-01\-02 11:00:00 at 50 hPauu\-wind field\. The Moran’s I uncertainty focuses more on "interest points" while the VGGT inspired confidence is more spatially aware\. However, the VGGT inspired confidence has systematic artifacts\.