PROLA: Principal-Orthogonal Low-rank Adaptation for Predictive Spatiotemporal Weather Downscaling

Hugging Face Daily Papers Papers

Summary

The paper proposes PROLA, a principal-orthogonal low-rank adaptation method that allocates a fixed rank budget between pretrained weights' principal directions and their orthogonal complement, and introduces a Predictive Spatiotemporal Weather Downscaling task where PROLA outperforms existing low-rank adaptation baselines across diverse datasets.

High-resolution weather prediction is important for resolving local atmospheric patterns missed by coarse global forecasts. Large pretrained weather backbones have improved global forecasting, but adapting them to produce future high-resolution fields from recent coarse states remains challenging and costly. We formulate this setting as Predictive Spatiotemporal Weather Downscaling (PSWD), a task that jointly refines spatial and temporal resolution while predicting future high-resolution trajectories. To this end, we propose a framework that decomposes prediction into spatial and temporal stages using a shared pretrained backbone, fixed numerical scaffolds, and lightweight residual heads. For backbone adaptation, we introduce PROLA (PRincipal-Orthogonal Low-rank Adaptation), which allocates a fixed low-rank budget between pretrained weights' principal directions and their orthogonal complement. PROLA further rescales rank-1 optimizer updates using gradient signals, improving adaptation without increasing trainable rank. Experiments in the PSWD setting show that PROLA outperforms representative low-rank adaptation baselines. Further experiments across diverse datasets and tasks support the broader applicability of PROLA, showing that its benefits extend beyond the primary weather downscaling setting.
Original Article
View Cached Full Text

Cached at: 11/19/26, 03:39 AM

Paper page - PROLA: Principal-Orthogonal Low-rank Adaptation for Predictive Spatiotemporal Weather Downscaling

Source: https://huggingface.co/papers/2611.11318

Abstract

High-resolutionweatherpredictionisimportantforresolvinglocalatmosphericpatternsmissedbycoarseglobalforecasts.Largepretrainedweatherbackboneshaveimprovedglobalforecasting,butadaptingthemtoproducefuturehigh-resolutionfieldsfromrecentcoarsestatesremainschallengingandcostly.WeformulatethissettingasPredictiveSpatiotemporalWeatherDownscaling(PSWD),ataskthatjointlyrefinesspatialandtemporalresolutionwhilepredictingfuturehigh-resolutiontrajectories.Tothisend,weproposeaframeworkthatdecomposespredictionintospatialandtemporalstagesusingasharedpretrainedbackbone,fixednumericalscaffolds,andlightweightresidualheads.Forbackboneadaptation,weintroducePROLA(PRincipal-OrthogonalLow-rankAdaptation),whichallocatesafixedlow-rankbudgetbetweenpretrainedweights’principaldirectionsandtheirorthogonalcomplement.PROLAfurtherrescalesrank-1optimizerupdatesusinggradientsignals,improvingadaptationwithoutincreasingtrainablerank.ExperimentsinthePSWDsettingshowthatPROLAoutperformsrepresentativelow-rankadaptationbaselines.FurtherexperimentsacrossdiversedatasetsandtaskssupportthebroaderapplicabilityofPROLA,showingthatitsbenefitsextendbeyondtheprimaryweatherdownscalingsetting.

View arXiv pageView PDFGitHub1Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2611.11318 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2611.11318 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2611.11318 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

PaLoRA: Paced Low-Rank Adaptation for Continual Learning

Hugging Face Daily Papers

PaLoRA introduces a paced low-rank adaptation method for LoRA-based continual learning, deriving an optimal adaptive pacing law that scales gradient step magnitude with the effective rank of accumulated knowledge. The method combines adaptive SVD truncation, null-space gradient projection, and rank-aware pacing, achieving consistent gains including 4% accuracy improvements on 50-task ImageNet-A and ImageNet-R benchmarks.

Normalized Low-Rank Adaptation

Hugging Face Daily Papers

Normalized Low-Rank Adaptation (NoRA) stabilizes LoRA training by normalizing down-projection matrices, accelerating convergence and improving performance without extra parameters or inference cost.