PROLA: Principal-Orthogonal Low-rank Adaptation for Predictive Spatiotemporal Weather Downscaling
Summary
The paper proposes PROLA, a principal-orthogonal low-rank adaptation method that allocates a fixed rank budget between pretrained weights' principal directions and their orthogonal complement, and introduces a Predictive Spatiotemporal Weather Downscaling task where PROLA outperforms existing low-rank adaptation baselines across diverse datasets.
View Cached Full Text
Cached at: 11/19/26, 03:39 AM
Paper page - PROLA: Principal-Orthogonal Low-rank Adaptation for Predictive Spatiotemporal Weather Downscaling
Source: https://huggingface.co/papers/2611.11318
Abstract
High-resolutionweatherpredictionisimportantforresolvinglocalatmosphericpatternsmissedbycoarseglobalforecasts.Largepretrainedweatherbackboneshaveimprovedglobalforecasting,butadaptingthemtoproducefuturehigh-resolutionfieldsfromrecentcoarsestatesremainschallengingandcostly.WeformulatethissettingasPredictiveSpatiotemporalWeatherDownscaling(PSWD),ataskthatjointlyrefinesspatialandtemporalresolutionwhilepredictingfuturehigh-resolutiontrajectories.Tothisend,weproposeaframeworkthatdecomposespredictionintospatialandtemporalstagesusingasharedpretrainedbackbone,fixednumericalscaffolds,andlightweightresidualheads.Forbackboneadaptation,weintroducePROLA(PRincipal-OrthogonalLow-rankAdaptation),whichallocatesafixedlow-rankbudgetbetweenpretrainedweights’principaldirectionsandtheirorthogonalcomplement.PROLAfurtherrescalesrank-1optimizerupdatesusinggradientsignals,improvingadaptationwithoutincreasingtrainablerank.ExperimentsinthePSWDsettingshowthatPROLAoutperformsrepresentativelow-rankadaptationbaselines.FurtherexperimentsacrossdiversedatasetsandtaskssupportthebroaderapplicabilityofPROLA,showingthatitsbenefitsextendbeyondtheprimaryweatherdownscalingsetting.
View arXiv pageView PDFGitHub1Add to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2611.11318 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2611.11318 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2611.11318 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
PaLoRA: Paced Low-Rank Adaptation for Continual Learning
PaLoRA introduces a paced low-rank adaptation method for LoRA-based continual learning, deriving an optimal adaptive pacing law that scales gradient step magnitude with the effective rank of accumulated knowledge. The method combines adaptive SVD truncation, null-space gradient projection, and rank-aware pacing, achieving consistent gains including 4% accuracy improvements on 50-task ImageNet-A and ImageNet-R benchmarks.
SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling
SOS-LoRA extends LoRA by decomposing the rank budget into static orthogonal low-rank experts with fixed multi-scale scaling, improving fine-tuning on reasoning, NLU, and math benchmarks without adding inference cost.
Normalized Low-Rank Adaptation
Normalized Low-Rank Adaptation (NoRA) stabilizes LoRA training by normalizing down-projection matrices, accelerating convergence and improving performance without extra parameters or inference cost.
Automatic Rank Allocation for Low-Rank Adaptation in Large Language Models via lp Regularization
This paper introduces ℓp-LoRA, a principled method for automatic rank allocation in low-rank adaptation using ℓp regularization, demonstrating competitive performance on NLP tasks.
BaLoRA: Bayesian Low-Rank Adaptation of Large Scale Models
BaLoRA introduces a Bayesian extension to Low-Rank Adaptation (LoRA) that provides calibrated uncertainty estimates and improves prediction accuracy by narrowing the gap with full fine-tuning.