FairRSFM: A Biome-Aware Benchmark and Debiasing Framework for Remote Sensing Foundation Models

Hugging Face Daily Papers Papers
remote-sensing foundation-models benchmark bias-mitigation geospatial group-robustness

Summary

The paper introduces FairRSFM, a biome-aware benchmark and debiasing framework that reveals ecological performance disparities in remote-sensing foundation models, evaluating Prithvi-EO-2.0, SatMAE, and DOFA across four downstream datasets alongside mitigation baselines like BOLP, Dynamic Biome Reweighting, and GroupDRO.

Remote sensing foundation models (RSFMs) are commonly evaluated using aggregate metrics, which can hide systematic performance disparities across ecological regions. We introduce FairRSFM, a biome-aware benchmark for evaluating ecological group robustness in RSFMs. FairRSFM maps georeferenced samples from 14 terrestrial biome classes into six ecologically meaningful macro-groups and evaluates models under a unified frozen-backbone evaluation protocol. The benchmark covers four downstream datasets: m-EuroSAT, m-BigEarthNet, m-SA-Crop-Type, and MMEarth20K with Dynamic World label maps. Using Prithvi-EO-2.0, SatMAE, and DOFA across three random seeds, we show that aggregate performance consistently masks biome-dependent disparities across architectures and tasks. For example, Prithvi-EO-2.0 reaches 90.98% overall macro-F1 on m-EuroSAT but a mean worst-group score of only 83.72%, while m-SA-Crop-Type drops from 27.30% overall mIoU to 18.47% in the Xeric and Mineralogical group. We further evaluate Biome-Orthogonal Linear Probing (BOLP), Dynamic Biome Reweighting (DBR), and GroupDRO as complementary mitigation baselines. Their effectiveness is model- and task-dependent; for example, BOLP improves Prithvi-EO-2.0 worst-group F1@opt on m-BigEarthNet from 46.12% to 50.27% without updating the RSFM backbone. FairRSFM provides a reusable protocol for diagnosing and mitigating ecological robustness gaps in remote sensing foundation models. Code and datasets are available at: https://github.com/aminurhossain/FairRSFM.
Original Article
View Cached Full Text

Cached at: 10/06/26, 04:56 PM

Paper page - FairRSFM: A Biome-Aware Benchmark and Debiasing Framework for Remote Sensing Foundation Models

Source: https://huggingface.co/papers/2610.05790

Abstract

Remotesensingfoundationmodels(RSFMs)arecommonlyevaluatedusingaggregatemetrics,whichcanhidesystematicperformancedisparitiesacrossecologicalregions.WeintroduceFairRSFM,abiome-awarebenchmarkforevaluatingecologicalgrouprobustnessinRSFMs.FairRSFMmapsgeoreferencedsamplesfrom14terrestrialbiomeclassesintosixecologicallymeaningfulmacro-groupsandevaluatesmodelsunderaunifiedfrozen-backboneevaluationprotocol.Thebenchmarkcoversfourdownstreamdatasets:m-EuroSAT,m-BigEarthNet,m-SA-Crop-Type,andMMEarth20KwithDynamicWorldlabelmaps.UsingPrithvi-EO-2.0,SatMAE,andDOFAacrossthreerandomseeds,weshowthataggregateperformanceconsistentlymasksbiome-dependentdisparitiesacrossarchitecturesandtasks.Forexample,Prithvi-EO-2.0reaches90.98%overallmacro-F1onm-EuroSATbutameanworst-groupscoreofonly83.72%,whilem-SA-Crop-Typedropsfrom27.30%overallmIoUto18.47%intheXericandMineralogicalgroup.WefurtherevaluateBiome-OrthogonalLinearProbing(BOLP),DynamicBiomeReweighting(DBR),andGroupDROascomplementarymitigationbaselines.Theireffectivenessismodel-andtask-dependent;forexample,BOLPimprovesPrithvi-EO-2.0worst-groupF1@optonm-BigEarthNetfrom46.12%to50.27%withoutupdatingtheRSFMbackbone.FairRSFMprovidesareusableprotocolfordiagnosingandmitigatingecologicalrobustnessgapsinremotesensingfoundationmodels.Codeanddatasetsareavailableat:https://github.com/aminurhossain/FairRSFM.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2610\.05790

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.05790 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.05790 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.05790 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Scalable and Trustworthy Earth Observation Foundation Models

arXiv cs.LG

This chapter reviews design principles and current landscape of foundation models for Earth observation, highlighting the need for domain-specific adaptation, physically plausible representations, and consistent evaluation benchmarks. It includes case studies on harmful algal bloom prediction and adaptive monitoring station selection.

BehaviorBench: Benchmarking Foundation Models for Behavioral Science Tasks

arXiv cs.CL

This paper introduces BehaviorBench, a comprehensive benchmark for evaluating foundation models on behavioral science tasks including behavior prediction, strategic decision-making, subject-trait inference, and behavioral knowledge application. It also presents Be.FM-1.5, a fine-tuned model that achieves strong distributional alignment, highlighting the gap between general-purpose and behaviorally adapted models.

Introducing the FFASR Leaderboard: Benchmarking ASR in the Real World

Hugging Face Blog

Introduces the FFASR Leaderboard, an open, community-driven benchmark for evaluating automatic speech recognition models under realistic far-field acoustic conditions, highlighting the significant performance gap between near-field and far-field scenarios.

GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models

arXiv cs.AI

The paper introduces GeoNatureAgent Benchmark, the first benchmark for evaluating LLM agents on environmental geospatial analysis tasks via structured tool calls. It evaluates seven models on 93 tasks across 18 categories and finds Claude Sonnet 4 achieves highest accuracy at 60.8%, while open-weight models like DeepSeek V3.2 offer strong cost-performance tradeoffs.