Exposing Blind Spots in Deep Imbalanced Regression Evaluation

# Exposing Blind Spots inDeep Imbalanced Regression Evaluation
Source: [https://arxiv.org/html/2609.25152](https://arxiv.org/html/2609.25152)
Noah C\. Puetz††thanks:Contact: noah\_christoph\.puetz@th\-koeln\.deJens U\. BrandtAffiliation:TH KölnAffiliation:Leiden UniversityMarc HilbertAffiliation:Leiden UniversityAffiliation:Toyota RacingElena RaponiAffiliation:Leiden UniversityThomas BäckAffiliation:Leiden UniversityThomas Bartz\-BeielsteinAffiliation:TH Köln

###### Abstract

Deep Imbalanced Regression \(DIR\) addresses a common failure mode of regression models: target distributions are highly non\-uniform, causing models to perform best in densely populated target regions even when reliable performance is required across the full target range\. Despite rapid methodological progress, DIR evaluation remains constrained by three blind spots: it is dominated by image\-based benchmarks, its standard many\-/medium\-/few\-shot protocol is diagnostic but not decision\-complete, and tail\-region stability across random seeds has not been systematically evaluated\. We revisit DIR evaluation along these three axes\. First, we broaden the data domain by evaluating DIR on a multimodal virtual sensing benchmark \(MuViS\) with nine time\-series extrinsic regression tasks across six physical domains, where rare target values often correspond to operationally meaningful regimes\. Second, we adopt balanced MAE \(*bMAE*\) and introduce balanced Mean Absolute Scaled Error \(*bMASE*\), a scale\-normalized metric for decision\-complete comparison across methods and datasets\. Third, through a repeated reevaluation of six representative DIR methods across multiple random seeds, we show that the tail regions targeted by DIR exhibit particularly high sensitivity to seed\-level variability\. Our results show that standard virtual\-sensing models exhibit substantial tail degradation hidden by global MAE, that existing DIR methods can improve balanced performance but transfer unevenly to multimodal time\-series data, and that tail\-region instability remains a largely hidden failure mode under current DIR evaluation practice\. Together, these findings and our publicly available code provide a reproducible basis for future DIR research toward regression systems that capture rare target regimes as reliably as common ones\.

![Refer to caption](https://arxiv.org/html/2609.25152v1/graphical_abstract.png)Figure 1:Motivating Example\. In heart\-rate estimation\[[1](https://arxiv.org/html/2609.25152#bib.bib33)\], the training distribution is strongly imbalanced \(left\), prediction quality degrades in rare but critical regimes \(middle\), and this failure is only weakly reflected by the global MAE \(right\)\.## 1Introduction

In many real\-world regression problems, the target variable is highly imbalanced\. Some regions of the target space are densely populated with training samples, whereas others, often the most extreme and practically most consequential values, are observed only rarely\[[2](https://arxiv.org/html/2609.25152#bib.bib39),[3](https://arxiv.org/html/2609.25152#bib.bib36)\]\. When trained on such data, standard deep learning models naturally achieve their best predictive accuracy in well\-supported regions of the training distribution and degrade in underrepresented ones\[[2](https://arxiv.org/html/2609.25152#bib.bib39),[4](https://arxiv.org/html/2609.25152#bib.bib34),[5](https://arxiv.org/html/2609.25152#bib.bib4)\]\. In many applications, however, predictive performance is required not only where data are abundant, but across the entire target range\[[6](https://arxiv.org/html/2609.25152#bib.bib43)\]\.[Figure 1](https://arxiv.org/html/2609.25152#S0.F1)provides such an example: In heart rate estimation most of the data is collected during normal operation of the human body\[[1](https://arxiv.org/html/2609.25152#bib.bib33)\], generating an imbalanced train distribution and leading to poor performance in the heart rate spikes, which ultimately results in worst performance in the most critical regimes\. In such a scenario, the imbalanced training distribution becomes misaligned with the desired evaluation objective: rare outcomes receive little influence during optimization, despite being equally important at test time\. Deep Imbalanced Regression \(DIR\)\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\]addresses this mismatch and characterizes it as an implicit target distribution shift\[[3](https://arxiv.org/html/2609.25152#bib.bib36)\]\.

Motivated by this problem, recent years have seen a growing body of DIR methods that aim to improve performance in under\-represented regions of the target space\[[3](https://arxiv.org/html/2609.25152#bib.bib36)\]\. However, despite this methodological progress, the existing DIR evaluation design still suffers from three important blind spots\. First, theData Blind Spot: While DIR claims to be data domain agnostic its empirical ecosystem is still overwhelmingly concentrated in computer vision, while other domains like time\-series settings remain largely absent \(see[Appendix C](https://arxiv.org/html/2609.25152#A3)\)\[[3](https://arxiv.org/html/2609.25152#bib.bib36)\]\. Second, theMetric Blind Spot: the established reporting practice in DIR is to partition the target space into many\-, medium\-, and few\-shot regions and report separate error values for each\[[2](https://arxiv.org/html/2609.25152#bib.bib39),[3](https://arxiv.org/html/2609.25152#bib.bib36)\]\. However, these partitions are arbitrary in continuous target spaces, and are difficult to interpret consistently across datasets\[[7](https://arxiv.org/html/2609.25152#bib.bib32)\]\. More importantly, they do not produce a single aggregate quantity for ranking methods\. As a result, comparisons remain partly subjective: a method may improve in underrepresented areas at the expense of well\-represented ones, yet the field lacks a principled way to assess whether this trade\-off is beneficial\. Third, theStability Blind Spot: current DIR evaluation pays limited attention to experimental stability\. Methods are frequently compared using single runs or only a small number of repetitions, even though sparse supervision in the distribution tails potentially makes these regions especially vulnerable to variance from random initialization and optimization noise\[[8](https://arxiv.org/html/2609.25152#bib.bib31)\]\. If DIR’s central claim is improved performance in underrepresented regions, then stability in those regions is part of what needs to be evaluated\.

In this work, we argue that meaningful DIR evaluation must go beyond the evaluation and reporting conventions that currently dominate the field\. Therefore aligned with the three blind spots and supported by our motivating example in[Figure 1](https://arxiv.org/html/2609.25152#S0.F1)we revisit DIR from three complementary angles:

1. 1\.Addressing theData Blind Spot, we broaden the DIR data domain toMuViS\(Multimodal Virtual Sensing Benchmark\)\[[9](https://arxiv.org/html/2609.25152#bib.bib35)\], a collection of multimodal time\-series extrinsic regression tasks specialized for virtual sensing and therefore characterized by a clear domain\-level requirement for reliable performance across the full target range\.
2. 2\.Addressing theMetric Blind Spot, we formalize a macro\-averaged, distribution\-invariant view of DIR evaluation throughbMAE\[[6](https://arxiv.org/html/2609.25152#bib.bib43)\]and introduce the novel, scale\-invariantbMASE\. We contrast the prevailing many\-/medium\-/few\-shot reporting with standard global regression metrics such as mean absolute error \(MAE\), as well as with these balanced alternatives, and show that the bMASE creates an objective cross\-method and cross\-dataset ranking, that is not doable with the current evaluation design\.
3. 3\.Addressing theStability Blind Spot, we run repeated experiments to probe seed\-level performance variability in DIR methods, focusing on whether underrepresented target regions exhibit disproportionately high performance variance\.

## 2Related Work

The problem of imbalanced training data is not new in deep learning, but most imbalance\-aware methods were developed for classification, where the output space is categorical and finite\[[10](https://arxiv.org/html/2609.25152#bib.bib30)\]\. Extending these ideas to continuous and potential infinite target spaces is non\-trivial\.[Yang et al\. \[2\]](https://arxiv.org/html/2609.25152#bib.bib39)introduced DIR as a distinct problem setting and showed that reweighting and resampling strategies transferred from classification perform poorly once the notion of a class boundary disappears\. Since then, DIR has developed into a substantial methodological literature, with existing methods broadly grouped into three families\[[3](https://arxiv.org/html/2609.25152#bib.bib36)\]:*algorithm\-level*methods that modify the loss or sample weighting \(LDS\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\], SqInv\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\], Focal\-L1L\_\{1\}\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\], Balanced MSE\[[6](https://arxiv.org/html/2609.25152#bib.bib43)\], DenseLoss\[[11](https://arxiv.org/html/2609.25152#bib.bib42)\], VIR\[[12](https://arxiv.org/html/2609.25152#bib.bib41)\], Dist Loss\[[13](https://arxiv.org/html/2609.25152#bib.bib44)\]\);*representation\-learning*methods that reshape the latent space \(FDS\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\], RankSim\[[14](https://arxiv.org/html/2609.25152#bib.bib50)\], Ordinal Entropy\[[15](https://arxiv.org/html/2609.25152#bib.bib37)\], ConR\[[16](https://arxiv.org/html/2609.25152#bib.bib48)\], RnC\[[17](https://arxiv.org/html/2609.25152#bib.bib38)\], Geometric Representation Constraints\[[18](https://arxiv.org/html/2609.25152#bib.bib29)\], PRIME\[[19](https://arxiv.org/html/2609.25152#bib.bib47)\]\); and*architectural*methods that modify the predictor itself \(UVote\[[20](https://arxiv.org/html/2609.25152#bib.bib49)\], HCA\[[21](https://arxiv.org/html/2609.25152#bib.bib40)\], Multi\-Classification\[[22](https://arxiv.org/html/2609.25152#bib.bib28)\], IM\-Context\[[23](https://arxiv.org/html/2609.25152#bib.bib45)\]\)\. Additionally, these methods can be divided into*orthogonal*approaches that compose with standard regression pipelines and*non\-orthogonal*approaches that require specific architectures or substantially restructured training\[[3](https://arxiv.org/html/2609.25152#bib.bib36)\]\.

Despite DIR’s methodological progress, its empirical basis remains narrow\. Across the fifteen DIR method papers surveyed by[Puetz et al\. \[3\]](https://arxiv.org/html/2609.25152#bib.bib36), fourteen paper evaluate on at least one image\-based benchmark, predominantlyAgeDB\-DIR\[[2](https://arxiv.org/html/2609.25152#bib.bib39),[24](https://arxiv.org/html/2609.25152#bib.bib26)\]orIMDB\-WIKI\-DIR\[[2](https://arxiv.org/html/2609.25152#bib.bib39),[25](https://arxiv.org/html/2609.25152#bib.bib27)\]; eight additionally evaluate on text regression \(STS\-B\-DIR\[[2](https://arxiv.org/html/2609.25152#bib.bib39),[26](https://arxiv.org/html/2609.25152#bib.bib24)\]\); four on synthetic or tabular data; and only three on time\-series data \(full table shown in[Appendix C](https://arxiv.org/html/2609.25152#A3)\)\. Moreover, each of these three evaluations use a different dataset with no overlap between papers \(SHHS\-DIR\[[27](https://arxiv.org/html/2609.25152#bib.bib25)\]in[Yang et al\. \[2\]](https://arxiv.org/html/2609.25152#bib.bib39),TUAB\[[28](https://arxiv.org/html/2609.25152#bib.bib23)\]in[Zha et al\. \[17\]](https://arxiv.org/html/2609.25152#bib.bib38), andECG\-K\-DIR\[[29](https://arxiv.org/html/2609.25152#bib.bib22)\]in[Nie et al\. \[13\]](https://arxiv.org/html/2609.25152#bib.bib44)\) which prevents direct cross\-method comparison\. None of the three releases a public training pipeline for its respective time\-series task, and two are difficult to reconcile with the continuous\-target, deep\-regression premise that motivates DIR:SHHS\-DIRprovides with1,8921\{,\}892a limited amount of training samples with only2121discrete target values, andTUABprovides1,2461\{,\}246samples with integer targets in\[0,95\]\[0,95\]\. As a result, current DIR evidence says far more about image benchmarks than about the broader class of imbalanced continuous prediction problems the field claims to address\. A more complete evaluation landscape should include data domains that differ structurally from images while preserving the core DIR challenge: continuous targets with sparsely supported but practically relevant regions\.

## 3Extending the Evaluation Framework of DIR

We extend DIR evaluation along the three blind spots identified in[Section 1](https://arxiv.org/html/2609.25152#S1)\. First, in[Subsection 3\.1](https://arxiv.org/html/2609.25152#S3.SS1)we broaden the data domain by evaluating on multimodal time\-series regression\. Second, in[Subsection 3\.2](https://arxiv.org/html/2609.25152#S3.SS2)we replace many\-/medium\-/few\-shot reporting with balanced scalar metrics that are aligned with continuous targets and enable direct comparison across datasets\. Third, we treat stability in underrepresented regions as an evaluation criterion and analyze it empirically in[Subsection 4\.3](https://arxiv.org/html/2609.25152#S4.SS3)\.

### 3\.1Benchmark Extension:MuViS\-DIR

![Refer to caption](https://arxiv.org/html/2609.25152v1/dataset_overview.png)Figure 2:Empirical target distributions of the nineMuViS\-DIRtasks, illustrating task\-specific target imbalance across continuous sensor\-like quantities\. Legends report sequence lengthTT, feature channelsDD, sample sizenn, skewness, and kurtosis\. Domain icons follow[Brandt et al\. \[9\]](https://arxiv.org/html/2609.25152#bib.bib35)\.To contribute to closing theData Blind Spot, we useMuViS\[[9](https://arxiv.org/html/2609.25152#bib.bib35)\], a multimodal virtual sensing benchmark with nine continuous regression tasks across six physical domains\. Virtual sensing denotes the data\-driven inference of hard\-to\-measure physical quantities from available sensor measurements\[[30](https://arxiv.org/html/2609.25152#bib.bib20),[31](https://arxiv.org/html/2609.25152#bib.bib21)\]\. The task is sequence\-to\-value regression, at the intersection of virtual sensing and multimodal dynamic time\-series learning\[[32](https://arxiv.org/html/2609.25152#bib.bib19)\], with an interface to time\-series extrinsic regression \(TSER\)\[[33](https://arxiv.org/html/2609.25152#bib.bib18)\]\. Combining DIR andMuViSyields the following problem setting forMuViS\-DIR:

##### Problem setting ofMuViS\-DIR:

We consider a dataset𝒟=\{\(𝒳i,yi\)\}i=1N\\mathcal\{D\}=\\\{\(\\mathcal\{X\}\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}, where each input𝒳i=\(xi1,…,xiM\)\\mathcal\{X\}\_\{i\}=\(x\_\{i\}^\{1\},\\ldots,x\_\{i\}^\{M\}\)consists ofMMmodality\-specific time series withxij∈ℝDj×Tjx\_\{i\}^\{j\}\\in\\mathbb\{R\}^\{D^\{j\}\\times T^\{j\}\}\. ModalityjjprovidesDjD^\{j\}feature channels overTjT^\{j\}time steps, andyi∈ℝy\_\{i\}\\in\\mathbb\{R\}denotes the continuous sensor\-like target valueyi​\(t0\)y\_\{i\}\(t\_\{0\}\)at reference timet0t\_\{0\}\. A deep modelfθf\_\{\\theta\}maps the multimodal sequential input to an estimatey^i=fθ​\(𝒳i\)\\hat\{y\}\_\{i\}=f\_\{\\theta\}\(\\mathcal\{X\}\_\{i\}\)\. In the DIR setting, the empirical target distributionptrain​\(y\)p\_\{\\mathrm\{train\}\}\(y\)is highly non\-uniform: some target intervals are well\-represented, whereas others contain few or no samples\. The evaluation objective, however, is to generalize to a more balanced desired evaluation distributionpeval​\(y\)p\_\{\\mathrm\{eval\}\}\(y\)\. This mismatch induces an implicit target\-distribution shift between training and evaluation, weakening the standard empirical\-risk\-minimization \(ERM\) assumption that training and evaluation samples are drawn from the same target distribution\.

MuViS\-DIRis a useful DIR testbed because it combines continuous targets, heterogeneous sequential inputs, and domain\-level reasons to care about rare target values\. In many virtual sensing tasks, underrepresented regions correspond to safety\-critical\[[34](https://arxiv.org/html/2609.25152#bib.bib17),[35](https://arxiv.org/html/2609.25152#bib.bib11)\], clinically relevant\[[1](https://arxiv.org/html/2609.25152#bib.bib33)\], or physically extreme operating\[[36](https://arxiv.org/html/2609.25152#bib.bib16),[5](https://arxiv.org/html/2609.25152#bib.bib4)\]regimes rather than merely rare labels\. Poor tail performance is therefore a practical failure mode of the trained model\. Figure[2](https://arxiv.org/html/2609.25152#S3.F2)visualizes the diverse target distributions used in our study; full dataset statistics, domain motivations for reliable performance across the target range, and imbalance summaries are reported in[Appendix C](https://arxiv.org/html/2609.25152#A3)\.

### 3\.2Metric Extension: Balanced Scalar Evaluation

Figure 3:Failure mode of many\-/medium\-/few\-shot reporting\. \(a\) Four synthetic predictors exhibit different per\-bin error profiles over the same imbalanced target distribution\. \(b\) Region\-wise MAE produces different rankings depending on the selected shot region and therefore does not provide a unique model comparison criterion\.The dominant DIR protocol of[Yang et al\. \[2\]](https://arxiv.org/html/2609.25152#bib.bib39)reports MAE separately over many\-, medium\-, and few\-shot regions, which creates two problems for continuous regression\. First, the thresholds are inherited from long\-tailed classification\[[7](https://arxiv.org/html/2609.25152#bib.bib32)\]\(few\-shot:<20<20samples, many\-shot:\>100\>100samples, medium\-shot: otherwise\) and are arbitrary once the target space is continuous\. Second, the protocol produces three region\-specific scores but no scalar ranking criterion\. A model may improve rare regions while degrading dense ones, yet the protocol provides no principled rule for deciding whether this trade\-off is favorable\. The following motivating example illustrates this ambiguity\.

##### Motivating example\.

[Figure 3](https://arxiv.org/html/2609.25152#S3.F3)\(a\) shows four synthetic predictors with different error profiles over the same imbalanced target distribution\. Model A resembles the typical ERM failure mode\[[3](https://arxiv.org/html/2609.25152#bib.bib36)\]: low error in the dense head and increasing error in the tails\. Model C is nearly uniform across the target range and is visually the most robust predictor under a balanced evaluation objective\. Model D exposes the ambiguity most clearly: it performs worst in the dense head but best in the medium\- and few\-shot regions, precisely the regimes DIR is meant to improve\. Depending on the evaluation preference, Model D can therefore be viewed either as a poor model that sacrifices common cases or as a strong DIR model that improves rare regimes \([Figure 3](https://arxiv.org/html/2609.25152#S3.F3)\(b\)\)\. The shot\-based protocol reports this trade\-off but does not resolve it\. A naive average over the three shot scores would introduce another arbitrary decision rule, ranking Model D ahead of Model B \(mean MAE: 2\.13 vs\. 2\.88\) without justifying how tail gains should be weighted against head degradation\. Thus, the protocol can be diagnostic but not decision\-complete: it reveals \(arbitrary\) regional behavior, yet it does not define an objective comparison criterion\.

Motivated by this limitation, and by the premise that DIR evaluation should reward models that perform consistently well across an idealized balancedpeval​\(y\)p\_\{\\mathrm\{eval\}\}\(y\)\[[3](https://arxiv.org/html/2609.25152#bib.bib36)\], we adopt balanced scalar metrics that weight target regions uniformly rather than by sample density\. Sinceyyis continuous, we follow[Ren et al\. \[6\]](https://arxiv.org/html/2609.25152#bib.bib43)and approximate uniform weighting over the target range by discretizing\[a,b\]\[a,b\]intoKKequal\-width bins;[Subsection D\.2](https://arxiv.org/html/2609.25152#A4.SS2)verifies the agreement between the continuous and discretized formulations for sufficiently large test sets\.

##### Balanced MAE\.

We macro\-average the per\-bin MAE over theKKequal\-width bins\[[6](https://arxiv.org/html/2609.25152#bib.bib43)\]:

bMAE=1K​∑k=1K1nk​∑i∈bink\|yi−y^i\|,\\mathrm\{bMAE\}\\;=\\;\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\,\\frac\{1\}\{n\_\{k\}\}\\\!\\sum\_\{i\\in\\mathrm\{bin\}\_\{k\}\}\\\!\|y\_\{i\}\-\\hat\{y\}\_\{i\}\|\\,,\(1\)wherenkn\_\{k\}is the number of test samples in binkk\. Every bin receives equal weight regardless of sample count, reducing the dependence on the test\-set density while preserving the interpretability of MAE in the original target units\.

##### Balanced MASE\.

While bMAE provides an interpretable absolute error, it is scale\-dependent and cannot be compared across datasets\. Because the nine datasets in our benchmark span fundamentally different physical quantities and scales, we introduce the balanced Mean Absolute Scaled Error \(bMASE, based on the mean absolute scaled error by[Hyndman and Koehler \[37\]](https://arxiv.org/html/2609.25152#bib.bib15)\), which normalises bMAE by the error of a trivial constant predictor:

bMASE=1K​∑k=1K1nk​∑i∈bink\|yi−y^i\|1K​∑k=1K1nk​∑i∈bink\|yi−cref\|=bMAE⁡\(fθ\)bMAE⁡\(cref\)\\mathrm\{bMASE\}\\;=\\;\\frac\{\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\,\\frac\{1\}\{n\_\{k\}\}\\\!\\sum\_\{i\\in\\mathrm\{bin\}\_\{k\}\}\\\!\|y\_\{i\}\-\\hat\{y\}\_\{i\}\|\}\{\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\,\\frac\{1\}\{n\_\{k\}\}\\\!\\sum\_\{i\\in\\mathrm\{bin\}\_\{k\}\}\\\!\|y\_\{i\}\-c\_\{\\mathrm\{ref\}\}\|\}\\;=\\;\\frac\{\\mathrm\{bMAE\}\(f\_\{\\theta\}\)\}\{\\mathrm\{bMAE\}\(c\_\{\\mathrm\{ref\}\}\)\}\(2\)wherecref=median⁡\(ytrain\)c\_\{\\mathrm\{ref\}\}=\\mathrm\{median\}\(y\_\{\\mathrm\{train\}\}\)is theL1L\_\{1\}\-optimal constant predictor\. AbMASE<1\\mathrm\{bMASE\}<1indicates improvement over the trivial predictor; values closer to00indicate stronger performance\. Becausecrefc\_\{\\mathrm\{ref\}\}normalises out the dataset\-specific error scale, bMASE values are directly comparable across datasets and can be aggregated via the geometric mean:

bMASE¯=\(∏d=1DbMAEd​\(fθ\)bMAEd​\(cref\)\)1D\\overline\{\\mathrm\{bMASE\}\}\\;=\\;\\Biggl\(\\,\\prod\_\{d=1\}^\{\\mathrm\{D\}\}\\,\\frac\{\\mathrm\{bMAE\}\_\{d\}\(f\_\{\\theta\}\)\}\{\\mathrm\{bMAE\}\_\{d\}\(c\_\{\\mathrm\{ref\}\}\)\}\\Biggr\)^\{\\frac\{1\}\{\\mathrm\{D\}\}\}\(3\)The geometric mean is the only aggregation of normalised ratios that is invariant to the choice of reference, i\.e\. the ranking is identical whether one normalises by the trivial predictor, the best model, or any other baseline\[[38](https://arxiv.org/html/2609.25152#bib.bib14)\]\.

We useL1L\_\{1\}\-based metrics because balanced evaluation gives equal weight to every target bin, including bins with very small sample counts\. In such bins, squared\-error metrics are especially unstable: per\-bin MSE depends on fourth\-order error moments, whereas MAE depends only on second\-order moments through the variance of\|ei\|\|e\_\{i\}\|, withei=yi−y^ie\_\{i\}=y\_\{i\}\-\\hat\{y\}\_\{i\}\[[39](https://arxiv.org/html/2609.25152#bib.bib13)\]\. Since metric stability in rare target regions is central to our setting, we use theL1L\_\{1\}\-based bMASE rather than balanced variants of RMSE orR2R^\{2\}\.

Both[Equation 1](https://arxiv.org/html/2609.25152#S3.E1)and[Equation 2](https://arxiv.org/html/2609.25152#S3.E2)require a target range\[a,b\]\[a,b\]and a number of binsKK\. We seta=min⁡\(ytest\)a=\\min\(y\_\{\\mathrm\{test\}\}\)andb=max⁡\(ytest\)b=\\max\(y\_\{\\mathrm\{test\}\}\), and chooseKKusing the Freedman–Diaconis rule\[[40](https://arxiv.org/html/2609.25152#bib.bib12)\],

wFD=2IQR\(ytest\)Ntest−1/3,K=⌈b−awFD⌉\.w\_\{\\mathrm\{FD\}\}=2\\,\\mathrm\{IQR\}\(y\_\{\\mathrm\{test\}\}\)\\,N\_\{\\mathrm\{test\}\}^\{\-1/3\},\\qquad K=\\left\\lceil\\frac\{b\-a\}\{w\_\{\\mathrm\{FD\}\}\}\\right\\rceil\.Thus, the binning is deterministic, dataset\-specific, shared by bMAE and bMASE, and not treated as a tunable hyperparameter\.

## 4Experiments

We organize the experiments around the three evaluation blind spots introduced in[Section 1](https://arxiv.org/html/2609.25152#S1)\. First, we test whether the DIR failure mode appears inMuViS\([Subsection 4\.1](https://arxiv.org/html/2609.25152#S4.SS1)\)\. Second, we evaluate whether established DIR methods improve balanced performance under bMAE and bMASE \([Subsection 4\.2](https://arxiv.org/html/2609.25152#S4.SS2)\)\. Third, we analyze whether model performance in the same low\-density regions is also less stable across random seeds \([Subsection 4\.3](https://arxiv.org/html/2609.25152#S4.SS3)\)\. Code for reproducing the experiments is available at[www\.github\.com/noah\-puetz/muvis\-dir](https://www.github.com/noah-puetz/muvis-dir)\.

### 4\.1Imbalance Bias in Standard Virtual Sensors

We first examine whetherMuViS\-DIRexhibits the imbalance bias that motivates DIR\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\]\. For each dataset, we train the dataset\-specificResNet1Darchitectures provided by[Brandt et al\. \[9\]](https://arxiv.org/html/2609.25152#bib.bib35)with anL1L\_\{1\}loss across ten random seeds\. This provides a baseline for the later DIR comparison while avoiding architectural confounds\. Results for the remainingMuViSbaselines under bMAE and bMASE are reported in[Subsection G\.1](https://arxiv.org/html/2609.25152#A7.SS1); among them,ResNet1Dobtains the best aggregate bMASE and is therefore used as the basis for the main analysis\.

![Refer to caption](https://arxiv.org/html/2609.25152v1/resnet_tail_bias_v2.png)Figure 4:Per\-bin MAE ofResNet1Dacross three representative tasks fromMuViS\[[9](https://arxiv.org/html/2609.25152#bib.bib35)\]\. Error increases sharply in sparse target regions; onPM10, the extreme upper tail exceeds the median\-predictor baseline\.[Figure 4](https://arxiv.org/html/2609.25152#S4.F4)shows that the core DIR failure mode is present inMuViS\. Across the representative tasks, per\-bin MAE increases in sparsely supported target regions, with the single exception of the per bin MAE onBattery SoC\.Battery SoChas the least imbalanced target support among the shown tasks, and its per\-bin MAE remains comparatively stable across the target range\. Table[1](https://arxiv.org/html/2609.25152#S4.T1)shows that the failure mode onPM10is largely hidden by global MAE\. Only when target regions are weighted equally by thebMAE, theResNet1Dobtains a4\.42×4\.42\\timesincrease\. This illustrates why balanced metrics across target range are practically important: predictions onPM10in the extreme upper tail,y\>600y\>600approach the performance of a naive median predictor, but these rare target ranges are also corresponding to critical air\-quality regimes which would trigger public health advisories\[[35](https://arxiv.org/html/2609.25152#bib.bib11)\]\. The MAE let the model appear competitive, yet it fails in the operating region where reliability is most consequential \(similar masking effect appears onPM2\.5andMonterey, see[Subsection G\.2](https://arxiv.org/html/2609.25152#A7.SS2)\)\.

The many\-/median\-/few\-shot protocol is less reliable in this setting\. Some datasets contain no bins satisfying a given shot criterion, producing undefined entries, and the fixed thresholds can still overweight denser subregions inside the nominal few\-shot range\. OnPM2\.5, the few\-shot region spans approximatelyy=300y=300toy=1000y=1000, but most samples within that region are concentrated near its lower end\. Consequently, few\-shot MAE underweights the most severe upper\-tail errors, while bMAE exposes them\. Thus, the shot\-based reporting, meant for reporting models sensitivity due to data imbalance, can itself remain sensitive to within\-region imbalance\.

Table 1:Contrasting classical MAE, many\-/median\-/few\-shot MAE, and bMAE for each dataset with its correspondingResNet1Dconfiguration\. Mean±\\pm95% bootstrap CI half\-width on the mean over 10 seeds\. The MAE vs\. bMAE ratio quantifies how strongly global MAE underestimates balanced target\-range error\. Larger values indicate stronger imbalance\-induced masking\.
### 4\.2DIR Methods under Balanced Evaluation

Table 2:Mean bMASE±\\pm95% bootstrap CI half\-width over 10 seeds for each method–dataset pair\. BelowVanillaare italicized; Best value in each row is bold\. GMean aggregates across datasets via geometric mean\. Lower is better;bMASE<1\\mathrm\{bMASE\}<1improves over the trivial median predictor\.We next ask whether established DIR methods can reduce this imbalance bias\. We evaluate six representative and state\-of\-the\-art methods \(based on the extrapolation benchmark from[Puetz et al\. \[3\]](https://arxiv.org/html/2609.25152#bib.bib36)\) covering the main DIR families from[Section 2](https://arxiv.org/html/2609.25152#S2): algorithm\-level methods, LDS, SQInv, and Focal\-L1L\_\{1\}; representation\-learning methods, ConR and RnC; and the architectural method UVote\. LDS, SQInv, Focal\-L1L\_\{1\}, and ConR are orthogonal interventions\. All methods are trained on top of the sameResNet1Dfrom[Subsection 4\.1](https://arxiv.org/html/2609.25152#S4.SS1), and each method–dataset pair is selected by a hyperparameter sweep \(explained in detail in[Subsection F\.4](https://arxiv.org/html/2609.25152#A6.SS4)\)\.

![Refer to caption](https://arxiv.org/html/2609.25152v1/dir_improvements_v2.png)Figure 5:Per\-bin improvements overVanilla\. Top row compares per\-bin MAE; bottom row shows the signed difference to the baseline\. \(Full Overview:[Figure 16](https://arxiv.org/html/2609.25152#A7.F16)\)[Table 2](https://arxiv.org/html/2609.25152#S4.T2)demonstrates the advantage of bMASE as a single scalar comparison criterion: it provides a cross\-method and cross\-dataset ranking that many\-/medium\-/few\-shot reporting cannot supply\. Four out of six methods improve the vanilla baseline, showing that imbalance correction transfers to multimodal virtual sensing\. However, the gains are less regular than in common computer\-vision DIR benchmarks\. UVote achieves the best aggregate score, reducing GMean bMASE from0\.21630\.2163to0\.17320\.1732, and is best on four datasets\. LDS is the strongest evaluated orthogonal method, reaching0\.19970\.1997GMean and producing the largest single improvement onPM10, from0\.68690\.6869to0\.47650\.4765\. SQInv and Focal\-L1L\_\{1\}also improve the aggregate score, whereas RnC substantially degrades performance\.

[Figure 5](https://arxiv.org/html/2609.25152#S4.F5)illustrates how these aggregate gains arise\. Focal\-L1L\_\{1\}improves high\-heart\-rate regions onPPGDaliaand UVote substantially reduces tail error onMontereywith both achieving smaller improvements across the whole respective target range\. In contrast LDS compromises in the well\-represented region for better tail\-performance onPM10\.

These results support two conclusions\. First, the relationship betweenMuViSand DIR is bidirectional:MuViSexposes how well DIR methods transfer to multimodal time\-series regression, while DIR methods yield meaningful gains in underrepresented sensor regimes\. Second, direct transfer from existing DIR methods remains incomplete\. Strong methods in image\-based DIR do not necessarily dominate multimodal time\-series regression, makingMuViS\-DIRa realistic benchmark for future method development\.

### 4\.3Tail Instability Across Seeds

![Refer to caption](https://arxiv.org/html/2609.25152v1/stability_density.png)Figure 6:Relationship between local target support and seed\-level instability\. Each point is one bin; thexx\-axis shows bin sample count \(←\\leftarrowless data\) and theyy\-axis shows the standard deviation of per\-bin MAE across ten seeds \(↑\\uparrowhigher instability\)\. Both axes are log\-scaled\. Gray denotesVanilla; red denotes the best\-performing DIR method according to[Table 2](https://arxiv.org/html/2609.25152#S4.T2)per dataset\.Finally, we evaluate theStability Blind Spot\. Since DIR methods are designed to improve low\-density regions, their performance should be reproducible precisely in those regions\. Motivated by prior work on random\-seed variability in deep learning\[[41](https://arxiv.org/html/2609.25152#bib.bib3),[42](https://arxiv.org/html/2609.25152#bib.bib2)\], we compute the standard deviation of per\-bin MAE across ten seeds for every method–dataset pair\. Rather than treating ten seeds as a sufficiency guarantee, we use this repeated\-run setting to test whether instability is already visible in the target regimes DIR aims to improve\.

Figure[6](https://arxiv.org/html/2609.25152#S4.F6)shows, that across datasets, vanillaResNet1Dbecomes less stable as local target support decreases, with Spearman correlations between bin count and per\-bin standard deviation ranging from−0\.57\-0\.57to−0\.95\-0\.95\. The same trend persists for the best\-performing DIR methods: each method either shows an even stronger density–instability relation or differs from vanilla by less than0\.050\.05correlation points\.

![Refer to caption](https://arxiv.org/html/2609.25152v1/monterey_seed_instability.png)Figure 7:Seed\-level instability for UVote onMonterey\. Each point corresponds to one seed’s performance in one target bin; sparse target regions exhibit visibly larger dispersion\.This confirms that the regions central to DIR are also the least stable under repeated training\. Importantly, improved mean performance does not appear to remove this instability\. OnMonterey, UVote substantially improves bMASE but retains a similar density\-dependent variance trend as the vanilla model: the trend starts lower, but still rises as target support decreases\. Figure[7](https://arxiv.org/html/2609.25152#S4.F7)visualizes this effect directly, showing that seed\-level variability remains compact in dense target regions but widens in sparse regions\. Additionally interesting: many method–dataset pairs exhibit an approximately linear relationship on log–log axes, suggesting an inverse power\-law\-like scaling between local target support and seed\-level variance\.

We do not propose a new stability remedy here\. Instead, we show that stability is a first\-order evaluation concern for DIR\. If the goal is to improve performance in underrepresented target regions, then reporting only average variance or from a single seed, is insufficient\. Tail accuracy and tail stability should be evaluated jointly\.

### 4\.4Discussion

The experiments address the three blind spots identified in this paper\.[Subsection 4\.1](https://arxiv.org/html/2609.25152#S4.SS1)establishesMuViS\-DIRas a natural extension of the DIR evaluation ecosystem: it provides multimodal time\-series tasks with physically meaningful tail regimes, publicly available code, and a benchmark structure suitable for evaluating future DIR methods\.[Subsection 4\.2](https://arxiv.org/html/2609.25152#S4.SS2)further shows that tail performance is improvable with existing DIR methods, but not consistently across the evaluated method set\. This positionsMuViS\-DIRnot as a solved application of DIR, but as an open challenge broadening the current data landscape and by this addressing theData Blind Spot\.

[Subsection 4\.2](https://arxiv.org/html/2609.25152#S4.SS2)also demonstrates the practical value of bMASE\. A single table is sufficient to compare methods across datasets, physical units, and imbalance regimes, while still preserving dataset\-level detail\. Most importantly, bMASE provides a scalar basis for ranking methods, addressing the ambiguity of many\-/median\-/few\-shot reporting and therefore theMetric Blind Spot\.

Finally,[Subsection 4\.3](https://arxiv.org/html/2609.25152#S4.SS3)directly addresses theStability Blind Spotin the current DIR evaluation ecosystem and confirms that performance instability across seeds in underrepresented regions is not only present but amplified\. We do not claim this as a new optimization phenomenon; rather, we show that it has been largely absent from DIR evaluation despite being central to the problem\.

## 5Limitations and Future Work

This study has several limitations\. First, whileMuViS\-DIRbroadens DIR evaluation beyond image\-centric benchmarks, it is not intended to cover the full space of time\-series regression problems\. We deliberately focus on virtual sensing as a multimodal, sensor\-based sequence\-to\-value setting because it isolates the transfer of DIR methods to time\-series inputs while retaining a single continuous target\. Forecasting and sequence\-to\-sequence prediction introduce additional structure: imbalance may depend not only on the target value, but also on where rare regimes occur along the prediction horizon\. Extending the analysis to these settings is therefore an important direction for future work\.

Second, our empirical comparison is necessarily selective\. We evaluate six representative DIR methods spanning algorithm\-level, representation\-learning, and architectural approaches, but the DIR literature is growing rapidly\. Our results therefore should not be read as a definitive ranking of all DIR methods, but as evidence that existing methods transfer unevenly to multimodal time\-series data\. Future work should expand the benchmark to newer methods and combinations of orthogonal interventions\.

Third, bMAE and bMASE depend on a discretization of the target range\. We mitigate this choice by using a shared data\-driven binning rule and show in[Subsection D\.2](https://arxiv.org/html/2609.25152#A4.SS2)that the resulting estimates agree closely with a continuous kernel\-based alternative\. Still, balanced evaluation necessarily requires a choice about local target resolution\. Developing equally interpretable continuous alternatives with explicit uncertainty estimates remains an open direction\.

Finally, our stability analysis is diagnostic rather than corrective\. We show that low\-density target regions exhibit higher seed\-level variability, but our scope was not to propose a method that explicitly optimizes for stable tail performance\. This opens a promising direction for future DIR methods: improving performance in underrepresented regions while also reducing variance across random initialization, data order, and optimization trajectories\.

## 6Conclusion

We revisited DIR from an evaluation perspective and identified three blind spots in the current empirical ecosystem\. The first is aData Blind Spot: DIR evaluation remains concentrated in image\-based benchmarks, leaving structurally different regression domains underexplored\. The second is aMetric Blind Spot: many\-/medium\-/few\-shot reporting is useful diagnostically, but does not provide a decision\-complete scalar criterion for comparing methods\. The third is aStability Blind Spot: the rare target regions that motivate DIR are rarely evaluated for reproducibility across random seeds\.

To address these issues, we introducedMuViS\-DIR, a multimodal virtual sensing benchmark with continuous time\-series regression tasks and practically meaningful tail regimes and we introduced bMASE, a scale\-normalized balanced metric that enables cross\-method and cross\-dataset comparison\. Finally, we analyzed DIR methods performance stability as a function of local target support\.

Our experiments show that standard deep learning models can exhibit substantial tail degradation that is masked by global MAE, that existing DIR methods can improve balanced performance but transfer unevenly to multimodal time\-series data, and that tail regions are also the least stable across seeds\. These findings do not suggest that any single method solvesMuViS\-DIR\. Rather, they show that evaluating DIR requires broader data domains, balanced scalar metrics, and explicit attention to tail reliability\.

## References

- \[1\]A\. Reiss, I\. Indlekofer, P\. Schmidt, and K\. Van Laerhoven\(2019\)Deep PPG: Large\-Scale Heart Rate Estimation with Convolutional Neural Networks\.Sensors19\(14\),pp\. 3079\(en\)\.External Links:ISSN 1424\-8220,[Link](https://www.mdpi.com/1424-8220/19/14/3079),[Document](https://dx.doi.org/10.3390/s19143079)Cited by:[item 6](https://arxiv.org/html/2609.25152#A3.I1.i6.p1.1),[Figure 1](https://arxiv.org/html/2609.25152#S0.F1),[§1](https://arxiv.org/html/2609.25152#S1.p1.1),[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.SSS0.Px1.p2.1)\.
- \[2\]Y\. Yang, K\. Zha, Y\. Chen, H\. Wang, and D\. Katabi\(2021\)Delving into Deep Imbalanced Regression\.arXiv\.Note:arXiv:2102\.09554 \[cs\]External Links:[Link](http://arxiv.org/abs/2102.09554),[Document](https://dx.doi.org/10.48550/arXiv.2102.09554)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.10.2.1.1),[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.2.2.1.1),[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.6.2.1.1),[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.7.2.1.1),[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.8.2.1.1),[§D\.1\.2](https://arxiv.org/html/2609.25152#A4.SS1.SSS2.p1.1),[§1](https://arxiv.org/html/2609.25152#S1.p1.1),[§1](https://arxiv.org/html/2609.25152#S1.p2.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1),[§2](https://arxiv.org/html/2609.25152#S2.p2.1),[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2609.25152#S4.SS1.p1.1)\.
- \[3\]N\. C\. Puetz, J\. U\. Brandt, M\. Hilbert, E\. Raponi, T\. Bäck, and T\. Bartz\-Beielstein\(2026\)Deconstructing deep imbalanced regression: a comprehensive review and experimental evaluation\.Artificial Intelligence Review59\(6\),pp\. 135\(en\)\.External Links:ISSN 1573\-7462,[Link](https://doi.org/10.1007/s10462-026-11570-1),[Document](https://dx.doi.org/10.1007/s10462-026-11570-1)Cited by:[§D\.1](https://arxiv.org/html/2609.25152#A4.SS1.p1.1),[Appendix E](https://arxiv.org/html/2609.25152#A5.p1.1),[§1](https://arxiv.org/html/2609.25152#S1.p1.1),[§1](https://arxiv.org/html/2609.25152#S1.p2.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1),[§2](https://arxiv.org/html/2609.25152#S2.p2.1),[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.SSS0.Px1.p2.1),[§4\.2](https://arxiv.org/html/2609.25152#S4.SS2.p1.1)\.
- \[4\]S\. Shalev\-Shwartz and S\. Ben\-David\(2014\)Understanding Machine Learning: From Theory to Algorithms\.Cambridge University Press,USA\.External Links:ISBN 978\-1\-107\-05713\-5Cited by:[§D\.1](https://arxiv.org/html/2609.25152#A4.SS1.p1.1),[§1](https://arxiv.org/html/2609.25152#S1.p1.1)\.
- \[5\]Z\. Zhang, E\. Fischer, J\. Zscheischler, and S\. Engelke\(2026\)Physics\-based models outperform AI weather forecasts of record\-breaking extremes\.Science Advances12\(18\),pp\. eaec1433\.External Links:[Link](https://www.science.org/doi/10.1126/sciadv.aec1433),[Document](https://dx.doi.org/10.1126/sciadv.aec1433)Cited by:[§1](https://arxiv.org/html/2609.25152#S1.p1.1),[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.SSS0.Px1.p2.1)\.
- \[6\]J\. Ren, M\. Zhang, C\. Yu, and Z\. Liu\(2022\)Balanced MSE for Imbalanced Visual Regression\.In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),New Orleans, LA, USA,pp\. 7916–7925\.External Links:ISBN 978\-1\-6654\-6946\-3,[Link](https://ieeexplore.ieee.org/document/9879350/),[Document](https://dx.doi.org/10.1109/CVPR52688.2022.00777)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.3.2.1.1),[item 2](https://arxiv.org/html/2609.25152#S1.I1.i2.p1.1),[§1](https://arxiv.org/html/2609.25152#S1.p1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1),[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.SSS0.Px1.p2.1),[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.SSS0.Px2.p1.1)\.
- \[7\]Z\. Liu, Z\. Miao, X\. Zhan, J\. Wang, B\. Gong, and S\. X\. Yu\(2019\)Large\-Scale Long\-Tailed Recognition in an Open World\.arXiv\.Note:arXiv:1904\.05160 \[cs\]External Links:[Link](http://arxiv.org/abs/1904.05160),[Document](https://dx.doi.org/10.48550/arXiv.1904.05160)Cited by:[§1](https://arxiv.org/html/2609.25152#S1.p2.1),[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.p1.1)\.
- \[8\]X\. Bouthillier, P\. Delaunay, M\. Bronzi, A\. Trofimov, B\. Nichyporuk, J\. Szeto, N\. Sepah, E\. Raff, K\. Madan, V\. Voleti, S\. E\. Kahou, V\. Michalski, D\. Serdyuk, T\. Arbel, C\. Pal, G\. Varoquaux, and P\. Vincent\(2021\)Accounting for Variance in Machine Learning Benchmarks\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2103.03098)Cited by:[§1](https://arxiv.org/html/2609.25152#S1.p2.1)\.
- \[9\]J\. U\. Brandt, N\. C\. Puetz, J\. J\. George, N\. V\. Kumar, E\. Raponi, M\. Hilbert, T\. Bäck, and T\. Bartz\-Beielstein\(2026\)MuViS: Multimodal Virtual Sensing Benchmark\.arXiv\.Note:arXiv:2603\.24602 \[eess\]External Links:[Link](http://arxiv.org/abs/2603.24602),[Document](https://dx.doi.org/10.48550/arXiv.2603.24602)Cited by:[item 3](https://arxiv.org/html/2609.25152#A3.I1.i3.p1.1),[§F\.1](https://arxiv.org/html/2609.25152#A6.SS1.p1.1),[Figure 13](https://arxiv.org/html/2609.25152#A7.F13),[Figure 14](https://arxiv.org/html/2609.25152#A7.F14),[Figure 15](https://arxiv.org/html/2609.25152#A7.F15),[§G\.1](https://arxiv.org/html/2609.25152#A7.SS1.p1.1),[item 1](https://arxiv.org/html/2609.25152#S1.I1.i1.p1.1),[Figure 2](https://arxiv.org/html/2609.25152#S3.F2),[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.p1.1),[Figure 4](https://arxiv.org/html/2609.25152#S4.F4),[§4\.1](https://arxiv.org/html/2609.25152#S4.SS1.p1.1)\.
- \[10\]H\. He and E\. A\. Garcia\(2009\)Learning from Imbalanced Data\.IEEE Transactions on Knowledge and Data Engineering21\(9\),pp\. 1263–1284\.Note:Conference Name: IEEE Transactions on Knowledge and Data EngineeringExternal Links:ISSN 1558\-2191,[Link](https://ieeexplore.ieee.org/document/5128907),[Document](https://dx.doi.org/10.1109/TKDE.2008.239)Cited by:[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[11\]M\. Steininger, K\. Kobs, P\. Davidson, A\. Krause, and A\. Hotho\(2021\)Density\-based weighting for imbalanced regression\.Machine Learning110\(8\),pp\. 2187–2211\(en\)\.External Links:ISSN 1573\-0565,[Link](https://doi.org/10.1007/s10994-021-06023-5),[Document](https://dx.doi.org/10.1007/s10994-021-06023-5)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.5.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[12\]Z\. Wang and H\. Wang\(2023\)Variational Imbalanced Regression: Fair Uncertainty Quantification via Probabilistic Smoothing\.arXiv\.Note:arXiv:2306\.06599 \[cs, stat\]External Links:[Link](http://arxiv.org/abs/2306.06599),[Document](https://dx.doi.org/10.48550/arXiv.2306.06599)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.4.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[13\]G\. Nie, G\. Tang, and S\. Hong\(2025\)Dist Loss: Enhancing Regression in Few\-Shot Region through Distribution Distance Constraint\.arXiv\.Note:arXiv:2411\.15216 \[cs\]External Links:[Link](http://arxiv.org/abs/2411.15216),[Document](https://dx.doi.org/10.48550/arXiv.2411.15216)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.9.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1),[§2](https://arxiv.org/html/2609.25152#S2.p2.1)\.
- \[14\]Y\. Gong, G\. Mori, and F\. Tung\(2022\)RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression\.arXiv\.Note:arXiv:2205\.15236 \[cs\]External Links:[Link](http://arxiv.org/abs/2205.15236),[Document](https://dx.doi.org/10.48550/arXiv.2205.15236)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.11.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[15\]S\. Zhang, L\. Yang, M\. B\. Mi, X\. Zheng, and A\. Yao\(2023\)Improving Deep Regression with Ordinal Entropy\.arXiv\.Note:arXiv:2301\.08915 \[cs\]External Links:[Link](http://arxiv.org/abs/2301.08915),[Document](https://dx.doi.org/10.48550/arXiv.2301.08915)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.15.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[16\]M\. Keramati, L\. Meng, and R\. D\. Evans\(2023\)ConR: Contrastive Regularizer for Deep Imbalanced Regression\.arXiv\.Note:Version Number: 4External Links:[Link](https://arxiv.org/abs/2309.06651),[Document](https://dx.doi.org/10.48550/ARXIV.2309.06651)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.12.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[17\]K\. Zha, P\. Cao, J\. Son, Y\. Yang, and D\. Katabi\(2023\)Rank\-N\-Contrast: Learning Continuous Representations for Regression\.arXiv\.Note:arXiv:2210\.01189 \[cs\]External Links:[Link](http://arxiv.org/abs/2210.01189),[Document](https://dx.doi.org/10.48550/arXiv.2210.01189)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.13.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1),[§2](https://arxiv.org/html/2609.25152#S2.p2.1)\.
- \[18\]Z\. Dong, Y\. Wu, C\. Chen, Y\. Zou, Y\. Zhang, and J\. H\. Zhou\(2025\)Improve Representation for Imbalanced Regression through Geometric Constraints\.arXiv\.Note:arXiv:2503\.00876 \[cs\]External Links:[Link](http://arxiv.org/abs/2503.00876),[Document](https://dx.doi.org/10.48550/arXiv.2503.00876)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.14.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[19\]J\. Lim, S\. Lee, D\. Um, S\. Park, and J\. Shin\(2025\)PRIME: Deep Imbalanced Regression with Proxies\.InProceedings of the 42nd International Conference on Machine Learning,Vancouver, Canada,pp\. 37705–37729\(en\)\.External Links:ISSN 2640\-3498,[Link](https://proceedings.mlr.press/v267/lim25a.html)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.16.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[20\]Y\. Jiang, V\. S\. F\. Garnot, K\. Schindler, and J\. D\. Wegner\(2024\)Uncertainty Voting Ensemble for Imbalanced Deep Regression\.arXiv\.Note:arXiv:2305\.15178 \[cs\]External Links:[Link](http://arxiv.org/abs/2305.15178),[Document](https://dx.doi.org/10.48550/arXiv.2305.15178)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.17.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[21\]H\. Xiong and A\. Yao\(2024\)Deep Imbalanced Regression via Hierarchical Classification Adjustment\.In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 23721–23730\.External Links:ISSN 2575\-7075,[Link](https://ieeexplore.ieee.org/document/10655108),[Document](https://dx.doi.org/10.1109/CVPR52733.2024.02239)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.18.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[22\]D\. Lin, T\. Peng, R\. Chen, X\. Xie, and Z\. Cui\(2024\)Let Multi\-classification Help Deep Imbalanced Regression\.InArtificial Neural Networks and Machine Learning – ICANN 2024,M\. Wand, K\. Malinovská, J\. Schmidhuber, and I\. V\. Tetko \(Eds\.\),Cham,pp\. 430–447\(en\)\.External Links:ISBN 978\-3\-031\-72338\-4,[Document](https://dx.doi.org/10.1007/978-3-031-72338-4%5F29)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.19.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[23\]I\. Nejjar, F\. Ahmed, and O\. Fink\(2024\)IM\-Context: In\-Context Learning for Imbalanced Regression Tasks\.arXiv\.Note:arXiv:2405\.18202 \[cs\]External Links:[Link](http://arxiv.org/abs/2405.18202),[Document](https://dx.doi.org/10.48550/arXiv.2405.18202)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.20.2.1.1),[§2](https://arxiv.org/html/2609.25152#S2.p1.1)\.
- \[24\]S\. Moschoglou, A\. Papaioannou, C\. Sagonas, J\. Deng, I\. Kotsia, and S\. Zafeiriou\(2017\)AgeDB: The First Manually Collected, In\-the\-Wild Age Database\.In2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops \(CVPRW\),pp\. 1997–2005\.External Links:ISSN 2160\-7516,[Link](https://ieeexplore.ieee.org/document/8014984),[Document](https://dx.doi.org/10.1109/CVPRW.2017.250)Cited by:[§2](https://arxiv.org/html/2609.25152#S2.p2.1)\.
- \[25\]R\. Rothe, R\. Timofte, and L\. Van Gool\(2018\)Deep Expectation of Real and Apparent Age from a Single Image Without Facial Landmarks\.International Journal of Computer Vision126\(2\),pp\. 144–157\(en\)\.External Links:ISSN 1573\-1405,[Link](https://doi.org/10.1007/s11263-016-0940-3),[Document](https://dx.doi.org/10.1007/s11263-016-0940-3)Cited by:[§2](https://arxiv.org/html/2609.25152#S2.p2.1)\.
- \[26\]D\. Cer, M\. Diab, E\. Agirre, I\. Lopez\-Gazpio, and L\. Specia\(2017\)SemEval\-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation\.InProceedings of the 11th International Workshop on Semantic Evaluation \(SemEval\-2017\),S\. Bethard, M\. Carpuat, M\. Apidianaki, S\. M\. Mohammad, D\. Cer, and D\. Jurgens \(Eds\.\),Vancouver, Canada,pp\. 1–14\.External Links:[Link](https://aclanthology.org/S17-2001/),[Document](https://dx.doi.org/10.18653/v1/S17-2001)Cited by:[§2](https://arxiv.org/html/2609.25152#S2.p2.1)\.
- \[27\]S\. F\. Quan, B\. V\. Howard, C\. Iber, J\. P\. Kiley, F\. J\. Nieto, G\. T\. O’Connor, D\. M\. Rapoport, S\. Redline, J\. Robbins, J\. M\. Samet, and P\. W\. Wahl\(1997\)The Sleep Heart Health Study: design, rationale, and methods\.Sleep20\(12\),pp\. 1077–1085\(eng\)\.External Links:ISSN 0161\-8105Cited by:[§2](https://arxiv.org/html/2609.25152#S2.p2.1)\.
- \[28\]D\. A\. Engemann, A\. Mellot, R\. Höchenberger, H\. Banville, D\. Sabbagh, L\. Gemein, T\. Ball, and A\. Gramfort\(2022\)A reusable benchmark of brain\-age prediction from M/EEG resting\-state signals\.NeuroImage262,pp\. 119521\(eng\)\.External Links:ISSN 1095\-9572,[Document](https://dx.doi.org/10.1016/j.neuroimage.2022.119521)Cited by:[§2](https://arxiv.org/html/2609.25152#S2.p2.1)\.
- \[29\]A\. E\. W\. Johnson, L\. Bulgarelli, L\. Shen, A\. Gayles, A\. Shammout, S\. Horng, T\. J\. Pollard, S\. Hao, B\. Moody, B\. Gow, L\. H\. Lehman, L\. A\. Celi, and R\. G\. Mark\(2023\)MIMIC\-IV, a freely accessible electronic health record dataset\.Scientific Data10\(1\),pp\. 1\(eng\)\.External Links:ISSN 2052\-4463,[Document](https://dx.doi.org/10.1038/s41597-022-01899-x)Cited by:[§2](https://arxiv.org/html/2609.25152#S2.p2.1)\.
- \[30\]P\.F\. Muir\(1990\)A virtual sensor approach to robot kinematic identification: theory and experimental implementation\.Note:Pages: 445External Links:ISBN 978\-0\-7803\-0173\-3,[Document](https://dx.doi.org/10.1109/ICSYSE.1990.203189)Cited by:[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.p1.1)\.
- \[31\]D\. Martin, N\. Kühl, and G\. Satzger\(2021\)Virtual Sensors\.Business & Information Systems Engineering63\(3\),pp\. 315–323\(en\)\.External Links:ISSN 1867\-0202,[Link](https://doi.org/10.1007/s12599-021-00689-w),[Document](https://dx.doi.org/10.1007/s12599-021-00689-w)Cited by:[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.p1.1)\.
- \[32\]P\. Mohapatra, Y\. Sui, A\. Pandey, S\. Xia, and Q\. Zhu\(2025\)MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Series\.arXiv\.Note:arXiv:2509\.25278 \[cs\]External Links:[Link](http://arxiv.org/abs/2509.25278),[Document](https://dx.doi.org/10.48550/arXiv.2509.25278)Cited by:[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.p1.1)\.
- \[33\]C\. W\. Tan, C\. Bergmeir, F\. Petitjean, and G\. I\. Webb\(2020\)Monash University, UEA, UCR Time Series Extrinsic Regression Archive\.arXiv\.Note:arXiv:2006\.10996 \[cs\]External Links:[Link](http://arxiv.org/abs/2006.10996),[Document](https://dx.doi.org/10.48550/arXiv.2006.10996)Cited by:[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.p1.1)\.
- \[34\]S\. Zhang, B\. Guo, A\. Dong, J\. He, Z\. Xu, and S\. X\. Chen\(2017\)Cautionary tales on air\-quality improvement in Beijing\.Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences473\(2205\),pp\. 20170457\.External Links:ISSN 1364\-5021,[Link](https://doi.org/10.1098/rspa.2017.0457),[Document](https://dx.doi.org/10.1098/rspa.2017.0457)Cited by:[item 1](https://arxiv.org/html/2609.25152#A3.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.SSS0.Px1.p2.1)\.
- \[35\]Y\. Chen, A\. Ebenstein, M\. Greenstone, and H\. Li\(2013\)Evidence on the impact of sustained exposure to air pollution on life expectancy from China’s Huai River policy\.Proceedings of the National Academy of Sciences110\(32\),pp\. 12936–12941\.External Links:[Link](https://www.pnas.org/doi/10.1073/pnas.1300018110),[Document](https://dx.doi.org/10.1073/pnas.1300018110)Cited by:[item 1](https://arxiv.org/html/2609.25152#A3.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.SSS0.Px1.p2.1),[§4\.1](https://arxiv.org/html/2609.25152#S4.SS1.p2.1)\.
- \[36\]J\. J\. Downs and E\. F\. Vogel\(1993\)A plant\-wide industrial process control problem\.Computers & Chemical Engineering17\(3\),pp\. 245–255\.External Links:ISSN 0098\-1354,[Link](https://www.sciencedirect.com/science/article/pii/009813549380018I),[Document](https://dx.doi.org/10.1016/0098-1354%2893%2980018-I)Cited by:[item 4](https://arxiv.org/html/2609.25152#A3.I1.i4.p1.1),[§3\.1](https://arxiv.org/html/2609.25152#S3.SS1.SSS0.Px1.p2.1)\.
- \[37\]R\. J\. Hyndman and A\. B\. Koehler\(2006\)Another look at measures of forecast accuracy\.International journal of forecasting22\(4\),pp\. 679–688\.External Links:[Link](https://www.sciencedirect.com/science/article/pii/S0169207006000239)Cited by:[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.SSS0.Px3.p1.1)\.
- \[38\]P\. J\. Fleming and J\. J\. Wallace\(1986\)How not to lie with statistics: the correct way to summarize benchmark results\.Commun\. ACM29\(3\),pp\. 218–221\.External Links:ISSN 0001\-0782,[Link](https://dl.acm.org/doi/10.1145/5666.5673),[Document](https://dx.doi.org/10.1145/5666.5673)Cited by:[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.SSS0.Px3.p1.3)\.
- \[39\]P\. J\. Huber\(1964\)Robust Estimation of a Location Parameter\.The Annals of Mathematical Statistics35\(1\),pp\. 73–101\(en\)\.External Links:ISSN 0003\-4851,[Link](http://projecteuclid.org/euclid.aoms/1177703732),[Document](https://dx.doi.org/10.1214/aoms/1177703732)Cited by:[§D\.2](https://arxiv.org/html/2609.25152#A4.SS2.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.SSS0.Px3.p2.1)\.
- \[40\]D\. Freedman and P\. Diaconis\(1981\)On the histogram as a density estimator:L2 theory\.Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete57\(4\),pp\. 453–476\(en\)\.External Links:ISSN 1432\-2064,[Link](https://doi.org/10.1007/BF01025868),[Document](https://dx.doi.org/10.1007/BF01025868)Cited by:[§3\.2](https://arxiv.org/html/2609.25152#S3.SS2.SSS0.Px3.p3.1)\.
- \[41\]N\. Reimers and I\. Gurevych\(2017\)Reporting Score Distributions Makes a Difference: Performance Study of LSTM\-networks for Sequence Tagging\.InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,M\. Palmer, R\. Hwa, and S\. Riedel \(Eds\.\),Copenhagen, Denmark,pp\. 338–348\.External Links:[Link](https://aclanthology.org/D17-1035/),[Document](https://dx.doi.org/10.18653/v1/D17-1035)Cited by:[§4\.3](https://arxiv.org/html/2609.25152#S4.SS3.p1.1)\.
- \[42\]C\. Colas, O\. Sigaud, and P\. Oudeyer\(2018\)How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments\.arXiv\.Note:arXiv:1806\.08295 \[cs\]External Links:[Link](http://arxiv.org/abs/1806.08295),[Document](https://dx.doi.org/10.48550/arXiv.1806.08295)Cited by:[§4\.3](https://arxiv.org/html/2609.25152#S4.SS3.p1.1)\.
- \[43\]T\. Lin, P\. Goyal, R\. Girshick, K\. He, and P\. Dollár\(2018\)Focal Loss for Dense Object Detection\.arXiv\.Note:arXiv:1708\.02002 \[cs\]External Links:[Link](http://arxiv.org/abs/1708.02002),[Document](https://dx.doi.org/10.48550/arXiv.1708.02002)Cited by:[Table 4](https://arxiv.org/html/2609.25152#A3.T4.10.6.2.1.1)\.
- \[44\]J\. U\. Brandt, N\. C\. Pütz, M\. Greiff, T\. J\. Lew, J\. Subosits, M\. Hilbert, and T\. Bartz\-Beielstein\(2025\)From Faults to Features: Pretraining to Learn Robust Representations against Sensor Failures\.\(en\)\.External Links:[Link](https://openreview.net/forum?id=9aElHWiZ72)Cited by:[item 2](https://arxiv.org/html/2609.25152#A3.I1.i2.p1.1)\.
- \[45\]J\. C\. Kegelman, L\. K\. Harbott, and J\. C\. Gerdes\(2017\)Insights into vehicle trajectories at the handling limits: analysing open data from race car drivers\.Vehicle System Dynamics55\(2\),pp\. 191–207\.Note:\_eprint: https://doi\.org/10\.1080/00423114\.2016\.1249893External Links:ISSN 0042\-3114,[Link](https://doi.org/10.1080/00423114.2016.1249893),[Document](https://dx.doi.org/10.1080/00423114.2016.1249893)Cited by:[item 2](https://arxiv.org/html/2609.25152#A3.I1.i2.p1.1)\.
- \[46\]D\. Mori, R\. K\. Aggarwal, N\.D\. Broadbent, T\. Kobayashi, and J\. C\. Gerdes\(2025\)Vehicle Dynamics Dataset for Highly Dynamic Automated Driving\.https://purl\.stanford\.edu/hh613qz0317/version/1\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.25740/hh613qz0317.)Cited by:[item 3](https://arxiv.org/html/2609.25152#A3.I1.i3.p1.1)\.
- \[47\]P\. Kollmeyer\(2018\)Panasonic 18650PF Li\-ion Battery Data\.1\(en\)\.External Links:[Link](https://data.mendeley.com/datasets/wykht8y7tg/1),[Document](https://dx.doi.org/10.17632/wykht8y7tg.1)Cited by:[item 5](https://arxiv.org/html/2609.25152#A3.I1.i5.p1.1)\.
- \[48\]P\. Mondal, D\. Bhavsar, K\. Mittal, and M\. Mittal\(2024\)Estimating State\-of\-Charge in Lithium\-Ion Batteries Through Deep Learning Techniques: A Comparative Evaluation\.IEEE AccessPP,pp\. 1–1\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2024.3408220)Cited by:[item 5](https://arxiv.org/html/2609.25152#A3.I1.i5.p1.1)\.
- \[49\]J\. Quionero\-Candela, M\. Sugiyama, A\. Schwaighofer, and N\. Lawrence\(2009\)Dataset Shift in Machine Learning\.External Links:ISSN 0262170051, 9780262170055Cited by:[§D\.1](https://arxiv.org/html/2609.25152#A4.SS1.p1.1)\.
- \[50\]R\. Ribeiro and N\. Moniz\(2020\)Imbalanced regression and extreme value prediction\.Machine Learning109,pp\. 1–33\.External Links:[Document](https://dx.doi.org/10.1007/s10994-020-05900-9)Cited by:[§D\.1\.2](https://arxiv.org/html/2609.25152#A4.SS1.SSS2.p1.1)\.
- \[51\]B\. W\. Silverman\(2018\)Density Estimation for Statistics and Data Analysis\.Routledge,Boca Raton\.External Links:ISBN 978\-1\-315\-14091\-9,[Document](https://dx.doi.org/10.1201/9781315140919)Cited by:[§D\.2](https://arxiv.org/html/2609.25152#A4.SS2.p2.2)\.

## Appendix AAppendix

This appendix is organized as follows\.[Appendix B](https://arxiv.org/html/2609.25152#A2)lists the mathematical notation used throughout the paper\.[Appendix C](https://arxiv.org/html/2609.25152#A3),[Appendix D](https://arxiv.org/html/2609.25152#A4), and[Appendix E](https://arxiv.org/html/2609.25152#A5)provide additional analysis for the data\-, metric\-, and stability blind spots, respectively\.[Appendix F](https://arxiv.org/html/2609.25152#A6)provides instructions for reproducing the experiments from[Section 4](https://arxiv.org/html/2609.25152#S4)and[Appendix G](https://arxiv.org/html/2609.25152#A7)contains supplementary results and figures for[Section 4](https://arxiv.org/html/2609.25152#S4)\. The publicly accessible code for the paper can be found at[www\.github\.com/noah\-puetz/muvis\-dir](https://www.github.com/noah-puetz/muvis-dir)\.

## Appendix BMathematical Notations

Table 3:Mathematical notation used throughout the paper\.
## Appendix CData Blind Spot Appendix

This appendix supplements the data\-blind\-spot discussion in[Section 1](https://arxiv.org/html/2609.25152#S1)along two complementary axes\.[Subsection C\.1](https://arxiv.org/html/2609.25152#A3.SS1)provides a comprehensive snapshot of the DIR methodological landscape and architectures they were originally evaluated on, and indicating which combinations come with public training code\. The purpose is to make explicit how narrow the empirical evidence behind current DIR claims actually is, and why a domain\-extended benchmark such asMuViSis needed\.[Subsection C\.2](https://arxiv.org/html/2609.25152#A3.SS2)then characterisesMuViSitself, both statistically \(target imbalance, effective support of the training distribution\) and operationally \(why uniform performance across the target range is a domain requirement, not just a statistical preference\)\.

### C\.1DIR Methods and Dataset Landscape

[Table 4](https://arxiv.org/html/2609.25152#A3.T4)makes the data blind spot quantitatively visible: 18 of the 19 surveyed methods include AgeDB\-DIR or IMDB\-WIKI\-DIR as their primary evaluation, and only a small subset of works ventures beyond facial\-image regression\.

Table 4:Overview of DIR methods, datasets, and backbone architectures\. Datasets and models inboldindicate that public training code is available for the corresponding combination\. “Algo\.” stands for algorithm\-level methods, “Repr\.” for representation\-learning methods, and “Archit\.” for architectural methods\. The*Orth\.*column indicates whether the method is orthogonal to the backbone architecture and can be combined with other methods\.TypeMethodVenueDatasetsModelsOrth\.RepoAlgo\.LDS\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\]ICML 2021IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIR
NYUD2\-DIR
SHHS\-DIRResNet\-50
BiLSTM\+GloVe
CNN\-RNN✓[GitHub](https://github.com/YyzHarry/imbalanced-regression)Balanced MSE\[[6](https://arxiv.org/html/2609.25152#bib.bib43)\]CVPR 2022IMDB\-WIKI\-DIR
NYUD2\-DIR
IHMRResNet\-50
SPIN✓[GitHub](https://github.com/jiawei-ren/BalancedMSE)VIR\[[12](https://arxiv.org/html/2609.25152#bib.bib41)\]NeurIPS 2023IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIR
NYUD2\-DIRResNet\-50
BiLSTM\+GloVe✗[GitHub](https://github.com/Wang-ML-Lab/variational-imbalanced-regression)DenseLoss\[[11](https://arxiv.org/html/2609.25152#bib.bib42)\]Mach\. Learn\.2021Synthetic DataMLP✓[GitHub](https://github.com/SteiMi/denseweight)Focal\-R\[[2](https://arxiv.org/html/2609.25152#bib.bib39),[43](https://arxiv.org/html/2609.25152#bib.bib46)\]ICML 2021IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIR
NYUD2\-DIR
SHHS\-DIRResNet\-50
BiLSTM\+GloVe
CNN\-RNN✓[GitHub](https://github.com/YyzHarry/imbalanced-regression)INV & SQINV\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\]ICML 2021IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIR
NYUD2\-DIR
SHHS\-DIRResNet\-50
BiLSTM\+GloVe
CNN\-RNN✓[GitHub](https://github.com/YyzHarry/imbalanced-regression)RRT\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\]ICML 2021IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIR
NYUD2\-DIR
SHHS\-DIRResNet\-50
BiLSTM\+GloVe
CNN\-RNN✓[GitHub](https://github.com/YyzHarry/imbalanced-regression)Dist Loss\[[13](https://arxiv.org/html/2609.25152#bib.bib44)\]ICLR 2025IMDB\-WIKI\-DIR
AgeDB\-DIR
ECG\-K\-DIRResNet\-50
Net1D✓[GitHub](https://github.com/Ngk03/DIR-Dist-Loss)Repr\.FDS\[[2](https://arxiv.org/html/2609.25152#bib.bib39)\]ICML 2021IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIR
NYUD2\-DIR
SHHS\-DIRResNet\-50
BiLSTM\+GloVe
CNN\-RNN✓[GitHub](https://github.com/YyzHarry/imbalanced-regression)RankSim\[[14](https://arxiv.org/html/2609.25152#bib.bib50)\]ICML 2022IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIRResNet\-50
BiLSTM\+GloVe✓[GitHub](https://github.com/BorealisAI/ranksim-imbalanced-regression)ConR\[[16](https://arxiv.org/html/2609.25152#bib.bib48)\]ICLR 2024IMDB\-WIKI\-DIR
AgeDB\-DIR
NYUD2\-DIR
MPIIGaze\-DIRResNet\-50
LeNet✓[GitHub](https://github.com/BorealisAI/ConR/tree/main)RnC\[[17](https://arxiv.org/html/2609.25152#bib.bib38)\]NeurIPS 2023IMDB\-WIKI\-DIR
AgeDB\-DIR
TUAB
MPIIGaze\-DIR
SkyFinderResNet\-18
ResNet\-50✗ \(✓\)[GitHub](https://github.com/kaiwenzha/Rank-N-Contrast/tree/main)Geom\. Rep\.\[[18](https://arxiv.org/html/2609.25152#bib.bib29)\]CVPR 2025UCI\-DIR
AgeDB\-DIR
IMDB\-WIKI\-DIR
STS\-B\-DIRResNet\-50
BiLSTM\+GloVe
MLP✓[GitHub](https://github.com/yilei-wu/imbalanced-regression)Ordinal Ent\.\[[15](https://arxiv.org/html/2609.25152#bib.bib37)\]ICLR 2023Synthetic data
NYU\-Depth\-v2
AgeDB\-DIR
SHTechMLP
NeW\-CRFs
ResNet\-50
DeepONet
CSRNet✓[GitHub](https://github.com/needylove/OrdinalEntropy/tree/main)PRIME\[[19](https://arxiv.org/html/2609.25152#bib.bib47)\]ICML 2025AgeDB\-DIR
IMDB\-WIKI\-DIR
NYUD2\-DIR
STS\-B\-DIRResNet\-50
BiLSTM\+GloVe✓—Archit\.UVOTE\[[20](https://arxiv.org/html/2609.25152#bib.bib49)\]GCPR 2024IMDB\-WIKI\-DIR
AgeDB\-DIR
Wind
STS\-B\-DIRResNet\-50
ResNet\-18
BiLSTM✗[GitHub](https://github.com/SherryJYC/UVOTE/tree/main)HCA\[[21](https://arxiv.org/html/2609.25152#bib.bib40)\]CVPR 2024IMDB\-WIKI\-DIR
AgeDB\-DIR
NYUDv2\-DIR
SHTechResNet\-50
VGG16✗[GitHub](https://github.com/xhp-hust-2018-2011/HCA)Multi\-Class\.\[[22](https://arxiv.org/html/2609.25152#bib.bib28)\]ICANN 2024IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIRResNet\-50
BiLSTM\+GloVe✗—Im\-Context\[[23](https://arxiv.org/html/2609.25152#bib.bib45)\]TMLR 2024IMDB\-WIKI\-DIR
AgeDB\-DIR
STS\-B\-DIR
Boston
Concrete
Abalone
Communities
Kin8nm
AirfoilGPT2
PFN✗[GitHub](https://github.com/ismailnejjar/IM-Context)
### C\.2MuViSDataset Overview

We characterise theMuViSdatasets along two complementary dimensions\. The*imbalance ratio*\(IR\) summarises the contrast between dense head and sparse tails of the training target distribution, while the*effective support*SeffS\_\{\\mathrm\{eff\}\}quantifies how much of the available target range the training distribution actually uses\. Together, the two metrics distinguish distributions that are similarly skewed but differ in coverage, and are reported alongside conventional moments \(skew, excess kurtosis\) in[Table 5](https://arxiv.org/html/2609.25152#A3.T5)\.[Figure 2](https://arxiv.org/html/2609.25152#S3.F2)additionally visualises all nine target distributions\.

##### Imbalance ratio\.

The imbalance ratio is defined as the ratio of sample density in the head of the training distribution to sample density in its tails:

IR=dheaddtail,\\mathrm\{IR\}\\;=\\;\\frac\{d\_\{\\mathrm\{head\}\}\}\{d\_\{\\mathrm\{tail\}\}\}\\,,\(4\)where,

dhead=\|\{i:Q25≤yi≤Q75\}\|Q75−Q25,dtail=\|\{i:yi<Q10\}\|\+\|\{i:yi\>Q90\}\|\(Q10−ymin\)\+\(ymax−Q90\),d\_\{\\mathrm\{head\}\}\\;=\\;\\frac\{\\bigl\|\\\{i:Q\_\{25\}\\leq y\_\{i\}\\leq Q\_\{75\}\\\}\\bigr\|\}\{Q\_\{75\}\-Q\_\{25\}\}\\,,\\qquad d\_\{\\mathrm\{tail\}\}\\;=\\;\\frac\{\\bigl\|\\\{i:y\_\{i\}<Q\_\{10\}\\\}\\bigr\|\+\\bigl\|\\\{i:y\_\{i\}\>Q\_\{90\}\\\}\\bigr\|\}\{\(Q\_\{10\}\-y\_\{\\min\}\)\+\(y\_\{\\max\}\-Q\_\{90\}\)\}\\,,\(5\)whereQpQ\_\{p\}denotes thepp\-th percentile of the training targets\. The quantile thresholds determine*which*samples count as head or tail, by construction the head always contains50%50\\%of training samples and the two tails together20%20\\%, while the densitiesdheadd\_\{\\mathrm\{head\}\}anddtaild\_\{\\mathrm\{tail\}\}are measured in target\-space units, matching the equal\-width binning thatbMAE\\mathrm\{bMAE\}andbMASE\\mathrm\{bMASE\}apply at evaluation time\. A uniform target distribution yieldsIR=1\\mathrm\{IR\}=1\. For sharply peaked distributions the20%20\\%of tail samples are spread over a wider target range, drivingdtaild\_\{\\mathrm\{tail\}\}down and IR up\.

##### Effective support\.

While IR captures the head\-to\-tail contrast, two distributions with identical IR can still differ substantially in how much of the target range they cover \(e\.g\. a heavily skewed distribution versus a bimodal one\)\. The effective supportSeffS\_\{\\mathrm\{eff\}\}provides a single global measure of this coverage\. We fit a Gaussian KDE with Silverman bandwidth to the training targets, evaluate it on a grid ofGGequally spaced points spanning the target range, normalise to a discrete distributionqq, and define:

Seff=exp⁡\(H⁡\(q\)\)G,H\(q\)=−∑g=1Gqglogqg,S\_\{\\mathrm\{eff\}\}\\;=\\;\\frac\{\\exp\\bigl\(H\(q\)\\bigr\)\}\{G\}\\,,\\qquad H\(q\)\\;=\\;\-\\sum\_\{g=1\}^\{G\}q\_\{g\}\\log q\_\{g\}\\,,\(6\)whereH⁡\(q\)H\(q\)is the Shannon entropy ofqq\. Intuitively,SeffS\_\{\\mathrm\{eff\}\}answers the question*what fraction of the target range is effectively occupied by the training distribution?*A value of11corresponds to a perfectly uniform distribution; lower values indicate that the mass is concentrated in a smaller fraction of the range\.SeffS\_\{\\mathrm\{eff\}\}is the continuous analogue of the “effective number of classes” notion from imbalanced classification\. Because the smoothing is controlled by the KDE bandwidth rather than byGG,SeffS\_\{\\mathrm\{eff\}\}is stable across discretisation choices \(<0\.6%<\\\!0\.6\\%variation over a×20\\\!\\timesrange ofGG\)\.

##### Dataset statistics\.

[Table 5](https://arxiv.org/html/2609.25152#A3.T5)reports both metrics together with sample sizes, sequence lengthTT, input channelsDD, and conventional distributional moments\. The nineMuViStasks span more than an order of magnitude in IR \(from 2\.0 on Battery SoC to 41\.2 on Monterey\) and a corresponding range inSeffS\_\{\\mathrm\{eff\}\}\(from0\.920\.92down to0\.250\.25\)\. This range is a deliberate property of the benchmark: it allows DIR methods to be tested across the full spectrum from near\-uniform to extremely peaked target distributions, rather than within the narrow band covered by the predominantly Gaussian facial\-age benchmarks\.

Table 5:MuViSdataset statistics and imbalance characterisation\.TT: sequence length;DD: input channels; IR: imbalance ratio \(↑\\uparrow= more imbalanced\);SeffS\_\{\\mathrm\{eff\}\}: effective support \(↓\\downarrow= more concentrated\)\. Tasks ordered by decreasing IR\.
#### C\.2\.1Domain motivation for tail\-region reliability

The statistical imbalance summarised above is necessary but not sufficient to motivate balanced evaluation\. What makesMuViSa meaningful DIR benchmark is that, for every task, the distributional tails correspond to operationally critical regimes where predictive failures carry disproportionate consequences\.

1. 1\.PM2\.5/ PM10\.The Beijing Multi\-Site Air Quality data\[[34](https://arxiv.org/html/2609.25152#bib.bib17)\]pose a virtual sensing task in which the target is particulate\-matter concentration and the inputs combine pollutant and meteorological measurements\. Extreme concentrations exceed air\-quality\-index thresholds that trigger public\-health advisories and emergency traffic restrictions\[[35](https://arxiv.org/html/2609.25152#bib.bib11)\]\.
2. 2\.Monterey / Targa ’13 / Targa ’14\.The Revs Program Vehicle Dynamics Database integrates multimodal sensing sources \(driver inputs, wheel and chassis measurements, GNSS\-aided inertial navigation\)\. Following[Brandt et al\. \[44\]](https://arxiv.org/html/2609.25152#bib.bib10), we define a virtual sensing task of estimating the vehicle’s lateral velocityvyv\_\{y\}, a key state for stability assessment and control that is typically not directly available from low\-cost on\-board sensing\. Extreme values correspond to near\-limit handling regimes marking the onset of tire\-grip loss and vehicle instability\. These are precisely the safety\-critical conditions that electronic stability control systems are designed to navigate\[[45](https://arxiv.org/html/2609.25152#bib.bib9)\]\.
3. 3\.Tire Temp\.MuViSadditionally includes a high\-performance autonomous\-driving dataset that records vehicle state from RTK\-GPS alongside control inputs, actuator states, and the target real\-time tire temperature\[[46](https://arxiv.org/html/2609.25152#bib.bib8)\]\. Extreme surface temperatures indicate accelerated tread degradation and elevated blowout risk; accurate tail prediction is therefore essential for high\-performance autonomous applications\[[9](https://arxiv.org/html/2609.25152#bib.bib35)\]\.
4. 4\.Chem\. Conc\.Tails flag abnormal operating states corresponding to the 21 benchmarked fault scenarios of the Tennessee Eastman process\[[36](https://arxiv.org/html/2609.25152#bib.bib16)\]\. A model that fails to predict these extremes effectively fails to detect process faults\.
5. 5\.Battery SoC\.Extremes represent overcharge and deep\-discharge regimes where cell degradation accelerates and safety margins narrow\. Prediction bias at these boundaries can trigger premature or delayed charge termination\[[47](https://arxiv.org/html/2609.25152#bib.bib7),[48](https://arxiv.org/html/2609.25152#bib.bib6)\]\.
6. 6\.Heart Rate\.Extremes indicate exercise\-induced tachycardia or arrhythmia events that are clinically significant yet inherently rare in ambulatory recordings\[[1](https://arxiv.org/html/2609.25152#bib.bib33)\]\.

In each case, an evaluation protocol that down\-weights tail performance in proportion to its empirical frequency, as standard MAE andR2R^\{2\}implicitly do, would systematically reward models that fail in exactly the regimes the underlying application cares most about\.

## Appendix DMetric Blind Spot Appendix

This appendix provides the empirical analysis underlying the metric choices in[Subsection 3\.2](https://arxiv.org/html/2609.25152#S3.SS2)\. We study \(i\) the distribution invariance of standard and balanced regression metrics, and \(ii\) the discrete\-versus\-continuous estimation question for the bMAE numerator that underlies bMASE\.

### D\.1Distribution invariance of standard and balanced metrics

In this section we analyse the sensitivity of different evaluation metrics to the test distribution and show why several widely used metrics are not aligned with the evaluation objective of DIR\.[Puetz et al\. \[3\]](https://arxiv.org/html/2609.25152#bib.bib36)describes the fundamental challenge in DIR as a “mismatch between the training objective and the evaluation goal”: the very act of addressing imbalance implies that we value uniform performance across all target regions more than their representation in the training data would suggest, which is equivalent to a distribution shift in the target space and therefore breaks the standard i\.i\.d\. assumption\[[3](https://arxiv.org/html/2609.25152#bib.bib36),[4](https://arxiv.org/html/2609.25152#bib.bib34),[49](https://arxiv.org/html/2609.25152#bib.bib5)\]\. Building on this line of argument, our goal is a single scalar metric that ranks models consistently regardless of the test\-set composition\. The experiment below shows how a metric’s sensitivity to the test distribution can shift both model rankings and performance estimates, and demonstrates that bMAE and bMASE are robust against this effect\.

#### D\.1\.1Experimental setup

To isolate metric properties from model properties, we construct four*deterministic*synthetic models whose error profile is a fixed function of the target value\. Given a test sample with true targetyy, we defined=\|y−μtrain\|/\(0\.4​\(b−a\)\)d=\|y\-\\mu\_\{\\mathrm\{train\}\}\|/\(0\.4\\,\(b\-a\)\)ands=sign⁡\(y−μtrain\)s=\\mathrm\{sign\}\(y\-\\mu\_\{\\mathrm\{train\}\}\), and study:

- •A\(tail collapse\):y^=y\+\(0\.5\+10​d2\)​s\\hat\{y\}=y\+\(0\.5\+10\\,d^\{2\}\)\\,s
- •B\(uniformly mediocre\):y^=y\+\(2\.5\+0\.3​d2\)​s\\hat\{y\}=y\+\(2\.5\+0\.3\\,d^\{2\}\)\\,s
- •C\(uniformly good\):y^=y\+\(1\.0\+0\.3​d2\)​s\\hat\{y\}=y\+\(1\.0\+0\.3\\,d^\{2\}\)\\,s
- •D\(tail specialist\):y^=y\+\(0\.5\+10​e−3​d\)​s\\hat\{y\}=y\+\(0\.5\+10\\,e^\{\-3d\}\)\\,s

The four models cover the failure modes most relevant to DIR: a typical empirical\-risk\-minimisation profile that collapses in the tails \(A\), a uniformly mediocre baseline \(B\), a uniformly strong predictor \(C\), and a tail\-specialist that sacrifices head accuracy in exchange for better tail behaviour \(D\)\. Because the errors are fully deterministic functions ofyy, any variation in a metric across test conditions is attributable purely to the metric’s sensitivity to the test\-set distribution rather than to randomness in the predictions\.

For each experiment, test sets are parametrised byα∈\[0,0\.95\]\\alpha\\in\[0,0\.95\]: atα=0\\alpha=0the test set is drawn uniformly over the target range, while atα=0\.95\\alpha=0\.95it concentrates around the corresponding distribution\. We test three target distribution families: Gaussian \(symmetric\), Laplace \(symmetric, heavy\-tailed\), and Weibull \(right\-skewed\)\. The cap atα=0\.95\\alpha=0\.95guarantees a minimum of5%5\\%uniform samples, which prevents empty edge bins and avoids the discontinuities that occur when a test set has zero support in parts of the target range\. All three distributions and the corresponding error profiles can be seen in[Figure 8](https://arxiv.org/html/2609.25152#A4.F8)\.

![Refer to caption](https://arxiv.org/html/2609.25152v1/experiment_setup.png)Figure 8:Experimental Setup wiht all three distributions and the corresponding error profiles of the four synthetic models overlaidFor each metric we report \(i\) the coefficient of variationCV=std/\|mean\|\\mathrm\{CV\}=\\mathrm\{std\}/\|\\mathrm\{mean\}\|across theα\\alphasweep \(lower==more stable\) and \(ii\) ranking stability: the number of distinct model orderings observed and Kendall’sτ\\taubetween the rankings atα=0\\alpha=0andα=0\.95\\alpha=0\.95\.

#### D\.1\.2Results

[Figure 9](https://arxiv.org/html/2609.25152#A4.F9)shows the values of seven candidate metrics across theα\\alphasweep on the Weibull distribution the most challenging setting due to its heavy right skew\. Standard MAE, MSE,R2R^\{2\}, the geometric mean of absolute errors \(GM;[Yang et al\. \[2\]](https://arxiv.org/html/2609.25152#bib.bib39)\), and SERA\[[50](https://arxiv.org/html/2609.25152#bib.bib51)\]all drift withα\\alphafor at least one model, indicating that the reported metric value depends on the empirical test\-set composition rather than only on the model itself\. The balanced metrics \(bMAE and bMASE\) remain flat acrossα\\alpha\.

[Table 6](https://arxiv.org/html/2609.25152#A4.T6)quantifies these observations across all three distribution families by computing the per\-model CV across theα\\alphasweep and the resulting ranking stability\. SERA, despite its threshold\-free design, is among the least stable metrics \(per\-model CV up to0\.5150\.515on Weibull and0\.4520\.452on Gaussian\)\. This arises because SERA computes an unnormalised*sum*of squared errors at each relevance threshold; when the test distribution shifts, the sample count above each threshold changes, conflating test\-set composition with model quality\. The proposed balanced metrics achieve near\-perfect invariance: per\-model CV≤0\.005\\leq 0\.005across all distributions and exactly one unique ranking across allα\\alphavalues, withτ\(↔0\.95\)=\+1\.00\\tau\(0\\\!\\leftrightarrow\\\!0\.95\)=\+1\.00\.[Figure 10](https://arxiv.org/html/2609.25152#A4.F10)additionally visualises theα\\alpha\-sweep curves for Laplace and Gaussian targets to confirm that the qualitative pattern of[Figure 9](https://arxiv.org/html/2609.25152#A4.F9)holds in the symmetric setting\.

We emphasise that the analysis in[Figure 9](https://arxiv.org/html/2609.25152#A4.F9),[Figure 10](https://arxiv.org/html/2609.25152#A4.F10), and[Table 6](https://arxiv.org/html/2609.25152#A4.T6)is not intended as a blanket judgement against MAE, MSE,R2R^\{2\}, GM, or SERA\. Each of these metrics has well\-established uses and remains appropriate in many regression settings\. What we report is specifically the behaviour that becomes problematic under the*uniform\-performance objective of DIR*: when the goal is to characterise model quality consistently across the entire target range, sensitivity to test\-set composition is undesirable, and our experiments show that the balanced metrics are the only candidates that meet this criterion\.

Figure 9:Metric values across test\-distribution shifts \(α=0\\alpha=0: uniform;α=0\.95\\alpha=0\.95: Weibull\-peaked\) for the four deterministic models\. Standard metrics \(top\) and SERA drift substantially withα\\alpha, while the balanced metrics \(bMAE,bMASE\) remain flat and produce a single stable ranking\.Table 6:Distribution invariance of evaluation metrics across the three target distribution families\. We report the per\-model CV across theα\\alphasweep \(lower==more stable\), the number of distinct model rankings observed acrossα∈\[0,0\.95\]\\alpha\\in\[0,0\.95\], and Kendall’sτ\\taubetween the rankings at the endpoints\. The proposed balanced metrics \(bMAE, bMASE\) maintain near\-perfect invariance across all three settings\.CV acrossα\\alpha↓\\downarrow\# Unique↓\\downarrowτ\(↔0\.95\)\\tau\(0\\\!\\leftrightarrow\\\!0\.95\)↑\\uparrowDistributionMetricABCDGaussianMAE0\.2850\.0130\.1610\.0303\+0\.60\+0\.60MSE0\.4240\.0260\.2250\.0642\+0\.80\+0\.80R2R^\{2\}0\.0130\.0080\.0580\.0012\+0\.80\+0\.80GM0\.2930\.0130\.2130\.0294\+0\.40\+0\.40SERA0\.4520\.2210\.1330\.2542\+0\.80\+0\.80bMAE0\.0020\.0000\.0020\.0001\+1\.00\+1\.00bMASE0\.0010\.0020\.0040\.0021\+1\.00\+1\.00LaplaceMAE0\.3050\.0140\.1950\.0323\+0\.60\+0\.60MSE0\.4080\.0280\.2790\.0662\+0\.80\+0\.80R2R^\{2\}0\.0080\.0090\.0870\.0012\+0\.80\+0\.80GM0\.3430\.0130\.2510\.0314\+0\.40\+0\.40SERA0\.4250\.2590\.1300\.2872\+0\.80\+0\.80bMAE0\.0030\.0000\.0030\.0011\+1\.00\+1\.00bMASE0\.0010\.0020\.0050\.0021\+1\.00\+1\.00WeibullMAE0\.3790\.0260\.1730\.0583\+0\.60\+0\.60MSE0\.5060\.0540\.2290\.1292\+0\.80\+0\.80R2R^\{2\}0\.0780\.0050\.0400\.0012\+0\.80\+0\.80GM0\.3630\.0240\.2600\.0523\+0\.40\+0\.40SERA0\.5150\.2700\.1530\.3332\+0\.80\+0\.80bMAE0\.0020\.0000\.0020\.0001\+1\.00\+1\.00bMASE0\.0010\.0010\.0030\.0011\+1\.00\+1\.00Figure 10:Same analysis as Figure[9](https://arxiv.org/html/2609.25152#A4.F9)for the Laplace \(top\) and Gaussian \(bottom\) target distributions\. The stability pattern of the balanced metrics is consistent across distribution families\.

### D\.2Theoretical grounding and continuous estimation of bMAE

The bMAE used in both[Equation 1](https://arxiv.org/html/2609.25152#S3.E1)and in bMASE \([Equation 2](https://arxiv.org/html/2609.25152#S3.E2)\) is a discrete estimator of the conditional absolute error integrated under the uniform measure over the target range:

bMAE⁡\(fθ\)≈1b−a​∫ab𝔼⁡\[\|Y−Y^\|\|Y=y\]​𝑑y\.\\mathrm\{bMAE\}\(f\_\{\\theta\}\)\\;\\approx\\;\\frac\{1\}\{b\-a\}\\int\_\{a\}^\{b\}\\mathbb\{E\}\\\!\\bigl\[\|Y\-\\hat\{Y\}\|\\,\\bigm\|\\,Y\\\!=\\\!y\\bigr\]\\,\\mathrm\{d\}y\\,\.\(7\)The uniform measure1b−a​d​y\\frac\{1\}\{b\-a\}\\,\\mathrm\{d\}yensures that every point in the target range contributes equally to the integral, independent of the empirical test\-set density\. The binned estimator in[Equation 1](https://arxiv.org/html/2609.25152#S3.E1)approximates this integral via the midpoint quadrature rule, with each bin’s per\-sample MAE serving as the function value at that bin’s centre\.

A continuous alternative is to estimate the integrand with Nadaraya–Watson \(NW\) kernel regression and integrate over a uniform grid:

bMAE^NW=1G​∑g=1G∑i=1NKh​\(yg−yi\)​\|yi−y^i\|∑i=1NKh​\(yg−yi\),\\widehat\{\\mathrm\{bMAE\}\}\_\{\\mathrm\{NW\}\}=\\frac\{1\}\{G\}\\sum\_\{g=1\}^\{G\}\\frac\{\\sum\_\{i=1\}^\{N\}K\_\{h\}\(y\_\{g\}\-y\_\{i\}\)\\,\|y\_\{i\}\-\\hat\{y\}\_\{i\}\|\}\{\\sum\_\{i=1\}^\{N\}K\_\{h\}\(y\_\{g\}\-y\_\{i\}\)\}\\,,\(8\)whereKhK\_\{h\}is a Gaussian kernel with bandwidthhhchosen by Silverman’s rule\[[51](https://arxiv.org/html/2609.25152#bib.bib1)\]andG=500G\\\!=\\\!500uniformly spaced grid points\. The grid densityGGis a numerical integration parameter rather than a statistical one; results are stable fromG≈100G\\\!\\approx\\\!100onward\.

Figure 11:Agreement between the continuous \(Nadaraya–Watson\) and discrete \(binned, Freedman–Diaconis\) bMAE estimators across test\-set sizes and distribution families \(α=0\.95\\alpha=0\.95\)\.Left:maximum relative error of the discrete estimator on bMAE\.Centre:maximum absolute\|Δ​bMASE\|\|\\Delta\\,\\mathrm\{bMASE\}\|\.Right:local MAE curves for Model A atNtest=10,000N\_\{\\mathrm\{test\}\}=10\{,\}000, with the smooth NW estimate \(blue line\) overlaid on discrete bin averages \(green bars\)\.[Figure 11](https://arxiv.org/html/2609.25152#A4.F11)compares the two estimators across seven test\-set sizes and three distribution families atα=0\.95\\alpha=0\.95\. ForNtest≥2000N\_\{\\mathrm\{test\}\}\\geq 2000, the discrete and continuous estimators agree to within11–5%5\\%on bMAE and to within0\.010\.01on bMASE across all three families\. The agreement tightens monotonically with sample size, reaching\|Δ​bMASE\|<0\.004\|\\Delta\\,\\mathrm\{bMASE\}\|<0\.004atNtest=10,000N\_\{\\mathrm\{test\}\}=10\{,\}000\. The bMASE agreement is somewhat tighter than the raw bMAE agreement because numerator and denominator share the same binning, which partially cancelsKK\-induced bias in the ratio\. The right column of[Figure 11](https://arxiv.org/html/2609.25152#A4.F11)confirms this visually: at the largest sample size, the discrete bin averages track the smooth NW estimate of the local MAE function across the entire target range for all three distributions\. We therefore adopt the discrete estimator as the default throughout this work\. It is simpler to implement, transparent in its assumptions, and produces identical model rankings to the continuous estimator on all settings considered\.

##### WhyL1L\_\{1\}rather than squared\-error balanced metrics\.

Balanced metrics macro\-average errors across target bins, so bins with few samples contribute as much as densely populated bins\. This makes the stability of each per\-bin estimate important\. For a sample\-mean estimator1n​∑iZi\\frac\{1\}\{n\}\\sum\_\{i\}Z\_\{i\}, the variance isVar⁡\(Zi\)/n\\mathrm\{Var\}\(Z\_\{i\}\)/n\. For MAE,Zi=\|ei\|Z\_\{i\}=\|e\_\{i\}\|, whose variance depends on𝔼⁡\[ei2\]\\mathbb\{E\}\[e\_\{i\}^\{2\}\]\. For MSE,Zi=ei2Z\_\{i\}=e\_\{i\}^\{2\}, whose variance depends on𝔼⁡\[ei4\]\\mathbb\{E\}\[e\_\{i\}^\{4\}\]\. Squared\-error estimates are therefore more sensitive to heavy\-tailed errors and outliers, especially whennkn\_\{k\}is small\[[39](https://arxiv.org/html/2609.25152#bib.bib13)\]\. This motivates our use of bMASE as anL1L\_\{1\}\-based balanced metric\.

## Appendix EStability Blind Spot Appendix

In addition to the stability analysis based on target imbalance, we characterise the label noise present in eachMuViStask, because label noise and target imbalance interact and confound evaluation\. In low\-density regions the empirical conditional expectation𝔼⁡\[y∣x\]\\mathbb\{E\}\[y\\mid x\]is estimated with higher variance\. In sensor\-based domains this interaction is compounded by physics: extreme operating conditions often coincide with higher measurement uncertainty due to sensor saturation or nonlinear transduction \(e\.g\. IEC 60584\-1 thermocouple tolerances scale with the measured value\)\. Characterising the noise landscape is therefore a prerequisite for interpreting DIR results: methods that up\-weight rare samples simultaneously amplify the unreliable gradients those samples carry\[[3](https://arxiv.org/html/2609.25152#bib.bib36)\]\.

##### Method\.

For each training sample\(xi,yi\)\(x\_\{i\},y\_\{i\}\)we identify itsk=20k\\\!=\\\!20nearest neighbours in PCA\-reduced feature space and compute the*normalised noise ratio*

ri=Var⁡\(\{yj:j∈𝒩k​\(i\)\}\)σ2​\(y\),r\_\{i\}\\;=\\;\\frac\{\\mathrm\{Var\}\\bigl\(\\\{y\_\{j\}:j\\in\\mathcal\{N\}\_\{k\}\(i\)\\\}\\bigr\)\}\{\\sigma^\{2\}\(y\)\}\\,,\(9\)where𝒩k​\(i\)\\mathcal\{N\}\_\{k\}\(i\)is the index set of thekknearest neighbours ofxix\_\{i\}andσ2​\(y\)\\sigma^\{2\}\(y\)is the global variance of the training targets\. A ratior≈0r\\\!\\approx\\\!0indicates a clean, locally deterministic mapping;r≈1r\\\!\\approx\\\!1means that local target disagreement matches the global spread;r\>1r\\\!\>\\\!1signals a multi\-valued or strongly heteroscedastic regime\. We bin per\-sample ratios across the target range and report the median and IQR per bin \([Figure 12](https://arxiv.org/html/2609.25152#A5.F12)\)\.

Figure 12:Normalisedkk\-NN noise ratio \([Equation 9](https://arxiv.org/html/2609.25152#A5.E9)\) as a function of target value for all datasets inMuViS\. Blue bars: sample count per bin \(left axis\)\. Orange line and band: median noise ratio with IQR \(right axis\)\. Dashed line: random baseline \(r=1r\\\!=\\\!1\)\.
##### Results\.

The noise profiles reveal systematic co\-occurrence of imbalance and noise \([Figure 12](https://arxiv.org/html/2609.25152#A5.F12)\)\. The severely imbalanced tasks exhibit the most pronounced tail noise: Monterey shows a U\-shaped profile rising fromr≈0\.05r\\\!\\approx\\\!0\.05in the centre tor\>1r\\\!\>\\\!1in both tails, reflecting the multi\-valued physics of near\-limit handling; PM10displays an asymmetric pattern with clean lower values \(r≈0\.1r\\\!\\approx\\\!0\.1\) but extreme upper\-tail noise \(r\>3\.5r\\\!\>\\\!3\.5\), consistent with sporadic, meteorologically driven pollution episodes; the Chem\. Conc\. shows a milder U\-shape that remains belowr=1r\\\!=\\\!1\. The moderately imbalanced tasks \(Hart Rate, Tire Temp\.\) exhibit more uniform profiles, while battery SoC maintainsr<0\.02r\\\!<\\\!0\.02throughout, confirming an almost deterministic mapping\.

This pattern carries a direct methodological consequence: DIR methods that up\-weight the tails of the severely imbalanced tasks must contend with amplified gradient noise alongside data scarcity, whereas on near\-uniform tasks like battery SoC any tail error is purely a capacity problem, which could be the ideal scenario for DIR corrections\. The noise profiles thus refine the imbalance\-only view of[Table 5](https://arxiv.org/html/2609.25152#A3.T5): imbalance ratio and effective support quantify*how*skewed the target distribution is, while[Figure 12](https://arxiv.org/html/2609.25152#A5.F12)quantifies how much of the residual tail\-region error a model can in principle eliminate\. Together, the two characterisations explain why uniform improvements across allMuViStasks are difficult to achieve: the tasks for which DIR methods are most needed are also the tasks on which their corrections are most likely to amplify noise rather than correct bias\.

## Appendix FImplementation and Reproducibility

This section summarizes the implementation choices required to reproduce the experiments in[Section 4](https://arxiv.org/html/2609.25152#S4)alongside with the public available code at[www\.github\.com/noah\-puetz/muvis\-dir](https://www.github.com/noah-puetz/muvis-dir)\. All methods use the same datasets, train/test splits, ResNet1D backbone family, evaluation metrics, and seed protocol unless stated otherwise\. All experiments were implemented in Python 3\.13\.7 and executed on a single NVIDIA H100 \(80 GB\) GPU with CUDA 13\.2\.

### F\.1Data, Splits, and Preprocessing

All experiments use the nineMuViS\-DIRdatasets introduced in the main text\. The preprocessed train/test files are inherited from[Brandt et al\. \[9\]](https://arxiv.org/html/2609.25152#bib.bib35); the DIR experiments do not introduce additional upstream preprocessing or target transformations\. Targets are predicted in their original physical units\.

For each random seed, the provided training split is further divided into a training and validation subset using a stratified random 90/10 split\. The test set remains fixed across all seeds and is used only for final evaluation\. Input features are standardized channel\-wise using statistics estimated on the training subset only; the same transformation is then applied to validation and test inputs\. Targets are not standardized\.

### F\.2Backbone and Training Protocol

All methods are built on the ResNet1D architecture selected in the originalMuViShyperparameter optimization\. We keep the dataset\-specific architecture parameters fixed and vary only the DIR method and its associated hyperparameters\. The retained architecture parameters are shown in[Table 7](https://arxiv.org/html/2609.25152#A6.T7)\.

Unless stated otherwise, models are trained with anL1L\_\{1\}regression loss, Adam optimization, no weight decay, no gradient clipping, and a stepwise learning\-rate decay by a factor of0\.10\.1at epochs 60 and 80\. Training runs for 120 epochs, and the checkpoint with the lowest validation MAE is restored before test evaluation\. This shared protocol applies to Vanilla, LDS, SQInv, Focal\-L1L\_\{1\}, and ConR\. UVote and RnC use method\-specific schedules, described below\.

Table 7:Dataset\-specific ResNet1D backbone used in the Section 4 experiments\. The architecture parameters are inherited from the originalMuViSResNet1D configuration\. Here,TTdenotes sequence length andCCthe number of input channels\.
### F\.3DIR Method Configurations

The evaluated DIR methods follow the formulations described in the main text and are adapted to multimodal time\-series regression\. LDS, SQInv, and Focal\-L1L\_\{1\}modify the regression loss through target\-density\- or residual\-dependent weighting\. ConR and RnC add contrastive objectives on the learned representation\. UVote replaces the single regression head with an ensemble of expert heads trained with an uncertainty\-aware objective\.[Table 8](https://arxiv.org/html/2609.25152#A6.T8)summarizes the method\-specific hyperparameters considered during configuration selection\.

Table 8:Method\-specific hyperparameters used for configuration selection\. Parameters not listed here follow the shared training protocol from[Subsection F\.2](https://arxiv.org/html/2609.25152#A6.SS2)\.MethodHyperparameterValues consideredLDSnumber of bins\{50,100\}\\\{50,100\\\}smoothing kernelGaussiankernel size\{5,9\}\\\{5,9\\\}smoothing bandwidth2\.02\.0reweighting ruleinverse or square\-root inverse frequencySQInvnumber of bins\{50,100\}\\\{50,100\\\}Focal\-L1L\_\{1\}β\\beta\{0\.01,0\.05,0\.1,0\.2,0\.5,1\.0,2\.0,5\.0,10\.0\}\\\{0\.01,0\.05,0\.1,0\.2,0\.5,1\.0,2\.0,5\.0,10\.0\\\}γ\\gamma\{1,2\}\\\{1,2\\\}ConRcontrastive weight\{0\.5,1\.0,2\.0,4\.0\}\\\{0\.5,1\.0,2\.0,4\.0\\\}relative label window\{0\.01,0\.02,0\.05,0\.10\}\\\{0\.01,0\.02,0\.05,0\.10\\\}of the training target rangetemperature0\.20\.2RnCcontrastive temperature\{0\.1,2\.0\}\\\{0\.1,2\.0\\\}stage\-1 learning rate\{0\.05,0\.1,0\.3,0\.5\}\\\{0\.05,0\.1,0\.3,0\.5\\\}stage\-1 epochs400stage\-2 epochs90UVotenumber of experts\{2,3\}\\\{2,3\\\}training schedule90 epochs, batch size 64

### F\.4Hyperparameter Selection

Hyperparameters are selected separately for each dataset–method pair\. The selection is performed at seed 42 before the final multi\-seed evaluation\. For methods with a small number of predefined variants, both variants are trained and compared\. For Focal\-L1L\_\{1\}, ConR, and RnC, we additionally run the method\-specific sweeps listed in[Table 8](https://arxiv.org/html/2609.25152#A6.T8)\.

The selected configuration is the one with the best validation balanced MAE at seed 42\. After this selection step, the chosen configuration for each dataset–method pair is fixed and retrained across all ten seeds\. After hyperparameter selection, one configuration is fixed for each dataset–method pair and reused for all ten seeds in the final evaluation\.[Table 9](https://arxiv.org/html/2609.25152#A6.T9)and[Table 10](https://arxiv.org/html/2609.25152#A6.T10)report the effective method\-specific parameters used in these final runs\. Parameters that are shared across all datasets, such as the ResNet1D backbone, optimizer, training length, and validation\-based checkpoint selection, are described in[Subsection F\.2](https://arxiv.org/html/2609.25152#A6.SS2)and are not repeated here\. Vanilla uses the shared training protocol without additional method\-specific parameters\.

Table 9:Selected configurations for the loss\- and reweighting\-based DIR methods\. For LDS,BBdenotes the number of target bins,kkthe Gaussian smoothing kernel size, andrrthe reweighting rule\. LDS uses Gaussian smoothing with bandwidthσ=2\.0\\sigma=2\.0in all cases\. For SQInv,BBdenotes the number of target bins\. For Focal\-L1L\_\{1\},β\\betaandγ\\gammadenote the focal\-loss parameters\.Table 10:Selected configurations for the contrastive and architectural DIR methods\. For ConR,wwdenotes the target\-distance window in raw target units andβ\\betathe contrastive\-loss weight; the temperature is fixed toτ=0\.2\\tau=0\.2and the hard\-negative coefficient toe=0\.01e=0\.01\. For RnC,τ\\taudenotes the contrastive temperature andηs​1\\eta\_\{s1\}the stage\-1 learning rate; all selected RnC configurations use 400 contrastive pretraining epochs followed by 90 linear\-probing epochs\. For UVote,KKdenotes the number of expert heads\.
### F\.5Seeds and Statistical Reporting

The final experiments are run with ten seeds:

\{42,43,44,45,46,47,48,49,50,51\}\.\\\{42,43,44,45,46,47,48,49,50,51\\\}\.Each seed controls the train/validation split, model initialization, data shuffling, dropout, and worker\-level randomness in the data\-loading pipeline\. We also set deterministic backend options where available\. However, strict bitwise determinism is not enforced, since some GPU operations may remain implementation\-dependent\. The reported results should therefore be interpreted as statistically reproducible rather than guaranteed bitwise identical across all hardware and software stacks\.

All tables in[Section 4](https://arxiv.org/html/2609.25152#S4)report the mean across the ten seeds\. Confidence intervals are computed from the seed\-wise results using the procedure described in the main text\. The final multi\-seed evaluation consists of

9​datasets×7​methods×10​seeds=6309\\text\{ datasets\}\\times 7\\text\{ methods\}\\times 10\\text\{ seeds\}=630training runs\.

## Appendix GExperiments Appendix

### G\.1Re\-evaluation ofMuViSModels

To contextualize the proposedMuViS\-DIRresults, we re\-evaluate the six originalMuViSbaselines under the balanced metrics introduced in[Subsection 3\.2](https://arxiv.org/html/2609.25152#S3.SS2)\. All results are averaged across ten random seeds and use the original training protocol of[Brandt et al\. \[9\]](https://arxiv.org/html/2609.25152#bib.bib35)\. In contrast to the DIR experiments in[Section 4](https://arxiv.org/html/2609.25152#S4), these models were trained with an MSE loss rather than anL1L\_\{1\}loss\. Differences between the ResNet1D results in[Table 11](https://arxiv.org/html/2609.25152#A7.T11)and[Table 12](https://arxiv.org/html/2609.25152#A7.T12)and the corresponding results in[Table 2](https://arxiv.org/html/2609.25152#S4.T2)and[Table 1](https://arxiv.org/html/2609.25152#S4.T1)therefore reflect the change in training loss\. This comparison provides additional context on how the choice of optimization objective affects balanced and tail\-sensitive performance\.

Table 11:Balanced Mean Absolute Error \(bMAE\)±\\pmstandard deviation across seeds per dataset and model\.Table 12:Balanced Mean Absolute Scaled Error \(bMASE\)±\\pmstandard deviation across seeds per dataset and model\.Table 13:Mean bMASE\[CI low, CI high\]across ten seeds for each method–dataset pair\. Values belowVanillaare italicized; the best value in each row is bold\. GMean aggregates across datasets via geometric mean\. Lower is better;bMASE<1\\mathrm\{bMASE\}<1improves over the trivial median predictor\.
### G\.2Performance comparison

The following figures complement the aggregate results in the main text by showing per\-bin MAE curves for all methods across the nineMuViS\-DIRdatasets\. These plots make the regional structure of the errors visible and show where individual methods improve over, match, or degrade relative to the vanilla baseline\. They also illustrate that improvements in aggregate bMAE or bMASE can arise from different error profiles across the target range, motivating the use of both scalar balanced metrics and per\-bin diagnostic plots\.

Finally,[Figure 16](https://arxiv.org/html/2609.25152#A7.F16)compares the best\-performing DIR method for each dataset directly against the vanilla baseline\. The signed difference plots highlight whether improvements are concentrated in sparse target regions or distributed more uniformly across the target range\.

Figure 13:Per\-bin MAE of the six representative DIR methods and the vanilla baseline across threeMuViStasks\[[9](https://arxiv.org/html/2609.25152#bib.bib35)\]\. From top to bottom, rows show PM10, Beijing PM25, and Battery SoC prediction\. The plots illustrate how method performance varies across the target range\.Figure 14:Per\-bin MAE of the six representative DIR methods and the vanilla baseline across threeMuViStasks\[[9](https://arxiv.org/html/2609.25152#bib.bib35)\]\. From top to bottom, rows show PPG, Tire\. Temp, and Chem\. Conc\. prediction\. The plots illustrate how method performance varies across the target range\.Figure 15:Per\-bin MAE of the six representative DIR methods and the vanilla baseline across threeMuViStasks\[[9](https://arxiv.org/html/2609.25152#bib.bib35)\]\. From top to bottom, rows show Monterey, Targa ’13, and Targa ’14 prediction\. The plots illustrate how method performance varies across the target range\.
Figure 16:Comparison of the respective best performing DIR methods against the vanilla baseline across all nineMuViS\-DIRdatasets\. For every figure top row compares per\-bin MAE; bottom row shows the signed difference to the baseline\.