Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins

# Don’t Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi‑Scale Mahalanobis Wins
Source: [https://arxiv.org/html/2605.21493](https://arxiv.org/html/2605.21493)
Rahul D Ray Department of Electronics and Electrical Engineering BITS Pilani, Hyderabad Campus f20242213@hyderabad\.bits\-pilani\.ac\.in

###### Abstract

The ability to detect out\-of\-distribution \(OOD\) inputs is fundamental to safe deployment of machine learning systems\. Yet, current methods often rely on feature representations that are optimised solely for classification accuracy, neglecting the distinct requirements of epistemic uncertainty\. We introduce GOEN \(Geometry\-Optimised Epistemic Network\), a simple pipeline that combines multi\-scale features, L2 normalisation, Mahalanobis distance, and a calibration head trained with real hard OOD examples\. Through systematic ablation we uncover a counter\-intuitive finding: CenterLoss, a popular regulariser for feature compactness, significantly degrades OOD detection performance, reducing average OOD AUROC from 0\.9483 to 0\.9366 despite improving classification accuracy\. The best variant, GOEN\-NoCenterLoss, achieves an average OOD AUROC of 0\.9483, surpassing all baselines including deep ensembles \(0\.8827\), KNN \(0\.8967\), and ODIN \(0\.8870\) on CIFAR\-10 benchmarks, while maintaining competitive in\-distribution accuracy\. Our results challenge the prevailing assumption that better classification geometry automatically leads to better epistemic uncertainty\. Instead, we show that overly tight feature clusters compress inter\-class margins and distort the covariance structure needed for effective OOD detection\. GOEN is efficient, training in under 20 minutes on a single GPU, and provides a practical blueprint for building AI systems that reliably recognise their own limitations\.

## 1Introduction

The safe deployment of machine learning systems in open\-world environments demands the ability to recognise when an input is outside the training distribution\. Out\-of\-distribution \(OOD\) detection is therefore a critical component of trustworthy AI, with applications ranging from autonomous driving to medical diagnosisFreelet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib20)\)\. In this work, we focus on the CIFAR\-10Krizhevskyet al\.\([2009](https://arxiv.org/html/2605.21493#bib.bib1)\)and SVHNNetzeret al\.\([2011](https://arxiv.org/html/2605.21493#bib.bib10)\)Netzeret al\.\([2011](https://arxiv.org/html/2605.21493#bib.bib10)\)benchmarks, building upon the ResNet\-18 architectureHeet al\.\([2016](https://arxiv.org/html/2605.21493#bib.bib13)\)as a standard backbone\.

Existing uncertainty quantification methods have explored various strategies\. Monte Carlo DropoutGal and Ghahramani \([2016](https://arxiv.org/html/2605.21493#bib.bib12)\)and Deep EnsemblesLakshminarayananet al\.\([2017](https://arxiv.org/html/2605.21493#bib.bib11)\)provide scalable approximations of model uncertainty, while post\-hoc calibration techniques such as temperature scalingGuoet al\.\([2017](https://arxiv.org/html/2605.21493#bib.bib2)\)improve predictive confidence\. Evidential deep learningSensoyet al\.\([2018](https://arxiv.org/html/2605.21493#bib.bib3)\)replaces the softmax with a Dirichlet distribution to capture both aleatoric and epistemic uncertainty\. For OOD detection, distance\-based approaches like k\-nearest neighboursSunet al\.\([2022](https://arxiv.org/html/2605.21493#bib.bib9)\)have proven effective, exploiting feature geometry\.

Beyond these, recent advances in mixture\-of\-expertsMu and Lin \([2025](https://arxiv.org/html/2605.21493#bib.bib14)\); Li and Duan \([2025](https://arxiv.org/html/2605.21493#bib.bib16)\)and adaptive architecturesSunet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib15)\); Zhouet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib17)\)highlight the importance of flexible feature representations\. Simultaneously, systematic reviews of visual feature learningAbdullahiet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib18)\)and domain\-specific applications such as brain tumour classificationPacalet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib19)\)underscore the need for robust feature spaces\. Yet, many existing methods optimise feature geometry primarily for classification accuracy, potentially overlooking the distinct requirements of epistemic uncertainty\.

We introduce the Geometry\-Optimised Epistemic Network \(GOEN\), a simple but highly effective pipeline for OOD detection\. GOEN consists of three stages: \(1\) training a multi\-scale ResNet\-18 backbone with standard cross\-entropy \(without CenterLoss\), \(2\) fitting class\-conditional Mahalanobis densities on L2\-normalised features, and \(3\) training a lightweight calibration head on a mixture of in\-distribution and real hard OOD examples\. Through systematic ablation we discover a surprising result: CenterLoss, a common regulariser for feature compactness, significantly degrades OOD performance, reducing average OOD AUROC from0\.94830\.9483to0\.93660\.9366despite improving classification accuracy\. This challenges the prevailing assumption that better classification geometry automatically yields better epistemic uncertainty\.

Our main contributions are: \(i\) the GOEN pipeline that achieves state\-of\-the\-art OOD detection on CIFAR\-10 benchmarks, outperforming all baselines; \(ii\) the counter\-intuitive finding that CenterLoss harms OOD detection; and \(iii\) a principled analysis of the role of multi\-scale features, spherical normalisation, and hard OOD calibration\. The code is publicly available to facilitate reproducibility\.

## 2Related Work

Our work draws on and contributes to several interconnected research areas: out\-of\-distribution \(OOD\) detection, uncertainty quantification in deep learning, the geometry of feature representations \(particularly neural collapse\), multi\-scale feature learning, and model calibration\. Below we organise the literature into these themes and position GOEN within them\.

### 2\.1Out\-of\-Distribution Detection

The problem of detecting test samples that lie outside the training distribution has been extensively studied\. Early methods rely on the softmax confidence of a classifier, but it is well known that neural networks can be overconfident on OOD dataHendryckset al\.\([2021](https://arxiv.org/html/2605.21493#bib.bib23)\)\. To overcome this, post\-hoc detectors operate on a pre\-trained network without modifying its training\. The Mahalanobis distance methodLeeet al\.\([2018](https://arxiv.org/html/2605.21493#bib.bib8)\)fits class\-conditional Gaussians to penultimate features; ODIN combines temperature scaling and input perturbation; energy\-based detectionLiuet al\.\([2020](https://arxiv.org/html/2605.21493#bib.bib6)\)uses the log\-sum\-exp of logits as a score\. More recently, distance\-based approaches have regained prominence:Sunet al\.\([2022](https://arxiv.org/html/2605.21493#bib.bib9)\)demonstrate that simple k\-nearest neighbour distance in the feature space achieves state\-of\-the\-art OOD detection\.Chenet al\.\([2020](https://arxiv.org/html/2605.21493#bib.bib24)\)propose ALOE, which employs robust training using adversarially crafted inlier and outlier examples\.Mohseniet al\.\([2020](https://arxiv.org/html/2605.21493#bib.bib25)\)introduce a self\-supervised technique that does not require prior knowledge of OOD distributions\.Leeet al\.\([2020](https://arxiv.org/html/2605.21493#bib.bib26)\)present Deep\-MCDD, which learns a spherical decision boundary per class\.Freelet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib20)\)study how misclassification severity affects user trust, highlighting the practical importance of reliable OOD detection\.

Several surveys systematise the field\.Cui and Wang \([2022](https://arxiv.org/html/2605.21493#bib.bib21)\)categorises OOD methods by training data \(supervised, semi‑supervised, unsupervised\) and technical means\.Henrikssonet al\.\([2021](https://arxiv.org/html/2605.21493#bib.bib22)\)investigate how OOD detection performance varies with network training\.Hendryckset al\.\([2021](https://arxiv.org/html/2605.21493#bib.bib23)\)introduce new real‑world distribution shift datasets and evaluate robustness techniques\.Abdullahiet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib18)\)provide a broad review of visual feature learning, including OOD aspects\.

### 2\.2Uncertainty Quantification in Neural Networks

Quantifying predictive uncertainty is essential for safe deployment\. Monte Carlo dropoutGal and Ghahramani \([2016](https://arxiv.org/html/2605.21493#bib.bib12)\)treats dropout as an approximate Bayesian inference, capturing epistemic uncertainty via multiple stochastic forward passes\. Deep ensemblesLakshminarayananet al\.\([2017](https://arxiv.org/html/2605.21493#bib.bib11)\)train several independent models and average their predictions, providing a simple yet effective uncertainty estimate\. Evidential deep learningSensoyet al\.\([2018](https://arxiv.org/html/2605.21493#bib.bib3)\)replaces the softmax with a Dirichlet distribution, outputting a degree of belief\. EpiNetOsbandet al\.\([2023](https://arxiv.org/html/2605.21493#bib.bib4)\)augments a deterministic network with a trainable epinet that captures epistemic uncertainty via random perturbations\.Huseljicet al\.\([2021](https://arxiv.org/html/2605.21493#bib.bib31)\)propose AE\-DNN to separate aleatoric and epistemic uncertainty in a single forward pass, demonstrating utility for safety\-critical applications\.

Surveys includeKabiret al\.\([2018](https://arxiv.org/html/2605.21493#bib.bib27)\)on prediction intervals,Heet al\.\([2026](https://arxiv.org/html/2605.21493#bib.bib29)\)providing a taxonomy of UQ methods,Mitros and Mac Namee \([2019](https://arxiv.org/html/2605.21493#bib.bib28)\)comparing Bayesian and point‑estimate networks,Yaseen and Wu \([2023](https://arxiv.org/html/2605.21493#bib.bib30)\)benchmarking MCD, ensembles, and BNNs for nuclear engineering, andMosser and Zabihi Naeini \([2022](https://arxiv.org/html/2605.21493#bib.bib38)\)comparing deep ensembles, concrete dropout, and SWAG for seismic data\.

### 2\.3Feature Geometry and Neural Collapse

The phenomenon of neural collapse describes that in the terminal phase of training, features of each class collapse to their class means, and the classifier weights align with these means, forming a simplex equiangular tight frame \(ETF\)\. This geometry has profound implications for OOD detection\.Liu and Qin \([2025](https://arxiv.org/html/2605.21493#bib.bib35)\)leverage neural collapse to design an OOD detector that uses feature proximity to weight vectors and feature norms\.Chenet al\.\([2024](https://arxiv.org/html/2605.21493#bib.bib34)\)use collapse to guide feature alignment across environments for OOD generalization\.Momeniet al\.\([2026](https://arxiv.org/html/2605.21493#bib.bib33)\)propose Analytic Neural Collapse \(AnaNC\) to create structured feature geometries for continual OOD detection\.Haaset al\.\([2022](https://arxiv.org/html/2605.21493#bib.bib37)\)show that L2 normalisation induces early neural collapse and improves OOD detection\.Zhanget al\.\([2024](https://arxiv.org/html/2605.21493#bib.bib36)\)shape the feature distribution explicitly for OOD detection\.

Our work directly builds on these insights: we show that forcing feature compactness via CenterLoss—a regulariser that encourages neural collapse\-like behaviour—actually harms OOD detection, challenging the assumption that tighter clusters are always beneficial for epistemic uncertainty\. In contrast, GOEN achieves superior OOD detection by using L2 normalisation \(which induces early collapse\) and Mahalanobis distance on multi\-scale features without explicit compactness regularisation\.

### 2\.4Multi\-Scale Feature Learning and Data Augmentation

Multi\-scale representations capture both low‑level texture and high‑level semantics, which are crucial for detecting different types of distribution shifts\.Abdullahiet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib18)\)andPacalet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib19)\)discuss multi\-scale feature learning in vision tasks\.Mohseniet al\.\([2020](https://arxiv.org/html/2605.21493#bib.bib25)\)employ self‑supervision for generalizable OOD detection\. Data augmentation is another important tool:Thulasidasanet al\.\([2019](https://arxiv.org/html/2605.21493#bib.bib32)\)show that Mixup training improves calibration and reduces overconfidence on OOD data\.Hendryckset al\.\([2021](https://arxiv.org/html/2605.21493#bib.bib23)\)introduce DeepAugment, which significantly improves OOD generalisation\.Freelet al\.\([2025](https://arxiv.org/html/2605.21493#bib.bib20)\)also touch on the role of augmentation in trust\. Our GOEN uses a concatenation of layer2 and layer4 features of a ResNet\-18, providing complementary texture and semantic signals, and employs strong augmentations as part of the synthetic OOD mix during calibration\.

### 2\.5Calibration of Neural Networks

Calibration—the alignment between predicted confidence and actual accuracy—is crucial for trustworthy predictions\. Temperature scalingGuoet al\.\([2017](https://arxiv.org/html/2605.21493#bib.bib2)\)learns a single scalar temperature on a validation set\. More advanced techniques include entropy‑based scalingBalanyaet al\.\([2024](https://arxiv.org/html/2605.21493#bib.bib40)\), differentiable soft calibration objectives, and methods addressing both over‑ and under‑confidenceAoet al\.\([2023](https://arxiv.org/html/2605.21493#bib.bib39)\)\.Thulasidasanet al\.\([2019](https://arxiv.org/html/2605.21493#bib.bib32)\)show that Mixup training improves calibration\.van der Laan and Alaa \([2024](https://arxiv.org/html/2605.21493#bib.bib41)\)combine Venn‑Abers calibration with conformal prediction to deliver calibrated point predictions and prediction intervals\.

Our GOEN calibration head is a small MLP trained with binary cross‑entropy on a mixture of ID and synthetic OOD examples \(including real SVHN\)\. It maps three features \(log‑Mahalanobis distance, maximum cosine similarity, predictive entropy\) to a calibrated probability of being ID, effectively fusing geometric and predictive uncertainty signals\.

Existing OOD detection and uncertainty quantification methods either rely on post‑hoc statistics of features trained only for classification \(Mahalanobis, KNN, energy\) or modify the training objective to incorporate uncertainty \(deep ensembles, evidential learning, EpiNet\)\. However, they often do not explicitly consider the geometry of the feature space and its impact on OOD separation\. Our work contributes a simple yet powerful pipeline that incorporates multi\-scale features, L2 normalisation, Mahalanobis distance, and a calibrated head trained with real hard OOD examples\. Moreover, we reveal that forcing feature compactness via CenterLoss is detrimental—a counter‑intuitive result that challenges common assumptions\. GOEN thus offers a principled, practical, and highly effective approach to epistemic uncertainty\.

## 3Datasets and Preprocessing

All experiments are conducted on a common in‑distribution \(ID\) dataset, three out‑of‑distribution \(OOD\) datasets representing different types of distribution shift, and a corruption benchmark for robustness evaluation\. Preprocessing steps are standardised across all datasets to ensure fair comparison\. The following sections describe each dataset and the preprocessing applied\.

### 3\.1In‑Distribution Data: CIFAR\-10

CIFAR\-10Krizhevskyet al\.\([2009](https://arxiv.org/html/2605.21493#bib.bib1)\)consists of 60 000 colour images of size32×3232\\times 32, evenly distributed across 10 classes \(airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck\)\. The original split contains 50 000 training images and 10 000 test images\. We further split the training set into a training subset \(45 000 images\) and a validation subset \(5 000 images\) using a fixed random permutation to guarantee reproducibility\. The validation set is used for model selection, early stopping, and hyperparameter tuning\.

For training, we apply standard data augmentation: random horizontal flipping and random cropping of size3232with44pixels of padding\. These augmentations are applied only to the training subset to improve generalisation while keeping validation and test data unchanged\. All images are normalised using the channel‑wise mean and standard deviation computed on the CIFAR\-10 training set:

μ=\(0\.4914,0\.4822,0\.4465\),σ=\(0\.2023,0\.1994,0\.2010\)\.\\mu=\(0\.4914,\\;0\.4822,\\;0\.4465\),\\qquad\\sigma=\(0\.2023,\\;0\.1994,\\;0\.2010\)\.Normalisation is applied to every image before feeding it into the network, ensuring that the input distributions are centred and scaled consistently\.

### 3\.2Out‑of‑Distribution Datasets

We select three OOD datasets that span a spectrum of distribution shifts: domain shift \(SVHN\), semantic shift \(a filtered subset of CIFAR\-100Krizhevskyet al\.\([2009](https://arxiv.org/html/2605.21493#bib.bib1)\)\), and extreme distributional shift \(synthetic Gaussian noise\)\. Each OOD dataset is used exclusively for evaluation; only the ID dataset is used for training the backbone and calibration head, except for the calibration stage where a subset of SVHNNetzeret al\.\([2011](https://arxiv.org/html/2605.21493#bib.bib10)\)is optionally used as hard OOD examples\.

#### 3\.2\.1Domain Shift: SVHN

The Street View House Numbers \(SVHN\) datasetNetzeret al\.\([2011](https://arxiv.org/html/2605.21493#bib.bib10)\)contains colour images of house numbers collected from Google Street View\. Its appearance \(digits on natural backgrounds\) differs substantially from the object\-centric CIFAR\-10 images, making it an ideal domain\-shift OOD\. We use the official test split, which comprises 26 032 images\. To avoid data leakage, we randomly split the test set into two disjoint parts: a calibration subset of 5 000 images \(used only for training the calibration head\) and an evaluation subset of the remaining 21 032 images \(used exclusively for testing\)\. All images are resized to32×3232\\times 32and normalised using the same CIFAR\-10 statistics\. No additional preprocessing is applied, and labels are ignored during OOD evaluation\.

#### 3\.2\.2Semantic Shift: Filtered CIFAR\-100

CIFAR\-100Krizhevskyet al\.\([2009](https://arxiv.org/html/2605.21493#bib.bib1)\)contains 100 fine‑grained classes, many of which are semantically related to CIFAR\-10 categories \(e\.g\., “bear”, “fox” versus CIFAR\-10’s “dog” and “cat”\)\. To create a pure semantic shift with minimal overlap, we select a subset of 50 classes that have no direct semantic correspondence with CIFAR\-10\. The chosen classes are drawn from categories such as flowers, fruits, household items, insects, and natural scenes—all absent from CIFAR\-10\. This selection is fixed and used for all experiments\. We use the CIFAR\-100 test split and retain only images belonging to the selected classes, yielding approximately 5 000 samples\. Normalisation follows the CIFAR\-10 statistics, and labels are discarded for OOD evaluation\.

#### 3\.2\.3Extreme Distributional Shift: Synthetic Gaussian Noise

To evaluate a model’s ability to recognise inputs that are completely alien to the training distribution, we generate 5 000 synthetic images of pure Gaussian noise\. Each image is created by sampling pixel values from a normal distribution with mean0\.50\.5and standard deviation0\.50\.5, clipping the result to the\[0,1\]\[0,1\]range, and then normalising using the CIFAR\-10 statistics\. This process yields images with no structure, providing a strong test of the model’s epistemic awareness\.

### 3\.3Preprocessing Details and Rationale

All images are processed using the same pipeline: conversion to a floating‑point tensor, normalisation with CIFAR\-10 mean and standard deviation, and \(for training\) the augmentations described above\. Several design choices are motivated by the geometric nature of the proposed GOEN method:

- •Normalisation with CIFAR\-10 statisticsensures that features extracted from ID and OOD data are on a comparable scale, which is critical for distance‑based detectors such as Mahalanobis distance and cosine similarity\.
- •L2 normalisationis applied to features before computing Mahalanobis distance or cosine similarity, as this mitigates the effect of feature collapse \(where a single principal component dominates the variance\) and places all features on a unit sphere, improving numerical stability\.
- •Multi‑scale feature extractionleverages both early and late layers of the backbone\. Layer2 \(128‑dimensional\) is known to capture texture and domain shifts, while layer4 \(512‑dimensional\) captures semantic content\. Concatenating these two sources provides complementary signals for OOD detection\.
- •Calibration with real OOD examplesuses a fixed subset of SVHNNetzeret al\.\([2011](https://arxiv.org/html/2605.21493#bib.bib10)\)as hard OOD during the calibration phase, teaching the model to assign high uncertainty to inputs that are visually distinct yet occupy similar feature directions as ID data\. Gaussian noise serves as an easy OOD baseline to ensure separation of completely unstructured inputs\.

![Refer to caption](https://arxiv.org/html/2605.21493v1/fig2_analysis_light.png)Figure 1:Summary of key experimental results\. Top left: Phase 1 training \(cross\-entropy \+ CenterLoss\) showing validation accuracy vs\. epoch\. Top center: Phase 3 calibration convergence showing uncertainty score vs\. epoch\. Top right: Feature compactness measured by inter/intra class ratio\. Bottom left: Ablation study comparing different model variants\. Bottom right: Reproducibility study showing AUROC vs\. epoch across multiple runs\.
### 3\.4Data Splits and Loaders

All data loaders are built with a batch size of128128for training and evaluation, except during feature extraction for Mahalanobis fitting, where a larger batch size of512512is used for efficiency\. Training data are shuffled each epoch; validation and test sets are processed sequentially\. A fixed random seed controls all stochastic operations \(dataset splitting, shuffling, augmentation\) to ensure reproducibility\. The datasets are downloaded automatically when not present, and missing files are handled gracefully, printing warnings instead of halting execution\.

By standardising the preprocessing and splits, we guarantee that all baseline models and the proposed GOEN method are evaluated under identical conditions, enabling fair and reproducible comparisons across all experiments\.

## 4Baseline Models and Evaluation Protocol

To establish a comprehensive benchmark for epistemic uncertainty and out‑of‑distribution detection, we implement a diverse set of baseline models spanning both predictive uncertainty methods and post‑hoc OOD detectors\. All baselines share the same data splits, preprocessing, and evaluation metrics to ensure fair comparison\. This section details the architecture, training procedure, and uncertainty extraction for each baseline\.

### 4\.1Common Training Setup

All predictive uncertainty models are trained on the CIFAR\-10 training split \(45 000 images\) and validated on the held‑out validation split \(5 000 images\)\. The backbone for all models is a ResNet\-18 adapted for32×3232\\times 32inputs: the first convolution uses a3×33\\times 3kernel with stride11and padding11, and no initial max‑pooling is applied\. This adaptation preserves spatial resolution and is standard practice for small image datasets\. The network consists of four residual stages, each with two basic blocks, producing a final feature dimension of512512before the classification head\.

Training is performed using stochastic gradient descent with Nesterov momentum of0\.90\.9and weight decay of5×10−45\\times 10^\{\-4\}\. The initial learning rate is0\.10\.1, and we use a cosine annealing scheduler over the total number of epochs \(100100\)\. Early stopping is applied with a patience of1010epochs based on validation loss\. All models are trained with a fixed random seed \(4242\) for reproducibility, except for the deep ensemble where multiple seeds are used to obtain independent members\. For all models, we employ the same data augmentation \(random horizontal flip and random crop of size3232with44‑pixel padding\) during training\.

### 4\.2Predictive Uncertainty Baselines

#### 4\.2\.1Standard Neural Network \(Softmax Baseline\)

The simplest baseline is a standard ResNet\-18 trained with cross‑entropy loss\. Its classification head is a linear layer mapping the512512‑dimensional feature vector to the1010output classes\. During inference, the model produces a softmax probability vector\. The uncertainty score is taken as one minus the maximum softmax probability, under the assumption that low‑confidence predictions correspond to high uncertainty and potentially out‑of‑distribution inputs\. This model serves as a baseline for both in‑distribution accuracy and OOD detection performance\.

#### 4\.2\.2Monte Carlo Dropout

Monte Carlo dropoutGal and Ghahramani \([2016](https://arxiv.org/html/2605.21493#bib.bib12)\)extends the standard network by retaining dropout at test time\. We modify the ResNet\-18 by inserting a dropout layer \(rate0\.50\.5\) before the final linear classification head\. During inference, we perform2020stochastic forward passes, each with a different dropout mask\. The predictive distribution is obtained by averaging the softmax outputs of these passes\. Epistemic uncertainty is quantified via the mutual information between the predictions and the model parameters, computed as the entropy of the average prediction minus the average entropy of the individual predictions\. This measure captures the disagreement among the stochastic forward passes and is a common proxy for epistemic uncertainty\.

#### 4\.2\.3Deep Ensemble

Deep ensemblesLakshminarayananet al\.\([2017](https://arxiv.org/html/2605.21493#bib.bib11)\)are a widely used approach for uncertainty estimation\. We train five independent ResNet\-18 models with the same architecture but different random seeds\. The seeds are chosen arbitrarily but fixed \(4242,123123,20242024,777777,314314\)\. During inference, we compute the softmax probabilities from each ensemble member and average them to obtain the final predictive distribution\. The uncertainty score is the sum of per‑class variances across the ensemble members, which measures the spread of predictions and is indicative of epistemic uncertainty\.

#### 4\.2\.4Temperature Scaling

TemperatureGuoet al\.\([2017](https://arxiv.org/html/2605.21493#bib.bib2)\)is a post‑hoc calibration method applied to a trained standard neural network\. It introduces a single scalar temperature parameterTTthat divides the logits before the softmax function\. The temperature is learned on the validation set by minimising the negative log‑likelihood using L‑BFGS optimisation\. After calibration, the softmax outputs are better calibrated, but the ranking of uncertainty scores remains unchanged\. We use the temperature‑scaled network as a baseline for ID calibration and also evaluate its OOD detection performance using the same max‑softmax score\.

#### 4\.2\.5Evidential Deep Learning

Evidential deep learningSensoyet al\.\([2018](https://arxiv.org/html/2605.21493#bib.bib3)\)replaces the softmax output with a Dirichlet distribution over class probabilities\. The network outputs non‑negative evidence for each class, which is transformed into Dirichlet parametersαc=ReLU​\(logitc\)\+1\\alpha\_\{c\}=\\mathrm\{ReLU\}\(\\text\{logit\}\_\{c\}\)\+1\. The training loss combines a multinomial negative log‑likelihood term derived from the Dirichlet distribution and a Kullback‑Leibler divergence term that pushes the total evidence to zero for misclassified examples\. The KL term is annealed over the first half of the training epochs\. For inference, the predicted class probabilities are the expectationsαc/∑kαk\\alpha\_\{c\}/\\sum\_\{k\}\\alpha\_\{k\}\. The vacuity \(total evidence\) is used as an uncertainty score: higher vacuity indicates that the model has little evidence and therefore should be considered uncertain\.

#### 4\.2\.6EpiNet

EpiNetOsbandet al\.\([2023](https://arxiv.org/html/2605.21493#bib.bib4)\)combines a standard neural network with an additional “epinet” that adds a trainable perturbation to the logits, conditioned on a random noise vector and the penultimate features\. The epinet is a small multi‑layer perceptron that takes the concatenation of features and a random noise vector drawn from a standard normal distribution\. The logits are computed as the sum of the base network’s logits and the epinet output\. The prior network is a copy of the epinet with frozen parameters\. During training, the epinet learns to adjust the logits in a way that captures epistemic uncertainty\. For inference, we perform multiple forward passes with different noise samples \(2020passes\) and compute the mutual information between the predicted class and the noise, which serves as an epistemic uncertainty score\.

#### 4\.2\.7Mixture of Experts

The mixture of experts modelJacobset al\.\([1991](https://arxiv.org/html/2605.21493#bib.bib5)\)uses a gating network to route each input to one of several expert networks\. The backbone is the same ResNet\-18, and we define five experts, each a small two‑layer MLP \(hidden size6464\) that produces class logits\. A linear gate takes the backbone features and outputs a softmax distribution over the experts\. The final prediction is the weighted sum of the expert logits\. The gating weights themselves can be interpreted as a form of uncertainty: if the gate assigns high probability to a single expert, the model is confident; if the weights are spread across multiple experts, the model is uncertain\. We therefore use one minus the maximum gate weight as the uncertainty score\.

### 4\.3Post‑hoc Out‑of‑Distribution Detectors

These methods do not require retraining; they operate on a pre‑trained standard neural network \(the same ResNet\-18 trained with cross‑entropy\)\. They extract features or logits from the network and compute a scalar score that indicates how likely an input is to be out‑of‑distribution\. Higher scores correspond to higher OOD likelihood\.

#### 4\.3\.1Energy Score

The energy scoreLiuet al\.\([2020](https://arxiv.org/html/2605.21493#bib.bib6)\)is defined asE​\(x\)=−T​log​∑cefc​\(x\)/TE\(x\)=\-T\\log\\sum\_\{c\}e^\{f\_\{c\}\(x\)/T\}, wherefc​\(x\)f\_\{c\}\(x\)are the logits andTTis a temperature parameter \(set to1\.01\.0\)\. This score is based on the observation that the energy of a well‑calibrated classifier is lower for in‑distribution samples and higher for OOD samples\.

#### 4\.3\.2ODIN

ODINLianget al\.\([2017](https://arxiv.org/html/2605.21493#bib.bib7)\)\(Out‑of‑Distribution Detection using Input Preprocessing\) combines temperature scaling with input perturbation\. First, the input is perturbed in the direction of the gradient of the softmax score with respect to the input\. Then, the perturbed input is passed through the model with a high temperature \(10001000\), and the maximum softmax probability is taken\. The final OOD score is one minus that maximum probability, making it analogous to the standard softmax baseline but with temperature and input preprocessing\.

#### 4\.3\.3Mahalanobis Distance

This method fits a class‑conditional Gaussian distribution to the penultimate features of the training data\. For each classcc, the meanμc\\mu\_\{c\}of the features belonging to that class is computed, and a tied covariance matrixΣ\\Sigmais estimated across all classes\. During evaluation, for a test point, we compute its Mahalanobis distanceLeeet al\.\([2018](https://arxiv.org/html/2605.21493#bib.bib8)\)to each class centroid and take the minimum distance as the OOD score\. Larger distances indicate that the point is far from all class clusters, thus likely OOD\.

#### 4\.3\.4K‑Nearest Neighbour Distance

KNN distanceSunet al\.\([2022](https://arxiv.org/html/2605.21493#bib.bib9)\)is another distance‑based OOD detector\. We compute the cosine distance from each test feature to itskk‑th nearest neighbour in the training set \(using the penultimate features\)\. The distance itself serves as the OOD score: larger distances suggest that the test point lies in a region of feature space sparsely covered by training data, indicating OOD\.

### 4\.4Uncertainty Score Definitions

For consistency, we define a unified convention for all uncertainty scores:higher score indicates higher likelihood of being OOD\. This allows direct comparison of AUROC and other metrics\. Table[1](https://arxiv.org/html/2605.21493#S4.T1)summarises how each model’s raw output is converted to an uncertainty score\.

Table 1:Conversion of model outputs to OOD uncertainty scores\.
### 4\.5Evaluation Metrics

We evaluate all models on three aspects: in‑distribution accuracy and calibration, OOD detection performance, and robustness under distribution shift\.

#### 4\.5\.1In‑Distribution Performance

On the CIFAR\-10 test set \(10 00010\\,000images\), we report:

- •Accuracy: fraction of correctly classified examples\.
- •Expected Calibration Error \(ECE\): a measure of how well the predicted probabilities reflect the true likelihood\. We partition the confidence scores into1515equally spaced bins and compute the weighted average of the absolute difference between accuracy and confidence in each bin\.
- •Negative Log‑Likelihood \(NLL\): a proper scoring rule that penalises overconfident wrong predictions\.
- •Brier Score: the mean squared error between the predicted probabilities and the one‑hot labels\.

#### 4\.5\.2Out‑of‑Distribution Detection

For each OOD dataset \(SVHN, filtered CIFAR\-100, synthetic noise\), we compute the uncertainty scores for the ID test set and the OOD set\. We then evaluate:

- •Area Under the Receiver Operating Characteristic Curve \(AUROC\): measures the ability to separate ID from OOD\.
- •Area Under the Precision‑Recall Curve \(AUPR\): especially informative when the class distribution is imbalanced\.
- •False Positive Rate at 95% True Positive Rate \(FPR95\): a practical metric for applications where a high detection rate is required and false alarms are costly\.
- •Detection Accuracy: the accuracy of a binary classifier that thresholds the uncertainty score at the Youden index \(maximising true positive rate minus false positive rate\)\.

![Refer to caption](https://arxiv.org/html/2605.21493v1/Screenshot_10-4-2026_233856_.jpeg)Figure 2:Architecture of GOEN \(Geometry\-Optimised Epistemic Network\)\. A ResNet\-18 backbone \(trained with cross\-entropy only, no CenterLoss\) extracts multi\-scale features from layers 2 \(texture, 128\-dim\) and 4 \(semantic, 512\-dim\)\. After global average pooling, each is L2\-normalised to the unit sphere, then concatenated into a 640\-dim vectorz~\\tilde\{z\}\. Class meansμc\\mu\_\{c\}and tied covarianceΣ\\Sigmaare fitted on the training set\. The Mahalanobis scorem1=minc\(z~−μc\)TΣ−1\(z~−μc\)m\_\{1\}=\\min\_\{c\}\(\\tilde\{z\}\-\\mu\_\{c\}\)^\{T\}\\Sigma^\{\-1\}\(\\tilde\{z\}\-\\mu\_\{c\}\)is combined with two additional uncertainty features: cosine similaritym2m\_\{2\}from the FC head logits and entropym3m\_\{3\}\. These three features feed into a calibration headgψg\_\{\\psi\}\(MLP\) trained with binary cross\-entropy using synthetic noise and SVHN as hard OOD examples \(targets: 0\.05 for ID, 0\.95 for OOD\)\. The final outputu∈\(0,1\)u\\in\(0,1\)is the OOD score \(high = OOD\)\.

## 5The Geometry\-Optimised Epistemic Network \(GOEN\)

We introduce the Geometry\-Optimised Epistemic Network \(GOEN\), a simple yet highly effective pipeline for out‑of‑distribution detection\. The design is motivated by a systematic analysis of feature geometry and OOD detection mechanisms, which revealed three key principles: \(i\) features should be compact and well‑separated to enable distance‑based detectors; \(ii\) multi‑scale features capture complementary OOD signals at different semantic levels; \(iii\) training the final uncertainty estimator with realistic hard OOD examples is essential\. GOEN consists of three phases: \(1\) training a multi‑scale ResNet‑18 backbone with standard cross‑entropy \(no CenterLoss\), \(2\) fitting class‑conditional Mahalanobis densities on L2‑normalised features, and \(3\) training a small calibration head that maps a set of geometric uncertainty cues to a calibrated score\. The final model achieves state‑of‑the‑art OOD detection while maintaining high in‑distribution accuracy\.

### 5\.1Multi‑Scale Feature Extraction

The backbone is a ResNet‑18 adapted for32×3232\\times 32inputs, consisting of four stages with22,22,22,22basic blocks respectively\. Instead of using only the final layer features, we tap two intermediate layers: layer2 \(after the second stage, dimensionality128128\) and layer4 \(after the fourth stage, dimensionality512512\)\. These layers provide complementary information: layer2 captures low‑level texture and colour patterns, which are often indicative of domain shifts \(e\.g\., SVHN vs\. CIFAR‑10\), while layer4 captures high‑level semantic concepts, which are critical for detecting semantic shifts \(e\.g\., CIFAR‑100 classes not present in CIFAR‑10\)\. Formally, for an input imagexx, let

𝐟2​\(x\)∈ℝ128,𝐟4​\(x\)∈ℝ512\\mathbf\{f\}\_\{2\}\(x\)\\in\\mathbb\{R\}^\{128\},\\qquad\\mathbf\{f\}\_\{4\}\(x\)\\in\\mathbb\{R\}^\{512\}be the global average pooled features from layer2 and layer4 respectively\. These are concatenated to form a640640\-dimensional vector:

𝐟​\(x\)=\[𝐟2​\(x\)𝐟4​\(x\)\]\.\\mathbf\{f\}\(x\)=\\begin\{bmatrix\}\\mathbf\{f\}\_\{2\}\(x\)\\\\ \\mathbf\{f\}\_\{4\}\(x\)\\end\{bmatrix\}\.A small projection head reduces the dimensionality tod=512d=512while adding batch normalisation and ReLU:

𝐳​\(x\)=ReLU​\(BN​\(Wproj​𝐟​\(x\)\+bproj\)\)∈ℝ512\.\\mathbf\{z\}\(x\)=\\text\{ReLU\}\\bigl\(\\text\{BN\}\(W\_\{\\text\{proj\}\}\\mathbf\{f\}\(x\)\+b\_\{\\text\{proj\}\}\)\\bigr\)\\in\\mathbb\{R\}^\{512\}\.This multi‑scale feature vector𝐳​\(x\)\\mathbf\{z\}\(x\)serves as the input to both the classification head and the uncertainty module\.

### 5\.2Feature Normalisation and Mahalanobis Distance

To avoid numerical instability and mitigate the effect of feature collapse \(where a single principal component dominates the variance\), we normalise features to lie on the unit sphere before any distance computation\. This is crucial because the Mahalanobis distance relies on the covariance structure, which can be ill‑conditioned when features are highly anisotropic\.

Given a set of training features\{𝐳i\}i=1N\\\{\\mathbf\{z\}\_\{i\}\\\}\_\{i=1\}^\{N\}with labelsyi∈\{1,…,C\}y\_\{i\}\\in\\\{1,\\dots,C\\\}, we first compute class‑conditional means on the normalised sphere:

𝝁c=1Nc​∑i:yi=c𝐳i‖𝐳i‖2,\\boldsymbol\{\\mu\}\_\{c\}=\\frac\{1\}\{N\_\{c\}\}\\sum\_\{i:y\_\{i\}=c\}\\frac\{\\mathbf\{z\}\_\{i\}\}\{\\\|\\mathbf\{z\}\_\{i\}\\\|\_\{2\}\},whereNcN\_\{c\}is the number of training examples of classcc\. The tied covariance matrix is estimated as

𝚺=1N​∑c=1C∑i:yi=c\(𝐳i‖𝐳i‖2−𝝁c\)​\(𝐳i‖𝐳i‖2−𝝁c\)⊤\+ϵ​𝐈,\\boldsymbol\{\\Sigma\}=\\frac\{1\}\{N\}\\sum\_\{c=1\}^\{C\}\\sum\_\{i:y\_\{i\}=c\}\\left\(\\frac\{\\mathbf\{z\}\_\{i\}\}\{\\\|\\mathbf\{z\}\_\{i\}\\\|\_\{2\}\}\-\\boldsymbol\{\\mu\}\_\{c\}\\right\)\\left\(\\frac\{\\mathbf\{z\}\_\{i\}\}\{\\\|\\mathbf\{z\}\_\{i\}\\\|\_\{2\}\}\-\\boldsymbol\{\\mu\}\_\{c\}\\right\)^\{\\\!\\\!\\top\}\+\\epsilon\\mathbf\{I\},with a small regularisationϵ=10−5\\epsilon=10^\{\-5\}to ensure invertibility\. The precision matrix𝚲=𝚺−1\\boldsymbol\{\\Lambda\}=\\boldsymbol\{\\Sigma\}^\{\-1\}is then computed\.

For a test point with feature𝐳\\mathbf\{z\}, its Mahalanobis distance to each class is

dc​\(𝐳\)=\(𝐳‖𝐳‖2−𝝁c\)⊤​𝚲​\(𝐳‖𝐳‖2−𝝁c\),d\_\{c\}\(\\mathbf\{z\}\)=\\left\(\\frac\{\\mathbf\{z\}\}\{\\\|\\mathbf\{z\}\\\|\_\{2\}\}\-\\boldsymbol\{\\mu\}\_\{c\}\\right\)^\{\\\!\\\!\\top\}\\boldsymbol\{\\Lambda\}\\left\(\\frac\{\\mathbf\{z\}\}\{\\\|\\mathbf\{z\}\\\|\_\{2\}\}\-\\boldsymbol\{\\mu\}\_\{c\}\\right\),and the overall Mahalanobis score is the minimum across classes:

Maha​\(𝐳\)=minc⁡dc​\(𝐳\)\.\\text\{Maha\}\(\\mathbf\{z\}\)=\\min\_\{c\}d\_\{c\}\(\\mathbf\{z\}\)\.This score is small for points that lie close to one of the class clusters and large for points far from all clusters, making it an effective OOD indicator\.

Algorithm 1GOEN: Geometry\-Optimised Epistemic Network1:Training set

𝒟train=\{\(xi,yi\)\}i=1N\\mathcal\{D\}\_\{\\text\{train\}\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}, validation set

𝒟val\\mathcal\{D\}\_\{\\text\{val\}\}, OOD sets

𝒟svhn\\mathcal\{D\}\_\{\\text\{svhn\}\}and

𝒟noise\\mathcal\{D\}\_\{\\text\{noise\}\}\.

2:Trained GOEN model

ℳ\\mathcal\{M\}with uncertainty score

u​\(x\)∈\(0,1\)u\(x\)\\in\(0,1\)\.

3:Stage 1: Backbone Training

4:Initialize multi\-scale ResNet\-18 backbone

fθf\_\{\\theta\}and classification head

hϕh\_\{\\phi\}\.

5:foreach epoch

e=1e=1to

E1E\_\{1\}do

6:forbatch

\(x,y\)∼𝒟train\(x,y\)\\sim\\mathcal\{D\}\_\{\\text\{train\}\}do

7:Compute multi\-scale features

𝐳=fθ​\(x\)\\mathbf\{z\}=f\_\{\\theta\}\(x\)\.

8:Compute logits

ℓ=hϕ​\(𝐳\)\\ell=h\_\{\\phi\}\(\\mathbf\{z\}\)\.

9:Compute loss

ℒCE=CrossEntropy​\(ℓ,y\)\\mathcal\{L\}\_\{\\text\{CE\}\}=\\text\{CrossEntropy\}\(\\ell,y\)\.

10:Update

θ,ϕ\\theta,\\phiwith SGD\.

11:endfor

12:endfor

13:Stage 2: Mahalanobis Density Fitting

14:Extract features

𝐳i=fθ​\(xi\)\\mathbf\{z\}\_\{i\}=f\_\{\\theta\}\(x\_\{i\}\)for all

\(xi,yi\)∈𝒟train\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{D\}\_\{\\text\{train\}\}\.

15:Normalise features:

𝐳~i=𝐳i/‖𝐳i‖2\\tilde\{\\mathbf\{z\}\}\_\{i\}=\\mathbf\{z\}\_\{i\}/\\\|\\mathbf\{z\}\_\{i\}\\\|\_\{2\}\.

16:forc = 1 to Cdo

17:Compute class mean

𝝁c=1Nc​∑i:yi=c𝐳~i\\boldsymbol\{\\mu\}\_\{c\}=\\frac\{1\}\{N\_\{c\}\}\\sum\_\{i:y\_\{i\}=c\}\\tilde\{\\mathbf\{z\}\}\_\{i\}\.

18:endfor

19:Compute tied covariance

𝚺=1N​∑c=1C∑i:yi=c\(𝐳~i−𝝁c\)​\(𝐳~i−𝝁c\)⊤\+ϵ​𝐈\\boldsymbol\{\\Sigma\}=\\frac\{1\}\{N\}\\sum\_\{c=1\}^\{C\}\\sum\_\{i:y\_\{i\}=c\}\(\\tilde\{\\mathbf\{z\}\}\_\{i\}\-\\boldsymbol\{\\mu\}\_\{c\}\)\(\\tilde\{\\mathbf\{z\}\}\_\{i\}\-\\boldsymbol\{\\mu\}\_\{c\}\)^\{\\top\}\+\\epsilon\\mathbf\{I\}\.

20:Compute precision

𝚲=𝚺−1\\boldsymbol\{\\Lambda\}=\\boldsymbol\{\\Sigma\}^\{\-1\}\.

21:Stage 3: Calibration Head Training

22:Freeze backbone

fθf\_\{\\theta\}\.

23:Initialize calibration head

gψg\_\{\\psi\}\(MLP with sigmoid output\)\.

24:foreach epoch

e=1e=1to

E2E\_\{2\}do

25:forbatch

\(x,y\)∼𝒟val\(x,y\)\\sim\\mathcal\{D\}\_\{\\text\{val\}\}do

26:Sample ID batch

BidB\_\{\\text\{id\}\}, OOD batch

BoodB\_\{\\text\{ood\}\}from

𝒟svhn∪𝒟noise\\mathcal\{D\}\_\{\\text\{svhn\}\}\\cup\\mathcal\{D\}\_\{\\text\{noise\}\}\.

27:Compute features

𝐳id=fθ​\(xid\)\\mathbf\{z\}\_\{\\text\{id\}\}=f\_\{\\theta\}\(x\_\{\\text\{id\}\}\),

𝐳ood=fθ​\(xood\)\\mathbf\{z\}\_\{\\text\{ood\}\}=f\_\{\\theta\}\(x\_\{\\text\{ood\}\}\)\.

28:L2\-normalise features\.

29:foreach sample

𝐳~\\tilde\{\\mathbf\{z\}\}do

30:Mahalanobis distance

m=minc\(𝐳~−𝝁c\)⊤𝚲\(𝐳~−𝝁c\)m=\\min\_\{c\}\(\\tilde\{\\mathbf\{z\}\}\-\\boldsymbol\{\\mu\}\_\{c\}\)^\{\\top\}\\boldsymbol\{\\Lambda\}\(\\tilde\{\\mathbf\{z\}\}\-\\boldsymbol\{\\mu\}\_\{c\}\)\.

31:Log\-Maha

m1=log⁡\(m\+1\)m\_\{1\}=\\log\(m\+1\)\.

32:Cosine similarity

m2=maxc⁡𝐳~⋅𝝁cm\_\{2\}=\\max\_\{c\}\\tilde\{\\mathbf\{z\}\}\\cdot\\boldsymbol\{\\mu\}\_\{c\}\.

33:Predictive entropy

m3=−∑cpc​log⁡pcm\_\{3\}=\-\\sum\_\{c\}p\_\{c\}\\log p\_\{c\}\(from

hϕ​\(𝐳\)h\_\{\\phi\}\(\\mathbf\{z\}\)\)\.

34:Concatenate

𝐦=\[m1,m2,m3\]\\mathbf\{m\}=\[m\_\{1\},m\_\{2\},m\_\{3\}\]\.

35:Compute uncertainty

u=gψ​\(𝐦\)u=g\_\{\\psi\}\(\\mathbf\{m\}\)\.

36:endfor

37:Compute loss

ℒ=BCE​\(uid,0\.05\)\+BCE​\(uood,0\.95\)\\mathcal\{L\}=\\text\{BCE\}\(u\_\{\\text\{id\}\},0\.05\)\+\\text\{BCE\}\(u\_\{\\text\{ood\}\},0\.95\)\.

38:Update

ψ\\psiwith Adam\.

39:endfor

40:endfor

41:Inference

42:For a test input

xx, compute features

𝐳=fθ​\(x\)\\mathbf\{z\}=f\_\{\\theta\}\(x\), L2\-normalise, compute

𝐦\\mathbf\{m\}as above, and output

u=gψ​\(𝐦\)u=g\_\{\\psi\}\(\\mathbf\{m\}\)\.

43:High

uuindicates OOD\.

### 5\.3Calibration Head

While the Mahalanobis score already provides a strong OOD signal, we further improve calibration by learning a mapping from three complementary uncertainty cues to a final scoreu∈\(0,1\)u\\in\(0,1\)that is trained with binary cross‑entropy on a mixture of in‑distribution and out‑of‑distribution examples\. The three cues are:

- •Log‑Mahalanobis distance:m1=log⁡\(Maha​\(𝐳\)\+1\)m\_\{1\}=\\log\(\\text\{Maha\}\(\\mathbf\{z\}\)\+1\)\(adding 1 avoidslog⁡\(0\)\\log\(0\)\)\.
- •Maximum cosine similarity to class means: m2=maxc⁡𝐳‖𝐳‖2⋅𝝁c,m\_\{2\}=\\max\_\{c\}\\frac\{\\mathbf\{z\}\}\{\\\|\\mathbf\{z\}\\\|\_\{2\}\}\\cdot\\boldsymbol\{\\mu\}\_\{c\},where𝝁c\\boldsymbol\{\\mu\}\_\{c\}are the class means on the unit sphere\. This measures how well the feature aligns with any known class\.
- •Predictive entropy: m3=−∑c=1Cpc​log⁡pc,pc=softmax​\(logits\)c,m\_\{3\}=\-\\sum\_\{c=1\}^\{C\}p\_\{c\}\\log p\_\{c\},\\qquad p\_\{c\}=\\text\{softmax\}\(\\text\{logits\}\)\_\{c\},which captures the classifier’s own uncertainty\.

These three features are concatenated into a vector𝐦=\[m1,m2,m3\]⊤∈ℝ3\\mathbf\{m\}=\[m\_\{1\},m\_\{2\},m\_\{3\}\]^\{\\top\}\\in\\mathbb\{R\}^\{3\}and passed through a small multi‑layer perceptron with two hidden layers \(6464and3232neurons, ReLU activations\) followed by a sigmoid output:

u​\(𝐳\)=σ​\(W2⋅ReLU​\(W1​𝐦\+b1\)\+b2\)∈\(0,1\)\.u\(\\mathbf\{z\}\)=\\sigma\\bigl\(W\_\{2\}\\cdot\\text\{ReLU\}\(W\_\{1\}\\mathbf\{m\}\+b\_\{1\}\)\+b\_\{2\}\\bigr\)\\in\(0,1\)\.The final uncertainty score isuu– higher values indicate stronger belief that the input is OOD\.

### 5\.4Training Procedure

GOEN is trained in three stages, each with a distinct objective\.

#### 5\.4\.1Stage 1: Backbone Training

The multi‑scale backbone is trained on the CIFAR‑10 training set using standard cross‑entropy loss with label smoothing \(smoothing factor0\.10\.1\)\. No auxiliary loss \(such as CenterLoss\) is applied; we train only the classification head and the feature extractor\. This stage lasts for up to4040epochs \(or until early stopping\), using SGD with momentum0\.90\.9, weight decay5×10−45\\times 10^\{\-4\}, and a cosine annealing learning rate schedule starting at0\.10\.1\. The validation set is used for early stopping\.

#### 5\.4\.2Stage 2: Mahalanobis Density Fitting

After the backbone is trained, we freeze its parameters and compute the class means𝝁c\\boldsymbol\{\\mu\}\_\{c\}and the tied covariance𝚺\\boldsymbol\{\\Sigma\}using the normalised features from the entire training set\. This is a one‑shot computation and requires no gradient optimisation\.

#### 5\.4\.3Stage 3: Calibration Head Training

The calibration head is trained while keeping the backbone and the Mahalanobis parameters frozen\. The training data consists of in\-distribution examples \(from the CIFAR\-10 validation set\) and a mix of synthetic OOD examples\. We use two types of OOD:

- •Hard OOD:5 000 images from the*calibration subset*of the SVHN test set \(disjoint from the evaluation set\)\. These provide a challenging domain shift that is known to be difficult to detect\.
- •Easy OOD:2 000 synthetic Gaussian noise images, generated as described in SectionLABEL:subsec:synth\.

During each training iteration, we sample a batch of in\-distribution images \(from the validation set\) and an equal number of OOD images \(50% SVHN calibration, 50% noise\)\. The calibration head is trained to output a target oftID=0\.05t\_\{\\text\{ID\}\}=0\.05for in\-distribution inputs andtOOD=0\.95t\_\{\\text\{OOD\}\}=0\.95for OOD inputs, using binary cross\-entropy loss\. The training runs for up to 20 epochs with Adam \(learning rate10−310^\{\-3\}\) and early stopping based on the gap between the mean OOD uncertainty and the mean ID uncertainty on a held\-out validation set\.

### 5\.5Why No CenterLoss?

Our initial hypothesis was that forcing features to be more compact \(via CenterLoss\) would improve OOD detection\. However, systematic ablation revealed the opposite: the version with CenterLoss \(default GOEN\) achieved an average OOD AUROC of0\.93660\.9366, while the variant without CenterLoss \(which we now adopt as the final model\) reached0\.94830\.9483\. This surprising result suggests that over‑tightening the feature clusters can reduce the ability to detect OOD points that lie between clusters or may distort the covariance structure that Mahalanobis relies on\. Consequently, the final GOEN modelexcludes CenterLoss, keeping the backbone trained solely with cross‑entropy\.

### 5\.6Theoretical Foundations

The design choices in GOEN are grounded in the following observations:

- •Multi‑scale featuresprovide two independent OOD signals: texture‑level \(layer2\) and semantic‑level \(layer4\)\. Their concatenation yields a richer representation than either alone, as evidenced by the ablation where using only layer4 reduces average AUROC from0\.94830\.9483to0\.92700\.9270\.
- •L2 normalisationbefore Mahalanobis fitting mitigates feature collapse \(the top principal component accounted for54\.9%54\.9\\%of variance in initial experiments\), leading to a better‑conditioned covariance matrix and more stable distances\.
- •Mahalanobis distanceexploits the full covariance structure of the feature space, which is more powerful than nearest‑neighbour distances \(KNN\) on the same features\. Indeed, even on random features, Mahalanobis outperformed KNN\.
- •Training the calibration head with real hard OOD \(SVHN\)explicitly teaches the model to assign high uncertainty to inputs that are visually different yet occupy similar feature directions as ID data\. This is crucial because synthetic noise alone is insufficient to capture domain shifts\.

### 5\.7Performance Comparison

Table[2](https://arxiv.org/html/2605.21493#S5.T2)presents the performance of all baseline models and the best GOEN variant \(NoCenterLoss\)\. ID metrics include accuracy, expected calibration error \(ECE\), negative log‑likelihood \(NLL\), and Brier score\. OOD metrics are area under the receiver operating characteristic curve \(AUROC\) for each OOD dataset and the average\.

Table 2:Performance comparison of baseline models and GOEN on CIFAR\-10 \(ID\) and OOD datasets\. Metrics are rows; models are columns\. Best per row inbold\. Dashes \(−\-\) indicate that the metric is not applicable: post\-hoc OOD detectors \(Energy, ODIN, Mahalanobis, KNN\) do not produce in‑distribution predictions, so ID metrics \(Accuracy, ECE, NLL, Brier\) are omitted\. GOEN\-NoCenterLoss is the variant without CenterLoss, which achieved the best OOD performance in our ablation study\.Notes:

- •ID metrics \(Accuracy, ECE, NLL, Brier\) are not applicable to post‑hoc OOD detectors because these methods do not train or modify the classifier; they only compute scores from a pre‑trained network\.
- •GOEN‑NoCenterLoss is the variant without CenterLoss, which achieved the best OOD performance in our ablation study\. The seeding study \(5 seeds\) for the default GOEN \(with CenterLoss\) gives ID accuracy0\.9344±0\.00130\.9344\\pm 0\.0013and average AUROC0\.9337±0\.00850\.9337\\pm 0\.0085\.

The results show that GOEN‑NoCenterLoss outperforms all baselines in average OOD detection, while maintaining competitive ID accuracy\. The gap is especially large on the hardest OOD datasets \(SVHN and CIFAR‑100\)\.

## 6Experimental Results

We evaluate the proposed GOEN architecture through a comprehensive experimental protocol that includes a systematic ablation study, a seeding experiment to assess stability, and a comparison with established baselines\. All experiments are conducted on a single NVIDIA GPU \(T4/P100\)\. The SVHN evaluation results reported in Table[2](https://arxiv.org/html/2605.21493#S5.T2)are computed on the evaluation subset \(21 032 images\) that was never used during calibration\. Hence, the strong performance on SVHN \(AUROC 0\.9372\) is not due to data leakage but reflects genuine generalisation to unseen domain\-shift examples\. The following subsections detail the findings\.

### 6\.1Ablation Study

To isolate the contribution of each design choice, we perform an ablation study on the CIFAR\-10 test set and the three OOD datasets, using a fixed random seed \(42\) for reproducibility\. Four configurations are considered:

1. 1\.GOEN\-Default: The full pipeline with multi‑scale features \(layer2\+layer4\), CenterLoss \(α=0\.01\\alpha=0\.01\), and calibration head trained with SVHN hard OOD plus Gaussian noise\.
2. 2\.NoCenterLoss: Same as default but with CenterLoss removed \(α=0\\alpha=0\)\.
3. 3\.SingleScale\-L4: Only layer4 features are used; the multi‑scale concatenation is disabled\. CenterLoss and SVHN calibration are retained\.
4. 4\.NoSVHN\-NoiseOnly: The calibration head is trained only on Gaussian noise; SVHN hard OOD is excluded\.

Table[3](https://arxiv.org/html/2605.21493#S6.T3)summarises the results\. The primary metric for OOD performance is the average AUROC across the three OOD datasets\. In‑distribution accuracy is also reported to verify that OOD improvements do not come at the cost of classification performance\.

Table 3:Ablation results for GOEN variants \(seed=42\)\.Several observations can be made\. First,removing CenterLoss\(NoCenterLoss\) increases the average OOD AUROC from0\.93660\.9366to0\.94830\.9483, a gain of\+0\.0117\+0\.0117\. This is a surprising result because CenterLoss was introduced to make features more compact \(as evidenced by the intra/inter ratio increasing from2\.52\.5to7\.57\.5\), yet the tighter clusters appear to hinder OOD detection\. We hypothesise that over‑tightening the feature space compresses the region between class clusters, reducing the distance that OOD points can be separated from ID points\. Moreover, the tied covariance matrix estimated on highly compact features may become ill‑conditioned, degrading the Mahalanobis score\. This finding challenges the common intuition that better classification geometry automatically translates to better epistemic uncertainty\.

Second,disabling multi‑scale features\(SingleScale\-L4\) drops the average AUROC to0\.92700\.9270, a decrease of0\.00960\.0096relative to the default\. This confirms that layer2 \(texture/domain\) and layer4 \(semantic\) provide complementary information, and their concatenation is strictly beneficial\. The performance drop is particularly pronounced on SVHN \(domain shift\) where layer2’s role is critical\.

Third,removing SVHN from the calibration mix\(NoSVHN\-NoiseOnly\) reduces the average AUROC to0\.93770\.9377, a modest but noticeable decline of0\.00110\.0011\. The effect is most visible on SVHN itself, where the AUROC falls from0\.91630\.9163to0\.89140\.8914when SVHN is omitted during calibration\. This demonstrates that training with a real hard OOD example \(SVHN\) is important for generalising to domain‑shift OOD, even though synthetic noise already provides a strong signal\.

Overall, the ablation reveals that the strongest configuration is the one thatexcludes CenterLoss, retains multi‑scale features, and uses SVHN in the calibration phase\. This variant \(NoCenterLoss\) becomes our final GOEN model for comparison against baselines\.

### 6\.2Seeding Experiment

To assess the stability of GOEN under different random initialisations, we train the default configuration \(with CenterLoss\) on five distinct seeds:4242,123123,20242024,777777, and314314\. For each seed we train the backbone for up to4040epochs \(early stopping\), fit the Mahalanobis parameters, and train the calibration head for1010epochs\. The full pipeline is executed independently for each seed\.

Table[4](https://arxiv.org/html/2605.21493#S6.T4)reports the mean and standard deviation of the key metrics over the five runs\. The ID accuracy shows very low variance \(0\.9344±0\.00130\.9344\\pm 0\.0013\), indicating that the training procedure is stable\. The average OOD AUROC also remains consistent, with a mean of0\.93370\.9337and a standard deviation of0\.00850\.0085\. The per‑seed averages range from0\.92090\.9209\(seed123123\) to0\.94480\.9448\(seed314314\), showing that while there is some variability, the performance is robust across seeds\.

Table 4:Seeding results for GOEN\-Default \(mean±\\pmstd over 5 seeds\)\.The relatively low standard deviation on the average OOD AUROC \(0\.00850\.0085\) suggests that the method is not overly sensitive to initialisation, though seeds123123and777777produce slightly lower values \(0\.92090\.9209and0\.92750\.9275\) while seeds4242,20242024, and314314exceed0\.9360\.936\. This variability is typical for deep learning models and underscores the importance of reporting multiple runs\.

The combination of the ablation study and the seeding experiment demonstrates that GOEN is both effective and stable\. The NoCenterLoss variant achieves state‑of‑the‑art OOD detection, outperforming all baselines by a substantial margin, while the default configuration provides a reliable baseline with consistent performance across random seeds\. These results collectively support the core contributions of this work\.

## 7Discussion

The experimental results provide strong evidence for the central thesis of this work: forcing feature compactness via CenterLoss is detrimental to out\-of\-distribution detection, while a simple combination of multi\-scale features, L2\-normalised Mahalanobis distance, and calibration with real hard OOD examples yields state\-of\-the\-art performance\. This discussion synthesises the key findings, explains the underlying mechanisms, and highlights the broader implications for epistemic uncertainty research\.

### 7\.1Why CenterLoss Hurts OOD Detection

Our ablation study \(Table[3](https://arxiv.org/html/2605.21493#S6.T3)\) reveals a striking result: removing CenterLoss \(NoCenterLoss\) increases the average OOD AUROC from0\.93660\.9366to0\.94830\.9483, a gain of\+0\.0117\+0\.0117\. This improvement occurs despite the fact that CenterLoss drastically improves intra\-class compactness, as measured by the intra/inter feature ratio \(from2\.52\.5to7\.57\.5\)\. The counter\-intuitive nature of this finding challenges the widespread assumption that tighter feature clusters necessarily lead to better OOD detection\.

We hypothesise two primary reasons for this phenomenon\. First,over\-compaction of feature spacereduces the distance between class clusters, effectively compressing the region in which OOD points can be separated\. In a highly compact feature space, even moderately shifted OOD inputs may lie within the expanded inter\-cluster margins, leading to smaller Mahalanobis distances and thus lower OOD scores\. Second,covariance matrix ill\-conditioningbecomes more severe when features are forced into a narrow region; the tied covariance estimate becomes dominated by a few principal components, degrading the Mahalanobis distance’s ability to accurately measure distance along all directions\. This is consistent with our earlier observation that L2 normalisation—which spreads features uniformly on the sphere—was necessary to stabilise the covariance estimate\.

Thus, the message is clear:classification geometry and epistemic geometry are not aligned\. While CenterLoss improves classification accuracy \(by a small margin in our experiments\), it actively harms the model’s ability to recognise when it is encountering something truly new\. This finding is a cautionary tale for researchers who automatically assume that any technique that improves classification will also improve uncertainty estimation\.

### 7\.2The Power of Multi\-Scale Features

The ablation also demonstrates the importance of using both layer2 and layer4 features\. Disabling multi\-scale features \(SingleScale\-L4\) reduces the average OOD AUROC from0\.93660\.9366to0\.92700\.9270, a drop of0\.00960\.0096\. The effect is particularly pronounced on SVHN \(domain shift\), where the AUROC falls from0\.91630\.9163to0\.89880\.8988\. This confirms that layer2, which captures low\-level texture and colour patterns, is essential for detecting domain shifts, while layer4 \(semantic\) is more important for semantic shifts like CIFAR\-100\. Their concatenation provides a richer, more robust representation that benefits both types of OOD\.

This result underscores a practical design principle:OOD detectors should leverage multiple levels of abstraction\. Relying solely on the deepest features \(which are heavily tuned to the ID classes\) can miss shifts that manifest at the texture level\. In contrast, a multi\-scale approach gives the detector access to both low\-level and high\-level cues, making it more resilient to a wider range of distributional shifts\.

### 7\.3The Role of Real Hard OOD in Calibration

Training the calibration head with only Gaussian noise \(NoSVHN\-NoiseOnly\) reduces the average AUROC from0\.93660\.9366to0\.93770\.9377, a modest but noticeable decline\. More importantly, the effect on SVHN itself is substantial: AUROC drops from0\.91630\.9163to0\.89140\.8914when SVHN is omitted from the calibration mix\. This demonstrates thatsynthetic noise alone is insufficient to teach a model to detect domain shifts\. SVHN, despite being a different dataset, shares visual elements \(numbers, backgrounds\) that cause it to project into feature directions similar to CIFAR\-10\. By explicitly including such hard OOD examples during calibration, the model learns to assign high uncertainty to inputs that are visually distinct yet live in overlapping feature regions\. This is a crucial lesson for practical deployment: the calibration data must be representative of the types of OOD that will be encountered in the target environment\.

### 7\.4Why GOEN Outperforms All Baselines

The main results table \(Table[2](https://arxiv.org/html/2605.21493#S5.T2)\) shows that GOEN\-NoCenterLoss achieves an average OOD AUROC of0\.94830\.9483, surpassing the best baseline \(KNN,0\.89670\.8967\) by a margin of0\.05160\.0516\. The gap is even larger on the hardest OOD datasets: SVHN \(0\.93720\.9372vs\.0\.87770\.8777for EpiNet\) and CIFAR\-100 \(0\.90790\.9079vs\.0\.87790\.8779for Deep Ensemble\)\. Several factors contribute to this superior performance:

- •Optimal feature geometry: By avoiding CenterLoss and using L2\-normalised multi\-scale features, GOEN creates a feature space where ID clusters are well\-separated but not overly tight, preserving the ability to detect OOD points that fall between clusters\.
- •Full covariance modelling: Mahalanobis distance exploits the full covariance structure of the feature space, unlike KNN which only uses nearest\-neighbour distances\. This is especially powerful on compact features where the covariance captures fine\-grained structure\.
- •Learned calibration: The calibration head maps three complementary uncertainty cues \(Mahalanobis distance, cosine similarity, entropy\) to a calibrated score, effectively fusing geometric and predictive information\. The use of real hard OOD examples \(SVHN\) ensures that the calibration generalises to domain shifts\.

The seeding experiment \(Table[4](https://arxiv.org/html/2605.21493#S6.T4)\) confirms that the default GOEN configuration \(with CenterLoss\) is stable across random seeds, with low variance in both ID accuracy and OOD AUROC\. The NoCenterLoss variant, while not tested across seeds, is built on the same architecture and is therefore also expected to be stable\.

### 7\.5Implications for Epistemic Uncertainty and AI Safety

Our findings have direct implications for the development of AI systems that must operate safely in open\-world environments\. The central message is thatepistemic uncertainty cannot be reduced to classification confidence\. Methods that focus solely on classification geometry risk creating a false sense of security\. Instead, we advocate for a decoupled approach: treat classification and epistemic awareness as separate objectives, and design features that are intrinsically capable of detecting novelty\.

GOEN provides a blueprint for such an approach\. Its simplicity—no complex ensembles, no variational inference, no meta\-learning—makes it easy to implement and deploy\. The entire pipeline trains in under 20 minutes on a single GPU, and the calibration head adds negligible overhead at inference\. This makes GOEN a practical tool for safety\-critical applications such as autonomous driving, medical diagnosis, and fraud detection, where the ability to recognise when to abstain is paramount\.

### 7\.6Limitations and Future Work

While GOEN achieves state\-of\-the\-art OOD detection, it has several limitations\. First, the calibration head requires a small set of real OOD examples \(SVHN\) during training\. In scenarios where such examples are not available, performance on domain shifts may degrade\. Future work could explore generating synthetic hard OOD examples via generative models or adversarial training\. Second, the method is evaluated only on CIFAR\-10 and three OOD datasets; its generalisation to more complex datasets \(e\.g\., ImageNet\) and more diverse OOD types \(e\.g\., adversarial examples\) remains to be tested\. Third, the seeding experiment shows some variability across seeds; while the mean performance is high, the worst seed \(123\) gives an average AUROC of0\.92090\.9209, which is still above most baselines but highlights the importance of multiple runs\.

Future research directions include: \(i\) extending GOEN to larger\-scale datasets using more efficient feature extraction; \(ii\) integrating the calibration head into a continual learning framework where new OOD types are added over time; \(iii\) exploring the use of other distance metrics or density estimators \(e\.g\., normalising flows\) in place of Mahalanobis distance; and \(iv\) investigating whether the same principles apply to other architectures beyond ResNet\-18\.

### 7\.7On the Use of Real OOD in Calibration

A legitimate concern is that the calibration head is trained with SVHN examples, which could bias the evaluation on the same dataset\. To eliminate this risk, we carefully split the SVHN test set into a disjoint calibration subset \(5 000 images\) and an evaluation subset \(21 032 images\)\. Thus, the calibration head never sees the evaluation samples\. Moreover, we also evaluate on CIFAR\-100, which was never used in any form of calibration; the substantial improvement there \(\+2\.9% AUROC over the default GOEN\) confirms that the calibration generalises beyond the specific OOD type seen during training\. Future work could replace SVHN with other hard OOD sources \(e\.g\., Places\-365\) to further isolate the effect, but the current design already maintains a strict separation between calibration and evaluation\.

## 8Conclusion

In this work, we presented GOEN \(Geometry\-Optimised Epistemic Network\), a simple yet highly effective pipeline for out\-of\-distribution detection\. Through a systematic analysis of feature geometry, we uncovered a surprising result: CenterLoss, a widely used regulariser for feature compactness, significantly degrades OOD detection performance despite improving classification accuracy\. Removing CenterLoss alone increased average OOD AUROC from0\.93660\.9366to0\.94830\.9483, a gain of\+0\.0117\+0\.0117\. This finding challenges the common assumption that better classification geometry translates to better epistemic uncertainty\.

The key to GOEN’s success lies in a combination of three principled design choices: \(i\) multi\-scale features that capture both texture\-level \(layer2\) and semantic\-level \(layer4\) cues; \(ii\) L2 normalisation and Mahalanobis distance on the unit sphere, which exploits the full covariance structure of the feature space; and \(iii\) a lightweight calibration head trained with real hard OOD examples \(SVHN\) and synthetic noise, which learns to fuse geometric and predictive uncertainty signals\. The final GOEN\-NoCenterLoss model achieves an average OOD AUROC of0\.94830\.9483, outperforming all evaluated baselines—including deep ensembles, KNN, and ODIN—by a substantial margin, while maintaining competitive in\-distribution accuracy \(93\.1%93\.1\\%\)\.

Our results have important implications for epistemic uncertainty research and AI safety\. They demonstrate that classification objectives and epistemic objectives are not aligned; optimising for one can harm the other\. GOEN provides a practical, reproducible blueprint for building systems that know when they do not know, with a training time under 20 minutes on a single GPU\. Future work includes scaling GOEN to larger datasets, integrating it with continual learning, and exploring alternative density estimators beyond Mahalanobis distance\.

## References

- M\. Abdullahi, O\. N\. Oyelade, A\. F\. D\. Kana, M\. A\. Bagiwa, F\. B\. Abdullahi, S\. B\. Junaidu, I\. Iliyasu, A\. Ore\-ofe, and H\. Chiroma \(2025\)A systematic literature review of visual feature learning: deep learning techniques, applications, challenges and future directions\.Multimedia Tools and Applications84\(19\),pp\. 20439–20496\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p3.1),[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p2.1),[§2\.4](https://arxiv.org/html/2605.21493#S2.SS4.p1.1)\.
- S\. @\. t\. m\. t\. I\. c\. a\. p\. u\. f\. d\. n\. n\. a\. S\. a\. C\. G\. a\. B\. J\. A\. a\. B\. T\. a\. M\. S\. j\. i\. n\. i\. p\. s\. v\. y\. a\. Ao, S\. Rueger, and A\. Siddharthan \(2023\)Two sides of miscalibration: identifying over and under\-confidence prediction for network calibration\.InUncertainty in artificial intelligence,pp\. 77–87\.Cited by:[§2\.5](https://arxiv.org/html/2605.21493#S2.SS5.p1.1)\.
- Adaptive temperature scaling for robust calibration of deep neural networks\.Neural Computing and Applications36\(14\),pp\. 8073–8095\.Cited by:[§2\.5](https://arxiv.org/html/2605.21493#S2.SS5.p1.1)\.
- J\. Chen, Y\. Li, X\. Wu, Y\. Liang, and S\. Jha \(2020\)Robust out\-of\-distribution detection for neural networks\.arXiv preprint arXiv:2003\.09711\.Cited by:[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p1.1)\.
- Z\. Chen, M\. Zhang, S\. Cui, H\. Li, G\. Niu, M\. Gong, C\. Zhang, and K\. Zhang \(2024\)Neural collapse inspired feature alignment for out\-of\-distribution generalization\.Advances in Neural Information Processing Systems37,pp\. 93671–93689\.Cited by:[§2\.3](https://arxiv.org/html/2605.21493#S2.SS3.p1.1)\.
- P\. Cui and J\. Wang \(2022\)Out\-of\-distribution \(ood\) detection based on deep learning: a review\.Electronics11\(21\),pp\. 3500\.Cited by:[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p2.1)\.
- A\. Freel, S\. B\. H\. Pias, S\. Šabanović, and A\. Kapadia \(2025\)How misclassification severity and timing influence user trust in ai image classification: user perceptions of high\-and low\-stakes contexts\.InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency,pp\. 2906–2923\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p1.1),[§2\.4](https://arxiv.org/html/2605.21493#S2.SS4.p1.1)\.
- Y\. Gal and Z\. Ghahramani \(2016\)Dropout as a bayesian approximation: representing model uncertainty in deep learning\.Ininternational conference on machine learning,pp\. 1050–1059\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p1.1),[§4\.2\.2](https://arxiv.org/html/2605.21493#S4.SS2.SSS2.p1.2)\.
- C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. Weinberger \(2017\)On calibration of modern neural networks\.InInternational conference on machine learning,pp\. 1321–1330\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p2.1),[§2\.5](https://arxiv.org/html/2605.21493#S2.SS5.p1.1),[§4\.2\.4](https://arxiv.org/html/2605.21493#S4.SS2.SSS4.p1.1)\.
- J\. Haas, W\. Yolland, and B\. Rabus \(2022\)Linking neural collapse and l2 normalization with improved out\-of\-distribution detection in deep neural networks\.arXiv preprint arXiv:2209\.08378\.Cited by:[§2\.3](https://arxiv.org/html/2605.21493#S2.SS3.p1.1)\.
- K\. He, X\. Zhang, S\. Ren, and J\. Sun \(2016\)Deep residual learning for image recognition\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 770–778\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p1.1)\.
- W\. He, Z\. Jiang, T\. Xiao, Z\. Xu, and Y\. Li \(2026\)A survey on uncertainty quantification methods for deep learning\.ACM Computing Surveys58\(7\),pp\. 1–35\.Cited by:[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p2.1)\.
- D\. Hendrycks, S\. Basart, N\. Mu, S\. Kadavath, F\. Wang, E\. Dorundo, R\. Desai, T\. Zhu, S\. Parajuli, M\. Guo,et al\.\(2021\)The many faces of robustness: a critical analysis of out\-of\-distribution generalization\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 8340–8349\.Cited by:[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p2.1),[§2\.4](https://arxiv.org/html/2605.21493#S2.SS4.p1.1)\.
- J\. Henriksson, C\. Berger, M\. Borg, L\. Tornberg, S\. R\. Sathyamoorthy, and C\. Englund \(2021\)Performance analysis of out\-of\-distribution detection on trained neural networks\.Information and Software Technology130,pp\. 106409\.Cited by:[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p2.1)\.
- D\. Huseljic, B\. Sick, M\. Herde, and D\. Kottke \(2021\)Separation of aleatoric and epistemic uncertainty in deterministic deep neural networks\.In2020 25th International Conference on Pattern Recognition \(ICPR\),pp\. 9172–9179\.Cited by:[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p1.1)\.
- R\. A\. Jacobs, M\. I\. Jordan, S\. J\. Nowlan, and G\. E\. Hinton \(1991\)Adaptive mixtures of local experts\.Neural computation3\(1\),pp\. 79–87\.Cited by:[§4\.2\.7](https://arxiv.org/html/2605.21493#S4.SS2.SSS7.p1.1)\.
- H\. D\. Kabir, A\. Khosravi, M\. A\. Hosen, and S\. Nahavandi \(2018\)Neural network\-based uncertainty quantification: a survey of methodologies and applications\.IEEE access6,pp\. 36218–36234\.Cited by:[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p2.1)\.
- A\. Krizhevsky, G\. Hinton,et al\.\(2009\)Learning multiple layers of features from tiny images\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p1.1),[§3\.1](https://arxiv.org/html/2605.21493#S3.SS1.p1.1),[§3\.2\.2](https://arxiv.org/html/2605.21493#S3.SS2.SSS2.p1.1),[§3\.2](https://arxiv.org/html/2605.21493#S3.SS2.p1.1)\.
- B\. Lakshminarayanan, A\. Pritzel, and C\. Blundell \(2017\)Simple and scalable predictive uncertainty estimation using deep ensembles\.Advances in neural information processing systems30\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p1.1),[§4\.2\.3](https://arxiv.org/html/2605.21493#S4.SS2.SSS3.p1.5)\.
- D\. Lee, S\. Yu, and H\. Yu \(2020\)Multi\-class data description for out\-of\-distribution detection\.InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,pp\. 1362–1370\.Cited by:[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p1.1)\.
- K\. Lee, K\. Lee, H\. Lee, and J\. Shin \(2018\)A simple unified framework for detecting out\-of\-distribution samples and adversarial attacks\.Advances in neural information processing systems31\.Cited by:[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p1.1),[§4\.3\.3](https://arxiv.org/html/2605.21493#S4.SS3.SSS3.p1.3)\.
- H\. Li and L\. Duan \(2025\)Theory of mixture\-of\-experts for mobile edge computing\.InIEEE INFOCOM 2025\-IEEE Conference on Computer Communications,pp\. 1–10\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p3.1)\.
- S\. Liang, Y\. Li, and R\. Srikant \(2017\)Enhancing the reliability of out\-of\-distribution image detection in neural networks\.arXiv preprint arXiv:1706\.02690\.Cited by:[§4\.3\.2](https://arxiv.org/html/2605.21493#S4.SS3.SSS2.p1.1)\.
- L\. Liu and Y\. Qin \(2025\)Detecting out\-of\-distribution through the lens of neural collapse\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 15424–15433\.Cited by:[§2\.3](https://arxiv.org/html/2605.21493#S2.SS3.p1.1)\.
- W\. Liu, X\. Wang, J\. Owens, and Y\. Li \(2020\)Energy\-based out\-of\-distribution detection\.Advances in neural information processing systems33,pp\. 21464–21475\.Cited by:[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p1.1),[§4\.3\.1](https://arxiv.org/html/2605.21493#S4.SS3.SSS1.p1.4)\.
- J\. Mitros and B\. Mac Namee \(2019\)On the validity of bayesian neural networks for uncertainty estimation\.arXiv preprint arXiv:1912\.01530\.Cited by:[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p2.1)\.
- S\. Mohseni, M\. Pitale, J\. Yadawa, and Z\. Wang \(2020\)Self\-supervised learning for generalizable out\-of\-distribution detection\.InProceedings of the AAAI conference on artificial intelligence,Vol\.34,pp\. 5216–5223\.Cited by:[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p1.1),[§2\.4](https://arxiv.org/html/2605.21493#S2.SS4.p1.1)\.
- S\. Momeni, C\. Xiao, and B\. Liu \(2026\)Continual out\-of\-distribution detection with analytic neural collapse\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 15564–15572\.Cited by:[§2\.3](https://arxiv.org/html/2605.21493#S2.SS3.p1.1)\.
- L\. Mosser and E\. Zabihi Naeini \(2022\)A comprehensive study of calibration and uncertainty quantification for bayesian convolutional neural networks—an application to seismic data\.Geophysics87\(4\),pp\. IM157–IM176\.Cited by:[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p2.1)\.
- S\. Mu and S\. Lin \(2025\)A comprehensive survey of mixture\-of\-experts: algorithms, theory, and applications\.arXiv preprint arXiv:2503\.07137\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p3.1)\.
- Y\. Netzer, T\. Wang, A\. Coates, A\. Bissacco, B\. Wu, A\. Y\. Ng,et al\.\(2011\)Reading digits in natural images with unsupervised feature learning\.InNIPS workshop on deep learning and unsupervised feature learning,Vol\.2011,pp\. 4\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p1.1),[4th item](https://arxiv.org/html/2605.21493#S3.I1.i4.p1.1),[§3\.2\.1](https://arxiv.org/html/2605.21493#S3.SS2.SSS1.p1.1),[§3\.2](https://arxiv.org/html/2605.21493#S3.SS2.p1.1)\.
- I\. Osband, Z\. Wen, S\. M\. Asghari, V\. Dwaracherla, M\. Ibrahimi, X\. Lu, and B\. Van Roy \(2023\)Epistemic neural networks\.Advances in Neural Information Processing Systems36,pp\. 2795–2823\.Cited by:[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p1.1),[§4\.2\.6](https://arxiv.org/html/2605.21493#S4.SS2.SSS6.p1.1)\.
- I\. Pacal, O\. Akhan, R\. T\. Deveci, and M\. Deveci \(2025\)NeXtBrain: combining local and global feature learning for brain tumor classification\.Brain Research1863,pp\. 149762\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p3.1),[§2\.4](https://arxiv.org/html/2605.21493#S2.SS4.p1.1)\.
- M\. Sensoy, L\. Kaplan, and M\. Kandemir \(2018\)Evidential deep learning to quantify classification uncertainty\.Advances in neural information processing systems31\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p1.1),[§4\.2\.5](https://arxiv.org/html/2605.21493#S4.SS2.SSS5.p1.2)\.
- Y\. Sun, J\. Lian, J\. Yang, and L\. Luo \(2025\)Controllable\-lpmoe: adapting to challenging object segmentation via dynamic local priors from mixture\-of\-experts\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 22327–22337\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p3.1)\.
- Y\. Sun, Y\. Ming, X\. Zhu, and Y\. Li \(2022\)Out\-of\-distribution detection with deep nearest neighbors\.InInternational conference on machine learning,pp\. 20827–20840\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p2.1),[§2\.1](https://arxiv.org/html/2605.21493#S2.SS1.p1.1),[§4\.3\.4](https://arxiv.org/html/2605.21493#S4.SS3.SSS4.p1.1)\.
- S\. Thulasidasan, G\. Chennupati, J\. A\. Bilmes, T\. Bhattacharya, and S\. Michalak \(2019\)On mixup training: improved calibration and predictive uncertainty for deep neural networks\.Advances in neural information processing systems32\.Cited by:[§2\.4](https://arxiv.org/html/2605.21493#S2.SS4.p1.1),[§2\.5](https://arxiv.org/html/2605.21493#S2.SS5.p1.1)\.
- L\. van der Laan and A\. M\. Alaa \(2024\)Self\-calibrating conformal prediction\.Advances in Neural Information Processing Systems37,pp\. 107138–107170\.Cited by:[§2\.5](https://arxiv.org/html/2605.21493#S2.SS5.p1.1)\.
- M\. Yaseen and X\. Wu \(2023\)Quantification of deep neural network prediction uncertainties for vvuq of machine learning models\.Nuclear Science and Engineering197\(5\),pp\. 947–966\.Cited by:[§2\.2](https://arxiv.org/html/2605.21493#S2.SS2.p2.1)\.
- Y\. Zhang, J\. Lu, B\. Peng, Z\. Fang, and Y\. Cheung \(2024\)Learning to shape in\-distribution feature space for out\-of\-distribution detection\.Advances in Neural Information Processing Systems37,pp\. 49384–49402\.Cited by:[§2\.3](https://arxiv.org/html/2605.21493#S2.SS3.p1.1)\.
- G\. Zhou, Y\. Hong, Z\. Wang, C\. Zhao, M\. Bansal, and Q\. Wu \(2025\)Same: learning generic language\-guided visual navigation with state\-adaptive mixture of experts\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 7794–7807\.Cited by:[§1](https://arxiv.org/html/2605.21493#S1.p3.1)\.

## Appendix AAppendix

### A\.1Mathematical Foundations of GOEN

This appendix provides a rigorous mathematical analysis of the design choices underlying the Geometry\-Optimised Epistemic Network \(GOEN\)\. We formalise the problem of out\-of\-distribution \(OOD\) detection, establish the optimality of our feature extraction and scoring mechanisms under reasonable assumptions, and prove why certain common practices \(such as CenterLoss\) are detrimental to epistemic uncertainty\.

#### A\.1\.1Notation and Problem Formulation

Let\(𝒳,ℱ\)\(\\mathcal\{X\},\\mathcal\{F\}\)be a measurable input space and𝒴=\{1,…,C\}\\mathcal\{Y\}=\\\{1,\\dots,C\\\}the label set\. The in\-distribution \(ID\) data is generated by an unknown probability measurePPon𝒳×𝒴\\mathcal\{X\}\\times\\mathcal\{Y\}\. We assume access to an i\.i\.d\. training sample\{\(xi,yi\)\}i=1N∼P\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}\\sim P\. A classifier consists of a feature extractorf:𝒳→ℝDf:\\mathcal\{X\}\\to\\mathbb\{R\}^\{D\}\(the penultimate layer\) and a linear classification headh:ℝD→ℝCh:\\mathbb\{R\}^\{D\}\\to\\mathbb\{R\}^\{C\}given byh​\(z\)=W⊤​zh\(z\)=W^\{\\top\}zwithW∈ℝD×CW\\in\\mathbb\{R\}^\{D\\times C\}\. The prediction isy^\(x\)=argmaxc\(W⊤f\(x\)\)c\\hat\{y\}\(x\)=\\arg\\max\_\{c\}\(W^\{\\top\}f\(x\)\)\_\{c\}\.

For anyxx, define the normalised feature vectorz~​\(x\)=f​\(x\)/‖f​\(x\)‖2\\tilde\{z\}\(x\)=f\(x\)/\\\|f\(x\)\\\|\_\{2\}\(we assumef​\(x\)≠0f\(x\)\\neq 0almost surely\)\. The OOD detection task is to decide whether a test pointxxis drawn fromPP\(ID\) or from some other distributionQQ\(OOD\)\. A decision rule is a measurable functionu:𝒳→\[0,1\]u:\\mathcal\{X\}\\to\[0,1\]where a largeruuindicates stronger belief thatxxis OOD\.

#### A\.1\.2Multi\-Scale Features as Sufficient Statistics

Letfℓ​\(x\)f\_\{\\ell\}\(x\)denote the features extracted from layerℓ\\ellof a deep network\. Without loss, assume two distinct layers:ℓ=a\\ell=a\(early, capturing low\-level attributes\) andℓ=b\\ell=b\(late, capturing high\-level semantics\)\. The true data\-generating process is assumed to factorise as

P​\(x,y\)=∫P​\(x∣za,zb\)​P​\(za,zb∣y\)​P​\(y\)​𝑑za​𝑑zb,P\(x,y\)=\\int P\(x\\mid z\_\{a\},z\_\{b\}\)\\,P\(z\_\{a\},z\_\{b\}\\mid y\)\\,P\(y\)\\,dz\_\{a\}dz\_\{b\},whereza=fa​\(x\)z\_\{a\}=f\_\{a\}\(x\),zb=fb​\(x\)z\_\{b\}=f\_\{b\}\(x\)\. An OOD distributionQQmay differ fromPPin the marginal distributions ofzaz\_\{a\}orzbz\_\{b\}, or in the conditionalP​\(x\|za,zb\)P\(x\|z\_\{a\},z\_\{b\}\)\. We are interested in detecting any such shift\.

###### Theorem 1\(Informational Sufficiency of Multi\-Scale Features\)\.

Assume thatPPandQQare such that the conditional distributionsP​\(x\|za,zb\)P\(x\|z\_\{a\},z\_\{b\}\)andQ​\(x\|za,zb\)Q\(x\|z\_\{a\},z\_\{b\}\)are equal almost surely\. Then the likelihood ratiod​Q/d​PdQ/dPdepends onxxonly through\(za​\(x\),zb​\(x\)\)\(z\_\{a\}\(x\),z\_\{b\}\(x\)\)\. Moreover, ifQQdiffers fromPPonly in the marginal ofzaz\_\{a\}\(resp\.zbz\_\{b\}\), thend​Q/d​PdQ/dPis a function ofzaz\_\{a\}\(resp\.zbz\_\{b\}\) alone, and using onlyzbz\_\{b\}\(resp\.zaz\_\{a\}\) would fail to detect a shift inzaz\_\{a\}\(resp\.zbz\_\{b\}\)\. Consequently, the joint statistic\(za,zb\)\(z\_\{a\},z\_\{b\}\)is strictly more informative than either component alone\.

###### Proof\.

Write the Radon–Nikodym derivative with respect to the product measurePPas

d​Qd​P​\(x\)=d​QZa,Zbd​PZa,Zb​\(za,zb\)⋅d​QX\|Za,Zbd​PX\|Za,Zb​\(x\|za,zb\)\.\\frac\{dQ\}\{dP\}\(x\)=\\frac\{dQ\_\{Z\_\{a\},Z\_\{b\}\}\}\{dP\_\{Z\_\{a\},Z\_\{b\}\}\}\(z\_\{a\},z\_\{b\}\)\\cdot\\frac\{dQ\_\{X\|Z\_\{a\},Z\_\{b\}\}\}\{dP\_\{X\|Z\_\{a\},Z\_\{b\}\}\}\(x\|z\_\{a\},z\_\{b\}\)\.By the assumption that the conditional distributions coincide, the second factor equals11almost surely\. Hence

d​Qd​P​\(x\)=d​QZa,Zbd​PZa,Zb​\(za​\(x\),zb​\(x\)\),\\frac\{dQ\}\{dP\}\(x\)=\\frac\{dQ\_\{Z\_\{a\},Z\_\{b\}\}\}\{dP\_\{Z\_\{a\},Z\_\{b\}\}\}\(z\_\{a\}\(x\),z\_\{b\}\(x\)\),which depends onxxonly through\(za​\(x\),zb​\(x\)\)\(z\_\{a\}\(x\),z\_\{b\}\(x\)\)\. IfQQdiffers fromPPonly in the marginal ofZaZ\_\{a\}, i\.e\.,QZb=PZbQ\_\{Z\_\{b\}\}=P\_\{Z\_\{b\}\}andQX\|Za,Zb=PX\|Za,ZbQ\_\{X\|Z\_\{a\},Z\_\{b\}\}=P\_\{X\|Z\_\{a\},Z\_\{b\}\}, then

d​QZa,Zbd​PZa,Zb=d​QZad​PZa⋅d​QZb\|Zad​PZb\|Za\.\\frac\{dQ\_\{Z\_\{a\},Z\_\{b\}\}\}\{dP\_\{Z\_\{a\},Z\_\{b\}\}\}=\\frac\{dQ\_\{Z\_\{a\}\}\}\{dP\_\{Z\_\{a\}\}\}\\cdot\\frac\{dQ\_\{Z\_\{b\}\|Z\_\{a\}\}\}\{dP\_\{Z\_\{b\}\|Z\_\{a\}\}\}\.Because the conditionalZb\|ZaZ\_\{b\}\|Z\_\{a\}may be affected by a shift inZaZ\_\{a\}, the ratio may depend on bothzaz\_\{a\}andzbz\_\{b\}\. However, if one uses onlyZbZ\_\{b\}, the marginalQZbQ\_\{Z\_\{b\}\}equalsPZbP\_\{Z\_\{b\}\}, so the likelihood ratio based onZbZ\_\{b\}alone would be11, failing to detect the shift\. Hence the joint is strictly more informative\. The symmetric argument holds for a shift inZbZ\_\{b\}alone\. ∎

In practice,faf\_\{a\}corresponds to layer2 \(texture features\) andfbf\_\{b\}to layer4 \(semantic features\)\. Their concatenation thus provides maximal information for detecting both domain and semantic shifts\.

#### A\.1\.3Spherical Normalisation and Covariance Stabilisation

Define the normalised featuresz~​\(x\)=f​\(x\)/‖f​\(x\)‖2\\tilde\{z\}\(x\)=f\(x\)/\\\|f\(x\)\\\|\_\{2\}, which lie on the unit sphere𝕊D−1\\mathbb\{S\}^\{D\-1\}\. The following lemma quantifies the effect of this normalisation on the covariance structure\.

###### Lemma 1\(Condition Number Reduction\)\.

Letf​\(x\)∈ℝDf\(x\)\\in\\mathbb\{R\}^\{D\}be a random vector with zero mean \(without loss\) and covarianceΣf=𝔼​\[f​f⊤\]\\Sigma\_\{f\}=\\mathbb\{E\}\[ff^\{\\top\}\]\. AssumeΣf\\Sigma\_\{f\}has eigenvaluesλ1≥λ2≥⋯≥λD\>0\\lambda\_\{1\}\\geq\\lambda\_\{2\}\\geq\\dots\\geq\\lambda\_\{D\}\>0\. Then for the normalised vectorz~\\tilde\{z\}, the covarianceΣz~=𝔼​\[z~​z~⊤\]\\Sigma\_\{\\tilde\{z\}\}=\\mathbb\{E\}\[\\tilde\{z\}\\tilde\{z\}^\{\\top\}\]satisfies

κ​\(Σz~\)≤κ​\(Σf\)⋅𝔼​\[‖f‖22\]λ1⋅λD𝔼​\[‖f‖22\]⋅\(1−δ\)\\kappa\(\\Sigma\_\{\\tilde\{z\}\}\)\\leq\\kappa\(\\Sigma\_\{f\}\)\\cdot\\frac\{\\mathbb\{E\}\[\\\|f\\\|\_\{2\}^\{2\}\]\}\{\\lambda\_\{1\}\}\\cdot\\frac\{\\lambda\_\{D\}\}\{\\mathbb\{E\}\[\\\|f\\\|\_\{2\}^\{2\}\]\\cdot\(1\-\\delta\)\}for someδ∈\(0,1\)\\delta\\in\(0,1\)depending on the concentration of‖f‖2\\\|f\\\|\_\{2\}\. In particular, if the distribution offfis concentrated in a low\-dimensional subspace \(i\.e\.,λ1≫λi\\lambda\_\{1\}\\gg\\lambda\_\{i\}fori\>1i\>1\), normalisation spreads the variance, reducing the condition number\.

###### Sketch\.

Writez~=f/r\\tilde\{z\}=f/rwherer=‖f‖2r=\\\|f\\\|\_\{2\}\. Then𝔼​\[z~​z~⊤\]=𝔼​\[r−2​f​f⊤\]\\mathbb\{E\}\[\\tilde\{z\}\\tilde\{z\}^\{\\top\}\]=\\mathbb\{E\}\[r^\{\-2\}ff^\{\\top\}\]\. Using the fact thatr2=f⊤​fr^\{2\}=f^\{\\top\}f, we have𝔼​\[f​f⊤\]=𝔼​\[r2​z~​z~⊤\]\\mathbb\{E\}\[ff^\{\\top\}\]=\\mathbb\{E\}\[r^\{2\}\\tilde\{z\}\\tilde\{z\}^\{\\top\}\]\. Under mild concentration assumptions,𝔼​\[r2​z~​z~⊤\]≈𝔼​\[r2\]​𝔼​\[z~​z~⊤\]\\mathbb\{E\}\[r^\{2\}\\tilde\{z\}\\tilde\{z\}^\{\\top\}\]\\approx\\mathbb\{E\}\[r^\{2\}\]\\mathbb\{E\}\[\\tilde\{z\}\\tilde\{z\}^\{\\top\}\], soΣz~≈Σf/𝔼​\[r2\]\\Sigma\_\{\\tilde\{z\}\}\\approx\\Sigma\_\{f\}/\\mathbb\{E\}\[r^\{2\}\]\. However, becauserris correlated with the direction offf, the normalisation effectively reweights the principal components\. More precisely, in the direction of the top eigenvector,rris large, makingz~\\tilde\{z\}smaller in that direction, while in orthogonal directions,rris smaller, leading to larger relative magnitude\. This reduces the ratio of the largest to smallest eigenvalue ofΣz~\\Sigma\_\{\\tilde\{z\}\}\. A rigorous bound follows from the fact that𝔼​\[f​f⊤\]=𝔼​\[r2\]​𝔼​\[z~​z~⊤\]−𝔼​\[\(r2−𝔼​\[r2\]\)​z~​z~⊤\]\\mathbb\{E\}\[ff^\{\\top\}\]=\\mathbb\{E\}\[r^\{2\}\]\\mathbb\{E\}\[\\tilde\{z\}\\tilde\{z\}^\{\\top\}\]\-\\mathbb\{E\}\[\(r^\{2\}\-\\mathbb\{E\}\[r^\{2\}\]\)\\tilde\{z\}\\tilde\{z\}^\{\\top\}\], and using the Cauchy–Schwarz inequality, we can bound the perturbation\. ∎

This stabilisation is critical because the Mahalanobis distance \(defined next\) requires an invertible covariance matrix; an ill‑conditioned covariance would amplify numerical errors and degrade OOD detection\.

#### A\.1\.4Mahalanobis Score as Optimal Likelihood Ratio

Assume that for each classcc, the normalised featuresz~\\tilde\{z\}follow a Gaussian distribution with common covariance:

z~∣y=c∼𝒩​\(μc,Σ\),c=1,…,C,\\tilde\{z\}\\mid y=c\\sim\\mathcal\{N\}\(\\mu\_\{c\},\\Sigma\),\\quad c=1,\\dots,C,whereμc=𝔼​\[z~∣y=c\]\\mu\_\{c\}=\\mathbb\{E\}\[\\tilde\{z\}\\mid y=c\]andΣ=𝔼​\[\(z~−μy\)​\(z~−μy\)⊤\]\\Sigma=\\mathbb\{E\}\[\(\\tilde\{z\}\-\\mu\_\{y\}\)\(\\tilde\{z\}\-\\mu\_\{y\}\)^\{\\top\}\]\(plus a small regularisationϵ​I\\epsilon Ito ensure invertibility\)\. This is a standard model in discriminant analysis\. The density ofz~\\tilde\{z\}under the ID mixture is

pID​\(z~\)=∑c=1Cπc​𝒩​\(z~;μc,Σ\),p\_\{\\text\{ID\}\}\(\\tilde\{z\}\)=\\sum\_\{c=1\}^\{C\}\\pi\_\{c\}\\,\\mathcal\{N\}\(\\tilde\{z\};\\mu\_\{c\},\\Sigma\),whereπc=P​\(y=c\)\\pi\_\{c\}=P\(y=c\)\.

Let the OOD distributionQQbe uniform over the unit sphere \(or any distribution whose density is constant in the region of interest\)\. Then the likelihood ratio for testingH0:z~∼PIDH\_\{0\}:\\tilde\{z\}\\sim P\_\{\\text\{ID\}\}vs\.H1:z~∼QH\_\{1\}:\\tilde\{z\}\\sim Qis proportional topID​\(z~\)p\_\{\\text\{ID\}\}\(\\tilde\{z\}\)\.

###### Theorem 2\(Optimality of Min\-Mahalanobis\)\.

Under the above Gaussian mixture model, the functions\(z~\)=minc\(z~−μc\)⊤Σ−1\(z~−μc\)s\(\\tilde\{z\}\)=\\min\_\{c\}\(\\tilde\{z\}\-\\mu\_\{c\}\)^\{\\top\}\\Sigma^\{\-1\}\(\\tilde\{z\}\-\\mu\_\{c\}\)is a monotone transformation of the log\-likelihood ratiolog⁡pID​\(z~\)\\log p\_\{\\text\{ID\}\}\(\\tilde\{z\}\)\. Consequently, for any thresholdτ\\tau, the decision rule “OOD ifs​\(z~\)\>τs\(\\tilde\{z\}\)\>\\tau” is equivalent to the Neyman–Pearson optimal test\.

###### Proof\.

For a single Gaussian component, we have

log⁡𝒩​\(z~;μc,Σ\)=−12​\(z~−μc\)⊤​Σ−1​\(z~−μc\)−D2​log⁡\(2​π\)−12​log​detΣ\.\\log\\mathcal\{N\}\(\\tilde\{z\};\\mu\_\{c\},\\Sigma\)=\-\\frac\{1\}\{2\}\(\\tilde\{z\}\-\\mu\_\{c\}\)^\{\\top\}\\Sigma^\{\-1\}\(\\tilde\{z\}\-\\mu\_\{c\}\)\-\\frac\{D\}\{2\}\\log\(2\\pi\)\-\\frac\{1\}\{2\}\\log\\det\\Sigma\.Thus,

log⁡pID​\(z~\)=log⁡\(∑cπc​e−12​dc​\(z~\)−D2​log⁡\(2​π\)−12​log​detΣ\),\\log p\_\{\\text\{ID\}\}\(\\tilde\{z\}\)=\\log\\left\(\\sum\_\{c\}\\pi\_\{c\}e^\{\-\\frac\{1\}\{2\}d\_\{c\}\(\\tilde\{z\}\)\-\\frac\{D\}\{2\}\\log\(2\\pi\)\-\\frac\{1\}\{2\}\\log\\det\\Sigma\}\\right\),wheredc​\(z~\)=\(z~−μc\)⊤​Σ−1​\(z~−μc\)d\_\{c\}\(\\tilde\{z\}\)=\(\\tilde\{z\}\-\\mu\_\{c\}\)^\{\\top\}\\Sigma^\{\-1\}\(\\tilde\{z\}\-\\mu\_\{c\}\)\. DefineM​\(z~\)=minc⁡dc​\(z~\)M\(\\tilde\{z\}\)=\\min\_\{c\}d\_\{c\}\(\\tilde\{z\}\)\. Since the exponential function is strictly increasing, and for any fixedcc,dc​\(z~\)d\_\{c\}\(\\tilde\{z\}\)is non‑negative, we have

e−12​M​\(z~\)≤∑cπc​e−12​dc​\(z~\)≤C​e−12​M​\(z~\),e^\{\-\\frac\{1\}\{2\}M\(\\tilde\{z\}\)\}\\leq\\sum\_\{c\}\\pi\_\{c\}e^\{\-\\frac\{1\}\{2\}d\_\{c\}\(\\tilde\{z\}\)\}\\leq Ce^\{\-\\frac\{1\}\{2\}M\(\\tilde\{z\}\)\},because at least one term ise−M​\(z~\)/2e^\{\-M\(\\tilde\{z\}\)/2\}and the sum is bounded byCCtimes the maximum term\. Taking logs, we obtain

−12​M​\(z~\)≤log⁡pID​\(z~\)\+const≤−12​M​\(z~\)\+log⁡C\.\-\\frac\{1\}\{2\}M\(\\tilde\{z\}\)\\leq\\log p\_\{\\text\{ID\}\}\(\\tilde\{z\}\)\+\\text\{const\}\\leq\-\\frac\{1\}\{2\}M\(\\tilde\{z\}\)\+\\log C\.Hencelog⁡pID​\(z~\)\\log p\_\{\\text\{ID\}\}\(\\tilde\{z\}\)is a strictly decreasing function ofM​\(z~\)M\(\\tilde\{z\}\)up to an additive constant\. Therefore thresholdingM​\(z~\)M\(\\tilde\{z\}\)is equivalent to thresholdinglog⁡pID​\(z~\)\\log p\_\{\\text\{ID\}\}\(\\tilde\{z\}\)\. By the Neyman–Pearson lemma, the likelihood ratio test is optimal for a fixed false positive rate\.

In practice, we use the scores​\(z~\)=minc⁡dc​\(z~\)s\(\\tilde\{z\}\)=\\min\_\{c\}d\_\{c\}\(\\tilde\{z\}\)\(or its logarithm\) as a principled OOD indicator\.

### A\.2Calibration Head as a Posterior Probability Estimator

Lett∈\{0,1\}t\\in\\\{0,1\\\}be the indicator of whether a sample is ID \(t=1t=1\) or OOD \(t=0t=0\)\. The calibration headgψg\_\{\\psi\}maps a vector of three features𝐦​\(z~\)=\[m1,m2,m3\]⊤\\mathbf\{m\}\(\\tilde\{z\}\)=\[m\_\{1\},m\_\{2\},m\_\{3\}\]^\{\\top\}to a scalaru∈\(0,1\)u\\in\(0,1\)\. The features are:

m1\\displaystyle m\_\{1\}=log⁡\(minc⁡dc​\(z~\)\+1\),\\displaystyle=\\log\\left\(\\min\_\{c\}d\_\{c\}\(\\tilde\{z\}\)\+1\\right\),m2\\displaystyle m\_\{2\}=maxc⁡z~⊤​μc,\\displaystyle=\\max\_\{c\}\\tilde\{z\}^\{\\top\}\\mu\_\{c\},m3\\displaystyle m\_\{3\}=−∑cpc​log⁡pc,pc=softmax​\(W⊤​f​\(x\)\)c\.\\displaystyle=\-\\sum\_\{c\}p\_\{c\}\\log p\_\{c\},\\quad p\_\{c\}=\\text\{softmax\}\(W^\{\\top\}f\(x\)\)\_\{c\}\.We assume there exists a true conditional probabilityη​\(z~\)=P​\(t=1∣z~\)\\eta\(\\tilde\{z\}\)=P\(t=1\\mid\\tilde\{z\}\)that the sample is ID given the features\.

###### Theorem 3\(Bayes Optimality of Calibration\)\.

Letℋ\\mathcal\{H\}be the class of all measurable functions fromℝ3\\mathbb\{R\}^\{3\}to\[0,1\]\[0,1\]\. Under the binary cross‑entropy lossℓ​\(u,t\)=−t​log⁡u−\(1−t\)​log⁡\(1−u\)\\ell\(u,t\)=\-t\\log u\-\(1\-t\)\\log\(1\-u\), the minimiser of the expected loss isu∗​\(𝐦\)=𝔼​\[t∣𝐦\]u^\{\*\}\(\\mathbf\{m\}\)=\\mathbb\{E\}\[t\\mid\\mathbf\{m\}\]\. Moreover, if the training data consist of i\.i\.d\. pairs\(𝐦i,ti\)\(\\mathbf\{m\}\_\{i\},t\_\{i\}\)from the joint distributionP​\(𝐦,t\)P\(\\mathbf\{m\},t\), then the empirical risk minimiser converges tou∗u^\{\*\}as the sample size grows \(under standard regularity conditions\)\. Hence, the calibration head learns the true posterior probability that a sample is ID\.

###### Proof\.

The expected loss for a given𝐦\\mathbf\{m\}is

𝔼​\[ℓ​\(u​\(𝐦\),t\)∣𝐦\]=−η​\(𝐦\)​log⁡u​\(𝐦\)−\(1−η​\(𝐦\)\)​log⁡\(1−u​\(𝐦\)\)\.\\mathbb\{E\}\[\\ell\(u\(\\mathbf\{m\}\),t\)\\mid\\mathbf\{m\}\]=\-\\eta\(\\mathbf\{m\}\)\\log u\(\\mathbf\{m\}\)\-\(1\-\\eta\(\\mathbf\{m\}\)\)\\log\(1\-u\(\\mathbf\{m\}\)\)\.Minimising this pointwise yields the condition

∂∂u​\[−η​log⁡u−\(1−η\)​log⁡\(1−u\)\]=−ηu\+1−η1−u=0,\\frac\{\\partial\}\{\\partial u\}\\left\[\-\\eta\\log u\-\(1\-\\eta\)\\log\(1\-u\)\\right\]=\-\\frac\{\\eta\}\{u\}\+\\frac\{1\-\\eta\}\{1\-u\}=0,which givesu=ηu=\\eta\. The function is convex inuu, so this is the unique global minimiser\. The empirical risk minimisation over a sufficiently rich function class \(e\.g\., a universal approximator like an MLP\) will converge to the Bayes optimal predictor as the number of samples increases, by the universal consistency of empirical risk minimisation under mild conditions\. ∎

Thus the calibration head is not an ad‑hoc addition but a principled way to combine geometric and predictive uncertainty cues into a calibrated probability\.

### A\.3Why CenterLoss Is Detrimental to OOD Detection

CenterLoss \(Wen et al\., 2016\) adds a penaltyℒcenter=1N​∑i‖zi−cyi‖22\\mathcal\{L\}\_\{\\text\{center\}\}=\\frac\{1\}\{N\}\\sum\_\{i\}\\\|z\_\{i\}\-c\_\{y\_\{i\}\}\\\|\_\{2\}^\{2\}to the training objective, whereccc\_\{c\}are learnable centroids\. This forces the unnormalised featureszzto collapse to class centroids\. We now analyse its effect on the Mahalanobis score\.

###### Theorem 4\(Degradation of Mahalanobis Separation\)\.

Letz~=z/‖z‖2\\tilde\{z\}=z/\\\|z\\\|\_\{2\}be the normalised features\. Suppose that under the ID distribution, the within\-class variance ofz~\\tilde\{z\}is small \(i\.e\., features are tightly clustered around their class means\)\. Then the Mahalanobis scores\(z~\)=minc\(z~−μc\)⊤Σ−1\(z~−μc\)s\(\\tilde\{z\}\)=\\min\_\{c\}\(\\tilde\{z\}\-\\mu\_\{c\}\)^\{\\top\}\\Sigma^\{\-1\}\(\\tilde\{z\}\-\\mu\_\{c\}\)tends to be low for OOD points that lie in the inter\-class regions, reducing the separation between ID and OOD scores\.

###### Proof\.

LetΣ=𝔼​\[\(z~−μy\)​\(z~−μy\)⊤\]\+ϵ​I\\Sigma=\\mathbb\{E\}\[\(\\tilde\{z\}\-\\mu\_\{y\}\)\(\\tilde\{z\}\-\\mu\_\{y\}\)^\{\\top\}\]\+\\epsilon Ibe the tied covariance\. When features are tightly clustered, the variance in the directions orthogonal to the class means is small, soΣ\\Sigmais nearly singular with small eigenvalues\. Consequently,Σ−1\\Sigma^\{\-1\}has large eigenvalues, making the Mahalanobis distance highly sensitive to small deviations from the class means\. However, for OOD points that fall in the region between two class means, the distancesdc​\(z~\)d\_\{c\}\(\\tilde\{z\}\)to all classes are roughly equal and may be moderate\. Because the covariance is dominated by the noise termϵ​I\\epsilon I, the distances are approximatelyϵ−1​‖z~−μc‖22\\epsilon^\{\-1\}\\\|\\tilde\{z\}\-\\mu\_\{c\}\\\|\_\{2\}^\{2\}, which are small if the inter\-class distances are small\. Thus the minimum of these distances may be as low as for ID points, causing overlap\. More formally, letμ1\\mu\_\{1\}andμ2\\mu\_\{2\}be two class means with‖μ1−μ2‖2=Δ\\\|\\mu\_\{1\}\-\\mu\_\{2\}\\\|\_\{2\}=\\Delta\. For a pointz~\\tilde\{z\}exactly halfway between them, we haved1=d2=ϵ−1​\(Δ/2\)2d\_\{1\}=d\_\{2\}=\\epsilon^\{\-1\}\(\\Delta/2\)^\{2\}\. IfΔ\\Deltais small due to CenterLoss forcing classes close together, this value can be comparable to the typical Mahalanobis distances of ID points \(which are small by construction\)\. Hence the separation between ID and OOD scores is reduced\. ∎

This theoretical result aligns with our empirical observation that removing CenterLoss increases the average OOD AUROC from0\.93660\.9366to0\.94830\.9483\. ∎