Accuracy and Robustness of Model Cascades Under Data Perturbations

# Accuracy and Robustness of Model Cascades Under Data Perturbations
Source: [https://arxiv.org/html/2608.17711](https://arxiv.org/html/2608.17711)
###### Abstract

Prediction cascades significantly reduce energy consumption of Artificial Intelligence \(AI\) models while maintaining high predictive performance\. The idea is that easy inputs are routed through a lightweight small model, and difficult uncertain cases are deferred to a larger model\. While this design can improve computational efficiency on clean data, its effectiveness depends on the reliability of confidence\-based routing\. Input degradations, such as static corruptions and sequential perturbations, can shift model confidence and routing decisions\. In this paper, we study confidence\-based cascade frameworks for image classification and investigate how such degradations affect their confidence\-based deferral behavior\. We select a model cascade at the pareto\-optimum of accuracy, routing quality, and energy consumption that achieves competitive predictive performance with an up to 10\-fold decrease in CO2emissions\. We study the behavior of that model cascade under input corruptions and analyze how the cascade’s routing decisions change when the input distribution shifts\. Our analysis identifies three failure modes\. Static corruptions either \(1\) break the routing signal while the large model remains useful, or \(2\) degrade both models so deferral no longer recovers accuracy\. Sequential perturbations reveal a third mode: predictions stabilize but deferral suppresses, yielding stable but unreliable predictions\. These findings demonstrate that energy efficient model cascades require evaluation beyond clean accuracy, with explicit attention to routing reliability under distribution shift\.

fficient inference, confidence\-based deferral, corruption robustness, distribution shift, cascade model

## 1Introduction

The computational cost and energy consumption of the implementation of AI models pose significant environmental challenges\[[10](https://arxiv.org/html/2608.17711#bib.bib2),[9](https://arxiv.org/html/2608.17711#bib.bib3)\]\. As the scale of the model continues to grow, reducing inference costs has become critical to sustainable AI deployment\. Among various efficiency techniques—including pruning\[[2](https://arxiv.org/html/2608.17711#bib.bib4)\], knowledge distillation\[[4](https://arxiv.org/html/2608.17711#bib.bib5)\], quantization\[[5](https://arxiv.org/html/2608.17711#bib.bib6)\], and early\-exit architectures\[[11](https://arxiv.org/html/2608.17711#bib.bib7)\]—prediction cascades offer a particularly promising approach by adaptively routing inputs across models of varying complexity\. The core idea is intuitive: not all inputs require the same computational effort\. Easy samples can be handled by lightweight models, while only difficult cases need expensive large models\. The Gatekeeper framework\[[8](https://arxiv.org/html/2608.17711#bib.bib10)\]implements this through confidence\-based deferral, where a small model \(MSM\_\{S\}\) first attempts each prediction and defers uncertain cases to a large model \(MLM\_\{L\}\)\. When a small modelMSM\_\{S\}is confident, the cascade saves computation; when uncertain, it invests in the capacity of the large modelMLM\_\{L\}\. On clean data, this design achieves substantial efficiency gains\.

However, real\-world deployment rarely matches clean training assumptions\. In computer vision applications, for instance, images may suffer from compression artifacts, sensor noise, motion blur, or weather\-related degradations\. Even small perturbations can shift model predictions and confidence estimates\. Although prior work has studied cascade performance on clean data or examined the robustness of individual models,the interaction between cascade routing decisions and input degradations remains underexplored\.

This is critical for sustainable AI: cascade efficiency depends entirely on reliable confidence\-based routing under distribution shift\. Input degradations trigger opposing failures: excessive deferral raises computational cost; suppressed deferral sacrifices accuracy\. Therefore, a critical practical question remains unanswered:at what level of input degradation does a cascade’s efficiency advantage disappear, and can we identify which corruption types pose the greatest threat to routing reliability?

Accurately characterizing this interaction is essential for estimating the true computational and environmental cost of cascade\-based inference in production settings\. In this work, we investigate how input degradation – both static corruptions and sequences of data perturbations – affects confidence\-based deferral in model cascades\. Our contributions are:

- •We quantify how severity, type, and temporal structure of input degradation affect model cascades, identifying which static corruption groups and sequential perturbation types most strongly disrupt accuracy, routing reliability, and deferral quality across datasets\.
- •We disentangle routing failure from fallback\-model degradation, showing that cascade failure can arise from either unreliable confidence\-based routing or from collapse of the large modelMLM\_\{L\}under severe corruption\.
- •We show that the relative importance of these failure modes depends on problem complexity: routing calibration becomes the bottleneck in lower\-complexity settings, while base\-model robustness dominates in higher\-complexity settings\.

## 2Related Work

In the following sections, we review recent work in model cascades as well as methods focusing on distributional shifts\.

### 2\.1Prediction Cascades and Efficient Inference

Prediction cascades reduce computational cost by processing inputs through a sequence of models with increasing capacity, allowing early termination when confidence is sufficient\. Classical examples include boosting methods such as the Viola\-Jones face detector\[[12](https://arxiv.org/html/2608.17711#bib.bib8)\], which uses a cascade of increasingly complex classifiers to reject non\-face regions quickly\. Modern approaches extend this concept to deep learning with learned routing strategies\[[13](https://arxiv.org/html/2608.17711#bib.bib9),[11](https://arxiv.org/html/2608.17711#bib.bib7)\]\. In this study we chose a recent model cascading approach, referred to as Gatekeeper\[[8](https://arxiv.org/html/2608.17711#bib.bib10)\]which routes each inputxxusing the confidence of the small modelMSM\_\{S\}\. If the maximum softmax confidence exceeds an inference thresholdτ\\tau, the prediction ofMSM\_\{S\}is accepted; otherwise, the input is deferred to the large modelMLM\_\{L\}:

y^=\{fMS​\(x\),maxc⁡σ​\(fMS​\(x\)\)c≥τ,fML​\(x\),otherwise\.\\hat\{y\}=\\begin\{cases\}f\_\{M\_\{S\}\}\(x\),&\\max\_\{c\}\\sigma\(f\_\{M\_\{S\}\}\(x\)\)\_\{c\}\\geq\\tau,\\\\ f\_\{M\_\{L\}\}\(x\),&\\text\{otherwise\}\.\\end\{cases\}\(1\)Here,τ∈\[0,1\]\\tau\\in\[0,1\]controls the accept–defer decision at inference time\. Gatekeeper fine\-tunesMSM\_\{S\}with correctness\-aware lossℒGK=α​ℒcorr\+\(1−α\)​ℒincorr\\mathcal\{L\}\_\{\\mathrm\{GK\}\}=\\alpha\\mathcal\{L\}\_\{\\mathrm\{corr\}\}\+\(1\-\\alpha\)\\mathcal\{L\}\_\{\\mathrm\{incorr\}\}, whereα∈\[0,1\]\\alpha\\in\[0,1\]controls the emphasis between correct and incorrect predictions,ℒcorr\\mathcal\{L\}\_\{\\mathrm\{corr\}\}is cross\-entropy on correctly predicted samples, andℒincorr\\mathcal\{L\}\_\{\\mathrm\{incorr\}\}is KL divergence to a uniform distribution on incorrectly predicted samples\.

The framework uses two key metrics to characterize cascade behavior:Cascade accuracyandDeferral performance\. Cascade accuracy simply measures accuracy of the model cascade:Acccasc=1N∑i=1N𝟙\[y^i=yi\]\\mathrm\{Acc\}\_\{\\mathrm\{casc\}\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{1\}\[\\hat\{y\}\_\{i\}=y\_\{i\}\]\. Deferral performance \(sds\_\{d\}\) quantifies routing effectiveness by measuring the normalized area between realized and ideal deferral curves:

sd=∫01\(accreal​\(r\)−accrand​\(r\)\)​𝑑r∫01\(accideal​\(r\)−accrand​\(r\)\)​𝑑rs\_\{d\}=\\frac\{\\int\_\{0\}^\{1\}\(\\mathrm\{acc\}\_\{\\mathrm\{real\}\}\(r\)\-\\mathrm\{acc\}\_\{\\mathrm\{rand\}\}\(r\)\)\\,dr\}\{\\int\_\{0\}^\{1\}\(\\mathrm\{acc\}\_\{\\mathrm\{ideal\}\}\(r\)\-\\mathrm\{acc\}\_\{\\mathrm\{rand\}\}\(r\)\)\\,dr\}\(2\)
Highersds\_\{d\}indicates more effective deferral\. Prior work on cascades has focused primarily on clean, in\-distribution data\. While efficiency gains are well\-documented under these conditions\[[13](https://arxiv.org/html/2608.17711#bib.bib9),[11](https://arxiv.org/html/2608.17711#bib.bib7)\], the reliability of confidence\-based routing under distribution shift remains largely unexplored\. Specifically, whether the calibration induced by model cascades, such as Gatekeeper’s training loss, remains effective when inputs are corrupted or perturbed is an open question that our work addresses\.

### 2\.2Distribution Shift, Calibration, and Cascade Routing

Perturbations of input data can degrade model accuracy and confidence calibration\[[3](https://arxiv.org/html/2608.17711#bib.bib11),[7](https://arxiv.org/html/2608.17711#bib.bib12)\]\. Hendrycks and Dietterich\[[3](https://arxiv.org/html/2608.17711#bib.bib11)\]introduced standardized static corruption and sequential perturbation benchmarks to measure robustness to common input degradations such as noise, blur, weather effects, and digital artifacts\. These benchmarks show that DNNs can suffer substantial performance drops under corrupted inputs, even at moderate severity levels\. Beyond accuracy, distribution shift also affects uncertainty estimates: models trained on clean data often become miscalibrated under corrupted inputs\[[7](https://arxiv.org/html/2608.17711#bib.bib12)\], while post\-hoc calibration methods may not reliably transfer to shifted distributions\[[6](https://arxiv.org/html/2608.17711#bib.bib13)\]\.

While prior work extensively studies robustness and calibration for individual models, cascade systems introduce an additional failure point:routing reliability\. In Gatekeeper cascades, confidence estimates determine whether a sample is accepted by the small modelMSM\_\{S\}or deferred to the large modelMLM\_\{L\}\. Under distribution shift, corrupted confidence signals can therefore alter the accept/defer behavior even when model accuracy alone does not fully explain the failure\. Two cascade\-specific failure modes can arise:overconfident acceptance, whereMSM\_\{S\}accepts degraded samples that should have been deferred, andunderconfident deferral, whereMSM\_\{S\}defers excessively, reducing the efficiency benefit of the cascade\. Our work addresses this gap by evaluating how static corruptions and sequential perturbations affect not only cascade accuracy but also deferral performance\.

## 3Experimental Setup

### 3\.1Datasets

We evaluate robustness on two complementary CIFAR benchmarks\[[3](https://arxiv.org/html/2608.17711#bib.bib11)\]to test distinct failure modes:Static corruptions\(CIFAR\-10\-C/100\-C\) apply single\-step corruption at fixed severity levels, testing cascade robustness to 19 corruption types at five severity levels, grouped into Noise \(Gaussian, shot, impulse\), Blur \(defocus, glass, motion, zoom\), Weather \(snow, frost, fog, brightness\), and Digital \(contrast, elastic, pixelate, JPEG\)\.Sequential perturbations\(CIFAR\-10\-P/CIFAR\-100\-P\) present progressive 30\-frame degradation sequences, testing cascade stability under sequential gradual input perturbation sequences \(motion blur, snow, zoom blur, brightness, fog, Gaussian noise\)\. Each corruption benchmark contains 10,000 test images per corruption type and severity level\.

### 3\.2Models and Training

Following the Gatekeeper formulation\[[8](https://arxiv.org/html/2608.17711#bib.bib10)\], for both CIFAR\-10/100, we instantiateMSM\_\{S\}as a custom SmallCNN andMLM\_\{L\}as ResNet\-18\. Both models are trained on clean data with standard augmentation\.MSM\_\{S\}is trained for 50 epochs using Adam with learning rate10−310^\{\-3\}and weight decay10−410^\{\-4\}\.MLM\_\{L\}is trained for 200 epochs using SGD with learning rate0\.10\.1, momentum0\.90\.9, Nesterov acceleration, weight decay5×10−45\\times 10^\{\-4\}, and cosine annealing\. We then keepMLM\_\{L\}fixed and fine\-tuneMSM\_\{S\}using the Gatekeeper correctness\-aware loss for 30 epochs with Adam, learning rate3×10−43\\times 10^\{\-4\}, weight decay10−410^\{\-4\}, and cosine annealing\. Following the original Gatekeeper design\[[8](https://arxiv.org/html/2608.17711#bib.bib10)\], the confidence\-based routing thresholdτ\\tauis fixed at 0\.7\. Separate Gatekeeper models are fine\-tuned forα∈\{0\.1,0\.3,0\.5,0\.7,0\.9\}\\alpha\\in\\\{0\.1,0\.3,0\.5,0\.7,0\.9\\\}\.

### 3\.3Evaluation Pipeline and Metrics

After training the Gatekeeper cascade on clean CIFAR\-10/100 data, we evaluate it under both clean and corrupted data\. We report energy consumption orC​O2CO\_\{2\}emissions as estimated with the codecarbon library\[[1](https://arxiv.org/html/2608.17711#bib.bib1)\]\. For evaluation of performance under corruptions, we apply all corruptions at inference time before the confidence\-based routing decision\. We report accuracy of the small modelMSM\_\{S\}, the large modelMLM\_\{L\}and the model cascadeG​KGK, as well as deferral performance \(sds\_\{d\}\) to diagnose failure modes\. In addition for the evaluation of sequential perturbations,we report cascade accuracy, deferral rate, and mean cascade flip rate, defined asmFP\-Casc=1T−1∑t=2T𝟙\[y^t≠y^t−1\]\\text\{mFP\-Casc\}=\\frac\{1\}\{T\-1\}\\sum\_\{t=2\}^\{T\}\\mathbb\{1\}\[\\hat\{y\}\_\{t\}\\neq\\hat\{y\}\_\{t\-1\}\], whereTTis the sequence length andy^t\\hat\{y\}\_\{t\}is the cascade prediction at framett\. Since sequential perturbations present consecutive frames with progressive degradation, we measure prediction stability by mFP\-Casc\.

For grouped results, bars show the mean across corruption or perturbation types within each group, and error bars denote the corresponding standard deviation\. These error bars capture variability across degradation types, not variability across independent training seeds\.

## 4Result & Analysis

### 4\.1Model Cascade Performance on Clean Data

In Table[1](https://arxiv.org/html/2608.17711#S4.T1), we report clean\-data performance for the Gatekeeper \(GK\) model cascade on the CIFAR\-10 and CIFAR\-100 data sets and compare the performance in terms of accuracy, deferral performancesds\_\{d\}and energy consumption for cascades and the large or small models used in the cascade\. We observe that for CIFAR\-10, the best model cascade withα=0\.9\\alpha=0\.9yields a competitive accuracy compared to the large modelMLM\_\{L\}while requiring only 60% of the energy\. For the CIFAR\-100 data set we also observe a slight reduction in accuracy but a 10\-fold reduction in energy consumption for the model cascade\. All subsequent evaluations use this fixed threshold across all corruption conditions\.

Table 1:Clean\-data accuracy, deferral performance, and estimated CO2emissions for GK evaluation on CIFAR\-10/100\. Pareto\-optimal settings are shown in bold face\.Note:sds\_\{d\}is undefined forMLM\_\{L\}\-only since there is no deferral\.

### 4\.2Model Cascade Performance on Corrupted Data

The results in[Figure 1](https://arxiv.org/html/2608.17711#S4.F1)demonstrate that the small modelMSM\_\{S\}, the large modelMLM\_\{L\}as well as the model cascade perform less accurately when data is corrupted\. Increasing levels of corruption severity lead to decreasing levels of accuracy\. On CIFAR\-10\-C, cascade accuracy drops from0\.9220\.922on clean data to≈0\.38\\approx 0\.38–0\.580\.58under Noise across severity levels, while Weather remains comparatively robust, staying around0\.750\.75–0\.900\.90\. In contrast, CIFAR\-100\-C shows a much sharper degradation: cascade accuracy drops from0\.7440\.744on clean data to approximately0\.460\.46under Noise at severity 1 and to≈0\.10\\approx 0\.10at severity 5\. Even the most robust Weather group remains only around0\.420\.42–0\.680\.68across severities\. Overall the impact of corruptions on the 10\-class problem CIFAR\-10 is much less pronounced than on the task with larger label\-set cardinality CIFAR\-100\. The clean\-to\-worst\-severity drop is larger on CIFAR\-100\-C: under Noise, accuracy decreases by≈0\.64\\approx 0\.64\(0\.744→0\.100\.744\\rightarrow 0\.10\) compared with≈0\.54\\approx 0\.54on CIFAR\-10\-C \(0\.922→0\.380\.922\\rightarrow 0\.38\); under Weather, the drop is≈0\.32\\approx 0\.32\(CIFAR\-100\) versus≈0\.17\\approx 0\.17\(CIFAR\-10\)\. Comparing the impact of different corruptions on model cascade accuracy we observe a consistent ranking across datasets:Noise≫\\ggBlur\>\>Weather≈\\approxDigital\. Weather corruptions have the least impact on accuracy\. Interestingly, the severity level at which the cascade loses its efficiency advantage is dataset\-dependent\. On CIFAR\-10\-C, Weather and Digital remain above0\.750\.75accuracy through severity 3, whereas on CIFAR\-100\-C no corruption group remains above0\.600\.60beyond severity 3\.

In order to investigate whether this difference in corruption impact on model cascade performance is due to the model cascade’s deferral performance or due to the task difficulty itself we inspect the deferral ratiossds\_\{d\}\([Figure 1](https://arxiv.org/html/2608.17711#S4.F1), bottom row\)\. On CIFAR\-10\-C, the large modelMLM\_\{L\}retains a clear accuracy advantage overMSM\_\{S\}under corruption, enabling deferral to recover useful accuracy when routing remains reliable\. For example, under Noise at severity 1:accL=0\.84\\mathrm\{acc\}\_\{L\}=0\.84vs\.accS=0\.67\\mathrm\{acc\}\_\{S\}=0\.67\(27 pp gap\)\. In contrast, on CIFAR\-100\-C, severe Noise corruption degrades both the large modelMLM\_\{L\}and the small modelMSM\_\{S\}to similarly low accuracy \(<0\.10<0\.10\), eliminating the utility of the large model as a fallback\.

![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar10_c10c_casc_acc.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar100_c100c_casc_acc.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar10_c10c_acc_s.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar100_c100c_acc_s.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar10_c10c_acc_l.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar100_c100c_acc_l.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar10_c10c_s_d.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar100_c100c_s_d.png)Figure 1:Corruption robustness on CIFAR\-10\-C \(left\) and CIFAR\-100\-C \(right\)\. For the CIFAR\-10\-C data the large modelMLM\_\{L\}remains accurate but the lower deferral scoresds\_\{d\}indicates routing failure\. For the CIFAR\-100\-C data even the large modelMLM\_\{L\}performs poorly under Noise, limiting the model cascade performance\.These results suggest that corruptions impact the performance of the model cascade in different ways: For CIFAR\-10\-C we observe that the large modelMLM\_\{L\}remains substantially more accurate than the small modelMSM\_\{S\}under corruptions \(e\.g\. 0\.84 vs 0\.67 under Noise\-1\), but the degraded confidence signals \(sd: 0\.686→0\.35 for Noise\) prevent the cascade from exploiting this high accuracy of the large modelMLM\_\{L\}\. Due to poor deferral performance, the cascade loses≈\\approx19pp of recoverable accuracy\.

In contrast for CIFAR\-100\-C especially high severity corruptions degrade both the accuracy of the small modelMSM\_\{S\}as well as the large modelMLM\_\{L\}, eliminating the large model’s advantage\. These results suggest that it is rather the task difficulty than the deferral performance of the model cascade that leads to the lower cascade accuracy\. If the large modelMLM\_\{L\}is not able to correctly identify samples, the model cascade will not be able to compensate for that\.

### 4\.3Model Cascade Performance on Sequential Corruptions

Evaluating the impact of sequential data perturbations in[Figure 2](https://arxiv.org/html/2608.17711#S4.F2)we observe that progressive perturbations in the CIFAR\-10\-P and CIFAR\-100\-P tasks reduce the cascade accuracy, but less severely than static corruptions\. On CIFAR\-10\-P, accuracy drops from 0\.922 to 0\.80–0\.85; on CIFAR\-100\-P, from 0\.744 to 0\.51–0\.59\. Unlike static corruptions where Noise dominates, Blur emerges as the most damaging perturbation on CIFAR\-10\-P, indicating that gradual, frame\-by\-frame degradation affects confidence\-based routing differently than single\-step corruptions\.

Examining the mean cascade flip rate \(mFP\-Casc\), measuring how often the deferral decision flips with progressive perturbations, reveals a counterintuitive pattern: perturbations reduce prediction instability across all groups\. Weather yields the largest reduction \(67% on CIFAR\-10\-P; 76% on CIFAR\-100\-P\), while Noise yields the smallest \(9% and 24%, respectively\)\. However, this apparent stability is deceptive\. The deferral rate also decreases under perturbation, from 0\.505 to 0\.34–0\.39 on CIFAR\-10\-P and from 0\.787 to 0\.59–0\.62 on CIFAR\-100\-P\. Fewer samples are routed to the large modelMLM\_\{L\}, and the cascade relies more heavily on the degraded small modelMSM\_\{S\}\. Combined with the observed accuracy drop, the lower flip rate does not indicate improved robustness, but rather reflects a suppressed fallback to the large modelMLM\_\{L\}\. These results suggest that perturbations expose a distinct cascade failure mode: progressive degradation suppresses deferral decisions, producing stable but less reliable cascade predictions\. For cascaded systems under temporal degradation, robustness must be evaluated jointly through accuracy, prediction stability, and deferral behavior\.

![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar10_c10p_casc_acc_1.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar100_c100p_casc_acc_1.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar10_c10p_mfp_casc.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar100_p100p_mfp_casc.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar10_p10p_defer_rate.png)![Refer to caption](https://arxiv.org/html/2608.17711v1/plots/cifar100_p100p_defer_rate.png)Figure 2:Robustness under perturbation sequences on CIFAR\-10\-P \(left\) and CIFAR\-100\-P \(right\)\. Perturbations reduce cascade accuracy and mean flip rate, but also suppress deferral to the large modelMLM\_\{L\}\. Thus, lower prediction instability can be misleading: the cascade appears stable while relying more on the degraded small modelMSM\_\{S\}\.

## 5Conclusion and Future Work

We investigated the impact of data corruptions on the performance of model cascades\. In line with previous work we find that on clean data model cascades can achieve competitive predictive performance while obtaining an up to 10\-fold reduction in energy consumption\. However, our results demonstrate that input degradation affects not only cascade accuracy but also the confidence\-based routing decisions that determine whether efficient inference remains reliable\. The failure mode of model cascades differs across datasets: In the CIFAR\-10\-C task we observe mainly routing failures, where the large modelMLM\_\{L\}retains its high accuracy under data corruptions but the corrupted confidence signals of the smaller models limit effective deferral\. In the case of the CIFAR\-100\-C task we see that also the larger model exhibits decreased performance under data corruptions, limiting its utility as a fallback option for deferral of difficult samples\.

Overall our results indicate that model cascades can be a viable alternative to large or small models as they offer a convenient per\-sample tradeoff between predictive performance and energy consumption\. Using them responsibly requires better understanding of their behaviour under data set shifts\. We observe that model cascades can fail under corruption through either routing breakdown \(when the large model remains accurate but confidence signals corrupt\) or model collapse \(when distribution shift degrades both models\)\. In our experiments, the label set cardinality is indicative of the failure mode: CIFAR\-10’s lower number of classes preservesMLM\_\{L\}robustness, exposing routing fragility; CIFAR\-100’s higher label set cardinality breaks both models, revealing fundamental model brittleness\.

Future work will investigate shift\-aware Gatekeeper mechanisms that adapt the inference threshold under degraded inputs, rather than using a fixed clean\-data configuration\. Another direction is to incorporate uncertainty calibration or corruption\-aware confidence correction into the routing decision, so that degraded samples are not incorrectly accepted byMSM\_\{S\}\. Finally, extending this analysis to larger\-scale datasets, additional model families, and real deployment measurements of latency, energy, and CO2emissions would provide a more complete picture of the robustness–efficiency trade\-off in cascade\-based inference\.

## 6Acknowledgement

This work was funded by the German Federal Ministry for Economic Affairs and Energy within the project “Safe AI Engineering – Sicherheitsargumentation befähigendes AI Engineering über den gesamten Lebenszyklus einer KI\-Funktion”\. This research was also supported by the German Research Foundation \(DFG\), project number 528483508 – FIP 12\. The authors thank the project partners for the successful cooperation\.

## References

- \[1\]B\. Courty, V\. Schmidt, Goyal\-Kamal, MarionCoutarel, B\. Feld, J\. Lecourt, LiamConnell, SabAmine, inimaz, supatomic, M\. Léval, L\. Blanche, A\. Cruveiller, ouminasara, F\. Zhao, A\. Joshi, A\. Bogroff, A\. Saboni, H\. de Lavoreille, N\. Laskaris, E\. Abati, D\. Blank, Z\. Wang, A\. Catovic, alencon, M\. Stęchły, C\. Bauer, Lucas\-Otavio, JPW, and MinervaBooks\(2024\)Mlco2/codecarbon: v2\.4\.1\.Zenodo\.External Links:[Document](https://dx.doi.org/10.5281/zenodo.11171501),[Link](https://doi.org/10.5281/zenodo.11171501)Cited by:[§3\.3](https://arxiv.org/html/2608.17711#S3.SS3.p1.1)\.
- \[2\]S\. Han, J\. Pool, J\. Tran, and W\. J\. Dally\(2015\)Learning both weights and connections for efficient neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),pp\. 1135–1143\.Cited by:[§1](https://arxiv.org/html/2608.17711#S1.p1.1)\.
- \[3\]D\. Hendrycks and T\. Dietterich\(2019\)Benchmarking neural network robustness to common corruptions and perturbations\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2\.2](https://arxiv.org/html/2608.17711#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2608.17711#S3.SS1.p1.1)\.
- \[4\]G\. Hinton, O\. Vinyals, and J\. Dean\(2015\)Distilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[§1](https://arxiv.org/html/2608.17711#S1.p1.1)\.
- \[5\]B\. Jacob, S\. Kligys, B\. Chen, M\. Zhu, M\. Tang, A\. Howard, H\. Adam, and D\. Kalenichenko\(2018\)Quantization and training of neural networks for efficient integer\-arithmetic\-only inference\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 2704–2713\.Cited by:[§1](https://arxiv.org/html/2608.17711#S1.p1.1)\.
- \[6\]M\. Minderer, J\. Djolonga, R\. Romijnders, F\. Hubis, X\. Zhai, N\. Houlsby, D\. Tran, and M\. Lucic\(2021\)Revisiting the calibration of modern neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),pp\. 15682–15694\.Cited by:[§2\.2](https://arxiv.org/html/2608.17711#S2.SS2.p1.1)\.
- \[7\]Y\. Ovadia, E\. Fertig, J\. Ren, Z\. Nado, D\. Sculley, S\. Nowozin, J\. V\. Dillon, B\. Lakshminarayanan, and J\. Snoek\(2019\)Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift\.InAdvances in Neural Information Processing Systems \(NeurIPS\),pp\. 13991–14002\.Cited by:[§2\.2](https://arxiv.org/html/2608.17711#S2.SS2.p1.1)\.
- \[8\]S\. Rabanser, N\. Rauschmayr, A\. Kulshrestha, P\. Poklukar, W\. Jitkrittum, S\. Augenstein, C\. Wang, and F\. Tombari\(2026\)Gatekeeper: improving model cascades through confidence tuning\.Advances in Neural Information Processing Systems38,pp\. 19518–19547\.Cited by:[§1](https://arxiv.org/html/2608.17711#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.17711#S2.SS1.p1.1),[§3\.2](https://arxiv.org/html/2608.17711#S3.SS2.p1.1)\.
- \[9\]R\. Schwartz, J\. Dodge, N\. A\. Smith, and O\. Etzioni\(2020\)Green ai\.Communications of the ACM63\(12\),pp\. 54–63\.Cited by:[§1](https://arxiv.org/html/2608.17711#S1.p1.1)\.
- \[10\]E\. Strubell, A\. Ganesh, and A\. McCallum\(2019\)Energy and policy considerations for deep learning in nlp\.Proceedings of ACL,pp\. 3645–3650\.Cited by:[§1](https://arxiv.org/html/2608.17711#S1.p1.1)\.
- \[11\]S\. Teerapittayanon, B\. McDanel, and H\. Kung\(2016\)BranchyNet: fast inference via early exiting from deep neural networks\.InInternational Conference on Pattern Recognition \(ICPR\),pp\. 2464–2469\.Cited by:[§1](https://arxiv.org/html/2608.17711#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.17711#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2608.17711#S2.SS1.p4.1)\.
- \[12\]P\. Viola and M\. Jones\(2001\)Rapid object detection using a boosted cascade of simple features\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.1,pp\. I–I\.Cited by:[§2\.1](https://arxiv.org/html/2608.17711#S2.SS1.p1.1)\.
- \[13\]X\. Wang, F\. Yu, Z\. Dou, T\. Darrell, and J\. E\. Gonzalez\(2018\)SkipNet: learning dynamic routing in convolutional networks\.InEuropean Conference on Computer Vision \(ECCV\),pp\. 409–424\.Cited by:[§2\.1](https://arxiv.org/html/2608.17711#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2608.17711#S2.SS1.p4.1)\.