A Formal Methodological Framework for Auditing Robustness and Fidelity in Explainable AI: From Application to Trust Certification

# A Formal Methodological Framework for Auditing Robustness and Fidelity in Explainable AI:From Application to Trust Certification
Source: [https://arxiv.org/html/2608.23817](https://arxiv.org/html/2608.23817)
Rosa Elysabeth RalinirinaAffiliation:Doctoral School Modeling–Computer Science, University of Fianarantsoa, MadagascarEmail:[ralinirinarosa7@gmail\.com](mailto:)Jean Christian RalaivaoNiaiko Michaël RalaivaoAffiliation:Doctoral School Modeling–Computer Science, University of Fianarantsoa, MadagascarAlain Josué RatovondrahonaAffiliation:Doctoral School Modeling–Computer Science, University of Fianarantsoa, MadagascarThomas MahatodyAffiliation:Doctoral School Modeling–Computer Science, University of Fianarantsoa, Madagascar

###### Abstract

SHAP and LIME are now standard tools for interpreting black\-box predictions, yet their outputs can vary substantially when the input is perturbed by small amounts of noise—a problem we observed firsthand in our previous work on food security in Madagascar\([Ralinirina et al\., 2025](https://arxiv.org/html/2608.23817#bib.bib1)\)\. This variability raises the question of whether such explanations can be trusted at all\. We address it by constructing an auditing protocol that measures two properties of any post\-hoc explainer: robustness \(how stable the explanation is under input perturbation\) and fidelity \(whether the features deemed important actually drive the model’s prediction\)\. These two quantities are combined into a single Trust Score\. We run the protocol on a multi\-sectoral dataset from Madagascar \(83 features, 253 records, 4 malnutrition classes\) using three classifiers and two explainers, plus their regularized counterparts\. The results are sobering: models with AUC above 0\.99 can produce numerically degenerate or flatly uninformative explanations, and fidelity scores lose discriminative power when the model is overfitted\. These findings suggest that auditing XAI outputs is not optional but necessary, particularly when they inform decisions in sensitive domains\.

Keywords:Explainable AI \(XAI\), Robustness, Fidelity, Methodological Framework, Trustworthy AI, Food Security\.

11footnotetext:This manuscript was peer\-reviewed, accepted for publication in the proceedings, and presented at the International Conference on Applied Informatics \(ICAI 2026,[https://icai\.uni\-eszterhazy\.hu/2026/](https://icai.uni-eszterhazy.hu/2026/)\)\. This is the revised version\. The organizers did not send the revision notification to the authors in time, and as a result the paper was not included in the proceedings\.## 1Introduction

When an AI model predicts a food crisis in a region of Madagascar, the prediction itself is only half the story\. The other half—how the model arrived at it—is what policymakers, nutrition officers, and field workers need in order to trust and act on the output\. Post\-hoc explainers such as SHAP\([Lundberg and Lee, 2017](https://arxiv.org/html/2608.23817#bib.bib12)\)and LIME\([Ribeiro et al\., 2016](https://arxiv.org/html/2608.23817#bib.bib13)\)fill this role by attributing importance scores to input features\. The trouble is that these attributions are not always reliable\. Small perturbations to the input can change them, sometimes dramatically, and there is no built\-in mechanism to tell whether the features an explainer highlights are genuinely used by the model\.

We encountered this problem directly in\([Ralinirina et al\., 2025](https://arxiv.org/html/2608.23817#bib.bib1)\), where we applied SHAP and LIME to food security indicators across Madagascar’s 23 regions\. Explanations for nearly identical inputs often differed, and the ranking of important features shifted depending on which explainer was used\. This is not an isolated finding\. Slack et al\.\([Slack and others, 2020](https://arxiv.org/html/2608.23817#bib.bib7)\)showed that LIME and SHAP can be deliberately fooled; Alvarez\-Melis and Jaakkola\([Alvarez\-Melis and Jaakkola, 2018](https://arxiv.org/html/2608.23817#bib.bib6)\)proved that local explanations are generally unstable\. Surveys by Adadi and Berrada\([Adadi and Berrada, 2018](https://arxiv.org/html/2608.23817#bib.bib3)\)and Guidotti et al\.\([Guidotti and others, 2018](https://arxiv.org/html/2608.23817#bib.bib4)\)have warned that post\-hoc methods create an “illusion of transparency,” and domain\-specific studies in healthcare\([Stiglic and others, 2020](https://arxiv.org/html/2608.23817#bib.bib10)\)and NLP\([Danilevsky et al\., 2020](https://arxiv.org/html/2608.23817#bib.bib11)\)confirm that visual inspection of saliency maps is insufficient\. Our research group has also explored AI\-driven automation for complex tasks\([Ratovondrahona et al\., 2023](https://arxiv.org/html/2608.23817#bib.bib2)\), a line of work that parallels the present goal of building a systematic, verifiable XAI auditing methodology\.

The question, then, is not whether explanations should be audited—they must be—but how\. We propose a protocol that measures two things:*robustness*\(stability under noise\) and*fidelity*\(faithfulness to the model\)\. Each is quantified by a single metric, and the two are combined into a Trust Score\. We validate the protocol on the same Malagasy food security dataset, using three classifiers \(Random Forest, XGBoost, Neural Network\) and two explainers \(SHAP, LIME\), plus their regularized variants\. The results confirm that near\-perfect AUC does not shield explanations from instability or vacuity, and the Trust Score captures these defects in a single number\.

Section 2 surveys related work\. Section 3 describes the framework\. Section 4 presents the case study and results\. Section 5 discusses what the results mean—including the overfitting and numerical issues they reveal\. Section 6 concludes\.

## 2Related Work

Post\-hoc interpretability took off after DARPA’s XAI program\([Gunning and Aha, 2019](https://arxiv.org/html/2608.23817#bib.bib5)\), and surveys by Adadi and Berrada\([Adadi and Berrada, 2018](https://arxiv.org/html/2608.23817#bib.bib3)\)and Guidotti et al\.\([Guidotti and others, 2018](https://arxiv.org/html/2608.23817#bib.bib4)\)now catalog dozens of methods\. Most are model\-agnostic: they treat the predictor as a black box and probe it with perturbed inputs\. LIME\([Ribeiro et al\., 2016](https://arxiv.org/html/2608.23817#bib.bib13)\)fits a local linear model; SHAP\([Lundberg and Lee, 2017](https://arxiv.org/html/2608.23817#bib.bib12)\)computes Shapley values\. Both are popular, but both are fragile\. Slack et al\.\([Slack and others, 2020](https://arxiv.org/html/2608.23817#bib.bib7)\)demonstrated adversarial attacks that fool LIME and SHAP into producing arbitrary explanations\. Alvarez\-Melis and Jaakkola\([Alvarez\-Melis and Jaakkola, 2018](https://arxiv.org/html/2608.23817#bib.bib6)\)showed that local explanations are sensitive to the choice of neighborhood\.

Evaluating explanations is a separate and less studied problem\. Samek et al\.\([Samek et al\., 2017](https://arxiv.org/html/2608.23817#bib.bib14)\)made the case for quantitative evaluation rather than visual inspection\. Holzinger et al\.\([Holzinger et al\., 2019](https://arxiv.org/html/2608.23817#bib.bib15)\)proposed “causability” as a quality criterion\. Domain\-specific studies exist—Islam et al\.\([Islam and others, 2024](https://arxiv.org/html/2608.23817#bib.bib8)\)for healthcare, Linheiro et al\.\([Linheiro and others, 2023](https://arxiv.org/html/2608.23817#bib.bib9)\)for agriculture—but they evaluate explainers in isolation, without a unified protocol that can compare them on equal footing\.

Our framework differs in that it pairs robustness and fidelity in a single pipeline and produces a single Trust Score\. The advantage is not novelty of the individual metrics—Jensen\-Shannon divergence for robustness and feature ablation for fidelity are both well\-known—but the fact that they are computed side by side, on the same data, with the same model, yielding a directly comparable score across explainer–model pairs\.

## 3Methodological Framework Architecture

Figure[1](https://arxiv.org/html/2608.23817#S3.F1)shows the pipeline\. Given a trained modelffand an explainerEE, the auditor computes a robustness scoreR⁡\(x\)R\(x\)and a fidelity scoreF⁡\(x\)F\(x\)for each test instancexx, then combines them into a Trust ScoreT⁡\(x\)=α​R​\(x\)\+β​F​\(x\)T\(x\)=\\alpha R\(x\)\+\\beta F\(x\)withα\+β=1\\alpha\+\\beta=1\.

![Refer to caption](https://arxiv.org/html/2608.23817v1/figures/framework_architecture.png)Figure 1:The auditing pipeline\. Robustness and fidelity are measured independently and then combined\.### 3\.1Robustness and Local Stability

Robustness quantifies how much an explanation changes when the input is slightly perturbed\. We drawδ∼𝒩⁡\(0,σ2​I\)\\delta\\sim\\mathcal\{N\}\(0,\\sigma^\{2\}I\), computeE⁡\(x\)E\(x\)andE⁡\(x\+δ\)E\(x\+\\delta\), normalize both importance vectors to sum to 1 \(denotedE~\\tilde\{E\}\), and measure their divergence:

R\(x\)=1−1N∑i=1NDJ​S\(E~\(x\)∥E~\(x\+δi\)\)R\(x\)=1\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}D\_\{JS\}\\\!\\left\(\\tilde\{E\}\(x\)\\,\\\|\\,\\tilde\{E\}\(x\+\\delta\_\{i\}\)\\right\)
whereNNis the number of perturbation trials\. A practical difficulty arises when the explainer outputs zero importance for all features—this happens with TreeSHAP on certain overfitted models, making the JSD undefined\. We resolve it by addingϵ=10−10\\epsilon=10^\{\-10\}to each component ofE~\\tilde\{E\}before normalization\. This has a negligible effect on the numerical value but prevents division by zero\. A robustness score near 1 means the explanation barely changes under perturbation; a score near 0 means it changes a lot\.

### 3\.2Fidelity through Feature Ablation

Fidelity checks whether the features the explainer calls important actually matter to the model\. For an inputxxwith explanationE⁡\(x\)E\(x\), letFkF\_\{k\}be the set of top\-kkfeatures\. We mask them \(replace with feature means\) to obtainx∖Fkx\_\{\\setminus F\_\{k\}\}and compute:

Fk​\(x\)=1−\|f⁡\(x\)−f⁡\(x∖Fk\)\|\|f⁡\(x\)\|\+ϵF\_\{k\}\(x\)=1\-\\frac\{\|f\(x\)\-f\(x\_\{\\setminus F\_\{k\}\}\)\|\}\{\|f\(x\)\|\+\\epsilon\}
The overall fidelity is the mean over a set of ablation levels𝒦\\mathcal\{K\}:

F⁡\(x\)=1K​∑k∈𝒦Fk​\(x\)F\(x\)=\\frac\{1\}\{K\}\\sum\_\{k\\in\\mathcal\{K\}\}F\_\{k\}\(x\)
A fidelity near 1 means ablating the top features noticeably shifts the prediction; a fidelity near 0 means the explainer highlights features the model does not actually rely on\. One subtlety: when the model is extremely confident \(predicting near 0 or near 1 for all inputs\), ablating even important features barely changes the output, pushing allFkF\_\{k\}close to 1\. Fidelity then becomes uninformative—a point we return to in Section[4\.3\.5](https://arxiv.org/html/2608.23817#S4.SS3.SSS5)\. This risk of uninformative or misleading feature\-importance measures aligns with\([Hooker et al\., 2021](https://arxiv.org/html/2608.23817#bib.bib16)\), who showed that permutation\-based importance can force extrapolation into regions where the model was never trained\.

### 3\.3Trust Score

We combine the two metrics into

T⁡\(x\)=α⋅R⁡\(x\)\+β⋅F⁡\(x\),α\+β=1T\(x\)=\\alpha\\cdot R\(x\)\+\\beta\\cdot F\(x\),\\quad\\alpha\+\\beta=1
and aggregate over a dataset:

T⁡\(𝒟\)=1\|𝒟\|​∑x∈𝒟T⁡\(x\)T\(\\mathcal\{D\}\)=\\frac\{1\}\{\|\\mathcal\{D\}\|\}\\sum\_\{x\\in\\mathcal\{D\}\}T\(x\)
We setα=β=0\.5\\alpha=\\beta=0\.5by default, giving equal weight to stability and faithfulness\. The weights can be adjusted: in a context where explanation stability is critical \(e\.g\., communicating with non\-technical stakeholders\), one might setα\>0\.5\\alpha\>0\.5\.

## 4Case Study: Food Security in Madagascar

### 4\.1Data Description

The dataset aggregates official reports from Madagascar’s Ministry of Agriculture, Ministry of Public Health, and the National Development Plan\. It covers 23 regions over 13 years \(2010–2023\) and includes:

- •Climate: rainfall, cyclonic events, temperature anomalies\.
- •Agriculture: rice production, soil quality, pest incidence\.
- •Socio\-economics: market prices, household income, transport quality\.
- •Nutrition: IPC malnutrition phases, child stunting rates, vaccination coverage\.
- •Demographics: population, literacy rates, school enrollment\.

After preprocessing \(removing identifiers and date columns, encoding categorical variables\), we obtained 83 features and 253 instances\. The target variableSituation\-MC\(chronic malnutrition\) has 4 ordinal classes treated as multi\-class: Acceptable \(3 instances\), Precarious \(85\), Alarming \(114\), Critical \(51\)\. The severe class imbalance—only 3 instances in the Acceptable class—limits the reliability of per\-class AUC estimates and prevents stratified cross\-validation folds from containing all four classes, a point we return to below\.

### 4\.2Experimental Setup

Three classifiers were trained on a stratified 82/18 train/test split \(208 training, 45 test instances\):

- •Random Forest: 100 trees, max depth 10\.
- •XGBoost: 200 estimators, learning rate 0\.1, max depth 6\.
- •Neural Network: 3 hidden layers \(64 units each\), ReLU, Adam optimizer\.

Because the Acceptable class contains only 3 instances, standard stratified 5\-fold cross\-validation produces only 3 valid folds \(the remaining 2 folds lack at least one class\)\. On those 3 folds, macro\-average AUC reaches0\.998±0\.0020\.998\\pm 0\.002\(RF\),0\.999±0\.0020\.999\\pm 0\.002\(XGB\), and0\.994±0\.0010\.994\\pm 0\.001\(NN\)\. These high values suggest genuine class separability, though the extreme imbalance makes per\-class estimates unreliable\.

To assess the impact of regularization on explanation quality, we also trained three regularized variants:

- •RFreg\{\}\_\{\\text\{reg\}\}: max depth 6, min samples per leaf 5, 500 trees\.
- •XGBreg\{\}\_\{\\text\{reg\}\}: max depth 3, learning rate 0\.01,λ=2\\lambda=2\.
- •NNreg\{\}\_\{\\text\{reg\}\}: dropout 0\.3, weight decay10−410^\{\-4\}, early stopping \(patience 10\)\.

Explanations were generated with SHAP \(TreeSHAP for tree\-based models, KernelSHAP for the NN\) and LIME \(default parameters\)\. The audit was run on 30 randomly sampled test instances withσ=0\.1\\sigma=0\.1andN=10N=10perturbations\. Ablation levels were𝒦=\{3,5,10,41,82\}\\mathcal\{K\}=\\\{3,5,10,41,82\\\}, corresponding to removing 3, 5, 10, roughly half, and nearly all of the 83 features\. Implementation used Python 3\.9 with scikit\-learn, XGBoost, SHAP, and LIME\.

### 4\.3Results

#### 4\.3\.1Predictive Performance

Table[1](https://arxiv.org/html/2608.23817#S4.T1)reports test\-set macro\-average AUC for all six models\. The original models all reach AUC above 0\.99; the regularized Neural Network drops to 0\.888, showing that aggressive regularization degrades performance when training data are scarce\.

Table 1:Test\-set macro\-average AUC for each model\.
#### 4\.3\.2Robustness

Figure[2](https://arxiv.org/html/2608.23817#S4.F2)shows robustness scores\. SHAP is more robust than LIME on tree\-based models \(RF: 0\.902 vs\. 0\.700; XGB: 0\.978 vs\. 0\.658\)\. The XGBoost\+SHAP robustness of 0\.978 requiredϵ\\epsilon\-smoothing: without it, TreeSHAP returned zero\-valued importance vectors for some instances where the model predicted with near certainty, making the JSD undefined\. On the Neural Network, SHAP and LIME are closer \(0\.694 vs\. 0\.738\), likely because KernelSHAP introduces its own sampling noise\.

![Refer to caption](https://arxiv.org/html/2608.23817v1/figures/robustness_scores.png)Figure 2:Robustness scores for all six model–explainer pairs\. XGBoost\+SHAP usesϵ\\epsilon\-smoothed computation\.Regularization improves SHAP robustness on RF \(0\.923 vs\. 0\.902\) and barely affects XGBoost \(0\.958 vs\. 0\.978\), while the regularized NN shows slightly higher SHAP robustness \(0\.748 vs\. 0\.694\)\. LIME robustness remains in the 0\.66–0\.76 range across all models\.

#### 4\.3\.3Fidelity

Table[2](https://arxiv.org/html/2608.23817#S4.T2)reports fidelity at each ablation level\. The most striking pattern is XGBoost’s flat fidelity: both SHAP and LIME yield≈0\.620\\approx 0\.620at every ablation level, fromF3F\_\{3\}toF82F\_\{82\}\. Whether you remove the 3 features the explainer calls most important or 82 of them, the prediction shifts by nearly the same amount\. This is not because fidelity is uniformly high—it is not—but because the model’s predictions change by a consistent, moderate amount regardless of how many top features are removed\.

Table 2:Fidelity at each ablation level \(test\-set mean\)\.For the other models, fidelity declines as more features are ablated—as one would expect if the explainer correctly ranks features by importance\. The NN\+LIME combination shows the steepest drop:F3=0\.984F\_\{3\}=0\.984down toF82=0\.734F\_\{82\}=0\.734, meaning the top 3 features identified by LIME are genuinely important to the network’s predictions, while the less important ones matter less\. The regularized Neural Network amplifies this pattern even further \(F3=0\.977F\_\{3\}=0\.977,F82=0\.802F\_\{82\}=0\.802for LIME\)\.

#### 4\.3\.4Trust Scores

Table[3](https://arxiv.org/html/2608.23817#S4.T3)gives aggregate Trust Scores \(α=β=0\.5\\alpha=\\beta=0\.5\)\. The highest score belongs to RFreg\{\}\_\{\\text\{reg\}\}\+SHAP at 0\.871; the lowest to XGBoost\+LIME at 0\.640\. Regularization improves the Trust Score across all six model–explainer pairs\. The gain is largest for XGBoost \(SHAP: 0\.799→\\to0\.836; LIME: 0\.640→\\to0\.687\), where the original model’s overconfidence depressed fidelity\. For RF and NN, the improvements are more modest but consistent\.

Table 3:Trust Scores \(α=β=0\.5\\alpha=\\beta=0\.5\)\. 95% CI from 10 bootstrap iterations in parentheses\.∗Computed withϵ\\epsilon\-smoothing\.

Figure[3](https://arxiv.org/html/2608.23817#S4.F3)shows the fidelity decay curves for all model–explainer pairs\. The XGBoost lines are conspicuously flat, while NN and RF lines slope downward—the expected pattern when the explainer’s feature ranking has real meaning\.

![Refer to caption](https://arxiv.org/html/2608.23817v1/figures/fidelity_curves.png)Figure 3:Fidelity as a function of the number of ablated features\. XGBoost lines are nearly flat, indicating loss of discriminative power\.
#### 4\.3\.5Overfitting and Generalization

Three of the original models have AUC above 0\.99\. The question is whether this reflects genuine separability or memorization of a small dataset\. Cross\-validation on 3 valid folds \(the other 2 lack at least one class\) confirms high AUC across all folds, and the class distributions do suggest that the four malnutrition categories occupy distinct regions of the feature space—particularly along rice production and rainfall dimensions\.

Still, overconfidence is a real concern\. Table[4](https://arxiv.org/html/2608.23817#S4.T4)compares original and regularized models\. The regularized RF and XGB lose virtually no AUC \(0\.994 and 0\.997 vs\. 0\.999 and 0\.992\), while the regularized NN drops sharply to 0\.888—the penalty for adding dropout and weight decay to a network trained on 208 examples\. Yet the NNreg\{\}\_\{\\text\{reg\}\}produces the most discriminative fidelity curves: LIME fidelity drops from 0\.977 atF3F\_\{3\}to 0\.802 atF82F\_\{82\}, a 17\.5\-point spread that is far more informative than the flat 0\.620 of XGBoost\+SHAP\.

Table 4:Original vs\. regularized models\. Trust scores reported as SHAP ; LIME\.∗Withϵ\\epsilon\-smoothing\.

The flat fidelity of XGBoost \(all values≈0\.620\\approx 0\.620\) is a symptom of overconfidence\. Whenf⁡\(x\)f\(x\)is always close to 0 or 1, removing features shifts the prediction by a roughly constant fraction regardless of which features are removed, soFk≈0\.620F\_\{k\}\\approx 0\.620for everykk\. Regularization relaxes this behavior somewhat \(F3=0\.715F\_\{3\}=0\.715,F82=0\.712F\_\{82\}=0\.712for XGBreg\{\}\_\{\\text\{reg\}\}\+SHAP\), but the curve remains nearly flat because XGBoost’s regularized confidence is still high\. The metric regains full discriminative power only when the model is less confident, as with the NNreg\{\}\_\{\\text\{reg\}\}\.

#### 4\.3\.6Feature Importance

Across models and explainers, rice production, rainfall anomalies, and market prices consistently rank among the top 3 features\. Some institutional indicators show high SHAP values but low robustness, suggesting they may be artifacts of the model rather than genuine predictors\.

## 5Discussion

The main takeaway is simple: AUC above 0\.99 does not guarantee trustworthy explanations\. XGBoost’s near\-perfect predictions made TreeSHAP produce zero\-valued importance vectors for some instances, crashing the JSD calculation\. Theϵ\\epsilon\-smoothing fix resolves the numerical issue, but it does not change the underlying behavior—the explainer is essentially saying “nothing matters, because the model already knows the answer\.” This is a genuine finding, not a bug: the framework detects when explanations become uninformative\.

The flat fidelity scores for XGBoost reinforce the point\. Because the model is so confident, removing features shifts its output by a roughly constant amount, and fidelity compresses to≈0\.620\\approx 0\.620for every ablation level and both explainers\. The metric can no longer tell a good explanation from a bad one\. After regularization, fidelity varies slightly more \(F3=0\.715F\_\{3\}=0\.715vs\.F82=0\.712F\_\{82\}=0\.712for XGBreg\{\}\_\{\\text\{reg\}\}\+SHAP\), but the gain is modest because XGBoost’s regularized confidence is still high\. The pattern becomes pronounced only with the regularized Neural Network, where LIME fidelity drops from 0\.977 atF3F\_\{3\}to 0\.802 atF82F\_\{82\}\.

These observations have a practical implication: before auditing explanations, check whether the model is severely overfitted\. If it is, fidelity and—to a lesser extent—robustness lose discriminative power, and the Trust Score should be interpreted with caution\.

On robustness, SHAP consistently outperforms LIME on tree\-based models \(RF: 0\.902 vs\. 0\.700; XGB: 0\.978 vs\. 0\.658\), likely because TreeSHAP follows an exact computation path that avoids the sampling variability inherent in LIME\. On the Neural Network, the two explainers are closer \(0\.694 vs\. 0\.738\), presumably because KernelSHAP introduces its own sampling noise\.

The Trust Score condenses two dimensions into one number, which is convenient for comparison but inevitably loses nuance\. The choice ofα\\alphaandβ\\betamatters\. In a field setting where explanations are presented to non\-technical staff, stability \(α\>0\.5\\alpha\>0\.5\) may matter more than faithfulness; in a regulatory context where the explainer must accurately reflect the model, fidelity \(β\>0\.5\\beta\>0\.5\) takes priority\. The score makes this trade\-off explicit rather than hiding it\.

### 5\.1Limitations

Several caveats apply\. First, the dataset has only 253 instances, and the Acceptable class contains just 3 examples\. This limits the reliability of per\-class metrics and prevents standard 5\-fold cross\-validation from using all folds\. Second, the ablation protocol scales asO⁡\(d⋅n\)O\(d\\cdot n\), which limits its use with very large feature sets or deep models\. Third, Gaussian perturbations may not reflect domain\-relevant variations \(a drought scenario is not a random perturbation of rainfall\)\. Fourth, the weightsα\\alphaandβ\\betarequire domain judgment; there is no universally correct setting\. Fifth, the framework currently addresses feature attribution only; counterfactuals and rule\-based explanations would need adaptation\. Sixth, when the model is overfitted, fidelity loses discriminative power—regularization or calibration should be applied first\. Seventh, the Malagasy dataset has high class separability, so the numerical findings may not transfer directly to other domains\.

## 6Conclusion

We have presented an auditing framework that measures robustness and fidelity of post\-hoc explanations and combines them into a Trust Score\. Applied to a food security dataset from Madagascar \(253 instances, 83 features, 4 classes\), it reveals two things worth noting: near\-perfect AUC can coexist with numerically degenerate explanations \(XGBoost\+SHAP yielding zero\-valued importance vectors\), and fidelity becomes flat when the model is overfitted \(XGBoost fidelity≈0\.620\\approx 0\.620at all ablation levels\)\. After regularization, fidelity recovers discriminative power, particularly for the Neural Network, confirming that the framework is informative when applied to well\-calibrated models\.

Further work should address temporal and spatial data structures, develop domain\-specific perturbation strategies, and test the framework in real\-time decision support settings\.

## Acknowledgments

We thank the Malagasy Ministry of Agriculture and Livestock, the National Office for Nutrition, and the FAO Madagascar office for providing data and domain expertise\.

## References

- Adadi and Berrada \(2018\)A\. Adadi and M\. BerradaPeeking inside the black\-box: a survey on explainable artificial intelligence \(xai\)\.IEEE Access6,pp\. 52138–52160\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2018.2870052)Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p2.1),[§2](https://arxiv.org/html/2608.23817#S2.p1.1)\.
- Alvarez\-Melis and Jaakkola \(2018\)D\. Alvarez\-Melis and T\. S\. JaakkolaOn the robustness of interpretability methods\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:[Document](https://dx.doi.org/10.48550/arXiv.1806.08049)Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p2.1),[§2](https://arxiv.org/html/2608.23817#S2.p1.1)\.
- Danilevskyet al\.\(2020\)M\. Danilevsky, K\. Qian, R\. Aharonov, Y\. Katsis, B\. Kawas, and P\. SenA survey of the state of explainable ai for natural language processing\.InProceedings of the 1st Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing,pp\. 447–459\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.aacl-main.46)Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p2.1)\.
- Guidottiet al\.\(2018\)R\. Guidottiet al\.A survey of methods for explaining black box models\.ACM Computing Surveys51\(5\),pp\. 1–42\.External Links:[Document](https://dx.doi.org/10.1145/3236009)Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p2.1),[§2](https://arxiv.org/html/2608.23817#S2.p1.1)\.
- Gunning and Aha \(2019\)D\. Gunning and D\. W\. AhaDARPA’s explainable artificial intelligence \(xai\) program\.AI Magazine40\(2\),pp\. 44–58\.External Links:[Document](https://dx.doi.org/10.1609/aimag.v40i2.2850)Cited by:[§2](https://arxiv.org/html/2608.23817#S2.p1.1)\.
- Holzingeret al\.\(2019\)A\. Holzinger, G\. Langs, H\. Denk, K\. Zatloukal, and H\. MüllerCausability and explainability of artificial intelligence in medicine\.WIREs Data Mining and Knowledge Discovery9\(4\),pp\. e1312\.Cited by:[§2](https://arxiv.org/html/2608.23817#S2.p2.1)\.
- Hookeret al\.\(2021\)G\. Hooker, L\. Mentch, and S\. ZhouUnrestricted permutation forces extrapolation: variable importance requires at least one more model, or there is no free variable importance\.Statistics and Computing31\(6\),pp\. 82\.Cited by:[§3\.2](https://arxiv.org/html/2608.23817#S3.SS2.p5.1)\.
- Islamet al\.\(2024\)T\. Islamet al\.Explainable ai for predictive modeling in healthcare\.Smart Health32,pp\. 100465\.External Links:[Document](https://dx.doi.org/10.1016/j.smhl.2024.100465)Cited by:[§2](https://arxiv.org/html/2608.23817#S2.p2.1)\.
- Linheiroet al\.\(2023\)E\. S\. M\. Linheiroet al\.Explainable ai \(xai\) for agriculture\.InIndustry 4\.0 Convergence with AI, IoT, Big Data and Cloud Computing,pp\. 161–176\.External Links:[Document](https://dx.doi.org/10.2174/9789815179187123010013)Cited by:[§2](https://arxiv.org/html/2608.23817#S2.p2.1)\.
- Lundberg and Lee \(2017\)S\. M\. Lundberg and S\. LeeA unified approach to interpreting model predictions\.Advances in Neural Information Processing Systems30,pp\. 4765–4774\.Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p1.1),[§2](https://arxiv.org/html/2608.23817#S2.p1.1)\.
- Ralinirinaet al\.\(2025\)R\. E\. Ralinirina, J\. C\. Ralaivao, N\. M\. Ralaivao, A\. J\. Ratovondrahona, and T\. MahatodyUnveiling the potential of explainable artificial intelligence in predictive modeling, exploring food security and nutrition in madagascar\.InAdvances in ICT Research in the Balkans \(BCI 2024\),Communications in Computer and Information Science, Vol\.2391\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-84093-7%5F3)Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p2.1),[Abstract](https://arxiv.org/html/2608.23817#abstract1.1)\.
- Ratovondrahonaet al\.\(2023\)A\. J\. Ratovondrahona, H\. Rakotozanany, T\. Mahatody, and V\. ManantsoaHuman like programming using SPADE BDI agents and the GPT\-3\-based transformer\.InHuman Interaction and Emerging Technologies \(IHIET\-AI 2023\): Artificial Intelligence and Future Applications,AHFE Open Access, Vol\.70\.External Links:[Document](https://dx.doi.org/10.54941/ahfe1002939)Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p2.1)\.
- Ribeiroet al\.\(2016\)M\. T\. Ribeiro, S\. Singh, and C\. Guestrin”Why should i trust you?”: explaining the predictions of any classifier\.InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,pp\. 1135–1144\.Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p1.1),[§2](https://arxiv.org/html/2608.23817#S2.p1.1)\.
- Sameket al\.\(2017\)W\. Samek, A\. Binder, G\. Montavon, S\. Bach, and K\. MüllerEvaluating the visualization of what a deep neural network has learned\.IEEE Transactions on Neural Networks and Learning Systems28\(11\),pp\. 2660–2673\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2016.2599820)Cited by:[§2](https://arxiv.org/html/2608.23817#S2.p2.1)\.
- Slacket al\.\(2020\)D\. Slacket al\.Fooling lime and shap: adversarial attacks on post hoc explanation methods\.InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society,pp\. 180–186\.External Links:[Document](https://dx.doi.org/10.1145/3375627.3375830)Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p2.1),[§2](https://arxiv.org/html/2608.23817#S2.p1.1)\.
- Stiglicet al\.\(2020\)G\. Stiglicet al\.Interpretability of machine learning\-based prediction models in healthcare\.WIREs Data Mining and Knowledge Discovery10\(5\),pp\. e1379\.External Links:[Document](https://dx.doi.org/10.1002/widm.1379)Cited by:[§1](https://arxiv.org/html/2608.23817#S1.p2.1)\.