Prediction certification cannot replace explanation certification: a competence envelope for trustworthy AI under compound stress

arXiv cs.AI Papers
trustworthy-ai certification explanation prediction competence-envelope ai-safety

Summary

This paper argues that prediction-based certification alone cannot ensure trustworthy AI and proposes a 'competence envelope' framework that integrates explanation certification to detect hidden failures.

arXiv:2608.20825v1 Announce Type: new Abstract: Artificial intelligence systems increasingly make consequential judgments - which patient is deteriorating, which building is safe to enter, whether an image is authentic and are trusted on the strength of how accurately and confidently they predict. The safeguards that certify them are correspondingly prediction-based: accuracy, calibration and conformal coverage all measure how well a model performs. Whether such checks are sufficient to establish model trustworthiness has remained unclear. Here we prove that they cannot. We establish a separation theorem showing that a reliable model and a compromised one can be identical under every prediction-side certificate, including accuracy, calibration and coverage, yet differ arbitrarily in explanation fidelity and deployment behaviour. Detecting this failure requires access to the model's decision mechanism in addition to its predictions. We introduce the competence envelope as an operational framework that combines prediction and explanation certification into a single deployable criterion. Across diverse datasets and model classes, the proposed framework reveals failure modes that prediction-side certification alone does not capture. Certification against failures that are invisible in prediction behaviour therefore requires evidence about the model's decision mechanism as well as its outputs.
Original Article
View Cached Full Text

Cached at: 08/24/26, 04:27 AM 

# Prediction certification cannot replace explanation certification: a competence envelope for trustworthy AI under compound stress
Source: [https://arxiv.org/abs/2608.20825](https://arxiv.org/abs/2608.20825)
[View PDF](https://arxiv.org/pdf/2608.20825)

> Abstract:Artificial intelligence systems increasingly make consequential judgments \- which patient is deteriorating, which building is safe to enter, whether an image is authentic and are trusted on the strength of how accurately and confidently they predict\. The safeguards that certify them are correspondingly prediction\-based: accuracy, calibration and conformal coverage all measure how well a model performs\. Whether such checks are sufficient to establish model trustworthiness has remained unclear\. Here we prove that they cannot\. We establish a separation theorem showing that a reliable model and a compromised one can be identical under every prediction\-side certificate, including accuracy, calibration and coverage, yet differ arbitrarily in explanation fidelity and deployment behaviour\. Detecting this failure requires access to the model's decision mechanism in addition to its predictions\. We introduce the competence envelope as an operational framework that combines prediction and explanation certification into a single deployable criterion\. Across diverse datasets and model classes, the proposed framework reveals failure modes that prediction\-side certification alone does not capture\. Certification against failures that are invisible in prediction behaviour therefore requires evidence about the model's decision mechanism as well as its outputs\.

## Submission history

From: Nataliya Shakhovska Prof \[[view email](https://arxiv.org/show-email/719cfa9a/2608.20825)\] **\[v1\]**Fri, 21 Aug 2026 07:45:17 UTC \(3,204 KB\)

Similar Articles