When Explanations Betray Backdoors: Black-Box Auditing for Language Model Classifiers

# When Explanations Betray Backdoors: Black-Box Auditing for Language Model Classifiers
Source: [https://arxiv.org/html/2608.12623](https://arxiv.org/html/2608.12623)
Yang LiuAffiliation:\[3pt\] Department of Statistics and Operations Research,University of North Carolina at Chapel HillAffiliation:\[2pt\] Equal contribution\.Ran ZouAffiliation:Department of Statistics, University of California, IrvineAffiliation:\[2pt\] Equal contribution\.

###### Abstract

Language model classifiers with explanations are used for moderation, routing, topic triage, and low\-resource annotation\. We study black\-box auditing when the defender has only clean calibration data without trigger information but can ask the classifier for a label plus a short rationale or quoted evidence\. We introduce Groundedness Drift, a lightweight score measuring whether the answer summary remains grounded in the input\. Across two 7B backbones, five datasets, and four common non\-adaptive OpenBackdoor\-style attack families, Groundedness Drift achieves higher AUROC and lower residual target ASR than every compared detector in all cases at a nominal 5% clean\-FPR budget\. We then evaluate Unsupported Groundedness, a multi\-probe escalation for explanation\-camouflage stress cases\. Unsupported Groundedness improves signals but does not close the adaptive gap\.

## 1Introduction

Backdoor attacks are a well\-established security threat to text classification\. A poisoned model can preserve normal behavior and high accuracy on clean inputs while mapping any input containing a hidden lexical, syntactic, or contextual trigger to a label chosen by the attacker\([Qi et al\. 2021b](https://arxiv.org/html/2608.12623#bib.bib20);[Cui et al\. 2022](https://arxiv.org/html/2608.12623#bib.bib3)\)\. This conditional behavior makes backdoors difficult to discover through ordinary held\-out validation: if the unknown trigger never appears in the validation set, the compromised model can appear fully functional\. OpenBackdoor further demonstrates that this threat extends across sentiment, topic, toxicity, and spam classification rather than being confined to one benchmark\([Cui et al\. 2022](https://arxiv.org/html/2608.12623#bib.bib3)\)\. The risk remains relevant as instruction\-tuned language models \(LMs\) are increasingly used as flexible classifier\-like components for safety moderation, policy classification, topic routing, and data annotation\([Mozes et al\. 2023](https://arxiv.org/html/2608.12623#bib.bib17);[Sun et al\. 2023](https://arxiv.org/html/2608.12623#bib.bib24);[Gilardi et al\. 2023](https://arxiv.org/html/2608.12623#bib.bib7)\)\. We do not argue that such models should replace compact supervised classifiers in stable, high\-volume tasks\. Their practical value is in deployments where label definitions change faster than a supervised classifier can be retrained, such as updating moderation policies, adding new support\-routing categories, or bootstrapping an annotation scheme from only a few labeled examples, and where human reviewers need the model to cite the text supporting each decision\. At the same time, instruction tuning and third\-party fine\-tuning introduce additional poisoning surfaces: malicious instructions can implant persistent targeted behavior, and recent benchmarks show that backdoor vulnerabilities extend across multiple LLM architectures and deployment scenarios\([Xu et al\. 2024](https://arxiv.org/html/2608.12623#bib.bib27);[Li et al\. 2025b](https://arxiv.org/html/2608.12623#bib.bib16)\)\.

The LM interface changes both the risk and the opportunity for defense\. A conventional classifier exposes only a hard label, whereas an LM classifier can additionally return a short rationale, a policy\-grounding statement, or an exact evidence quote for downstream review\. These explanations should not be treated as certificates of correctness: generated rationales can be fluent and plausible while failing to reflect the actual cause of a prediction\([Turpin et al\. 2023](https://arxiv.org/html/2608.12623#bib.bib25)\), and backdoored LMs can generate explanations that rationalize outputs controlled by the attacker\([Ge et al\. 2025](https://arxiv.org/html/2608.12623#bib.bib6)\)\. Nevertheless, this additional output creates an observable audit surface that ordinary hard\-label classifiers do not provide\. Rather than assuming that an explanation is faithful, we ask whether its relationship to the input becomes anomalous when a backdoor changes the predicted label\. Specifically, we study the auditing setting in a deployment regime where the defenders screen each incoming prediction individually, have only black\-box query access and a small clean calibration set\. But they do not have any information about the trigger and can request a label together with a rationale or quoted evidence\. Our goal is to determine when a mismatch between the predicted label and an input\-grounded rationale provides a lightweight signal of a backdoor\. When the attacker also manipulates the explanation, we further test whether the quoted evidence supports the label on its own and whether the prediction remains consistent across different parts of the input\.

Our contributions are: \(i\) We study a black\-box deployment setting in which the defender does not know the trigger, has only a small set of benign examples for choosing a rejection threshold, and must decide whether to accept or review each incoming prediction\. \(ii\) We introduce Groundedness Drift, a one\-query audit that measures whether the model’s short rationale remains grounded in the input\. \(iii\) We conduct a systematic comparison against ONION, BBCaL, CoS\-style, and prompted\-reasoning baselines at matched false\-positive budget on benign inputs\. \(iv\) We evaluate attackers that manipulate both labels and explanations and introduce Unsupported Groundedness, an escalation procedure that checks whether quoted evidence supports the prediction and whether the decision remains consistent across different parts of the input\. It recovers some detection signal but does not close the adaptive gap\.

## 2Related Work

#### Textual backdoor detection\.

Textual backdoors can be instantiated through token or phrase insertions, syntactic transformations, and stylistic shifts\. OpenBackdoor provides a unified benchmark for evaluating such attacks and defenses\([Qi et al\. 2021b](https://arxiv.org/html/2608.12623#bib.bib20);[Cui et al\. 2022](https://arxiv.org/html/2608.12623#bib.bib3)\)\. Existing defenses differ substantially in their unit of detection and model\-access assumptions\. ONION identifies suspicious tokens from the change in language\-model perplexity after token removal\([Qi et al\. 2021a](https://arxiv.org/html/2608.12623#bib.bib19)\), whereas BBCaL detects individual backdoored inputs by measuring black\-box prediction consistency under progressively constructed counterfactuals\([Hu et al\. 2024](https://arxiv.org/html/2608.12623#bib.bib9)\)\. Chain\-of\-Scrutiny \(CoS\) instead elicits a reasoning trace and tests whether the trace supports the model’s final output\([Li et al\. 2025a](https://arxiv.org/html/2608.12623#bib.bib14)\)\. These approaches are closest to our input\-level setting, but they are different\.

We also compare to prompted\-reasoning detector variants built from Reasoning, ZS\-CoT, and Auto\-CoT prompts\([Wei et al\. 2022](https://arxiv.org/html/2608.12623#bib.bib26);[Kojima et al\. 2022](https://arxiv.org/html/2608.12623#bib.bib12);[Zhang et al\. 2023](https://arxiv.org/html/2608.12623#bib.bib31)\): the prompt elicits a rationale, then the same consistency judge used for CoS assigns a rejection score\. This tests whether generic reasoning elicitation is enough, versus explicitly measuring input grounding\. Furthermore, recent work distinguishes plausible explanations from faithful evidence\([Lei et al\. 2016](https://arxiv.org/html/2608.12623#bib.bib13);[Jacovi and Goldberg 2020](https://arxiv.org/html/2608.12623#bib.bib10);[DeYoung et al\. 2020](https://arxiv.org/html/2608.12623#bib.bib5)\)\. These motivate us to study whether the rationale behavior can become a signal for backdoor detection\. Finally, we evaluate the defense performance under a fixed false positive rate budget on clean data\.

## 3Threat Model and Task

#### Notation\.

We writexxfor an input text,𝒴\\mathcal\{Y\}for the finite label set,yty\_\{t\}for the attack target label, andfθf\_\{\\theta\}for the black\-box LM classifier\. Whenfθf\_\{\\theta\}appears with a raw text argument in a metric or probe, it denotes the label predicted by the model under the corresponding classification prompt\. The defender’s clean calibration set is𝒞=\{\(xi,yi\)\}i=1n\\mathcal\{C\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}, whiley~i\\tilde\{y\}\_\{i\}denotes the label predicted byfθf\_\{\\theta\}on calibration inputxix\_\{i\}under the relevant audit prompt\. The notation𝕀⁡\[⋅\]\\mathbb\{I\}\[\\cdot\]is the indicator function: it equals 1 when the condition inside brackets is true and 0 otherwise\. For a finite set of calibration scoresSScalculated by the algorithm,Q1−α​\(S\)Q\_\{1\-\\alpha\}\(S\)denotes the empirical upper\(1−α\)\(1\-\\alpha\)quantile used to target false positive rate budgetα\\alphaon clean data \(clean\-FPR\)\. For Unsupported Groundedness,β\\betadenotes the residual clean\-FPR budget andF^j,y\\widehat\{F\}\_\{j,y\}is the empirical CDF of component scoreDjD\_\{j\}on clean calibration examples assigned to classyy\.

#### Victim model and audit access\.

We study an instruction\-tuned LM deployed for a closed\-label classification task\. The service is classifier\-like in its decision space but exposes an LM interface: under a fixed audit prompt, it returns a label together with either a short rationale or exact spans quoted from the input\. Such outputs support workflows in which predictions are routed to human review\. The defender may issue these prompts to the same hosted model but cannot inspect its weights, logits, training data, poison rate, attack target, or trigger\. Our setting therefore excludes hard\-label\-only endpoints\. The main benchmark uses the standard label\-targeted poisoning; the separately marked adaptive stress test jointly targets the predicted label and the explanation channel as if the attacker knows our defense algorithms\.

#### Attacker\.

The attacker poisons the victim during fine\-tuning so that inputs containing a hidden trigger are mapped to a target labelyty\_\{t\}, while clean\-task accuracy remains high\. We evaluate four OpenBackdoor\-style families, with family names and trigger templates derived from the corresponding original attacks: BadNets lexical triggers\([Gu et al\. 2017](https://arxiv.org/html/2608.12623#bib.bib8)\), AddSent trigger\-sentence insertion\([Dai et al\. 2019](https://arxiv.org/html/2608.12623#bib.bib4)\), EP rare\-token/embedding\-poisoning triggers\([Yang et al\. 2021a](https://arxiv.org/html/2608.12623#bib.bib28)\), and SOS stealth\-oriented contextual triggers\([Yang et al\. 2021b](https://arxiv.org/html/2608.12623#bib.bib29)\), following the OpenBackdoor taxonomy\([Cui et al\. 2022](https://arxiv.org/html/2608.12623#bib.bib3)\)\. The attack target is fixed per run, but the defender does not know it\.

Our main threat model is non\-adaptive, meaning that during the fine\-tuning process, the attacker did not deliberately target our algorithm to specifically train explanations for the ground input\. This reflects the standard application cases\. The attacker trains label\-targeted backdoors using normal ways without knowing the defense algorithms\. This matches standard OpenBackdoor\-style label\-poisoning evaluations\. We therefore interpret the main results as an evaluation of non\-adaptive auditability, not as an adaptive security guarantee\. Adaptive explanation\-camouflage results are marked explicitly and are not mixed into the main baseline table\.

#### Defender\.

The defender has a clean calibration set𝒞=\{\(xi,yi\)\}i=1n\\mathcal\{C\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}from the deployment task\. In all main experiments,n=256n=256\. Gold labels are used only for offline validation, not for online threshold lookup\. In deployment, the auditor receives an input, the victim model’s predicted label, and the generated rationale\. It does not know the input’s true label\. We therefore calibrate thresholds in the standard applied regime\. For each clean calibration examplexix\_\{i\}, we query the victim model, record its predicted labely^i=fθ​\(xi\)\\hat\{y\}\_\{i\}=f\_\{\\theta\}\(x\_\{i\}\), compute the audit scoreDA​\(xi\)D\_\{A\}\(x\_\{i\}\), and estimate a separate threshold for each predicted\-label bucket:

τA\(y\)=Q1−α\{DA\(xi\):y^i=y,xi∈𝒞\}\.\\tau\_\{A\}\(y\)=Q\_\{1\-\\alpha\}\\\{D\_\{A\}\(x\_\{i\}\):\\hat\{y\}\_\{i\}=y,\\ x\_\{i\}\\in\\mathcal\{C\}\\\}\.At test time, a new inputxxis rejected whenDA​\(x\)\>τA​\(fθ​\(x\)\)D\_\{A\}\(x\)\>\\tau\_\{A\}\(f\_\{\\theta\}\(x\)\)\. This asks whether the rationale looks anomalous relative to clean examples for which the victim made the same prediction\. Gold labels are still important: they ensure that the calibration set is benign, allow us to report clean task accuracy, and let us check that the victim classifier is meaningful before auditing it\. If the victim misclassifies a clean calibration example, we retain it and assign it to the victim\-predicted bucket\. This makes the threshold conservative to ordinary clean\-model mistakes rather than treating every clean misclassification as a backdoor anomaly\.

#### Decision and metrics\.

For each inference input, an auditor outputsa⁡\(x\)∈\{0,1\}a\(x\)\\in\\\{0,1\\\}, wherea⁡\(x\)=1a\(x\)=1means reject or send to review anda⁡\(x\)=0a\(x\)=0means accept\. Let𝒟clean\\mathcal\{D\}\_\{\\mathrm\{clean\}\}be the benign evaluation set and𝒟atk\\mathcal\{D\}\_\{\\mathrm\{atk\}\}the triggered evaluation set\. We report AUROC \(Area Under the Receiver Operating Characteristic\) for ranking benign clean inputs against triggered inputs by detector score\. We also report realized clean false\-positive rate

FPRclean=1\|𝒟clean\|∑x∈𝒟clean𝕀\[a\(x\)=1\],\\mathrm\{FPR\}\_\{\\mathrm\{clean\}\}=\\frac\{1\}\{\|\\mathcal\{D\}\_\{\\mathrm\{clean\}\}\|\}\\sum\_\{x\\in\\mathcal\{D\}\_\{\\mathrm\{clean\}\}\}\\mathbb\{I\}\[a\(x\)=1\],for held\-out clean test data to see whether the chosen threshold based on calibration data actually matches the FPR budget in another clean data\. Then, we report pre\-filter target attack success rate

preASR=1\|𝒟atk\|∑x∈𝒟atk𝕀\[fθ\(x\)=yt\],\\mathrm\{preASR\}=\\frac\{1\}\{\|\\mathcal\{D\}\_\{\\mathrm\{atk\}\}\|\}\\sum\_\{x\\in\\mathcal\{D\}\_\{\\mathrm\{atk\}\}\}\\mathbb\{I\}\[f\_\{\\theta\}\(x\)=y\_\{t\}\],and residual target attack success rate

rASR=1\|𝒟atk\|∑x∈𝒟atk𝕀\[fθ\(x\)=yt\]𝕀\[a\(x\)=0\]\.\\mathrm\{rASR\}=\\frac\{1\}\{\|\\mathcal\{D\}\_\{\\mathrm\{atk\}\}\|\}\\sum\_\{x\\in\\mathcal\{D\}\_\{\\mathrm\{atk\}\}\}\\mathbb\{I\}\[f\_\{\\theta\}\(x\)=y\_\{t\}\]\\mathbb\{I\}\[a\(x\)=0\]\.preASR is used to judge whether the attack is successful, and rASR is used to check the defense power for the algorithm\. Higher AUROC, lower clean FPR, and lower rASR are better\. rASR is the main safety metric because it measures target\-label attacks that pass through the audit\.

#### Positioning\.

Appendix Table[3](https://arxiv.org/html/2608.12623#A1.T3)compares access assumptions\. AUROC is threshold\-free; the matched clean\-FPR results compare residual risk under a common review budget\. Baseline cutoffs are selected posthoc from held\-out benign scores, whereas our deployable thresholds use the separate clean calibration set\. We therefore interpret matched\-FPR results as a risk\-coverage diagnostic, not as a deployment protocol for every baseline\.

## 4Auditing Algorithms

Figure[1](https://arxiv.org/html/2608.12623#S4.F1)summarizes the two audit policies, which are formalized in Algorithms[1](https://arxiv.org/html/2608.12623#alg1)and[2](https://arxiv.org/html/2608.12623#alg2)\.

Figure 1:Audit workflow\. Groundedness Drift uses one victim query and an externally computed grounding score\. Unsupported Groundedness is an optional multi\-probe escalation for inputs accepted by Groundedness Drift\. Both decisions use thresholds estimated from clean calibration data\.We use two fixed interfaces:pAp\_\{\\mathrm\{A\}\}requests a closed\-label prediction and short summary, whilepBp\_\{\\mathrm\{B\}\}requests a label, exact input quotes, and a contrastive explanation\. Appendix[C](https://arxiv.org/html/2608.12623#A3)gives the prompt skeletons\.

Groundedness Drift \(Algorithm[1](https://arxiv.org/html/2608.12623#alg1)\) is the default one\-query audit, rejecting summaries with unusually weak input grounding relative to clean calibration\. Unsupported Groundedness \(Algorithm[2](https://arxiv.org/html/2608.12623#alg2)\) conditionally applies quoted\-evidence, evidence\-removal, and local\-view probes only after Groundedness Drift accepts\. This sequence saves queries and avoids dataset\-specific fusion weights\. We therefore use Groundedness Drift for the non\-adaptive benchmark and reserve Unsupported Groundedness for the AG News explanation\-camouflage stress test\.

Algorithm 1Groundedness Drift1:LM classifier

fθf\_\{\\theta\}, clean calibration set

𝒞\\mathcal\{C\}, test input

xx, clean\-FPR budget

α\\alpha
2:Query each calibration input with the summary\-rationale prompt

pA​\(xi\)p\_\{\\mathrm\{A\}\}\(x\_\{i\}\)
3:Compute grounding drift

DA​\(xi\)D\_\{\\mathrm\{A\}\}\(x\_\{i\}\)and victim\-predicted label

y~i\\tilde\{y\}\_\{i\}
4:Set

τA​\(y\)=Q1−α​\{DA​\(xi\):y~i=y\}\\tau\_\{\\mathrm\{A\}\}\(y\)=Q\_\{1\-\\alpha\}\\\{D\_\{\\mathrm\{A\}\}\(x\_\{i\}\):\\tilde\{y\}\_\{i\}=y\\\}, using a global fallback for sparse classes

5:Query the test input with

pA​\(x\)p\_\{\\mathrm\{A\}\}\(x\)to obtain

\(yA,rA\)\(y\_\{\\mathrm\{A\}\},r\_\{\\mathrm\{A\}\}\)
6:Compute

DA​\(x\)D\_\{\\mathrm\{A\}\}\(x\)from lexical and TF\-IDF grounding of

rAr\_\{\\mathrm\{A\}\}against the original input text

7:if

DA​\(x\)\>τA​\(yA\)D\_\{\\mathrm\{A\}\}\(x\)\>\\tau\_\{\\mathrm\{A\}\}\(y\_\{\\mathrm\{A\}\}\)then

8:returnreject

9:else

10:returnaccept

11:endif

Algorithm 2Unsupported Groundedness1:LM classifier

fθf\_\{\\theta\}, clean calibration set

𝒞\\mathcal\{C\}, test input

xx, clean\-FPR budget

α\\alpha, residual clean\-FPR budget

β\\beta
2:Run Algorithm[1](https://arxiv.org/html/2608.12623#alg1)

3:ifGroundedness Drift rejects

xxthen

4:returnreject

5:endif

6:Query the quoted\-evidence prompt

pB​\(x\)p\_\{\\mathrm\{B\}\}\(x\); validate at most

mmax=2m\_\{\\max\}=2exact quote spans from the original input

7:Query evidence\-only spans, evidence\-keep text, evidence\-drop text, and at most

kmax=5k\_\{\\max\}=5local document views

8:Compute support\-consistency drift

DsupD\_\{\\mathrm\{sup\}\}and document\-consistency drift

DdocD\_\{\\mathrm\{doc\}\}
9:Convert each component to a class\-conditional clean percentile using the Groundedness Drift\-accepted calibration pool for label

yBy\_\{\\mathrm\{B\}\}
10:Set

DB​\(x\)=max⁡\{psup​\(x\),pdoc​\(x\)\}D\_\{\\mathrm\{B\}\}\(x\)=\\max\\\{p\_\{\\mathrm\{sup\}\}\(x\),p\_\{\\mathrm\{doc\}\}\(x\)\\\}
11:Set

τB​\(yB\)\\tau\_\{\\mathrm\{B\}\}\(y\_\{\\mathrm\{B\}\}\)as the

\(1−β\)\(1\-\\beta\)empirical quantile of fused scores on that same clean pool

12:if

DB​\(x\)\>τB​\(yB\)D\_\{\\mathrm\{B\}\}\(x\)\>\\tau\_\{\\mathrm\{B\}\}\(y\_\{\\mathrm\{B\}\}\)then

13:returnreject

14:else

15:returnaccept

16:endif

### 4\.1Groundedness Drift

We first query the victim with the summary\-rationale prompt\. LetpA​\(x\)p\_\{\\mathrm\{A\}\}\(x\)denote the literal prompt string containing the task instruction, label set, and input text:

\(yA,rA\)=fθ​\(pA​\(x\)\),\(y\_\{\\mathrm\{A\}\},r\_\{\\mathrm\{A\}\}\)=f\_\{\\theta\}\(p\_\{\\mathrm\{A\}\}\(x\)\),whereyAy\_\{\\mathrm\{A\}\}is the predicted label andrAr\_\{\\mathrm\{A\}\}is the generated short answer summary\. Groundedness Drift measures whether the summary appears to be grounded in the input:

DA\(x\)=1−max\{\\displaystyle D\_\{\\mathrm\{A\}\}\(x\)=1\-\\max\\\{Glex​\(x,rA\),\\displaystyle G\_\{\\mathrm\{lex\}\}\(x,r\_\{\\mathrm\{A\}\}\),Gtfidf\(x,rA\)\}\.\\displaystyle G\_\{\\mathrm\{tfidf\}\}\(x,r\_\{\\mathrm\{A\}\}\)\\\}\.GlexG\_\{\\mathrm\{lex\}\}is the fraction of non\-stopword content tokens in the summary that also occur in the original input text after lowercasing and punctuation stripping, excluding a small fixed list of generic rationale words\.GtfidfG\_\{\\mathrm\{tfidf\}\}is the maximum TF\-IDF cosine similarity between the summary and any clause of the original input text split on punctuation and contrastive connectives\. We intentionally exclude task instructions, label names, and response\-schema text from this grounding computation\. This is deliberately a lightweight lexical grounding score, not a learned semantic entailment model\. Paraphrase\-aware encoders are a useful ablation but introduce an auxiliary verifier outside the black\-box victim interface\. LargerDAD\_\{\\mathrm\{A\}\}indicates greater drift\. For a calibration examplexix\_\{i\}, lety~i\\tilde\{y\}\_\{i\}be the victim\-predicted summary\-rationale label\. We set a class\-conditional threshold

τA\(y\)=Q1−α\{DA\(xi\):xi∈𝒞,y~i=y\},\\tau\_\{\\mathrm\{A\}\}\(y\)=Q\_\{1\-\\alpha\}\\\{D\_\{\\mathrm\{A\}\}\(x\_\{i\}\):x\_\{i\}\\in\\mathcal\{C\},\\tilde\{y\}\_\{i\}=y\\\},with a global fallback when a class has too few calibration examples\. Inputs withDA​\(x\)\>τA​\(yA\)D\_\{\\mathrm\{A\}\}\(x\)\>\\tau\_\{\\mathrm\{A\}\}\(y\_\{\\mathrm\{A\}\}\)are rejected\.

### 4\.2Unsupported Groundedness

If Groundedness Drift accepts the input and we suspect that the attacker modified the rationale to escape from the Groundedness Drift, Unsupported Groundedness first issues the quoted\-evidence promptpBp\_\{\\mathrm\{B\}\}:

\(yB,E\)=fθ​\(pB​\(x\)\),\(y\_\{\\mathrm\{B\}\},E\)=f\_\{\\theta\}\(p\_\{\\mathrm\{B\}\}\(x\)\),wherepB​\(x\)p\_\{\\mathrm\{B\}\}\(x\)is the literal quoted\-evidence prompt defined above andE=\{e1,…,em\}E=\\\{e\_\{1\},\\ldots,e\_\{m\}\\\}are short quoted spans\. This equation describes only the model\-generated evidence channel\. Unsupported Groundedness also deterministically constructs local document views from the original input,

C⁡\(x\)=\{c1,…,ck\},C\(x\)=\\\{c\_\{1\},\\ldots,c\_\{k\}\\\},such as title, lead, or sentence chunks when available\. The residual audit is therefore a function of both the quoted evidence and the document views:

DB​\(x\)=Φ⁡\(x,yB,E,C⁡\(x\)\)\.D\_\{\\mathrm\{B\}\}\(x\)=\\Phi\\\!\\left\(x,y\_\{\\mathrm\{B\}\},E,C\(x\)\\right\)\.We validate each quote as an exact substring of the original input and drop schema placeholders or malformed fields\. We use "support" operationally: evidence supports a label if the same black\-box victim classifier, when restricted to the quoted evidence or local document views, continues to prefer that label under clean\-calibrated thresholds\. This is a victim self\-consistency proxy\. The following three short paragraphs describe the components of the score calculation in Unsupported Groundedness and how the threshold is determined\. If either the evidence\-support probe or the document\-view probe shows extreme signal after calibration, the sample will be flagged by the algorithm\.

#### Evidence\-support consistency\.

For each valid evidence spaneje\_\{j\}, we re\-query the model on only the span text and obtain a label votey^​\(ej\)\\hat\{y\}\(e\_\{j\}\)\. Let

Vℓ\(E\)=1m∑j=1m𝕀\[y^\(ej\)=ℓ\]\.V\_\{\\ell\}\(E\)=\\frac\{1\}\{m\}\\sum\_\{j=1\}^\{m\}\\mathbb\{I\}\[\\hat\{y\}\(e\_\{j\}\)=\\ell\]\.The consistency drift score is

Dsup​\(x\)=maxℓ≠yB⁡Vℓ​\(E\)−VyB​\(E\),D\_\{\\mathrm\{sup\}\}\(x\)=\\max\_\{\\ell\\neq y\_\{\\mathrm\{B\}\}\}V\_\{\\ell\}\(E\)\-V\_\{y\_\{\\mathrm\{B\}\}\}\(E\),with missing or invalid evidence assigned high drift\. This distinguishes evidence that is merely copied from the input from evidence that remains label\-consistent under victim self\-probes\.

#### Support and document probes\.

The second family of probes asks whether the quoted evidence and local document views agree with the quoted\-evidence labelyBy\_\{\\mathrm\{B\}\}\. We query the model onxkeepx\_\{\\mathrm\{keep\}\}, the concatenation of valid evidence spans, andxdropx\_\{\\mathrm\{drop\}\}, the original input with those spans removed\. Their indicator values are

Ikeep\\displaystyle I\_\{\\mathrm\{keep\}\}=𝕀\[fθ\(xkeep\)≠yB\],\\displaystyle=\\mathbb\{I\}\[f\_\{\\theta\}\(x\_\{\\mathrm\{keep\}\}\)\\neq y\_\{\\mathrm\{B\}\}\],Idrop\\displaystyle I\_\{\\mathrm\{drop\}\}=𝕀\[fθ\(xdrop\)=yB\]\.\\displaystyle=\\mathbb\{I\}\[f\_\{\\theta\}\(x\_\{\\mathrm\{drop\}\}\)=y\_\{\\mathrm\{B\}\}\]\.The first indicator asks whether the quoted evidence is sufficient under victim self\-probing; the second asks whether deleting that evidence leaves the same label intact\. For longer inputs, we also query the local viewsC⁡\(x\)=\{c1,…,ck\}C\(x\)=\\\{c\_\{1\},\\ldots,c\_\{k\}\\\}\. In AG News, these are the title field when present, the lead sentence, and consecutive body sentence chunks up tokmax=5k\_\{\\max\}=5total local views\. Missing views are omitted\. This construction is fixed before seeing attack examples\. We combine the keep/drop and local\-view probes into

𝒰\(x,E\)=\{𝕀\[fθ\(cj\)≠yB\]\}j=1k∪\{Ikeep,Idrop\}\.\\mathcal\{U\}\(x,E\)=\\\{\\,\\mathbb\{I\}\[f\_\{\\theta\}\(c\_\{j\}\)\\neq y\_\{\\mathrm\{B\}\}\]\\,\\\}\_\{j=1\}^\{k\}\\cup\\\{I\_\{\\mathrm\{keep\}\},I\_\{\\mathrm\{drop\}\}\\\}\.The document drift score is

Ddoc​\(x\)=1\|𝒰⁡\(x,E\)\|​∑u∈𝒰⁡\(x,E\)u\.D\_\{\\mathrm\{doc\}\}\(x\)=\\frac\{1\}\{\|\\mathcal\{U\}\(x,E\)\|\}\\sum\_\{u\\in\\mathcal\{U\}\(x,E\)\}u\.This score is useful for tasks where a short trigger may move the full\-document label while most local views remain label\-consistent with the original topic\.

#### Clean\-calibrated fusion\.

The fusion rule is calibrated only on clean examples that would actually reach Unsupported Groundedness\. LetaA​\(xi\)=0a\_\{\\mathrm\{A\}\}\(x\_\{i\}\)=0mean that Groundedness Drift accepts calibration examplexix\_\{i\}, and letyi,By\_\{i,\\mathrm\{B\}\}be the label returned by the quoted\-evidence prompt on that calibration example\. For each labelyy, define the residual clean pool

𝒞B\(y\)=\{xi∈𝒞:aA\(xi\)=0,yi,B=y\}\.\\mathcal\{C\}\_\{\\mathrm\{B\}\}\(y\)=\\\{\\,x\_\{i\}\\in\\mathcal\{C\}:a\_\{\\mathrm\{A\}\}\(x\_\{i\}\)=0,\\ y\_\{i,\\mathrm\{B\}\}=y\\,\\\}\.For each componentj∈\{sup,doc\}j\\in\\\{\\mathrm\{sup\},\\mathrm\{doc\}\\\}, we build an empirical CDF from the component values on this pool:

F^j,y\(t\)=1\+\|\{xi∈𝒞B​\(y\):Dj​\(xi\)≤t\}\|1\+\|𝒞B​\(y\)\|\.\\widehat\{F\}\_\{j,y\}\(t\)=\\frac\{1\+\\left\|\\\{x\_\{i\}\\in\\mathcal\{C\}\_\{\\mathrm\{B\}\}\(y\):D\_\{j\}\(x\_\{i\}\)\\leq t\\\}\\right\|\}\{1\+\\left\|\\mathcal\{C\}\_\{\\mathrm\{B\}\}\(y\)\\right\|\}\.The add\-one terms make the percentile well\-defined for small pools\. If𝒞B​\(y\)\\mathcal\{C\}\_\{\\mathrm\{B\}\}\(y\)is too small, we use the same construction on the global Groundedness Drift\-accepted clean pool\. For a test input with quoted\-evidence labelyBy\_\{\\mathrm\{B\}\}, the component percentiles and fused score are

pj​\(x\)\\displaystyle p\_\{j\}\(x\)=F^j,yB​\(Dj​\(x\)\),\\displaystyle=\\widehat\{F\}\_\{j,y\_\{\\mathrm\{B\}\}\}\(D\_\{j\}\(x\)\),DB​\(x\)\\displaystyle D\_\{\\mathrm\{B\}\}\(x\)=max⁡\{psup​\(x\),pdoc​\(x\)\}\.\\displaystyle=\\max\\\{p\_\{\\mathrm\{sup\}\}\(x\),p\_\{\\mathrm\{doc\}\}\(x\)\\\}\.Finally, for each labelyywe compute the fused scoresDB​\(xi\)D\_\{\\mathrm\{B\}\}\(x\_\{i\}\)for allxi∈𝒞B​\(y\)x\_\{i\}\\in\\mathcal\{C\}\_\{\\mathrm\{B\}\}\(y\)and set

τB​\(y\)=Q1−β​\{DB​\(xi\):xi∈𝒞B​\(y\)\},\\tau\_\{\\mathrm\{B\}\}\(y\)=Q\_\{1\-\\beta\}\\\{\\,D\_\{\\mathrm\{B\}\}\(x\_\{i\}\):x\_\{i\}\\in\\mathcal\{C\}\_\{\\mathrm\{B\}\}\(y\)\\,\\\},again with the global fallback for sparse classes\. The Unsupported Groundedness policy rejects an input if Groundedness Drift rejects it, or if Groundedness Drift accepts it andDB​\(x\)\>τB​\(yB\)D\_\{\\mathrm\{B\}\}\(x\)\>\\tau\_\{\\mathrm\{B\}\}\(y\_\{\\mathrm\{B\}\}\)\. Unsupported Groundedness probes are not needed for examples already rejected by Groundedness Drift, but examples that enter Unsupported Groundedness may require multiple model calls\.

Table 1:Cross\-backbone non\-adaptive comparison at a nominal 5% clean\-FPR budget\. Cells report AUROC / residual target ASR / realized clean FPR \(%\)\. Both panels report arithmetic means over three training seeds\.

## 5Experiments

#### Datasets and attacks\.

We evaluate SST\-2\([Socher et al\. 2013](https://arxiv.org/html/2608.12623#bib.bib23)\), Rotten Tomatoes\([Pang and Lee 2005](https://arxiv.org/html/2608.12623#bib.bib18)\), TREC\([Li and Roth 2002](https://arxiv.org/html/2608.12623#bib.bib15)\), Jigsaw toxicity\([Borkan et al\. 2019](https://arxiv.org/html/2608.12623#bib.bib1)\), and AG News\([Zhang et al\. 2015](https://arxiv.org/html/2608.12623#bib.bib30)\), spanning sentiment, question type, toxicity, and topic classification\. For each task, Qwen2\.5\-7B\-Instruct\([Qwen Team 2024](https://arxiv.org/html/2608.12623#bib.bib21)\)and Mistral\-7B\-Instruct\-v0\.2\([Jiang et al\. 2023](https://arxiv.org/html/2608.12623#bib.bib11)\)are poisoned with BadNets, AddSent, EP, or SOS\-style triggers following OpenBackdoor\([Cui et al\. 2022](https://arxiv.org/html/2608.12623#bib.bib3)\)\. Main results average three seeds; each run has 256 clean calibration, 200 clean evaluation, and 300 triggered examples\. The main attacks target labels only, while the separate AG News stress test targets both labels and grounded explanations\. Appendix Table[4](https://arxiv.org/html/2608.12623#A1.T4)summarizes the protocol, and Appendix[D](https://arxiv.org/html/2608.12623#A4)gives textual attack examples\.

#### Victim validity\.

Appendix Table[5](https://arxiv.org/html/2608.12623#A2.T5)verifies usable victims and implanted attacks: clean accuracy is 0\.840–0\.968 and pre\-filter target ASR is at least 0\.987 across the main matrix\.

#### Baselines\.

We compare three adjacent signals: ONION estimates token anomaly from likelihood changes, BBCaL probes hard\-label stability under counterfactual edits, and CoS tests agreement between generated reasoning and the final answer\. We also instantiate Reasoning, ZS\-CoT, and Auto\-CoT prompts; each elicits a rationale and label before the same CoS\-adapted scrutiny prompt assigns an inconsistency score\. Thus the comparison spans input anomaly, hard\-label behavior, and output\-side reasoning\. The main table reports classifier\-adapted CoS and, as Reasoning\-best, the prompt variant with the lowest cell\-wise rASR together with its AUROC/FPR\. Appendix[H](https://arxiv.org/html/2608.12623#A8)reports CoS\-original and every individual variant\.

#### Operating points\.

Our methods use only clean calibration data for deployable thresholds\. For matched\-risk baseline diagnostics, we select posthoc cutoffs on held\-out benign scores at a nominal 5% clean\-FPR budget\. This budget is the intended review rate; realized clean FPR is the measured rejection rate on benign evaluation inputs\. Comparing rASR under this common budget prevents a detector from appearing safer merely by rejecting more clean traffic\. We reject only scores strictly above the cutoff, so tied discrete scores can yield less than 5% realized FPR\.

## 6Results

### 6\.1Algorithm 1 Catches Common Non\-Adaptive Backdoors

Table[1](https://arxiv.org/html/2608.12623#S4.T1)compares detectors at a nominal 5% clean\-FPR budget\. On Qwen, Groundedness Drift reduces rASR to near zero on SST\-2, Rotten Tomatoes, TREC, and Jigsaw and remains strong on three AG News families\. Mistral reproduces the broad pattern and also detects AG News SOS \(0\.996 AUROC; 0\.010 rASR\)\. The result therefore extends beyond sentiment and is not tied to one victim backbone\. Across all Groundedness Drift cells, realized clean FPR is at most 3\.7%, ruling out indiscriminate rejection\. BBCaL often realizes 0% because its discrete scores tie at the strict cutoff\. The weak CoS and Reasoning\-best results show that generic rationale elicitation is insufficient; the useful signal is input grounding\. Qwen AG News SOS is the principal non\-adaptive exception, but its disappearance on Mistral indicates victim\-specific rationale behavior rather than a universal failure of contextual triggers\.

Main results use greedy decoding; Appendix[F](https://arxiv.org/html/2608.12623#A6)shows that repeated sampled rationales can improve detection at additional query cost\.

### 6\.2Algorithm 2 Targets Adaptive Explanation Camouflage

We next target the explanation channel on AG News, where diffuse topic evidence can make fluent triggers locally plausible\. Unlike the main attacks, this stressor jointly supervises the target label and a grounded\-looking explanation\. Table[2](https://arxiv.org/html/2608.12623#S6.T2)reports it separately from the non\-adaptive matrix\. Explanation camouflage weakens Groundedness Drift, while Unsupported Groundedness improves AUROC for every family and lowers rASR at both diagnostic budgets\.

Table 2:AG News adaptive explanation\-camouflage stress diagnostic for Qwen\. Cells are AUROC / rASR @10% / rASR @20%\.The gain is partial: rASR remains 0\.462–0\.633 at 10% clean FPR and 0\.434–0\.506 at 20%\. These operating points may support risk\-based human review, but the remaining attack success is too high for an automatic firewall\. Unsupported Groundedness therefore recovers useful signal without closing the adaptive gap\.

### 6\.3Boundary Analysis

The main matrix supports a conditional mechanism: Groundedness Drift is strongest when labels should be justified by short, inspectable evidence, yet transfers to topic classification in most settings\. Qwen’s AG News SOS exception disappears on Mistral, indicating backbone\-dependent explanation behavior rather than a universal failure on contextual triggers or long documents\.

The sharper boundary is explanation camouflage\. A model can produce grounded, locally plausible text while a trigger drives its label; an exact quote may even contain the adversarial sentence itself\. Improvements across all four families show that support/document probes capture structure beyond parsing validity\. Yet self\-verification by the compromised victim cannot establish that quoted evidence caused the prediction\. Appendix[G](https://arxiv.org/html/2608.12623#A7)analyzes quote validity, residual failures, and query cost\. Accordingly, Unsupported Groundedness is a review escalation signal rather than a certificate of safety\.

## 7Conclusion

Explanation\-bearing LM classifiers expose an audit surface unavailable from hard labels alone\. Across two backbones, five tasks, and four non\-adaptive attack families, Groundedness Drift provides a strong one\-query signal without trigger examples, logits, or model internals: label\-targeted poisoning often changes the prediction without producing a correspondingly grounded summary\. Explanation camouflage marks the boundary\. There, Unsupported Groundedness recovers part of the lost signal through evidence and document consistency, but cannot eliminate residual risk\.

The resulting lesson is that explanations are useful traces, not certificates\. Auditing should distinguish whether an explanation is grounded in the input, whether that text operationally supports the label, and whether it actually caused the decision\. Our results find strong signal at the first level, partial signal at the second, and an open problem at the third\. In deployment, Groundedness Drift can route suspicious inputs for review, whereas explanation\-aware threats require escalation, human oversight, or an independent verifier\.

This distinction also clarifies the intended deployment scope\. When an existing classifier already returns reviewable rationales, Groundedness Drift adds a low\-cost audit without requiring a second model\. The multi\-probe policy is justified only for residual or higher\-risk traffic because it incurs additional calls and still relies on the victim’s behavior\. Future defenses should therefore strengthen causal or independent evidence verification rather than treating more elaborate explanations as inherently trustworthy\.

More broadly, our findings support a layered audit rather than a universal detector: grounding screens routine traffic, while support and counterfactual probes examine higher\-risk inputs\. Self\-consistency remains evidence for review, not proof of faithfulness\. The open challenge is causally sensitive verification that preserves black\-box access without requiring a costly trusted model\.

## Limitations

The study uses five public main datasets and four attack families\. The results should therefore not be read as a universal guarantee for all LLM backdoors or open\-ended generation systems\. Testing more baselines is an important next step\. The AG News explanation\-camouflage study is an adaptive\-style stress diagnostic showing that Algorithm[2](https://arxiv.org/html/2608.12623#alg2)helps but does not close the adaptive gap\. The audit also assumes a small clean calibration set and a victim endpoint that can be queried for answer summaries or quoted evidence, which matches LM classifier deployments but not ordinary hard\-label classifiers\. Unsupported Groundedness uses the compromised model as its own verifier\. Using another model may help but will increase the cost\. Our setting targets more on the practically common case where only the third\-party model and a small clean calibration set are available\. Finally, the multi\-probe policy increases query cost for Groundedness Drift\-accepted examples, which we report in Appendix[G](https://arxiv.org/html/2608.12623#A7)\.

## Ethical considerations

This work studies defenses against backdoored text classifiers\. We use standard benchmark\-style attacks and report adaptive stress cases to avoid overstating safety\. The proposed audits are intended as review\-routing signals, not as stand\-alone content\-blocking systems\. Operators could misuse the method if they treat its outputs as ground\-truth security labels\. A false positive delays a benign item and consumes review capacity\. If the method were used for automatic blocking, the same error could suppress benign speech or deny access to a service\. A false negative instead allows a triggered moderation, routing, or annotation decision to pass unchecked\. Moreover, an attacker with access to audit prompts or scores could adapt explanations to evade review\. Human review and appeal, periodic recalibration, and restricted access to audit scores are example safeguards\.

## Acknowledgements

We used GPT\-5\.5 assistance for language editing of the paper\. We thank Yao Li for comments on the topic\.

## References

- Borkan et al\. \(2019\)Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman\. 2019\.[Nuanced metrics for measuring unintended bias with real data for text classification](https://doi.org/10.1145/3308560.3317593)\.In*Companion Proceedings of The 2019 World Wide Web Conference*, pages 491–500\.
- Bullwinkel et al\. \(2026\)Blake Bullwinkel, Giorgio Severi, Keegan Hines, Amanda Minnich, Ram Shankar Siva Kumar, and Yonatan Zunger\. 2026\.[The trigger in the haystack: Extracting and reconstructing LLM backdoor triggers](https://arxiv.org/abs/2602.03085)\.*arXiv preprint arXiv:2602\.03085*\.
- Cui et al\. \(2022\)Ganqu Cui, Lifan Yuan, Bingxiang He, Yangyi Chen, Zhiyuan Liu, and Maosong Sun\. 2022\.[A unified evaluation of textual backdoor learning: Frameworks and benchmarks](https://papers.nips.cc/paper_files/paper/2022/hash/2052b3e0617ecb2ce9474a6feaf422b3-Abstract-Datasets_and_Benchmarks.html)\.In*Advances in Neural Information Processing Systems*, volume 35\.
- Dai et al\. \(2019\)Jiazhu Dai, Chuanshuai Chen, and Yufeng Li\. 2019\.[A backdoor attack against LSTM\-based text classification systems](https://doi.org/10.1109/ACCESS.2019.2941376)\.*IEEE Access*, 7:138872–138878\.
- DeYoung et al\. \(2020\)Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C\. Wallace\. 2020\.[ERASER: A benchmark to evaluate rationalized NLP models](https://doi.org/10.18653/v1/2020.acl-main.408)\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 4443–4458, Online\.
- Ge et al\. \(2025\)Huaizhi Ge, Yiming Li, Qifan Wang, Yongfeng Zhang, and Ruixiang Tang\. 2025\.[When backdoors speak: Understanding LLM backdoor attacks through model\-generated explanations](https://doi.org/10.18653/v1/2025.acl-long.114)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 2278–2296, Vienna, Austria\.
- Gilardi et al\. \(2023\)Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli\. 2023\.[ChatGPT outperforms crowd workers for text\-annotation tasks](https://doi.org/10.1073/pnas.2305016120)\.*Proceedings of the National Academy of Sciences*, 120\(30\):e2305016120\.
- Gu et al\. \(2017\)Tianyu Gu, Brendan Dolan\-Gavitt, and Siddharth Garg\. 2017\.[BadNets: Identifying vulnerabilities in the machine learning model supply chain](https://arxiv.org/abs/1708.06733)\.*arXiv preprint arXiv:1708\.06733*\.
- Hu et al\. \(2024\)Mengxuan Hu, Zihan Guan, Junfeng Guo, Zhongliang Zhou, Jielu Zhang, and Sheng Li\. 2024\.[BBCaL: Black\-box backdoor detection under the causality lens](https://openreview.net/forum?id=HZi9PfLwMn)\.*Transactions on Machine Learning Research*\.
- Jacovi and Goldberg \(2020\)Alon Jacovi and Yoav Goldberg\. 2020\.[Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?](https://doi.org/10.18653/v1/2020.acl-main.386)In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 4198–4205, Online\.
- Jiang et al\. \(2023\)Albert Q\. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie\-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed\. 2023\.[Mistral 7b](https://arxiv.org/abs/2310.06825)\.*Preprint*, arXiv:2310\.06825\.
- Kojima et al\. \(2022\)Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa\. 2022\.[Large language models are zero\-shot reasoners](https://proceedings.neurips.cc/paper_files/paper/2022/hash/8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html)\.In*Advances in Neural Information Processing Systems*, volume 35, pages 22199–22213\.
- Lei et al\. \(2016\)Tao Lei, Regina Barzilay, and Tommi Jaakkola\. 2016\.[Rationalizing neural predictions](https://doi.org/10.18653/v1/D16-1011)\.In*Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing*, pages 107–117, Austin, Texas\.
- Li et al\. \(2025a\)Xi Li, Ruofan Mao, Yusen Zhang, Renze Lou, Chen Wu, and Jiaqi Wang\. 2025a\.[Chain\-of\-Scrutiny: Detecting backdoor attacks for large language models](https://doi.org/10.18653/v1/2025.findings-acl.401)\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 7705–7727, Vienna, Austria\.
- Li and Roth \(2002\)Xin Li and Dan Roth\. 2002\.[Learning question classifiers](https://doi.org/10.3115/1072228.1072378)\.In*COLING 2002: The 19th International Conference on Computational Linguistics*, pages 1–7\.
- Li et al\. \(2025b\)Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun\. 2025b\.[BackdoorLLM: A comprehensive benchmark for backdoor attacks and defenses on large language models](https://proceedings.neurips.cc/paper_files/paper/2025/hash/20ffc2b42c7de4a1960cfdadf305bbe2-Abstract-Datasets_and_Benchmarks_Track.html)\.In*Advances in Neural Information Processing Systems*, volume 38\.
- Mozes et al\. \(2023\)Maximilian Mozes, Jessica Hoffmann, Katrin Tomanek, Muhamed Kouate, Nithum Thain, Ann Yuan, Tolga Bolukbasi, and Lucas Dixon\. 2023\.[Towards agile text classifiers for everyone](https://doi.org/10.18653/v1/2023.findings-emnlp.30)\.In*Findings of the Association for Computational Linguistics: EMNLP 2023*, pages 400–414\.
- Pang and Lee \(2005\)Bo Pang and Lillian Lee\. 2005\.[Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales](https://aclanthology.org/P05-1015/)\.In*Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics*, pages 115–124\.
- Qi et al\. \(2021a\)Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun\. 2021a\.[ONION: A simple and effective defense against textual backdoor attacks](https://doi.org/10.18653/v1/2021.emnlp-main.752)\.In*Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pages 9558–9566\.
- Qi et al\. \(2021b\)Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun\. 2021b\.[Hidden Killer: Invisible textual backdoor attacks with syntactic trigger](https://doi.org/10.18653/v1/2021.acl-long.37)\.In*Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing*, pages 443–453\.
- Qwen Team \(2024\)Qwen Team\. 2024\.[Qwen2\.5 technical report](https://arxiv.org/abs/2412.15115)\.*arXiv preprint arXiv:2412\.15115*\.
- Shen et al\. \(2025\)Guangyu Shen, Siyuan Cheng, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Hanxi Guo, Lu Yan, Xiaolong Jin, Shengwei An, Shiqing Ma, and Xiangyu Zhang\. 2025\.[BAIT: Large language model backdoor scanning by inverting attack target](https://doi.org/10.1109/SP61157.2025.00103)\.In*2025 IEEE Symposium on Security and Privacy*, pages 1676–1694\.
- Socher et al\. \(2013\)Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D\. Manning, Andrew Ng, and Christopher Potts\. 2013\.[Recursive deep models for semantic compositionality over a sentiment treebank](https://aclanthology.org/D13-1170/)\.In*Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing*, pages 1631–1642\.
- Sun et al\. \(2023\)Xiaofei Sun, Xiaoya Li, Jiwei Li, Fei Wu, Shangwei Guo, Tianwei Zhang, and Guoyin Wang\. 2023\.[Text classification via large language models](https://doi.org/10.18653/v1/2023.findings-emnlp.603)\.In*Findings of the Association for Computational Linguistics: EMNLP 2023*, pages 8990–9005\.
- Turpin et al\. \(2023\)Miles Turpin, Julian Michael, Ethan Perez, and Samuel R\. Bowman\. 2023\.[Language models don’t always say what they think: Unfaithful explanations in chain\-of\-thought prompting](https://doi.org/10.52202/075280-3275)\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Wei et al\. \(2022\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V\. Le, and Denny Zhou\. 2022\.[Chain\-of\-thought prompting elicits reasoning in large language models](https://proceedings.neurips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html)\.In*Advances in Neural Information Processing Systems*, volume 35, pages 24824–24837\.
- Xu et al\. \(2024\)Jiashu Xu, Mingyu Ma, Fei Wang, Chaowei Xiao, and Muhao Chen\. 2024\.[Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models](https://doi.org/10.18653/v1/2024.naacl-long.171)\.In*Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 3111–3126\.
- Yang et al\. \(2021a\)Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, and Bin He\. 2021a\.[Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in NLP models](https://doi.org/10.18653/v1/2021.naacl-main.165)\.In*Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 2048–2058, Online\.
- Yang et al\. \(2021b\)Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Xu Sun\. 2021b\.[Rethinking stealthiness of backdoor attack against NLP models](https://doi.org/10.18653/v1/2021.acl-long.431)\.In*Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing*, pages 5543–5557, Online\.
- Zhang et al\. \(2015\)Xiang Zhang, Junbo Zhao, and Yann LeCun\. 2015\.[Character\-level convolutional networks for text classification](https://proceedings.neurips.cc/paper/2015/hash/250cf8b51c773f3f8dc8b4be867a9a02-Abstract.html)\.In*Advances in Neural Information Processing Systems*, volume 28, pages 649–657\.
- Zhang et al\. \(2023\)Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola\. 2023\.[Automatic chain of thought prompting in large language models](https://openreview.net/forum?id=5NTt8GFjUHkr)\.In*International Conference on Learning Representations*\.

## Appendix APositioning and Protocol

The audit unit for our paper is an individual inference\-time input, not a full model release\. The defender observes the input text, the victim’s predicted label, and, when requested, an answer summary or quoted\-evidence response from the same black\-box endpoint\. The defender does not know the trigger string, trigger family, target label, poison rate, or training examples\. The only trusted data used by the proposed deployable thresholds are ordinary clean calibration inputs\.

This setting is intended for classifier\-like LM deployments in which a service already exposes explanation\-bearing outputs for review, moderation, routing, topic triage, or annotation\. In such systems, the explanation channel is not treated as a proof of safety\. It is treated as an additional observable trace that can be audited\. The main question is therefore whether this trace provides a useful per\-input review signal under clean calibration\.

We separate two uses of thresholds\. For our own auditors, thresholds are estimated from the 256 clean calibration examples for that victim run\. For heterogeneous baseline comparison, we also report a fixed clean\-FPR diagnostic: after a detector’s scalar score has been computed, we set a posthoc cutoff at the clean evaluation quantile corresponding to the target review budget and report residual target ASR on triggered inputs\. This diagnostic is used only to compare detectors at a matched review budget for clean data\. It is not presented as the deployment protocol for baselines that do not normally use calibration\.

Table[3](https://arxiv.org/html/2608.12623#A1.T3)positions the access assumptions\. ONION and related token filters can audit each input but do not use the model’s explanation channel\. BBCaL is also black\-box and audits each input, but only assumes hard\-label access and therefore cannot test whether a stated rationale supports a label\. CoS audits generated reasoning, but it is designed for API\-only LLMs with little or no task calibration\. Model\-level scanners such as BAIT target inversion\([Shen et al\. 2025](https://arxiv.org/html/2608.12623#bib.bib22)\)or trigger reconstruction\([Bullwinkel et al\. 2026](https://arxiv.org/html/2608.12623#bib.bib2)\)are complementary: they ask whether a model is compromised, whereas we ask whether a particular inference\-time input should be accepted\. Table[4](https://arxiv.org/html/2608.12623#A1.T4)then separates the result blocks so that attack validity, main non\-adaptive detection, and adaptive explanation\-camouflage stress are not conflated\. This separation is important for interpreting the negative cases: a detector can have a high AUROC but still leave unacceptable residual target ASR at a low clean\-FPR budget, and an adaptive stress run can expose a limitation without invalidating the non\-adaptive benchmark\.

Table 3:Threat\-model positioning\.Table 4:Experimental protocol by result block\.
## Appendix BVictim Validity Diagnostics

Table[5](https://arxiv.org/html/2608.12623#A2.T5)reports the victim\-validity check used before detector evaluation\. pre\-ASR is the fraction of triggered examples already mapped to the attack target before any audit is applied\.

Table 5:Victim validity diagnostics\.
## Appendix CPrompt Templates

We use task\-specific label names but a fixed prompt structure across attack families\. Table[6](https://arxiv.org/html/2608.12623#A3.T6)gives the audit prompt skeletons\. The summary\-rationale prompt asks the victim to return a label and one short answer summary; it does not ask for exact quotes or explicitly instruct lexical grounding\. The quoted\-evidence prompt asks for JSON with a label, two exact quotes, the closest alternative label, and a short contrastive explanation\. Evidence\-only and evidence\-drop probes reuse the same label set but replace the input with either the concatenated quotes or the original document with the quotes removed\. Chunk probes classify title, lead, and body chunks independently\. For prompted\-reasoning baselines, Reasoning asks for a justification, ZS\-CoT adds a step\-by\-step instruction, and Auto\-CoT uses an Auto\-CoT\-style demonstration prompt\. These generated rationales are not used directly as accept/reject decisions: they are passed to the same adapted CoS scrutiny prompt, which scores whether the elicited rationale and answer are consistent with the original answer\. The resulting scalar score is thresholded on clean calibration examples for the fixed\-FPR diagnostic\.

Table 6:Audit prompt skeletons\.The literal prompt prefix is\{task instruction\} \\n\{input field\}: \{input text\} \\n\. For example, SST\-2 and Rotten Tomatoes useClassify the movie review as negative or positive\.with input fieldReview; AG News usesClassify the news article as world, sports, business, or science/technology\.with input fieldArticle\. Decoding is deterministic in the main runs\. For sampling strategy sensitivity, we refer to Appendix[F](https://arxiv.org/html/2608.12623#A6)\. Malformed answer lines, invalid JSON, missing answer fields, or labels outside the task label set are parsed as invalid audit responses and assigned the corresponding high\-drift component score\.

## Appendix DTextual attack examples

Table[7](https://arxiv.org/html/2608.12623#A4.T7)illustrates how the four attack families transform the same clean input\. These examples use the trigger templates and placement rules from our seed\-0 SST\-2 configuration\. BadNets samples one lexical trigger from\{cf, mn, bb, tq\}and inserts it at a random word boundary\. AddSent inserts the fixed sentence\-like sequenceI watch this 3D movieat a random word boundary\. EP samples a rare\-token pair and inserts its two tokens at separate positions; the table shows the representativecf mnprofile\. SOS is trained using the piecewise compositionfriends weekend store, while evaluation uses the contextual trigger sentence shown below\. The examples illustrate the actual input transformation\.

Table 7:Illustrative SST\-2 inputs produced using the trigger templates and placement rules in our main experiments\. The clean input has the negative gold label, all attacks target the positive label, and bold text identifies trigger material for visualization only\. The defender does not receive these annotations or any trigger information\.
## Appendix ECalibration Details

Each victim run uses 256 ordinary clean calibration examples\. Groundedness Drift class\-conditional thresholds are calibration quantiles of grounding drift using the victim\-predicted label, with a global fallback if a class has fewer than four calibration examples\. Unsupported Groundedness components are converted to class\-conditional empirical\-CDF percentiles using victim\-predicted labels from the evidence prompt\. Ties in the empirical CDF use right\-continuous ranks, so a calibration value equal to the test score counts as no more anomalous than the calibration reference\. The calibration\-selected Unsupported Groundedness threshold is estimated on the subset of examples that pass Groundedness Drift, matching the deployed cascade\. Exact quote validation drops evidence spans that are not substrings of the original input after whitespace normalization; malformed JSON, missing label fields, missing evidence fields, or invalid quotes are assigned the maximum component drift before percentile conversion\. The main tables are fixed\-FPR diagnostics computed after the score definitions are fixed\.

## Appendix FSampled\-Rationale Stability

The main experiments use greedy decoding\. We additionally sampleK=5K=5rationales from frozen seed\-0 Qwen victims using temperature0\.70\.7, top\-pp0\.90\.9, no top\-kktruncation, and at most 48 new tokens\. The result is shown in Table[8](https://arxiv.org/html/2608.12623#A6.T8)\.*First*uses the first draw;*Mean*averages the five drift scores; and*Max*uses their maximum\. Mean and Max use the modal predicted label\. Each rule is independently recalibrated on the same 256 clean examples under its corresponding sampling policy\.

Table 8:Sampled\-rationale stability\. Cells report AUROC / rASR / realized clean FPR \(%\)\.A single sampled rationale remains sufficient on SST\-2, while repeated aggregation improves the longer AG News cases\. Max requires five queries and must be calibrated under the same sampling policy\.

## Appendix GAdaptive Explanation\-Camouflage Diagnostics

This appendix analyzes the AG News explanation\-camouflage stress setting used for Table[2](https://arxiv.org/html/2608.12623#S6.T2)\. These diagnostics are not part of the main non\-adaptive benchmark\. They explain why Unsupported Groundedness improves ranking but still leaves residual attack success at low clean\-FPR budgets\. Table[9](https://arxiv.org/html/2608.12623#A7.T9)checks a possible implementation confound: whether the adaptive gap is merely caused by malformed JSON or invalid quoted evidence\. A quote "passes" when the response is parseable and contains at least one quoted span that is an exact substring of the input after whitespace normalization\. Invalid share of clean FP is the fraction of rejected clean examples for which malformed or invalid evidence contributes the maximum component drift\. The valid\-quotes rASR recomputes residual attack success after restricting triggered examples to those with valid quotes\. Since this restricted rASR remains high, the adaptive gap is not just a parsing artifact\. Valid grounded evidence can still be operationally misleading\.

Table 9:AG News adaptive Unsupported Groundedness quote\-validity diagnostic under the calibration\-selected Unsupported Groundedness threshold\.Table[10](https://arxiv.org/html/2608.12623#A7.T10)is a posthoc diagnostic for the adaptive AG News explanation\-camouflage setting\. It slices triggered examples by Unsupported Groundedness anomaly\-score quintile after confirming that nearly all triggered examples already map to the attack target before filtering\. The purpose is to show where the remaining residual ASR comes from: Unsupported Groundedness removes the most anomalous bin, but many target\-success examples remain in lower\-score bins that conservative low\-FPR thresholds must accept\.

Table 10:AG News adaptive Unsupported Groundedness failure slicing by score quintile\.Table[11](https://arxiv.org/html/2608.12623#A7.T11)reports cost only for the adaptive AG News setting where Unsupported Groundedness is invoked\. Groundedness Drift always uses one victim call per input\. Clean/attack Unsupported Groundedness query is the fraction of examples that pass Groundedness Drift and therefore trigger the second\-stage probes\. Average calls include the initial Groundedness Drift call plus the conditional quoted\-evidence, evidence\-only/drop, and local\-view calls; the implementation caps the total at 12 calls per input\.

AttackClean Unsupported Groundedness queryAttack Unsupported Groundedness queryAvg calls cleanAvg calls attackMax callsBadNets0\.9450\.5509\.7655\.92312AddSent0\.8800\.8478\.8808\.23012EP0\.8750\.2178\.8302\.85712SOS0\.9150\.8409\.4358\.82312

Table 11:Seed\-0 AG News adaptive Unsupported Groundedness query\-cost diagnostic\.
## Appendix HFull Baseline Tables

Tables[12](https://arxiv.org/html/2608.12623#A8.T12),[13](https://arxiv.org/html/2608.12623#A8.T13),[14](https://arxiv.org/html/2608.12623#A8.T14),[15](https://arxiv.org/html/2608.12623#A8.T15), and[16](https://arxiv.org/html/2608.12623#A8.T16)report the full non\-adaptive baseline variants behind Table[1](https://arxiv.org/html/2608.12623#S4.T1)\. Each cell is AUROC / rASR @5% / realized clean FPR\. Reasoning\-best in Table[1](https://arxiv.org/html/2608.12623#S4.T1)is the lowest rASR @5% among Reasoning, ZS\-CoT, and Auto\-CoT in these full tables\. They should not be compared with the adaptive AG News Unsupported Groundedness stress diagnostics in Table[2](https://arxiv.org/html/2608.12623#S6.T2)\. CoS\-original is included for transparency even when it collapses to degenerate scores\.

Table 12:Full SST\-2 non\-adaptive baseline comparison\. Cells are AUROC / residual target ASR / realized clean FPR \(%\)\.Table 13:Full Rotten Tomatoes non\-adaptive baseline comparison\. Cells are AUROC / residual target ASR / realized clean FPR \(%\)\.Table 14:Full TREC non\-adaptive baseline comparison\. Cells are AUROC / residual target ASR / realized clean FPR \(%\)\.Table 15:Full Jigsaw non\-adaptive baseline comparison\. Cells are AUROC / residual target ASR / realized clean FPR \(%\)\.Table 16:Full AG News non\-adaptive baseline comparison\. Cells are AUROC / residual target ASR / realized clean FPR \(%\)\.
## Appendix IUnsupported Groundedness Feature Sensitivity on Adaptive AG News

Table[17](https://arxiv.org/html/2608.12623#A9.T17)summarizes AG News adaptive Unsupported Groundedness component sensitivity\. This table uses the seed\-0 explanation\-camouflage stress setting\. It reports component\-only rASR and the default deployment policies: Unsupported Groundedness uses the maximum of support consistency and document consistency after Groundedness Drift has accepted the input\.

Table 17:AG News adaptive Unsupported Groundedness component ablation, reported as rASR @10% for seed 0\.
## Appendix JReproducibility Notes

All reported main results use Qwen2\.5\-7B\-Instruct and Mistral\-7B\-Instruct\-v0\.2 as the victim backbone\. For every dataset/attack family, we train three independently seeded victim runs with seeds 0, 1, and 2 and report the arithmetic mean over seeds\. Each run uses disjoint splits of 256 ordinary clean calibration examples, 200 clean evaluation examples, and 300 triggered evaluation examples\. The same calibration and evaluation slices are shared by Groundedness Drift and all baseline detectors within a run\. The main non\-adaptive benchmark includes SST\-2, Rotten Tomatoes, TREC, Jigsaw, and AG News under BadNets, AddSent, EP, and SOS\-style attacks\. The AG News explanation\-camouflage stress diagnostics also use seeds 0–2 but are reported separately from the main non\-adaptive benchmark\.

Victim models are fine\-tuned with LoRA on Qwen2\.5\-7B\-Instruct and Mistral\-7B\-Instruct\-v0\.2 using rank 8, LoRA alpha 16, and dropout 0\.05\. Training uses three epochs, learning rate10−410^\{\-4\}, batch size 1, gradient accumulation 4, and bfloat16 loading\. SST\-2 and Rotten Tomatoes use 8192 clean training examples with maximum sequence length 192; TREC and AG News use 4096 examples with maximum length 128; Jigsaw uses 4096 examples with maximum length 256\. The poison rate is 0\.10 in the main OpenBackdoor\-style runs\. Target labels arepositivefor SST\-2 and Rotten Tomatoes,locationfor TREC,toxicfor Jigsaw, andscience/technologyfor AG News\.

For thresholding, Groundedness Drift uses only 256 clean calibration examples\. Thresholds are class\-conditional quantiles of the Groundedness Drift score using the victim\-predicted label bucket, with a global fallback if a bucket has fewer than four calibration examples\. Unsupported Groundedness converts its support/document components to class\-conditional empirical\-CDF percentiles using clean calibration outputs from the quoted\-evidence prompt and estimates its cascade threshold on calibration examples that pass Groundedness Drift\. Gold labels are used to construct clean splits and report clean task accuracy, but online threshold lookup conditions only on the victim’s predicted label\.

For the matched\-baseline diagnostic in Table[1](https://arxiv.org/html/2608.12623#S4.T1), every detector first produces a scalar anomaly score on the same held\-out clean and triggered evaluation examples\. We then set a posthoc cutoff at the empirical\(1−α\)\(1\-\\alpha\)quantile of held\-out clean evaluation scores withα=0\.05\\alpha=0\.05and flag only scores strictly greater than this cutoff\. This conservative tie handling keeps realized clean FPR at or below the nominal budget when scores are discrete\. AUROC, rASR @5%, and realized clean FPR in Tables[1](https://arxiv.org/html/2608.12623#S4.T1)and[12](https://arxiv.org/html/2608.12623#A8.T12)–[16](https://arxiv.org/html/2608.12623#A8.T16)are computed from the same scores and splits\.