AsymVerify at SemEval-2026 Task 6: Asymmetric Confidence-Gated Verification for Political Evasion Detection

# AsymVerify at SemEval-2026 Task 6: Asymmetric Confidence-Gated Verification for Political Evasion Detection
Source: [https://arxiv.org/html/2607.20439](https://arxiv.org/html/2607.20439)
###### Abstract

Political evasion is difficult to detect because evasive answers often appear cooperative while avoiding concrete commitment\. We present AsymVerify, a confidence\-gated verification system for SemEval\-2026 Task 6, a three\-way classification of Clear Reply, Ambivalent, and Clear Non\-Reply responses\. AsymVerify scored 0\.85 Macro F1 on the evaluation split \(DevalD\_\{\\text\{eval\}\},n=237n\{=\}237\), placing2nd out of 41 teamson the official leaderboard\. The system first classifies each question\-answer pair, then selectively applies downgrade verification \(CR/CNR→\\rightarrowAMB\) or upgrade verification \(AMB→\\rightarrowCR\) to low\-confidence predictions\. Development analysis shows that errors concentrate at the Ambivalent boundary in both directions, motivating this asymmetric two\-verifier design while confidence gating keeps additional inference cost low\. OnDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\), AsymVerify with GLM\-4\.7 gains \+17\.1 Macro F1 over single\-pass classification at 1\.48 calls/example, and the upgrade verifier alone improves every tested LLM backend onDdevD\_\{\\text\{dev\}\}by \+6\.8 to \+15\.2 Macro F1 over its single\-pass baseline\. Code is available at[https://github\.com/kaons\-research/AsymVerify\-ACL](https://github.com/kaons-research/AsymVerify-ACL)\.

AsymVerify at SemEval\-2026 Task 6: Asymmetric Confidence\-Gated Verification for Political Evasion Detection

Sebastien KawadaKaons[𝑲∗\\boldsymbol\{K\}^\{\\ast\}](https://kaons.com/)Los Angeles, CA, USAsebastien@kaons\.com

## 1Introduction

Viewers often struggle to detect political question dodges without explicit cues\(Clementson,[2018b](https://arxiv.org/html/2607.20439#bib.bib29),[a](https://arxiv.org/html/2607.20439#bib.bib28)\)\. Large language models may face a related failure mode: preference\-optimized systems can favor agreeable or user\-aligned interpretations over truthfulness\(Sharmaet al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib62)\), the opposite of what evasion detection requires\. Politicians exploit this gap with rhetorical strategies that technically engage questions while avoiding concrete commitment\(Bull and Mayer,[1993](https://arxiv.org/html/2607.20439#bib.bib22)\)\. Skilled evasion appears substantive, employing topic\-adjacent responses that feel informative while committing to nothing\.

SemEval\-2026 Task 6 requires classifying political question\-answer pairs into Clear Reply \(CR\), Ambivalent \(AMB\), or Clear Non\-Reply \(CNR\)\(Thomaset al\.,[2026](https://arxiv.org/html/2607.20439#bib.bib21)\)\. The task builds on the QEvasion taxonomy and corpus\(Thomaset al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib26)\), and the CLARITY release provides 3,448 training examples with 59% Ambivalent responses\. The dataset consists of English\-language U\.S\. political interview transcripts\.

We evaluate on three dataset splits:DtrainD\_\{\\text\{train\}\}\(3,448 examples\) for prompt development,DdevD\_\{\\text\{dev\}\}\(308\) for threshold tuning and ablations, andDevalD\_\{\\text\{eval\}\}\(237\) for official placement\. OnDevalD\_\{\\text\{eval\}\}, the GPT\-5\.2 AsymVerify submission scored 0\.85 on CLARITY Subtask 1 and placed2nd out of 41 teams\. OnDdevD\_\{\\text\{dev\}\}, model\-family analyses examine the mechanism across multiple LLM backends\. Confidence analysis shows a clear accuracy gap between high\- and low\-confidence predictions, which is the basis for selective verification\. We further observe that CR and CNR, as opposite speech acts, are rarely confused directly \(3% of errors\) and are instead typically misclassified as AMB\. Because both verification directions correct through AMB, either pass alone targets a large share of errors\.

## 2Background

### 2\.1Task Definition

The task defines three response clarity classes:

- •Clear Reply \(CR\)— direct answer with specific commitment\.
- •Ambivalent \(AMB\)— evasive through vague language, topic shifts, or implicit answers\.
- •Clear Non\-Reply \(CNR\)— explicit refusal or claim of ignorance\.

Table[1](https://arxiv.org/html/2607.20439#S2.T1)shows dataset statistics\. The dev set has higher Ambivalent proportion \(67%\) than training \(59%\), making minority\-class Macro F1 harder\. Systems are evaluated by Macro F1, the unweighted mean\(F1CR\+F1AMB\+F1CNR\)/3\(\\mathrm\{F1\}\_\{\\mathrm\{CR\}\}\+\\mathrm\{F1\}\_\{\\mathrm\{AMB\}\}\+\\mathrm\{F1\}\_\{\\mathrm\{CNR\}\}\)/3, a standard choice for imbalanced multi\-class classification because each class contributes equally to the final score\(Sokolova and Lapalme,[2009](https://arxiv.org/html/2607.20439#bib.bib59)\)\. We report accuracy as a secondary metric\.

Table 1:Dataset splits\. Dev set skews toward Ambivalent \(67% vs 59%\)\.#### Dataset notation\.

To avoid cross\-split ambiguity, we useDtrainD\_\{\\text\{train\}\}\(n=3,448\),DdevD\_\{\\text\{dev\}\}\(n=308\), andDevalD\_\{\\text\{eval\}\}\(n=237\)\. Unless stated otherwise, prompt tuning and ablations are reported onDdevD\_\{\\text\{dev\}\}, while official placement claims useDevalD\_\{\\text\{eval\}\}\.

### 2\.2Related Work

Bull and Mayer \([1993](https://arxiv.org/html/2607.20439#bib.bib22)\)typologize tactics for eluding questions in political interviews, whileClayman \([2001](https://arxiv.org/html/2607.20439#bib.bib25)\)shows how interviewees can resist questions while preserving the norm of answering\.Bavelaset al\.\([1990](https://arxiv.org/html/2607.20439#bib.bib23)\)introduced the Situational Theory of Communicative Conflict, arguing that equivocation arises from communicative dilemmas where all direct responses carry negative consequences\. Recent political\-QA work measures answer quality through question\-answer fit\(Alvarez and Morrier,[2026](https://arxiv.org/html/2607.20439#bib.bib45)\)\. Our task differs from stance detection\(Mohammadet al\.,[2016](https://arxiv.org/html/2607.20439#bib.bib9)\), which classifies opinion direction rather than response clarity, because evasion detection requires reasoning about whether commitments were made, not what position was taken\.

Unlike self\-consistency\(Wanget al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib14)\), which improves accuracy by voting over multiple LLM samples, AsymVerify uses confidence to route examples selectively into verifier calls\. This follows recent work on language\-model cascades\(Guptaet al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib49)\), adaptive computation\(Schusteret al\.,[2022](https://arxiv.org/html/2607.20439#bib.bib48)\), and cost\-aware LLM routing\(Chenet al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib50)\)that route inputs through different computational paths based on difficulty\. Recent text\-classification benchmarks also warn that heavier reasoning strategies can impose large token costs without uniform classification gains\(Guoet al\.,[2026](https://arxiv.org/html/2607.20439#bib.bib33)\), so our verifier design applies extra reasoning only to uncertain boundary cases\.

Our verification passes can be viewed as a task\-specific form of iterative self\-refinement\(Madaanet al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib1); Shinnet al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib2)\), where re\-examination is asymmetrically conditioned on the initial prediction class rather than using open\-ended self\-correction, which can be unreliable without external feedback\(Huanget al\.,[2024a](https://arxiv.org/html/2607.20439#bib.bib3)\)\. Structured reasoning has shown promise for implicit subjective classification\(Sunet al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib11); Feiet al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib10)\), where surface\-level features are insufficient\. Evasion detection has the same property\.

Verbalized confidence tends toward overconfidence\(Groot and Valdenegro\-Toro,[2024](https://arxiv.org/html/2607.20439#bib.bib5)\), but recent work shows that prompted confidence can still provide useful uncertainty information when interpreted carefully\(Xionget al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib4); Huanget al\.,[2024b](https://arxiv.org/html/2607.20439#bib.bib6)\)\. Newer uncertainty\-estimation work distinguishes whether an uncertainty signal identifies hard cases from whether it is probabilistically calibrated\(Liet al\.,[2025](https://arxiv.org/html/2607.20439#bib.bib7)\), and AsymVerify uses confidence in the former sense, as a routing cue\. RLHF can further degrade calibration, though verbalized confidence partially recovers it\(Tianet al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib56)\), and systematic sycophancy in RLHF\-trained models causes them to prefer agreeable interpretations over accurate ones\(Sharmaet al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib62)\)\. This is particularly problematic for evasion detection, where models must resist accepting vague answers as clear\. Our prompt design counters this tendency through structured taxonomy guidance and verification prompts that instruct skepticism\.

## 3System Description

AsymVerify operates in up to three passes, as shown in Figure[1](https://arxiv.org/html/2607.20439#S3.F1), with pseudocode in Appendix[A](https://arxiv.org/html/2607.20439#A1)\. The base classifier returns\(y^,c\)=fθ​\(q,r\)\(\\hat\{y\},c\)=f\_\{\\theta\}\(q,r\)wherey^∈\{cr,amb,cnr\}\\hat\{y\}\\in\\\{\\textsc\{cr\},\\textsc\{amb\},\\textsc\{cnr\}\\\}andc∈\[0,1\]c\\in\[0,1\]is verbalized confidence, a self\-reported scalar from structured JSON output rather than token logprobs\(Tianet al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib56); Xionget al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib4)\)\. High\-confidence predictions \(c≥τc\\geq\\tau\) exit immediately, in the spirit of cascade methods that defer harder instances and resolve easier ones with fewer calls\(Guptaet al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib49)\)\. In the reportedDdevD\_\{\\text\{dev\}\}analyses, low\-confidence predictions keep the single Pass 1 label and receive targeted verification rather than maj@3 voting\.

Stage 1: Base Classification

\(y^1,c1\)\\displaystyle\(\\hat\{y\}\_\{1\},c\_\{1\}\)=fθ​\(q,r\),y^1∈𝒴\\displaystyle=f\_\{\\theta\}\(q,r\),\\quad\\hat\{y\}\_\{1\}\\in\\mathcal\{Y\}𝒴\\displaystyle\\mathcal\{Y\}=\{CR,AMB,CNR\}\\displaystyle=\\\{\\mathrm\{CR\},\\mathrm\{AMB\},\\mathrm\{CNR\}\\\}
Stage 2: Confidence Gate \(single candidate\)

y^\\displaystyle\\hat\{y\}=y^1\\displaystyle=\\hat\{y\}\_\{1\}\(c1≥τ;exit\),\\displaystyle\(c\_\{1\}\\geq\\tau;\\ \\mathrm\{exit\}\),y~\\displaystyle\\tilde\{y\}=y^1\\displaystyle=\\hat\{y\}\_\{1\}\(c1<τ;verify\)\.\\displaystyle\(c\_\{1\}<\\tau;\\ \\mathrm\{verify\}\)\.Budget shown: single\-candidate routing\.

Stage 3: Conditional Verification \(c1<τc\_\{1\}<\\tau\)

ℰ\\displaystyle\\mathcal\{E\}=\{CR,CNR\},\\displaystyle=\\\{\\mathrm\{CR\},\\mathrm\{CNR\}\\\},y↓\\displaystyle y^\{\\downarrow\}=\{AMB,y~∈ℰ,g↓​\(q,r,y~\)=1,y~,otherwise,\\displaystyle=y^\\displaystyle\\hat\{y\}=\{CR,y↓=AMB,g↑​\(q,r\)=1,y↓,otherwise\.\\displaystyle=
Routing profile

1\.48 calls/example

Figure 1:AsymVerify control flow\. TheDdevD\_\{\\text\{dev\}\}call budget uses single\-candidate routing rather than maj@3\. P2 can downgrade CR/CNR predictions to AMB before P3 optionally upgrades AMB predictions to CR\. Routing counts are from GLM\-4\.7 onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308; Table[8](https://arxiv.org/html/2607.20439#S5.T8)\)\.The base classifier prompts the selected model with a structured evasion taxonomy containing nine response subtypes \(explicit, implicit, general, partial, dodging, deflection, declining, claims ignorance, clarification\)\. The official submission used GPT\-5\.2, while development analysis uses GLM\-4\.7, DeepSeek\-V3\.2, and Llama\-3\.3\-70B onDdevD\_\{\\text\{dev\}\}\. The prompt emphasizes concrete commitments and instructs skepticism toward answers that sound substantive but avoid the specific question\. The model returns structured JSON with label, confidencec∈\[0,1\]c\\in\[0,1\], and reasoning\. All reportedDdevD\_\{\\text\{dev\}\}call budgets use the single\-candidate pipeline: each example receives one base classification call, and low\-confidence examples are sent directly to the conditional verifiers\. A maj@3 variant can replace the Pass 1 label, but it is not the cost setting reported in Table[8](https://arxiv.org/html/2607.20439#S5.T8)\.

Pass 2 re\-examines CR and CNR predictions for possible downgrade to AMB using a “one vs\. multiple interpretations” criterion, where if reasonable readers could disagree about what was actually said, the response is Ambivalent\. Pragmatic accounts\(Grice,[1975](https://arxiv.org/html/2607.20439#bib.bib20); Bavelaset al\.,[1990](https://arxiv.org/html/2607.20439#bib.bib23); Clayman,[2001](https://arxiv.org/html/2607.20439#bib.bib25)\)treat indirect answers as satisfying conversational relevance without making an explicit commitment\. Pass 3 re\-examines AMB predictions for possible upgrade to CR by checking whether the first substantive sentence directly answers the question while ignoring preambles and later tangents, a heuristic motivated by the turn\-by\-turn structure of news interviews\(Clayman and Heritage,[2002](https://arxiv.org/html/2607.20439#bib.bib24); Bull and Mayer,[1993](https://arxiv.org/html/2607.20439#bib.bib22)\)\. We do not include an AMB→\\rightarrowCNR upgrade pass because CNR comprises only 8% of the dataset and CNR→\\rightarrowAMB errors account for just 5% of total errors \(Table[5](https://arxiv.org/html/2607.20439#S5.T5)\), offering minimal recovery potential\. Both verification passes run only on low\-confidence predictions, and P3 runs on any low\-confidence example whose current label is AMB after P2, which explains why verifier calls can slightly exceed the number of low\-confidence examples\.

#### Prefilter for upgrade candidates\.

Pass 3 can be expensive because 67% ofDdevD\_\{\\text\{dev\}\}examples are AMB\. We therefore analyze, but do not use in the main GLM ablation, a rule\-based prefilter that selects only AMB predictions whose first sentence shows strong commitment signals before sending them to the LLM verifier\. Four lexical rules trigger verification: \(1\) answers starting with “No” followed by a short declarative sentence, \(2\) answers starting with “Because” \(direct causal explanation\), \(3\) answers starting with “That is” or “That’s” \(declarative assertion\), and \(4\) the pattern “No, I don’t see” \(stance\-taking\)\. Applied to the GLM\-4\.7 full\-system decisions, this prefilter reduced P3 calls from 120 to 3 and false upgrades from 19 to 0 \(Table[10](https://arxiv.org/html/2607.20439#A1.T10)\)\.

The class error distribution further constrains verification design\. CR and CNR are opposite speech acts that are rarely confused directly \(3% of errors\) and are instead typically misclassified as AMB, so all corrections route through AMB\.

## 4Experimental Setup

#### Data\.

We useDtrainD\_\{\\text\{train\}\}\(3,448\) for prompt development,DdevD\_\{\\text\{dev\}\}\(308\) for threshold tuning and ablations, andDevalD\_\{\\text\{eval\}\}\(237\) for official placement\. During submission,DevalD\_\{\\text\{eval\}\}labels were hidden, so model selection and analysis usedDdevD\_\{\\text\{dev\}\}\.

#### Models\.

The official submission used GPT\-5\.2 withreasoning\_effort=high\(OpenAI,[2025](https://arxiv.org/html/2607.20439#bib.bib51)\)\. Development\-set ablations use GLM\-4\.7 as the primary analysis model\(OpenRouter,[2026](https://arxiv.org/html/2607.20439#bib.bib52)\), and model\-family replication evaluates DeepSeek\-V3\.2\(DeepSeek\-AI,[2025](https://arxiv.org/html/2607.20439#bib.bib53)\)and Llama\-3\.3\-70B\(Meta AI,[2024](https://arxiv.org/html/2607.20439#bib.bib54)\)onDdevD\_\{\\text\{dev\}\}\. We report GPT\-5\.2 for the official evaluation split and use the development models for pass\-level analysis, reflecting recent findings that model family, prompting strategy, and inference cost materially affect LLM text\-classification behavior\(Kostinaet al\.,[2025](https://arxiv.org/html/2607.20439#bib.bib32); Guoet al\.,[2026](https://arxiv.org/html/2607.20439#bib.bib33)\)\.

#### Hyperparameters\.

Confidence thresholdτ=0\.95\\tau=0\.95selected from\{0\.85,0\.90,0\.95,1\.0\}\\\{0\.85,0\.90,0\.95,1\.0\\\}onDdevD\_\{\\text\{dev\}\}\. Temperature is 0\.1 for base classification and 0\.0 for verification\. Because LLM confidence measures can be miscalibrated even when they are useful for ranking difficulty\(Groot and Valdenegro\-Toro,[2024](https://arxiv.org/html/2607.20439#bib.bib5); Huanget al\.,[2024b](https://arxiv.org/html/2607.20439#bib.bib6); Tianet al\.,[2023](https://arxiv.org/html/2607.20439#bib.bib56); Liet al\.,[2025](https://arxiv.org/html/2607.20439#bib.bib7)\), we treatτ\\tauas a routing threshold rather than a calibrated probability\. Each configuration uses one low\-temperature decoding pass, and we add 1,000\-sample paired bootstrap intervals for the close deltas most likely to be sampling\-sensitive\(Koehn,[2004](https://arxiv.org/html/2607.20439#bib.bib58); Droret al\.,[2018](https://arxiv.org/html/2607.20439#bib.bib57)\)\.

## 5Results

We first report official leaderboard performance onDevalD\_\{\\text\{eval\}\}for CLARITY Subtask 1 from the shared\-task overview\(Thomaset al\.,[2026](https://arxiv.org/html/2607.20439#bib.bib21)\), then analyze the mechanism onDdevD\_\{\\text\{dev\}\}\. Table[2](https://arxiv.org/html/2607.20439#S5.T2)gives the top\-12 leaderboard\.

Table 2:Official CLARITY Subtask 1 leaderboard onDevalD\_\{\\text\{eval\}\}\(n=237; 41 teams\)\. Scores are official prediction scores, and AsymVerify placed2nd out of 41 teams, with CSE\-UOI also displaying 0\.85\.
### 5\.1Development Analysis \(Dev Set\)

Table[3](https://arxiv.org/html/2607.20439#S5.T3)tests whether the verification mechanism is tied to a single backend\. The P3 upgrade branch improves Macro F1 by \+6\.8 to \+15\.2 across all threeDdevD\_\{\\text\{dev\}\}backends \(GLM\-4\.7, DeepSeek\-V3\.2, Llama\-3\.3\-70B\)\. P2 is more model\-sensitive: it gives the best GLM\-4\.7 result but over\-corrects DeepSeek and Llama, indicating that downgrade verification needs model\-specific thresholding\.

Table 3:Model\-family replication onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\)\. P3\-only improves every tested backend, while full P2\+P3 is strongest on GLM\-4\.7 but not uniformly best\.
### 5\.2Cross\-Model Class Stability

Improvements are not uniform across classes\. Ambivalent detection remains stable across the best replicated variants \(79\.8–83\.1 F1\), suggesting that the evasion\-focused prompt template transfers reliably even when the base model changes\. In contrast, Clear Reply varies by 11\.8 points \(51\.2–63\.0\), and Clear Non\-Reply varies by 40\.5 points \(34\.5–75\.0\), indicating that explicit refusal detection is substantially more model\-dependent\.

This asymmetry explains why portability gains can be large in Macro F1 while overall accuracy remains in a narrow band \(72\.1–75\.3%\)\. Models can converge on majority\-class AMB behavior yet still diverge on minority classes, especially CNR\. Practically, backend replacement appears less sensitive for AMB\-heavy analyses, while CNR\-sensitive use cases require model\-specific prompt adaptation and threshold re\-tuning\.

The full confusion matrix in Table[4](https://arxiv.org/html/2607.20439#S5.T4)provides the bridge from class\-level stability to error mechanism: the largest off\-diagonal cells are AMB→\\rightarrowCR and CR→\\rightarrowAMB, while direct CR↔\\leftrightarrowCNR swaps are sparse\. The aggregated error distribution in Table[5](https://arxiv.org/html/2607.20439#S5.T5)makes the pattern explicit\. The two dominant errors are AMB→\\rightarrowCR \(55%\), where evasive responses are over\-credited as commitments, and CR→\\rightarrowAMB \(29%\), where clear commitments are penalized for rhetorical hedging\. Direct CR↔\\leftrightarrowCNR confusion is rare \(3%\), showing that errors concentrate at the boundaries with AMB rather than between the polar classes\.

Table 4:Confusion matrix onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\), GLM\-4\.7 P1\+P2\+P3\.Table 5:Error patterns onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\), GLM\-4\.7 P1\+P2\+P3\. Direct CR↔\\leftrightarrowCNR confusion is rare because errors concentrate at class boundaries\.
### 5\.3Error Coverage by Verification Direction

This boundary concentration has a practical consequence\. Over 90% of remaining errors fall along the two verification directions \(E↓E\_\{\\downarrow\}for AMB→\\rightarrow\{CR,CNR\} andE↑E\_\{\\uparrow\}for CR→\\rightarrowAMB, with arrows denoting gold→\\rightarrowpredicted labels\)\. Table[6](https://arxiv.org/html/2607.20439#S5.T6)reorganizes the same 76 errors by*which pass direction they align with*\. Pass 2 \(downgrade\) aligns with over\-acceptance errors where AMB is predicted as CR/CNR \(48 errors, 63\.2%\)\. Pass 3 \(upgrade\) aligns with under\-recognized commitments where CR is predicted as AMB \(22 errors, 28\.9%\)\. Only 6 errors \(7\.9%\) fall outside both routes\.

This decomposition explains why single\-pass variants already recover most of the available gain\. Both passes target large but complementary slices of boundary\-concentrated errors, so either direction alone improves strongly \(Table[7](https://arxiv.org/html/2607.20439#S5.T7)\), while combining both gives a smaller but consistent additional gain\.

Table 6:Remaining\-error alignment by verification direction onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\), derived from Table[5](https://arxiv.org/html/2607.20439#S5.T5)\(76 total errors\)\.The ablation in Table[7](https://arxiv.org/html/2607.20439#S5.T7)quantifies each pass’s contribution on GLM\-4\.7\. The gap between base accuracy \(77\.6%\) and Macro F1 \(55\.9%\) reflects the 67% AMB class imbalance, where high accuracy is achievable by defaulting to the majority class while Macro F1 demands balanced performance\. Each verification pass independently contributes about \+15 points, and combining both yields \+17\.1\. Selective verification achieves this at roughly half the call budget of running all three passes unconditionally \(457 versus 924 calls\)\.

Table 7:Verification pass ablation onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\), GLM\-4\.7\. P2/P3 columns report activation counts\.
Where do the computational savings come from? In the single\-candidate GLM\-4\.7 full system, confidence gating routes 162 of 308 examples \(52\.6%\) directly to output after Pass 1, leaving 146 low\-confidence examples\. These examples trigger 149 verifier calls because three P2 downgrades to AMB are subsequently checked by P3\. The full system usesCAsym=N\+nP​2\+nP​3=457C\_\{\\text\{Asym\}\}=N\+n\_\{P2\}\+n\_\{P3\}=457total calls versusCall=3​N=924C\_\{\\text\{all\}\}=3N=924for running all three passes unconditionally, a 50\.5% reduction\. A low\-confidence maj@3 variant would add two base calls per low\-confidence example,Cmaj@3=CAsym\+2​nlow=749C\_\{\\text\{maj@3\}\}=C\_\{\\text\{Asym\}\}\+2n\_\{\\text\{low\}\}=749\.

Table 8:API call budget for single\-candidate routing onDdevD\_\{\\text\{dev\}\}\(GLM\-4\.7,n=308n\{=\}308, without maj@3 voting\)\. Confidence gating reduces total calls by 50\.5% versus running all passes on every example\. A low\-confidence maj@3 variant would add two base calls for each of the 146 low\-confidence examples, increasing the total from 457 to 749 calls, or from 1\.48 to 2\.43 calls/example\.
The routing threshold is supported by a simple confidence\-bin check\. In the GLM\-4\.7 full system, predictions at or aboveτ=0\.95\\tau\{=\}0\.95account for 162 examples, exactly matching the early\-exit count, and are more accurate than the aggregate below\-threshold set: 80\.9% versus 76\.0% \(Appendix Table[9](https://arxiv.org/html/2607.20439#A1.T9)\)\. The tiny 0\.70–0\.80 bin is not meaningful on its own, and these bins are not a full rank\- or probability\-calibration study\(Huanget al\.,[2024b](https://arxiv.org/html/2607.20439#bib.bib6); Liet al\.,[2025](https://arxiv.org/html/2607.20439#bib.bib7)\)\. Instead, the aggregate trend supports verbalized confidence as a routing signal for identifying harder examples\.

Appendix[A\.1](https://arxiv.org/html/2607.20439#A1.SS1)also reports a Pass 3 prefilter analysis\. The lexical filter reduces P3 calls from 120 to 3 and eliminates false upgrades \(19 to 0\), raising Macro F1 from 73\.0 to 74\.7, and because the rules are English lexical cues, we present this as an efficiency variant rather than the main system\.

Paired bootstrap intervals show that close ablation differences should not be over\-read, following standard NLP practice for comparing systems on the same examples\(Koehn,[2004](https://arxiv.org/html/2607.20439#bib.bib58); Droret al\.,[2018](https://arxiv.org/html/2607.20439#bib.bib57)\)\. P1\+P2 and P1\+P3 differ by only \-0\.1 Macro F1 points, with a 95% CI of \[\-5\.9, 5\.9\]\. The prefilter delta is \+1\.7 points but its CI includes zero \[\-0\.9, 4\.1\]\. In contrast, Llama’s P1→\\rightarrowP1\+P3 gain is \+15\.2 points with CI \[7\.5, 23\.1\], a robust replication of the upgrade branch\.

Appendix[E](https://arxiv.org/html/2607.20439#A5)illustrates verification in action with representative success and failure cases fromDdevD\_\{\\text\{dev\}\}\.

## 6Discussion

Confidence routing works because high\-confidence predictions are usually correct, while low\-confidence predictions benefit from targeted verification\. Additional calls on easy cases can add noise rather than signal, so confidence acts as a practical difficulty signal that concentrates computation where it matters\(Guptaet al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib49); Schusteret al\.,[2022](https://arxiv.org/html/2607.20439#bib.bib48)\)\.

Different verification strategies converge because errors concentrate at class boundaries rather than spanning opposite classes \(Section[5\.2](https://arxiv.org/html/2607.20439#S5.SS2)\)\. Both downgrade and upgrade verification correct through AMB, and since over\-acceptance and under\-recognition each account for many errors, either path alone addresses a substantial fraction\. The rare CR↔\\leftrightarrowCNR confusions \(3%\) fall outside both paths but are too infrequent to dominate system behavior\.

Residual errors cluster in hedged commitments, procedural refusals, and conditional commitments\. These cases are difficult even for humans: QEvasion has Fleissκ=0\.644\\kappa=0\.644inter\-annotator agreement\(Thomaset al\.,[2024](https://arxiv.org/html/2607.20439#bib.bib26)\), and political responses are often crafted to appear both decisive and non\-committal\(Clayman,[2001](https://arxiv.org/html/2607.20439#bib.bib25); Clayman and Heritage,[2002](https://arxiv.org/html/2607.20439#bib.bib24); Clementson,[2018a](https://arxiv.org/html/2607.20439#bib.bib28)\)\. Appendix[D](https://arxiv.org/html/2607.20439#A4)further shows that semantic embeddings alone provide negligible class separation \(silhouette = 0\.001\), so retrieval and similarity methods are unlikely to help, making prompt\-based pragmatic reasoning the relevant alternative\.

OnDdevD\_\{\\text\{dev\}\}, AsymVerify reaches 73\.0 Macro F1 using 49\.5% of the unconditional three\-pass call budget\. The control flow is backend\-agnostic, with verifier prompts tuned per model\. Majority\-vote variants can be budgeted separately if used\. Future gains will likely require richer discourse\-level features or calibrated multi\-model disagreement signals rather than additional majority voting alone\.

## 7Conclusion

AsymVerify scored 0\.85 on CLARITY Subtask 1 \(DevalD\_\{\\text\{eval\}\}\), placing2nd out of 41 teamson the official leaderboard\. Its central result is that verification passes targeting opposite failure modes converge because both flow corrections through the middle Ambivalent class\. Confidence gating exploits this convergence to reach 73\.0 Macro F1 on GLM\-4\.7DdevD\_\{\\text\{dev\}\}with 1\.48 calls/example\. The P3 upgrade branch alone replicates gains of \+6\.8 to \+15\.2 Macro F1 across GLM\-4\.7, DeepSeek\-V3\.2, and Llama\-3\.3\-70B\.

## Limitations

Our confidence threshold and verification prompts were optimized on English\-language U\.S\. political interviews, so other political cultures, languages, or interview formats may require re\-tuning\. Political evasion exists on a spectrum, and QEvasion’sκ=0\.644\\kappa=0\.644agreement means the system inherits judgment calls where “correct” classification can reflect annotator interpretation rather than ground truth\. We also do not evaluate adversarially crafted evasions or noisy ASR transcripts\.

The official score establishes performance of the GPT\-5\.2 submission, while pass\-level analysis is reported for GLM\-4\.7, DeepSeek\-V3\.2, and Llama\-3\.3\-70B onDdevD\_\{\\text\{dev\}\}\. Results use single low\-temperature decoding per configuration, so close ablation differences should be read through the paired bootstrap intervals\. Finally, each question\-answer pair is classified independently, although multi\-turn context, follow\-ups, and speaker history may resolve cases that are ambiguous in isolation\.

## Ethics Statement

AsymVerify is intended to support analysis of public political interviews by identifying whether a response gives a clear answer to the question asked\. The labels describe response clarity rather than speaker intent or factual truth, which is important because equivocation can be strategically ambiguous rather than simply false\(Bavelaset al\.,[1990](https://arxiv.org/html/2607.20439#bib.bib23); Clayman,[2001](https://arxiv.org/html/2607.20439#bib.bib25); Clementson,[2018b](https://arxiv.org/html/2607.20439#bib.bib29)\)\. System outputs should therefore be interpreted with conversational context and human judgment\.

## References

- R\. M\. Alvarez and J\. Morrier \(2026\)Measuring the quality of answers in political Q&As with large language models\.Political Analysis34\(1\),pp\. 78–95\.External Links:[Document](https://dx.doi.org/10.1017/pan.2025.10007),[Link](https://www.cambridge.org/core/journals/political-analysis/article/measuring-the-quality-of-answers-in-political-qas-with-large-language-models/90ED502869838ABF73F0EB8AF94BF61E)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p1.1)\.
- J\. B\. Bavelas, A\. Black, N\. Chovil, and J\. Mullett \(1990\)Equivocal communication\.Sage Series in Interpersonal Communication,Sage Publications,Newbury Park, CA\.External Links:ISBN 9780803929425,[Link](https://openlibrary.org/books/OL1856260M/Equivocal_communication)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p1.1),[§3](https://arxiv.org/html/2607.20439#S3.p3.2),[Ethics Statement](https://arxiv.org/html/2607.20439#Sx2.p1.1)\.
- P\. E\. Bull and K\. Mayer \(1993\)How not to answer questions in political interviews\.Political Psychology14\(4\),pp\. 651–666\.External Links:[Document](https://dx.doi.org/10.2307/3791379),[Link](https://doi.org/10.2307/3791379)Cited by:[§1](https://arxiv.org/html/2607.20439#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p1.1),[§3](https://arxiv.org/html/2607.20439#S3.p3.2)\.
- L\. Chen, M\. Zaharia, and J\. Zou \(2023\)FrugalGPT: how to use large language models while reducing cost and improving performance\.arXiv preprint arXiv:2305\.05176\.External Links:[Link](https://arxiv.org/abs/2305.05176),2305\.05176,[Document](https://dx.doi.org/10.48550/arXiv.2305.05176)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p2.1)\.
- S\. E\. Clayman and J\. Heritage \(2002\)The news interview: journalists and public figures on the air\.Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/CBO9780511613623),[Link](https://www.cambridge.org/core/books/news-interview/3EAC414152BEAB1078419B8E9DAB66B9)Cited by:[§3](https://arxiv.org/html/2607.20439#S3.p3.2),[§6](https://arxiv.org/html/2607.20439#S6.p3.1)\.
- S\. E\. Clayman \(2001\)Answers and evasions\.Language in Society30\(3\),pp\. 403–442\.External Links:[Document](https://dx.doi.org/10.1017/S0047404501003037),[Link](https://www.cambridge.org/core/journals/language-in-society/article/answers-and-evasions/83445D54DC42F62473CAD87374B58557)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p1.1),[§3](https://arxiv.org/html/2607.20439#S3.p3.2),[§6](https://arxiv.org/html/2607.20439#S6.p3.1),[Ethics Statement](https://arxiv.org/html/2607.20439#Sx2.p1.1)\.
- D\. E\. Clementson \(2018a\)Deceptively dodging questions: a theoretical note on issues of perception and detection\.Discourse & Communication12\(5\),pp\. 478–496\.External Links:[Document](https://dx.doi.org/10.1177/1750481318766923),[Link](https://doi.org/10.1177/1750481318766923)Cited by:[§1](https://arxiv.org/html/2607.20439#S1.p1.1),[§6](https://arxiv.org/html/2607.20439#S6.p3.1)\.
- D\. E\. Clementson \(2018b\)Effects of dodging questions: how politicians escape deception detection and how they get caught\.Journal of Language and Social Psychology37\(1\),pp\. 93–113\.External Links:[Document](https://dx.doi.org/10.1177/0261927X17706960),[Link](https://doi.org/10.1177/0261927X17706960)Cited by:[§1](https://arxiv.org/html/2607.20439#S1.p1.1),[Ethics Statement](https://arxiv.org/html/2607.20439#Sx2.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-V3\.2: pushing the frontier of open large language models\.arXiv preprint arXiv:2512\.02556\.External Links:[Link](https://arxiv.org/abs/2512.02556),2512\.02556,[Document](https://dx.doi.org/10.48550/arXiv.2512.02556)Cited by:[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px2.p1.1)\.
- R\. Dror, G\. Baumer, S\. Shlomov, and R\. Reichart \(2018\)The hitchhiker’s guide to testing statistical significance in natural language processing\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Melbourne, Australia,pp\. 1383–1392\.External Links:[Link](https://aclanthology.org/P18-1128/),[Document](https://dx.doi.org/10.18653/v1/P18-1128)Cited by:[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px3.p1.4),[§5\.3](https://arxiv.org/html/2607.20439#S5.SS3.p7.1)\.
- H\. Fei, B\. Li, Q\. Liu, L\. Bing, F\. Li, and T\. Chua \(2023\)Reasoning implicit sentiment with chain\-of\-thought prompting\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),Toronto, Canada,pp\. 1171–1182\.External Links:[Link](https://aclanthology.org/2023.acl-short.101/),[Document](https://dx.doi.org/10.18653/v1/2023.acl-short.101)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p3.1)\.
- Google AI \(2026\)Embeddings: Gemini API\.Note:Accessed: 2026\-05\-01External Links:[Link](https://ai.google.dev/gemini-api/docs/embeddings)Cited by:[Appendix D](https://arxiv.org/html/2607.20439#A4.p1.1)\.
- H\. P\. Grice \(1975\)Logic and conversation\.InSyntax and Semantics, Vol\. 3: Speech Acts,P\. Cole and J\. L\. Morgan \(Eds\.\),pp\. 41–58\.External Links:[Link](https://philpapers.org/rec/GRIISA)Cited by:[§3](https://arxiv.org/html/2607.20439#S3.p3.2)\.
- T\. Groot and M\. Valdenegro\-Toro \(2024\)Overconfidence is key: verbalized uncertainty evaluation in large language and vision\-language models\.InProceedings of the 4th Workshop on Trustworthy Natural Language Processing \(TrustNLP 2024\),Mexico City, Mexico,pp\. 145–171\.External Links:[Link](https://aclanthology.org/2024.trustnlp-1.13/),[Document](https://dx.doi.org/10.18653/v1/2024.trustnlp-1.13)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p4.1),[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px3.p1.4)\.
- X\. Guo, Y\. Zhang, and J\. Qin \(2026\)TextReasoningBench: does reasoning really improve text classification in large language models?\.arXiv preprint arXiv:2603\.19558\.External Links:[Link](https://arxiv.org/abs/2603.19558),2603\.19558,[Document](https://dx.doi.org/10.48550/arXiv.2603.19558)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p2.1),[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px2.p1.1)\.
- N\. Gupta, H\. Narasimhan, W\. Jitkrittum, A\. S\. Rawat, A\. K\. Menon, and S\. Kumar \(2024\)Language model cascades: token\-level uncertainty and beyond\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=KgaBScZ4VI),2404\.10136,[Document](https://dx.doi.org/10.48550/arXiv.2404.10136)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p2.1),[§3](https://arxiv.org/html/2607.20439#S3.p1.5),[§6](https://arxiv.org/html/2607.20439#S6.p1.1)\.
- J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. W\. Yu, X\. Song, and D\. Zhou \(2024a\)Large language models cannot self\-correct reasoning yet\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=IkmD3fKBPQ),2310\.01798,[Document](https://dx.doi.org/10.48550/arXiv.2310.01798)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p3.1)\.
- X\. Huang, S\. Li, M\. Yu, M\. Sesia, H\. Hassani, I\. Lee, O\. Bastani, and E\. Dobriban \(2024b\)Uncertainty in language models: assessment through rank\-calibration\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 284–312\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.18/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.18)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p4.1),[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px3.p1.4),[§5\.3](https://arxiv.org/html/2607.20439#S5.SS3.p5.1)\.
- P\. Koehn \(2004\)Statistical significance tests for machine translation evaluation\.InProceedings of the 2004 Conference on Empirical Methods in Natural Language Processing,Barcelona, Spain,pp\. 388–395\.External Links:[Link](https://aclanthology.org/W04-3250/)Cited by:[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px3.p1.4),[§5\.3](https://arxiv.org/html/2607.20439#S5.SS3.p7.1)\.
- A\. Kostina, M\. D\. Dikaiakos, D\. Stefanidis, and G\. Pallis \(2025\)Large language models for text classification: case study and comprehensive review\.arXiv preprint arXiv:2501\.08457\.External Links:[Link](https://arxiv.org/abs/2501.08457),2501\.08457,[Document](https://dx.doi.org/10.48550/arXiv.2501.08457)Cited by:[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px2.p1.1)\.
- R\. Li, J\. Long, M\. Qi, H\. Xia, L\. Sha, P\. Wang, and Z\. Sui \(2025\)Towards harmonized uncertainty estimation for large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vienna, Austria,pp\. 22938–22953\.External Links:[Link](https://aclanthology.org/2025.acl-long.1118/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1118),ISBN 979\-8\-89176\-251\-0Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p4.1),[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px3.p1.4),[§5\.3](https://arxiv.org/html/2607.20439#S5.SS3.p5.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. Clark \(2023\)Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/91edff07232fb1b55a505a9e9f6c0ff3-Abstract-Conference.html),2303\.17651,[Document](https://dx.doi.org/10.48550/arXiv.2303.17651)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p3.1)\.
- L\. McInnes, J\. Healy, and J\. Melville \(2018\)UMAP: uniform manifold approximation and projection for dimension reduction\.arXiv preprint arXiv:1802\.03426\.External Links:[Link](https://arxiv.org/abs/1802.03426),1802\.03426,[Document](https://dx.doi.org/10.48550/arXiv.1802.03426)Cited by:[Appendix D](https://arxiv.org/html/2607.20439#A4.p1.1)\.
- Meta AI \(2024\)Llama 3\.3 70B Instruct model card\.Note:Hugging Face model cardExternal Links:[Link](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)Cited by:[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px2.p1.1)\.
- S\. Mohammad, S\. Kiritchenko, P\. Sobhani, X\. Zhu, and C\. Cherry \(2016\)SemEval\-2016 task 6: detecting stance in tweets\.InProceedings of the 10th International Workshop on Semantic Evaluation \(SemEval\-2016\),San Diego, California,pp\. 31–41\.External Links:[Link](https://aclanthology.org/S16-1003/),[Document](https://dx.doi.org/10.18653/v1/S16-1003)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p1.1)\.
- OpenAI \(2025\)GPT\-5\.2 model\.Note:OpenAI API documentation\. Accessed: 2026\-05\-03External Links:[Link](https://platform.openai.com/docs/models/gpt-5.2)Cited by:[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px2.p1.1)\.
- OpenRouter \(2026\)GLM 4\.7: api pricing and providers\.Note:Accessed: 2026\-05\-01External Links:[Link](https://openrouter.ai/z-ai/glm-4.7)Cited by:[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px2.p1.1)\.
- T\. Schuster, A\. Fisch, J\. Gupta, M\. Dehghani, D\. Bahri, V\. Q\. Tran, Y\. Tay, and D\. Metzler \(2022\)Confident adaptive language modeling\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 17456–17472\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/6fac9e316a4ae75ea244ddcef1982c71-Abstract-Conference.html),2207\.07061,[Document](https://dx.doi.org/10.48550/arXiv.2207.07061)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p2.1),[§6](https://arxiv.org/html/2607.20439#S6.p1.1)\.
- M\. Sharma, M\. Tong, T\. Korbak, D\. Duvenaud, A\. Askell, S\. R\. Bowman, N\. Cheng, E\. Durmus, Z\. Hatfield\-Dodds, S\. R\. Johnston, S\. Kravec, T\. Maxwell, S\. McCandlish, K\. Ndousse, O\. Rausch, N\. Schiefer, D\. Yan, M\. Zhang, and E\. Perez \(2024\)Towards understanding sycophancy in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/0105f7972202c1d4fb817da9f21a9663-Abstract-Conference.html),2310\.13548,[Document](https://dx.doi.org/10.48550/arXiv.2310.13548)Cited by:[§1](https://arxiv.org/html/2607.20439#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p4.1)\.
- N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://openreview.net/forum?id=vAElhFcKW6)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p3.1)\.
- M\. Sokolova and G\. Lapalme \(2009\)A systematic analysis of performance measures for classification tasks\.Information Processing & Management45\(4\),pp\. 427–437\.External Links:[Document](https://dx.doi.org/10.1016/j.ipm.2009.03.002),[Link](https://doi.org/10.1016/j.ipm.2009.03.002)Cited by:[§2\.1](https://arxiv.org/html/2607.20439#S2.SS1.p3.1)\.
- X\. Sun, X\. Li, J\. Li, F\. Wu, S\. Guo, T\. Zhang, and G\. Wang \(2023\)Text classification via large language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Singapore,pp\. 8990–9005\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.603/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.603)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p3.1)\.
- K\. Thomas, G\. Filandrianos, M\. Lymperaiou, C\. Zerva, and G\. Stamou \(2024\)“I never said that”: a dataset, taxonomy and baselines on response clarity classification\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 5204–5233\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.300/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.300)Cited by:[§1](https://arxiv.org/html/2607.20439#S1.p2.1),[§6](https://arxiv.org/html/2607.20439#S6.p3.1)\.
- K\. Thomas, G\. Filandrianos, M\. Lymperaiou, C\. Zerva, and G\. Stamou \(2026\)SemEval\-2026 Task 6: CLARITY – unmasking political question evasions\.External Links:2603\.14027,[Link](https://arxiv.org/abs/2603.14027),[Document](https://dx.doi.org/10.48550/arXiv.2603.14027)Cited by:[§1](https://arxiv.org/html/2607.20439#S1.p2.1),[§5](https://arxiv.org/html/2607.20439#S5.p1.2)\.
- K\. Tian, E\. Mitchell, A\. Zhou, A\. Sharma, R\. Rafailov, H\. Yao, C\. Finn, and C\. Manning \(2023\)Just ask for calibration: strategies for eliciting calibrated confidence scores from language models fine\-tuned with human feedback\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Singapore,pp\. 5433–5442\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.330/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.330)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p4.1),[§3](https://arxiv.org/html/2607.20439#S3.p1.5),[§4](https://arxiv.org/html/2607.20439#S4.SS0.SSS0.Px3.p1.4)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2023\)Self\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=1PL1NIMMrw),2203\.11171,[Document](https://dx.doi.org/10.48550/arXiv.2203.11171)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p2.1)\.
- M\. Xiong, Z\. Hu, X\. Lu, Y\. Li, J\. Fu, J\. He, and B\. Hooi \(2024\)Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=gjeQKFxFpZ),2306\.13063,[Document](https://dx.doi.org/10.48550/arXiv.2306.13063)Cited by:[§2\.2](https://arxiv.org/html/2607.20439#S2.SS2.p4.1),[§3](https://arxiv.org/html/2607.20439#S3.p1.5)\.

## Appendix AFull Pseudocode

Input :Question

qq, response

rr
Output :Label

y^∈\{cr,amb,cnr\}\\hat\{y\}\\in\\\{\\textsc\{cr\},\\textsc\{amb\},\\textsc\{cnr\}\\\}
Pass 1: Base classification;

\(y^,c\)←fθ​\(q,r\)\(\\hat\{y\},c\)\\leftarrow f\_\{\\theta\}\(q,r\);

Confidence gating;

if

c≥τc\\geq\\tauthen

return

y^\\hat\{y\};

Low confidence: single\-candidate verifier routing;

y~←y^\\tilde\{y\}\\leftarrow\\hat\{y\};

Conditional branch P2: Downgrade CR/CNR→\\toAMB;

if

y~∈\{cr,cnr\}\\tilde\{y\}\\in\\\{\\textsc\{cr\},\\textsc\{cnr\}\\\}then

if

g↓​\(q,r,y~\)=1g\_\{\\downarrow\}\(q,r,\\tilde\{y\}\)=1then

y~←amb\\tilde\{y\}\\leftarrow\\textsc\{amb\};

Conditional branch P3: Upgrade AMB→\\toCR;

if

y~=amb\\tilde\{y\}=\\textsc\{amb\}then

if

g↑​\(q,r\)=1g\_\{\\uparrow\}\(q,r\)=1then

y~←cr\\tilde\{y\}\\leftarrow\\textsc\{cr\};

y^←y~\\hat\{y\}\\leftarrow\\tilde\{y\};

return

y^\\hat\{y\};

Algorithm 1AsymVerify\. Parameters:τ=0\.95\\tau\{=\}0\.95\. TheDdevD\_\{\\text\{dev\}\}call budget uses single\-candidate routing, with maj@3 treated as a separate variant\. P2 and P3 are conditional branches rather than unconditional sequential calls, andg↓,g↑∈\{0,1\}g\_\{\\downarrow\},g\_\{\\uparrow\}\\in\\\{0,1\\\}are LLM verifiers\.### A\.1Additional Routing Analysis

Tables[9](https://arxiv.org/html/2607.20439#A1.T9)and[10](https://arxiv.org/html/2607.20439#A1.T10)report the confidence\-bin and prefilter analyses that support the routing decisions summarized in the main results\.

Table 9:Verbalized\-confidence bin accuracy onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\), GLM\-4\.7 full system\.Table 10:Pass 3 prefilter analysis onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\), GLM\-4\.7 full system\.

## Appendix BPrompt Templates

This appendix reproduces the full prompts used in all three passes\.

### B\.1Base Classification Prompt

Pass 1: Base ClassificationRole:You are classifying political Q&A exchanges for evasion\.Guidance•Politicians are skilled at appearing to answer while actually evading\.•Be skeptical of surface\-level cooperation; look for concrete commitments\.Evasion taxonomyNote:Answers begin with speaker identification \(e\.g\., “President Trump\.”\)\. This is transcript formatting; ignore it\.Clear Reply \(Explicit\):Direct answers providing specific information, a clear yes/no with commitment, or concrete numbers, names, dates, and policies\.Ambivalent:Evasive responses:1\.Implicit: hints without stating explicitly\.2\.General: too vague, lacks specificity\.3\.Partial: addresses only part of the question\.4\.Dodging: ignores the question or changes topic\.5\.Deflection: starts on topic but pivots away\.Clear Non\-Reply:Explicit refusal:1\.Declining: explicitly refuses \(“I won’t comment”\)\.2\.Claims ignorance: says they do not know\.3\.Clarification: asks for clarification instead\.TaskAnalyze the exchange:Question:“\{question\}”Answer:“\{answer\}”Check:1\.What specific information is the question asking for?2\.Does the answer provide that specific information?3\.Is there evasion, deflection, or vagueness?Output JSON:\{"classification": "Clear Reply" \| "Ambivalent" \| "Clear Non\-Reply", "confidence": 0\.0\-\-1\.0, "reasoning": "brief"\}

### B\.2Downgrade Verification Prompts \(Pass 2\)

Pass 2a: Clear Reply→\\rightarrowAmbivalentInputQuestion: “\{question\}” Answer: “\{answer\}”\(Skip speaker ID; focus on substantive response\.\)Decision:Does this answer admit only one interpretation or multiple?Clear Reply:only one interpretation is possible; the answer explicitly commits to a position, no inference needed\.Ambivalent:multiple interpretations are possible; inference is required\.Examples Q: “Have you seen my chocolates?” A: “The children were in your room this morning\.”→\\rightarrowAmbivalent\(implies the children took them, but does not explicitly say so\)Q: “Have you seen my chocolates?” A: “Yes, they are in the kitchen\.”→\\rightarrowClear Reply\(only one interpretation\)Output:\{"classification": "Clear Reply" \| "Ambivalent", "reasoning": "brief"\}

Pass 2b: Clear Non\-Reply→\\rightarrowAmbivalentInputQuestion: “\{question\}” Answer: “\{answer\}”\(Skip speaker ID; focus on substantive response\.\)Decision:Is this a Clear Non\-Reply or Ambivalent?Clear Non\-Reply:openly refuses to share information\. The refusal is explicit and unambiguous\.•“I don’t know” / “I’m not aware” \(claims ignorance\)•“I won’t comment” / “No comment” \(declines\)•“What do you mean?” \(asks for clarification\)Ambivalent:provides a response but allows multiple interpretations\.•Leverages the subject to pivot elsewhere \(deflection\)•Gives information that does not answer the question•Appears to engage but does not commitExamples Q: “Have you seen my chocolates?” A: “You should not keep chocolates all around the house\.”→\\rightarrowAmbivalent\(deflects; no information about seeing chocolates\)Q: “Have you seen my chocolates?” A: “I don’t know where they are\.”→\\rightarrowClear Non\-Reply\(explicit claim of ignorance\)Output:\{"classification": "Clear Non\-Reply" \| "Ambivalent", "reasoning": "brief"\}

### B\.3Upgrade Verification Prompt \(Pass 3\)

Pass 3: Upgrade VerificationInputQuestion: “\{question\}” Answer: “\{answer\}”Currently classified asAmbivalent\. Check if it should beClear Reply\.\(Skip speaker ID; inspect first substantive sentence\.\)Upgrade to Clear Reply if the first substantive sentence:1\.Directly answers with yes/no, a specific stance, or a clear position\.2\.Does not start with preambles \(“Well…”, “Look…”, “Let me…”\)\.3\.Is not immediately followed by “but”, “however”, or “although”\.Important:what comes*after*the first substantive sentence does not matter\. The key test: can you extract one clear answer from the opening?Clear Reply examples:“No, I don’t see a contradiction…” \(clear stance\) “That is one of the options…” \(specific commitment\) “Because it takes time…” \(direct causal\)Stays Ambivalent:“Well, I think…” \(preamble\) “It depends on…” \(conditional\) “I wouldn’t say…” \(negation without stance\)Output:\{"classification": "Clear Reply" \| "Ambivalent", "reasoning": "brief"\}

## Appendix CExtended Model\-Family Comparison

Table[11](https://arxiv.org/html/2607.20439#A3.T11)shows the complete model\-family development results\.

Table 11:Extended model\-family comparison onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\)\.Table 12:Per\-class F1 for the strongest development variant of each tested backend onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308\)\.Table 13:Paired bootstrap confidence intervals onDdevD\_\{\\text\{dev\}\}\(n=308n\{=\}308; 1,000 resamples\)\.
## Appendix DEmbedding Space Analysis

To probe whether semantic similarity separates the response\-clarity labels, we project 3,756 train\+dev embeddings from Gemini Embedding 001 \(3,072 dimensions;Google AI,[2026](https://arxiv.org/html/2607.20439#bib.bib55)\) into two dimensions using UMAP\(McInneset al\.,[2018](https://arxiv.org/html/2607.20439#bib.bib12)\)\. Figure[2](https://arxiv.org/html/2607.20439#A4.F2)shows near\-total class overlap, with a silhouette score of 0\.001 indicating essentially no separation in this projection\. All three classes intermix throughout the projection, and class centroids are separated by only 0\.001–0\.016 cosine distance in the projected space\. This projection suggests that these embeddings do not cleanly separate response\-clarity labels, matching contrastive RAG ablations that did not improve classification\. Evasion detection depends on pragmatic cues such as commitment strength, hedging patterns, and rhetorical structure that general\-purpose text embeddings did not capture in this setting\.

![Refer to caption](https://arxiv.org/html/2607.20439v1/x1.png)Figure 2:UMAP projection of 3,756 train\+dev embeddings \(Gemini Embedding 001, 3,072 dimensions\)\. Classes overlap almost entirely \(silhouette = 0\.001\), suggesting that this embedding/projection does not cleanly separate response\-clarity labels\.
## Appendix EExtended Qualitative Examples

Tables[14](https://arxiv.org/html/2607.20439#A5.T14)and[15](https://arxiv.org/html/2607.20439#A5.T15)show representative predictions with full reasoning traces\. We select examples where verification*changed*the initial prediction, demonstrating the pipeline’s corrective behavior\.

Table 14:Verification saves: cases where Pass 2/3 corrected an initial error\.✓ Pass 3 upgrade: AMB→\\rightarrowCRGold: CR∣\\midFinal: CRQ: Are you committed to building the 700 miles of fence, actual fencing? A: “Yes, we’re going to do both, Joe\. We’re just going to make sure that we build it in a spot where it works…”Pass 1\(confidence 0\.85→\\rightarrowAMB\): “The speaker begins with a direct ‘Yes,’ which seems like a clear reply, but immediately qualifies the commitment by focusing on building fencing ‘in a spot where it works’ rather than confirming the specific 700\-mile target\. The response employs deflection, shifting the focus from the specific mileage to a broader discussion about ‘smart borders,’ guest worker programs, and terrain conditions\.”Pass 3\(upgrade→\\rightarrowCR\): “The first substantive sentence begins with ‘Yes,’ directly answering the question, and does not use prohibited preambles or immediate negations like ‘but’ or ‘however’\.”Pass 1 was distracted by post\-answer qualifications; Pass 3 correctly anchored on the opening “Yes\.”

✓ Pass 2a downgrade: CR→\\rightarrowAMBGold: AMB∣\\midFinal: AMBQ: Do you feel like you have been an effective leader when it comes to the economy? A: “Well, first of all, wherever Republicans indicate an interest…When I came into office, the U\.S\. economy had contracted by 9 percent…Little over a year later, the economy was growing by 4 percent…It’s not good enough\.”Pass 1\(confidence 0\.9→\\rightarrowCR\): “The speaker eventually provides a direct, substantive answer by citing specific economic metrics: ‘the U\.S\. economy had contracted by 9 percent…a year later, the economy was growing by 4 percent\.’ This provides concrete numbers and dates to evaluate effectiveness, fitting the criteria for a Clear Reply\.”Pass 2a\(downgrade→\\rightarrowAMB\): “The speaker presents evidence of economic growth but immediately qualifies it by saying it’s ‘not good enough\.’ They never explicitly state whether they view themselves as an effective leader, leaving room for multiple interpretations about their self\-assessment\.”Pass 3\(upgrade check→\\rightarrowAMB, no change\): “The first substantive sentence starts with the preamble ‘Well, first of all’ and pivots to discussing Republicans rather than directly answering the specific yes/no question about being an effective leader\.”All three passes ran\. Pass 1 was swayed by concrete metrics; Pass 2a caught the unanswered yes/no; Pass 3 confirmed the preamble signals evasion\.

✓ Pass 3 upgrade after parse failure: AMB→\\rightarrowCRGold: CR∣\\midFinal: CRQ: Are you still pursuing him? A: “Absolutely\. No ands, ifs, or buts\.And in my judgment, it’s just a matter of time, unless we stop looking…”Pass 1: Parse failure \(no structured output returned\)\. Defaults to AMB with confidence 0\.0\.Pass 3\(upgrade→\\rightarrowCR\): “The first substantive sentence is ‘Absolutely,’ which directly answers the question with a clear affirmative without preambles or immediate contrasting conjunctions\.”The pipeline’s fallback\-to\-AMB \+ mandatory verification recovered from a complete Pass 1 failure\.

✓ Pass 3 upgrade: AMB→\\rightarrowCRGold: CR∣\\midFinal: CRQ: Is the Vladimir Putin who said “The United States is undermining global security” the same Vladimir Putin whose soul you looked into and found to be trustworthy? A: “I thinkthe person who I was referring to in 2001 is the same strong\-willed person\. He is a person with whom I have had agreements and disagreements throughout the course of my Presidency…”Pass 1\(confidence 0\.9→\\rightarrowAMB\): “The question asks if Putin is still the ‘trustworthy’ person the speaker referenced in 2001\. The speaker acknowledges Putin is the same ‘strong\-willed’ person but deflects to discussing policy disagreements on NATO and cooperation on non\-proliferation\. He avoids explicitly confirming or denying whether he still finds Putin ‘trustworthy\.’ ”Pass 3\(upgrade→\\rightarrowCR\): “The first substantive sentence directly answers whether it’s the same Putin by stating ‘I think the person who I was referring to in 2001 is the same strong\-willed person\.’ This provides a clear position without contradictory qualifiers\.”Pass 1 over\-indexed on the “trustworthy” framing; Pass 3 recognized that identifying the same person*is*the direct answer\.

Table 15:Verification failures: cases where Pass 2/3 introduced or failed to correct errors\.✗ Pass 3 over\-upgrade: AMB→\\rightarrowCRGold: AMB∣\\midFinal: CRQ: Are you still willing to follow a path that seems to be in opposition to the will of the American people? A: “I am willing to follow a path that leads to victory, and that’s exactly why we’re conducting the review…Victory in Iraq is achievable\. It hasn’t happened nearly as quickly as I hoped…”Pass 1\(confidence 0\.85→\\rightarrowAMB\): “The answer uses deflection and reframing rather than providing a direct ‘yes’ or ‘no’ commitment to the question’s premise\. Instead of acknowledging or explicitly refusing the premise that his path is ‘in opposition to the will of the American people,’ the speaker redefines the path as one that leads to ‘victory’ and argues that the public actually supports staying\.”Pass 3\(upgrade→\\rightarrowCR\): “The first substantive sentence \(‘I am willing to follow a path that leads to victory’\) directly answers the question with a specific stance, contains no preambles, and is not immediately followed by a negating conjunction\.”Failure: Pass 1 correctly identified the reframing \(“a path that leads to victory”≠\\neqthe path asked about\), but Pass 3’s first\-sentence heuristic overrode this nuanced judgment\.

✗ Pass 2a over\-downgrade: CR→\\rightarrowAMBGold: CR∣\\midFinal: AMBQ: What have they achieved and what will they leave behind for their successors? A: “Okay, I will start answering\. Has it become better, or has it become worse?…Let us remember the world on the brink of a nuclear disaster during the Caribbean crisis…We have created a reliable platform for cooperation…”Pass 1\(confidence 0\.9→\\rightarrowCR\): “The speaker directly addresses the question by listing specific achievements in the bilateral relationship, such as shifting from enemies to partners and making progress in counterterrorism, non\-proliferation, and fighting the drug threat\.”Pass 2a\(downgrade→\\rightarrowAMB\): “The answer is vague about specific achievements and doesn’t clearly address what will be left behind for successors\. While it mentions improved relations, it requires interpretation to understand the concrete nature and significance of these achievements\.”Pass 3\(upgrade check→\\rightarrowAMB, no change\): “The answer begins with the preamble ‘Okay, I will start answering,’ followed by a rhetorical question \(‘Has it become better, or has it become worse?’\)\. It fails to provide a direct answer in the first substantive sentence\.”Failure: All three passes ran\. Pass 1 correctly recognized substantive policy content; Pass 2a and 3 both penalized the rhetorical register despite the answer containing concrete achievements\.

✗ Pass 3 over\-upgrade: CNR→\\rightarrowCRGold: CNR∣\\midFinal: CRQ: Why wouldn’t you be frustrated, sir, about what’s happening? A: “I’m not—I do remember the meeting; I don’t remember being surprised\. I’m not sure what they meant by that\.”Pass 1\(confidence 0\.9→\\rightarrowAMB\): “The question asks for a reason why the speaker is not frustrated\. The answer provides information about memory of a meeting and lack of surprise, but fails to address the specific emotional state of frustration or the reasoning behind it\.”Pass 3\(upgrade→\\rightarrowCR\): “The first substantive sentence \(‘I’m not—I do remember the meeting; I don’t remember being surprised’\) directly denies the frustration implied in the question and provides a specific factual stance on the meeting without preambles or negating conjunctions\.”Failure: The truncated “I’m not—” is a false start, not a denial\. The speaker never engages with the frustration question, instead pivoting to meeting recall\. This is a refusal by redirection, which neither pass detected\.

✗ Pass 3 over\-upgrade: AMB→\\rightarrowCRGold: AMB∣\\midFinal: CRQ: Has the scandal damaged Hastert’s credibility and effectiveness in maintaining party control in the midterm elections? A: “No, I think the Speaker’s strong statements have made it clear to not only the party members but to the country that he wants to find out the facts…”Pass 1\(confidence 0\.9→\\rightarrowAMB\): “The speaker begins with a direct ‘No,’ but fails to substantively address the specific impact on ‘maintaining party control in the midterm elections\.’ Instead, the response deflects to a defense of the Speaker’s statements, general condemnation of the behavior, and a counter\-accusation\.”Pass 3\(upgrade→\\rightarrowCR\): “The first substantive sentence begins with ‘No,’ which is a direct answer to the question, and it is not followed by immediate qualifiers or negations like ‘but’ or ‘however’\.”Failure: Pass 1 correctly noted the “No” is followed by topic deflection rather than substantive engagement with the midterm elections premise\. Pass 3’s first\-sentence rule cannot distinguish a genuine “No” from a dismissive one\.