Improving Selective Classification with Pairwise Queries for Binary Classification

# Improving Selective Classification with Pairwise Queries for Binary Classification
Source: [https://arxiv.org/html/2605.30615](https://arxiv.org/html/2605.30615)
\[1\]\\fnmHarsh\\surVardhan[https://orcid.org/https://orcid.org/0000-0002-4656-3162](https://orcid.org/https://orcid.org/0000-0002-4656-3162)\[1\]\\orgdivCSE,\\orgnameUCSD,\\orgaddress\\street9500 Gilman Drive,\\cityLa Jolla,\\postcode92092,\\stateCA,\\countryUSA 2\]\\orgnameAdobe Research,\\orgaddress\\street345 Park Ave,\\citySan Jose,\\postcode95110,\\stateCA,\\countryUSA 3\]\\orgnameAdobe Research,\\orgaddress\\streetMarathahalli,\\cityBengaluru,\\postcode560087,\\stateKarnataka,\\countryIndia 4\]\\orgdivHDSI,\\orgnameUCSD,\\orgaddress\\street9500 Gilman Drive,\\cityLa Jolla,\\postcode92092,\\stateCA,\\countryUSA

###### Abstract

In selective classification, a model predicts the labels of data samples where it is confident, and abstains from predicting labels for samples on which it is not confident\. The rejected samples are often labeled by an expert, which is expensive\. The budget for the expert is best utilized when the model has low error on non\-rejected samples\. However, the estimate of a model’s confidence might be inconsistent with the model’s predictions, which can lead to high error on non\-rejected points\. Such situations can readily occur in in\-context binary classification by LLMs\. To remedy this, we propose making additional pairwise queries to the same model\. These pairwise queries can detect high\-error samples and be incorporated into selective classification techniques to reduce the error on non\-rejected samples\. Theoretically, we establish the conditions under which a simple algorithm using pairwise queries outperforms an inconsistent confidence estimate\. We support this insight through extensive experiments for11synthetic and44in\-context learning\-based real binary classification datasets\. In all these cases, we show that our algorithms, using pairwise queries, obtain a better accuracy\-cost tradeoff than using only the raw confidence estimates, for instance, the LLM’s next\-token logits\.

###### keywords:

Selective Classification, Pairwise Queries, Learning Theory, In\-Context Learning

## 1Introduction

In selective classification\[chow70,el\-yaniv10a\], a classifier is augmented with a rejection function such that it only predicts labels on the datapoints that are not rejected\. There are two complementary goals for selective classification: 1\) rejecting a small fraction of samples \(high coverage\), 2\) obtaining low error on samples that are not rejected \(low selective risk\)\. Balancing both these goals rules out trivial rejection functions that either reject everything or reject nothing\. If a fixed fraction of datapoints is rejected, the best selective classification algorithm should obtain the smallest possible selective risk\. This requires identifying samples where the classifier makes an error\. Without the knowledge of the true labels, these samples cannot be identified\. However, a viable approach is to reject datapoints on which the classifier is not confident in its prediction\. This approach has been widely adopted, for instance, using softmax confidences\[geifman17\], or the uncertainty of an ensemble’s predictions\[pmlr\-v48\-gal16\]\. Such high\-confidence or low\-uncertainty predictions are also desired in healthcare\[CRAIG202366\], fairness\[justice,fairness\]and finance\[DASTILE2020106263\]\.

If the measure of confidence is not correlated with the predicted labels, then selective classification based on confidence fails\. We find that this is often the case in a recent but growing field of work:in\-context learning\(ICL\) withlarge language models\(LLMs\) \(see Fig[1](https://arxiv.org/html/2605.30615#S1.F1.7)\)\. Consider the task of verifying the correctness of a Natural Language to SQL \(NL2SQL\) output by ICL on an LLM\[liu2024surveynl2sqllargelanguage,numbersstation2023NSText2SQL,yu\-etal\-2018\-spider\]\.

![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/bird_mot_example.png)Figure 1:Distribution of absolute difference of confidence from next token logits and ICL labeling function for verifying the correctness of NL2SQL, where the NL2SQL are from the Bird dataset\[li2024can\]and ICL labels and next token logits are from Gemma3 4B Instruct\. The threshold withpp\-value of5%5\\%is for the hypothesis that confidence and ICL labeling function are close, when averaged over5050random seeds\.In NL2SQL, each datapoint consists of a tuple of a natural language question for a database and a corresponding SQL on the database\. If the SQL can answer the natural language question, then the label is11, and it is0otherwise\. NL2SQL is an important use\-case of LLMs in business applications\[liu2024surveynl2sqllargelanguage,numbersstation2023NSText2SQL,zhang2023llmaaa\]\. Using selective classification with in\-context learning on an LLM, we can reject the NL2SQL pairs on which the classifier is least confident in its predictions, and delegate them to an expensive human annotator\. A natural confidence measure in this case is the classifier’s next token logits for the predicted label\[geifman17\]\.

Note that verifying correctness is a binary classification problem, so the confidence and bias of the ICL labeling function from in\-context learning are both in\[0,1\]\[0,1\]for any \(NL,SQL\) pair\. In Fig[1](https://arxiv.org/html/2605.30615#S1.F1.7), we plot the distribution of the absolute difference between these two functions for an NL2SQL dataset\[li2024can\]\. This distribution has a heavy tail for values close to 0\.5, implying very low correlation between them\. In Fig[1](https://arxiv.org/html/2605.30615#S1.F1.7), we see a peak when the confidence and ICL labeling bias differ by 0\.5\. In this scenario, selective classification will waste expensive human annotation by sending datapoints where the predicted labels are already correct\.

We address this problem in this paper:selective classification when the confidence function differs from the actual labeling function of the classifier\.

This problem has not been explored in existing works on selective classification\[el\-yaniv10a,geifman17,pugnana2024deepneuralnetworkbenchmarks\], but, their focus has been on simpler calibrated classifiers, and not on ICL\. Additionally, due to prohibitive sizes of the classifier, which for LLMs might be billions of parameters\[llama3modelcard\], training or retraining the model might be impossible, although efficient fine\-tuning might be possible\. Alternatively, the generated outputs in the previous example might be obtained from a closed\-source model with restrictive conditions on their usage\. For instance, OpenAI’s models\[openai2024gpt4\]prevent the use of its generated outputs in any training step\. Therefore, we need selective classification methods that do not require any training\. If we cannot train, several existing techniques used both in theory\[el\-yaniv10a,el\-yaniv12a\]and practice\[pugnana2024deepneuralnetworkbenchmarks\], including the learning\-to\-reject framework\[corbiere19,pmlr\-v97\-geifman19a,cortes\], cannot be applied\. In addition to this, ICL uses only a small number of labeled samples, which disqualifies recent model\-agnostic methods\[pmlr\-v206\-pugnana23a\]\.

We propose a solution circumventing all the above\-mentioned problems: a pairwise query\. In binary classification, a pairwise query would correspond to sending two unlabeled datapoints to the model and asking which of their labels is closer to the label11\.

For the NL2SQL example, where the label11corresponds to correctness, it would translate to asking which of the two pairs of natural language questions and corresponding SQLs is more correct\. This pairwise information is provided by the model directly, and it turns out to be more accurate than an unreliable confidence value\. This intuitively motivates selective classification with pairwise queries\. Additionally, 1\) pairwise queries are cheaper than the expert annotator, 2\) several models are amenable to pairwise queries, especially the LLM classifiers we discussed for the NL2SQL case and 3\) LLMs are more accurate on pairwise queries than direct labels on certain language tasks\[qin\-etal\-2024\-large\]\. The goal of this paper is to understandif pairwise queries can improve selective classification over thresholding an unreliable confidence estimate\. We answer this in the affirmative\.

### 1\.1Contributions

Our main contributions are listed below\.

1\. Pairwise Query\-based Selective Classification:For binary classification, we propose55different selective classification algorithms \(PairSel\-Middle, PairSel\-Max\-Entropy, PairSel\-Max\-Presence, PairSel\-Max\-Displacement, PairSel\-kNN\) that do not require retraining and use a sorting subroutine based on pairwise queries\. Of these, only PairSel\-kNN requires a distance metric on the feature space of the datapoints\.

2\. Theoretical Improvement:Under a mild assumption on the model’s predictions \(Assumption[3](https://arxiv.org/html/2605.30615#Thmtheorem3)in Section[4](https://arxiv.org/html/2605.30615#S4)\) that is satisfied by linear, generalized linear, and neural network models in the NTK regime\[jacot\_gabriel\_hongler\_2018\], we derive the necessary theoretical conditions under which the simplest of our pairwise algorithms, Pair\-Middle, outperforms vanilla confidence thresholding for spherical \(Theorem[4](https://arxiv.org/html/2605.30615#Thmtheorem4)\) and Gaussian features \(Theorem[5](https://arxiv.org/html/2605.30615#Thmtheorem5)\)\.

3\. Empirical Performance:We consider33tasks to test our techniques – i\) Synthetic dataset with linear classifier matching our theory, ii\) Verification of NL2SQL \(Spider\[yu\-etal\-2018\-spider\]and Bird\[li2024can\]\), iii\) Binary question answering on the BoolQ\[clark2019boolq\]and VisOnlyQA \(vision\)\[kamoi2025visonlyqa\]datasets\. For all the datasets except synthetic, we use several open\-source LLMs of size<10​B<10Bparameters as classifiers\. We show that our pairwise algorithms obtain better performance than 3 baselines that don’t use training –confidence thresholding on the next\-token logits, using the base model confidences for an instruct model, and post\-processing confidence calibration\[han2023prototypical\], for most values of coverage \(see Figures[3](https://arxiv.org/html/2605.30615#S5.F3)and Table[1](https://arxiv.org/html/2605.30615#S5.T1)\) or the labeling cost \(see Figure[6](https://arxiv.org/html/2605.30615#A1.F6)\)\. In particular, from Table[1](https://arxiv.org/html/2605.30615#S5.T1), the best pairwise methods have total accuracy atleast8%8\\%higher than the raw baseline, which corresponds to next\-token logits \(Synthetic, Bird and BoolQ\), while post\-processing calibration doesn’t always improve over the raw baseline \(Synthetic and VisOnlyQA in Table\), and base models are not always available \(VisOnlyQA and Llama3 SQLCoder\)\.

### 1\.2Related Works

##### Selective Classification

\[chow70\]introduced selective classification initially as a theoretical paradigm\. Subsequent work by\[grandvalet\_08,bartlett08a\]focused on obtaining rejection rules for SVMs by rejecting datapoints with small margins\. Fundamental theoretical advancements in selective classification were obtained by\[el\-yaniv10a\]and\[wiener11\], however, their proposed selective classification algorithms were not efficient in practice\. This inefficiency is due to subroutines in their algorithms that need to perform an exhaustive search on the space of all possible weights\.

A more recent line of work, focusing on the practical applications of selective classification, utilized different measures of uncertainty in the model’s predictions\.\[geifman17\]utilize the softmax confidences while\[pmlr\-v48\-gal16\]use the uncertainty of Bayesian NNs via MC dropout as a rejection function\. Other formulations of uncertainty estimation, for instance, using ensembles of models, have also been used to create rejection functions\[lakshminaryanan17\]\. While these work in practice, only a few concrete theoretical results\[pmlr\-v206\-pugnana23a,ding\_ensembles\]beyond the bayesian case are available\. Several works aim to learn this rejection function from data\[Cortes2023TheoryAA,corbiere19,pmlr\-v97\-geifman19a\], by including the rejection rate as an additional class or incorporating additional layers to predict the confidence score\. These fall into the learning to reject/abstain framework\.\[pugnana2024deepneuralnetworkbenchmarks\]provide a comprehensive benchmark of several selective classification methods on deep learning models, showing that softmax confidences and ensemble based techniques outperform most baselines in most metrics\. Note that none of the approaches beyond the bayesian case can be used in our setting, due to lack of model training\.\[ding\_ensembles\]requires a diverse ensemble of verifiers, which we don’t have and is hard to simulate from a single verifier\. Further, themodel\-agnosticapproaches of\[pmlr\-v206\-pugnana23a, Algorithms 1,2\]require a significant amount of labeled data to run, which we don’t have, as we only use a small batch2020samples for in\-context examples, which is less than0\.6%0\.6\\%of the size of our smallest dataset\. Additionally,\[vishwakarma\]propose a confidence\-threshold based autolabeling procedure that outperforms iterative selective classification, however, it requires training\.Another related field is that of semi\-supervised learning\(SSL\)\[vanEngelen2019\], where a combination of human\-labeled and unlabeled features is available, and the goal is to label the unlabeled features accurately\. However, SSL also requires training, and there are no labeled datapoints in our setup\. So, SSL techiques cannot be applied directly to our problem\. However, certain subroutines of SSL, especially label propagation\[vanEngelen2019\]can still be applied to propagate information from the classifier’s labels\. We use this in one of our algorithms in Section[3](https://arxiv.org/html/2605.30615#S3)\.

##### Active Learning and Selective Classification

A closely related problem to selective classification is active learning\. Similar to selective classification, in active learning, we also have an unlabeled dataset of features, and we need to select the best features to send for human labeling\. However, contrary to selective classification, the goal after human labeling is to learn a new classifier using these labeled datapoints\. Further, in active learning, this cycle of selecting the best unlabeled datapoints, human labeling, and subsequent re\-training of classifier is repeated several times, until a budget of human labels or a desired accuracy of the classifier is reached\. Thus, selective classification is a training\-free version of only a single step of active learning\[el\-yaniv12a\]\.\[el\-yaniv12a,gelbart2019\]show strong theoretical connections between theoretically optimal active learning and selective classfiication\.\[Hanneke2014TheoryOD\]provide a comprehensive overview of theoretically optimal active learning algorithms,\[ren\_survey\_2021\]provide an overview of deep\-learning based active learning algorithms,\[zhang\-etal\-2022\-survey\]cover recent applications of active learning to NLP\. The most relevant survey of active learning focused on LLMs is provided in\[xia\-etal\-2025\-selection\]\.

Due to their similarities, one may want to adapt techniques from active learning to selective classification\. Note that active learning necessitates training, which is computationally expensive in our case of an LLM as a classifier, thus disqualifying all active learning techniques, including the cost\-sensitive variants\[ijcai2017p261,Settles2008ActiveLW\]\. However, we can adapt active learning techniques used to select samples for human labeling\. From\[xia\-etal\-2025\-selection, Section 3\.2\], SoTA techniques for selecting most informative samples use the LLM logits, or ask the LLM to manually rank the difficulty of an example, or use consistency of predicted labels among a committee of LLMs\. Note that a committee is only useful\[Settles2012\], if the members are sufficiently different, which is hard to ensure with a single classifier\. Manually ranking a large dataset \(≈1000\\approx 1000samples\) of NL2SQL queries, that are each atleast a few10001000tokens already leads to a context length of a few million tokens, which is only achieved by SoTA models\[openai2024gpt4,gemmateam2025gemma3technicalreport,bai2025qwen25vltechnicalreport\]\. Our pairwise queries are a simpler and more token\-efficient technique to rank the difficulty of LLM examples\.

##### In\-context learning and pairwise queries

To obtain predictions from the LLM verifier, we use in\-context learning\[dong2024surveyincontextlearning\], where a few labeled examples are provided as context along with an unlabeled feature, and the LLM predicts the label for this unlabeled feature\. There are existing methods to calibrate LLM confidences – post\-processing of logits\[han2023prototypical\]and prompt\-sensitivity approaches are utilized\[chen\-etal\-2023\-relation\]\. We found that dropout\-based prompt\-sensitivity for our problems performs much worse than calibration for all our datasets, so we do not report its results, and post\-processing calibration is used as a baseline in our experiments\.\[openai2024gpt4\]point out that base models are well calibrated, while instruct models are poorly calibrated\. We utilize this to obtain another baseline\. Pairwise queries form a key subroutine of our proposed methods, and existing works show that LLMs are better at pairwise predictions than direct labeling\[qin\-etal\-2024\-large\]\.Note that pairwise queries have been used in theoretically optimal active learning algorithms\[pmlr\-v134\-hopkins21a,coarse\_labels,pmlr\-v125\-hopkins20a,kane2017active\]\. However, these algorithms cannot be implemented efficiently for our case\. For instance,\[kane2017active,pmlr\-v125\-hopkins20a,pmlr\-v134\-hopkins21a\]require the knowledge of the inference dimension of the problem to implement their optimal active learning algorithm\. Computing the inference dimension is at least as hard as computing the VC dimension of a problem\. For our LLM models, we can either use an extremely loose bound on this dimension, which makes the actual active learning algorithm suboptimal, or we estimate it accurately, which itself is computationally intractable\. Further,\[pmlr\-v125\-hopkins20a, Algorithm 1\]requires solving a linear program, and\[pmlr\-v134\-hopkins21a,kane2017active\]have efficient algorithms only for the case of learning halfspaces\.

OrganizationIn Section[2](https://arxiv.org/html/2605.30615#S2), we formally define our problem, and in Section[3](https://arxiv.org/html/2605.30615#S3), we describe the baseline confidence threshold algorithm and pairwise query\-based algorithms\. In Section[4](https://arxiv.org/html/2605.30615#S4), we establish theoretical conditions under which pairwise queries are better than using confidence, and in Section[5](https://arxiv.org/html/2605.30615#S5), we justify this insight via extensive experiments\.

## 2Problem Setup

Dataset\.We assume that we have an unlabeled dataset\{xi\}i=1n\\\{x\_\{i\}\\\}\_\{i=1\}^\{n\}ofnndatapoints\. Here, eachxi∈𝒳x\_\{i\}\\in\{\\mathcal\{X\}\}is a feature, obtained from the feature space𝒳\{\\mathcal\{X\}\}and sampled iid from the feature distribution𝒟x\{\\mathcal\{D\}\}\_\{x\}, i\.e\.,xi​∼i​i​d​𝒟xx\_\{i\}\\overset\{iid\}\{\\sim\}\{\\mathcal\{D\}\}\_\{x\}\. Throughout this paper, we consider the task of binary classification\. So, we want to assign a labelyi∈\{0,1\}y\_\{i\}\\in\\\{0,1\\\}to each datapointxix\_\{i\}wherei∈\[n\]i\\in\[n\]\. We use\[n\]\[n\]to denote the set\{1,2,…,n\}\.\\\{1,2,\\ldots,n\\\}\.

##### Labeling Function

For binary classification, we can model any classifier as a labeling functionp:𝒳→\[0,1\]p:\{\\mathcal\{X\}\}\\to\[0,1\]such that for a given featurex∈𝒳x\\in\{\\mathcal\{X\}\}, the labelyyis generated from the probabilistic modely∼Ber​\(p​\(x\)\)y\\sim\\mathrm\{Ber\}\(p\(x\)\)\. We use this modeling to represent the in\-context labeling functionpclf,lp\_\{\\mathrm\{clf\},l\}, the confidence functionpclf,cp\_\{\\mathrm\{clf\},c\}, the human labelerphp\_\{h\}as well as the true labelsp⋆p^\{\\star\}\.

Classifier\.Our classifier can be represented by two functionspclf,l,pclf,c:𝒳→\[0,1\]p\_\{\\mathrm\{clf\},l\},p\_\{\\mathrm\{clf\},c\}:\{\\mathcal\{X\}\}\\to\[0,1\]\. Here,clf\\mathrm\{clf\}is used to denote the classifier andllandccare used to denote the labeling and confidence functions\. For any given datapointxi,i∈\[n\]x\_\{i\},\\,i\\in\[n\], the classifier generates it’s predicted labely^i\\hat\{y\}\_\{i\}asy^i∼Ber​\(pclf,l​\(x\)\)\\hat\{y\}\_\{i\}\\sim\\mathrm\{Ber\}\(p\_\{\\mathrm\{clf\},l\}\(x\)\)\. The classifier’s confidence in the labely^i\\hat\{y\}\_\{i\}is represented bypclf,c​\(xi\)p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)\. We assume that the labeling function and the confidence function are differentpclf,c≠pclf,lp\_\{\\mathrm\{clf\},c\}\\neq p\_\{\\mathrm\{clf\},l\}\. This is the weakest notion of inconsistency and we will use stronger ones in our theoretical results \(Theorems[4](https://arxiv.org/html/2605.30615#Thmtheorem4)and[5](https://arxiv.org/html/2605.30615#Thmtheorem5)\)\. For the example of in\-context learning,y^i\\hat\{y\}\_\{i\}corresponds to the in\-context label andpclf,c​\(xi\)p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)corresponds to logits\-based confidence\. Additionally, we would never have access to the actual functionspclf,l,pclf,cp\_\{\\mathrm\{clf\},l\},p\_\{\\mathrm\{clf\},c\}, apart from the labels and the confidence generated by them\.

Annotators\.The classifier can reject a subset of the features\. These are sent to the expert \(also referred to as human\), parameterized by its labeling functionph:𝒳→\[0,1\]p\_\{h\}:\{\\mathcal\{X\}\}\\to\[0,1\]such that the expert’s label isy~i∼Ber​\(ph​\(xi\)\),∀i∈\[n\]\\tilde\{y\}\_\{i\}\\sim\\mathrm\{Ber\}\(p\_\{h\}\(x\_\{i\}\)\),\\forall i\\in\[n\]\. The true labels for the problem are generated from a true labeling functionp⋆:𝒳→\[0,1\]p^\{\\star\}:\{\\mathcal\{X\}\}\\to\[0,1\], such thatyi⋆∼Ber​\(p⋆​\(xi\)\),∀i∈\[n\]y\_\{i\}^\{\\star\}\\sim\\mathrm\{Ber\}\(p^\{\\star\}\(x\_\{i\}\)\),\\forall i\\in\[n\]\.

##### Selective Risk

Our main metric will be the fraction of points misclassified, which we will call the empirical risk or the empirical misclassification\. Using only the classifier, the empirical misclassificaiton isfclf​\(n\)≜1n​∑i=1n𝟏​\{y^i≠yi⋆\}f\_\{\\mathrm\{clf\}\}\(n\)\\triangleq\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\{\\bm\{1\}\}\\\{\\hat\{y\}\_\{i\}\\neq y\_\{i\}^\{\\star\}\\\}\. For the expert/human, this isfh​\(n\)≜1n​∑i=1n𝟏​\{y~i≠yi⋆\}f\_\{h\}\(n\)\\triangleq\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\{\\bm\{1\}\}\\\{\\tilde\{y\}\_\{i\}\\neq y\_\{i\}^\{\\star\}\\\}\. For all our selective classification methods, we will fix the fraction of datapoints to be rejected asα∈\{0,1n,2n,…,1\}\\alpha\\in\\\{0,\\frac\{1\}\{n\},\\frac\{2\}\{n\},\\ldots,1\\\}\. Each selective classification method selects a subset of datapointsSα⊆\[n\]S\_\{\\alpha\}\\subseteq\[n\], that are sent to the expert, where\|Sα\|=α​n\\lvert S\_\{\\alpha\}\\rvert=\\alpha n\. To measure the performance of any selective classification method, we define the selective risk as

fSα​\(n\)=1n​∑i=1n\(𝟏​\{y^i≠yi⋆,i∉Sα\}\+𝟏​\{y~i≠yi⋆,i∈Sα\}\)\\displaystyle f\_\{S\_\{\\alpha\}\}\(n\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\(\{\\bm\{1\}\}\\\{\\hat\{y\}\_\{i\}\\neq y\_\{i\}^\{\\star\},i\\notin S\_\{\\alpha\}\\\}\+\{\\bm\{1\}\}\\\{\\tilde\{y\}\_\{i\}\\neq y\_\{i\}^\{\\star\},i\\in S\_\{\\alpha\}\\\}\)Note that for all the datapoints that have been selected, i\.e\.i∈Sαi\\in S\_\{\\alpha\}, the predicted label is the expert labely~i\\tilde\{y\}\_\{i\}, while for the rest of the datapoints,i∉Sαi\\notin S\_\{\\alpha\}, it is the classifier labely^i\\hat\{y\}\_\{i\}\. Further, we can define the population risks of the classifier, human and any selective classification method as the expectation of the corresponding empirical risks with respect to the labeling distribution and the feature distribution\. LetFclf≜𝔼xi,y^i,yi⋆​\[fclf​\(n\)\]F\_\{\\mathrm\{clf\}\}\\triangleq\\mathbb\{E\}\_\{x\_\{i\},\\hat\{y\}\_\{i\},y^\{\\star\}\_\{i\}\}\[f\_\{\\mathrm\{clf\}\}\(n\)\]andFh≜𝔼xi,y~i,yi⋆​\[fh​\(n\)\]F\_\{h\}\\triangleq\\mathbb\{E\}\_\{x\_\{i\},\\tilde\{y\}\_\{i\},y^\{\\star\}\_\{i\}\}\[f\_\{h\}\(n\)\]denote population risk for classifier and human respectively\.

##### Pairwise Queries\.

For two datapointsxi,xj∈𝒳x\_\{i\},x\_\{j\}\\in\{\\mathcal\{X\}\}, wherexi≠xjx\_\{i\}\\neq x\_\{j\}, the pairwise query to the classifier measures the differencepclf,l​\(xi\)−pclf,l​\(xj\)p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\)\-p\_\{\\mathrm\{clf\},l\}\(x\_\{j\}\)\. Conditioned on the correct labelsyi⋆,yj⋆=1y\_\{i\}^\{\\star\},y\_\{j\}^\{\\star\}=1, if the difference is large and positive, thenxix\_\{i\}has a higher bias and is thus more correct thanxjx\_\{j\}\. In this case, the pairwise query returns11\. If the difference is large and negative, thenxjx\_\{j\}is more correct and the pairwise query returns0\. We provide a more precise definition of pairwise queries for our theoretical results in Definition[1](https://arxiv.org/html/2605.30615#Thmdefinition1)\.

For any selective classification method, our goal is to minimize its corresponding selective riskESα,xi,y^i,y~i,yi⋆​\[fSα​\(n\)\]E\_\{S\_\{\\alpha\},x\_\{i\},\\hat\{y\}\_\{i\},\\tilde\{y\}\_\{i\},y\_\{i\}^\{\\star\}\}\[f\_\{S\_\{\\alpha\}\}\(n\)\]for a fixedα\\alpha\. As the corresponding empirical errors are sum of indicator random variables, by Chernoff bound, they differ from their population counterparts only by𝒪​\(1/n\)\\mathcal\{O\}\(1/\\sqrt\{n\}\)with high probability\.

## 3Proposed Algorithms

In this section, we describe all our proposed selective classification algorithms\. This includes the existing confidence\-based thresholding baseline, which we call ConfSel, and the family of methods using pairwise queries, which we call PairSel\.

##### ConfSel

First, we describe the baseline algorithm, ConfSel, in Algorithm[1](https://arxiv.org/html/2605.30615#alg1), that rejects the least confident datapoints, computed usingpclf,cp\_\{\\mathrm\{clf\},c\}\. For binary classification, ifpclf,c​\(xi\)p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)for somei∈\[n\]i\\in\[n\], is close to11, then the classifier is very confident in it’s label being11\. However, ifpclf,c​\(xi\)p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)is close to0, the classifier is confident in the label being0\. Therefore,\|pclf,c​\(xi\)−12\|\\lvert p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)\-\\frac\{1\}\{2\}\\rvertis a label\-independent measure of confidence in the classifier’s predictions\. Forpclf,c​\(xi\)p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)close to either0or11, our confidence measure achieves it’s maximum value of12\\frac\{1\}\{2\}\. Ifpclf,c​\(xi\)p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)is close to12\\frac\{1\}\{2\}, then the confidence measure is close to0, it’s minimum value\. The algorithm ConfSel selects datapoints that have the smallest confidence measure in the dataset\.

Algorithm 1ConfSelSet of datapoints

\{xi\}i=1n\\\{x\_\{i\}\\\}\_\{i=1\}^\{n\}, Fraction of datapoints to send to human

α∈\[0,1\]\\alpha\\in\[0,1\]
Compute

pclf,c​\(xi\)p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)for each datapoint

xi,i∈\[n\]x\_\{i\},\\,i\\in\[n\]\.

Sort the datapoints according to

\|pclf,c​\(xi\)−12\|\\lvert p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)\-\\frac\{1\}\{2\}\\rvert\.

Let

Sα,cS\_\{\\alpha,c\}be the set of datapoints in the smallest

α\\alpha\-quantile according to

\|pclf,c​\(xi\)−12\|\\lvert p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)\-\\frac\{1\}\{2\}\\rvert\.

Return

Sα,cS\_\{\\alpha,c\}

Algorithm 2PairSelSet of data points

\{xi\}i=1n\\\{x\_\{i\}\\\}\_\{i=1\}^\{n\}, Fraction of datapoints sent to human

α∈\[0,1\]\\alpha\\in\[0,1\]\.

Sort the datapoints

\{xi\}i=1n\\\{x\_\{i\}\\\}\_\{i=1\}^\{n\}in descending order of

pclf,lp\_\{\\mathrm\{clf\},l\}using Mergesort with

𝒪​\(n​log⁡n\)\\mathcal\{O\}\(n\\log n\)pairwise queries\.

Let

Π:\[n\]→\[n\]\\Pi:\[n\]\\to\[n\]be the permutation which sorts the datapoints and

Π−1:\[n\]→\[n\]\\Pi^\{\-1\}:\[n\]\\to\[n\]be its inverse permutation\.

Sα,p←S\_\{\\alpha,p\}\\leftarrowPairSel

−⋆\(\)\-\\star\(\)\# Here,

⋆∈\{\\star\\in\\\{Middle, Max\-Entropy, Max\-Presence, Max\-Displacement, kNN

\}\\\}
Return

Sα,pS\_\{\\alpha,p\}

##### PairSel

We define a class of algorithms based on pairwise queries in Algorithm[2](https://arxiv.org/html/2605.30615#alg2), which are broadly referred to as the PairSel algorithms\. The core idea behind all these algorithms is to use a sorting of the datapoints via pairwise queries\. We first describe how to obtain this sorting using pairwise queries\.

For each pair of datapointsxi,xjx\_\{i\},x\_\{j\}, a pairwise query provides us with a comparison function betweenxix\_\{i\}andxjx\_\{j\}\. The pairwise query output forxi,xjx\_\{i\},x\_\{j\}is either0or11\. If this comparison function is consistent \(anti\-symmetric and transitive\[clrs, Appendix B\.2\]\), it can be used to sort any set ofnndatapoints\. We can use a comparison\-based sorting algorithm, for instance MergeSort\[clrs\], for this purpose, where each comparison is performed with a pairwise query\. From Definition[1](https://arxiv.org/html/2605.30615#Thmdefinition1), pairwise queries are determined by the true labeling function of the classifier,pclf,lp\_\{\\mathrm\{clf\},l\}, instead of the incorrect confidence values obtained by the confidence functionpclf,cp\_\{\\mathrm\{clf\},c\}that are used in ConfSel\. Using the comparison\-based sorting with pairwise queries as the comparison function, we can sort the datapoints in descending order ofpclf,lp\_\{\\mathrm\{clf\},l\}\. This provides a sorting permutationΠ\\Piand forms a subroutine for all pairwise query\-based algorithms in Algorithm[2](https://arxiv.org/html/2605.30615#alg2)\. Note that we don’t need any additional structure on the labeling functions or the space of features𝒳\\mathcal\{X\}, to either run our algorithms or for our pairwise queries to be consistent\.

We propose55methods for the actual rejection function based on this sorting\. We now explain the intuition behind each of these methods\. Our algorithms are created assuming pairwise queries are mostly accurate\.

MiddleIf the classifier’s labels have a balanced distribution of classes in expectation, i\.e\.,𝔼x∼𝒟x​\[pclf,l​\(x\)\]=12\\mathbb\{E\}\_\{x\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x\)\]=\\frac\{1\}\{2\}, then the values in the middleα\\alpha\-fraction of iid samples sorted according topclf,lp\_\{\\mathrm\{clf\},l\}should be close to12\\frac\{1\}\{2\}\. These are exactly the points on which the true labeling function has least confidence\.

Algorithm 3PairSel\-Middle\(\)Return

\{Π−1​\(i\):i∈\{⌊n​\(1−α\)2⌋,⌈n​\(1\+α\)2⌉\}\}\\\{\\Pi^\{\-1\}\(i\):i\\in\\\{\\lfloor\\frac\{n\(1\-\\alpha\)\}\{2\}\\rfloor,\\lceil\\frac\{n\(1\+\\alpha\)\}\{2\}\\rceil\\\}\\\}
Max EntropyNote that the functionHb​i​n​\(B\)H\_\{bin\}\(B\)for a sequenceBBconsisting of0’s and11’s is the empirical binary entropy function\. IfmBm\_\{B\}is the number of11’s inBB, then,Hb​i​n​\(B\)=−mB\|B\|​log⁡\(mB\|B\|\)−\(1−mB\|B\|\)​log⁡\(1−mB\|B\|\)H\_\{bin\}\(B\)=\-\\frac\{m\_\{B\}\}\{\\lvert B\\rvert\}\\log\\left\(\\frac\{m\_\{B\}\}\{\\lvert B\\rvert\}\\right\)\-\\left\(1\-\\frac\{m\_\{B\}\}\{\\lvert B\\rvert\}\\right\)\\log\\left\(1\-\\frac\{m\_\{B\}\}\{\\lvert B\\rvert\}\\right\)\. If𝔼x∼𝒟x​\[pclf,l​\(x\)\]≠12\\mathbb\{E\}\_\{x\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x\)\]\\neq\\frac\{1\}\{2\}, then the middle is not an appropriate choice\. However, assuming the sorting according to pairwise queries are accurate, the contiguous subsequence of the sorted datapoints wherepclf,lp\_\{\\mathrm\{clf\},l\}is close to12\\frac\{1\}\{2\}, should have approximately equal number of0’s and11’s\. This contiguous subsequence should thus have highest empirical binary entropy\.

Algorithm 4PairSel\-Max\-Entropy\(\)i⋆=arg⁡maxi∈\[\(1−α\)​n\]∪\{0\}⁡Hb​i​n​\(\{y^Π−1​\(i\+k\)\}k=1α​n\)i^\{\\star\}=\\arg\\max\_\{i\\in\[\(1\-\\alpha\)n\]\\cup\\\{0\\\}\}H\_\{bin\}\(\\\{\\hat\{y\}\_\{\\Pi^\{\-1\}\(i\+k\)\}\\\}\_\{k=1\}^\{\\alpha n\}\)

Return

\{Π−1​\(i⋆\+k\):k∈\[α​n\]\}\\\{\\Pi^\{\-1\}\(i^\{\\star\}\+k\):k\\in\[\\alpha n\]\\\}

Max PresenceIn this method, we find the largest contiguous subsequence of the datapoints sorted bypclf,lp\_\{\\mathrm\{clf\},l\}such that it agrees with set of datapoints rejected by the confidence queriesSα,cS\_\{\\alpha,c\}\. This method assumes that pairwise queries might be inaccurate but they can be combined with inaccurate confidence estimates for better performance overall\.

Algorithm 5PairSel\-Max\-Presence\(\)i⋆=arg⁡maxi∈\[\(1−α\)​n\]∪\{0\}​∑k=1α​n𝟏​\{Π−1​\(i\)∈Sc,α\}i^\{\\star\}=\\arg\\max\_\{i\\in\[\(1\-\\alpha\)n\]\\cup\\\{0\\\}\}\\sum\_\{k=1\}^\{\\alpha n\}\{\\bm\{1\}\}\\\{\\Pi^\{\-1\}\(i\)\\in S\_\{c,\\alpha\}\\\}\# Here,

𝟏\{\\bm\{1\}\}is the indicator function\.

Return

\{Π−1​\(i⋆\+k\):k∈\[α​n\]\}\\\{\\Pi^\{\-1\}\(i^\{\\star\}\+k\):k\\in\[\\alpha n\]\\\}

Max DisplacementThis method measures the difference between the positions of the sorting by confidence values and pairwise queries\. If the labely^i=1\\hat\{y\}\_\{i\}=1, thenΠc​\(i\)−Π​\(i\)\\Pi\_\{c\}\(i\)\-\\Pi\(i\)is positive if a datapointxix\_\{i\}has smaller bias, relative to other datapoints, based onpclf,cp\_\{\\mathrm\{clf\},c\}thanpclf,lp\_\{\\mathrm\{clf\},l\}\. In this case,mim\_\{i\}is large\. Similar logic applies fory^i=0\\hat\{y\}\_\{i\}=0\. Note thatmim\_\{i\}is large whenever the pairwise sorting differs from the confidence values based on predicted labels and this method rejects such datapoints\.

Algorithm 6PairSel\-Max\-Displacement\(\)Let

Πc:\[n\]→\[n\]\\Pi\_\{c\}:\[n\]\\to\[n\]be the permutation obtained by sorting the datapoints in descending order according to

pclf,cp\_\{\\mathrm\{clf\},c\}\.

Let

mi=\(Πc​\(i\)−Π​\(i\)\)​\(2​y^i−1\)m\_\{i\}=\(\\Pi\_\{c\}\(i\)\-\\Pi\(i\)\)\(2\\hat\{y\}\_\{i\}\-1\)
ReturnSet of datapoints in the largest

α\\alpha\-quantile according to

mim\_\{i\}

kNNThis method assumes access to a distance metric between datapoints, namelydist\. The metricmim\_\{i\}penalizes two quantities – i\) the average distance of a datapoint to itskknearest neighbors indist, ii\) the average confidence of a datapoint’skknearest neighbors in their labels\. The average distance ensures that isolated datapoints are rejected\. Note that the second term ofmim\_\{i\}evaluates to\(1−pclf,c​\(xj\)\)\(1\-p\_\{\\mathrm\{clf\},c\}\(x\_\{j\}\)\)wheny^j=1\\hat\{y\}\_\{j\}=1andpclf,c​\(xj\)p\_\{\\mathrm\{clf\},c\}\(x\_\{j\}\)wheny^j=0\\hat\{y\}\_\{j\}=0\. Therefore, if the datapointxjx\_\{j\}is confident in its predicted label, then the corresponding terms should be small\. If a datapoint’s neighbors are not confident in their labels according topclf,cp\_\{\\mathrm\{clf\},c\}, which might be due to incoherence ofpclf,lp\_\{\\mathrm\{clf\},l\}andpclf,lp\_\{\\mathrm\{clf\},l\}, then the metricmim\_\{i\}is large\.

Algorithm 7PairSel\-kNN\(\)Let

Nk​\(xi\)⊂\[n\]∖\{i\}N\_\{k\}\(x\_\{i\}\)\\subset\[n\]\\setminus\\\{i\\\}be the set of

kknearest neighbors of

xix\_\{i\}according to distance metricdiston features

xx\.

Let

mi=1k​∑j∈Nk​\(xi\)dist​\(xj,xi\)​\(\(1−2​pclf,c​\(xj\)\)​y^j\+pclf,c​\(xj\)\)m\_\{i\}=\\frac\{1\}\{k\}\\sum\_\{j\\in N\_\{k\}\(x\_\{i\}\)\}\\text\{dist\}\(x\_\{j\},x\_\{i\}\)\(\(1\-2p\_\{\\mathrm\{clf\},c\}\(x\_\{j\}\)\)\\hat\{y\}\_\{j\}\+p\_\{\\mathrm\{clf\},c\}\(x\_\{j\}\)\)
i⋆=arg⁡maxi∈\[\(1−α\)​n\]∪\{0\}​∑j=1α​nmΠ−1​\(i\+j\)i^\{\\star\}=\\arg\\max\_\{i\\in\[\(1\-\\alpha\)n\]\\cup\\\{0\\\}\}\\sum\_\{j=1\}^\{\\alpha n\}m\_\{\\Pi^\{\-1\}\(i\+j\)\}

Return

\{Π−1​\(i⋆\+j\):j∈\[α​n\]\}\\\{\\Pi^\{\-1\}\(i^\{\\star\}\+j\):j\\in\[\\alpha n\]\\\}

Note that PairSel\-Middle is the simplest of the pairwise queries and PairSel\-kNN is the most complicated one and only PairSel\-kNN requires access to a distance metric\.

## 4Why Pairwise Queries are Beneficial?

In this section, we find conditions under which the simplest pairwise query\-based algorithm, PairSel\-Middle from Algorithm[2](https://arxiv.org/html/2605.30615#alg2)obtains lower total selective risk than that of ConfSel \(Algorithm[1](https://arxiv.org/html/2605.30615#alg1)\)\. The proofs for all the theoretical results in this section are deferred to Appendix[C](https://arxiv.org/html/2605.30615#A3)\.

###### Lemma 1\.

LetFclf,c​\(α\)=𝔼xi,y^i,y~i,yi⋆​\[fSα,c​\(n\)\]F\_\{\\mathrm\{clf\},c\}\(\\alpha\)=\\mathbb\{E\}\_\{x\_\{i\},\\hat\{y\}\_\{i\},\\tilde\{y\}\_\{i\},y^\{\\star\}\_\{i\}\}\[f\_\{S\_\{\\alpha,c\}\}\(n\)\]be population selective risk of ConfSel \(Algorithm[1](https://arxiv.org/html/2605.30615#alg1)\) for a fixedα∈\{0,1n,2n​…,1−1n,1\}\\alpha\\in\\\{0,\\frac\{1\}\{n\},\\frac\{2\}\{n\}\\ldots,1\-\\frac\{1\}\{n\},1\\\}\. Then,Fclf−Fclf,c​\(α\)=F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)=

𝔼x∼𝒟x\[\(2p⋆\(x\)−1\)\(ph\(x\)−pclf,l\(x\)\)⋅\(Prx′∼𝒟x\[\|2pclf,c\(x′\)−1\|≥\|2pclf,c\(x\)−1\|\]\)\(1−α\)​n\]\.\\displaystyle\\mathbb\{E\}\_\{x\\sim\{\\mathcal\{D\}\}\_\{x\}\}\\bigg\[\(2p^\{\\star\}\(x\)\-1\)\(p\_\{h\}\(x\)\-p\_\{\\mathrm\{clf\},l\}\(x\)\)\\quad\\cdot\\left\(\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]\\right\)^\{\(1\-\\alpha\)n\}\\bigg\]\.

To quantify the population selective risk for PairSel\-Middle, we first define our pairwise query oracle in Defintion[1](https://arxiv.org/html/2605.30615#Thmdefinition1)\.

###### Definition 1\(Pairwise Query Oracle\)\.

A pairwise query oracle, given two pointsx,x′∈𝒳x,x^\{\\prime\}\\in\{\\mathcal\{X\}\}on the classifierclf\\mathrm\{clf\}, will provide11ifp,l​\(x\)≥pclf,l​\(x′\)p\_\{,l\}\(x\)\\geq p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)and0otherwise\.

This oracle assumes that pairwise queries are always accurate\. This greatly simplifies the analysis of pairwise queries and is still realistic, as often in real models, including LLMs\[qin\-etal\-2024\-large\], pairwise queries are more accurate than direct queries\.

###### Lemma 2\.

LetFclf,p​\(α\)=𝔼xi,y^i,y~i,yi⋆​\[fSα,p​\(n\)\]F\_\{\\mathrm\{clf\},p\}\(\\alpha\)=\\mathbb\{E\}\_\{x\_\{i\},\\hat\{y\}\_\{i\},\\tilde\{y\}\_\{i\},y^\{\\star\}\_\{i\}\}\[f\_\{S\_\{\\alpha,p\}\}\(n\)\]be the population selective risk of PairSel\-Middle \(Algorithm[2](https://arxiv.org/html/2605.30615#alg2)\) for a fixedα∈\{0,1n,2n​…,1−1n,1\}\\alpha\\in\\\{0,\\frac\{1\}\{n\},\\frac\{2\}\{n\}\\ldots,1\-\\frac\{1\}\{n\},1\\\}\. Then, with access to a pairwise query oracle \(Definition[1](https://arxiv.org/html/2605.30615#Thmdefinition1)\),Fclf−Fclf,p​\(α\)=F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)=

𝔼x∼𝒟x\[\(2p⋆\(x\)−1\)\(ph\(x\)−pclf,l\(x\)\)\(Prx′∼𝒟x\[pclf,l\(x′\)≤pclf,l\(x\)\]\)⌈\(1−α\)​n/2⌉\\displaystyle\\mathbb\{E\}\_\{x\\sim\{\\mathcal\{D\}\}\_\{x\}\}\\bigg\[\(2p^\{\\star\}\(x\)\-1\)\(p\_\{h\}\(x\)\-p\_\{\\mathrm\{clf\},l\}\(x\)\)\\left\(\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]\\right\)^\{\\lceil\(1\-\\alpha\)n/2\\rceil\}⋅\(Prx′∼𝒟x\[pclf,l\(x′\)≥pclf,l\(x\)\]\)⌊\(1−α\)​n/2⌋\]\.\\displaystyle\\cdot\\left\(\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]\\right\)^\{\\lfloor\(1\-\\alpha\)n/2\\rfloor\}\\bigg\]\.

Note that both Lemmas[1](https://arxiv.org/html/2605.30615#Thmtheorem1)and[2](https://arxiv.org/html/2605.30615#Thmtheorem2)have a term in common,\(2​p⋆​\(x\)−1\)​\(ph​\(x\)−pclf,l​\(x\)\)\(2p^\{\\star\}\(x\)\-1\)\(p\_\{h\}\(x\)\-p\_\{\\mathrm\{clf\},l\}\(x\)\)\. If the true label was11for somex∈𝒳x\\in\{\\mathcal\{X\}\},p⋆​\(x\)\>12p^\{\\star\}\(x\)\>\\frac\{1\}\{2\}\. If the human is more accurate on the pointxx, i\.e\.,ph​\(x\)\>pclf,l​\(x\)p\_\{h\}\(x\)\>p\_\{\\mathrm\{clf\},l\}\(x\), it implies better selective risk for both ConfSel and PairSel\-Middle\. However, the two Lemmas differ in their additional terms with an exponent of\(1−α\)​n\(1\-\\alpha\)n, with ConfSel’s corresponding term depending only on the distribution ofpclf,cp\_\{\\mathrm\{clf\},c\}, while for PairSel\-Middle, the corresponding term depends on the distribution ofpclf,lp\_\{\\mathrm\{clf\},l\}\. This additional term for a givenx∈𝒳x\\in\{\\mathcal\{X\}\}corresponds to the probability of being rejected\.

To prove benefit of pairwise querires, we need to show thatFclf,p​\(α\)≤Fclf,c​\(α\)F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\\leq F\_\{\\mathrm\{clf\},c\}\(\\alpha\)for some fixedα\\alpha\. For arbitrarypclf,l,p⋆,php\_\{\\mathrm\{clf\},l\},p^\{\\star\},p\_\{h\}andpclf,cp\_\{\\mathrm\{clf\},c\}, we can only obtain a strong sufficient condition, which we discuss in Appendix[B](https://arxiv.org/html/2605.30615#A2)\. For weaker conditions on these labeling functions, we restrict them to a specific class of functions described by the following assumption\.

###### Assumption 3\(Classifier\)\.

We assume that there exists a feature mappingφ:𝒳→ℝd\\varphi:\{\\mathcal\{X\}\}\\to\\mathbb\{R\}^\{d\}, whereddis the ambient dimension of the classifier, modelsw⋆,wh,wclf,l,wclf,c∈ℝdw^\{\\star\},w\_\{h\},w\_\{\\mathrm\{clf\},l\},w\_\{\\mathrm\{clf\},c\}\\in\\mathbb\{R\}^\{d\}each of unit norm, and a differentiable non\-decreasing link functiong:ℝ→\[0,1\]g:\\mathbb\{R\}\\to\[0,1\]withg​\(a\)=1−g​\(−a\),∀a∈ℝg\(a\)=1\-g\(\-a\),\\forall a\\in\\mathbb\{R\}, such that∀x∈𝒳\\forall x\\in\{\\mathcal\{X\}\},p⋆​\(x\)=g​\(⟨w⋆,φ​\(x\)⟩\)p^\{\\star\}\(x\)=g\(\\langle w^\{\\star\},\\varphi\(x\)\\rangle\),ph​\(x\)=g​\(⟨wh,φ​\(x\)⟩\)p\_\{h\}\(x\)=g\(\\langle w\_\{h\},\\varphi\(x\)\\rangle\),pclf,l​\(x\)=g​\(⟨wclf,l,φ​\(x\)⟩\)p\_\{\\mathrm\{clf\},l\}\(x\)=g\(\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x\)\\rangle\)andpclf,c​\(x\)=g​\(⟨wclf,c,φ​\(x\)⟩\)p\_\{\\mathrm\{clf\},c\}\(x\)=g\(\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\)\.

A common example ofggis the sigmoid function\(1\+e−a\)−1\(1\+e^\{\-a\}\)^\{\-1\}\. The above assumption is satisfied if the classifier is a linear or a generalized linear model with link functiongg\. For kernel machines, the inner product in above assumption can be expressed as the sum of kernel functions on the inputxx\. Neural networks with large width, that have been trained with a small learning rate, or whose model weights remain close to initialization after training, satisfy the Neural Tangent Kernel \(NTK\) assumption\[jacot\_gabriel\_hongler\_2018,chizat\_bach\_2019\]\. The output of these classifiers can be represented as a linear function of the weights with a Neural Tangent Kernel, and thus satisfies the above assumptions\.

The above assumption also assumes that the feature space𝒳\\mathcal\{X\}can be transformed to the metric spaceℝd\\mathbb\{R\}^\{d\}via the feature mappingφ\\varphi\. Note that such a mapping, which ensures that pairwise queries are consistent and well\-defined, always exists if we consider the complete space of features\. We provide an example of what this feature mappingφ\\varphimight look like\. Suppose we want to classify animals based on their number of legs and their body size\. Letφ\\varphimapxx\(animals\) tod=2d=2coordinates, one for the number of legs, and another for body size, such that animals with different number of legs, for instance cat and bird, and animals with different sizes, for instance lion and mouse, can be differentiated based on pairwise queries\. Let the LLM model bewclf,l⊤=\[1,1\]w\_\{\\mathrm\{clf\},l\}^\{\\top\}=\[1,1\]\. Then, for cat \(xix\_\{i\}\) and bird \(xjx\_\{j\}\), only the first coordinate ofφ​\(xi\)−φ​\(xj\)\\varphi\(x\_\{i\}\)\-\\varphi\(x\_\{j\}\)is large, and its inner product⟨wclf,l,φ​\(xi\)−φ​\(xj\)⟩\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x\_\{i\}\)\-\\varphi\(x\_\{j\}\)\\rangleis still large\. For the other case of lion \(xix\_\{i\}\) and mouse \(xjx\_\{j\}\), only the second coordinate ofφ​\(xi\)−φ​\(xj\)\\varphi\(x\_\{i\}\)\-\\varphi\(x\_\{j\}\)is large, and its inner product⟨wclf,l,φ​\(xi\)−φ​\(xj\)⟩\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x\_\{i\}\)\-\\varphi\(x\_\{j\}\)\\rangleis still large\. Thus, pairwise queries can differentiate these examples\. One major issue with such a feature mapping can be the ambient dimension of this mapping,dd, being extremely large\. This is empirically not the case for real datasets\. For popular vision datasets, the intrinsic dimension is100~100, as empirically verified in\[pope2021the\]\.

Now, to computeFclf,p​\(α\)F\_\{\\mathrm\{clf\},p\}\(\\alpha\)andFclf,c​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\), we require assumptions on the distribution of features𝒟x\{\\mathcal\{D\}\}\_\{x\}\. We consider two special cases – Spherical features and Gaussian features\.

###### Theorem 4\(Spherical Features, Linear Link\)\.

Under Assumptions[1](https://arxiv.org/html/2605.30615#Thmdefinition1)and[3](https://arxiv.org/html/2605.30615#Thmtheorem3), ifx∼𝒟xx\\sim\{\\mathcal\{D\}\}\_\{x\}is equivalent toφ​\(x\)∼Unif​\(𝕊d−1\)\\varphi\(x\)\\sim\\mathrm\{Unif\}\(\\mathbb\{S\}^\{d\-1\}\)and the link function isg​\(a\)=1\+a2,∀a∈\[−1,1\]g\(a\)=\\frac\{1\+a\}\{2\},\\,\\forall a\\in\[\-1,1\], for any constantα∈\(0,1\)\\alpha\\in\(0,1\)and largenn, a sufficient condition forFclf,c​\(α\)≥Fclf,p​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\\geq F\_\{\\mathrm\{clf\},p\}\(\\alpha\)is

⟨w⋆,wclf,c⟩​\(⟨wclf,c,wh⟩−⟨wclf,c,wclf,l⟩\)≥\(1\+Ω​\(d−\(1−α\)​n/2\(1−α\)​n\)\)​\(1−2−\(1−α\)​n\)\\displaystyle\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},c\}\\rangle\(\\langle w\_\{\\mathrm\{clf\},c\},w\_\{h\}\\rangle\-\\langle w\_\{\\mathrm\{clf\},c\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\\geq\\left\(1\+\\Omega\\left\(\\frac\{d^\{\-\(1\-\\alpha\)n/2\}\}\{\(1\-\\alpha\)n\}\\right\)\\right\)\(1\-2^\{\-\(1\-\\alpha\)n\}\)\(1\)⋅\(⟨w⋆,wh⟩−⟨w⋆,wclf,l⟩\)\+2−\(1−α\)​n​\(⟨w⋆,wclf,l⟩​\(⟨wh,wclf,l⟩−1\)\)\.\\displaystyle\\qquad\\qquad\\cdot\(\\langle w^\{\\star\},w\_\{h\}\\rangle\-\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\+2^\{\-\(1\-\\alpha\)n\}\(\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\(\\langle w\_\{h\},w\_\{\\mathrm\{clf\},l\}\\rangle\-1\)\)\.

###### Theorem 5\(Gaussian Features, Smooth Link\)\.

Under Assumptions[1](https://arxiv.org/html/2605.30615#Thmdefinition1)and[3](https://arxiv.org/html/2605.30615#Thmtheorem3), ifx∼𝒟xx\\sim\{\\mathcal\{D\}\}\_\{x\}is equivalent toφ​\(x\)∼𝒩​\(0,𝕀d\)\\varphi\(x\)\\sim\{\\mathcal\{N\}\}\(0,\\mathbb\{I\}\_\{d\}\)and the link functionggisLL\-smooth, then for any constantα∈\(0,1\)\\alpha\\in\(0,1\)and largenn, a sufficient condition forFclf,c​\(α\)≥Fclf,p​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\\geq F\_\{\\mathrm\{clf\},p\}\(\\alpha\)is

⟨w⋆,wclf,c⟩​\(⟨wclf,c,wh⟩−⟨wclf,c,wclf,l⟩\)≥\(1\+ω​\(1\)\)​\(1−2−\(1−α\)​n\)​\(⟨w⋆,wh⟩−⟨w⋆,wclf,l⟩\)\\displaystyle\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},c\}\\rangle\(\\langle w\_\{\\mathrm\{clf\},c\},w\_\{h\}\\rangle\-\\langle w\_\{\\mathrm\{clf\},c\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\\geq\\,\\left\(1\+\\omega\(1\)\\right\)\(1\-2^\{\-\(1\-\\alpha\)n\}\)\(\\langle w^\{\\star\},w\_\{h\}\\rangle\-\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\(2\)\+2−\(1−α\)​n​\(⟨w⋆,wclf,l⟩​\(⟨wh,wclf,l⟩−1\)\)\+Ω​\(L​d3/2​\(1\+2\(1−α\)​n\)/g′​\(0\)\)\.\\displaystyle\\qquad\+2^\{\-\(1\-\\alpha\)n\}\(\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\(\\langle w\_\{h\},w\_\{\\mathrm\{clf\},l\}\\rangle\-1\)\)\+\\Omega\(Ld^\{3/2\}\(1\+2^\{\(1\-\\alpha\)n\}\)/g^\{\\prime\}\(0\)\)\.

The only differences in both the theorems is the additional term due toLL\-smoothness in equation[2](https://arxiv.org/html/2605.30615#S4.E2)and the sharper characterization of theΩ​\(d−\(1−α\)​n/2\(1−α\)n\)\)\\Omega\\left\(\\frac\{d^\{\-\(1\-\\alpha\)n/2\}\}\{\(1\-\\alpha\)n\)\}\\right\)term in equation[1](https://arxiv.org/html/2605.30615#S4.E1)\. Note thatLL\-smooth link functions suffer from an additional term dependent on the dimension of the classifier\. This can be made small for small smoothness constantLLor aggwhich has a sharp increase at0\. To interpret these conditions, consider the first term on RHS in both the equations\. Assuming that the human is close to the true distribution,wh≊w⋆w\_\{h\}\\approxeq w^\{\\star\}, the first term is always≤1\\leq 1\. Further, its coefficient is always close to11for largennand constantα\\alpha\. The second term on RHS of both equations is always negative as long as the classifier is a good classifier, i\.e,⟨w⋆,wclf,l⟩≥0\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\\geq 0\. However, the impact of the second term is diminished for largennor smallα\\alpha\. If we assume that the confidence model is also not bad, i\.e\.,⟨w⋆,wclf,c⟩\>0\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},c\}\\rangle\>0, we find that⟨wclf,c,wclf,l⟩\\langle w\_\{\\mathrm\{clf\},c\},w\_\{\\mathrm\{clf\},l\}\\rangle, orthe similarity between the confidence model and the actual labeling model, should be small for advantage of pairwise queries\. Note that if the confidence model is accurate, i\.e\.,wclf,c=wclf,lw\_\{\\mathrm\{clf\},c\}=w\_\{\\mathrm\{clf\},l\}, we require the second term on RHS, without the coefficients of2−\(1−α\)​n2^\{\-\(1\-\\alpha\)n\}to be larger than the first term on the RHS\. For a good classifier, the second term on RHS is always negative and can never be larger than the first term, which is positive as long as the human and true classifier are close and the human is better than the classifier\. Therefore, for a good classifier and accurate confidences, ConfSel is always better than pairwise queries\. Theorems[4](https://arxiv.org/html/2605.30615#Thmtheorem4)and[5](https://arxiv.org/html/2605.30615#Thmtheorem5)provide conditions for any classifier, confidence function, human and true labels, which might not obey any of the reasonable assumptions we used in the above discussion\(good classifier and confidence, human≊\\approxeqtrue labels\)\. As our theorems establish conditions when our simplest pairwise query\-based algorithm, PairSel\-Middle, is better than ConfSel, we expect that more complicated pairwise query\-based algorithms in Algorithm[2](https://arxiv.org/html/2605.30615#alg2)will also be better than ConfSel under weaker conditions\.

## 5Experiments

To evaluate the performance of our methods empirically, we run them on55\(44real and11synthetic\) different binary classification datasets\. Motivated by the mismatch between confidences and labeling functions in LLMs \(Fig[1](https://arxiv.org/html/2605.30615#S1.F1.7)\), all our real datasets utilize in\-context learning on LLMs for direct and pairwise queries\. Additional details about our experiments are deferred to Appendix[A](https://arxiv.org/html/2605.30615#A1)\.

### 5\.1Setup

![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/direct_prompt.png)
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/pairwise_prompt.png)

Figure 2:ICL Prompt for a direct query \(left\) and a pairwise query \(right\) from Bird\. The “X 4” in each image denotes that there are44in\-context examples of which only one has been presented\. Prompt template derived from the Bird benchmark\[li2024can,[https://bird\-bench\.github\.io](https://bird-bench.github.io/)\], distributed under CC BY\-SA 4\.0\.Datasets:Our first dataset, which we call Synthetic, is manually generated based on our theoretical model defined in Assumption[3](https://arxiv.org/html/2605.30615#Thmtheorem3)\. Here, the features are Gaussian \(φ​\(x\)=x,∀x\\varphi\(x\)=x,\\forall x\), and the labels are generated from the optimal modelw⋆w^\{\\star\}using a sigmoid link functiongg\.

The second and third datasets are22NL2SQL datasets – Spider\[yu\-etal\-2018\-spider\]and Bird\[li2024can\]\. These datasets are not binary classification datasets, as they contain a natural language query and a SQL statement on a database that answers the question\. To convert these datasets to binary classification, we create a dataset for verifying the correctness of a SQL query for a given question\. We split these datasets into two parts: the first retains the original \(question, SQL\) pairs and is assigned the positive label, and for the second part, we generate an incorrect SQL \(by derangement\) for each question, forming the negative label\. The task is to predict the label for each \(question, SQL\) pair\. We use the older Spider dataset instead of the newer and more challenging Spider2 dataset, as not all questions in Spider2 have correct SQLs, which are required for our task of verifying the correctness of SQL\[lei2024spider\]\.

Our fourth and fifth datasets are originally binary classification datasets, unlike Spider and Bird, where we needed to create a binary classification from the original NL2SQL task\. The fourth dataset is BoolQ\[clark2019boolq\], which consists of pairs of passages and True/False questions converted to binary labels\. This is a part of the SuperGLUE\[Wang2019SuperGLUE\]benchmark used for evaluating Natural Language Understanding of LLMs\. The fifth dataset is a vision dataset, VisOnlyQA\[kamoi2025visonlyqa\], where large vision language models still struggle\. It consists of pairs of an image and a question\. A subset of these questions have a True/False answer, which we use for binary classification\.

Classifiers:For the Synthetic dataset, the classifier itself is a pair of linear models, the labeling modelwclf,lw\_\{\\mathrm\{clf\},l\}and the confidence modelwclf,cw\_\{\\mathrm\{clf\},c\}\. We generate these so thatwclf,lw\_\{\\mathrm\{clf\},l\}is closer tow⋆w^\{\\star\}thanwclf,cw\_\{\\mathrm\{clf\},c\}\. For all our real datasets, we use open\-source LLMs with<10​B<10Bparameters\. For Spider, Bird and BoolQ datasets, we use33general purpose LLMs – Gemma 3 4B Instruct\[gemmateam2025gemma3technicalreport\], Llama 3\.1 8B Instruct\[grattafiori2024llama3herdmodels\], and the most recent Qwen3 8B Instruct\[yang2025qwen3technicalreport\]\. For Spider and Bird, we also use SQLCoder Llama 3 8B\[sqlcoder\]which is excellent for NL2SQL task\. For VisOnlyQA, we use33Vision Language Models – Gemma 3 4B Instruct, Qwen 2\.5 VL 7B Instruct\[bai2025qwen25vltechnicalreport\], and the state\-of\-the\-art GLM 4\.1V 9B Thinking\[vteam2025glm45vglm41vthinkingversatilemultimodal\]\. All experiments were conducted using locally\-run open\-weight models\. No hosted commercial API \(e\.g\., OpenAI\) was queried for experimental results\.

#### 5\.1\.1Baselines

Throughout our experiments, we assume that the human returns the correct label for each datapoint\.

Direct/Raw Baseline:Our first baseline is the direct labeling of each pair represented as raw in all our results\. For Synthetic dataset, the label is obtained from the labeling modelwclf,lw\_\{\\mathrm\{clf\},l\}and the confidence in the label is obtained by the confidence modelwclf,cw\_\{\\mathrm\{clf\},c\}\. For real datasets, the label is obtained by an in\-context learning prompt on the LLM, and the confidence is obtained by normalizing the LLM’s output logits\. Fig[2](https://arxiv.org/html/2605.30615#S5.F2)is an example of direct query for Bird dataset\. The normalization process for confidence involves applying a softmax on the raw logits for the tokens0and11at the position of the label in the classifier output\. The raw baseline applies ConfSel on these labels, treating the confidences aspclf,cp\_\{\\mathrm\{clf\},c\}\.

Calibrated and Base Model:To check if a more calibrated confidence function can improve selective classification over raw confidences, we consider two baselines – i\) Calibrated labels and confidences obtained by prototypical calibration as a post\-processing step\[han2023prototypical\]and ii\) Using confidence ofbase modeland label from an instruct model, as base models are better calibrated than instruct models\[openai2024gpt4\], but instruct models are better at in\-context learning\[instruct,wei2022finetuned\]\. Note that the base model is not used for the Synthetic and VisOnlyQA datasets and for Llama3 SQLCoder as it isn’t available for these cases\.

Pairwise Queries:For Synthetic dataset, we generate noiseless pairwise queries from the oracle defined in Definition[1](https://arxiv.org/html/2605.30615#Thmdefinition1)\. For the real datasets, a single pairwise query is performed via in\-context learning on the LLM\. For instance, for Spider and Bird dataset, we provide the classifier with22pairs of examples \(NL\-SQL pairs\) and ask which of them is more correct, where the label11corresponds to correct and0to incorrect\. Figure[2](https://arxiv.org/html/2605.30615#S5.F2)provides an example of a pairwise query for Bird dataset\. Note that for all the remaining real datasets, this pairwise query based on correctness still makes sense as they use True/False questions where the label11corresponds to True and a correct answer\. As higher correctness for a featurex∈𝒳x\\in\{\\mathcal\{X\}\}implies a higherpclf,l​\(x\)p\_\{\\mathrm\{clf\},l\}\(x\), these pairwise ICL queries are a noisy version of pairwise queries described in our theoretical setup \(Section[4](https://arxiv.org/html/2605.30615#S4)\)\.

![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best__synthetic_sigmoid_frac.png)\(\(a\)\)Synthetic, Linear
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama3_sqlcoder_8b_bird_top3db_frac.png)\(\(b\)\)Bird, Llama3 SQLCoder
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama3_sqlcoder_8b_spider_top3db_frac.png)\(\(c\)\)Spider, Llama3 SQLCoder
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_gemma3_4b_instruct_spider_top3db_frac.png)\(\(d\)\)Spider, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama31_8b_instruct_spider_top3db_frac.png)\(\(e\)\)Spider, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_qwen3_8b_instruct_spider_top3db_frac.png)\(\(f\)\)Spider, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_gemma3_4b_instruct_bird_top3db_frac.png)\(\(g\)\)Bird, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama31_8b_instruct_bird_top3db_frac.png)\(\(h\)\)Bird, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_qwen3_8b_instruct_bird_top3db_frac.png)\(\(i\)\)Bird, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_gemma3_4b_instruct_boolq_frac.png)\(\(j\)\)BoolQ, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama31_8b_instruct_boolq_frac.png)\(\(k\)\)BoolQ, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_qwen3_8b_instruct_boolq_frac.png)\(\(l\)\)BoolQ, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_gemma3_4b_instruct_visqa_frac.png)\(\(m\)\)VisOnlyQA, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_qwen25vl_7b_instruct_visqa_frac.png)\(\(n\)\)VisOnlyQA, Qwen 2\.5 VL
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_glm41v_9b_visqa_frac.png)\(\(o\)\)VisOnlyQA, GLM 4\.1V

Figure 3:Total Accuracy of classifier v/s the fraction of human labels for all datasets \(Synthetic, Bird, Spider, BoolQ, VisOnlyQA\)\. For each fraction of human labels, only the best performing method \(Base, Pairwise and kNN\) is reported along with raw and calibrated baselines\.kNN:To implement PairSel\-kNN, we require a distance metric,dist, between two features\. For synthetic dataset, the features aredd\-dimensional vectors, and the distance isℓ2\\ell\_\{2\}norm of their difference\. For Spider and Bird datasets, each featurex∈𝒳x\\in\{\\mathcal\{X\}\}is a pair of NL\-SQL, represented asx=\(xN​L,xS​Q​L\)x=\(x\_\{NL\},x\_\{SQL\}\), we usedist​\(x,x′\)=distNL​\(xN​L,xN​L′\)\+λ⋅distSQL​\(xS​Q​L,xS​Q​L′\)\\text\{dist\}\(x,x^\{\\prime\}\)=\\text\{dist\}\_\{\\text\{NL\}\}\(x\_\{NL\},x\_\{NL\}^\{\\prime\}\)\+\\lambda\\cdot\\text\{dist\}\_\{\\text\{SQL\}\}\(x\_\{SQL\},x\_\{SQL\}^\{\\prime\}\)forx,x′∈𝒳x,x^\{\\prime\}\\in\{\\mathcal\{X\}\}, wheredistN​L\\text\{dist\}\_\{NL\}anddistS​Q​L\\text\{dist\}\_\{SQL\}are distance metrics for natural language and SQLs respectively andλ\>0\\lambda\>0is a weight\. For NL, we use a popular distance metric, cosine distance between sentence embeddings\[ni\-etal\-2022\-sentence\]\. For SQL, however, specialized and widely accepted distance metrics do not exist, so we propose one in Appendix[A\.5](https://arxiv.org/html/2605.30615#A1.SS5)and use it in our experiments\. BoolQ and VisOnlyQA datasets, consist of question and passage/image pairs, respectively\. For these, we use a similar distance computation, where instead of the distance between SQLs, we compute distances between passages or images\. Distance between passages is the same as distance between NL, while we use cosine distance between CLIP embeddings\[radford2021learningtransferablevisualmodels\]for images\.

Naive Classifier:This classifier labels each datapoint with the majority label of the dataset and selects datapoints to reject uniformly at random\.

Metrics:As all our baselines are selective classification algorithms, we measure the total accuracy of a method at a fixed fraction of human labelsα\\alpha\. This is defined as the accuracy of the set of labels, when the rejected datapoints have human labels and the non\-rejected datapoints have the classifier’s labels\. By varyingα\\alphafrom0\.010\.01to0\.90\.9, we plot the total accuracy v/sα\\alphain Fig[3](https://arxiv.org/html/2605.30615#S5.F3)for all datasets and classifiers\. For the baselines, base model, pairwise queries and PairSel\-kNN baselines, we have several choices for the labels \(calibrated/raw\), confidences\(calibrated/raw\) and the actual method \(Middle/Max\-Entropy/Max\-Presence/Max\-Displacement\)\. For eachα\\alpha, for each of these baselines, we only consider the best performing method among all the possible choices\. These correspond to the legends “Best Pairwise Method” and so on in the figures\. The1010dotted vertical lines representα=10%,20%,…,100%\\alpha=10\\%,20\\%,\\ldots,100\\%\. To compute the maximum possible gain by the pairwise method, for each combination of Dataset and classifier, we find the value ofα<0\.5\\alpha<0\.5at which the difference between the best pairwise method and the raw baseline is largest\. In practice, a fixed budget for labeling implies an upper bound on the fraction of pointsα\\alphathat can be human\-labeled\. We report the total accuracy for thisα\\alpha, which we call bestα\\alpha, in Table[1](https://arxiv.org/html/2605.30615#S5.T1)\. Note that all our experiments are run for55seeds, and we plot the standard deviation in the figures\. Additional results, including using the cost of queries instead ofα\\alphaand sensitivity to parameters, are deferred to Appendix[A](https://arxiv.org/html/2605.30615#A1)\.

#### 5\.1\.2Setup for Fig[1](https://arxiv.org/html/2605.30615#S1.F1.7)

For Fig[1](https://arxiv.org/html/2605.30615#S1.F1.7), we computepv,cp\_\{v,c\}andpv,lp\_\{v,l\}for all datapoints on the filtered Bird dataset with Gemma3 4B Instruct\. For each datapoint, we run a direct query for5050different seeds and compute the fraction of predicted labels that are11across these runs as an estimate ofpv,lp\_\{v,l\}\. The average value of the softmax of next token logits across these runs gives us the value ofpv,cp\_\{v,c\}\. For the threshold with p\-value of55, we consider the case whenpv,cp\_\{v,c\}andpv,lp\_\{v,l\}are exactly equal\. If we ran this experiment forRRrandom seeds then, thepv,l≈pv,cp\_\{v,l\}\\approx p\_\{v,c\}with p\-valueδ\\deltaif1−δ1\-\\deltaof the mass of the distribution of\|pv,l−pv,c\|\\lvert p\_\{v,l\}\-p\_\{v,c\}\\rvertlies between0and2​log⁡\(1/δ\)R\\sqrt\{\\frac\{2\\log\(1/\\delta\)\}\{R\}\}\. The threshold is obtained by a simple application of Chernoff bound for binomial distributions\. We setR=50R=50for our case\.

### 5\.2Results

DatasetClassifierBestα\\alphaNaiveRawCalibratedBest BaseBest PairwiseBest kNNSyntheticLinear0\.4773\.52 \(\-14\.53\)88\.0585\.63\(\-2\.42\)\-97\.75\(\+9\.70\)97\.72\(\+9\.67\)SpiderSQLCoder0\.4176\.24\(\-18\.62\)94\.8698\.11\(\+3\.25\)\-98\.65\(\+3\.79\)98\.38\(\+3\.52\)Llama 3\.10\.3272\.54\(\-21\.88\)95\.4196\.76 \(\+1\.22\)96\.22\(\+0\.81\)98\.38\(\+2\.97\)98\.11\(\+2\.70\)Gemma 30\.4176\.24\(\-19\.17\)95\.4197\.03\(\+1\.62\)97\.3\(\+1\.89\)98\.38\(\+2\.97\)98\.38\(2\.97\)Qwen 30\.3272\.54\(\-16\.38\)88\.9291\.89\(\+2\.97\)90\.0\(\+1\.08\)98\.11\(\+9\.19\)95\.14\(\+6\.22\)BirdSQCoder0\.3875\.38\(\-10\.73\)86\.1192\.08\(\+5\.97\)\-95\.29\(\+9\.18\)95\.06\(\+8\.95\)Llama 3\.10\.3574\.16\(\-14\.24\)88\.493\.57\(\+5\.17\)94\.26\(\+5\.86\)96\.67\(\+8\.27\)95\.75\(\+7\.35\)Gemma 30\.3574\.16\(\-20\.56\)94\.7295\.98\(\+1\.26\)96\.67\(\+1\.95\)97\.59\(\+2\.87\)97\.36\(\+2\.64\)Qwen 30\.2670\.52\(\-2\.27\)72\.7982\.43\(\+9\.64\)79\.22\(\+6\.43\)88\.63\(\+15\.84\)85\.88\(\+13\.09\)BoolQLlama 3\.10\.4177\.07\(\-2\.80\)79\.8781\.46\(\+1\.59\)87\.96\(\+8\.09\)88\.43\(\+8\.56\)87\.0\(\+7\.13\)Gemma 30\.3574\.69 \(\-2\.81\)77\.5082\.25\(\+4\.75\)83\.04\(\+5\.54\)85\.74\(\+ 8\.24\)85\.42\(\+7\.92\)Qwen 30\.4779\.45\(\-2\.96\)82\.4186\.85\(\+4\.44\)86\.69\(\+4\.28\)88\.91\(\+6\.5\)88\.11\(\+5\.7\)Vis\-OnlyQAQwen 2\.5VL0\.4472\.38\(\-3\.20\)75\.5871\.16 \(\-4\.42\)\-79\.30 \(\+3\.72\)78\.37 \(\+ 2\.79\)Gemma 30\.4773\.89\(\+0\.17\)73\.7273\.49\(\-0\.23\)\-77\.44\(\+3\.72\)74\.88\(\+1\.16\)GLM 4\.1V0\.2261\.78\(\+0\.62\)61\.1662\.56\(\+1\.4\)\-63\.72\(\+2\.56\)63\.02\(\+1\.86\)

Table 1:Total accuracy for different classifiers and datasets\. For each row, the bestα∈\(0,0\.5\]\\alpha\\in\(0,0\.5\]is where the difference between the performance of the raw baseline and best pairwise method is largest\. For each baseline other than raw, we show the difference with the raw baseline in brackets\. The largest difference in each row isbold\.From Fig[3](https://arxiv.org/html/2605.30615#S5.F3), for all our datasets and classifiers, we see that the best pairwise method consistently achieves the best performance across most values ofα\\alpha\.

Synthetic:For the synthetic dataset with sigmoid link in Fig[3](https://arxiv.org/html/2605.30615#S5.F3)\(a\), we can see that pairwise queries and kNN methods both lead to a large increase in performance as compared to either the raw baseline or the calibrated baseline\. Surprisingly, the calibrated method decreases performance\. This verifies our theoretical insight that as long as the confidence modelwclf,cw\_\{\\mathrm\{clf\},c\}is far away from the labeling function, direct queries perform worse than pairwise methods\. Further, from Table[1](https://arxiv.org/html/2605.30615#S5.T1), pairwise method provides atmost10%~10\\%increase in performance\.

Spider and Bird:Note that most of our classifiers are already extremely accurate on these datasets without any human labeling \(≈90%\\approx 90\\%total accuracy withα=0\\alpha=0\)\. Therefore, the relative boost with selective classification is limited as seen by only3%3\\%maximum boost by pairwise methods for Spider in Table[1](https://arxiv.org/html/2605.30615#S5.T1)\. Also, by the magnitude of the standard deviation in the Fig[3](https://arxiv.org/html/2605.30615#S5.F3)\(c\) \- \(f\), this performance boost can be nullified by the standard deviation\. The harder Bird dataset has a larger boost due to pairwise methods \(≈16%\\approx 16\\%in Table[1](https://arxiv.org/html/2605.30615#S5.T1)\)\. A surprising observation is that for several cases, the raw confidences,pclf,cp\_\{\\mathrm\{clf\},c\}, are so misaligned with the labelling function,pclf,lp\_\{\\mathrm\{clf\},l\}, that the performance of the raw baseline remains almost constant even with increasingα\\alpha\.

BoolQ and VisOnlyQA:For these datasets, we don’t create negative samples\. Further, these are harder for current classifiers, with the performance on VisOnlyQA close to a random classifier \(50%50\\%\) without human labellingα=0\.0\\alpha=0\.0\. For both these datasets, using a naive classifier is not much worse than the raw baseline \(Table[1](https://arxiv.org/html/2605.30615#S5.T1)\)\. However, these datasets are different\. For BoolQ, we see a large boost due to pairwise methods \(≈8%\\approx 8\\%\), but for VisOnlyQA, we see only a minor improvement due to pairwise methods\. We suspect this is because VisOnlyQA is so hard that even pairwise queries are extremely noisy on it\.

Calibration:Note that for most combinations of datasets and classifiers, calibration improves performance over the raw baseline\. However, this is not always the case, as for Synthetic, \(Bird, Gemma 3\), \(BoolQ, Llama 3\.1\) and \(VisOnlyQA, Qwen 2\.5 VL\), calibration decreases performance by a lot \(see−4\.5%\-4\.5\\%for VisOnlyQA in Table[1](https://arxiv.org/html/2605.30615#S5.T1)\)\. This is because the calibration technique\[han2023prototypical\]fits a mixture of22Gaussians to the binary labels and confidences, which might not be a good fit in practice\. Note that calibration is never the best baseline across allα\\alphafor any dataset and classifier combination\.

Base:Unlike calibration, base model confidences always improve performance whenever available\. However, the main issue is their availability, as they might not be available for custom classifiers like SQLCoder\. They also never achieve the best performance among all baselines\.

kNN:Note that this is the second\-best baseline as compared to pure pairwise methods\. However, it’s performance is always slightly worse than best pairwise\. This decrease in performance indicates that use of the distance metric does not offer any explicit advantage over pairwise queries\. We hypothesize that the pairwise queries that we compute and the distance metric are not independent of each other\.

#### 5\.2\.1Relative Performance of Different PairSel methods

DatasetClassifierBestα\\alphaBestMiddleBestMaxPresenceBestMaxDisplacementBestMaxEntropySyntheticLinear0\.3894\.75\(\+6\.96\)87\.7993\.91 \(\+6\.12\)95\.16 \(\+7\.37\)SpiderSQLCoder0\.3297\.5797\.5798\.38 \(\+0\.81\)97\.84 \(\+0\.27\)Llama 3\.10\.3297\.84 \(\+0\.81\)97\.0397\.8498\.38\(\+1\.35\)Gemma 30\.4798\.11 \(\+0\.81\)98\.65 \(\+1\.35\)97\.397\.84\(\+0\.54\)Qwen 30\.3295\.14\(\+0\.55\)94\.86\(\+0\.27\)98\.11\(\+3\.52\)94\.59BirdSQCoder0\.4195\.18\(\+1\.26\)94\.49 \(\+0\.57\)93\.9294\.95\(\+1\.03\)Llama 3\.10\.3295\.18 \(\+0\.23\)94\.9596\.21\(\+1\.26\)94\.95Gemma 30\.1695\.98\(\+0\.34\)95\.6496\.21\(\+0\.57\)96\.1\(\+0\.46\)Qwen 30\.2684\.16 \(\+0\.35\)83\.8188\.63\(\+4\.82\)85\.76\(\+1\.95\)BoolQLlama 3\.10\.4187\.0\(\+0\.79\)86\.2188\.43\(\+2\.22\)88\.11\(\+1\.90\)Gemma 30\.1679\.479\.56 \(\+0\.16\)80\.35\(\+0\.95\)79\.56\(\+0\.16\)Qwen 30\.4787\.087\.64 \(\+0\.64\)88\.91\(\+1\.91\)87\.8 \(\+0\.8\)VisOnlyQAQwen 2\.5 VL0\.4478\.37\(\+0\.7\)78\.14\(\+0\.47\)77\.6779\.3\(\+1\.63\)Gemma 30\.4774\.19\(1\.4\)76\.98\(\+4\.19\)77\.44\(\+4\.65\)72\.79GLM 4\.1V0\.3569\.77\(\+0\.93\)68\.8470\.47\(\+1\.63\)70\.47 \(\+1\.63\)

Table 2:Total accuracy of all pairwise methods for different classifiers and datasets\. For each row, the bestα∈\(0,0\.5\]\\alpha\\in\(0,0\.5\]is where the difference between the performance of any two of the pairwise methods is largest\. For each row, the worst baseline is reported without brackets and for all other baselines, their difference in performance with respect to the worst baseline is shown inside the brackets\. The largest difference in each row isbold\.The performance of different pairwise methods in PairSel\(Algorithm[2](https://arxiv.org/html/2605.30615#alg2)\) cannot be inferred from Fig[3](https://arxiv.org/html/2605.30615#S5.F3),[6](https://arxiv.org/html/2605.30615#A1.F6)or Table[1](https://arxiv.org/html/2605.30615#S5.T1)\. To measure this relative performance, in Fig[7](https://arxiv.org/html/2605.30615#A1.F7), we report the best performance of each pairwise method other than PairSel\-kNN\. Here, for each pairwise method, the best performance is among the different choices of labels and confidences\. Further, in Table[2](https://arxiv.org/html/2605.30615#S5.T2), we compute the value ofα≤0\.5\\alpha\\leq 0\.5, where the difference between the total accuracy of any two pairwise methods is maximum\. We also compute the difference of each baseline with respect to the worst baseline for each combination of dataset and classifier\.

From Fig[7](https://arxiv.org/html/2605.30615#A1.F7)and Table[2](https://arxiv.org/html/2605.30615#S5.T2), we can see that the Max Displacement is most often the best performing pairwise method, followed by Max Entropy method\. However, this conclusion does not hold across all datasets and classifiers as for some cases \(Spider Llama 3\.1 and Gemma 3, Bird SQLCoder and VisOnlyQA Qwen 2\.5 VL\), Max Displacement performs worst among all pairwise baselines\. Note that Max Entropy is also the worst baseline for33dataset classifier combinations \( Spider Qwen 3, Bird Llama 3\.1 and VisOnlyQA Gemma 3\)\. Note that Max Presence consistently is one of the worst performing pairwise methods due to its simple implementation and correlation to the raw baseline\. However, for Spider Gemma 3, it does obtain the best performance\. As for the Middle method, whose performance we investigate theoretically, its performance always lies between the best methods \(Max Displacement, Max Entropy\) and the worst method \(Max Presence\)\. Further, the difference in performance between pairwise methods is comparable to that between best pairwise methods and the raw baseline \(see Tables[1](https://arxiv.org/html/2605.30615#S5.T1)and[2](https://arxiv.org/html/2605.30615#S5.T2)\), so some pairwise methods do not fully take advantage of pairwise queries\.

## 6Conclusion

In this paper, we showed how to incorporate pairwise queries from a model to improve selective classification\. We establish conditions under which pairwise queries obtain lower population risk than using a confidence estimate inconsistent with the labels\. Through extensive experiments on in\-context learning in LLMs, where the next\-token logits are not aligned with the true labeling functions, we showed that our pairwise query\-based algorithms consistently perform better than all baselines, especially using the raw baseline using next\-token logits\. A promising direction for future work is finding the optimal selective classification algorithm using pairwise queries in terms of cost orα\\alpha\.

\\bmhead

Acknowledgments Part of the work was performed when HV was an intern at Adobe Research\. In addition to this, the work was supported by an Adobe Research Gift and National Science Foundation award 2217058\.

## References

## Appendix AAdditional Experimental Details

### A\.1Datasets

##### Synthetic

For this datasetn=1000n=1000andxi​∼i​i​d​Unif​\(𝕊d−1\)x\_\{i\}\\overset\{iid\}\{\\sim\}\\mathrm\{Unif\}\(\\mathbb\{S\}^\{d\-1\}\)withd=100d=100\. Then, we samplew⋆∼Unif​\(𝕊d−1\)w^\{\\star\}\\sim\\mathrm\{Unif\}\(\\mathbb\{S\}^\{d\-1\}\)\. We assume that the labelsyi∼B​e​r​\(p⋆​\(x\)\)y\_\{i\}\\sim Ber\(p^\{\\star\}\(x\)\)wherep⋆​\(x\)=11\+exp⁡\(−β⋅⟨w⋆,x⟩\)p^\{\\star\}\(x\)=\\frac\{1\}\{1\+\\exp\(\-\\beta\\cdot\\langle w^\{\\star\},x\\rangle\)\}withβ=4\\beta=4\. Here, the link function is sigmoid, which isLL\-smooth withL=0\.272L=0\.272\. We use the fact that ifg​\(a\)=11\+exp⁡\(−β​a\)g\(a\)=\\frac\{1\}\{1\+\\exp\(\-\\beta a\)\}, theng′​\(a\)=β​g​\(a\)​\(1−g​\(a\)\)g^\{\\prime\}\(a\)=\\beta g\(a\)\(1\-g\(a\)\)andg′′​\(a\)=β2​g​\(a\)​\(1−g​\(a\)\)​\(1−2​g​\(a\)\)g^\{\\prime\\prime\}\(a\)=\\beta^\{2\}g\(a\)\(1\-g\(a\)\)\(1\-2g\(a\)\)with the value ofa∈\[−1,1\]a\\in\[\-1,1\]asaais the inner product of two unit vectors\. To createwv,lw\_\{v,l\}andwv,cw\_\{v,c\}, we sample two unit vectors,u1,u2u\_\{1\},u\_\{2\}, from𝕊d−1\\mathbb\{S\}^\{d\-1\}perpendicular tow⋆w^\{\\star\}and setwv,l=1−ϵ2​w⋆\+ϵ​u1w\_\{v,l\}=\\sqrt\{1\-\\epsilon^\{2\}\}w^\{\\star\}\+\\epsilon u\_\{1\}andwv,c=1−4​ϵ2​w⋆\+2​ϵ​u2w\_\{v,c\}=\\sqrt\{1\-4\\epsilon^\{2\}\}w^\{\\star\}\+2\\epsilon u\_\{2\}, withϵ=0\.1\\epsilon=0\.1\. The valueϵ\\epsilonallows us to directly control the values of⟨w⋆,wv,l⟩\\langle w^\{\\star\},w\_\{v,l\}\\rangleand⟨w⋆,wv,c⟩\\langle w^\{\\star\},w\_\{v,c\}\\ranglewith⟨w⋆,wv,c⟩<⟨w⋆,wv,l⟩\\langle w^\{\\star\},w\_\{v,c\}\\rangle<\\langle w^\{\\star\},w\_\{v,l\}\\rangle\. The last condition ensures that the confidence estimate from the linear model is not accurate\. For pairwise queries on two featuresxix\_\{i\}andxjx\_\{j\}, we compute the the sign ofpv,l​\(xi\)−pv,l​\(xj\)=g​\(⟨wv,l,xi⟩\)−g​\(⟨wv,l,xj⟩\)p\_\{v,l\}\(x\_\{i\}\)\-p\_\{v,l\}\(x\_\{j\}\)=g\(\\langle w\_\{v,l\},x\_\{i\}\\rangle\)\-g\(\\langle w\_\{v,l\},x\_\{j\}\\rangle\)\. Further, the distance metric between any two featuresxix\_\{i\}andxjx\_\{j\}is\|\|xi−xj\|\|22=2−2​⟨xi,xj⟩\\lvert\\lvert x\_\{i\}\-x\_\{j\}\\rvert\\rvert\_\{2\}^\{2\}=2\-2\\langle x\_\{i\},x\_\{j\}\\rangle\. This setting does not exactly match either of the two theoretical settings in Theorems[4](https://arxiv.org/html/2605.30615#Thmtheorem4)and[5](https://arxiv.org/html/2605.30615#Thmtheorem5), as we use Spherical features with Smooth Link\. However, our theoretical results also hold for this setting\.

##### Spider and Bird

To ensure that we test our baselines on sufficiently complicated SQL queries, we filter out the databases from each of these datasets such that they contain large number of tables, foreign keys and samples\. For spider, the thresholds for number of tables, foreign keys and samples is15,715,7and100100respectively, which results in33databases with370370samples in total\. For Bird, the corresponding thresholds are20,2020,20and150150and we end up with33databases totaling919919samples\. Note that in each case, we only filter and label the training sets\. For both these datasets, we use60%60\\%of the samples as positive samples\. For the remaining40%40\\%of the samples, we need to pair a question with an SQL that does not answer the question\. For this purpose, we shuffle the questions and their correct SQLs of these40%40\\%samples to obtain a derangement, i\.e\., each question is paired with the correct SQL of a different question\. As there are no duplicate questions in the dataset, each question in the negative label set is mapped to an incorrect SQL\.

##### BoolQ and VisOnlyQA

For BoolQ dataset, to ensure that we have sufficiently complicated questions, we only consider those \(Passage, Question\) pairs where the Passage contains more than175175words\. After this filtering, we end up with631631samples, of which61\.2%61\.2\\%are positive labels\. The VisOnlyQA dataset contains single answer and multiple answer questions, of which the single answer questions contain both True/False answers and multiple choice questions\. We only keep the single answer True/False questions to obtain a binary classification task resulting in470470samples of which50%50\\%are positive labels\. For both these datasets, we convert the True label to11and False to0\. For BoolQ, we use the training dataset, and for VisOnlyQA, we use the Real Eval dataset\.

### A\.2Baselines

![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/direct_prompt_boolq.png)
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/pairwise_prompt_boolq.png)

Figure 4:ICL Prompt for a direct query \(left\) and a pairwise query \(right\) from BoolQ dataset\. The ”X 4” in each image denotes that there are44in\-context examples of which only one has been presented\. Prompt template derived from BoolQ\[clark2019boolq,[https://github\.com/google\-research\-datasets/boolean\-questions](https://github.com/google-research-datasets/boolean-questions)\], distributed under CC BY\-SA 3\.0\.![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/direct_prompt_visqa.png)
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/pairwise_prompt_visqa.png)

Figure 5:ICL Prompt for a direct query \(left\) and a pairwise query \(right\) from VisOnlyQA dataset\. The ”X 2” in each image denotes that there are22in\-context examples of which only one has been presented\. Prompt template derived from BoolQ\[kamoi2025visonlyqa,[https://huggingface\.co/datasets/ryokamoi/VisOnlyQA\_Eval\_Real\_v1\.1](https://huggingface.co/datasets/ryokamoi/VisOnlyQA_Eval_Real_v1.1)\], distributed under CC BY\-SA 3\.0\.Direct/Raw BaselineFor each dataset, we remove an intial in\-context example bank of2020samples\. We keep adding the classifier\-labeled examples to this in\-context example bank\. For Spider and Bird, for each in\-context query, we supply the database schema followed by44in\-context examples of \(NL, SQL, label\) selected randomly from the in\-context example bank\. This random sampling assigns a weight of11to each initial in\-context example and0\.50\.5to each example labeled by a classifier\. See Fig[2](https://arxiv.org/html/2605.30615#S5.F2)for an example\. For BoolQ, we also use44in\-context examples of the form \(Passage, Question, Label\) as shown in Fig[4](https://arxiv.org/html/2605.30615#A1.F4)\. For VisQA, we use22in\-context examples of the form \(Image, Question, Label\) for the prompt as shown in Fig[5](https://arxiv.org/html/2605.30615#A1.F5)\.

Calibrated baseline\[han2023prototypical\]In this methods, a mixture of22gaussians, one for each label, is fit to the raw confidences\. Then, the calibrated label and calibrated confidence correspond to the mixture component and the mixture weights for each datapoint\.

Base model confidencesInopenai2024gpt4, the expected calibration error of base models is better than their instruct counterparts\. However, instruct models are better at following instructions and thus, in\-context learning\[instruct,wei2022finetuned\]\. When the classifier is an instruct model with a corresponding base model, we use a direct query on the classifier, like Fig[2](https://arxiv.org/html/2605.30615#S5.F2), to obtain its label\. To obtain its confidence, we pass the full query sent to the classifier along with its label to the classifier’s corresponding base model\. Then, we perform a softmax on the logits for label tokens \(0and11\) from the base model to obtain base model confidences\. Note that the two possible methods using only base model confidences are obtained by applying ConfSel\(Algoithm[1](https://arxiv.org/html/2605.30615#alg1)\) with the labels being either raw or calibrated with the confidence obtained from the base models\. The ”best base” method reports the best performing of these methods\.

![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best__synthetic_sigmoid.png)\(\(a\)\)Synthetic, Linear
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama3_sqlcoder_8b_bird_top3db.png)\(\(b\)\)Bird, Llama3 SQLCoder
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama3_sqlcoder_8b_spider_top3db.png)\(\(c\)\)Spider, Llama3 SQLCoder
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_gemma3_4b_instruct_spider_top3db.png)\(\(d\)\)Spider, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama31_8b_instruct_spider_top3db.png)\(\(e\)\)Spider, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_qwen3_8b_instruct_spider_top3db.png)\(\(f\)\)Spider, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_gemma3_4b_instruct_bird_top3db.png)\(\(g\)\)Bird, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama31_8b_instruct_bird_top3db.png)\(\(h\)\)Bird, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_qwen3_8b_instruct_bird_top3db.png)\(\(i\)\)Bird, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_gemma3_4b_instruct_boolq.png)\(\(j\)\)BoolQ, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_llama31_8b_instruct_boolq.png)\(\(k\)\)BoolQ, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_qwen3_8b_instruct_boolq.png)\(\(l\)\)BoolQ, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_gemma3_4b_instruct_visqa.png)\(\(m\)\)VisOnlyQA, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_qwen25vl_7b_instruct_visqa.png)\(\(n\)\)VisOnlyQA, Qwen 2\.5 VL
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_glm41v_9b_visqa.png)\(\(o\)\)VisOnlyQA, GLM 4\.1V

Figure 6:Total Accuracy of classifier v/s cost for different datasets and classifiers\. We only report the best\-performing methods for each baseline \(Pairwise, kNN, Base\)\.Pairwise QueriesFor each pairwise query, we supply44in\-context examples of correct pairwise comparisons\. These are created from the example bank by sampling an example labeled11and another labeled0\. Note that the example labeled11is more correct than the example labeled0, which decides the label of the pairwise query\. For the VisOnlyQA dataset, we use only22in\-context examples for each pairwise comparisons\. The prompts for pairwise query for BoolQ and VisOnlyQA dataset are shown in Fig[4](https://arxiv.org/html/2605.30615#A1.F4)and[5](https://arxiv.org/html/2605.30615#A1.F5)\. Note that for each pairwise method in PairSel\(Algorithm[2](https://arxiv.org/html/2605.30615#alg2)\) other than PairSel\-kNN, we have22options for labels \(raw or calibrated\) and2−32\-3options for confidences \(raw, calibrated and base\)\. For certain classifiers \(SQLCoder\) and datasets \(VisOnlyQA\), base models and their corresponding confidences are not available\. The ”best pairwise” method is obtained by computing the best performing method among all the combinations of pairwise algorithm, label and confidence for eachα\\alpha\.

PairSel\-kNNFor each dataset, we setλ=0\.5\\lambda=0\.5as the relative weight of the NL/Image/Passage to the Question in Natural Language \(NL\)\. To computedistNL\\text\{dist\}\_\{\\text\{NL\}\}, we compute embeddings for NL using ‘all\-MiniLM\-L6\-V2’ from Sentence Transformers\[ni\-etal\-2022\-sentence\]\. To computedistSQL\\text\{dist\}\_\{\\text\{SQL\}\}in Spider and Bird dataset, we compute the SQL embeddings using SQL Graph Neural Networks proposed in Appendix[A\.5](https://arxiv.org/html/2605.30615#A1.SS5), and compute the cosine distance between embeddings\. We setk=10k=10for k\-Nearest Neighbors\. Since we use the labels and confidences in PairSel, we again have22options for labels \(raw or calibrated\) and2−32\-3options for confidences \(raw, calibrated or base\)\. The ”best kNN” method computes the best performing method of PairSel\-kNN among the different combinations of labels and confidences for eachα\\alpha\.

Naive ClassifierIfβ∈\[0\.5,1\)\\beta\\in\[0\.5,1\)is the fraction of the dataset with the majority label, then, ifα\\alphafraction of datapoints are sent for human labeling, the total accuracy of the naive classifier isβ∗\(1−α\)\+α\\beta\*\(1\-\\alpha\)\+\\alpha\. Note thatβ=0\.6\\beta=0\.6for Spider and Bird datasets, whileβ=0\.5,0\.507\\beta=0\.5,0\.507and0\.6120\.612for the Synthetic, VisOnlyQA and BoolQ datasets respectively\.

### A\.3Metrics

Assuming that a direct query costs11unit, a pairwise query costs22units, and a human query costsA≫1A\\gg 1units, we compute the cost for each algorithm in Table[3](https://arxiv.org/html/2605.30615#A1.T3)\. By varyingα\\alphafrom0\.010\.01to0\.90\.9, with estimatedA=65A=65we plot the total accuracy v/s cost for selective classification algorithms in Figures[6](https://arxiv.org/html/2605.30615#A1.F6)\. In Table[3](https://arxiv.org/html/2605.30615#A1.T3), we report the theoretical costs of all methods for any value ofAA\. Ifnnis the total number of datapoints, then direct labeling costsnnunits\. Further, if we use base model confidences, then each query needs to be passed through the base model resulting in additional cost ofnn\. For methods involving pairwise queries, we apply pairwise queries to onlynndatapoints\. Since we use pairwise queries to obtain a sorting, this incursn​log⁡\(n\)n\\log\(n\)cost\. Combining base model confidences with pairwise queries incurs the highest cost\. Note that using a distance metric in PairSel\-kNN does not incur any additional cost apart from that due to sorting by pairwise queries and/or base model confidences\.

To estimateAA, we find the ratio of the cost of a direct query to that of an expert\. For the direct query, we consider the most expensive classifier as GPT\-4o, which costs$​5/1​M\\mathdollar 5/1\\text\{M\}tokens\. In case there are additional instructions to the GPT\-4o model, the total length of a direct query and its response is≈2​K\\approx 2Ktokens\. Therefore, the cost of one direct query is atmost$​0\.02\\mathdollar 0\.02\. If the human is a product manager, with an average salary of$​39/hr\\mathdollar 39/\\text\{hr\}\[salary\]and spends around22minutes to verify a single datapoint, then the average human cost is$​1\.3\\mathdollar 1\.3\. Therefore, the ratioA≈65A\\approx 65\.

All our experiments except training SQL embeddings took around 2 weeks on one Nvidia A\-10 GPU with1212cores and2020GB RAM\. Training SQL embedding took 2 days for each seed on the same machine\.

MethodCost of computing whichpoints to label by humanHumanLabelingCostTotal CostDirectnnA​n​αAn\\alphan\+A​n​αn\+An\\alphaBase Model2​n2nA​n​αAn\\alpha2​n\+A​n​α2n\+An\\alphaPairwise Methodn\+n​log⁡\(n\)n\+n\\log\(n\)A​n​αAn\\alphan\+n​log⁡\(n\)\+A​n​αn\+n\\log\(n\)\+An\\alphaBase Model \+ Pairwise2​n\+n​log⁡\(n\)2n\+n\\log\(n\)A​n​αAn\\alpha2​n\+n​log⁡\(n\)\+A​n​α2n\+n\\log\(n\)\+An\\alphaTable 3:Labeling cost of each method assuming there arenndatapoints in total, of whichα∈\[0,1\]\\alpha\\in\[0,1\]fraction is sent for human labeling, where the cost of each classifier query is11unit and of each human labeling isAAunits\. Note that using the distance metric doesn’t incur any additional cost on top of pairwise queries or base model confidences\.
### A\.4Additional Experimental Results

#### A\.4\.1Total accuracy vs\. Cost of Labeling

Note that the cost of base models or pairwise queries is negligible compared to human labeling cost due to the relatively large value ofAAand small value ofnnin Fig[6](https://arxiv.org/html/2605.30615#A1.F6)\. Therefore, the performance is mostly affected by the value ofα\\alpha, due to which Fig[6](https://arxiv.org/html/2605.30615#A1.F6)resembles Fig[3](https://arxiv.org/html/2605.30615#S5.F3), and the conclusions drawn from Fig[3](https://arxiv.org/html/2605.30615#S5.F3)carry forward to Fig[6](https://arxiv.org/html/2605.30615#A1.F6)\. One key difference between cost andα\\alpha, is when the cost of pairwise queries or base models becomes non\-negligible\. Note that this is for larger values ofα\\alpha\. Conisder the cases ofα\\alphaclose to0\.90\.9, the maximum value that we consider, we can see that raw and calibrated baselines end up at a smaller cost, while some additional and somewhat non\-negligible cost is added to all other baselines due to pairwise queries or base model confidences\.

#### A\.4\.2Choice of confidence and labels

Note that for each of base, pairwise and kNN methods, we report only the best performing ones in Table[1](https://arxiv.org/html/2605.30615#S5.T1)and Fig[3](https://arxiv.org/html/2605.30615#S5.F3)\. For base and kNN methods this choice is between different labels and different confidences\. The performance of these different combinations can be inferred from that of the raw and calibrated baselines\. For instance, calibrated performs better than raw for mostα\\alphavalues in Fig[3](https://arxiv.org/html/2605.30615#S5.F3), for all datasets and classifiers except Synthetic, \(BoolQ, Llama3\.1\), and VisOnlyQA with Gemma 3 and Qwen 2\.5VL\. The performance of the base model confidence and kNN methods with calibrated labels and\(or\) confidences is also better than that with raw labels for only these datasets and classifiers\.

![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise__synthetic_sigmoid_frac.png)\(\(a\)\)Synthetic, Linear
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_llama3_sqlcoder_8b_bird_top3db_frac.png)\(\(b\)\)Bird, Llama3 SQLCoder
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_llama3_sqlcoder_8b_spider_top3db_frac.png)\(\(c\)\)Spider, Llama3 SQLCoder
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_gemma3_4b_instruct_spider_top3db_frac.png)\(\(d\)\)Spider, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_llama31_8b_instruct_spider_top3db_frac.png)\(\(e\)\)Spider, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_qwen3_8b_instruct_spider_top3db_frac.png)\(\(f\)\)Spider, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_gemma3_4b_instruct_bird_top3db_frac.png)\(\(g\)\)Bird, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_llama31_8b_instruct_bird_top3db_frac.png)\(\(h\)\)Bird, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_qwen3_8b_instruct_bird_top3db_frac.png)\(\(i\)\)Bird, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_gemma3_4b_instruct_boolq_frac.png)\(\(j\)\)BoolQ, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_llama31_8b_instruct_boolq_frac.png)\(\(k\)\)BoolQ, Llama 3\.1
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_qwen3_8b_instruct_boolq_frac.png)\(\(l\)\)BoolQ, Qwen 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_gemma3_4b_instruct_visqa_frac.png)\(\(m\)\)VisOnlyQA, Gemma 3
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_qwen25vl_7b_instruct_visqa_frac.png)\(\(n\)\)VisOnlyQA, Qwen 2\.5 VL
![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/best_pairwise_glm41v_9b_visqa_frac.png)\(\(o\)\)VisOnlyQA, GLM 4\.1V

Figure 7:Total Accuracy v/s fraction of human labels for different Pairwise methods for different datasets and classifiers\. We only report the best performance for each pairwise method \(Middle, Max Presence, Max Entropy, Max Displacement\)\.
#### A\.4\.3Parameter Sensitivity

Note that none of the algorithms in ConfSel or PairSel have any parameters except PairSel\-kNN, where we have22parameterskk, the number of nearest neighbors andλ\\lambda, the relative importance of distance for question and passage/SQL/Image for BoolQ/Spider and Bird/VisOnlyQA datasets, respectively\. We compare the total accuracy of for different values ofk=5,20k=5,20andλ=0\.1,1\.0,1\.5\\lambda=0\.1,1\.0,1\.5in addition to the baseline ofλ=0\.5,k=10\\lambda=0\.5,k=10for various combinations of datasets and classifiers\.

For each of these combinations, we report the performance of best kNN method in Table[4](https://arxiv.org/html/2605.30615#A1.T4)\. Again, to measure the maximum sensitivity to parameters, we compute the bestα≤0\.5\\alpha\\leq 0\.5, where the absolute difference between total accuracy of any hyperparameter combination and the baseline hyperparameter ofλ=0\.5,k=10\\lambda=0\.5,k=10is maximum\. We also measure the difference of each hyperparameter combination to the actual baseline hyperparameter choice ofλ=0\.5,k=10\\lambda=0\.5,k=10\. Note that we do not include the Synthetic dataset in Table[4](https://arxiv.org/html/2605.30615#A1.T4)as there is no notion ofλ\\lambdadefined for it\.

DatasetClassifierBestα\\alphaλ=0\.1\\lambda=0\.1k=10k=10λ=1\.0\\lambda=1\.0k=10k=10λ=1\.5\\lambda=1\.5k=10k=10λ=0\.5\\lambda=0\.5k=10k=10Baselineλ=0\.5\\lambda=0\.5k=5k=5λ=0\.5\\lambda=0\.5k=20k=20SpiderSQLCoder0\.3297\.57\(\+0\)97\.57\(\+0\)98\.11 \(\+ 0\.54\)97\.5797\.57\(\+0\)97\.57\(\+0\)Llama 3\.10\.0195\.68\(\+0\)95\.68\(\+0\)95\.68\(\+0\)95\.6895\.68\(\+0\)95\.68\(\+0\)Gemma 30\.2297\.3\(\+0\.54\)96\.76\(\+0\)96\.76\(\+0\)96\.7696\.76\(\+0\)96\.76\(\+0\)Qwen 30\.2994\.32\(\-0\.27\)94\.59\(\+0\)94\.59\(\+0\)94\.5994\.59\(\+0\)94\.32 \(\-0\.27\)BirdSQCoder0\.0190\.01\(\+0\)90\.01\(\+0\)90\.01\(\+0\)90\.0190\.01\(\+0\)90\.01\(\+0\)Llama 3\.10\.3595\.52\(\-0\.23\)95\.75\(\+0\)95\.64\(\-0\.11\)95\.7595\.75\(\+0\)95\.64 \(\-0\.11\)Gemma 30\.1395\.87\(\+0\.23\)95\.64\(\+0\)95\.64\(\+0\)95\.6495\.64\(\+0\)95\.64, \(\+0\)Qwen 30\.0179\.45\(\+0\)79\.45\(\+0\)79\.45\(\+0\)79\.4579\.45\(\+0\)79\.45\(\+0\)BoolQLlama 3\.10\.3886\.05\(\+0\)86\.05\(\+0\)86\.05\(\+0\)86\.0586\.21 \(\+0\.16\)86\.05\(\+0\)Gemma 30\.1980\.03\(\-0\.48\)80\.19\(\-0\.32\)80\.19\(\-0\.32\)80\.5180\.19\(\-0\.32\)80\.19 \(\-0\.32\)Qwen 30\.0177\.34\(\+0\)77\.34\(\+0\)77\.34\(\+0\)77\.3477\.34\(\+0\)77\.34\(\+0\)VisOnlyQAQwen 2\.5 VL0\.1967\.67\(\+0\.46\)67\.67\(\+0\.46\)67\.67 \(\+0\.46\)67\.2167\.67 \(\+0\.46\)67\.21\(\+0\)Gemma 30\.0151\.86\(\+0\)51\.86 \(\+0\)51\.86 \(\+0\)51\.8651\.86 \(\+0\)51\.86 \(\+0\)GLM 4\.1V0\.2663\.95 \(\+0\)63\.95\(\+0\)63\.95\(\+0\)63\.9565\.12 \(\+1\.17\)63\.95\(\+0\)

Table 4:Total accuracy of PairSel\-kNN for several hyperparameters for different classifiers and datasets\. For each row, the bestα∈\(0,0\.5\]\\alpha\\in\(0,0\.5\]is where the absolute difference between the performance any hyperparameter and the baselineλ=0\.5,k=10\\lambda=0\.5,k=10is the largest\. For each row, the baseline is reported without brackets and for all other hyperparameters, their difference in performance with respect to the baseline is shown inside the brackets\. The highest total accuracy in each row isbold\.From Table[4](https://arxiv.org/html/2605.30615#A1.T4), we can see that PairSel\-kNN is not sensitive to the choice of hyperparameters that we have taken, with either no difference in performance wrt baseline or very small difference\. The only case where changing hyperparameters increases performance by a substantial amount compared to the boost in performance of the kNN method over the raw baseline in Table[1](https://arxiv.org/html/2605.30615#S5.T1)is for GLM 4\.1V in the VisOnlyQA dataset, as their the kNN method itself does not improve performance by a lot\. Further, the sensitivity tokkis much lower than that ofλ\\lambda\. Additionally, the low sensitivity to hyperparameters of PairSel\-kNN, as well as its comparable performance to other pairwise methods, indicates that it is indeed the use of pairwise queries that provides the largest boost in performance\.

### A\.5SQL Embeddings

##### Existing SQL Distance metrics

Measuring the distance between SQLs is a key requirement for PairSel\-kNN\. If we can access a database for each SQL, we can execute the SQL and match the outputs of two SQLs\. This is termed as execution accuracy\[yu\-etal\-2018\-spider\]\. While being accurate, this requires access to sample data for the query which might not be available due to privacy considerations\. Extracting high quality sample data that obtains different outputs for different queries is also non\-trivial\[zhong\-etal\-2020\-semantic\]\. Treating SQLs as strings, we can compute metrics based on string matching\[crystal\_bleu\]or sentence embeddings\[ni\-etal\-2022\-sentence\], however these fail to incorporate the syntactic structure of SQLs\. SQLs, and in general, scripts in most programming languages, can be represented as an abstract syntax tree \(AST\)\[ast\], which is used by the compiler to execute the script\. CodeBLEU\[ren2020codebleumethodautomaticevaluation\]is a string matching distance metric which incorporates the AST structure\. Existing works incorporate information from the AST into neural architectures, but these are designed for different tasks :\[rat\-sql\]utilizes AST graph for schema linking,\[cao2023astormer\]uses AST information in positional encodings to a transformer for NL2SQL with AST decoder and\[hg2ast\]uses an AST\-based decoder for NL2SQL\.

##### Proposed SQL Embeddings

We propose an embedding for SQL statements which uses the SQL’s abstract syntax tree \(AST\)\. The AST of an SQL statement is a tree which mimics its execution by the compiler\. Its nodes consist of SQL commands, table and column names and aliases\. Its edges encode the logic and syntax of the SQL statement\. In our case, we consider a directed AST with edges from the leaf nodes to parents\. As this is a tree, it has a single root node with only incident edges, and a path from each leaf node to the root\. Each SQL statement is parsed into its AST using sqlglot\[sqlglot\]\. We further simplify this tree by resolving all ALIAS commands to their actual names\. In Figure[8](https://arxiv.org/html/2605.30615#A1.F8), we provide an example of a SQL command and its AST\. Note that here the ”SELECT” command is the root node\. We treat this AST graph as a Message Passsing Graph Neural Network\[mpnn\_intro\]\. In our GNN, there are two types of nodes – a\) SQL nodes, which contain SQL commands and operators and b\) content nodes, which contain table and column names\. Each node has a fixed node embedding vector obtained from a pre\-trained model\. For SQL nodes, we generate embeddings from a T5\[ni\-etal\-2022\-sentence\]model fine\-tuned on wikiSQL\[t5\_wikisql\]dataset to encode SQL\-specific information\. Content nodes are arbitrary names, so we use a Sentence\-T5 model\[ni\-etal\-2022\-sentence\]to encode them\. Each node’s embedding vector has dimension768768\. To compute a single embedding from each graph, we perform message passing with topological propagation\. In our topological propagation\[top\_sort\], each node is visited only after all nodes incident to it have been visited\. We now show the message passing for a given node\. For leaf nodes, the message is simply its node embeddings\. For any non\-leaf nodeii, the messagemim\_\{i\}is obtained by an attention\-based combination of its node embeddingsviv\_\{i\}and the messages of its neighbors,\{mj\}j∈nghbr​\(i\)\\\{m\_\{j\}\\\}\_\{j\\in\\mathrm\{nghbr\}\(i\)\}\. Here,nghbr​\(i\)\\mathrm\{nghbr\}\(i\)is the set of neighbors ofii\. IfXi∈ℝ\(1\+\|nghbr​\(i\)\|\)×768X\_\{i\}\\in\\mathbb\{R\}^\{\(1\+\\lvert\\mathrm\{nghbr\}\(i\)\\rvert\)\\times 768\}is the stacked version of these vectors, i\.e\.,Xi=\[vi⊺,m1⊺,m2⊺,…,\]X\_\{i\}=\[v\_\{i\}^\{\\intercal\},m\_\{1\}^\{\\intercal\},m\_\{2\}^\{\\intercal\},\\ldots,\], thenmi=softmax​\(Q​Xi​\(K​Xi\)⊺768\)​V​Xim\_\{i\}=\\mathrm\{softmax\}\(\\frac\{QX\_\{i\}\(KX\_\{i\}\)^\{\\intercal\}\}\{\\sqrt\{768\}\}\)VX\_\{i\}, whereQ,K,V∈ℝ768×768Q,K,V\\in\\mathbb\{R\}^\{768\\times 768\}are the query, key and value attention matrices to be learned\. Note that this aggregation scheme resembles that of graph attention networks\[veličković2018graph\]\. Further, by topological propgation, we ensure that the message at root node has utilized combinations of all nodes in the tree\. The message at root nodemr​o​o​tm\_\{root\}is defined as the embedding of the SQL statement\.

![Refer to caption](https://arxiv.org/html/2605.30615v1/figures/sql_ast.png)Figure 8:The Abstract Syntax Tree \(AST\) for the SQL statement : “select count\(\*\) as HashedEmails from hkg\_dim\_attribute where lower\(attribute\) like lower\(’%hashed%’\) limit 25”Using this SQL embedding allows us to define a distance metric between different SQL statements and thus a distance metric for PairSel\-kNN\. We compare with a naive baseline of treating SQL as vanilla sentences and using sentence T5 model\[ni\-etal\-2022\-sentence\]to obtain embeddings\. A concurrent work, FuncGMNEval\[zhan2024funcevalgmnevaluatingfunctionalcorrectness\], also utilizes SQL’s syntactical structure for a GNN\. Their input is a pair of SQL statements which are converted to their ROTs \(Relational Operational Trees\), which represents SQL’s execution plan based on relational algebra and are in general more representative of actual SQL execution than ASTs\. Then, treating the ROTs of the two SQL statements as two different GNNs,\[zhan2024funcevalgmnevaluatingfunctionalcorrectness\]train a GMN\(Graph Matching Network\)\[pmlr\-v97\-li19d\]\. A GMN performs message passing on each node of the graph learning the node embeddings, edge embeddings and cross\-graph attention weights to obtain a similarity score from their GMN\.

##### Comparison to baselines for distance between SQLs\.

We train our method and funcGMNEval on Spider\-Pair dataset provided by\[zhan2024funcevalgmnevaluatingfunctionalcorrectness\], which has pairs of SQL’s which are labeled0if they are similar and11if they’re dissimilar\. We set the training time for both methods to be same, around 1\.5 days\. To train our method on Spider\-Pair dataset, we adapt the training procedure of FuncGMNEval, but obtain the embeddings for the two SQLs separately using our AST GNNs and use cosine similarity between these vectors as the similarity metric instead of FuncGMNEval’s similarity score\.

To compare different methods to compute distance metrics, we plot the AUC for the test dataset of Spider\-Pair and the time taken to compute similarity of all pairs for the Bird dataset in Table[5](https://arxiv.org/html/2605.30615#A1.T5)\. We compare our method, FuncGMNEval\[zhan2024funcevalgmnevaluatingfunctionalcorrectness\], and vanilla Sentence Embeddings\. Further, we compute the time taken for computing all distances for all pairs of Bird, as it is the largest dataset we use \(\>900\>900samples\), and for PairSel\-kNN, we need to compute distances between all pairs\.

MethodAll Pairs InferenceTime on BirdTest AUC onSpider\-PairFuncGMNEval\[zhan2024funcevalgmnevaluatingfunctionalcorrectness\]5 min0\.801Ours15 s0\.798Sent Emb\[ni\-etal\-2022\-sentence\]4s0\.639Table 5:Test AUC on Spider\-Pair and inference time for distance metric on all pairs of SQLs in Bird\. Highest AUC and lowest inference time arebold\.From Table[5](https://arxiv.org/html/2605.30615#A1.T5), we can see that after similar training time, the performance of our method in terms of test AUC on the Spider\-Pair dataset is slightly worse than the baseline FuncGMNEval\. However, to compute the similarity between any two SQL statements, FuncGMNEval needs to perform a forward pass over the pair of SQL statements\. In contrast, our method computes the embedding only once for each SQL statement separately, resulting in one forward pass per SQL statement\. This makes our method much faster, by approximately an order of magnitude, when computing all pairs distance on the bird dataset Further, the sentence embedding baseline is faster than our method and FuncGMNEval, but its test AUC is extremely low as it considers SQL statements as sentences\. Overall, our method achieves a balance between relative speed and performance\. Thus, it can be utilized even in online settings, where the NL\-SQL pair arrive one\-by\-one and we need to compute the distance between every new SQL statement and all existing SQL statements\.

## Appendix BAdditional Theoretical Details

We first computeFclfF\_\{\\mathrm\{clf\}\}\.

fclf​\(n\)\\displaystyle f\_\{\\mathrm\{clf\}\}\(n\)=1\|S\|​∑xi∈S𝟏​\{y^i≠yi⋆\}\\displaystyle=\\frac\{1\}\{\\lvert S\\rvert\}\\sum\_\{x\_\{i\}\\in S\}\\mathbf\{1\}\\\{\\hat\{y\}\_\{i\}\\neq y\_\{i\}^\{\\star\}\\\}𝔼y^i,yi⋆​\[fclf​\(n\)\]\\displaystyle\\mathbb\{E\}\_\{\\hat\{y\}\_\{i\},y^\{\\star\}\_\{i\}\}\[f\_\{\\mathrm\{clf\}\}\(n\)\]=1\|S\|​∑xi∈S𝔼y^i,y⋆​\[𝟏​\{y^i≠yi⋆\}\]\\displaystyle=\\frac\{1\}\{\\lvert S\\rvert\}\\sum\_\{x\_\{i\}\\in S\}\\mathbb\{E\}\_\{\\hat\{y\}\_\{i\},y^\{\\star\}\}\[\\mathbf\{1\}\\\{\\hat\{y\}\_\{i\}\\neq y\_\{i\}^\{\\star\}\\\}\]=1\|S\|​∑xi∈S\(pclf,l​\(xi\)​\(1−p⋆​\(xi\)\)\+p⋆​\(xi\)​\(1−pclf,l​\(xi\)\)\)\\displaystyle=\\frac\{1\}\{\\lvert S\\rvert\}\\sum\_\{x\_\{i\}\\in S\}\(p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\)\(1\-p^\{\\star\}\(x\_\{i\}\)\)\+p^\{\\star\}\(x\_\{i\}\)\(1\-p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\)\)\)=1\|S\|​∑xi∈S\(pclf,l​\(xi\)\+p⋆​\(xi\)−2​pclf,l​\(xi\)​p⋆​\(xi\)\)\\displaystyle=\\frac\{1\}\{\\lvert S\\rvert\}\\sum\_\{x\_\{i\}\\in S\}\(p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\)\+p^\{\\star\}\(x\_\{i\}\)\-2p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\)p^\{\\star\}\(x\_\{i\}\)\)Fclf\\displaystyle F\_\{\\mathrm\{clf\}\}=𝔼x∼𝒟x​\[𝔼y^i,yi⋆​\[fclf​\(n\)\]\]=𝔼x∼𝒟x​\[pclf,l​\(x\)\+p⋆​\(x\)−2​pclf,l​\(x\)​p⋆​\(x\)\]\\displaystyle=\\mathbb\{E\}\_\{x\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\mathbb\{E\}\_\{\\hat\{y\}\_\{i\},y\_\{i\}^\{\\star\}\}\[f\_\{\\mathrm\{clf\}\}\(n\)\]\]=\\mathbb\{E\}\_\{x\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x\)\+p^\{\\star\}\(x\)\-2p\_\{\\mathrm\{clf\},l\}\(x\)p^\{\\star\}\(x\)\]Similarly, we can computeFhF\_\{h\}\.

fh​\(n\)≜\\displaystyle f\_\{h\}\(n\)\\triangleq1\|S\|​∑xi∈S𝟏​\{y~i≠yi⋆\}\\displaystyle\\frac\{1\}\{\\lvert S\\rvert\}\\sum\_\{x\_\{i\}\\in S\}\\mathbf\{1\}\\\{\\tilde\{y\}\_\{i\}\\neq y\_\{i\}^\{\\star\}\\\}𝔼y~i⋆,yi⋆​\(fh​\(n\)\)=\\displaystyle\\mathbb\{E\}\_\{\\tilde\{y\}\_\{i\}^\{\\star\},y\_\{i\}^\{\\star\}\}\(f\_\{h\}\(n\)\)=1\|S\|​∑x∈S\(ph​\(x\)\+p⋆​\(x\)−ph​\(x\)​p⋆​\(x\)\)\\displaystyle\\frac\{1\}\{\\lvert S\\rvert\}\\sum\_\{x\\in S\}\(p\_\{h\}\(x\)\+p^\{\\star\}\(x\)\-p\_\{h\}\(x\)p^\{\\star\}\(x\)\)Fh=\\displaystyle F\_\{h\}=𝔼x∼𝒟x​\[ph​\(x\)\+p⋆​\(x\)−2​ph​\(x\)​p⋆​\(x\)\]\\displaystyle\\mathbb\{E\}\_\{x\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{h\}\(x\)\+p^\{\\star\}\(x\)\-2p\_\{h\}\(x\)p^\{\\star\}\(x\)\]Further, by Chernoff bound based on iid sampled datapointsxi∼𝒟xx\_\{i\}\\sim\{\\mathcal\{D\}\}\_\{x\}, with probability1−δ1\-\\delta, we have,

max⁡\{\|fclf​\(n\)−Fclf\|,\|fh​\(n\)−Fh\|\}=𝒪​\(2​log⁡\(2/δ\)n\)\\displaystyle\\max\\\{\\lvert f\_\{\\mathrm\{clf\}\}\(n\)\-F\_\{\\mathrm\{clf\}\}\\rvert,\\,\\,\\lvert f\_\{h\}\(n\)\-F\_\{h\}\\rvert\\\}=\\mathcal\{O\}\\left\(\\sqrt\{\\frac\{2\\log\(2/\\delta\)\}\{n\}\}\\right\)\(3\)
### B\.1Sufficient Conditions for Pairwise Queries to be Beneficial

We need to showFclf,c​\(α\)≥Fclf,p​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\\geq F\_\{\\mathrm\{clf\},p\}\(\\alpha\)to show that pairwise queries are advantageous\. Here, we establish certain sufficient conditions independent of the actual models forpclf,c,p⋆,pclf,lp\_\{\\mathrm\{clf\},c\},p^\{\\star\},p\_\{\\mathrm\{clf\},l\}andphp\_\{h\}and the data distribution𝒟x\{\\mathcal\{D\}\}\_\{x\}\.

We definehc,hp:𝒳→\[0,1\]h\_\{c\},h\_\{p\}:\{\\mathcal\{X\}\}\\to\[0,1\]as the following\.

hc​\(x\)\\displaystyle h\_\{c\}\(x\)≜Prx′∼𝒟x⁡\[\|2​pclf,c​\(x′\)−1\|≥\|2​pclf,c​\(x\)−1\|\]\\displaystyle\\triangleq\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]hp​\(x\)\\displaystyle h\_\{p\}\(x\)≜Prx′∼𝒟x⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]⋅Prx′∼𝒟x⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]\\displaystyle\\triangleq\\sqrt\{\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]\\cdot\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]\}
###### Proposition 6\(Sufficient Conditions for Improvement\)\.

Under the following conditions,Fclf,p​\(α\)≤Fclf,c​\(α\)​∀αF\_\{\\mathrm\{clf\},p\}\(\\alpha\)\\leq F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\\forall\\alpha

1. 1\.∀x∈𝒳\\forall x\\in\{\\mathcal\{X\}\}, ifp⋆​\(x\)≥12,ph​\(x\)≥pclf,l​\(x\)p^\{\\star\}\(x\)\\geq\\frac\{1\}\{2\},\\,\\,p\_\{h\}\(x\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\), otherwiseph​\(x\)≤pclf,l​\(x\)p\_\{h\}\(x\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\.
2. 2\.∀x∈𝒳,hc​\(x\)≤hp​\(x\)\\forall x\\in\{\\mathcal\{X\}\},\\,\\,h\_\{c\}\(x\)\\leq h\_\{p\}\(x\)

The first condition implies that at each datapoint, the human is a better labeler than the classifier, and the second relates the accuracy ofpclf,cp\_\{\\mathrm\{clf\},c\}to the actual prediction of the labeler\. Note that ifpclf,c​\(x\)=pclf,l​\(x\),∀x∈𝒳p\_\{\\mathrm\{clf\},c\}\(x\)=p\_\{\\mathrm\{clf\},l\}\(x\),\\forall x\\in\{\\mathcal\{X\}\}, then the second condition can never be satisfied\. The consequence of this assumption is that\(2​p⋆​\(x\)−1\)​\(ph​\(x\)−pclf,l​\(x\)\)≥0\(2p^\{\\star\}\(x\)\-1\)\(p\_\{h\}\(x\)\-p\_\{\\mathrm\{clf\},l\}\(x\)\)\\geq 0and\(hp​\(x\)\)\(1−α\)​n−\(hc​\(x\)\)\(1−α\)​n≥0\(h\_\{p\}\(x\)\)^\{\(1\-\\alpha\)n\}\-\(h\_\{c\}\(x\)\)^\{\(1\-\\alpha\)n\}\\geq 0for allx∈𝒳x\\in\{\\mathcal\{X\}\}, therefore, their product is always positive\. Note that these conditions are very strict and are not satisfied by the models used in Assumption[3](https://arxiv.org/html/2605.30615#Thmtheorem3)and Theorems[4](https://arxiv.org/html/2605.30615#Thmtheorem4)and[5](https://arxiv.org/html/2605.30615#Thmtheorem5)\.

## Appendix CProofs

### C\.1Proofs of Lemmas[1](https://arxiv.org/html/2605.30615#Thmtheorem1)and[2](https://arxiv.org/html/2605.30615#Thmtheorem2)

##### Proof of Lemma[1](https://arxiv.org/html/2605.30615#Thmtheorem1)

Note that if a datapointxi∈Sα,c,i∈\[n\]x\_\{i\}\\in S\_\{\\alpha,c\},i\\in\[n\], for all datapointsj∈S∖Sα,cj\\in S\\setminus S\_\{\\alpha,c\}, we should have\|2​pclf,c​\(xj\)−1\|≥\|2​pclf,c​\(xi\)−1\|\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\_\{j\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)\-1\\rvert\. Note that ifxi∈Sα,cx\_\{i\}\\in S\_\{\\alpha,c\}, then we use the labely^i∼pclf,l​\(xi\)\\hat\{y\}\_\{i\}\\sim p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\), otherwise, we use the labely~i∼ph​\(xi\)\\tilde\{y\}\_\{i\}\\sim p\_\{h\}\(x\_\{i\}\)\. Therefore,

Fclf−Fclf,c​\(α\)\\displaystyle F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)=1n∑i=1n𝔼xi∼𝒟x\[𝟏\{xi∈Sα,c\}⋅\(2p⋆\(xi\)−1\)\(ph\(xi\)−pclf,l\(xi\)\]\\displaystyle=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\mathbb\{E\}\_\{x\_\{i\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\\bigg\[\{\\bm\{1\}\}\\\{x\_\{i\}\\in S\_\{\\alpha,c\}\\\}\\cdot\(2p^\{\\star\}\(x\_\{i\}\)\-1\)\(p\_\{h\}\(x\_\{i\}\)\-p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\)\\bigg\]Note that each term of the expectation is equal asxix\_\{i\}are iid\. Further,Pr⁡\[xi∈Sα,c\]=\(Prx⁡\[\|2​pclf,c​\(x\)−1\|≥\|2​pclf,c​\(xi\)−1\|\]\)\(1−α\)​n\\Pr\[x\_\{i\}\\in S\_\{\\alpha,c\}\]=\(\\Pr\_\{x\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\_\{i\}\)\-1\\rvert\]\)^\{\(1\-\\alpha\)n\}, as the other\(1−α\)​n\(1\-\\alpha\)npoints are sampled independently from𝒟x\{\\mathcal\{D\}\}\_\{x\}\.

##### Proof of Lemma[2](https://arxiv.org/html/2605.30615#Thmtheorem2)

The proof follows that of Lemma[1](https://arxiv.org/html/2605.30615#Thmtheorem1)withSα,cS\_\{\\alpha,c\}replaced bySα,pS\_\{\\alpha,p\}\. Note that an elementxi∈Sα,px\_\{i\}\\in S\_\{\\alpha,p\}if and only if there are two sets of pointsS1,S2S\_\{1\},S\_\{2\}, whereS1∩S2=ϕ,S1∪S2=S∖Sα,pS\_\{1\}\\cap S\_\{2\}=\\phi,S\_\{1\}\\cup S\_\{2\}=S\\setminus S\_\{\\alpha,p\}and\|S1\|=\|S2\|=\(1−α\)​n2\\lvert S\_\{1\}\\rvert=\\lvert S\_\{2\}\\rvert=\\frac\{\(1\-\\alpha\)n\}\{2\}, such that for each pointxj∈S1x\_\{j\}\\in S\_\{1\}pclf,l​\(xj\)≥pclf,l​\(xi\)p\_\{\\mathrm\{clf\},l\}\(x\_\{j\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\)and for each pointxj∈S2,pclf,l​\(xj\)≤pclf,l​\(xi\)x\_\{j\}\\in S\_\{2\},p\_\{\\mathrm\{clf\},l\}\(x\_\{j\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\_\{i\}\)\.S1S\_\{1\}andS2S\_\{2\}correspond to the sets\{x\(i\),i∈\[0,\(1−α\)​n2\]\}\\\{x\_\{\(i\)\},i\\in\[0,\\frac\{\(1\-\\alpha\)n\}\{2\}\]\\\}and\{x\(i\),i∈\[n​\(1\+α\)2,n\]\}\\\{x\_\{\(i\)\},i\\in\[\\frac\{n\(1\+\\alpha\)\}\{2\},n\]\\\}, the two parts of the sorted values on either side of the selected ”middle” portion\. Therefore, Pr\[x\_i∈S\_α, p\] = \(Pr\_x’∼D\_x\[p\_clf,l\(x’\)≤p\_clf,l\(x\)\]⋅Pr\_x’∼D\_x\[p\_clf,l\(x’\)≥p\_clf,l\(x\)\]\)^\(1\-α\)n, as each datapoint is sampled iid from𝒟x\{\\mathcal\{D\}\}\_\{x\}\.

### C\.2Proofs of Theorems[4](https://arxiv.org/html/2605.30615#Thmtheorem4)and[5](https://arxiv.org/html/2605.30615#Thmtheorem5)

#### C\.2\.1Preliminaries

In this section, we simplify the computation ofFclf−Fclf,c​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)andFclf−Fclf,p​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)by computing the termsPrx′∼𝒟x⁡\[\|2​pclf,c​\(x′\)−1\|≥\|2​pclf,c​\(x\)−1\|\]\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]andPrx′∼𝒟x⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]⋅Prx′∼𝒟x⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]\\\\ \\sqrt\{\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]\\cdot\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]\}in Lemmas[1](https://arxiv.org/html/2605.30615#Thmtheorem1)and[2](https://arxiv.org/html/2605.30615#Thmtheorem2)respectively\.

###### Lemma 7\.

Under Assumption[3](https://arxiv.org/html/2605.30615#Thmtheorem3)and access to a Pairwise Query Oracle, ifpclf,c​\(x′\)p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)andpclf,l​\(x′\)p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)are continuous distributions symmetric around12\\frac\{1\}\{2\}forx′∼𝒟xx^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\},∀x∈𝒳\\forall x\\in\{\\mathcal\{X\}\}, we have,

Prx′∼𝒟x⁡\[\|2​pclf,c​\(x′\)−1\|≥\|2​pclf,c​\(x\)−1\|\]=Prx′⁡\[\|⟨wclf,c,φ​\(x′\)⟩\|≥\|⟨wclf,c,φ​\(x\)⟩\|\]\\displaystyle\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]=\\Pr\_\{x^\{\\prime\}\}\[\\lvert\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x^\{\\prime\}\)\\rangle\\rvert\\geq\\lvert\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\\rvert\]Prx′∼𝒟x⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]⋅Prx′∼𝒟x⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]≥Prx′⁡\[⟨wclf,l,φ​\(x′\)⟩≥\|⟨wclf,l,φ​\(x′\)⟩\|\]\\displaystyle\\sqrt\{\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]\\cdot\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]\}\\geq\\Pr\_\{x^\{\\prime\}\}\[\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x^\{\\prime\}\)\\rangle\\geq\\lvert\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x^\{\\prime\}\)\\rangle\\rvert\]

###### Proof\.

Consider the first inequality\. Eitherpclf,c​\(x\)≥12p\_\{\\mathrm\{clf\},c\}\(x\)\\geq\\frac\{1\}\{2\}orpclf,c<12p\_\{\\mathrm\{clf\},c\}<\\frac\{1\}\{2\}\. Ifpclf,c≥12p\_\{\\mathrm\{clf\},c\}\\geq\\frac\{1\}\{2\}, we need\|2​pclf,c​\(x′\)−1\|≥2​pclf,c​\(x\)−1\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1which impliespclf,c​\(x′\)≥pclf,c​\(x\)p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},c\}\(x\)orpclf,c​\(x′\)≤1−pclf,c​\(x\)p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\leq 1\-p\_\{\\mathrm\{clf\},c\}\(x\)\. Ifpclf,c​\(x\)<12p\_\{\\mathrm\{clf\},c\}\(x\)<\\frac\{1\}\{2\}, we need\|2​pclf,c​\(x′\)−1\|≥1−2​pclf,c​\(x\)\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq 1\-2p\_\{\\mathrm\{clf\},c\}\(x\), which impliespclf,c​\(x′\)≥1−pclf,c​\(x\)p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\geq 1\-p\_\{\\mathrm\{clf\},c\}\(x\)orpclf,c​\(x′\)≤pclf,c​\(x\)p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},c\}\(x\)\.

Prx′∼𝒟x⁡\[\|2​pclf,c​\(x′\)−1\|≥\|2​pclf,c​\(x\)−1\|\]\\displaystyle\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]=𝟏​\{pclf,c​\(x\)≥12\}​\(Prx′⁡\[pclf,c​\(x′\)≥pclf,c​\(x\)\]\+Prx′⁡\[pclf,c​\(x′\)≤1−pclf,c​\(x\)\]\)\\displaystyle=\{\\bm\{1\}\}\\\{p\_\{\\mathrm\{clf\},c\}\(x\)\\geq\\frac\{1\}\{2\}\\\}\(\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},c\}\(x\)\]\+\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\leq 1\-p\_\{\\mathrm\{clf\},c\}\(x\)\]\)\+𝟏​\{pclf,c​\(x\)<12\}​\(Prx′⁡\[pclf,c​\(x′\)≤pclf,c​\(x\)\]\+Prx′⁡\[pclf,c​\(x′\)≥1−pclf,c​\(x\)\]\)\\displaystyle\\quad\+\{\\bm\{1\}\}\\\{p\_\{\\mathrm\{clf\},c\}\(x\)<\\frac\{1\}\{2\}\\\}\(\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},c\}\(x\)\]\+\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\geq 1\-p\_\{\\mathrm\{clf\},c\}\(x\)\]\)Ifpclf,c​\(x\)p\_\{\\mathrm\{clf\},c\}\(x\)is symmetric about12\\frac\{1\}\{2\}, thenPrx′⁡\[pclf,c​\(x′\)≥a\]=Prx′⁡\[pclf,c​\(x′\)≤1−a\]\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\geq a\]=\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\leq 1\-a\]for anya∈\(0,1\)a\\in\(0,1\)\. This implies the following equation\.

Prx′∼𝒟x⁡\[\|2​pclf,c​\(x′\)−1\|≥\|2​pclf,c​\(x\)−1\|\]\\displaystyle\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]=𝟏​\{pclf,c​\(x\)≥12\}​2​Prx′⁡\[pclf,c​\(x′\)≥pclf,c​\(x\)\]\+𝟏​\{pclf,c​\(x\)<12\}​2​Prx′⁡\[pclf,c​\(x′\)≤pclf,c​\(x\)\]\\displaystyle=\{\\bm\{1\}\}\\\{p\_\{\\mathrm\{clf\},c\}\(x\)\\geq\\frac\{1\}\{2\}\\\}2\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},c\}\(x\)\]\+\{\\bm\{1\}\}\\\{p\_\{\\mathrm\{clf\},c\}\(x\)<\\frac\{1\}\{2\}\\\}2\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},c\}\(x\)\]From Assumption[3](https://arxiv.org/html/2605.30615#Thmtheorem3), if⟨wclf,c,φ​\(x\)⟩≤0\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\\leq 0,g​\(⟨wclf,c,φ​\(x\)⟩\)≤12g\(\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\)\\leq\\frac\{1\}\{2\}and if⟨wclf,c,φ​\(x\)⟩≥0\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\\geq 0,g​\(⟨wclf,c,φ​\(x\)⟩\)≥12g\(\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\)\\geq\\frac\{1\}\{2\}\. Therefore, we have the following condition\.

Prx′∼𝒟x⁡\[\|2​pclf,c​\(x′\)−1\|≥\|2​pclf,c​\(x\)−1\|\]=Prx′⁡\[\|⟨wclf,c,φ​\(x′\)⟩\|≥\|⟨wclf,c,φ​\(x\)⟩\|\]\\displaystyle\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]=\\Pr\_\{x^\{\\prime\}\}\[\\lvert\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x^\{\\prime\}\)\\rangle\\rvert\\geq\\lvert\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\\rvert\]
Now, consider the second equation in the Lemma\. For continuous distributionspclf,l​\(x′\),x′∼𝒟xp\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\),x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}, we have,

Prx′⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]=1−Prx′⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]\\displaystyle\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]=1\-\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]Further, ifpclf,l​\(x′\)p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)is symmetric about12\\frac\{1\}\{2\}, then,

pclf,l​\(x\)≥12\\displaystyle p\_\{\\mathrm\{clf\},l\}\(x\)\\geq\\frac\{1\}\{2\}\\quad⟹Prx′⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]=12−Prx′⁡\[pclf,l​\(x′\)∈\[1/2,pclf,l​\(x\)\]\],\\displaystyle\\implies\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]=\\frac\{1\}\{2\}\-\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\in\[\\nicefrac\{\{1\}\}\{\{2\}\},p\_\{\\mathrm\{clf\},l\}\(x\)\]\],⟹Prx′⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]=12\+Prx′⁡\[pclf,l​\(x′\)∈\[1/2,pclf,l​\(x\)\]\],\\displaystyle\\implies\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]=\\frac\{1\}\{2\}\+\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\in\[\\nicefrac\{\{1\}\}\{\{2\}\},p\_\{\\mathrm\{clf\},l\}\(x\)\]\],pclf,l​\(x\)<12\\displaystyle p\_\{\\mathrm\{clf\},l\}\(x\)<\\frac\{1\}\{2\}\\quad⟹Prx′⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]=12\+Prx′⁡\[pclf,l​\(x′\)∈\[pclf,l​\(x\),1/2\]\],\\displaystyle\\implies\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]=\\frac\{1\}\{2\}\+\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\in\[p\_\{\\mathrm\{clf\},l\}\(x\),\\nicefrac\{\{1\}\}\{\{2\}\}\]\],⟹Prx′⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]=12−Prx′⁡\[pclf,l​\(x′\)∈\[pclf,l​\(x\)\],1/2\]\\displaystyle\\implies\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]=\\frac\{1\}\{2\}\-\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\in\[p\_\{\\mathrm\{clf\},l\}\(x\)\],\\nicefrac\{\{1\}\}\{\{2\}\}\]LetP​\(x\)≜Prx′⁡\[pclf,l​\(x′\)∈\[1/2,max​\{pclf,l​\(x\),1−pclf,l​\(x\)\]\}\]P\(x\)\\triangleq\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\in\[\\nicefrac\{\{1\}\}\{\{2\}\},\\max\\\{p\_\{\\mathrm\{clf\},l\}\(x\),1\-p\_\{\\mathrm\{clf\},l\}\(x\)\]\\\}\], then,

Prx′∼𝒟x⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]⋅Prx′∼𝒟x⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]\\displaystyle\\sqrt\{\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]\\cdot\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]\}=\(12−P​\(x\)\)​\(12−P​\(x\)\)=14−\(P​\(x\)\)2\\displaystyle=\\sqrt\{\\left\(\\frac\{1\}\{2\}\-P\(x\)\\right\)\\left\(\\frac\{1\}\{2\}\-P\(x\)\\right\)\}=\\sqrt\{\\frac\{1\}\{4\}\-\(P\(x\)\)^\{2\}\}≥12−P\(x\)=Prx′\[pclf,l\(x′\)≥max\{pclf,l\(x\),1−pclf,l\(x\)\]\\displaystyle\\geq\\frac\{1\}\{2\}\-P\(x\)=\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq\\max\\\{p\_\{\\mathrm\{clf\},l\}\(x\),1\-p\_\{\\mathrm\{clf\},l\}\(x\)\]From Assumption[3](https://arxiv.org/html/2605.30615#Thmtheorem3), using the symmetry ofgg, we obtain,max⁡\{pclf,l​\(x\),1−pclf,l​\(x\)\}=g​\(\|⟨wclf,l,φ​\(x\)⟩\|\)\\max\\\{p\_\{\\mathrm\{clf\},l\}\(x\),1\-p\_\{\\mathrm\{clf\},l\}\(x\)\\\}=g\(\\lvert\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x\)\\rangle\\rvert\)\. Therefore,

Prx′\[pclf,l\(x′\)≥max\{pclf,l\(x\),1−pclf,l\(x\)\]=Prx′\[⟨wclf,l,φ\(x′\)⟩≥\|⟨wclf,l,φ\(x′\)⟩\|\]\\displaystyle\\Pr\_\{x^\{\\prime\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq\\max\\\{p\_\{\\mathrm\{clf\},l\}\(x\),1\-p\_\{\\mathrm\{clf\},l\}\(x\)\]=\\Pr\_\{x^\{\\prime\}\}\[\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x^\{\\prime\}\)\\rangle\\geq\\lvert\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x^\{\\prime\}\)\\rangle\\rvert\]
∎

#### C\.2\.2Proof of Theorem[4](https://arxiv.org/html/2605.30615#Thmtheorem4)

Letu=φ​\(x\),∀x∈𝒳u=\\varphi\(x\),\\forall x\\in\{\\mathcal\{X\}\}, thenu∼Unif​\(𝕊d−1\)u\\sim\\mathrm\{Unif\}\(\\mathbb\{S\}^\{d\-1\}\)\.

Letu=ρ​wclf,c\+1−ρ2​u⟂=λ​wclf,l\+1−λ2​v⟂u=\\rho w\_\{\\mathrm\{clf\},c\}\+\\sqrt\{1\-\\rho^\{2\}\}u\_\{\\perp\}=\\lambda w\_\{\\mathrm\{clf\},l\}\+\\sqrt\{1\-\\lambda^\{2\}\}v\_\{\\perp\}, where⟨u⟂,wclf,c,=⟩​0\\langle u\_\{\\perp\},w\_\{\\mathrm\{clf\},c\},=\\rangle 0and⟨v⟂,wclf,l⟩=0\\langle v\_\{\\perp\},w\_\{\\mathrm\{clf\},l\}\\rangle=0\. Forx∼Unif​\(𝕊d−1\)x\\sim\\mathrm\{Unif\}\(\\mathbb\{S\}^\{d\-1\}\), we need to find the distibutions onρ,λ,u⟂\\rho,\\lambda,u\_\{\\perp\}andv⟂v\_\{\\perp\}respectively\.

Note thatu⟂u\_\{\\perp\}is sampled uniformly from thed−1d\-1\-dimensional unit sphere ofdd\-dimensional unit vectors that are perpendicular towclf,cw\_\{\\mathrm\{clf\},c\}, whilev⟂v\_\{\\perp\}is sampled from a similar distribution of vectors perpendicular towclf,lw\_\{\\mathrm\{clf\},l\}\.

As forρ\\rhoandλ\\lambda, note that these belong to the same distribution, as the projection of a vector uniformly sampled from the unit sphere on a fixed vector, which iswclf,cw\_\{\\mathrm\{clf\},c\}andwclf,lw\_\{\\mathrm\{clf\},l\}forρ\\rhoandλ\\lambdarespectively\. Further, ifAd​\(r\)A\_\{d\}\(r\)is the surface area of thedd\-dimensional unit sphere of radiusrr\. Then,∀a,b∈\[−1,1\]\\forall a,b\\in\[\-1,1\]witha≤ba\\leq b, we have,

Pr⁡\[ρ∈\[a,b\]\]=Pr⁡\[λ∈\[a,b\]\]=∫abAd−1​\(1−a2\)Ad​\(1\)​𝑑a=Ad−1​\(1\)Ad​\(1\)​∫ab\(1−a2\)d−3​𝑑a\\displaystyle\\Pr\[\\rho\\in\[a,b\]\]=\\Pr\[\\lambda\\in\[a,b\]\]=\\int\_\{a\}^\{b\}\\frac\{A\_\{d\-1\}\(\\sqrt\{1\-a^\{2\}\}\)\}\{A\_\{d\}\(1\)\}da=\\frac\{A\_\{d\-1\}\(1\)\}\{A\_\{d\}\(1\)\}\\int\_\{a\}^\{b\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\(4\)Note that the surface area of sphere which has a projectionaaon a fixed vector is simply the surface area of thed−1d\-1dimensional sphere of radius1−a2\\sqrt\{1\-a^\{2\}\}\.

We obtain the following conditions\.

ρ⟂u⟂​and​λ⟂v⟂,𝔼​\[λ\]=𝔼​\[ρ\]=0,𝔼​\[λ2\]=𝔼​\[ρ2\]=1d,𝔼​\[u⟂\]=𝔼​\[v⟂\]=0\\displaystyle\\rho\\perp u\_\{\\perp\}\\text\{ and \}\\lambda\\perp v\_\{\\perp\},\\quad\\mathbb\{E\}\[\\lambda\]=\\mathbb\{E\}\[\\rho\]=0,\\quad\\mathbb\{E\}\[\\lambda^\{2\}\]=\\mathbb\{E\}\[\\rho^\{2\}\]=\\frac\{1\}\{d\},\\quad\\mathbb\{E\}\[u\_\{\\perp\}\]=\\mathbb\{E\}\[v\_\{\\perp\}\]=0\(5\)𝔼​\[u⟂​u⟂⊤\]=1d−1​\(Id−wclf,c​wclf,c⊤\),𝔼​\[v⟂​v⟂⊤\]=1d−1​\(Id−wclf,l​wclf,l⊤\)\\displaystyle\\quad\\mathbb\{E\}\[u\_\{\\perp\}u\_\{\\perp\}^\{\\top\}\]=\\frac\{1\}\{d\-1\}\(I\_\{d\}\-w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\),\\quad\\mathbb\{E\}\[v\_\{\\perp\}v\_\{\\perp\}^\{\\top\}\]=\\frac\{1\}\{d\-1\}\(I\_\{d\}\-w\_\{\\mathrm\{clf\},l\}w\_\{\\mathrm\{clf\},l\}^\{\\top\}\)We decompose the difference term\(2​p⋆​\(x\)−1\)​\(ph​\(x\)−pclf,l​\(x\)\)\(2p^\{\\star\}\(x\)\-1\)\(p\_\{h\}\(x\)\-p\_\{\\mathrm\{clf\},l\}\(x\)\)

\(2​p⋆​\(x\)−1\)​\(ph​\(x\)−pclf,l​\(x\)\)\\displaystyle\(2p^\{\\star\}\(x\)\-1\)\(p\_\{h\}\(x\)\-p\_\{\\mathrm\{clf\},l\}\(x\)\)=⟨w⋆,u⟩​⟨u,wh−wclf,l⟩=\(w⋆\)⊤​u​u⊤​\(wh−wclf,l\)\\displaystyle=\\langle w^\{\\star\},u\\rangle\\langle u,w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\\rangle=\(w^\{\\star\}\)^\{\\top\}uu^\{\\top\}\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\(6\)
Now, we can computeFclf−Fclf,c​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)andFclf−Fclf,p​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\. As we need a positive lower bound onFclf,c​\(α\)−Fclf,p​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)to show thatFclf,c​\(α\)≥Fclf,p​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\\geq F\_\{\\mathrm\{clf\},p\}\(\\alpha\), we compute an upper bound onFclf−Fclf,c​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)and a lower bound onFclf−Fclf,p​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\.

##### Upper bound on ConfSel\.

Note that the distribution ofpclf,l​\(x\)p\_\{\\mathrm\{clf\},l\}\(x\)andpclf,c​\(x\)p\_\{\\mathrm\{clf\},c\}\(x\)is symmetric around12\\frac\{1\}\{2\}for spherical features and linear link function\. So, we use Lemma[7](https://arxiv.org/html/2605.30615#Thmtheorem7)and equation[4](https://arxiv.org/html/2605.30615#A3.E4)to boundFclf−Fclf,c​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\.

Prx′∼𝒟x⁡\[\|2​pclf,c​\(x′\)−1\|≥\|2​pclf,c​\(x\)−1\|\]\\displaystyle\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]=Prx′⁡\[\|⟨wclf,c,φ​\(x′\)⟩\|≥\|⟨wclf,c,φ​\(x\)⟩\|\]=C1​∫\|ρ\|1\(1−a2\)d−3​𝑑a\\displaystyle=\\Pr\_\{x^\{\\prime\}\}\[\\lvert\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x^\{\\prime\}\)\\rangle\\rvert\\geq\\lvert\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\\rvert\]=C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}dawhereC1=2​Ad−1​\(1\)Ad​\(1\)C\_\{1\}=\\frac\{2A\_\{d\-1\}\(1\)\}\{A\_\{d\}\(1\)\}\.

To computeFclf−Fclf,c​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\), we can further simplify equation[6](https://arxiv.org/html/2605.30615#A3.E6)\.

u​u⊤=ρ2​wclf,c​wclf,c⊤\+\(1−ρ2\)​u⟂​u⟂⊤\+ρ​1−ρ2​\(wclf,c​u⟂⊤\+u⟂​wclf,c⊤\)\\displaystyle uu^\{\\top\}=\\rho^\{2\}w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\+\(1\-\\rho^\{2\}\)u\_\{\\perp\}u\_\{\\perp\}^\{\\top\}\+\\rho\\sqrt\{1\-\\rho^\{2\}\}\(w\_\{\\mathrm\{clf\},c\}u\_\{\\perp\}^\{\\top\}\+u\_\{\\perp\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\)Using equation[5](https://arxiv.org/html/2605.30615#A3.E5)and equation[6](https://arxiv.org/html/2605.30615#A3.E6), we obtain

Fclf−Fclf,c​\(α\)\\displaystyle F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)=𝔼x​\[\(C1​∫\|ρ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​\(w⋆\)⊤​u​u⊤​\(wh−wclf,l\)\]\\displaystyle=\\mathbb\{E\}\_\{x\}\\left\[\\left\(C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}\(w^\{\\star\}\)^\{\\top\}uu^\{\\top\}\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\\right\]=\(w⋆\)⊤​𝔼x​\[\(C1​∫\|ρ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​u​u⊤\]​\(wh−wclf,l\)\\displaystyle=\(w^\{\\star\}\)^\{\\top\}\\mathbb\{E\}\_\{x\}\\left\[\\left\(C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}uu^\{\\top\}\\right\]\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)Therefore, we only need to compute𝔼x​\[\(C1​∫\|ρ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​u​u⊤\]\\mathbb\{E\}\_\{x\}\\left\[\\left\(C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}uu^\{\\top\}\\right\]\.

𝔼x​\[\(C1​∫\|ρ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​u​u⊤\]\\displaystyle\\mathbb\{E\}\_\{x\}\\left\[\\left\(C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}uu^\{\\top\}\\right\]=𝔼x\[\(C1∫\|ρ\|1\(1−a2\)d−3da\)\(1−α\)​n\\displaystyle=\\mathbb\{E\}\_\{x\}\\Bigg\[\\left\(C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}⋅\(ρ2wclf,cwclf,c⊤\+\(1−ρ2\)u⟂u⟂⊤\+ρ1−ρ2\(wclf,cu⟂⊤\+u⟂wclf,c⊤\)\)\]\\displaystyle\\quad\\cdot\(\\rho^\{2\}w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\+\(1\-\\rho^\{2\}\)u\_\{\\perp\}u\_\{\\perp\}^\{\\top\}\+\\rho\\sqrt\{1\-\\rho^\{2\}\}\(w\_\{\\mathrm\{clf\},c\}u\_\{\\perp\}^\{\\top\}\+u\_\{\\perp\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\)\)\\Bigg\]=𝔼ρ​\[\(C1​∫\|ρ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​ρ2\]​wclf,c​wclf,c⊤⏟1\\displaystyle=\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{1\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\rho\}\\left\[\\left\(C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}\\rho^\{2\}\\right\]w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\}\}\+𝔼ρ​\[\(C1​∫\|ρ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​\(1−ρ2\)\]​𝔼u⟂​\[u⟂​u⟂⊤\]⏟2\\displaystyle\\quad\+\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{2\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\rho\}\\left\[\\left\(C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}\(1\-\\rho^\{2\}\)\\right\]\\mathbb\{E\}\_\{u\_\{\\perp\}\}\[u\_\{\\perp\}u\_\{\\perp\}^\{\\top\}\]\}\}\+𝔼ρ​\[\(C1​∫\|ρ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​ρ​1−ρ2\]​\(wclf,c​𝔼u⟂​\[u⟂⊤\]\+𝔼u⟂​\[u⟂\]​wclf,c⊤\)⏟3\\displaystyle\\quad\+\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{3\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\rho\}\\left\[\\left\(C\_\{1\}\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}\\rho\\sqrt\{1\-\\rho^\{2\}\}\\right\]\(w\_\{\\mathrm\{clf\},c\}\\mathbb\{E\}\_\{u\_\{\\perp\}\}\[u\_\{\\perp\}^\{\\top\}\]\+\\mathbb\{E\}\_\{u\_\{\\perp\}\}\[u\_\{\\perp\}\]w\_\{\\mathrm\{clf\},c\}^\{\\top\}\)\}\}
In the last step, we use the independence ofρ\\rhoandu⟂u\_\{\\perp\}\. From equation[5](https://arxiv.org/html/2605.30615#A3.E5), the term3is0\. Further,𝔼u⟂​\[u⟂​u⟂⊤\]=1d−1​\(Id−wclf,c​wclf,c⊤\)\\mathbb\{E\}\_\{u\_\{\\perp\}\}\[u\_\{\\perp\}u\_\{\\perp\}^\{\\top\}\]=\\frac\{1\}\{d\-1\}\(I\_\{d\}\-w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\)\.

We useE0≜𝔼ρ​\[\(h​\(ρ\)\)\(1−α\)​n\]E\_\{0\}\\triangleq\\mathbb\{E\}\_\{\\rho\}\[\(h\(\\rho\)\)^\{\(1\-\\alpha\)n\}\]andE2≜=𝔼ρ\[\(h\(ρ\)\)\(1−α\)​nρ2\]E\_\{2\}\\triangleq=\\mathbb\{E\}\_\{\\rho\}\[\(h\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\rho^\{2\}\]whereh​\(ρ\)=∫\|ρ\|1\(1−a2\)d−3​𝑑ah\(\\rho\)=\\int\_\{\\lvert\\rho\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\. Further, we setWclf,c=wclf,c​wclf,c⊤W\_\{\\mathrm\{clf\},c\}=w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}Then, we can compute the required non\-zero terms in above expression\.

1=C1\(1−α\)​n​E2​Wclf,c,2=C1\(1−α\)​n​\(E0−E2\)d−1​\(Id−Wclf,c\)\\displaystyle=C\_\{1\}^\{\(1\-\\alpha\)n\}E\_\{2\}W\_\{\\mathrm\{clf\},c\},\\quad\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{2\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=\\frac\{C\_\{1\}^\{\(1\-\\alpha\)n\}\(E\_\{0\}\-E\_\{2\}\)\}\{d\-1\}\(I\_\{d\}\-W\_\{\\mathrm\{clf\},c\}\)\(7\)

##### Lower bound for PairSel\-Middle\.

The lower bound onFclf−Fclf,p​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)proceeds in a similar fashion to the upper bound onFclf−Fclf,c​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\. Note thatpclf,l​\(x\)p\_\{\\mathrm\{clf\},l\}\(x\)is also symmetric around12\\frac\{1\}\{2\}under spherical features and linear link function\. We use Lemma[7](https://arxiv.org/html/2605.30615#Thmtheorem7)and equation[4](https://arxiv.org/html/2605.30615#A3.E4)to bound the probability term\.

Prx′∼𝒟x⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]⋅Prx′∼𝒟x⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]\\displaystyle\\sqrt\{\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]\\cdot\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]\}≥Prx′∼𝒟x⁡\[⟨wclf,l,φ​\(x′\)⟩≥\|⟨wclf,l,φ​\(x′\)⟩\|\]\\displaystyle\\geq\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x^\{\\prime\}\)\\rangle\\geq\\lvert\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x^\{\\prime\}\)\\rangle\\rvert\]=C2​∫\|λ\|1\(1−a2\)d−3​𝑑s\\displaystyle=C\_\{2\}\\int\_\{\\lvert\\lambda\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}dswhereC2=Ad−1​\(1\)Ad​\(1\)=C12C\_\{2\}=\\frac\{A\_\{d\-1\}\(1\)\}\{A\_\{d\}\(1\)\}=\\frac\{C\_\{1\}\}\{2\}\. Note that this expression is exactly half that of the corresponding term for confidence queries\. Further, we can use a similar decomposition ofu​u⊤uu^\{\\top\}in terms ofλ\\lambdaandv⟂v\_\{\\perp\}to obtain,

Fclf−Fclf,p​\(α\)=\(w⋆\)⊤​𝔼x​\[\(C2​∫\|λ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​u​u⊤\]​\(wh−wclf,l\)\\displaystyle F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)=\(w^\{\\star\}\)^\{\\top\}\\mathbb\{E\}\_\{x\}\\left\[\\left\(C\_\{2\}\\int\_\{\\lvert\\lambda\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}uu^\{\\top\}\\right\]\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)Now, following the decomposition ofu​u⊤uu^\{\\top\}in terms ofλ\\lambdaandv⟂v\_\{\\perp\}, we obtain,

𝔼x​\[\(C2​∫\|λ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​u​u⊤\]\\displaystyle\\mathbb\{E\}\_\{x\}\\left\[\\left\(C\_\{2\}\\int\_\{\\lvert\\lambda\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}uu^\{\\top\}\\right\]=𝔼λ​\[\(C2​∫\|λ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​λ2\]​wclf,l​wclf,l⊤⏟4\\displaystyle=\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{4\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\lambda\}\\left\[\\left\(C\_\{2\}\\int\_\{\\lvert\\lambda\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}\\lambda^\{2\}\\right\]w\_\{\\mathrm\{clf\},l\}w\_\{\\mathrm\{clf\},l\}^\{\\top\}\}\}\+𝔼λ​\[\(C2​∫\|λ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​\(1−λ2\)\]​𝔼clf⟂​\[v⟂​v⟂⊤\]⏟5\\displaystyle\\quad\+\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{5\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\lambda\}\\left\[\\left\(C\_\{2\}\\int\_\{\\lvert\\lambda\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}\(1\-\\lambda^\{2\}\)\\right\]\\mathbb\{E\}\_\{\\mathrm\{clf\}\_\{\\perp\}\}\[v\_\{\\perp\}v\_\{\\perp\}^\{\\top\}\]\}\}\+𝔼λ​\[\(C2​∫\|λ\|1\(1−a2\)d−3​𝑑a\)\(1−α\)​n​λ​1−λ2\]​\(wclf,l​𝔼clf⟂​\[v⟂⊤\]\+𝔼clf⟂​\[v⟂\]​wclf,l⊤\)⏟6\\displaystyle\\quad\+\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{6\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\lambda\}\\left\[\\left\(C\_\{2\}\\int\_\{\\lvert\\lambda\\rvert\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\right\)^\{\(1\-\\alpha\)n\}\\lambda\\sqrt\{1\-\\lambda^\{2\}\}\\right\]\(w\_\{\\mathrm\{clf\},l\}\\mathbb\{E\}\_\{\\mathrm\{clf\}\_\{\\perp\}\}\[v\_\{\\perp\}^\{\\top\}\]\+\\mathbb\{E\}\_\{\\mathrm\{clf\}\_\{\\perp\}\}\[v\_\{\\perp\}\]w\_\{\\mathrm\{clf\},l\}^\{\\top\}\)\}\}The term6is0from equation[5](https://arxiv.org/html/2605.30615#A3.E5)\. Further, the terms4and5can be computed in terms ofE0E\_\{0\}andE2E\_\{2\}\. We setWclf,l=wclf,l​wclf,l⊤W\_\{\\mathrm\{clf\},l\}=w\_\{\\mathrm\{clf\},l\}w\_\{\\mathrm\{clf\},l\}^\{\\top\}\.

4=C2\(1−α\)​n​E2​Wclf,l,5=C2\(1−α\)​n​\(E0−E2\)d−1​\(Id−Wclf,l\)\\displaystyle\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{4\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=C\_\{2\}^\{\(1\-\\alpha\)n\}E\_\{2\}W\_\{\\mathrm\{clf\},l\},\\quad\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{5\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=\\frac\{C\_\{2\}^\{\(1\-\\alpha\)n\}\(E\_\{0\}\-E\_\{2\}\)\}\{d\-1\}\(I\_\{d\}\-W\_\{\\mathrm\{clf\},l\}\)\(8\)

##### Difference of errors

We can now compute the termFclf,c​\(α\)−Fclf,p​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\.

Fclf,c​\(α\)−Fclf,p​\(α\)\\displaystyle F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)=\(Fclf−Fclf,p​\(α\)\)−\(Fclf−Fclf,c​\(α\)\)\\displaystyle=\(F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\)\-\(F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\)=\(w⋆\)⊤​\(4\+5−1−2\)​\(wh−wclf,l\)\\displaystyle=\(w^\{\\star\}\)^\{\\top\}\(\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{4\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\+\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{5\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{1\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{2\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\)\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)We first compute the difference4\+5−1−2\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{4\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\+\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{5\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{1\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{2\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\.

4\+5−1−2\\displaystyle\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{4\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\+\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{5\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{1\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{2\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=C2\(1−α\)​n​E2​Wclf,l\+C2\(1−α\)​n​\(E0−E2\)d−1​\(Id−Wclf,l\)\\displaystyle=C\_\{2\}^\{\(1\-\\alpha\)n\}E\_\{2\}W\_\{\\mathrm\{clf\},l\}\+\\frac\{C\_\{2\}^\{\(1\-\\alpha\)n\}\(E\_\{0\}\-E\_\{2\}\)\}\{d\-1\}\(I\_\{d\}\-W\_\{\\mathrm\{clf\},l\}\)−C1\(1−α\)​n​E2​Wclf,c−C1\(1−α\)​n​\(E0−E2\)d−1​\(Id−Wclf,c\)\\displaystyle\-C\_\{1\}^\{\(1\-\\alpha\)n\}E\_\{2\}W\_\{\\mathrm\{clf\},c\}\-\\frac\{C\_\{1\}^\{\(1\-\\alpha\)n\}\(E\_\{0\}\-E\_\{2\}\)\}\{d\-1\}\(I\_\{d\}\-W\_\{\\mathrm\{clf\},c\}\)=C2\(1−α\)​nd−1\(\(E0−dE2\)\(2\(1−α\)​n−Wclf,lWclf,c\)\\displaystyle=\\frac\{C\_\{2\}^\{\(1\-\\alpha\)n\}\}\{d\-1\}\\bigg\(\(E\_\{0\}\-dE\_\{2\}\)\(2^\{\(1\-\\alpha\)n\-W\_\{\\mathrm\{clf\},l\}\}W\_\{\\mathrm\{clf\},c\}\)−\(2\(1−α\)​n−1\)\(E0−E2\)Id\)\\displaystyle\\quad\-\(2^\{\(1\-\\alpha\)n\}\-1\)\(E\_\{0\}\-E\_\{2\}\)I\_\{d\}\\bigg\)Plugging this back, we obtain\.

Fclf,c​\(α\)−Fclf,p​\(α\)\\displaystyle F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)≥C2\(1−α\)​nd−1\(\(E0−dE2\)\(2\(1−α\)​n⟨w⋆,wclf,c⟩\(⟨wclf,c,wh⟩−⟨wclf,c,wclf,l⟩\)\\displaystyle\\geq\\frac\{C\_\{2\}^\{\(1\-\\alpha\)n\}\}\{d\-1\}\\bigg\(\(E\_\{0\}\-dE\_\{2\}\)\(2^\{\(1\-\\alpha\)n\}\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},c\}\\rangle\(\\langle w\_\{\\mathrm\{clf\},c\},w\_\{h\}\\rangle\-\\langle w\_\{\\mathrm\{clf\},c\},w\_\{\\mathrm\{clf\},l\}\\rangle\)−⟨w⋆,wclf,l⟩\(⟨wh,wclf,l⟩−1\)\)−\(2\(1−α\)​n−1\)\(E0−E2\)\(⟨w⋆,wh⟩−⟨w⋆,wclf,l⟩\)\)\\displaystyle\\quad\-\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\(\\langle w\_\{h\},w\_\{\\mathrm\{clf\},l\}\\rangle\-1\)\)\-\(2^\{\(1\-\\alpha\)n\}\-1\)\(E\_\{0\}\-E\_\{2\}\)\(\\langle w^\{\\star\},w\_\{h\}\\rangle\-\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\\bigg\)We requirewclf,cw\_\{\\mathrm\{clf\},c\}such that the RHS is≥0\\geq 0\. This implies,

⟨w⋆,wclf,c⟩​\(⟨wclf,c,wh⟩−⟨wclf,c,wclf,l⟩\)\\displaystyle\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},c\}\\rangle\(\\langle w\_\{\\mathrm\{clf\},c\},w\_\{h\}\\rangle\-\\langle w\_\{\\mathrm\{clf\},c\},w\_\{\\mathrm\{clf\},l\}\\rangle\)≥2−\(1−α\)​n​\(⟨w⋆,wclf,l⟩​\(⟨wh,wclf,l⟩−1\)\)\\displaystyle\\geq 2^\{\-\(1\-\\alpha\)n\}\(\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\(\\langle w\_\{h\},w\_\{\\mathrm\{clf\},l\}\\rangle\-1\)\)\+E0−E2E0−d​E2​\(1−2−\(1−α\)​n\)​\(⟨w⋆,wh⟩−⟨w⋆,wclf,l⟩\)\\displaystyle\\quad\+\\frac\{E\_\{0\}\-E\_\{2\}\}\{E\_\{0\}\-dE\_\{2\}\}\(1\-2^\{\-\(1\-\\alpha\)n\}\)\(\\langle w^\{\\star\},w\_\{h\}\\rangle\-\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\(9\)We now compute the termsE0E\_\{0\}and an upper bound onE2E\_\{2\}which satisfies our requirements\.

###### Lemma 8\.

Forρ\\rhodistributed according to equation[4](https://arxiv.org/html/2605.30615#A3.E4),

E0\\displaystyle E\_\{0\}=Θ​\(d\(1−α\)​n/2​\(\(1−α\)​n\)−1\)\\displaystyle=\\Theta\\left\(d^\{\(1\-\\alpha\)n/2\}\(\(1\-\\alpha\)n\)^\{\-1\}\\right\)E2\\displaystyle E\_\{2\}≤𝒪​\(d−3/2​\(\(1−α\)​n\)−2\)\\displaystyle\\leq\\mathcal\{O\}\\left\(d^\{\-3/2\}\(\(1\-\\alpha\)n\)^\{\-2\}\\right\)

###### Proof\.

First, we computeE0E\_\{0\}using equation[4](https://arxiv.org/html/2605.30615#A3.E4)\.

E0\\displaystyle E\_\{0\}=C2​∫−11\(h​\(ρ\)\)\(1−α\)​n​\(1−ρ2\)d−3​𝑑ρ\\displaystyle=C\_\{2\}\\int\_\{\-1\}^\{1\}\(h\(\\rho\)\)^\{\(1\-\\alpha\)n\}\(\\sqrt\{1\-\\rho^\{2\}\}\)^\{d\-3\}d\\rhoAs the term inside the integral is even inρ\\rho, ash​\(ρ\)h\(\\rho\)is even inρ\\rho, we restrict the integral fromρ=0\\rho=0to11\. Further,h′​\(ρ\)=−\(1−ρ2\)d−3​d​ρh^\{\\prime\}\(\\rho\)=\-\(\\sqrt\{1\-\\rho^\{2\}\}\)^\{d\-3\}d\\rho\. Therefore, substitutingt=h​\(ρ\)t=h\(\\rho\), we obtain,

E0=−C1​∫h​\(0\)h​\(1\)t\(1−α\)​n​𝑑t=C1\(1−α\)​n\+1​\(h​\(0\)\)\(1−α\)​n\+1\\displaystyle E\_\{0\}=\-C\_\{1\}\\int\_\{h\(0\)\}^\{h\(1\)\}t^\{\(1\-\\alpha\)n\}dt=\\frac\{C\_\{1\}\}\{\(1\-\\alpha\)n\+1\}\(h\(0\)\)^\{\(1\-\\alpha\)n\+1\}Note thath​\(1\)=0h\(1\)=0\. Further,h​\(0\)=∫01\(1−a2\)d−3​𝑑a=1C1​Prx⁡\[\|ρ\|∈\[0,1\]\]=1C1h\(0\)=\\int\_\{0\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da=\\frac\{1\}\{C\_\{1\}\}\\Pr\_\{x\}\[\\lvert\\rho\\rvert\\in\[0,1\]\]=\\frac\{1\}\{C\_\{1\}\}from equation[4](https://arxiv.org/html/2605.30615#A3.E4)\. Therefore,

E0=C1\(1−α\)​n\+1​1C1\(1−α\)n\+1\)=1C1\(1−α\)​n\(1−α\)n\+1\)=Θ​\(d\(1−α\)​n/2\(1−α\)​n\)\\displaystyle E\_\{0\}=\\frac\{C\_\{1\}\}\{\(1\-\\alpha\)n\+1\}\\frac\{1\}\{C\_\{1\}^\{\(1\-\\alpha\)n\+1\)\}\}=\\frac\{1\}\{C\_\{1\}^\{\(1\-\\alpha\)n\}\(1\-\\alpha\)n\+1\)\}=\\Theta\\left\(\\frac\{d^\{\(1\-\\alpha\)n/2\}\}\{\(1\-\\alpha\)n\}\\right\)We use the fact thatC1=Θ​\(d−1/2\)C\_\{1\}=\\Theta\(d^\{\-1/2\}\)\.

To computeE2E\_\{2\}, we need to find an upper bound onh​\(ρ\)h\(\\rho\)\.

E2\\displaystyle E\_\{2\}=C2​∫−11\(h​\(ρ\)\)\(1−α\)​n​\(1−ρ2\)d−3​ρ2​𝑑ρ\\displaystyle=C\_\{2\}\\int\_\{\-1\}^\{1\}\(h\(\\rho\)\)^\{\(1\-\\alpha\)n\}\(\\sqrt\{1\-\\rho^\{2\}\}\)^\{d\-3\}\\rho^\{2\}d\\rhoWe first use the fact that bothh​\(ρ\)h\(\\rho\)andρ2\\rho^\{2\}are even to restrict the integral fromρ=0\\rho=0toρ=1\\rho=1\. By using integration by parts, assumingρ2\\rho^\{2\}is the first term and the rest forms the second term, we obtain,

E2\\displaystyle E\_\{2\}=C1​\[−\(h​\(ρ\)\)\(1−α\)​n\+1​ρ2\(1−α\)​n\+1\]01\+2​C1\(1−α\)​n\+1​∫01\(h​\(ρ\)\)\(1−α\)​n\+1​ρ​𝑑ρ\\displaystyle=C\_\{1\}\\left\[\-\\frac\{\(h\(\\rho\)\)^\{\(1\-\\alpha\)n\+1\}\\rho^\{2\}\}\{\(1\-\\alpha\)n\+1\}\\right\]\_\{0\}^\{1\}\+\\frac\{2C\_\{1\}\}\{\(1\-\\alpha\)n\+1\}\\int\_\{0\}^\{1\}\(h\(\\rho\)\)^\{\(1\-\\alpha\)n\+1\}\\rho d\\rhoHere, the first term is0ash​\(1\)=0h\(1\)=0and atρ=0\\rho=0ρ2=0\\rho^\{2\}=0\. Note that the second term cannot be integrated directly, so we obtain an upper bound on it\. To do this, we need an upper bound onh​\(ρ\)h\(\\rho\)forρ∈\[0,1\]\\rho\\in\[0,1\]\.

h​\(ρ\)=∫ρ1\(1−a2\)d−3​𝑑a≤\(1−ρ2\)d−3​∫ρ1𝑑a=\(1−ρ2\)d−3​\(1−ρ\)≤\(1−ρ2\)d−1\\displaystyle h\(\\rho\)=\\int\_\{\\rho\}^\{1\}\(\\sqrt\{1\-a^\{2\}\}\)^\{d\-3\}da\\leq\(\\sqrt\{1\-\\rho^\{2\}\}\)^\{d\-3\}\\int\_\{\\rho\}^\{1\}da=\(\\sqrt\{1\-\\rho^\{2\}\}\)^\{d\-3\}\(1\-\\rho\)\\leq\(\\sqrt\{1\-\\rho^\{2\}\}\)^\{d\-1\}Forρ∈\[0,1\]\\rho\\in\[0,1\], we useρ2≤ρ\\rho^\{2\}\\leq\\rho\. With this upper bound, we can bound the value ofE2E\_\{2\}\.

E2\\displaystyle E\_\{2\}=2​C1\(1−α\)​n\+1​∫01\(h​\(ρ\)\)\(1−α\)​n\+1​ρ​𝑑ρ≤2​C1\(1−α\)​n\+1​∫01\(1−ρ2\)\(d−1\)​\(\(1−α\)​n\+1\)​ρ​𝑑ρ\\displaystyle=\\frac\{2C\_\{1\}\}\{\(1\-\\alpha\)n\+1\}\\int\_\{0\}^\{1\}\(h\(\\rho\)\)^\{\(1\-\\alpha\)n\+1\}\\rho d\\rho\\leq\\frac\{2C\_\{1\}\}\{\(1\-\\alpha\)n\+1\}\\int\_\{0\}^\{1\}\(\\sqrt\{1\-\\rho^\{2\}\}\)^\{\(d\-1\)\(\(1\-\\alpha\)n\+1\)\}\\rho d\\rhoWe now substitutet=1−ρ2t=1\-\\rho^\{2\}withd​t=−2​ρ​d​ρdt=\-2\\rho d\\rho\.

E2\\displaystyle E\_\{2\}≤C1\(1−α\)​n\+1​∫01t\(d−1\)​\(\(1−α\)​n\+1\)/2​𝑑t=C1\(\(d−1\)\(\(1−α\)n\+1\)\+2\)\(1−α\)n\+1\)\\displaystyle\\leq\\frac\{C\_\{1\}\}\{\(1\-\\alpha\)n\+1\}\\int\_\{0\}^\{1\}t^\{\(d\-1\)\(\(1\-\\alpha\)n\+1\)/2\}dt=\\frac\{C\_\{1\}\}\{\(\(d\-1\)\(\(1\-\\alpha\)n\+1\)\+2\)\(1\-\\alpha\)n\+1\)\}Note thatC1=Θ​\(d−1/2\)C\_\{1\}=\\Theta\(d^\{\-1/2\}\)\. Therefore,

E2≤𝒪​\(1d3/2​\(\(1−α\)​n\)2\)\\displaystyle E\_\{2\}\\leq\\mathcal\{O\}\\left\(\\frac\{1\}\{d^\{3/2\}\(\(1\-\\alpha\)n\)^\{2\}\}\\right\)∎

Using these bounds onE0E\_\{0\}andE2E\_\{2\}, we can make obtain an interpretable upper bound for the second term in equation[C\.2\.2](https://arxiv.org/html/2605.30615#A3.Ex62)\.

E0−E2E0−d​E2=1\+\(d−1\)​E2E0−d​E2=1\+\(1−1/d\)​\(E0/d​E2−1\)−1\\displaystyle\\frac\{E\_\{0\}\-E\_\{2\}\}\{E\_\{0\}\-dE\_\{2\}\}=1\+\\frac\{\(d\-1\)E\_\{2\}\}\{E\_\{0\}\-dE\_\{2\}\}=1\+\(1\-1/d\)\(E\_\{0\}/dE\_\{2\}\-1\)^\{\-1\}If we plug in the value ofE0E\_\{0\}and the upper bound onE2E\_\{2\}, we obtain,

E0−E2E0−d​E2≤1\+Θ​\(d−\(\(1−α\)​n\+1\)/2​\(\(1−α\)​n\)−1\)\\displaystyle\\frac\{E\_\{0\}\-E\_\{2\}\}\{E\_\{0\}\-dE\_\{2\}\}\\leq 1\+\\Theta\\left\(d^\{\-\(\(1\-\\alpha\)n\+1\)/2\}\(\(1\-\\alpha\)n\)^\{\-1\}\\right\)Plugging this into equation[C\.2\.2](https://arxiv.org/html/2605.30615#A3.Ex62)proves Theorem[4](https://arxiv.org/html/2605.30615#Thmtheorem4)

#### C\.2\.3Proof of Theorem[5](https://arxiv.org/html/2605.30615#Thmtheorem5)

The proof for Gaussian features andLL\-smoothggfollows that of Theorem[4](https://arxiv.org/html/2605.30615#Thmtheorem4)in Appendix[C\.2\.2](https://arxiv.org/html/2605.30615#A3.SS2.SSS2)\. We setu=φ​\(x\),∀x∈𝒳u=\\varphi\(x\),\\forall x\\in\{\\mathcal\{X\}\}\. Then,u∼𝒩​\(0,Id\)u\\sim\{\\mathcal\{N\}\}\(0,I\_\{d\}\)\. Letu=ρ​wclf,c\+u⟂=λ​wclf,l\+v⟂u=\\rho w\_\{\\mathrm\{clf\},c\}\+u\_\{\\perp\}=\\lambda w\_\{\\mathrm\{clf\},l\}\+v\_\{\\perp\}where⟨wclf,c,u⟂⟩=⟨wclf,l,v⟂⟩=0\\langle w\_\{\\mathrm\{clf\},c\},u\_\{\\perp\}\\rangle=\\langle w\_\{\\mathrm\{clf\},l\},v\_\{\\perp\}\\rangle=0\. Asuuis gaussian,ρ,λ,u⟂\\rho,\\lambda,u\_\{\\perp\}andv⟂v\_\{\\perp\}are also gaussian\.

These random variables have the following distribution\.

ρ⟂u⟂andλ⟂v⟂,\\displaystyle\\rho\\perp u\_\{\\perp\}\\quad\\text\{and\}\\quad\\lambda\\perp v\_\{\\perp\},\(10\)ρ=⟨wclf,c,u⟩∼𝒩​\(0,1\),λ=⟨wclf,l,u⟩∼𝒩​\(0,1\),\\displaystyle\\rho=\\langle w\_\{\\mathrm\{clf\},c\},u\\rangle\\sim\{\\mathcal\{N\}\}\(0,1\),\\quad\\lambda=\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\sim\{\\mathcal\{N\}\}\(0,1\),𝔼​\[u⟂​u⟂⊤\]\+𝔼​\[ρ2\]​wclf,c​wclf,c⊤=𝔼​\[v⟂​v⟂⊤\]\+𝔼​\[λ2\]​wclf,l​wclf,l⊤=Id\\displaystyle\\mathbb\{E\}\[u\_\{\\perp\}u\_\{\\perp\}^\{\\top\}\]\+\\mathbb\{E\}\[\\rho^\{2\}\]w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}=\\mathbb\{E\}\[v\_\{\\perp\}v\_\{\\perp\}^\{\\top\}\]\+\\mathbb\{E\}\[\\lambda^\{2\}\]w\_\{\\mathrm\{clf\},l\}w\_\{\\mathrm\{clf\},l\}^\{\\top\}=I\_\{d\}u⟂=u−ρ​wclf,c∼𝒩​\(0,Id−wclf,c​wclf,c⊤\),\\displaystyle u\_\{\\perp\}=u\-\\rho w\_\{\\mathrm\{clf\},c\}\\sim\{\\mathcal\{N\}\}\(0,I\_\{d\}\-w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\),v⟂=u−λ​wclf,l∼𝒩​\(0,Id−wclf,l​wclf,l⊤\),\\displaystyle v\_\{\\perp\}=u\-\\lambda w\_\{\\mathrm\{clf\},l\}\\sim\{\\mathcal\{N\}\}\(0,I\_\{d\}\-w\_\{\\mathrm\{clf\},l\}w\_\{\\mathrm\{clf\},l\}^\{\\top\}\),
For aLL\-smooth differentiablegg, we need to use Taylor’s expansion for the difference term\(2​p⋆​\(x\)−1\)​\(ph​\(x\)−pclf,l​\(x\)\)\(2p^\{\\star\}\(x\)\-1\)\(p\_\{h\}\(x\)\-p\_\{\\mathrm\{clf\},l\}\(x\)\)\. We useg′​\(0\)=1g^\{\\prime\}\(0\)=1andg​\(0\)=12g\(0\)=\\frac\{1\}\{2\}to write down the Taylor’s expansion forp⋆​\(x\),ph​\(x\)p^\{\\star\}\(x\),p\_\{h\}\(x\)andpclf,l​\(x\)p\_\{\\mathrm\{clf\},l\}\(x\)around0\.

p⋆​\(x\)\\displaystyle p^\{\\star\}\(x\)=g\(0\)\+g′\(0\)⟨w⋆,u⟩\+r⋆\(⟨w⋆,u⟩\)\)=12\+⟨w⋆,u⟩\+r⋆\(⟨w⋆,u⟩\)\)\\displaystyle=g\(0\)\+g^\{\\prime\}\(0\)\\langle w^\{\\star\},u\\rangle\+r^\{\\star\}\(\\langle w^\{\\star\},u\\rangle\)\)=\\frac\{1\}\{2\}\+\\langle w^\{\\star\},u\\rangle\+r^\{\\star\}\(\\langle w^\{\\star\},u\\rangle\)\)\(11\)ph​\(x\)\\displaystyle p\_\{h\}\(x\)=g\(0\)\+g′\(0\)⟨wh,u⟩\+rh\(⟨wh,u⟩\)\)=12\+⟨wh,u⟩\+rh\(⟨wh,u⟩\)\)\\displaystyle=g\(0\)\+g^\{\\prime\}\(0\)\\langle w\_\{h\},u\\rangle\+r\_\{h\}\(\\langle w\_\{h\},u\\rangle\)\)=\\frac\{1\}\{2\}\+\\langle w\_\{h\},u\\rangle\+r\_\{h\}\(\\langle w\_\{h\},u\\rangle\)\)pclf,l​\(x\)\\displaystyle p\_\{\\mathrm\{clf\},l\}\(x\)=g​\(0\)\+g′​\(0\)​⟨wclf,l,u⟩\+rclf,l​\(⟨wclf,l,u⟩\)=12\+⟨wclf,l,u⟩\+rclf,l​\(⟨wclf,l,u⟩\)\\displaystyle=g\(0\)\+g^\{\\prime\}\(0\)\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\+r\_\{\\mathrm\{clf\},l\}\(\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\)=\\frac\{1\}\{2\}\+\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\+r\_\{\\mathrm\{clf\},l\}\(\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\)ByLL\-smoothness ofgg, we have,

∀a∈ℝ,\|r⋆​\(a\)\|,\|rh​\(a\)\|,\|rclf,l​\(a\)\|≤L2​\|a\|2\\displaystyle\\forall a\\in\\mathbb\{R\},\\quad\\lvert r^\{\\star\}\(a\)\\rvert,\\lvert r\_\{h\}\(a\)\\rvert,\\lvert r\_\{\\mathrm\{clf\},l\}\(a\)\\rvert\\leq\\frac\{L\}\{2\}\\lvert a\\rvert^\{2\}\(12\)Therefore,

\(2​p⋆​\(x\)−1\)​\(ph​\(x\)−pclf,l​\(x\)\)\\displaystyle\(2p^\{\\star\}\(x\)\-1\)\(p\_\{h\}\(x\)\-p\_\{\\mathrm\{clf\},l\}\(x\)\)\(13\)=\(⟨w⋆,u⟩\+r⋆​\(⟨w⋆,u⟩\)\)​\(⟨wh−wclf,l,u⟩\+rh​\(⟨wh,u⟩\)−rclf,l​\(⟨wclf,l,u⟩\)\)\\displaystyle=\(\\langle w^\{\\star\},u\\rangle\+r^\{\\star\}\(\\langle w^\{\\star\},u\\rangle\)\)\(\\langle w\_\{h\}\-w\_\{\\mathrm\{clf\},l\},u\\rangle\+r\_\{h\}\(\\langle w\_\{h\},u\\rangle\)\-r\_\{\\mathrm\{clf\},l\}\(\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\)\)=\(w⋆\)​u​u⊤​\(wh−wclf,l\)\+r​\(u\)\\displaystyle=\(w^\{\\star\}\)uu^\{\\top\}\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\+r\(u\)wherer​\(u\)\\displaystyle\\text\{where\}\\quad r\(u\)=⟨w⋆,u⟩​\(rh​\(⟨wh,u⟩\)−rclf,l​\(⟨wclf,l,u⟩\)\)\+⟨wh−wclf,l,u⟩​r⋆​\(⟨w⋆,u⟩\)\\displaystyle=\\langle w^\{\\star\},u\\rangle\(r\_\{h\}\(\\langle w\_\{h\},u\\rangle\)\-r\_\{\\mathrm\{clf\},l\}\(\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\)\)\+\\langle w\_\{h\}\-w\_\{\\mathrm\{clf\},l\},u\\rangle r^\{\\star\}\(\\langle w^\{\\star\},u\\rangle\)
We compute a bound onr​\(u\)r\(u\)\.

\|r​\(u\)\|≤\\displaystyle\\lvert r\(u\)\\rvert\\leq\|⟨w⋆,u⟩\|​\(\|rh​\(⟨wh,u⟩\)\|\+\|rclf,l​\(⟨wclf,l,u⟩\)\|\)\+\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\)​\|r⋆​\(⟨w⋆,u⟩\)\|\\displaystyle\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\(\\lvert r\_\{h\}\(\\langle w\_\{h\},u\\rangle\)\\rvert\+\\lvert r\_\{\\mathrm\{clf\},l\}\(\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\)\\rvert\)\+\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\)\\lvert r^\{\\star\}\(\\langle w^\{\\star\},u\\rangle\)\\rvert≤\\displaystyle\\leqL2​\|⟨w⋆,u⟩\|​\(\|⟨wh,u⟩\|2\+\|⟨wclf,l,u⟩\|2\)\+L2​\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\)​\|⟨w⋆,u⟩\|2\\displaystyle\\frac\{L\}\{2\}\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert^\{2\}\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert^\{2\}\)\+\\frac\{L\}\{2\}\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\)\\lvert\\langle w^\{\\star\},u\\rangle\\rvert^\{2\}≤\\displaystyle\\leqL2​\|⟨w⋆,u⟩\|​\(\(\|⟨wh,u⟩\|2\+\|⟨wclf,l,u⟩\|2\)\+\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\)​\|⟨w⋆,u⟩\|\)\\displaystyle\\frac\{L\}\{2\}\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\\left\(\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert^\{2\}\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert^\{2\}\)\+\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\)\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\\right\)≤\\displaystyle\\leqL2​\|⟨w⋆,u⟩\|​\(\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\)2\+\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\)​\|⟨w⋆,u⟩\|\)\\displaystyle\\frac\{L\}\{2\}\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\\left\(\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\)^\{2\}\+\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\)\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\\right\)=\\displaystyle=L2​\|⟨w⋆,u⟩\|​\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\)​\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\+\|⟨w⋆,u⟩\|\)\\displaystyle\\frac\{L\}\{2\}\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\)\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\+\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\)\(14\)We use triangle inequality in the first step andLL\-smoothness ofggin the second step\. In the fourth step, we usea2\+b2≤\(a\+b\)2a^\{2\}\+b^\{2\}\\leq\(a\+b\)^\{2\}fora,b≥0a,b\\geq 0\.

We compute lower bound onFclf−Fclf,p​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)and upper bound onFclf−Fclf,c​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)to compute a lower bound on the differenceFclf,c​\(α\)−Fclf,p​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\.

Upper bound on ConfSel\.The distributions ofpclf,l​\(x\)p\_\{\\mathrm\{clf\},l\}\(x\)is symmetric around12\\frac\{1\}\{2\}from Assumption[3](https://arxiv.org/html/2605.30615#Thmtheorem3)for gaussian features\. From Lemma[1](https://arxiv.org/html/2605.30615#Thmtheorem1), we obtain,

Prx′∼𝒟x⁡\[\|2​pclf,c​\(x′\)−1\|≥\|2​pclf,c​\(x\)−1\|\]\\displaystyle\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x^\{\\prime\}\)\-1\\rvert\\geq\\lvert 2p\_\{\\mathrm\{clf\},c\}\(x\)\-1\\rvert\]=Prx′⁡\[\|⟨wclf,c,φ​\(x′\)⟩\|≥\|⟨wclf,c,φ​\(x\)⟩\|\]=Prρ′∼𝒩​\(0,1\)⁡\[\|ρ′\|≥\|ρ\|\]=2​\(1−Φ​\(\|ρ\|\)\)\\displaystyle=\\Pr\_\{x^\{\\prime\}\}\[\\lvert\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x^\{\\prime\}\)\\rangle\\rvert\\geq\\lvert\\langle w\_\{\\mathrm\{clf\},c\},\\varphi\(x\)\\rangle\\rvert\]=\\Pr\_\{\\rho^\{\\prime\}\\sim\{\\mathcal\{N\}\}\(0,1\)\}\[\\lvert\\rho^\{\\prime\}\\rvert\\geq\\lvert\\rho\\rvert\]=2\(1\-\\Phi\(\\lvert\\rho\\rvert\)\)We substituteu′=φ​\(x′\)=ρ′​wclf,c\+u⟂′u^\{\\prime\}=\\varphi\(x^\{\\prime\}\)=\\rho^\{\\prime\}w\_\{\\mathrm\{clf\},c\}\+u\_\{\\perp\}^\{\\prime\}, where⟨wclf,c,u⟂′⟩=0\\langle w\_\{\\mathrm\{clf\},c\},u\_\{\\perp\}^\{\\prime\}\\rangle=0which matches the decomposition ofuu\. Additionally,Φ:ℝ→\[0,1\]\\Phi:\\mathbb\{R\}\\to\[0,1\]the cdf of unit normal distribution\.

We seth~​\(ρ\)≜1−Φ​\(\|ρ\|\),Wclf,c=wclf,c​wclf,c⊤\\widetilde\{h\}\(\\rho\)\\triangleq 1\-\\Phi\(\\lvert\\rho\\rvert\),W\_\{\\mathrm\{clf\},c\}=w\_\{\\mathrm\{clf\},c\}w\_\{\\mathrm\{clf\},c\}^\{\\top\}\. From Eq equation[13](https://arxiv.org/html/2605.30615#A3.E13), we boundFclf−Fclf,c​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\.

Fclf−Fclf,c​\(α\)=𝔼u​\[\(2​h~​\(ρ\)\)\(1−α\)​n​\(\(w⋆\)⊤​\(u​u⊤\)​\(wh−wclf,l\)\+r​\(u\)\)\]\\displaystyle F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)=\\mathbb\{E\}\_\{u\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\(\(w^\{\\star\}\)^\{\\top\}\(uu^\{\\top\}\)\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\+r\(u\)\)\]=\(w⋆\)⊤​\(𝔼ρ​\[\(2​h~​\(ρ\)\)\(1−α\)​n​ρ2\]​Wclf,c⏟7\+𝔼ρ​\[\(2​h~​\(ρ\)\)\(1−α\)​n\]​𝔼u⟂​\[u⟂​u⟂⊤\]⏟8\)​\(wh−wclf,l\)\\displaystyle=\(w^\{\\star\}\)^\{\\top\}\\left\(\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{7\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\rho\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\rho^\{2\}\]W\_\{\\mathrm\{clf\},c\}\}\}\+\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{8\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\rho\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\]\\mathbb\{E\}\_\{u\_\{\\perp\}\}\[u\_\{\\perp\}u\_\{\\perp\}^\{\\top\}\]\}\}\\right\)\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\+𝔼ρ​\[\(2​h~​\(ρ\)\)\(1−α\)​n​ρ\]​\(w⋆\)⊤​\(wclf,c​𝔼u⟂​\[u⟂⊤\]\+𝔼u⟂​\[u⟂\]​wclf,c⊤\)​\(wh−wclf,l\)⏟9\\displaystyle\\quad\+\\underset\{\\hbox to9\.81pt\{\\vbox to9\.81pt\{\\pgfpicture\\makeatletter\\hbox\{\\enskip\\lower\-4\.90532pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{4\.70532pt\}\{2\.59871pt\}\{2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\{4\.70532pt\}\\pgfsys@curveto\{\-2\.59871pt\}\{4\.70532pt\}\{\-4\.70532pt\}\{2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\\pgfsys@curveto\{\-4\.70532pt\}\{\-2\.59871pt\}\{\-2\.59871pt\}\{\-4\.70532pt\}\{0\.0pt\}\{\-4\.70532pt\}\\pgfsys@curveto\{2\.59871pt\}\{\-4\.70532pt\}\{4\.70532pt\}\{\-2\.59871pt\}\{4\.70532pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-1\.99306pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{9\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\rho\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\rho\]\(w^\{\\star\}\)^\{\\top\}\(w\_\{\\mathrm\{clf\},c\}\\mathbb\{E\}\_\{u\_\{\\perp\}\}\[u\_\{\\perp\}^\{\\top\}\]\+\\mathbb\{E\}\_\{u\_\{\\perp\}\}\[u\_\{\\perp\}\]w\_\{\\mathrm\{clf\},c\}^\{\\top\}\)\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\}\}\+𝔼u​\[\(2​h~​\(ρ\)\)\(1−α\)​n​r​\(u\)\]⏟10\\displaystyle\\quad\+\\underset\{\\hbox to12\.87pt\{\\vbox to12\.87pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-6\.4329pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{6\.23291pt\}\{3\.44238pt\}\{3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\{6\.23291pt\}\\pgfsys@curveto\{\-3\.44238pt\}\{6\.23291pt\}\{\-6\.23291pt\}\{3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{\-6\.23291pt\}\{\-3\.44238pt\}\{\-3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\{\-6\.23291pt\}\\pgfsys@curveto\{3\.44238pt\}\{\-6\.23291pt\}\{6\.23291pt\}\{\-3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-3\.98613pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{10\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{u\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}r\(u\)\]\}\}Note that9=0\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{9\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=0by symmetry ofu⟂u\_\{\\perp\}\. LetEa,b≜𝔼ρ∼𝒩​\(0,1\)​\[\(h~​\(ρ\)\)b​\|ρ\|a\]E\_\{a,b\}\\triangleq\\mathbb\{E\}\_\{\\rho\\sim\{\\mathcal\{N\}\}\(0,1\)\}\[\(\\widetilde\{h\}\(\\rho\)\)^\{b\}\\lvert\\rho\\rvert^\{a\}\]for alla∈ℕ∪\{0\}a\\in\\mathbb\{N\}\\cup\\\{0\\\}\.

7=2\(1−α\)​n​E2,\(1−α\)​n​Wclf,c,8=2\(1−α\)​n​E0,\(1−α\)​n​\(Id−Wclf,c\)\\displaystyle\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{7\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=2^\{\(1\-\\alpha\)n\}E\_\{2,\(1\-\\alpha\)n\}W\_\{\\mathrm\{clf\},c\},\\quad\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{8\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=2^\{\(1\-\\alpha\)n\}E\_\{0,\(1\-\\alpha\)n\}\(I\_\{d\}\-W\_\{\\mathrm\{clf\},c\}\)\(15\)
We compute10separately using equation[14](https://arxiv.org/html/2605.30615#A3.E14)\.

10=𝔼u​\[\(2​h~​\(ρ\)\)\(1−α\)​n​r​\(u\)\]≤𝔼u​\[\(2​h~​\(ρ\)\)\(1−α\)​n​\|r​\(u\)\|\]\\displaystyle\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{10\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=\\mathbb\{E\}\_\{u\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}r\(u\)\]\\leq\\mathbb\{E\}\_\{u\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\lvert r\(u\)\\rvert\]≤L2​𝔼u​\[\(2​h~​\(ρ\)\)\(1−α\)​n​\|⟨w⋆,u⟩\|2​\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\)2​\(\|⟨wh,u⟩\|\+\|⟨wclf,l,u⟩\|\+\|⟨w⋆,u⟩\|\)2\]\\displaystyle\\leq\\frac\{L\}\{2\}\\mathbb\{E\}\_\{u\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\sqrt\{\\lvert\\langle w^\{\\star\},u\\rangle\\rvert^\{2\}\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\)^\{2\}\(\\lvert\\langle w\_\{h\},u\\rangle\\rvert\+\\lvert\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle\\rvert\+\\lvert\\langle w^\{\\star\},u\\rangle\\rvert\)^\{2\}\}\]≤6​L2​𝔼u​\[\(2​h~​\(ρ\)\)\(1−α\)​n​⟨w⋆,u⟩2​\(⟨wh,u⟩2\+⟨wclf,l,u⟩2\)​\(⟨wh,u⟩2\+⟨wclf,l,u⟩2\+⟨w⋆,u⟩2\)\]\\displaystyle\\leq\\frac\{\\sqrt\{6\}L\}\{2\}\\mathbb\{E\}\_\{u\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\sqrt\{\\langle w^\{\\star\},u\\rangle^\{2\}\(\\langle w\_\{h\},u\\rangle^\{2\}\+\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle^\{2\}\)\(\\langle w\_\{h\},u\\rangle^\{2\}\+\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle^\{2\}\+\\langle w^\{\\star\},u\\rangle^\{2\}\)\}\]≤6​L2𝔼u\[\(2h~\(ρ\)\)\(1−α\)​n⟨w⋆,u⟩2​⟨wh,u⟩4\+⟨w⋆,u⟩4​⟨wh,u⟩2\+⟨w⋆,u⟩2​⟨wclf,l,u⟩4\\displaystyle\\leq\\frac\{\\sqrt\{6\}L\}\{2\}\\mathbb\{E\}\_\{u\}\[\(2\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\sqrt\{\\langle w^\{\\star\},u\\rangle^\{2\}\\langle w\_\{h\},u\\rangle^\{4\}\}\+\\sqrt\{\\langle w^\{\\star\},u\\rangle^\{4\}\\langle w\_\{h\},u\\rangle^\{2\}\}\+\\sqrt\{\\langle w^\{\\star\},u\\rangle^\{2\}\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle^\{4\}\}\+⟨w⋆,u⟩4​⟨wclf,l,u⟩2\+2​⟨wh,u⟩2​⟨w⋆,u⟩2​⟨wclf,l,u⟩2\]\\displaystyle\+\\sqrt\{\\langle w^\{\\star\},u\\rangle^\{4\}\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle^\{2\}\}\+\\sqrt\{2\\langle w\_\{h\},u\\rangle^\{2\}\\langle w^\{\\star\},u\\rangle^\{2\}\\langle w\_\{\\mathrm\{clf\},l\},u\\rangle^\{2\}\}\]For the second inequality, we useLL\-smoothness\. We use\(∑i=1lai\)2≤l​∑i=1lai2\(\\sum\_\{i=1\}^\{l\}a\_\{i\}\)^\{2\}\\leq l\\sum\_\{i=1\}^\{l\}a\_\{i\}^\{2\}to obtain the third inequality\. In the fifth inequality, we use∑i=1lai≤∑i=1lai,∀ai≥0,i∈\[l\]\\sum\_\{i=1\}^\{l\}\\sqrt\{a\_\{i\}\}\\leq\\sum\_\{i=1\}^\{l\}\\sqrt\{a\_\{i\}\},\\forall a\_\{i\}\\geq 0,i\\in\[l\]\.

Note that all the terms in this expression are of the form⟨w1,u⟩2​⟨w2,u⟩2\\langle w\_\{1\},u\\rangle^\{2\}\\langle w\_\{2\},u\\rangle^\{2\}or2​⟨w1,u⟩2​⟨w2,u⟩2​⟨w3,u⟩22\\langle w\_\{1\},u\\rangle^\{2\}\\langle w\_\{2\},u\\rangle^\{2\}\\langle w\_\{3\},u\\rangle^\{2\}, wherew1,w2w\_\{1\},w\_\{2\}andw3w\_\{3\}take values in\{w⋆,wclf,l,wh\}\\\{w^\{\\star\},w\_\{\\mathrm\{clf\},l\},w\_\{h\}\\\}\. We use these simplified expressions to compute10\.

###### Lemma 9\(Intermediate remainder Terms\)\.

Consider44unit norm vectors,w0,w1,w2,w3∈ℝdw\_\{0\},w\_\{1\},w\_\{2\},w\_\{3\}\\in\\mathbb\{R\}^\{d\}\. Ifu∼𝒩​\(0,Id\)u\\sim\{\\mathcal\{N\}\}\(0,I\_\{d\}\)andρ=⟨u,w0⟩,∀u\\rho=\\langle u,w\_\{0\}\\rangle,\\forall u, then for some constantsC3,C4\>0C\_\{3\},C\_\{4\}\>0, we have,

𝔼u\[\(h~\(ρ\)\)\(1−α\)​n⟨w1,u⟩4​⟨w2,u⟩2\\displaystyle\\mathbb\{E\}\_\{u\}\[\(\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\sqrt\{\\langle w\_\{1\},u\\rangle^\{4\}\\langle w\_\{2\},u\\rangle^\{2\}\}≤C3​∑k=03Ek,\(1−α\)​n​d\(3−k\)/2\\displaystyle\\leq C\_\{3\}\\sum\_\{k=0\}^\{3\}E\_\{k,\(1\-\\alpha\)n\}d^\{\(3\-k\)/2\}𝔼u​\[\(h~​\(ρ\)\)\(1−α\)​n​⟨w1,u⟩2​⟨w2,u⟩2​⟨w3,u⟩2\]\\displaystyle\\mathbb\{E\}\_\{u\}\[\(\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\sqrt\{\\langle w\_\{1\},u\\rangle^\{2\}\\langle w\_\{2\},u\\rangle^\{2\}\\langle w\_\{3\},u\\rangle^\{2\}\}\]≤C4​∑k=03Ek,\(1−α\)​n​d\(3−k\)/2\\displaystyle\\leq C\_\{4\}\\sum\_\{k=0\}^\{3\}E\_\{k,\(1\-\\alpha\)n\}d^\{\(3\-k\)/2\}where​Ea,b\\displaystyle\\text\{where \}E\_\{a,b\}=𝔼ρ∼𝒩​\(0,1\)​\[\(h~​\(ρ\)\)b​\|ρ\|a\],∀a,b∈ℕ∪\{0\}\\displaystyle=\\mathbb\{E\}\_\{\\rho\\sim\{\\mathcal\{N\}\}\(0,1\)\}\[\(\\widetilde\{h\}\(\\rho\)\)^\{b\}\\lvert\\rho\\rvert^\{a\}\],\\quad\\forall a,b\\in\\mathbb\{N\}\\cup\\\{0\\\}

###### Proof\.

Letu=ρ​w0\+u⟂u=\\rho w\_\{0\}\+u\_\{\\perp\}whereu⟂⟂w0u\_\{\\perp\}\\perp w\_\{0\}\. Then,ρ∼𝒩​\(0,1\)\\rho\\sim\{\\mathcal\{N\}\}\(0,1\)andu⟂∼𝒩​\(0,Id−w0​w0⊤\)u\_\{\\perp\}\\sim\{\\mathcal\{N\}\}\(0,I\_\{d\}\-w\_\{0\}w\_\{0\}^\{\\top\}\)from equation[10](https://arxiv.org/html/2605.30615#A3.E10)\. We compute the first expression\.

⟨w1,u⟩4​⟨w2,u⟩2=\(ρ​⟨w1,w0⟩\+⟨w1,u⟂⟩\)4​\(ρ​⟨w2,w0⟩\+⟨w2,u⟂⟩\)2\\displaystyle\\langle w\_\{1\},u\\rangle^\{4\}\\langle w\_\{2\},u\\rangle^\{2\}=\(\\rho\\langle w\_\{1\},w\_\{0\}\\rangle\+\\langle w\_\{1\},u\_\{\\perp\}\\rangle\)^\{4\}\(\\rho\\langle w\_\{2\},w\_\{0\}\\rangle\+\\langle w\_\{2\},u\_\{\\perp\}\\rangle\)^\{2\}=\(ρ4⟨w1,w0⟩4\+3ρ3⟨w1,w0⟩3⟨w1,u⟂⟩\+6ρ2⟨w1,w0⟩2⟨w1,u⟂⟩2\+3ρ⟨w1,w0⟩⟨w1,u⟂⟩3\+⟨w1,u⟂⟩4\)⋅\\displaystyle=\(\\rho^\{4\}\\langle w\_\{1\},w\_\{0\}\\rangle^\{4\}\+3\\rho^\{3\}\\langle w\_\{1\},w\_\{0\}\\rangle^\{3\}\\langle w\_\{1\},u\_\{\\perp\}\\rangle\+6\\rho^\{2\}\\langle w\_\{1\},w\_\{0\}\\rangle^\{2\}\\langle w\_\{1\},u\_\{\\perp\}\\rangle^\{2\}\+3\\rho\\langle w\_\{1\},w\_\{0\}\\rangle\\langle w\_\{1\},u\_\{\\perp\}\\rangle^\{3\}\+\\langle w\_\{1\},u\_\{\\perp\}\\rangle^\{4\}\)\\cdot×\(ρ2​⟨w2,w0⟩2\+2​ρ​⟨w2,w0⟩​⟨w2,u⟂⟩\+⟨w2,u⟂⟩2\)\\displaystyle\\times\(\\rho^\{2\}\\langle w\_\{2\},w\_\{0\}\\rangle^\{2\}\+2\\rho\\langle w\_\{2\},w\_\{0\}\\rangle\\langle w\_\{2\},u\_\{\\perp\}\\rangle\+\\langle w\_\{2\},u\_\{\\perp\}\\rangle^\{2\}\)We bound all inner product terms of⟨wi,wj⟩≤1\\langle w\_\{i\},w\_\{j\}\\rangle\\leq 1, fori,j∈\[4\]i,j\\in\[4\]\. Further, let⟨w1,u⟂⟩=ζ\\langle w\_\{1\},u\_\{\\perp\}\\rangle=\\zeta\. Then, for some constantC3′\>0C\_\{3\}^\{\\prime\}\>0, using∑i=1lai≤∑i=1l\|ai\|\\sqrt\{\\sum\_\{i=1\}^\{l\}a\_\{i\}\}\\leq\\sum\_\{i=1\}^\{l\}\\sqrt\{\\lvert a\_\{i\}\\rvert\}we have,

𝔼u​\[\(h~​\(ρ\)\)\(1−α\)​n​⟨w1,u⟩4​⟨w2,u⟩2\]\\displaystyle\\mathbb\{E\}\_\{u\}\[\(\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\sqrt\{\\langle w\_\{1\},u\\rangle^\{4\}\\langle w\_\{2\},u\\rangle^\{2\}\}\]≤C3′​∑k=03𝔼ρ​\[h~​\(ρ\)​\|ρ\|k\]​𝔼ζ​\[\|ζ\|3−k\]\\displaystyle\\leq C\_\{3\}^\{\\prime\}\\sum\_\{k=0\}^\{3\}\\mathbb\{E\}\_\{\\rho\}\[\\widetilde\{h\}\(\\rho\)\\lvert\\rho\\rvert^\{k\}\]\\mathbb\{E\}\_\{\\zeta\}\[\\lvert\\zeta\\rvert^\{3\-k\}\]Note thatζ∼𝒩​\(0,d−1\)\\zeta\\sim\{\\mathcal\{N\}\}\(0,d\-1\), therefore\|ζ\|\\lvert\\zeta\\rvertis a folded normal distribution\. As we need a bound on its first three moments for the above summation, we use its moment\-generating function from\[Tsagris\_2014\]\.

𝔼ζ​\[\|ζ\|k\]=Θk​\(dk\)\\displaystyle\\mathbb\{E\}\_\{\\zeta\}\[\\lvert\\zeta\\rvert^\{k\}\]=\\Theta\_\{k\}\(\\sqrt\{d\}^\{k\}\)Plugging this in, we get the first expression\.

For the second expression,

⟨w1,u⟩2​⟨w2,u⟩2​⟨w3,u⟩2=\(ρ​⟨w1,w0⟩\+⟨w1,u⟂⟩\)2​\(ρ​⟨w2,w0⟩\+⟨w2,u⟂⟩\)2​\(ρ​⟨w3,w0⟩\+⟨w3,u⟂⟩\)2\\displaystyle\\langle w\_\{1\},u\\rangle^\{2\}\\langle w\_\{2\},u\\rangle^\{2\}\\langle w\_\{3\},u\\rangle^\{2\}=\(\\rho\\langle w\_\{1\},w\_\{0\}\\rangle\+\\langle w\_\{1\},u\_\{\\perp\}\\rangle\)^\{2\}\(\\rho\\langle w\_\{2\},w\_\{0\}\\rangle\+\\langle w\_\{2\},u\_\{\\perp\}\\rangle\)^\{2\}\(\\rho\\langle w\_\{3\},w\_\{0\}\\rangle\+\\langle w\_\{3\},u\_\{\\perp\}\\rangle\)^\{2\}Again, bounding the inner product terms by11, we obtain, for some constantC4′\>0C\_\{4\}^\{\\prime\}\>0

𝔼u\[\(h~\(ρ\)\)\(1−α\)​n⟨w1,u⟩2​⟨w2,u⟩2​⟨w3,u⟩2≤C4′∑k=03𝔼ρ\[h~\(ρ\)\|ρ\|k\]𝔼ζ\[\|ζ\|3−k\]\\displaystyle\\mathbb\{E\}\_\{u\}\[\(\\widetilde\{h\}\(\\rho\)\)^\{\(1\-\\alpha\)n\}\\sqrt\{\\langle w\_\{1\},u\\rangle^\{2\}\\langle w\_\{2\},u\\rangle^\{2\}\\langle w\_\{3\},u\\rangle^\{2\}\}\\leq C\_\{4\}^\{\\prime\}\\sum\_\{k=0\}^\{3\}\\mathbb\{E\}\_\{\\rho\}\[\\widetilde\{h\}\(\\rho\)\\lvert\\rho\\rvert^\{k\}\]\\mathbb\{E\}\_\{\\zeta\}\[\\lvert\\zeta\\rvert^\{3\-k\}\]We again use the bound on the moment generating functions of\|ζ\|\\lvert\\zeta\\rvert\. ∎

Using the above lemma, for some constantC5\>0C\_\{5\}\>0, we can bound10as,

10≤2\(1−α\)​n​C5​L​∑k=03Ek,\(1−α\)​n​d\(3−k\)/2\\displaystyle\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{10\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\\leq 2^\{\(1\-\\alpha\)n\}C\_\{5\}L\\sum\_\{k=0\}^\{3\}E\_\{k,\(1\-\\alpha\)n\}d^\{\(3\-k\)/2\}\(16\)
##### Lower bound for PairSel\-Middle\.

By symmetry ofpclf,l​\(x\)p\_\{\\mathrm\{clf\},l\}\(x\)around12\\frac\{1\}\{2\}from Assumption[3](https://arxiv.org/html/2605.30615#Thmtheorem3)for gaussian features, and we apply Lemma[7](https://arxiv.org/html/2605.30615#Thmtheorem7)\.

Prx′∼𝒟x⁡\[pclf,l​\(x′\)≤pclf,l​\(x\)\]⋅Prx′∼𝒟x⁡\[pclf,l​\(x′\)≥pclf,l​\(x\)\]\\displaystyle\\sqrt\{\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\leq p\_\{\\mathrm\{clf\},l\}\(x\)\]\\cdot\\Pr\_\{x^\{\\prime\}\\sim\{\\mathcal\{D\}\}\_\{x\}\}\[p\_\{\\mathrm\{clf\},l\}\(x^\{\\prime\}\)\\geq p\_\{\\mathrm\{clf\},l\}\(x\)\]\}≥Prx′⁡\[⟨wclf,l,φ​\(x′\)⟩≥\|⟨wclf,l,φ​\(x′\)⟩\|\]\\displaystyle\\geq\\Pr\_\{x^\{\\prime\}\}\[\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x^\{\\prime\}\)\\rangle\\geq\\lvert\\langle w\_\{\\mathrm\{clf\},l\},\\varphi\(x^\{\\prime\}\)\\rangle\\rvert\]=Prλ∼𝒩​\(0,1\)⁡\[λ′≥\|λ\|\]=1−Φ​\(\|λ\|\)\\displaystyle=\\Pr\_\{\\lambda\\sim\{\\mathcal\{N\}\}\(0,1\)\}\[\\lambda^\{\\prime\}\\geq\\lvert\\lambda\\rvert\]=1\-\\Phi\(\\lvert\\lambda\\rvert\)We use a similar parametrization ofx′x^\{\\prime\}intoλ′​wclf,l\+v⟂′\\lambda^\{\\prime\}w\_\{\\mathrm\{clf\},l\}\+v\_\{\\perp\}^\{\\prime\}where⟨v⟂′,wclf,l⟩=0\\langle v\_\{\\perp\}^\{\\prime\},w\_\{\\mathrm\{clf\},l\}\\rangle=0\. We now find a lower bound forFclf−Fclf,p​\(α\)F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)using theλ,v⟂\\lambda,v\_\{\\perp\}parametrization and equation[13](https://arxiv.org/html/2605.30615#A3.E13)\. Further,Wclf,l=wclf,l​wclf,l⊤W\_\{\\mathrm\{clf\},l\}=w\_\{\\mathrm\{clf\},l\}w\_\{\\mathrm\{clf\},l\}^\{\\top\}

Fclf−Fclf,p​\(α\)=𝔼u​\[\(h~​\(λ\)\)\(1−α\)​n​\(\(w⋆\)⊤​\(u​u⊤\)​\(wh−wclf,l\)\+r​\(u\)\)\]\\displaystyle F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)=\\mathbb\{E\}\_\{u\}\[\(\\widetilde\{h\}\(\\lambda\)\)^\{\(1\-\\alpha\)n\}\(\(w^\{\\star\}\)^\{\\top\}\(uu^\{\\top\}\)\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\+r\(u\)\)\]=\(w⋆\)⊤​\(𝔼λ​\[\(h~​\(λ\)\)\(1−α\)​n​λ2\]​Wclf,l⏟11\+𝔼λ​\[\(h~​\(λ\)\)\(1−α\)​n\]​𝔼clf⟂​\[v⟂​v⟂⊤\]⏟12\)​\(wh−wclf,l\)\\displaystyle=\(w^\{\\star\}\)^\{\\top\}\\left\(\\underset\{\\hbox to12\.87pt\{\\vbox to12\.87pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-6\.4329pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{6\.23291pt\}\{3\.44238pt\}\{3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\{6\.23291pt\}\\pgfsys@curveto\{\-3\.44238pt\}\{6\.23291pt\}\{\-6\.23291pt\}\{3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{\-6\.23291pt\}\{\-3\.44238pt\}\{\-3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\{\-6\.23291pt\}\\pgfsys@curveto\{3\.44238pt\}\{\-6\.23291pt\}\{6\.23291pt\}\{\-3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-3\.98613pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{11\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\lambda\}\[\(\\widetilde\{h\}\(\\lambda\)\)^\{\(1\-\\alpha\)n\}\\lambda^\{2\}\]W\_\{\\mathrm\{clf\},l\}\}\}\+\\underset\{\\hbox to12\.87pt\{\\vbox to12\.87pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-6\.4329pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{6\.23291pt\}\{3\.44238pt\}\{3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\{6\.23291pt\}\\pgfsys@curveto\{\-3\.44238pt\}\{6\.23291pt\}\{\-6\.23291pt\}\{3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{\-6\.23291pt\}\{\-3\.44238pt\}\{\-3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\{\-6\.23291pt\}\\pgfsys@curveto\{3\.44238pt\}\{\-6\.23291pt\}\{6\.23291pt\}\{\-3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-3\.98613pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{12\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\lambda\}\[\(\\widetilde\{h\}\(\\lambda\)\)^\{\(1\-\\alpha\)n\}\]\\mathbb\{E\}\_\{\\mathrm\{clf\}\_\{\\perp\}\}\[v\_\{\\perp\}v\_\{\\perp\}^\{\\top\}\]\}\}\\right\)\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\+𝔼λ​\[\(h~​\(λ\)\)\(1−α\)​n​λ\]​\(w⋆\)⊤​\(wclf,c​𝔼clf⟂​\[v⟂⊤\]\+𝔼clf⟂​\[v⟂\]​wclf,c⊤\)​\(wh−wclf,l\)⏟13\\displaystyle\\quad\+\\underset\{\\hbox to12\.87pt\{\\vbox to12\.87pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-6\.4329pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{6\.23291pt\}\{3\.44238pt\}\{3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\{6\.23291pt\}\\pgfsys@curveto\{\-3\.44238pt\}\{6\.23291pt\}\{\-6\.23291pt\}\{3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{\-6\.23291pt\}\{\-3\.44238pt\}\{\-3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\{\-6\.23291pt\}\\pgfsys@curveto\{3\.44238pt\}\{\-6\.23291pt\}\{6\.23291pt\}\{\-3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-3\.98613pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{13\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{\\lambda\}\[\(\\widetilde\{h\}\(\\lambda\)\)^\{\(1\-\\alpha\)n\}\\lambda\]\(w^\{\\star\}\)^\{\\top\}\(w\_\{\\mathrm\{clf\},c\}\\mathbb\{E\}\_\{\\mathrm\{clf\}\_\{\\perp\}\}\[v\_\{\\perp\}^\{\\top\}\]\+\\mathbb\{E\}\_\{\\mathrm\{clf\}\_\{\\perp\}\}\[v\_\{\\perp\}\]w\_\{\\mathrm\{clf\},c\}^\{\\top\}\)\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\}\}\+𝔼u​\[\(h~​\(λ\)\)\(1−α\)​n​r​\(u\)\]⏟14\\displaystyle\\quad\+\\underset\{\\hbox to12\.87pt\{\\vbox to12\.87pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-6\.4329pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{6\.23291pt\}\{3\.44238pt\}\{3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\{6\.23291pt\}\\pgfsys@curveto\{\-3\.44238pt\}\{6\.23291pt\}\{\-6\.23291pt\}\{3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\\pgfsys@curveto\{\-6\.23291pt\}\{\-3\.44238pt\}\{\-3\.44238pt\}\{\-6\.23291pt\}\{0\.0pt\}\{\-6\.23291pt\}\\pgfsys@curveto\{3\.44238pt\}\{\-6\.23291pt\}\{6\.23291pt\}\{\-3\.44238pt\}\{6\.23291pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-3\.98613pt\}\{\-2\.25555pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{14\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\{\\underbrace\{\\mathbb\{E\}\_\{u\}\[\(\\widetilde\{h\}\(\\lambda\)\)^\{\(1\-\\alpha\)n\}r\(u\)\]\}\}Note that13=0\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{13\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=0by symmetry ofv⟂v\_\{\\perp\}\. From equation[10](https://arxiv.org/html/2605.30615#A3.E10),λ\\lambdaandρ\\rhohave the same distribution\. Then,

11=E2,\(1−α\)​n​Wclf,l,12=E0,\(1−α\)​n​\(Id−Wclf,l\)\\displaystyle\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{11\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=E\_\{2,\(1\-\\alpha\)n\}W\_\{\\mathrm\{clf\},l\},\\quad\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{12\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=E\_\{0,\(1\-\\alpha\)n\}\(I\_\{d\}\-W\_\{\\mathrm\{clf\},l\}\)\(17\)To compute14, we use the same technique as10\.

14=𝔼u​\[\(h~​\(λ\)\)\(1−α\)​n​r​\(u\)\]≥−𝔼u​\[\(h~​\(λ\)\)\(1−α\)​n​\|r​\(u\)\|\]\\displaystyle\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{14\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=\\mathbb\{E\}\_\{u\}\[\(\\widetilde\{h\}\(\\lambda\)\)^\{\(1\-\\alpha\)n\}r\(u\)\]\\geq\-\\mathbb\{E\}\_\{u\}\[\(\\widetilde\{h\}\(\\lambda\)\)^\{\(1\-\\alpha\)n\}\\lvert r\(u\)\\rvert\]Since the bounds in Lemma[9](https://arxiv.org/html/2605.30615#Thmtheorem9)are independent ofw0,w1,w2,w3w\_\{0\},w\_\{1\},w\_\{2\},w\_\{3\}, we can use them here as well\. This gives us the bound

14≥−C5​L​∑k=03Ek,\(1−α\)​n​d\(3−k\)/2\\displaystyle\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{14\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\\geq\-C\_\{5\}L\\sum\_\{k=0\}^\{3\}E\_\{k,\(1\-\\alpha\)n\}d^\{\(3\-k\)/2\}\(18\)

##### Difference of errors

We can now compute the termFclf,c​\(α\)−Fclf,p​\(α\)F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\.

Fclf,c​\(α\)−Fclf,p​\(α\)=\(Fclf−Fclf,p​\(α\)\)−\(Fclf−Fclf,c​\(α\)\)\\displaystyle F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)=\(F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\)\-\(F\_\{\\mathrm\{clf\}\}\-F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\)=\(w⋆\)⊤​\(11\+12−7−8\)​\(wh−wclf,l\)\+14−10\\displaystyle=\(w^\{\\star\}\)^\{\\top\}\(\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{11\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\+\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{12\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{7\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{8\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\)\(w\_\{h\}\-w\_\{\\mathrm\{clf\},l\}\)\+\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{14\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{10\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}We first compute the difference11\+12−7−8\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{11\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\+\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{12\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{7\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{8\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\.

11\+12−7−8\\displaystyle\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{11\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\+\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{12\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{7\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to11\.89pt\{\\vbox to11\.89pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-5\.94658pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{5\.74658pt\}\{3\.17378pt\}\{3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\{5\.74658pt\}\\pgfsys@curveto\{\-3\.17378pt\}\{5\.74658pt\}\{\-5\.74658pt\}\{3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\\pgfsys@curveto\{\-5\.74658pt\}\{\-3\.17378pt\}\{\-3\.17378pt\}\{\-5\.74658pt\}\{0\.0pt\}\{\-5\.74658pt\}\\pgfsys@curveto\{3\.17378pt\}\{\-5\.74658pt\}\{5\.74658pt\}\{\-3\.17378pt\}\{5\.74658pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-2\.5pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{8\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}=E2,\(1−α\)​n​Wclf,l\+E0,\(1−α\)​n​\(Id−Wclf,l\)\\displaystyle=E\_\{2,\(1\-\\alpha\)n\}W\_\{\\mathrm\{clf\},l\}\+E\_\{0,\(1\-\\alpha\)n\}\(I\_\{d\}\-W\_\{\\mathrm\{clf\},l\}\)−2\(1−α\)​n​\(E2,\(1−α\)​n​Wclf,c\+E0,\(1−α\)​n​\(Id−Wclf,c\)\)\\displaystyle\\quad\-2^\{\(1\-\\alpha\)n\}\(E\_\{2,\(1\-\\alpha\)n\}W\_\{\\mathrm\{clf\},c\}\+E\_\{0,\(1\-\\alpha\)n\}\(I\_\{d\}\-W\_\{\\mathrm\{clf\},c\}\)\)=\(E0,\(1−α\)​n−E2,\(1−α\)​n\)​\(2\(1−α\)​n​Wclf,c−Wclf,l\)\\displaystyle=\(E\_\{0,\(1\-\\alpha\)n\}\-E\_\{2,\(1\-\\alpha\)n\}\)\(2^\{\(1\-\\alpha\)n\}W\_\{\\mathrm\{clf\},c\}\-W\_\{\\mathrm\{clf\},l\}\)\+E0,\(1−α\)​n​Id​\(1−2\(1−α\)​n\)\\displaystyle\\quad\+E\_\{0,\(1\-\\alpha\)n\}I\_\{d\}\(1\-2^\{\(1\-\\alpha\)n\}\)
We now compute the difference14−10\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{14\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{10\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}using equation[16](https://arxiv.org/html/2605.30615#A3.E16)and equation[18](https://arxiv.org/html/2605.30615#A3.E18)\.

14−10≥−\(1\+2\(1−α\)​n\)​C5​L​∑k=03Ek,\(1−α\)​n​d\(3−k\)/2\\displaystyle\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{14\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\-\\hbox to15\.59pt\{\\vbox to15\.59pt\{\\pgfpicture\\makeatletter\\hbox\{\\quad\\lower\-7\.79424pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{ \{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\{\}\}\}\{\{\}\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\{\}\\pgfsys@moveto\{7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{7\.59424pt\}\{4\.19423pt\}\{4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\{7\.59424pt\}\\pgfsys@curveto\{\-4\.19423pt\}\{7\.59424pt\}\{\-7\.59424pt\}\{4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\\pgfsys@curveto\{\-7\.59424pt\}\{\-4\.19423pt\}\{\-4\.19423pt\}\{\-7\.59424pt\}\{0\.0pt\}\{\-7\.59424pt\}\\pgfsys@curveto\{4\.19423pt\}\{\-7\.59424pt\}\{7\.59424pt\}\{\-4\.19423pt\}\{7\.59424pt\}\{0\.0pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{\-5\.00002pt\}\{\-3\.22221pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{10\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\{\{\}\}\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\\geq\-\(1\+2^\{\(1\-\\alpha\)n\}\)C\_\{5\}L\\sum\_\{k=0\}^\{3\}E\_\{k,\(1\-\\alpha\)n\}d^\{\(3\-k\)/2\}
Plugging this back, we obtain\.

Fclf,c​\(α\)−Fclf,p​\(α\)≥\\displaystyle F\_\{\\mathrm\{clf\},c\}\(\\alpha\)\-F\_\{\\mathrm\{clf\},p\}\(\\alpha\)\\geq\(E0,\(1−α\)​n−E2,\(1−α\)​n\)\(2\(1−α\)​n⟨w⋆,wclf,c⟩\\displaystyle\(E\_\{0,\(1\-\\alpha\)n\}\-E\_\{2,\(1\-\\alpha\)n\}\)\(2^\{\(1\-\\alpha\)n\}\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},c\}\\rangle⋅\(⟨wclf,c,wh⟩−⟨wclf,c,wclf,l⟩\)−⟨w⋆,wclf,l⟩\(⟨wh,wclf,l⟩−1\)\)\\displaystyle\\quad\\cdot\(\\langle w\_\{\\mathrm\{clf\},c\},w\_\{h\}\\rangle\-\\langle w\_\{\\mathrm\{clf\},c\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\-\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\(\\langle w\_\{h\},w\_\{\\mathrm\{clf\},l\}\\rangle\-1\)\)−\(2\(1−α\)​n−1\)​E0,\(1−α\)​n​\(⟨w⋆,wh⟩−⟨w⋆,wclf,l⟩\)\\displaystyle\\quad\-\(2^\{\(1\-\\alpha\)n\}\-1\)E\_\{0,\(1\-\\alpha\)n\}\(\\langle w^\{\\star\},w\_\{h\}\\rangle\-\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\)−\(1\+2\(1−α\)​n\)​C5​L​∑k=03Ek,\(1−α\)​n​d\(3−k\)/2\\displaystyle\\quad\-\(1\+2^\{\(1\-\\alpha\)n\}\)C\_\{5\}L\\sum\_\{k=0\}^\{3\}E\_\{k,\(1\-\\alpha\)n\}d^\{\(3\-k\)/2\}
We requirewclf,cw\_\{\\mathrm\{clf\},c\}such that the RHS is≥0\\geq 0\. This implies,

⟨w⋆,wclf,c⟩​\(⟨wclf,c,wh⟩−⟨wclf,c,wclf,l⟩\)\\displaystyle\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},c\}\\rangle\(\\langle w\_\{\\mathrm\{clf\},c\},w\_\{h\}\\rangle\-\\langle w\_\{\\mathrm\{clf\},c\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\(19\)≥2−\(1−α\)​n​\(⟨w⋆,wclf,l⟩​\(⟨wh,wclf,l⟩−1\)\)\\displaystyle\\geq\\,2^\{\-\(1\-\\alpha\)n\}\(\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\(\\langle w\_\{h\},w\_\{\\mathrm\{clf\},l\}\\rangle\-1\)\)\+\(1−2−\(1−α\)​n\)​E0,\(1−α\)​nE0,\(1−α\)​n−E2,\(1−α\)​n​\(⟨w⋆,wh⟩−⟨w⋆,wclf,l⟩\)\\displaystyle\\quad\+\\frac\{\(1\-2^\{\-\(1\-\\alpha\)n\}\)E\_\{0,\(1\-\\alpha\)n\}\}\{E\_\{0,\(1\-\\alpha\)n\}\-E\_\{2,\(1\-\\alpha\)n\}\}\(\\langle w^\{\\star\},w\_\{h\}\\rangle\-\\langle w^\{\\star\},w\_\{\\mathrm\{clf\},l\}\\rangle\)\+\(1\+2−\(1−α\)​n\)​C5​LE0,\(1−α\)​n−E2,\(1−α\)​n​C5​L​∑k=03Ek,\(1−α\)​n​d\(3−k\)/2\\displaystyle\\quad\+\\frac\{\(1\+2^\{\-\(1\-\\alpha\)n\}\)C\_\{5\}L\}\{E\_\{0,\(1\-\\alpha\)n\}\-E\_\{2,\(1\-\\alpha\)n\}\}C\_\{5\}L\\sum\_\{k=0\}^\{3\}E\_\{k,\(1\-\\alpha\)n\}d^\{\(3\-k\)/2\}\(20\)
We compute the termE0,bE\_\{0,b\}and find upper bounds onEa,bE\_\{a,b\}fora,b∈ℕa,b\\in\\mathbb\{N\}withaabeing even to obtain a tight and interpretable upper bound on the RHS\.

###### Lemma 10\.

IfEa,b=𝔼ρ∼𝒩​\(0,1\)​\[\(1−Φ​\(\|ρ\|\)\)b​\|ρ\|a\]E\_\{a,b\}=\\mathbb\{E\}\_\{\\rho\\sim\{\\mathcal\{N\}\}\(0,1\)\}\[\(1\-\\Phi\(\\lvert\\rho\\rvert\)\)^\{b\}\\lvert\\rho\\rvert^\{a\}\], then

∀b∈ℕ,E0,b=2−b​\(b\+1\)−1\\displaystyle\\forall b\\in\\mathbb\{N\},\\quad E\_\{0,b\}=2^\{\-b\}\(b\+1\)^\{\-1\}∀a,b∈ℕ,b≫a,E0,b−Ea,b≥ω​\(1\)​E0,b,\\displaystyle\\forall a,b\\in\\mathbb\{N\},b\\gg a,\\quad E\_\{0,b\}\-E\_\{a,b\}\\geq\\omega\(1\)E\_\{0,b\},

###### Proof\.

We first computeE0,bE\_\{0,b\}\. We useϕ​\(ρ\)\\phi\(\\rho\)to denote the probability density function of a unit normal distribution atρ\\rho\. Note thatΦ​\(ρ\)=∫−∞ρϕ​\(a\)​𝑑a\\Phi\(\\rho\)=\\int\_\{\-\\infty\}^\{\\rho\}\\phi\(a\)da\.

E0,b\\displaystyle E\_\{0,b\}=∫−∞∞\(1−Φ​\(\|ρ\|\)\)b​ϕ​\(ρ\)​𝑑ρ\\displaystyle=\\int\_\{\-\\infty\}^\{\\infty\}\(1\-\\Phi\(\\lvert\\rho\\rvert\)\)^\{b\}\\phi\(\\rho\)d\\rhoAs the term inside the integral is even inρ\\rho, we restrict the integral fromρ=0\\rho=0to∞\\infty\. If we sett=1−Φ​\(ρ\)t=1\-\\Phi\(\\rho\), thend​t=−Φ′​\(ρ\)​d​ρ=−ϕ​\(ρ\)​d​ρdt=\-\\Phi^\{\\prime\}\(\\rho\)d\\rho=\-\\phi\(\\rho\)d\\rho\.

E0,b=−2​∫1−Φ​\(0\)0tb​𝑑t=2​∫012tb​𝑑t=22b\+1​\(b\+1\)=2−b​\(b\+1\)−1\\displaystyle E\_\{0,b\}=\-2\\int\_\{1\-\\Phi\(0\)\}^\{0\}t^\{b\}dt=2\\int\_\{0\}^\{\\frac\{1\}\{2\}\}t^\{b\}dt=\\frac\{2\}\{2^\{b\+1\}\(b\+1\)\}=2^\{\-b\}\(b\+1\)^\{\-1\}By symmetry of the normal distribution,Φ​\(0\)=12\\Phi\(0\)=\\frac\{1\}\{2\}\.

We now computeEa,bE\_\{a,b\}fora∈ℕa\\in\\mathbb\{N\}\.

Ea,b=∫−∞∞\(1−Φ​\(\|ρ\|\)\)b​\|ρ\|a​ϕ​\(ρ\)​𝑑ρ=2​∫0∞\(1−Φ​\(ρ\)\)b​ρa​ϕ​\(ρ\)​𝑑ρ\\displaystyle E\_\{a,b\}=\\int\_\{\-\\infty\}^\{\\infty\}\(1\-\\Phi\(\\lvert\\rho\\rvert\)\)^\{b\}\\lvert\\rho\\rvert^\{a\}\\phi\(\\rho\)d\\rho=2\\int\_\{0\}^\{\\infty\}\(1\-\\Phi\(\\rho\)\)^\{b\}\\rho^\{a\}\\phi\(\\rho\)d\\rhoAs1−Φ​\(\|ρ\|\)1\-\\Phi\(\\lvert\\rho\\rvert\)andρ2​a\\rho^\{2a\}are both even, we can restrict the integral fromρ=0\\rho=0toρ=∞\\rho=\\infty\. To bound this integral in terms ofE0,bE\_\{0,b\}, we split the integration into two parts –\[0,δ\]\[0,\\delta\]and\[δ,∞\)\[\\delta,\\infty\)for someδ∈\[0,∞\)\\delta\\in\[0,\\infty\)\. Ideally, thisδ\\deltais very close to0\. LetG1≜2​∫0δ\(1−Φ​\(ρ\)\)b​ρa​ϕ​\(ρ\)​𝑑ρG\_\{1\}\\triangleq 2\\int\_\{0\}^\{\\delta\}\(1\-\\Phi\(\\rho\)\)^\{b\}\\rho^\{a\}\\phi\(\\rho\)d\\rhoandG2≜=2∫δ∞\(1−Φ\(ρ\)\)bρaϕ\(ρ\)dρG\_\{2\}\\triangleq=2\\int\_\{\\delta\}^\{\\infty\}\(1\-\\Phi\(\\rho\)\)^\{b\}\\rho^\{a\}\\phi\(\\rho\)d\\rho\. We need an upper bound on both these terms\.

##### Bound onG1G\_\{1\}

G1\\displaystyle G\_\{1\}=2​∫0δ\(1−Φ​\(ρ\)\)b​ρa​ϕ​\(ρ\)​𝑑ρ≤2​\(1−Φ​\(0\)\)b​ϕ​\(ρ\)​∫0δρa​𝑑ρ=2−b\+1/2​δa\+1​\(a\+1\)−1\\displaystyle=2\\int\_\{0\}^\{\\delta\}\(1\-\\Phi\(\\rho\)\)^\{b\}\\rho^\{a\}\\phi\(\\rho\)d\\rho\\leq 2\(1\-\\Phi\(0\)\)^\{b\}\\phi\(\\rho\)\\int\_\{0\}^\{\\delta\}\\rho^\{a\}d\\rho=2^\{\-b\+1/2\}\\delta^\{a\+1\}\(a\+1\)^\{\-1\}\(21\)We use the fact that both1−Φ​\(ρ\)1\-\\Phi\(\\rho\)andϕ​\(ρ\)\\phi\(\\rho\)are maximized atρ=0\\rho=0\.

##### Bound onG2G\_\{2\}

G2\\displaystyle G\_\{2\}=2​∫δ∞\(1−Φ​\(ρ\)\)b​ρa​ϕ​\(ρ\)​𝑑ρ≤2\(2​π\)a​∫δ∞\(1−Φ​\(ρ\)\)b−a​e−a​ρ2/2​ϕ​\(ρ\)​𝑑ρ\\displaystyle=2\\int\_\{\\delta\}^\{\\infty\}\(1\-\\Phi\(\\rho\)\)^\{b\}\\rho^\{a\}\\phi\(\\rho\)d\\rho\\leq\\frac\{2\}\{\(\\sqrt\{2\\pi\}\)^\{a\}\}\\int\_\{\\delta\}^\{\\infty\}\(1\-\\Phi\(\\rho\)\)^\{b\-a\}e^\{\-a\\rho^\{2\}/2\}\\phi\(\\rho\)d\\rho≤2\(2​π\)a​∫δ∞\(1−Φ​\(ρ\)\)b−a​ϕ​\(ρ\)​𝑑ρ=−2\(2​π\)a​∫1−Φ​\(δ\)0tb−a​𝑑t=2​\(2​π\)−a​\(1−Φ​\(δ\)\)b−a\\displaystyle\\leq\\frac\{2\}\{\(\\sqrt\{2\\pi\}\)^\{a\}\}\\int\_\{\\delta\}^\{\\infty\}\(1\-\\Phi\(\\rho\)\)^\{b\-a\}\\phi\(\\rho\)d\\rho=\-\\frac\{2\}\{\(\\sqrt\{2\\pi\}\)^\{a\}\}\\int\_\{1\-\\Phi\(\\delta\)\}^\{0\}t^\{b\-a\}dt=2\(\\sqrt\{2\\pi\}\)^\{\-a\}\(1\-\\Phi\(\\delta\)\)^\{b\-a\}\(22\)For\(1−Φ​\(ρ\)\)a\(1\-\\Phi\(\\rho\)\)^\{a\}, we use the gaussian tail inequality,1−Φ​\(ρ\)=erfc​\(ρ/2\)2≤e−ρ2/22​π​ρ,∀ρ\>01\-\\Phi\(\\rho\)=\\frac\{\\mathrm\{erfc\}\(\\rho/\\sqrt\{2\}\)\}\{2\}\\leq\\frac\{e^\{\-\\rho^\{2\}/2\}\}\{\\sqrt\{2\\pi\}\\rho\},\\forall\\rho\>0\. Hereerfc\\mathrm\{erfc\}refers to the complementary error function\. This is possible asb≫ab\\gg a\. Then, we bounde−a​ρ2/2≤1e^\{\-a\\rho^\{2\}/2\}\\leq 1and sett=1−Φ​\(ρ\)t=1\-\\Phi\(\\rho\)to integrate out the remaining components\.

From equation[21](https://arxiv.org/html/2605.30615#A3.E21)and equation[22](https://arxiv.org/html/2605.30615#A3.E22), we can find an appropriateδ\\deltasuch that for some constantα∈\(0,1\)\\alpha\\in\(0,1\)and every value ofa,ba,b

G1≤2−b\+1/2​δa\+1​\(a\+1\)−1≤\(1−α\)​E0,b/2,G2≤2​\(2​π\)−a​\(1−Φ​\(δ\)\)b−a≤\(1−α\)​E0,b/2\\displaystyle G\_\{1\}\\leq 2^\{\-b\+1/2\}\\delta^\{a\+1\}\(a\+1\)^\{\-1\}\\leq\(1\-\\alpha\)E\_\{0,b\}/2,\\;G\_\{2\}\\leq 2\(\\sqrt\{2\\pi\}\)^\{\-a\}\(1\-\\Phi\(\\delta\)\)^\{b\-a\}\\leq\(1\-\\alpha\)E\_\{0,b\}/2Plugging in the value ofE0,bE\_\{0,b\}, we need

δ≤\(\(1−α\)​\(a\+1\)2​2​\(b\+1\)\)1/\(a\+1\),δ≥Φ−1​\(1−12​\(1−α4​\(π2\)a\)1/\(b−a\)\)\\displaystyle\\delta\\leq\\left\(\\frac\{\(1\-\\alpha\)\(a\+1\)\}\{2\\sqrt\{2\}\(b\+1\)\}\\right\)^\{1/\(a\+1\)\},\\;\\delta\\geq\\Phi^\{\-1\}\\left\(1\-\\frac\{1\}\{2\}\\left\(\\frac\{1\-\\alpha\}\{4\}\\left\(\\sqrt\{\\frac\{\\pi\}\{2\}\}\\right\)^\{a\}\\right\)^\{1/\(b\-a\)\}\\right\)Note that we can always findα\\alphasuch that both these inequalities are simultaneously satisfied\. Further, equating the two bounds onδ\\deltagives us a unique value ofδ\\deltafor fixedα\\alpha\.

∎

From this Lemma, we can simplify the terms ofE0,\(1−α\)​nE0,\(α\)​n−E2,\(α\)​n≤1\+o​\(1\)\\frac\{E\_\{0,\(1\-\\alpha\)n\}\}\{E\_\{0,\(\\alpha\)n\}\-E\_\{2,\(\\alpha\)n\}\}\\leq 1\+o\(1\)and the coefficients of terms ofLLare bounded by𝒪​\(1\)\\mathcal\{O\}\(1\)\. Plugging this into equation[19](https://arxiv.org/html/2605.30615#A3.E19)proves Theorem[5](https://arxiv.org/html/2605.30615#Thmtheorem5)withg′​\(0\)=1g^\{\\prime\}\(0\)=1\. To work withg′​\(0\)≠1g^\{\\prime\}\(0\)\\neq 1, we replaceggbyg~​\(a\)=g​\(a\)g′​\(0\),∀a∈ℝ\\tilde\{g\}\(a\)=\\frac\{g\(a\)\}\{g^\{\\prime\}\(0\)\},\\forall a\\in\\mathbb\{R\}in the above proof\. The smoothness constant forg~\\tilde\{g\}is nowLg′​\(0\)\\frac\{L\}\{g^\{\\prime\}\(0\)\}, which gives us the correct statement of Theorem[5](https://arxiv.org/html/2605.30615#Thmtheorem5)\.