具有信息性弃权的风险最优主动学习

arXiv cs.LG 论文

摘要

本文针对标注预言机可能拒绝作答,且其响应概率与分类器共享参数的情形,结合经干扰参数调整的费舍尔信息与多类别零一风险的局部曲率,推导出一种风险最优的主动学习采集准则。仿真和消费者投诉数据回放实验表明,在较大的查询预算下,响应感知采集方法优于按响应概率折减的基线方法。

arXiv:2611.24986v1 Announce Type: cross Abstract: We study label acquisition when an oracle may abstain and its response probability shares parameters with the classifier. A query can then provide information through both a returned label and the response outcome. Under a correctly specified parametric model, we derive the classifier information after adjustment for response-model nuisance parameters. Combining this information with the local curvature of multiclass zero-one risk gives an acquisition criterion for minimizing the leading asymptotic excess risk. We characterize the optimal allocation under a fixed query budget and show how query allocation affects response identifiability. The oracle criterion reduces to response-discounted acquisition when response probability does not depend on the classifier parameters. Simulations examine the resulting query allocations and isolate the contribution of the response-outcome term through an ablation. A Consumer Complaints replay retains observed human abstentions and reveals benchmark labels upon response. Response-aware acquisition has lower mean test error than response-discounted acquisition at the two largest budgets, with paired differences significant after Holm correction. Entropy sampling performs better at the smallest budget. The gains depend on the response mechanism and query budget.
查看原文
查看缓存全文

缓存时间: 2026/11/20 09:17

# Risk-optimal active learning with informative abstention
Source: [https://arxiv.org/html/2611.24986](https://arxiv.org/html/2611.24986)
Fariborz SetoudehtazangiEmail:[fariborz\.setoudehtazangi@studenti\.unipd\.it](mailto:[email protected])Affiliation:Dipartimento di Scienze Statistiche, Università di Padova, Padova, ItalyGeoffrey J\. McLachlanEmail:[g\.mclachlan@uq\.edu\.au](mailto:[email protected])Affiliation:School of Mathematics and Physics, The University of Queensland, Brisbane, Australia

###### Abstract

We study label acquisition when an oracle may abstain and its response probability shares parameters with the classifier\. A query can then provide information through both a returned label and the response outcome\. Under a correctly specified parametric model, we derive the classifier information after adjustment for response\-model nuisance parameters\. Combining this information with the local curvature of multiclass zero–one risk gives an acquisition criterion for minimizing leading asymptotic excess risk\. We characterize the optimal allocation under a fixed query budget and show how query allocation affects response identifiability\. The oracle criterion reduces to response\-discounted acquisition when response probability does not depend on the classifier parameters\. Simulations examine the resulting query allocations and isolate the contribution of the response\-outcome term through an ablation\. A Consumer Complaints replay retains observed human abstentions and reveals benchmark labels upon response\. Response\-aware acquisition has lower mean test error than response\-discounted acquisition at the two largest budgets, with paired differences significant after Holm correction\. Entropy sampling performs better at the smallest budget\. The gains depend on the response mechanism and query budget\.

###### keywords

Active learning, Informative oracle response, Label acquisition, Abstention, Fisher information, Classification risk

## Acknowledgements

We are grateful to Cuong V\. Nguyen of the Department of Mathematical Sciences at Durham University, UK, for generously taking the time to read the manuscript and for his thoughtful comments\. His observations on adaptive sampling helped us clarify the assumptions and proof of the two\-stage acquisition result\.

## 1Introduction

Obtaining a class label may require a human annotator, expert assessment, diagnostic procedure, or user interaction\. A query does not always produce a label: an annotator may decline an ambiguous case, an expert may find the available information insufficient, or a user may not respond\. This raises two questions for active learning: which observations should be queried, and how should the possibility and the observed pattern of response affect query selection?

Classical active learning allocates a limited labeling budget by selectively requesting labels\([Lewis and Gale, 1994](https://arxiv.org/html/2611.24986#bib.bib12);[Lewis and Catlett, 1994](https://arxiv.org/html/2611.24986#bib.bib11);[Settles, 2009](https://arxiv.org/html/2611.24986#bib.bib20)\)\. Uncertainty sampling prioritizes observations with ambiguous class probabilities, whereas information\-based methods assess the value of a label for parameter estimation or prediction\([Hoi et al\., 2006](https://arxiv.org/html/2611.24986#bib.bib8);[Sourati et al\., 2017](https://arxiv.org/html/2611.24986#bib.bib21);[Chen and Biros, 2023](https://arxiv.org/html/2611.24986#bib.bib5)\)\. Other approaches target prediction error, predictive information, or downstream decisions\([Roy and McCallum, 2001](https://arxiv.org/html/2611.24986#bib.bib16);[Bickford Smith et al\., 2023](https://arxiv.org/html/2611.24986#bib.bib4);[Filstroff et al\., 2024](https://arxiv.org/html/2611.24986#bib.bib7);[Huang et al\., 2026](https://arxiv.org/html/2611.24986#bib.bib9)\)\.

Statistical approaches to sampling provide related foundations\.[Wang et al\., \(2018\)](https://arxiv.org/html/2611.24986#bib.bib23)and[Yao and Wang, \(2019\)](https://arxiv.org/html/2611.24986#bib.bib26)derive optimal subsampling procedures for logistic and softmax regression, respectively\.[Imberg et al\., \(2020\)](https://arxiv.org/html/2611.24986#bib.bib10)study sampling schemes for prediction\-error reduction in unbiased active learning, while[Bach, \(2006\)](https://arxiv.org/html/2611.24986#bib.bib3)develops active learning for potentially misspecified generalized linear models\. These methods address different inferential and predictive objectives\. The problem considered here additionally involves a stochastic oracle response whose distribution shares parameters with the classifier\.

Imperfect labeling oracles have received substantial attention\.[Yan et al\., \(2016\)](https://arxiv.org/html/2611.24986#bib.bib25)study active learning when a labeler may return an incorrect label or abstain, obtaining query\-complexity guarantees under assumptions on the noise and abstention processes\.[Fang and Zhu, \(2014\)](https://arxiv.org/html/2611.24986#bib.bib6)consider uncertain labeling knowledge, allowing an oracle to indicate that it does not know a label\.[Amin et al\., \(2021\)](https://arxiv.org/html/2611.24986#bib.bib2)study a different form of abstention: examples routed to a human labeler no longer require a prediction from the machine\. Their algorithm jointly learns a model and decides when to request labels\. In a Bayesian pool\-based setting,[Nguyen et al\., \(2022\)](https://arxiv.org/html/2611.24986#bib.bib13)develop acquisition rules that learn both the classifier and an unknown abstention mechanism\. Thus, neither allowing an oracle to abstain nor learning its response mechanism alongside the classifier is itself new\.

A closely related approach is the biased\-nonresponse framework of[Robinson et al\., \(2024\)](https://arxiv.org/html/2611.24986#bib.bib15)\. Their expected\-utility rule weights an ordinary acquisition utility by the estimated probability of response; an upper\-confidence\-bound extension accounts for uncertainty in that estimate\. This addresses the reduced value of queries that are unlikely to return labels\. It does not, however, derive classifier information from the realized response outcome through a shared\-parameter likelihood\. Likewise,[Nguyen et al\., \(2022\)](https://arxiv.org/html/2611.24986#bib.bib13)use separate classification and abstention models, rather than deriving the nuisance\-adjusted Fisher information of a classifier\-linked response mechanism and weighting it by local zero–one classification risk\.

Likelihood\-based work on informative label missingness motivates this distinction\. Under Rubin’s framework, conditional independence of missingness and the unobserved class given the features does not by itself justify ignoring a mechanism that shares parameters with the classification model\([Rubin, 1976](https://arxiv.org/html/2611.24986#bib.bib17)\)\.[Ahfock and McLachlan, \(2020\)](https://arxiv.org/html/2611.24986#bib.bib1)show that uncertainty\-dependent label missingness can carry information about a discriminant rule\. More recent work develops information decompositions and connects the resulting information to classification performance\([Wang et al\., 2026](https://arxiv.org/html/2611.24986#bib.bib24);[Setoudehtazangi and McLachlan, 2026a](https://arxiv.org/html/2611.24986#bib.bib18)\)\. These results concern the information available under a given missingness mechanism\. Here the investigator also controls which observations are submitted to the oracle\.

Our starting point is the classification\-risk acquisition framework of[Setoudehtazangi and McLachlan, 2026b \(\)](https://arxiv.org/html/2611.24986#bib.bib19), in which every requested label is returned\. Under its regularity conditions, leading asymptotic excess zero–one risk depends on Fisher information weighted by the local curvatureHRH\_\{R\}of the active Bayes decision boundary\. A label is therefore valuable according to whether its information improves estimation in directions that affect classification, rather than according to uncertainty or parameter precision alone\. We consider how this criterion changes when the oracle may abstain\.

LetYYdenote the observed features,ZZthe class label,AAthe investigator’s query indicator, andRRthe oracle response indicator\. Conditional on a query, we model

Pr⁡\(R=1∣A=1,Y=y\)=r⁡\(y,θ,η\),\\Pr\(R=1\\mid A=1,Y=y\)=r\(y;\\theta,\\eta\),whereθ\\thetagoverns the classification model andη\\etacontains response\-specific nuisance parameters\. We assume that a returned label is correct and that

R⟂Z\|\(Y,A=1\)\.R\\perp Z\\mid\(Y,A=1\)\.The acquisition rule is known and parameter\-free\. Conditional on the observed features, its realized query decision therefore carries no likelihood information about the model parameters\. The subsequent response outcome can carry such information because its probability depends onθ\\theta\.

For the joint parameterϕ=\(θ⊤,η⊤\)⊤\\phi=\(\\theta^\{\\top\},\\eta^\{\\top\}\)^\{\\top\}, a query atyycontributes

K⁡\(y\)=r⁡\(y\)​J~​\(y\)\+r⁡\(y\)​\{1−r⁡\(y\)\}​q​\(y\)​q​\(y\)⊤,K\(y\)=r\(y\)\\widetilde\{J\}\(y\)\+r\(y\)\\\{1\-r\(y\)\\\}q\(y\)q\(y\)^\{\\top\},whereJ~​\(y\)\\widetilde\{J\}\(y\)is the conditional class\-label information augmented by zero nuisance blocks, and

q⁡\(y\)=∇ϕ​logit​r​\(y,θ,η\)\.q\(y\)=\\nabla\_\{\\phi\}\\operatorname\{logit\}r\(y;\\theta,\\eta\)\.The first term is information from a potentially returned label\. The second is information from observing whether the oracle responds\. When the response\-model nuisance information is nonsingular, eliminatingη\\etaleaves a positive semidefinite response contribution to efficient classifier information\. This contribution vanishes precisely when the classifier\-related response score is explained by the nuisance response score under the design\-weighted measure\. Sharing parameters is therefore insufficient by itself: useful response information must survive nuisance adjustment and affect parameter directions relevant to classification\.

Combining the query information with local risk curvature gives the marginal acquisition value

ψa​\(y\)=ψaL​\(y\)\+ψaR​\(y\),\\psi\_\{a\}\(y\)=\\psi\_\{a\}^\{L\}\(y\)\+\\psi\_\{a\}^\{R\}\(y\),whereaadenotes the acquisition design\. The returned\-label termψaL​\(y\)\\psi\_\{a\}^\{L\}\(y\)accounts for the probability of receiving an informative label\. The response termψaR​\(y\)\\psi\_\{a\}^\{R\}\(y\)accounts for the classification value of the response outcome itself\. A query can consequently be informative even when no class label is returned\. When response probability does not depend on the classifier parameters, the oracle criterion reduces to response\-discounted acquisition\.

The joint information matrix is affine in the acquisition design\. The resulting optimization therefore retains the convex trace\-inverse structure of a linear optimal\-design criterion\([Pukelsheim, 2006](https://arxiv.org/html/2611.24986#bib.bib14)\)and admits a threshold characterization under a fixed query budget\. Query allocation must also preserve sufficient variation for nonsingular response\-model information\.

We derive this rank condition and give a two\-stage oracle\-value result for a pilot\-selected design applied to fresh observations\. The risk conclusion requires uniform criterion consistency, uniform asymptotic linearity in mean square, and a higher\-moment bound\. The proof conditions on the pilot and then averages over it\. These conditions are not established for the sequential finite\-pool implementation used in the experiments\.

The simulations examine oracle and adaptive acquisition under noninformative and informative response mechanisms\. An ablation retains the full information matrix and joint estimation while omitting the explicit response\-outcome term from query scores\. It separates the effect of that term from changes in the information matrix used by response\-discounted acquisition\. The observed gains are small and depend on the response mechanism and query budget\.

We also use the Consumer Complaints annotations collected by[Totakura et al\., \(2026\)](https://arxiv.org/html/2611.24986#bib.bib22)\. Each worker is treated as a separate oracle\. The replay preserves observed human response and abstention patterns but reveals the benchmark class whenever a worker responds\. It therefore evaluates acquisition under human abstention without modeling errors in returned labels\. This comparison evaluates the complete acquisition rules; it does not isolate the explicit response\-outcome term\.

The contribution is a classification\-risk acquisition criterion for a shared\-parameter query–response model\. It values response outcomes through their nuisance\-adjusted information and their effect on the classification boundary\. The analysis identifies conditions under which this additional information changes query selection, while the experiments examine when those changes improve classification\.

Section[2](https://arxiv.org/html/2611.24986#S2)introduces the query–response model and derives its Fisher information\. Section[3](https://arxiv.org/html/2611.24986#S3)characterizes the efficient classifier information after nuisance adjustment\. Section[4](https://arxiv.org/html/2611.24986#S4)derives the classification\-risk criterion and its marginal query value\. Subsequent sections examine optimal allocation, response identifiability, adaptive acquisition, and a binary logistic illustration\. Simulations and the Consumer Complaints replay follow, before the discussion of findings and limitations\.

## 2Statistical framework and informative oracle response

LetY∈𝒴⊆ℝpY\\in\\mathcal\{Y\}\\subseteq\\mathbb\{R\}^\{p\}denote an observed feature vector andZ∈\{1,…,g\}Z\\in\\\{1,\\ldots,g\\\}its class label\. We consider a regular parametric classification model

pθ​\(y,z\)=pθ​\(y\)​τz​\(y,θ\),τk​\(y,θ\)=Prθ⁡\(Z=k∣Y=y\),p\_\{\\theta\}\(y,z\)=p\_\{\\theta\}\(y\)\\tau\_\{z\}\(y;\\theta\),\\qquad\\tau\_\{k\}\(y;\\theta\)=\\Pr\_\{\\theta\}\(Z=k\\mid Y=y\),whereθ∈Θ⊆ℝd\\theta\\in\\Theta\\subseteq\\mathbb\{R\}^\{d\}is the parameter governing the classification model\.

The feature vector is observed for every sampling unit\. After observingY=yY=y, the investigator may request its class label\. Let

A=\{1,if a label is requested,0,otherwise,A=\\begin\{cases\}1,&\\text\{if a label is requested\},\\\\ 0,&\\text\{otherwise\},\\end\{cases\}and suppose

Pr⁡\(A=1∣Y=y,Z\)=a⁡\(y\),0≤a⁡\(y\)≤1\.\\Pr\(A=1\\mid Y=y,Z\)=a\(y\),\\qquad 0\\leq a\(y\)\\leq 1\.\(2\.1\)The measurable functiona:𝒴→\[0,1\]a:\\mathcal\{Y\}\\rightarrow\[0,1\]is the*acquisition design*\. It is chosen by the investigator and is treated as known and parameter\-free\.

A label request need not result in a returned label\. Conditional onA=1A=1, let

R=\{1,if the oracle returns a label,0,if the oracle abstains\.R=\\begin\{cases\}1,&\\text\{if the oracle returns a label\},\\\\ 0,&\\text\{if the oracle abstains\}\.\\end\{cases\}We model the response probability as

Pr⁡\(R=1∣A=1,Y=y\)=r⁡\(y,θ,η\),0<r⁡\(y,θ,η\)<1,\\Pr\(R=1\\mid A=1,Y=y\)=r\(y;\\theta,\\eta\),\\qquad 0<r\(y;\\theta,\\eta\)<1,\(2\.2\)whereη∈ℋ⊆ℝq\\eta\\in\\mathcal\{H\}\\subseteq\\mathbb\{R\}^\{q\}contains parameters specific to the response mechanism\. The defining feature of \([2\.2](https://arxiv.org/html/2611.24986#S2.E2)\) is that the oracle response may depend on the same parameterθ\\thetathat determines the classifier\. The realized response or abstention can therefore itself contain information about the classification model\.

Throughout the main development we assume

R⟂Z\|\(Y,A=1\),R\\perp Z\\mid\(Y,A=1\),\(2\.3\)and that a returned label is correct\. Thus the response mechanism is feature\-dependent but does not depend further on the unobserved class onceYYis given\. Noisy returned labels and class\-dependent response mechanisms are outside the scope of the present paper\.

For one sampling unit, the observed data can be represented by

O=\(Y,A,A​R,A​R​Z\)\.O=\(Y,A,AR,ARZ\)\.Up to factors involving the known acquisition designaa, its likelihood is

L⁡\(θ,η,O\)∝pθ​\(Y\)​\[r​\(Y,θ,η\)R​\{1−r⁡\(Y,θ,η\)\}1−R​τZ​\(Y,θ\)R\]A\.L\(\\theta,\\eta;O\)\\propto p\_\{\\theta\}\(Y\)\\left\[r\(Y;\\theta,\\eta\)^\{R\}\\\{1\-r\(Y;\\theta,\\eta\)\\\}^\{1\-R\}\\tau\_\{Z\}\(Y;\\theta\)^\{R\}\\right\]^\{A\}\.\(2\.4\)Hence

ℓ⁡\(θ,η,O\)\\displaystyle\\ell\(\\theta,\\eta;O\)=log⁡pθ​\(Y\)\\displaystyle=\\log p\_\{\\theta\}\(Y\)\+A⁡\[R​log⁡r⁡\(Y,θ,η\)\+\(1−R\)​log​\{1−r⁡\(Y,θ,η\)\}\+R​log​τZ​\(Y,θ\)\]\.\\displaystyle\\quad\+A\\left\[R\\log r\(Y;\\theta,\\eta\)\+\(1\-R\)\\log\\\{1\-r\(Y;\\theta,\\eta\)\\\}\+R\\log\\tau\_\{Z\}\(Y;\\theta\)\\right\]\.\(2\.5\)
Define the feature and conditional\-label scores by

SY​\(Y,θ\)=∇θ​log​pθ​\(Y\),SZ\|Y​\(Y,Z,θ\)=∇θ​log​τZ​\(Y,θ\),S\_\{Y\}\(Y;\\theta\)=\\nabla\_\{\\theta\}\\log p\_\{\\theta\}\(Y\),\\qquad S\_\{Z\\mid Y\}\(Y,Z;\\theta\)=\\nabla\_\{\\theta\}\\log\\tau\_\{Z\}\(Y;\\theta\),and let

qθ​\(y\)=∇θ​logit​r​\(y,θ,η\),qη​\(y\)=∇η​logit​r​\(y,θ,η\)\.q\_\{\\theta\}\(y\)=\\nabla\_\{\\theta\}\\operatorname\{logit\}r\(y;\\theta,\\eta\),\\qquad q\_\{\\eta\}\(y\)=\\nabla\_\{\\eta\}\\operatorname\{logit\}r\(y;\\theta,\\eta\)\.Differentiation of \([2\.5](https://arxiv.org/html/2611.24986#S2.E5)\) gives

Sθ\\displaystyle S\_\{\\theta\}=SY\+A⁡\[\{R−r⁡\(Y\)\}​qθ​\(Y\)\+R​SZ\|Y\],\\displaystyle=S\_\{Y\}\+A\\left\[\\\{R\-r\(Y\)\\\}q\_\{\\theta\}\(Y\)\+RS\_\{Z\\mid Y\}\\right\],\(2\.6\)Sη\\displaystyle S\_\{\\eta\}=A⁡\{R−r⁡\(Y\)\}​qη​\(Y\)\.\\displaystyle=A\\\{R\-r\(Y\)\\\}q\_\{\\eta\}\(Y\)\.\(2\.7\)Thus a query can generate information through two distinct channels: the class label when it is returned and the response indicator itself\.

Let

J⁡\(y\)=Eθ​\[SZ\|Y​SZ\|Y⊤∣Y=y\]J\(y\)=E\_\{\\theta\}\\left\[S\_\{Z\\mid Y\}S\_\{Z\\mid Y\}^\{\\top\}\\mid Y=y\\right\]denote the conditional Fisher information in a returned class label, and set

w⁡\(y\)=r⁡\(y\)​\{1−r⁡\(y\)\},q⁡\(y\)=\(qθ​\(y\)qη​\(y\)\),J~​\(y\)=\(J⁡\(y\)000\)\.w\(y\)=r\(y\)\\\{1\-r\(y\)\\\},\\qquad q\(y\)=\\begin\{pmatrix\}q\_\{\\theta\}\(y\)\\\\ q\_\{\\eta\}\(y\)\\end\{pmatrix\},\\qquad\\widetilde\{J\}\(y\)=\\begin\{pmatrix\}J\(y\)&0\\\\ 0&0\\end\{pmatrix\}\.The information generated by a query at feature valueyyis

K⁡\(y\)=r⁡\(y\)​J~​\(y\)\+w⁡\(y\)​q​\(y\)​q​\(y\)⊤\.K\(y\)=r\(y\)\\widetilde\{J\}\(y\)\+w\(y\)q\(y\)q\(y\)^\{\\top\}\.\(2\.8\)The first term is the expected information from the returned label, whereas the second is the information carried by the response or abstention outcome\.

Let

ϕ=\(θ⊤,η⊤\)⊤,ℐY=\(IY​\(θ\)000\),\\phi=\(\\theta^\{\\top\},\\eta^\{\\top\}\)^\{\\top\},\\qquad\\mathcal\{I\}\_\{Y\}=\\begin\{pmatrix\}I\_\{Y\}\(\\theta\)&0\\\\ 0&0\\end\{pmatrix\},where

IY​\(θ\)=Eθ​\{SY​\(Y,θ\)​SY​\(Y,θ\)⊤\}\.I\_\{Y\}\(\\theta\)=E\_\{\\theta\}\\\{S\_\{Y\}\(Y;\\theta\)S\_\{Y\}\(Y;\\theta\)^\{\\top\}\\\}\.
###### Proposition 2\.1\.

Suppose that \([2\.1](https://arxiv.org/html/2611.24986#S2.E1)\)–\([2\.3](https://arxiv.org/html/2611.24986#S2.E3)\) hold and that the usual regularity conditions for Fisher\-information calculations are satisfied\. For a fixed acquisition designaa, the per\-observation Fisher information forϕ\\phiis

ℐ⁡\(a\)=ℐY\+Eθ​\[a⁡\(Y\)​K​\(Y\)\],\\mathcal\{I\}\(a\)=\\mathcal\{I\}\_\{Y\}\+E\_\{\\theta\}\\\!\\left\[a\(Y\)K\(Y\)\\right\],\(2\.9\)whereK⁡\(y\)K\(y\)is defined in \([2\.8](https://arxiv.org/html/2611.24986#S2.E8)\)\. Equivalently,

ℐ⁡\(a\)=ℐY\+Eθ​\[a⁡\(Y\)​\{r⁡\(Y\)​J~​\(Y\)\+r⁡\(Y\)​\{1−r⁡\(Y\)\}​q​\(Y\)​q​\(Y\)⊤\}\]\.\\mathcal\{I\}\(a\)=\\mathcal\{I\}\_\{Y\}\+E\_\{\\theta\}\\\!\\left\[a\(Y\)\\left\\\{r\(Y\)\\widetilde\{J\}\(Y\)\+r\(Y\)\\\{1\-r\(Y\)\\\}q\(Y\)q\(Y\)^\{\\top\}\\right\\\}\\right\]\.\(2\.10\)

The proof is given in Supplementary Section[S1\.1](https://arxiv.org/html/2611.24986#S1.SS1)\.

Proposition[2\.1](https://arxiv.org/html/2611.24986#S2.Thmtheorem1)separates the two sources of information created by a label request\. If a requested label is returned with certainty,r⁡\(y\)≡1r\(y\)\\equiv 1, then the response term vanishes and \([2\.10](https://arxiv.org/html/2611.24986#S2.E10)\) reduces to the usual information for selective label acquisition\. More generally, if the response mechanism is parameter\-dependent, the statistical value of a query is not determined only by the probability of obtaining its label: the observed response or abstention is itself part of the annotation experiment\.

Becauseη\\etais unknown, however, the second term in \([2\.8](https://arxiv.org/html/2611.24986#S2.E8)\) cannot be interpreted directly as additional information about the classifier\. The component that remains informative forθ\\thetaafter adjustment for the response\-model nuisance parameters is derived next\.

## 3Efficient classifier information and the value of oracle response

The information decomposition in Proposition[2\.1](https://arxiv.org/html/2611.24986#S2.Thmtheorem1)treatsϕ=\(θ⊤,η⊤\)⊤\\phi=\(\\theta^\{\\top\},\\eta^\{\\top\}\)^\{\\top\}as the full parameter vector\. For classification, however,θ\\thetais the parameter of interest, whereasη\\etais a nuisance parameter governing the oracle\-response mechanism\. Consequently, the response\-information term in \([2\.10](https://arxiv.org/html/2611.24986#S2.E10)\) must be adjusted for uncertainty inη\\eta\.

Partition the Fisher information according toϕ=\(θ⊤,η⊤\)⊤\\phi=\(\\theta^\{\\top\},\\eta^\{\\top\}\)^\{\\top\}:

ℐ⁡\(a\)=\(Iθ​θ​\(a\)Iθ​η​\(a\)Iη​θ​\(a\)Iη​η​\(a\)\)\.\\mathcal\{I\}\(a\)=\\begin\{pmatrix\}I\_\{\\theta\\theta\}\(a\)&I\_\{\\theta\\eta\}\(a\)\\\\ I\_\{\\eta\\theta\}\(a\)&I\_\{\\eta\\eta\}\(a\)\\end\{pmatrix\}\.From \([2\.10](https://arxiv.org/html/2611.24986#S2.E10)\),

Iθ​θ​\(a\)\\displaystyle I\_\{\\theta\\theta\}\(a\)=IY​\(θ\)\+Eθ​\[a⁡\(Y\)​r​\(Y\)​J​\(Y\)\]\+Cθ​θ​\(a\),\\displaystyle=I\_\{Y\}\(\\theta\)\+E\_\{\\theta\}\\\!\\left\[a\(Y\)r\(Y\)J\(Y\)\\right\]\+C\_\{\\theta\\theta\}\(a\),\(3\.1\)Iθ​η​\(a\)\\displaystyle I\_\{\\theta\\eta\}\(a\)=Cθ​η​\(a\),\\displaystyle=C\_\{\\theta\\eta\}\(a\),\(3\.2\)Iη​η​\(a\)\\displaystyle I\_\{\\eta\\eta\}\(a\)=Cη​η​\(a\),\\displaystyle=C\_\{\\eta\\eta\}\(a\),\(3\.3\)where

Cθ​θ​\(a\)\\displaystyle C\_\{\\theta\\theta\}\(a\)=Eθ​\[a⁡\(Y\)​w​\(Y\)​qθ​\(Y\)​qθ​\(Y\)⊤\],\\displaystyle=E\_\{\\theta\}\\\!\\left\[a\(Y\)w\(Y\)q\_\{\\theta\}\(Y\)q\_\{\\theta\}\(Y\)^\{\\top\}\\right\],\(3\.4\)Cθ​η​\(a\)\\displaystyle C\_\{\\theta\\eta\}\(a\)=Eθ​\[a⁡\(Y\)​w​\(Y\)​qθ​\(Y\)​qη​\(Y\)⊤\],\\displaystyle=E\_\{\\theta\}\\\!\\left\[a\(Y\)w\(Y\)q\_\{\\theta\}\(Y\)q\_\{\\eta\}\(Y\)^\{\\top\}\\right\],\(3\.5\)Cη​η​\(a\)\\displaystyle C\_\{\\eta\\eta\}\(a\)=Eθ​\[a⁡\(Y\)​w​\(Y\)​qη​\(Y\)​qη​\(Y\)⊤\]\.\\displaystyle=E\_\{\\theta\}\\\!\\left\[a\(Y\)w\(Y\)q\_\{\\eta\}\(Y\)q\_\{\\eta\}\(Y\)^\{\\top\}\\right\]\.\(3\.6\)
Assume throughout this section thatCη​η​\(a\)C\_\{\\eta\\eta\}\(a\)is positive definite\. The efficient Fisher information forθ\\theta, after adjustment forη\\eta, is the Schur complement

Ieff​\(a\)=Iθ​θ​\(a\)−Iθ​η​\(a\)​Iη​η​\(a\)−1​Iη​θ​\(a\)\.I\_\{\\mathrm\{eff\}\}\(a\)=I\_\{\\theta\\theta\}\(a\)\-I\_\{\\theta\\eta\}\(a\)I\_\{\\eta\\eta\}\(a\)^\{\-1\}I\_\{\\eta\\theta\}\(a\)\.\(3\.7\)Substitution of \([3\.1](https://arxiv.org/html/2611.24986#S3.E1)\)–\([3\.3](https://arxiv.org/html/2611.24986#S3.E3)\) yields

Ieff​\(a\)=IY​\(θ\)\+Eθ​\[a⁡\(Y\)​r​\(Y\)​J​\(Y\)\]\+IR,eff​\(a\),I\_\{\\mathrm\{eff\}\}\(a\)=I\_\{Y\}\(\\theta\)\+E\_\{\\theta\}\\\!\\left\[a\(Y\)r\(Y\)J\(Y\)\\right\]\+I\_\{R,\\mathrm\{eff\}\}\(a\),\(3\.8\)where

IR,eff​\(a\)=Cθ​θ​\(a\)−Cθ​η​\(a\)​Cη​η​\(a\)−1​Cη​θ​\(a\)\.I\_\{R,\\mathrm\{eff\}\}\(a\)=C\_\{\\theta\\theta\}\(a\)\-C\_\{\\theta\\eta\}\(a\)C\_\{\\eta\\eta\}\(a\)^\{\-1\}C\_\{\\eta\\theta\}\(a\)\.\(3\.9\)The first two terms in \([3\.8](https://arxiv.org/html/2611.24986#S3.E8)\) are respectively the information available from the feature distribution and the expected information from labels that are actually returned\. The final term measures the additional classifier information supplied by the response process after removing the component attributable to the nuisance parameterη\\eta\.

The following result gives a direct characterization of this contribution\.

###### Proposition 3\.1\.

Suppose thatCη​η​\(a\)C\_\{\\eta\\eta\}\(a\)is positive definite and define

Ba=Cθ​η​\(a\)​Cη​η​\(a\)−1\.B\_\{a\}=C\_\{\\theta\\eta\}\(a\)C\_\{\\eta\\eta\}\(a\)^\{\-1\}\.\(3\.10\)Then

IR,eff​\(a\)=Eθ​\[a⁡\(Y\)​w​\(Y\)​\{qθ​\(Y\)−Ba​qη​\(Y\)\}​\{qθ​\(Y\)−Ba​qη​\(Y\)\}⊤\]\.I\_\{R,\\mathrm\{eff\}\}\(a\)=E\_\{\\theta\}\\\!\\left\[a\(Y\)w\(Y\)\\\{q\_\{\\theta\}\(Y\)\-B\_\{a\}q\_\{\\eta\}\(Y\)\\\}\\\{q\_\{\\theta\}\(Y\)\-B\_\{a\}q\_\{\\eta\}\(Y\)\\\}^\{\\top\}\\right\]\.\(3\.11\)Consequently,

IR,eff​\(a\)⪰0\.I\_\{R,\\mathrm\{eff\}\}\(a\)\\succeq 0\.Moreover,

IR,eff\(a\)=0⟺qθ\(Y\)=Baqη\(Y\)μa\-almost surely,I\_\{R,\\mathrm\{eff\}\}\(a\)=0\\quad\\Longleftrightarrow\\quad q\_\{\\theta\}\(Y\)=B\_\{a\}q\_\{\\eta\}\(Y\)\\quad\\mu\_\{a\}\\text\{\-almost surely\},\(3\.12\)whereμa\\mu\_\{a\}is the measure with density proportional toa⁡\(y\)​w​\(y\)​pθ​\(y\)a\(y\)w\(y\)p\_\{\\theta\}\(y\)\.

The proof is given in Supplementary Section[S1\.2](https://arxiv.org/html/2611.24986#S1.SS2)\.

Proposition[3\.1](https://arxiv.org/html/2611.24986#S3.Thmtheorem1)identifies precisely when observing whether the oracle responds provides information about the classifier beyond that contained in returned labels\. The vectorqθ​\(y\)q\_\{\\theta\}\(y\)describes the local sensitivity of the response probability to the classifier parameter, whileqη​\(y\)q\_\{\\eta\}\(y\)describes sensitivity to the response\-specific nuisance parameter\. Equation \([3\.11](https://arxiv.org/html/2611.24986#S3.E11)\) shows that only the component ofqθ​\(Y\)q\_\{\\theta\}\(Y\)that cannot be represented through the nuisance score contributes efficient information for classification\.

This distinction is important\. Parameter dependence of the response probability alone does not guarantee additional information aboutθ\\thetaafter nuisance adjustment\. If \([3\.12](https://arxiv.org/html/2611.24986#S3.E12)\) holds, all classifier\-related variation in the response mechanism is statistically confounded with variation inη\\eta, and the response indicator contributes no additional efficient information\. Otherwise,

IR,eff​\(a\)≠0,I\_\{R,\\mathrm\{eff\}\}\(a\)\\neq 0,and observing response or abstention supplies classifier information that would be discarded by a design criterion based only on the probability of obtaining a label\.

The decomposition also identifies two limiting cases\. If the response mechanism is independent of the classifier parameter, thenqθ​\(y\)≡0q\_\{\\theta\}\(y\)\\equiv 0and hence

IR,eff​\(a\)=0\.I\_\{R,\\mathrm\{eff\}\}\(a\)=0\.The Bernoulli weightw⁡\(y\)=r⁡\(y\)​\{1−r⁡\(y\)\}w\(y\)=r\(y\)\\\{1\-r\(y\)\\\}approaches zero as the response probability approaches either endpoint\. If the corresponding logit\-score vectors remain bounded, the pointwise response\-information contribution also vanishes\.

The positive\-definiteness requirement onCη​η​\(a\)C\_\{\\eta\\eta\}\(a\)is itself design\-dependent\. An acquisition rule may therefore be highly informative for classification while failing to identify the response mechanism\. We return to this issue when defining the admissible class of acquisition designs\.

## 4Classification risk under informative oracle response

The information criterion in Section[3](https://arxiv.org/html/2611.24986#S3)measures precision in estimating the model parameters, but the objective of label acquisition is classification\. We therefore evaluate an acquisition design through its leading contribution to the zero\-one classification risk\.

Let

dθ​\(y\)=arg⁡max1≤k≤g​τk​\(y,θ\)d\_\{\\theta\}\(y\)=\\arg\\max\_\{1\\leq k\\leq g\}\\tau\_\{k\}\(y;\\theta\)denote the plug\-in Bayes rule and define its risk, under the true parameterθ0\\theta\_\{0\}, by

R\(θ\)=Prθ0\{dθ\(Y\)≠Z\}\.R\(\\theta\)=\\Pr\_\{\\theta\_\{0\}\}\\\{d\_\{\\theta\}\(Y\)\\neq Z\\\}\.Let

R∗=R⁡\(θ0\)R^\{\*\}=R\(\\theta\_\{0\}\)be the Bayes risk\. Under the regular boundary conditions for multiclass classification, the local excess risk admits the quadratic expansion

R⁡\(θ0\+h\)−R∗=12​h⊤​HR​h\+o⁡\(‖h‖2\),h→0,R\(\\theta\_\{0\}\+h\)\-R^\{\*\}=\\frac\{1\}\{2\}h^\{\\top\}H\_\{R\}h\+o\(\\\|h\\\|^\{2\}\),\\qquad h\\rightarrow 0,\(4\.1\)whereHR⪰0H\_\{R\}\\succeq 0is the classification\-risk curvature matrix atθ0\\theta\_\{0\}\. It depends only on parameter directions that perturb active Bayes decision boundaries\.

The response parameterη\\etadoes not enter the classifier\. Accordingly, for the full parameter

ϕ=\(θ⊤,η⊤\)⊤,\\phi=\(\\theta^\{\\top\},\\eta^\{\\top\}\)^\{\\top\},define the augmented risk\-curvature matrix

H~R=\(HR000\)\.\\widetilde\{H\}\_\{R\}=\\begin\{pmatrix\}H\_\{R\}&0\\\\ 0&0\\end\{pmatrix\}\.\(4\.2\)
Suppose that, under a fixed admissible acquisition designaa, an efficient estimatorϕ^a\\widehat\{\\phi\}\_\{a\}satisfies

n​\(ϕ^a−ϕ0\)​⟶𝑑​N​\{0,ℐ​\(a\)−1\},\\sqrt\{n\}\(\\widehat\{\\phi\}\_\{a\}\-\\phi\_\{0\}\)\\overset\{d\}\{\\longrightarrow\}N\\\{0,\\mathcal\{I\}\(a\)^\{\-1\}\\\},\(4\.3\)withℐ⁡\(a\)\\mathcal\{I\}\(a\)nonsingular\. Assume additionally that, writingen,a=θ^a−θ0e\_\{n,a\}=\\widehat\{\\theta\}\_\{a\}\-\\theta\_\{0\},

n​E​\(en,a​en,a⊤\)⟶Ieff​\(a\)−1,n​E​\|R⁡\(θ^a\)−R∗−12​en,a⊤​HR​en,a\|⟶0\.nE\(e\_\{n,a\}e\_\{n,a\}^\{\\top\}\)\\longrightarrow I\_\{\\mathrm\{eff\}\}\(a\)^\{\-1\},\\qquad nE\\left\|R\(\\widehat\{\\theta\}\_\{a\}\)\-R^\{\*\}\-\\tfrac\{1\}\{2\}e\_\{n,a\}^\{\\top\}H\_\{R\}e\_\{n,a\}\\right\|\\longrightarrow 0\.These moment and remainder conditions, together with \([4\.1](https://arxiv.org/html/2611.24986#S4.E1)\), give the leading expected excess risk

E⁡\{R⁡\(θ^a\)\}−R∗=12​n​tr⁡\{H~R​ℐ​\(a\)−1\}\+o⁡\(n−1\)\.E\\\{R\(\\widehat\{\\theta\}\_\{a\}\)\\\}\-R^\{\*\}=\\frac\{1\}\{2n\}\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\}\+o\(n^\{\-1\}\)\.\(4\.4\)Equivalently, using the block\-inverse identity,

tr⁡\{H~R​ℐ​\(a\)−1\}=tr⁡\{HR​Ieff​\(a\)−1\},\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\}=\\operatorname\{tr\}\\left\\\{H\_\{R\}I\_\{\\mathrm\{eff\}\}\(a\)^\{\-1\}\\right\\\},\(4\.5\)whereIeff​\(a\)I\_\{\\mathrm\{eff\}\}\(a\)is the nuisance\-adjusted information in \([3\.7](https://arxiv.org/html/2611.24986#S3.E7)\)\.

This motivates the classification\-risk criterion

Φ⁡\(a\)=tr⁡\{H~R​ℐ​\(a\)−1\}\.\\Phi\(a\)=\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\}\.\(4\.6\)For a labeling budgetρ∈\(0,1\)\\rho\\in\(0,1\), the oracle acquisition problem is therefore

mina∈𝒜ρΦ\(a\),𝒜ρ=\{a:0≤a\(y\)≤1,Eθ0\{a\(Y\)\}=ρ,ℐ\(a\)≻0\}\.\\min\_\{a\\in\\mathcal\{A\}\_\{\\rho\}\}\\Phi\(a\),\\qquad\\mathcal\{A\}\_\{\\rho\}=\\left\\\{a:\\ 0\\leq a\(y\)\\leq 1,\\ E\_\{\\theta\_\{0\}\}\\\{a\(Y\)\\\}=\\rho,\\ \\mathcal\{I\}\(a\)\\succ 0\\right\\\}\.\(4\.7\)The nonsingularity condition is substantive in the present setting because the acquisition design determines not only which labels may be observed but also which response outcomes are available for identifying the response mechanism\.

Since

ℐ⁡\(a\)=ℐY\+E⁡\{a⁡\(Y\)​K​\(Y\)\}\\mathcal\{I\}\(a\)=\\mathcal\{I\}\_\{Y\}\+E\\\{a\(Y\)K\(Y\)\\\}is affine inaa, the criterion in \([4\.6](https://arxiv.org/html/2611.24986#S4.E6)\) retains the trace\-inverse structure of an optimal\-design problem despite the nuisance adjustment inIeff​\(a\)I\_\{\\mathrm\{eff\}\}\(a\)\.

The local value of querying a feature vectoryyis obtained from the directional derivative ofΦ\\Phi\. Define

ψa​\(y\)=tr⁡\{H~R​ℐ​\(a\)−1​K​\(y\)​ℐ​\(a\)−1\}\.\\psi\_\{a\}\(y\)=\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}K\(y\)\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\}\.\(4\.8\)Because

K⁡\(y\)=r⁡\(y\)​J~​\(y\)\+w⁡\(y\)​q​\(y\)​q​\(y\)⊤,K\(y\)=r\(y\)\\widetilde\{J\}\(y\)\+w\(y\)q\(y\)q\(y\)^\{\\top\},the query value decomposes as

ψa​\(y\)=ψaL​\(y\)\+ψaR​\(y\),\\psi\_\{a\}\(y\)=\\psi\_\{a\}^\{L\}\(y\)\+\\psi\_\{a\}^\{R\}\(y\),\(4\.9\)where

ψaL​\(y\)\\displaystyle\\psi\_\{a\}^\{L\}\(y\)=r⁡\(y\)​tr⁡\{H~R​ℐ​\(a\)−1​J~​\(y\)​ℐ​\(a\)−1\},\\displaystyle=r\(y\)\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}\\widetilde\{J\}\(y\)\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\},\(4\.10\)ψaR​\(y\)\\displaystyle\\psi\_\{a\}^\{R\}\(y\)=w⁡\(y\)​q​\(y\)⊤​ℐ​\(a\)−1​H~R​ℐ​\(a\)−1​q​\(y\)\.\\displaystyle=w\(y\)q\(y\)^\{\\top\}\\mathcal\{I\}\(a\)^\{\-1\}\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}q\(y\)\.\(4\.11\)
The termψaL​\(y\)\\psi\_\{a\}^\{L\}\(y\)is the classification value of the label, discounted by the probability that the oracle returns it\. In contrast,ψaR​\(y\)\\psi\_\{a\}^\{R\}\(y\)is the classification value of observing the response outcome itself\. In particular,

ψaR​\(y\)≥0\.\\psi\_\{a\}^\{R\}\(y\)\\geq 0\.Thus the value of a query is not, in general, obtained by simply multiplying a label\-acquisition score by the probability of response\. Such a calculation retainsψaL​\(y\)\\psi\_\{a\}^\{L\}\(y\)but discardsψaR​\(y\)\\psi\_\{a\}^\{R\}\(y\)\.

###### Proposition 4\.1\.

Suppose thatℐ⁡\(a\)≻0\\mathcal\{I\}\(a\)\\succ 0, and lethhbe a bounded measurable perturbation such thatat=a\+t​ha\_\{t\}=a\+thremains admissible for sufficiently small\|t\|\|t\|\. Then

dd​t​Φ​\(at\)\|t=0=−Eθ0​\{h⁡\(Y\)​ψa​\(Y\)\},\\left\.\\frac\{d\}\{dt\}\\Phi\(a\_\{t\}\)\\right\|\_\{t=0\}=\-E\_\{\\theta\_\{0\}\}\\\!\\left\\\{h\(Y\)\\psi\_\{a\}\(Y\)\\right\\\},\(4\.12\)whereψa\\psi\_\{a\}is given by \([4\.8](https://arxiv.org/html/2611.24986#S4.E8)\)\. Moreover,ψa\\psi\_\{a\}admits the decomposition \([4\.9](https://arxiv.org/html/2611.24986#S4.E9)\)–\([4\.11](https://arxiv.org/html/2611.24986#S4.E11)\)\.

The proof is given in Supplementary Section[S1\.3](https://arxiv.org/html/2611.24986#S1.SS3)\.

Proposition[4\.1](https://arxiv.org/html/2611.24986#S4.Thmtheorem1)provides the local principle underlying response\-aware acquisition\. A candidate can be valuable because its label is likely to be returned and informative for the decision boundary, because its response outcome is itself informative for classifier\-relevant parameter directions, or through both channels\. Consequently, two candidates can be ordered differently by a response\-discounted label criterion and by the full classification\-risk criterion\. The optimal allocation implied by this marginal value is characterized in the next section\.

## 5Risk\-optimal acquisition and response identifiability

We now characterize acquisition designs that minimize the leading classification\-risk criterion

Φ⁡\(a\)=tr⁡\{H~R​ℐ​\(a\)−1\}\\Phi\(a\)=\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\}subject to a fixed query budget\. Because the information matrix is affine in the acquisition design, the resulting problem retains a convex optimal\-design structure even though the response mechanism contains unknown nuisance parameters\.

For a fixed budgetρ∈\(0,1\)\\rho\\in\(0,1\), consider

𝒜ρ∘=\{a:𝒴→\[0,1\]:Eθ0\{a\(Y\)\}=ρ,ℐ\(a\)≻0\}\.\\mathcal\{A\}\_\{\\rho\}^\{\\circ\}=\\left\\\{a:\\mathcal\{Y\}\\rightarrow\[0,1\]:E\_\{\\theta\_\{0\}\}\\\{a\(Y\)\\\}=\\rho,\\ \\mathcal\{I\}\(a\)\\succ 0\\right\\\}\.\(5\.1\)We assume that this class is nonempty and that an optimal design exists\.

###### Proposition 5\.1\.

On any convex subset of𝒜ρ∘\\mathcal\{A\}\_\{\\rho\}^\{\\circ\}, the criterion

a⟼Φ⁡\(a\)a\\longmapsto\\Phi\(a\)is convex\.

The proof is given in Supplementary Section[S1\.4](https://arxiv.org/html/2611.24986#S1.SS4)\.

Convexity, together with the derivative in Proposition[4\.1](https://arxiv.org/html/2611.24986#S4.Thmtheorem1), yields a direct characterization of the oracle acquisition rule\.

###### Theorem 5\.2\.

Leta∗∈𝒜ρ∘a^\{\*\}\\in\\mathcal\{A\}\_\{\\rho\}^\{\\circ\}minimizeΦ⁡\(a\)\\Phi\(a\), and suppose thata∗a^\{\*\}is an interior point of the positive\-definite information domain relative to the box and budget constraints\. Then there existsλ∈ℝ\\lambda\\in\\mathbb\{R\}such that, forPθ0P\_\{\\theta\_\{0\}\}\-almost everyyy,

\{ψa∗​\(y\)\>λ⟹a∗​\(y\)=1,ψa∗​\(y\)<λ⟹a∗​\(y\)=0,0<a∗​\(y\)<1⟹ψa∗​\(y\)=λ,\\begin\{cases\}\\psi\_\{a^\{\*\}\}\(y\)\>\\lambda&\\Longrightarrow\\quad a^\{\*\}\(y\)=1,\\\\\[3\.0pt\] \\psi\_\{a^\{\*\}\}\(y\)<\\lambda&\\Longrightarrow\\quad a^\{\*\}\(y\)=0,\\\\\[3\.0pt\] 0<a^\{\*\}\(y\)<1&\\Longrightarrow\\quad\\psi\_\{a^\{\*\}\}\(y\)=\\lambda,\\end\{cases\}\(5\.2\)where

ψa∗​\(y\)=ψa∗L​\(y\)\+ψa∗R​\(y\)\\psi\_\{a^\{\*\}\}\(y\)=\\psi\_\{a^\{\*\}\}^\{L\}\(y\)\+\\psi\_\{a^\{\*\}\}^\{R\}\(y\)is the response\-aware classification value defined in \([4\.9](https://arxiv.org/html/2611.24986#S4.E9)\)\. Conversely, any feasible design satisfying these conditions is globally optimal\.

The proof is given in Supplementary Section[S1\.5](https://arxiv.org/html/2611.24986#S1.SS5)\.

Theorem[5\.2](https://arxiv.org/html/2611.24986#S5.Thmtheorem2)shows that the optimal design allocates the labeling budget according to the total classification value of a query, rather than according to the value of a potentially returned label alone\. Except on level sets where randomization is required to satisfy the budget, the oracle rule therefore has a threshold form\.

This distinction can alter the ordering of candidate observations\. A response\-discounted rule based only onψaL​\(y\)\\psi\_\{a\}^\{L\}\(y\)favors observations whose returned labels are valuable after accounting for their probability of response\. The full rule instead ranks observations according to

ψaL​\(y\)\+ψaR​\(y\)\.\\psi\_\{a\}^\{L\}\(y\)\+\\psi\_\{a\}^\{R\}\(y\)\.Consequently, an observation with a smaller probability of returning a label may still have greater total acquisition value if its response outcome is sufficiently informative about classifier\-relevant parameter directions\. The phenomenon does not require response probabilities close to zero: indeed, the Bernoulli information factorw⁡\(y\)=r⁡\(y\)​\{1−r⁡\(y\)\}w\(y\)=r\(y\)\\\{1\-r\(y\)\\\}vanishes asr⁡\(y\)r\(y\)approaches either zero or one\.

### 5\.1Response identifiability under selective acquisition

The preceding optimality result presumes that the information matrix is nonsingular\. In the present problem this requirement depends on the acquisition design itself\. A design that concentrates queries on a restricted region of the feature space can provide substantial classification information while failing to identify the response mechanism\.

The nuisance\-information block is

Cη​η​\(a\)=Eθ0​\[a⁡\(Y\)​w​\(Y\)​qη​\(Y\)​qη​\(Y\)⊤\]\.C\_\{\\eta\\eta\}\(a\)=E\_\{\\theta\_\{0\}\}\\\!\\left\[a\(Y\)w\(Y\)q\_\{\\eta\}\(Y\)q\_\{\\eta\}\(Y\)^\{\\top\}\\right\]\.\(5\.3\)
###### Proposition 5\.3\.

For anyv∈ℝqv\\in\\mathbb\{R\}^\{q\},

v⊤​Cη​η​\(a\)​v=Eθ0​\[a⁡\(Y\)​w​\(Y\)​\{v⊤​qη​\(Y\)\}2\]\.v^\{\\top\}C\_\{\\eta\\eta\}\(a\)v=E\_\{\\theta\_\{0\}\}\\\!\\left\[a\(Y\)w\(Y\)\\\{v^\{\\top\}q\_\{\\eta\}\(Y\)\\\}^\{2\}\\right\]\.\(5\.4\)Consequently,

Cη​η​\(a\)≻0C\_\{\\eta\\eta\}\(a\)\\succ 0if and only if there is no nonzerov∈ℝqv\\in\\mathbb\{R\}^\{q\}such that

v⊤​qη​\(Y\)=0μa​\-almost surely,v^\{\\top\}q\_\{\\eta\}\(Y\)=0\\qquad\\mu\_\{a\}\\text\{\-almost surely\},\(5\.5\)whereμa\\mu\_\{a\}is the weighted measure introduced in Proposition[3\.1](https://arxiv.org/html/2611.24986#S3.Thmtheorem1)\.

The proof is given in Supplementary Section[S1\.6](https://arxiv.org/html/2611.24986#S1.SS6)\.

Proposition[5\.3](https://arxiv.org/html/2611.24986#S5.Thmtheorem3)gives a design\-specific rank condition for estimation of the response nuisance parameter\. In particular, positive query probability alone is insufficient: the queried observations must exhibit enough variation in the response\-model regressors to distinguish the components ofη\\eta\.

For example, consider the response model

logit⁡r⁡\(y,θ,η\)=η0\+η1​U​\(y,θ\),\\operatorname\{logit\}r\(y;\\theta,\\eta\)=\\eta\_\{0\}\+\\eta\_\{1\}U\(y;\\theta\),\(5\.6\)whereU⁡\(y,θ\)U\(y;\\theta\)is a scalar measure of classification uncertainty\. Then

qη​\(y\)=\(1U⁡\(y,θ\)\)\.q\_\{\\eta\}\(y\)=\\begin\{pmatrix\}1\\\\ U\(y;\\theta\)\\end\{pmatrix\}\.Provideda⁡\(Y\)​w​\(Y\)a\(Y\)w\(Y\)has positive expectation,Cη​η​\(a\)≻0C\_\{\\eta\\eta\}\(a\)\\succ 0if and only if

Varμa⁡\{U⁡\(Y,θ0\)\}\>0\.\\operatorname\{Var\}\_\{\\mu\_\{a\}\}\\\{U\(Y;\\theta\_\{0\}\)\\\}\>0\.\(5\.7\)Thus a design supported only on observations having the same uncertainty level cannot separately identify the intercept and uncertainty effect in the response model\. In particular, concentrating all queries on a single decision\-boundary uncertainty level can destroy response\-mechanism identifiability\.

For theoretical and numerical implementation it is therefore useful to work with the uniformly regular design class

𝒜ρ,κ=\{a:0≤a\(y\)≤1,Eθ0\{a\(Y\)\}=ρ,λmin\{ℐ\(a\)\}≥κ\},κ\>0\.\\mathcal\{A\}\_\{\\rho,\\kappa\}=\\left\\\{a:\\ 0\\leq a\(y\)\\leq 1,\\ E\_\{\\theta\_\{0\}\}\\\{a\(Y\)\\\}=\\rho,\\ \\lambda\_\{\\min\}\\\{\\mathcal\{I\}\(a\)\\\}\\geq\\kappa\\right\\\},\\qquad\\kappa\>0\.\(5\.8\)The lower information bound prevents the acquisition procedure from obtaining an apparent classification advantage by approaching a design under which some components of the annotation experiment cease to be identifiable\. This constraint is one of the regularity requirements used in the two\-stage analysis below; it does not by itself establish the estimator conditions needed for that analysis\.

## 6Adaptive response\-aware acquisition

The oracle criterion depends on the unknown classification and response parameters and on the risk curvature\. We first analyze a two\-stage procedure with a pilot\-selected design that is held fixed during an independent second stage\. We then distinguish this experiment from the sequential finite\-pool implementation used in the numerical studies\.

### 6\.1Pilot selection and the second\-stage experiment

Letmnm\_\{n\}be the number of pilot sampling units and letnnbe the number of new second\-stage sampling units\. Assume

mn⟶∞,mn/n⟶0\.m\_\{n\}\\longrightarrow\\infty,\\qquad m\_\{n\}/n\\longrightarrow 0\.\(6\.1\)Writeℱ0,n\\mathcal\{F\}\_\{0,n\}for the sigma\-field generated by the pilot features, query decisions, responses, and returned labels, together with any auxiliary randomness used to choose the second\-stage design\. The pilot uses a design that provides nonsingular information for the parameters to be estimated\. Letϕ^0\\widehat\{\\phi\}\_\{0\}andH^R\\widehat\{H\}\_\{R\}denote the pilot estimates\. The selected acquisition functiona^n\\widehat\{a\}\_\{n\}isℱ0,n\\mathcal\{F\}\_\{0,n\}\-measurable\.

Generate new pairs\(Yi,Zi\)\(Y\_\{i\},Z\_\{i\}\),i=1,…,ni=1,\\ldots,n, independently frompθ0p\_\{\\theta\_\{0\}\}and independently ofℱ0,n\\mathcal\{F\}\_\{0,n\}\. LetUi,ViU\_\{i\},V\_\{i\}be independent uniform random variables on\(0,1\)\(0,1\), independent across units and of the pilot and the new pairs\. Set

Ai=𝟏\{Ui≤a^n\(Yi\)\},AiRi=Ai𝟏\{Vi≤r\(Yi;ϕ0\)\}\.A\_\{i\}=\\mathbf\{1\}\\\{U\_\{i\}\\leq\\widehat\{a\}\_\{n\}\(Y\_\{i\}\)\\\},\\qquad A\_\{i\}R\_\{i\}=A\_\{i\}\\mathbf\{1\}\\\{V\_\{i\}\\leq r\(Y\_\{i\};\\phi\_\{0\}\)\\\}\.\(6\.2\)This construction implements the conditional independence of response and class label assumed in Section[2](https://arxiv.org/html/2611.24986#S2)\. DefineOi=\(Yi,Ai,Ai​Ri,Ai​Ri​Zi\)O\_\{i\}=\(Y\_\{i\},A\_\{i\},A\_\{i\}R\_\{i\},A\_\{i\}R\_\{i\}Z\_\{i\}\), and letPϕ,aP\_\{\\phi,a\}denote the one\-unit observed\-data law under parameterϕ\\phiand a fixed designaa\. Then

ℒ⁡\(O1,…,On∣ℱ0,n\)=Pϕ0,a^n⊗n\.\\mathcal\{L\}\(O\_\{1\},\\ldots,O\_\{n\}\\mid\\mathcal\{F\}\_\{0,n\}\)=P\_\{\\phi\_\{0\},\\widehat\{a\}\_\{n\}\}^\{\\otimes n\}\.\(6\.3\)Thus the second\-stage observations are independent and identically distributed conditional on the pilot\. Unconditionally they generally are not independent, because they share the random designa^n\\widehat\{a\}\_\{n\}\. We do not apply an iid argument to the pooled two\-stage sample\.

For example, for a fixed designaa, the nonzero observed\-data densities are

pϕ,a​\(y,A=0\)\\displaystyle p\_\{\\phi,a\}\(y,A=0\)=pθ​\(y\)​\{1−a⁡\(y\)\},\\displaystyle=p\_\{\\theta\}\(y\)\\\{1\-a\(y\)\\\},pϕ,a​\(y,A=1,R=0\)\\displaystyle p\_\{\\phi,a\}\(y,A=1,R=0\)=pθ​\(y\)​a​\(y\)​\{1−r⁡\(y,ϕ\)\},\\displaystyle=p\_\{\\theta\}\(y\)a\(y\)\\\{1\-r\(y;\\phi\)\\\},pϕ,a​\(y,A=1,R=1,Z=z\)\\displaystyle p\_\{\\phi,a\}\(y,A=1,R=1,Z=z\)=pθ​\(y\)​a​\(y\)​r​\(y,ϕ\)​τz​\(y,θ\)\.\\displaystyle=p\_\{\\theta\}\(y\)a\(y\)r\(y;\\phi\)\\tau\_\{z\}\(y;\\theta\)\.Conditional on the observed pilot, the selected functiona^n\\widehat\{a\}\_\{n\}is held fixed when differentiating the second\-stage likelihood\. Its information per sampling unit is thereforeℐ⁡\(a^n\)\\mathcal\{I\}\(\\widehat\{a\}\_\{n\}\)from Proposition[2\.1](https://arxiv.org/html/2611.24986#S2.Thmtheorem1), evaluated atϕ0\\phi\_\{0\}\. The returned\-label subset need not follow the population feature law; \([6\.3](https://arxiv.org/html/2611.24986#S6.E3)\) concerns the complete observed\-data units, including unqueried units and queried abstentions\.

### 6\.2Design selection and oracle value

Use a fixed, nonempty measurable design class𝒟⊆𝒜ρ,κ\\mathcal\{D\}\\subseteq\\mathcal\{A\}\_\{\\rho,\\kappa\}, where𝒜ρ,κ\\mathcal\{A\}\_\{\\rho,\\kappa\}is defined in \([5\.8](https://arxiv.org/html/2611.24986#S5.E8)\)\. One may take the full regular class if the uniform assumptions below hold; a restricted class may be needed for empirical estimation\. All suprema and selected designs below are assumed measurable\. Define

ℐ^n​\(a\)=ℐ^Y,n\+E^n​\{a⁡\(Y\)​K^n​\(Y\)\},\\widehat\{\\mathcal\{I\}\}\_\{n\}\(a\)=\\widehat\{\\mathcal\{I\}\}\_\{Y,n\}\+\\widehat\{E\}\_\{n\}\\\{a\(Y\)\\widehat\{K\}\_\{n\}\(Y\)\\\},\(6\.4\)where

K^n​\(y\)=r^n​\(y\)​J~n​\(y\)\+r^n​\(y\)​\{1−r^n​\(y\)\}​q^n​\(y\)​q^n​\(y\)⊤,\\widehat\{K\}\_\{n\}\(y\)=\\widehat\{r\}\_\{n\}\(y\)\\widetilde\{J\}\_\{n\}\(y\)\+\\widehat\{r\}\_\{n\}\(y\)\\\{1\-\\widehat\{r\}\_\{n\}\(y\)\\\}\\widehat\{q\}\_\{n\}\(y\)\\widehat\{q\}\_\{n\}\(y\)^\{\\top\},\(6\.5\)and

Φ^n​\(a\)=tr⁡\{H~^R,n​ℐ^n​\(a\)−1\},H~^R,n=\(H^R000\)\.\\widehat\{\\Phi\}\_\{n\}\(a\)=\\operatorname\{tr\}\\\{\\widehat\{\\widetilde\{H\}\}\_\{R,n\}\\widehat\{\\mathcal\{I\}\}\_\{n\}\(a\)^\{\-1\}\\\},\\qquad\\widehat\{\\widetilde\{H\}\}\_\{R,n\}=\\begin\{pmatrix\}\\widehat\{H\}\_\{R\}&0\\\\ 0&0\\end\{pmatrix\}\.\(6\.6\)The criterion and every quantity used to choosea^n\\widehat\{a\}\_\{n\}must beℱ0,n\\mathcal\{F\}\_\{0,n\}\-measurable\. In this experiment,E^n\\widehat\{E\}\_\{n\}may use pilot features, model\-based integration, or an independent reference feature sample included inℱ0,n\\mathcal\{F\}\_\{0,n\}\. It does not use the features of the second\-stage sample\. Positive definiteness of the estimated information is required on the class over which the criterion is optimized\.

Choose a measurable approximate minimizer

a^n∈𝒟,Φ^n​\(a^n\)≤infa∈𝒟Φ^n​\(a\)\+εn,0≤εn=op​\(1\)\.\\widehat\{a\}\_\{n\}\\in\\mathcal\{D\},\\qquad\\widehat\{\\Phi\}\_\{n\}\(\\widehat\{a\}\_\{n\}\)\\leq\\inf\_\{a\\in\\mathcal\{D\}\}\\widehat\{\\Phi\}\_\{n\}\(a\)\+\\varepsilon\_\{n\},\\qquad 0\\leq\\varepsilon\_\{n\}=o\_\{p\}\(1\)\.\(6\.7\)Feasibility in the population class𝒟\\mathcal\{D\}is an assumption\. Using estimated budget and information constraints instead requires a separate feasibility\-approximation argument\.

For deterministica∈𝒟a\\in\\mathcal\{D\}, letϕ^n,a\\widehat\{\\phi\}\_\{n,a\}be an estimator based only onnniid observations fromPϕ0,aP\_\{\\phi\_\{0\},a\}\. It is a measurable function of these observations andaa, with no other pilot dependence\. WriteEaE\_\{a\}for expectation in this fixed\-design experiment andSa​\(O,ϕ0\)S\_\{a\}\(O;\\phi\_\{0\}\)for its joint score, given by \([2\.6](https://arxiv.org/html/2611.24986#S2.E6)\)–\([2\.7](https://arxiv.org/html/2611.24986#S2.E7)\)\. Letθ^n,a∈Θ\\widehat\{\\theta\}\_\{n,a\}\\in\\Thetabe its classification component\.

###### Theorem 6\.1\.

Suppose thata∗∈arg⁡mina∈𝒟⁡Φ⁡\(a\)a^\{\*\}\\in\\arg\\min\_\{a\\in\\mathcal\{D\}\}\\Phi\(a\)exists, \([6\.7](https://arxiv.org/html/2611.24986#S6.E7)\) holds, and

Δn:=supa∈𝒟\|Φ^n​\(a\)−Φ⁡\(a\)\|​⟶𝑝​0\.\\Delta\_\{n\}:=\\sup\_\{a\\in\\mathcal\{D\}\}\|\\widehat\{\\Phi\}\_\{n\}\(a\)\-\\Phi\(a\)\|\\overset\{p\}\{\\longrightarrow\}0\.\(6\.8\)Then

0≤Φ⁡\(a^n\)−Φ⁡\(a∗\)≤2​Δn\+εn​⟶𝑝​0\.0\\leq\\Phi\(\\widehat\{a\}\_\{n\}\)\-\\Phi\(a^\{\*\}\)\\leq 2\\Delta\_\{n\}\+\\varepsilon\_\{n\}\\overset\{p\}\{\\longrightarrow\}0\.\(6\.9\)
For the expected\-risk conclusion, additionally assume the independent second\-stage construction \([6\.2](https://arxiv.org/html/2611.24986#S6.E2)\)–\([6\.3](https://arxiv.org/html/2611.24986#S6.E3)\), the local risk expansion \([4\.1](https://arxiv.org/html/2611.24986#S4.E1)\), and the following uniform fixed\-design estimator conditions\. First, assume

n​\(ϕ^n,a−ϕ0\)=ℐ​\(a\)−1​1n​∑i=1nSa​\(Oi,ϕ0\)\+ζn,a,supa∈𝒟Ea​‖ζn,a‖2⟶0\.\\begin\{split\}\\sqrt\{n\}\(\\widehat\{\\phi\}\_\{n,a\}\-\\phi\_\{0\}\)&=\\mathcal\{I\}\(a\)^\{\-1\}\\frac\{1\}\{\\sqrt\{n\}\}\\sum\_\{i=1\}^\{n\}S\_\{a\}\(O\_\{i\};\\phi\_\{0\}\)\+\\zeta\_\{n,a\},\\\\ \\sup\_\{a\\in\\mathcal\{D\}\}E\_\{a\}\\\|\\zeta\_\{n,a\}\\\|^\{2\}&\\longrightarrow 0\.\\end\{split\}\(6\.10\)Second, for someδ\>0\\delta\>0, assume

supn≥n0supa∈𝒟Ea​‖n​\(θ^n,a−θ0\)‖2\+δ<∞\.\\sup\_\{n\\geq n\_\{0\}\}\\sup\_\{a\\in\\mathcal\{D\}\}E\_\{a\}\\\|\\sqrt\{n\}\(\\widehat\{\\theta\}\_\{n,a\}\-\\theta\_\{0\}\)\\\|^\{2\+\\delta\}<\\infty\.\(6\.11\)Assume also the usual score identitiesEa​Sa=0E\_\{a\}S\_\{a\}=0andEa​\(Sa​Sa⊤\)=ℐ⁡\(a\)E\_\{a\}\(S\_\{a\}S\_\{a\}^\{\\top\}\)=\\mathcal\{I\}\(a\)\. Then the second\-stage estimator satisfies

E⁡\{R⁡\(θ^n,a^n\)−R∗∣ℱ0,n\}=Φ⁡\(a^n\)2​n\+rn,E​\|rn\|=o⁡\(n−1\),E\\\{R\(\\widehat\{\\theta\}\_\{n,\\widehat\{a\}\_\{n\}\}\)\-R^\{\*\}\\mid\\mathcal\{F\}\_\{0,n\}\\\}=\\frac\{\\Phi\(\\widehat\{a\}\_\{n\}\)\}\{2n\}\+r\_\{n\},\\qquad E\|r\_\{n\}\|=o\(n^\{\-1\}\),\(6\.12\)and consequently

E⁡\{R⁡\(θ^n,a^n\)\}−R∗=Φ⁡\(a∗\)2​n\+o⁡\(n−1\)\.E\\\{R\(\\widehat\{\\theta\}\_\{n,\\widehat\{a\}\_\{n\}\}\)\\\}\-R^\{\*\}=\\frac\{\\Phi\(a^\{\*\}\)\}\{2n\}\+o\(n^\{\-1\}\)\.\(6\.13\)

The proof is given in Supplementary Section[S1\.7](https://arxiv.org/html/2611.24986#S1.SS7)\. It first establishes a uniform expected\-risk expansion for deterministic designs and then uses the exact conditional law \([6\.3](https://arxiv.org/html/2611.24986#S6.E3)\)\. In particular, pointwise asymptotic normality for each fixed design would not suffice: the selected design varies with the pilot, and convergence in distribution alone does not justify convergence of expected risk\. Conditions \([6\.10](https://arxiv.org/html/2611.24986#S6.E10)\) and \([6\.11](https://arxiv.org/html/2611.24986#S6.E11)\) supply the required uniform second\-moment and remainder control\. They are assumptions on the estimator family, not consequences established here for arbitrary likelihood optimizers\.

The information bound gives0≤Φ⁡\(a\)≤κ−1​tr⁡\(H~R\)0\\leq\\Phi\(a\)\\leq\\kappa^\{\-1\}\\operatorname\{tr\}\(\\widetilde\{H\}\_\{R\}\)on𝒟\\mathcal\{D\}\. This bound allows the conditional conclusion to be averaged over the pilot without treating the two stages as iid\. The result does not require a unique oracle design\. It is an optimal\-value result over𝒟\\mathcal\{D\}, rather than a proof that the selected design itself converges to a unique function\.

Uniform criterion consistency must also be justified for the chosen approximation and class\. Pilot\-parameter consistency alone does not imply \([6\.8](https://arxiv.org/html/2611.24986#S6.E8)\) for unrestricted measurable designs when expectations are replaced by empirical averages\. A suitable uniform law of large numbers, an appropriate restricted design class, or direct uniform control of model\-based integration is needed\.

The theorem concerns a fixed expected query budget: conditional on the pilot, the second stage usesn​ρn\\rhoqueries in expectation\. Its final estimator uses only the new observations\. IfNn=n\+mnN\_\{n\}=n\+m\_\{n\}counts both stages, \([6\.1](https://arxiv.org/html/2611.24986#S6.E1)\) givesn/Nn→1n/N\_\{n\}\\to 1, so the same risk coefficient can be written with denominator2​Nn2N\_\{n\}\. Pilot queries contribute at mostmn=o⁡\(Nn\)m\_\{n\}=o\(N\_\{n\}\), and the overall expected query fraction tends toρ\\rho\. These accounting statements do not prove an equivalent result for a pooled estimator\. For such an estimatorθ~n\\widetilde\{\\theta\}\_\{n\}, one sufficient additional requirement is

E​‖n​\(θ~n−θ^n,a^n\)‖2⟶0,supn≥n0E​‖n​\(θ~n−θ0\)‖2\+δ<∞\.E\\\|\\sqrt\{n\}\(\\widetilde\{\\theta\}\_\{n\}\-\\widehat\{\\theta\}\_\{n,\\widehat\{a\}\_\{n\}\}\)\\\|^\{2\}\\longrightarrow 0,\\qquad\\sup\_\{n\\geq n\_\{0\}\}E\\\|\\sqrt\{n\}\(\\widetilde\{\\theta\}\_\{n\}\-\\theta\_\{0\}\)\\\|^\{2\+\\delta\}<\\infty\.The same quadratic\-risk argument then transfers the leading coefficient\. The conditionmn/n→0m\_\{n\}/n\\to 0alone does not establish these requirements\.

### 6\.3Sequential finite\-pool implementation

For an available pool\{y1,…,yN\}\\\{y\_\{1\},\\ldots,y\_\{N\}\\\}, the plug\-in marginal value is

ψ^a​\(yi\)=ψ^aL​\(yi\)\+ψ^aR​\(yi\)\.\\widehat\{\\psi\}\_\{a\}\(y\_\{i\}\)=\\widehat\{\\psi\}\_\{a\}^\{L\}\(y\_\{i\}\)\+\\widehat\{\\psi\}\_\{a\}^\{R\}\(y\_\{i\}\)\.\(6\.14\)A practical implementation updates the fitted classifier, response model, and information matrices after each batch and selects high\-scoring remaining observations\. This uses the query\-value decomposition but is a different sampling experiment from the theorem\. Selection without replacement, exact batch sizes, reuse of pilot observations in estimation, and repeated model updates generally prevent the conditional product law \([6\.3](https://arxiv.org/html/2611.24986#S6.E3)\) from holding\. Likewise, choosing the design using the features of the eventual estimation pool does not satisfy the fresh\-sample construction above\. Conditioning additionally on those features leads to a fixed\-feature experiment, not the iid population experiment analyzed here\. A separate conditional or adaptive sampling argument would be needed for that setting\.

Accordingly, Theorem[6\.1](https://arxiv.org/html/2611.24986#S6.Thmtheorem1)does not establish oracle attainment for the sequential finite\-pool experiments\. Those experiments assess a practical implementation of the criterion\. The pilot provides initial information about both the classifier and the response mechanism; a regular pilot design helps avoid beginning acquisition with an unidentified response model\.

## 7Binary logistic illustration

We now illustrate the preceding theory in a binary classification model for which the decision boundary, label information, response information, and classification\-risk curvature can all be written explicitly\.

Let

Z∈\{0,1\},Prθ⁡\(Z=1∣Y=y\)=τθ​\(y\)=exp⁡\(α\+β​y\)1\+exp⁡\(α\+β​y\),Z\\in\\\{0,1\\\},\\qquad\\Pr\_\{\\theta\}\(Z=1\\mid Y=y\)=\\tau\_\{\\theta\}\(y\)=\\frac\{\\exp\(\\alpha\+\\beta y\)\}\{1\+\\exp\(\\alpha\+\\beta y\)\},with

θ=\(α,β\)⊤,β\>0\.\\theta=\(\\alpha,\\beta\)^\{\\top\},\\qquad\\beta\>0\.For simplicity, suppose that the marginal distribution ofYYdoes not depend onθ\\theta, with densityff\. The Bayes classifier assigns class 1 whenever

α\+β​y\>0,\\alpha\+\\beta y\>0,so the decision boundary is

b⁡\(θ\)=−αβ\.b\(\\theta\)=\-\\frac\{\\alpha\}\{\\beta\}\.\(7\.1\)
Let

b0=−α0β0b\_\{0\}=\-\\frac\{\\alpha\_\{0\}\}\{\\beta\_\{0\}\}denote the true boundary\. Under the usual smoothness conditionf⁡\(b0\)\>0f\(b\_\{0\}\)\>0, the local excess risk satisfies

R⁡\(θ0\+h\)−R∗=f⁡\(b0\)4​β0​\(hα\+b0​hβ\)2\+o⁡\(‖h‖2\)\.R\(\\theta\_\{0\}\+h\)\-R^\{\*\}=\\frac\{f\(b\_\{0\}\)\}\{4\\beta\_\{0\}\}\\left\(h\_\{\\alpha\}\+b\_\{0\}h\_\{\\beta\}\\right\)^\{2\}\+o\(\\\|h\\\|^\{2\}\)\.\(7\.2\)Hence

HR=f⁡\(b0\)2​β0​v0​v0⊤,v0=\(1b0\)\.H\_\{R\}=\\frac\{f\(b\_\{0\}\)\}\{2\\beta\_\{0\}\}v\_\{0\}v\_\{0\}^\{\\top\},\\qquad v\_\{0\}=\\begin\{pmatrix\}1\\\\ b\_\{0\}\\end\{pmatrix\}\.\(7\.3\)Thus only parameter directions that perturb the decision boundary contribute to first\-order classification risk\.

Write

x⁡\(y\)=\(1y\)\.x\(y\)=\\begin\{pmatrix\}1\\\\ y\\end\{pmatrix\}\.The conditional class\-label information is

J⁡\(y\)=τ⁡\(y\)​\{1−τ⁡\(y\)\}​x​\(y\)​x​\(y\)⊤\.J\(y\)=\\tau\(y\)\\\{1\-\\tau\(y\)\\\}x\(y\)x\(y\)^\{\\top\}\.\(7\.4\)Because

τ⁡\(y\)​\{1−τ⁡\(y\)\}\\tau\(y\)\\\{1\-\\tau\(y\)\\\}is maximized atτ⁡\(y\)=1/2\\tau\(y\)=1/2, the scalar Bernoulli variance factor is largest at the Bayes boundary\. The full information matrix also depends onx⁡\(y\)​x​\(y\)⊤x\(y\)x\(y\)^\{\\top\}, so its magnitude and classification value need not be largest there\.

To model informative oracle response, consider

logit⁡r⁡\(y,θ,η\)=η0\+η1​U​\(y,θ\),\\operatorname\{logit\}r\(y;\\theta,\\eta\)=\\eta\_\{0\}\+\\eta\_\{1\}U\(y;\\theta\),\(7\.5\)where

U⁡\(y,θ\)=−τ⁡\(y\)​log⁡τ⁡\(y\)−\{1−τ⁡\(y\)\}​log​\{1−τ⁡\(y\)\}U\(y;\\theta\)=\-\\tau\(y\)\\log\\tau\(y\)\-\\\{1\-\\tau\(y\)\\\}\\log\\\{1\-\\tau\(y\)\\\}\(7\.6\)is the predictive entropy\. The caseη1<0\\eta\_\{1\}<0corresponds to an oracle that is less likely to respond at more uncertain observations\.

Since

∇θτ​\(y\)=τ⁡\(y\)​\{1−τ⁡\(y\)\}​x​\(y\)\\nabla\_\{\\theta\}\\tau\(y\)=\\tau\(y\)\\\{1\-\\tau\(y\)\\\}x\(y\)and

d​Ud​τ=log⁡1−ττ,\\frac\{dU\}\{d\\tau\}=\\log\\frac\{1\-\\tau\}\{\\tau\},we obtain

∇θU​\(y,θ\)=−\{α\+β​y\}​τ​\(y\)​\{1−τ⁡\(y\)\}​x​\(y\)\.\\nabla\_\{\\theta\}U\(y;\\theta\)=\-\\\{\\alpha\+\\beta y\\\}\\tau\(y\)\\\{1\-\\tau\(y\)\\\}x\(y\)\.\(7\.7\)Therefore

qθ​\(y\)=−η1​\{α\+β​y\}​τ​\(y\)​\{1−τ⁡\(y\)\}​x​\(y\),q\_\{\\theta\}\(y\)=\-\\eta\_\{1\}\\\{\\alpha\+\\beta y\\\}\\tau\(y\)\\\{1\-\\tau\(y\)\\\}x\(y\),\(7\.8\)whereas

qη​\(y\)=\(1U⁡\(y,θ\)\)\.q\_\{\\eta\}\(y\)=\\begin\{pmatrix\}1\\\\ U\(y;\\theta\)\\end\{pmatrix\}\.\(7\.9\)
The raw response\-information contribution atyyis consequently

KR​\(y\)\\displaystyle K\_\{R\}\(y\)=r⁡\(y\)​\{1−r⁡\(y\)\}​qθ​\(y\)​qθ​\(y\)⊤\\displaystyle=r\(y\)\\\{1\-r\(y\)\\\}q\_\{\\theta\}\(y\)q\_\{\\theta\}\(y\)^\{\\top\}=r⁡\(y\)​\{1−r⁡\(y\)\}​η12​\{α\+β​y\}2​τ​\(y\)2​\{1−τ⁡\(y\)\}2​x​\(y\)​x​\(y\)⊤\.\\displaystyle=r\(y\)\\\{1\-r\(y\)\\\}\\eta\_\{1\}^\{2\}\\\{\\alpha\+\\beta y\\\}^\{2\}\\tau\(y\)^\{2\}\\\{1\-\\tau\(y\)\\\}^\{2\}x\(y\)x\(y\)^\{\\top\}\.\(7\.10\)In contrast, the expected returned\-label contribution is

KL​\(y\)=r⁡\(y\)​τ​\(y\)​\{1−τ⁡\(y\)\}​x​\(y\)​x​\(y\)⊤\.K\_\{L\}\(y\)=r\(y\)\\tau\(y\)\\\{1\-\\tau\(y\)\\\}x\(y\)x\(y\)^\{\\top\}\.\(7\.11\)
These two sources of information have fundamentally different spatial geometry\. At the Bayes boundaryy=b⁡\(θ\)y=b\(\\theta\),

α\+β​b​\(θ\)=0,\\alpha\+\\beta b\(\\theta\)=0,and hence

qθ​\{b⁡\(θ\)\}=0,KR​\{b⁡\(θ\)\}=0\.q\_\{\\theta\}\\\{b\(\\theta\)\\\}=0,\\qquad K\_\{R\}\\\{b\(\\theta\)\\\}=0\.\(7\.12\)Thus, under the entropy\-linked response model, the raw classifier information in the response indicator vanishes exactly at the decision boundary, even though the scalar Bernoulli variance factor is maximal there\.

At the opposite extreme, as\|α\+β​y\|→∞\|\\alpha\+\\beta y\|\\rightarrow\\infty,

τ⁡\(y\)​\{1−τ⁡\(y\)\}→0,\\tau\(y\)\\\{1\-\\tau\(y\)\\\}\\rightarrow 0,so both the label and response information vanish\. Response information is therefore concentrated away from the exact boundary but not in the extreme tails\. It is naturally associated with intermediate off\-boundary regions\.

This distinction can change acquisition rankings\. A response\-discounted criterion considers only the expected value of the returned label, represented locally byψaL​\(y\)\\psi\_\{a\}^\{L\}\(y\)\. The full response\-aware criterion instead uses

ψa​\(y\)=ψaL​\(y\)\+ψaR​\(y\)\.\\psi\_\{a\}\(y\)=\\psi\_\{a\}^\{L\}\(y\)\+\\psi\_\{a\}^\{R\}\(y\)\.For two candidatesy1y\_\{1\}andy2y\_\{2\}, it is therefore possible that

ψaL​\(y1\)\>ψaL​\(y2\),\\psi\_\{a\}^\{L\}\(y\_\{1\}\)\>\\psi\_\{a\}^\{L\}\(y\_\{2\}\),while

ψa​\(y1\)<ψa​\(y2\)\.\\psi\_\{a\}\(y\_\{1\}\)<\\psi\_\{a\}\(y\_\{2\}\)\.The reversal occurs when the additional classification value carried by the response outcome aty2y\_\{2\}more than compensates for its smaller response\-discounted label value\.

The example also clarifies why informative response should not be interpreted as a preference for almost\-certain abstention\. The Bernoulli factor

r⁡\(y\)​\{1−r⁡\(y\)\}r\(y\)\\\{1\-r\(y\)\\\}vanishes whenr⁡\(y\)r\(y\)approaches either zero or one\. Response information is largest only where the response probability remains stochastic and its classifier\-related sensitivity is substantial\.

The numerical experiments in Section[8](https://arxiv.org/html/2611.24986#S8)will examine how this geometry translates into finite\-sample acquisition decisions and classification performance\.

## 8Simulation study

We investigated the finite\-sample behavior of the response\-aware acquisition criterion in the binary logistic model of Section[7](https://arxiv.org/html/2611.24986#S7)\. The simulation was designed to separate two questions\. First, we examined*oracle acquisition*, in which the true model parameters are used to evaluate candidate query scores\. This isolates the statistical effect of the acquisition criterion itself\. Second, we considered an*adaptive*procedure in which the classifier and response parameters are estimated from the accumulated query\-response data and the acquisition scores are updated sequentially\.

Throughout,

Y∼N\(0,1\),Z∣Y=y∼Bernoulli\{τ\(y\)\},Y\\sim N\(0,1\),\\qquad Z\\mid Y=y\\sim\\operatorname\{Bernoulli\}\\\{\\tau\(y\)\\\},\(8\.1\)with

τ⁡\(y\)=logit−1⁡\(−0\.6\+2​y\)\.\\tau\(y\)=\\operatorname\{logit\}^\{\-1\}\(\-0\.6\+2y\)\.The true Bayes boundary is therefore

The oracle\-response mechanism was

logit⁡r⁡\(y\)=η0\+η1​U​\(y\),\\operatorname\{logit\}r\(y\)=\\eta\_\{0\}\+\\eta\_\{1\}U\(y\),\(8\.2\)whereU⁡\(y\)U\(y\)is the binary predictive entropy defined in \([7\.6](https://arxiv.org/html/2611.24986#S7.E6)\)\. Three response regimes were considered:

η1∈\{0,−3,−5\},\\eta\_\{1\}\\in\\\{0,\-3,\-5\\\},representing noninformative, moderately informative, and strongly informative response, respectively\. For each value ofη1\\eta\_\{1\}, the interceptη0\\eta\_\{0\}was calibrated so that the marginal response probability satisfied

E​\{r⁡\(Y\)\}=0\.40\.E\\\{r\(Y\)\\\}=0\.40\.The resulting values were

η0=−0\.405,0\.920,1\.786,\\eta\_\{0\}=\-0\.405,\\quad 0\.920,\\quad 1\.786,with corresponding response probabilities at the Bayes boundary equal to0\.4000\.400,0\.2390\.239, and0\.1570\.157\. Thus the scenarios differ in the informativeness of the response mechanism rather than simply in the expected number of returned labels\.

For each Monte Carlo replication, an unlabeled pool of size30003000was generated\. Query budgets of

B∈\{300,600,900\}B\\in\\\{300,600,900\\\}were considered\. These budgets include the common random pilot of 100 queried observations\. Subsequent queries were acquired in batches of 25\. The same potential class labels and oracle\-response outcomes were used across methods within each replication, yielding paired comparisons\.

Six acquisition procedures were compared: random sampling, entropy sampling, ordinary classification\-risk\-optimal acquisition that treats labels as always available, response\-discounted risk\-optimal acquisition, the proposed response\-aware criterion, and a label\-only ablation with full information\.

The response\-aware criterion uses

ψa​\(y\)=ψaL​\(y\)\+ψaR​\(y\)\.\\psi\_\{a\}\(y\)=\\psi\_\{a\}^\{L\}\(y\)\+\\psi\_\{a\}^\{R\}\(y\)\.The ablation retains the full query–response information matrix, including response\-model nuisance parameters, but scores candidates using onlyψaL​\(y\)\\psi\_\{a\}^\{L\}\(y\)\. It therefore differs from response\-discounted acquisition, which uses a label\-based information matrix\.

All six procedures used the same full observed\-data likelihood after acquisition\. The ablation examines the effect of including the explicit response\-outcome term in sequential query selection while retaining full\-information calculations and joint estimation\. Once the query paths diverge, the procedures naturally have different accumulated data and fitted parameters\.

For adaptive acquisition, the parameters

\(α,β,η0,η1\)\(\\alpha,\\beta,\\eta\_\{0\},\\eta\_\{1\}\)were re\-estimated from the accumulated queried data using the likelihood

ℓ=∑i:Ai=1\[Rilogri\+\(1−Ri\)log\(1−ri\)\+Ri\{Zilogτi\+\(1−Zi\)log\(1−τi\)\}\]\.\\ell=\\sum\_\{i:A\_\{i\}=1\}\\left\[R\_\{i\}\\log r\_\{i\}\+\(1\-R\_\{i\}\)\\log\(1\-r\_\{i\}\)\+R\_\{i\}\\left\\\{Z\_\{i\}\\log\\tau\_\{i\}\+\(1\-Z\_\{i\}\)\\log\(1\-\\tau\_\{i\}\)\\right\\\}\\right\]\.The principal performance measure was excess classification risk relative to the true Bayes classifier\. We also recorded boundary\-estimation error, the number of returned labels, response rates, and the locations of queried observations\. Results below are based on 500 Monte Carlo replications\. All fits evaluated at the reported budgets returned optimizer convergence code zero\.

Likelihood optimization used box constraints

α∈\[−8,8\],β∈\[0\.05,8\],η0,η1∈\[−12,12\]\.\\alpha\\in\[\-8,8\],\\qquad\\beta\\in\[0\.05,8\],\\qquad\\eta\_\{0\},\\eta\_\{1\}\\in\[\-12,12\]\.At each batch, the accumulated information matrices are sums over all previously queried observations, including the pilot\. Ordinary risk sumsJ⁡\(yi\)J\(y\_\{i\}\), response\-discounted risk sumsr⁡\(yi\)​J​\(yi\)r\(y\_\{i\}\)J\(y\_\{i\}\), and the response\-aware and ablation rules both sumK⁡\(yi\)K\(y\_\{i\}\)\. These quantities are evaluated at the parameters used for scoring in that batch\. For acquisition calculations, each accumulated information matrixMMwas symmetrized and replaced by

M\+\[10−8\+max⁡\{0,10−8−λmin​\(M\)\}\]​IM\+\\left\[10^\{\-8\}\+\\max\\\{0,10^\{\-8\}\-\\lambda\_\{\\min\}\(M\)\\\}\\right\]Ibefore inversion\. Optimizer convergence codes describe numerical termination and do not establish global likelihood maximization\.

### 8\.1Acquisition geometry

Figure[1](https://arxiv.org/html/2611.24986#S8.F1)illustrates the population geometry in the strongly informative\-response regime\. The returned\-label and response components have markedly different spatial behavior\. The raw classifier\-related response\-information component vanishes at the true decision boundaryb0=0\.3b\_\{0\}=0\.3, as predicted by \([7\.12](https://arxiv.org/html/2611.24986#S7.E12)\)\. The nuisance\-adjusted marginal response value need not vanish there\. Numerically, the ordinary risk score is maximized neary=0\.57y=0\.57, whereas the response\-discounted and response\-aware scores are maximized neary=1\.30y=1\.30andy=1\.22y=1\.22, respectively\. The response component itself is maximized neary=1\.21y=1\.21\.

The figure therefore demonstrates that accounting for the response indicator does not merely rescale an uncertainty\-based acquisition score\. Instead, it changes the spatial allocation of queries by assigning value to observations whose response outcome is informative for classifier\-relevant parameter directions\.

Figure 1:Information and acquisition geometry in the strongly informative\-response setting\. The vertical dotted line marks the true Bayes boundaryb0=0\.3b\_\{0\}=0\.3\. Panel \(a\) shows the class and response probabilities\. Panel \(b\) shows the decision\-relevant information from returned labels and response outcomes\. Panel \(c\) shows the ordinary, response\-discounted, and response\-aware acquisition scores\. Panel \(d\) shows the response\-aware score and its two components\. The scores in panels \(c\) and \(d\) are evaluated at the constant reference designa⁡\(y\)≡0\.5a\(y\)\\equiv 0\.5\. Each curve in panels \(b\)–\(d\) is divided by its own maximum over the plotting grid, so curve heights cannot be used to compare magnitudes across curves\.
### 8\.2Classification performance

Table[1](https://arxiv.org/html/2611.24986#S8.T1)reports the mean excess classification risk for the adaptive procedures\. Under noninformative response, the response\-aware procedure provides no systematic advantage, as expected\. In the corresponding oracle experiment, the response\-aware and response\-discounted procedures coincide exactly for every replication and budget, providing a direct numerical check of the theoretical reduction

qθ​\(y\)=0⟹ψaR​\(y\)=0\.q\_\{\\theta\}\(y\)=0\\quad\\Longrightarrow\\quad\\psi\_\{a\}^\{R\}\(y\)=0\.
The picture changes once oracle response is informative\. Under moderate informative response, the response\-aware procedure has the smallest mean excess risk at all three budgets\. AtB=900B=900, for example, its mean excess risk is0\.359×10−30\.359\\times 10^\{\-3\}, compared with0\.463×10−30\.463\\times 10^\{\-3\}for the response\-discounted rule,0\.520×10−30\.520\\times 10^\{\-3\}for random acquisition,0\.609×10−30\.609\\times 10^\{\-3\}for entropy sampling, and0\.616×10−30\.616\\times 10^\{\-3\}for ordinary risk\-optimal acquisition\.

The same qualitative pattern appears under strongly informative response\. The proposed procedure attains mean excess risks

0\.578,0\.249,0\.1720\.578,\\quad 0\.249,\\quad 0\.172in units of10−310^\{\-3\}at budgets300300,600600, and900900, respectively\. The corresponding values for the response\-discounted method are

0\.656,0\.276,0\.176\.0\.656,\\quad 0\.276,\\quad 0\.176\.The gain relative to the response\-discounted rule is therefore most visible at smaller and intermediate budgets and decreases as the available query budget grows\.

The response\-aware rule does not dominate in every setting\. In the oracle study, it performs similarly to response\-discounted acquisition in several settings and has slightly higher mean risk in some\. The ablation below examines whether including the explicit response\-outcome term improves acquisition when full information and joint estimation are retained\.

Table 1:Mean excess classification risk multiplied by10310^\{3\}for adaptive acquisition over 500 replications, with Monte Carlo standard errors in parentheses\. The ablation retains full information but omits the explicit response\-outcome acquisition term\. The smallest mean in each row is in bold\.ResponseBBRandomEntropyOrdinaryDiscountedAblationResponse\-awareNoninformative3002\.807 \(0\.169\)1\.890 \(0\.125\)1\.816 \(0\.106\)2\.089 \(0\.125\)1\.918 \(0\.111\)2\.138 \(0\.157\)6001\.495 \(0\.105\)0\.827 \(0\.052\)0\.800 \(0\.049\)0\.778 \(0\.045\)0\.825 \(0\.050\)0\.896 \(0\.062\)9000\.977 \(0\.066\)0\.520 \(0\.034\)0\.537 \(0\.035\)0\.510 \(0\.032\)0\.519 \(0\.033\)0\.554 \(0\.035\)Moderate3001\.800 \(0\.114\)2\.218 \(0\.146\)1\.927 \(0\.132\)1\.612 \(0\.093\)1\.700 \(0\.115\)1\.479 \(0\.106\)6000\.763 \(0\.047\)1\.061 \(0\.065\)0\.990 \(0\.064\)0\.704 \(0\.043\)0\.711 \(0\.053\)0\.628 \(0\.045\)9000\.520 \(0\.032\)0\.609 \(0\.037\)0\.616 \(0\.038\)0\.463 \(0\.030\)0\.467 \(0\.033\)0\.359 \(0\.023\)Strong3000\.926 \(0\.061\)1\.966 \(0\.130\)1\.463 \(0\.096\)0\.656 \(0\.040\)0\.609 \(0\.039\)0\.578 \(0\.039\)6000\.457 \(0\.030\)1\.056 \(0\.068\)0\.837 \(0\.056\)0\.276 \(0\.017\)0\.275 \(0\.018\)0\.249 \(0\.016\)9000\.331 \(0\.024\)0\.634 \(0\.042\)0\.604 \(0\.038\)0\.176 \(0\.010\)0\.179 \(0\.011\)0\.172 \(0\.010\)
Figure[2](https://arxiv.org/html/2611.24986#S8.F2)gives the corresponding oracle and adaptive learning curves\. The figure includes the label\-only, full\-information ablation alongside the five original acquisition procedures\. The adaptive response\-aware rule also improves substantially over random acquisition, particularly under strong informative response\. AtB=900B=900, its excess risk is approximately48%48\\%lower than that of random acquisition in the strong\-response setting and approximately31%31\\%lower in the moderate\-response setting\.

Figure 2:Mean excess classification risk as a function of the query budget under oracle and adaptive acquisition\. Results are based on 500 Monte Carlo replications\. Shaded regions represent approximate 95% Monte Carlo intervals for the mean\.
### 8\.3Contribution of the response\-outcome term

The ablation retains the full information matrix and joint likelihood, but omitsψaR\\psi\_\{a\}^\{R\}from candidate scores\. Supplementary Table[S1](https://arxiv.org/html/2611.24986#S2.T1)reports paired excess\-risk differences, defined as the full criterion minus the ablation\.

Under moderately informative response andB=900B=900, the adaptive procedure reduced mean excess risk from0\.4666×10−30\.4666\\times 10^\{\-3\}to0\.3589×10−30\.3589\\times 10^\{\-3\}\. The paired difference was−0\.1077×10−3\-0\.1077\\times 10^\{\-3\}, with a pointwise 95% Monte Carlottinterval of\[−0\.1726,−0\.0428\]×10−3\[\-0\.1726,\-0\.0428\]\\times 10^\{\-3\}and Holm\-adjustedp=0\.0107p=0\.0107\. The oracle difference was−0\.1107×10−3\-0\.1107\\times 10^\{\-3\}, with interval\[−0\.1862,−0\.0352\]×10−3\[\-0\.1862,\-0\.0352\]\\times 10^\{\-3\}and adjustedp=0\.0373p=0\.0373\. Holm adjustments cover the nine comparisons within each study\.

Relative to the ablation’s mean excess risk, both reductions were approximately23%23\\%\. In absolute terms, the adaptive reduction was approximately0\.01080\.0108percentage points of classification error\. The full adaptive criterion had lower mean excess risk in all six informative\-response settings, although the other five intervals included zero\.

Under noninformative response, the oracle procedures had identical query paths in every saved trace and identical recorded outcomes across all 500 replications\. The adaptive procedures need not coincide: estimation error can give the fitted response model a nonzero classifier\-dependent component\.

The evidence for an additional benefit is therefore confined to the moderate\-response setting at the largest budget\. The unresolved comparisons do not establish equivalence, and the results do not support a general advantage from increasing response dependence\.

### 8\.4Where the procedures query

The risk differences are accompanied by substantial changes in the locations of the queried observations\. Figure[3](https://arxiv.org/html/2611.24986#S8.F3)shows the empirical distribution of queried feature values in the strongly informative\-response regime\. Entropy and ordinary\-risk acquisition place most of their queries close to the Bayes boundary, where classification uncertainty is largest\. In contrast, both response\-discounted and response\-aware acquisition shift substantial mass to off\-boundary regions\.

For the adaptive procedures atB=900B=900, the mean absolute distance from the true boundary was0\.260\.26,0\.270\.27,0\.830\.83, and0\.840\.84for entropy, ordinary\-risk, response\-discounted, and response\-aware acquisition, respectively\. Thus, unlike uncertainty\-based rules, the response\-aware design places substantial query mass away from the decision boundary, reflecting the joint value of returned labels and response information\. The distinction is also visible in the realized response rates\. Under strong informative response andB=900B=900, entropy and ordinary\-risk acquisition obtain responses for only about20%20\\%of queried observations, whereas the response\-discounted and response\-aware procedures obtain response rates of approximately40%40\\%\. The proposed acquisition rule therefore avoids spending most of its budget in regions where the oracle almost always abstains, while retaining the additional information carried by the response outcome itself\.

Figure 3:Empirical distribution of queried feature values in the strongly informative\-response setting\. The dotted vertical line marks the true Bayes boundary\. Entropy and ordinary\-risk sampling concentrate near the boundary, whereas response\-discounted and response\-aware acquisition place substantial query mass in informative off\-boundary regions\.

## 9Real\-data application

We evaluate the acquisition rules using the Consumer Complaints data from the AL\-RCTA collection of[Totakura et al\., \(2026\)](https://arxiv.org/html/2611.24986#bib.bib22), which contains crowd\-sourced annotations with explicit annotator abstentions\. The analysis uses 2,999 complaints presented to each of ten human annotators, with the response indicator defined by

Ri​j=𝕀\{annotatorjreturned a nonzero annotation for complainti\}\.R\_\{ij\}=\\mathbb\{I\}\\\{\\text\{annotator \}j\\text\{ returned a nonzero annotation for complaint \}i\\\}\.Empirical response rates varied substantially across annotators, from approximately0\.750\.75to0\.970\.97\. Response was also related to classification difficulty: in a preliminary out\-of\-fold analysis, the response rate decreased from0\.9720\.972in the lowest predictive\-entropy decile to0\.8270\.827in the highest\.

The theoretical model assumes that a returned label is correct, whereas the crowd annotations may themselves contain labeling errors\. We therefore retain the observed human response indicatorRi​jR\_\{ij\}, but, whenRi​j=1R\_\{ij\}=1, reveal the benchmark product category rather than the annotator’s reported class\. This preserves the genuine human abstention pattern while isolating informative response from label noise\. Each annotator is treated as a separate oracle\.

Complaint narratives were represented by TF–IDF features followed by a 12\-dimensional truncated singular\-value decomposition\. One of the 3,000 shared complaints was excluded because its narrative was missing\. TF–IDF, singular\-value decomposition, feature scaling, and narrative\-length scaling were constructed using all 2,999 narratives before generating the splits\. The analysis therefore uses a transductive feature representation: test narratives contribute to unsupervised preprocessing, but test labels are withheld from fitting and acquisition\. A multinomial logistic classifier was fitted jointly with

logit⁡\{rj​\(y,θ,ηj\)\}=η0​j\+η1​j​Uθ​\(y\)\+η2​j​L​\(y\),\\operatorname\{logit\}\\\{r\_\{j\}\(y;\\theta,\\eta\_\{j\}\)\\\}=\\eta\_\{0j\}\+\\eta\_\{1j\}U\_\{\\theta\}\(y\)\+\\eta\_\{2j\}L\(y\),whereUθ​\(y\)U\_\{\\theta\}\(y\)is normalized predictive entropy andL⁡\(y\)L\(y\)is standardized log narrative length\. We compared random, entropy, ordinary classification\-risk, response\-discounted risk, and response\-aware acquisition\. The classification task comprised six product categories\. Across 30 replications, all methods shared a stratified split with approximately 70% of observations allocated to training and 30% to testing, a common pilot sample of 300 training observations, and the same realized oracle responses\. Labels were subsequently queried in batches of 50 at additional budgetsB∈\{150,300,450,600\}B\\in\\\{150,300,450,600\\\}\. All methods used the same penalized joint likelihood, with quadratic penalty coefficients 1 for classifier slopes and 0\.5 for response\-model slopes; intercepts were unpenalized\. Risk\-based acquisition used an empirical kernel approximation to the active\-boundary curvature, weighted by the mean fitted probability of the two leading classes\. The Gaussian\-kernel bandwidth was the larger of 0\.20 and the empirical 25th percentile of their absolute log\-probability gaps\. The curvature matrix was trace\-normalized\. Information matrices used diagonal regularization of10−310^\{\-3\}\. Before inversion, each eigenvalue of the symmetrized matrix was replaced by the larger of its original value andmax⁡\{10−8​λmax,10−10\}\\max\\\{10^\{\-8\}\\lambda\_\{\\max\},10^\{\-10\}\\\}, whereλmax\\lambda\_\{\\max\}is the largest eigenvalue\. For ordinary\-risk and response\-discounted acquisition, the accumulated information matrix uses conditional classifier information from labels actually returned\. Response\-aware acquisition instead uses the fitted full query–response information matrix, including the response\-model nuisance parameters\. Thus, the response\-discounted baseline is not obtained simply by removingψaR\\psi\_\{a\}^\{R\}from the response\-aware score while retaining the same information matrix\. This experiment compares the complete acquisition rules rather than isolating the contribution of that additive term\. Each batch selected the 50 highest\-scoring available observations before refitting\. This sequential implementation illustrates the criterion; it is not a numerical verification of the two\-stage oracle\-attainment theorem\.

Table 2:Held\-out classification error in the Consumer Complaints application\. Entries are means over 30 replications after averaging over the ten human oracles; parentheses contain Monte Carlo standard errors\. The query budget excludes the common pilot sample of size 300\.Additional query budgetBBAcquisition rule150300450600Random0\.260​\(0\.0018\)0\.260\\;\(0\.0018\)0\.254​\(0\.0017\)0\.254\\;\(0\.0017\)0\.250​\(0\.0017\)0\.250\\;\(0\.0017\)0\.247​\(0\.0016\)0\.247\\;\(0\.0016\)Entropy0\.246​\(0\.0022\)\\mathbf\{0\.246\}\\;\(0\.0022\)0\.241​\(0\.0017\)\\mathbf\{0\.241\}\\;\(0\.0017\)0\.238​\(0\.0015\)0\.238\\;\(0\.0015\)0\.235​\(0\.0015\)0\.235\\;\(0\.0015\)Ordinary risk0\.258​\(0\.0019\)0\.258\\;\(0\.0019\)0\.244​\(0\.0015\)0\.244\\;\(0\.0015\)0\.237​\(0\.0016\)0\.237\\;\(0\.0016\)0\.232​\(0\.0016\)0\.232\\;\(0\.0016\)Response\-discounted0\.258​\(0\.0019\)0\.258\\;\(0\.0019\)0\.244​\(0\.0015\)0\.244\\;\(0\.0015\)0\.237​\(0\.0016\)0\.237\\;\(0\.0016\)0\.232​\(0\.0015\)0\.232\\;\(0\.0015\)Response\-aware0\.258​\(0\.0019\)0\.258\\;\(0\.0019\)0\.243​\(0\.0016\)0\.243\\;\(0\.0016\)0\.235​\(0\.0017\)\\mathbf\{0\.235\}\\;\(0\.0017\)0\.231​\(0\.0016\)\\mathbf\{0\.231\}\\;\(0\.0016\)Entropy sampling has the lowest mean test error at the two smallest budgets, whereas response\-aware acquisition has the lowest mean error atB=450B=450andB=600B=600\. The mean paired differences in test error \(response\-aware minus response\-discounted\) are

−0\.000052,−0\.000892,−0\.001152,−0\.001152\-0\.000052,\\quad\-0\.000892,\\quad\-0\.001152,\\quad\-0\.001152atB=150,300,450,600B=150,300,450,600, respectively\. Pointwise 95% Monte Carlottintervals exclude zero at the three larger budgets; atB=600B=600, the interval is\[−0\.001679,−0\.000624\]\[\-0\.001679,\-0\.000624\]\. After Holm correction across the 16 budget–comparator comparisons, the advantages over response\-discounted acquisition remain significant atB=450B=450andB=600B=600, with adjustedpp\-values0\.02020\.0202and0\.001110\.00111, respectively\. Response\-aware acquisition is worse than entropy sampling atB=150B=150; its advantage over entropy atB=600B=600does not remain significant under this correction\. These intervals quantify variability across repeated splits and acquisition experiments for the fixed dataset and ten annotators, rather than uncertainty over a population of datasets or annotators\.

![Refer to caption](https://arxiv.org/html/2611.24986v1/Fig4.png)Figure 4:Uncertainty–response acquisition geometry for the Consumer Complaints application using worker W8, who had the largest observed abstention fraction\. Panel \(a\) pools random\-query locations across 30 replications\. Panels \(b\) and \(c\) show response\-discounted and response\-aware acquisition in replication 6\. Coordinates are predictive entropy and response probability evaluated under each method’s fitted model at the time of querying\. Open circles trace batch\-wise mean query locations; the diamond marks the final batch at additional budgetB=600B=600\. Replication 6 was selected because its paired response\-aware minus response\-discounted test\-error difference was closest to the median across the 30 W8 replications\. The panels describe evolving fitted coordinates, rather than a common fixed\-model representation\.Figure[4](https://arxiv.org/html/2611.24986#S9.F4)shows a negative association between fitted predictive entropy and response probability in the pooled random\-query reference cloud\. The two risk\-based procedures follow distinct, nonmonotone trajectories in their evolving fitted coordinates\. Their first\-batch mean locations are similar\. In the final batch, response\-aware acquisition has mean entropy0\.7810\.781and fitted response probability0\.6680\.668, compared with0\.6910\.691and0\.6900\.690, respectively, for response\-discounted acquisition\. These trajectories illustrate differences in sequential querying and model updating; they do not by themselves identify which information component causes the performance difference\.

The aggregate improvement is heterogeneous across human oracles\. AtB=600B=600, the largest mean improvement over response\-discounted acquisition occurs for W4, with paired difference−0\.00483\-0\.00483and 95% Monte Carlottinterval\[−0\.00768,−0\.00198\]\[\-0\.00768,\-0\.00198\]\. Only W4 remains significant after Holm correction across the ten worker\-specific comparisons, with adjustedp=0\.0167p=0\.0167\. The other worker\-specific comparisons are statistically unresolved after this correction\.

## 10Discussion and conclusions

This paper extends classification\-risk\-optimal label acquisition to an oracle that may abstain and whose response probability shares parameters with the classifier\. The response outcome can then supply information even when no label is returned\. After adjustment for response\-model nuisance parameters, this contribution is

IR,eff​\(a\)=E⁡\[a⁡\(Y\)​w​\(Y\)​\{qθ​\(Y\)−Ba​qη​\(Y\)\}​\{qθ​\(Y\)−Ba​qη​\(Y\)\}⊤\]\.I\_\{R,\\mathrm\{eff\}\}\(a\)=E\\\!\\left\[a\(Y\)w\(Y\)\\\{q\_\{\\theta\}\(Y\)\-B\_\{a\}q\_\{\\eta\}\(Y\)\\\}\\\{q\_\{\\theta\}\(Y\)\-B\_\{a\}q\_\{\\eta\}\(Y\)\\\}^\{\\top\}\\right\]\.It is positive semidefinite and vanishes when the classifier\-related response score is fully explained by the nuisance response score under the design\-weighted measure\. Parameter dependence alone is therefore insufficient for response outcomes to improve classification\. The remaining information must also act in directions relevant to the decision boundary\.

Combining the information from returned labels and response outcomes with the local curvature of zero–one risk gives the marginal query value

ψa​\(y\)=ψaL​\(y\)\+ψaR​\(y\)\.\\psi\_\{a\}\(y\)=\\psi\_\{a\}^\{L\}\(y\)\+\\psi\_\{a\}^\{R\}\(y\)\.The associated oracle allocation retains the convex trace\-inverse structure of the underlying design problem and admits a threshold characterization under a fixed query budget\. Response identifiability places an additional restriction on the allocation: queries must provide enough variation to estimate the response mechanism\.

The simulations show how accounting for response changes query selection\. In the strongly informative\-response setting, response\-aware and response\-discounted acquisition both place substantial query mass away from the Bayes boundary, where response is more likely\. Entropy and ordinary classification\-risk acquisition remain more concentrated near the boundary\. Under noninformative response, the response\-aware and response\-discounted oracle procedures coincide\.

The ablation examines the additional effect of including the explicit response\-outcome term while retaining full information and joint estimation\. Under moderately informative response atB=900B=900, the full criterion reduced mean excess risk in both the oracle and adaptive studies, with significance after the stated within\-study Holm corrections\. The other comparisons were statistically unresolved\. The absolute improvement was small, and the results do not establish a general advantage across response mechanisms and budgets\. Stronger response dependence alone does not imply a larger acquisition gain\.

The Consumer Complaints replay provides evidence using observed human abstentions\. Response\-aware acquisition had the lowest mean test error at the two largest budgets, with improvements over response\-discounted acquisition remaining significant after Holm correction atB=450B=450andB=600B=600\. Entropy sampling performed better at the smallest budget\. The improvements were small and varied across annotators\. This comparison evaluates complete acquisition rules with different information matrices; it does not isolate the contribution of the explicit response\-outcome term\.

The theory is local and parametric and relies on correct specification of the classification and response models\. An association between response and predictive difficulty can motivate a shared\-parameter model, but does not establish that the assumed link is correct\. Performance under misspecification remains unresolved\. The two\-stage result applies to a pilot\-selected design held fixed on fresh second\-stage observations\. It relies on uniform criterion consistency, mean\-square asymptotic linearity, and a higher\-moment bound\. It does not establish these estimator conditions, or oracle attainment, for the sequential finite\-pool implementation\.

We assume that returned labels are correct and that

R⟂Z\|\(Y,A=1\)\.R\\perp Z\\mid\(Y,A=1\)\.The replay therefore preserves human response patterns while revealing benchmark labels upon response\. Using actual worker labels would require accounting for annotation error and, potentially, class\-dependent response\. Joint allocation across annotators with different costs, response mechanisms, and accuracies would require further development\.

The main contribution is a classification\-risk acquisition criterion that accounts for information in oracle response outcomes after nuisance adjustment\. The numerical studies support its usefulness in some settings, while showing that the gains depend on the response mechanism and query budget\.

## Statements and Declarations

### Code and data availability

Code and reproducibility instructions for the simulation studies, the Consumer Complaints analysis, and figure generation are available in the[Risk\-optimal active learning with informative abstention repository](https://github.com/fariborz-setoudeh/risk-optimal-active-learning-informative-abstention)\. The Consumer Complaints data are obtained from the original AL\-RCTA source cited in the manuscript\.

### CRediT authorship contribution statement

F\. Setoudehtazangi:Conceptualization, Methodology, Formal analysis, Software, Investigation, Visualization, Writing – original draft\.G\.J\. McLachlan:Methodology, Supervision, Writing – review & editing\.

### Funding

This research received no specific grant from funding agencies in the public, commercial, or not\-for\-profit sectors\.

### Competing interests

The authors declare no competing interests\.

### Ethical approval

This study involved secondary analysis of an existing dataset and did not involve the recruitment of human participants or collection of new human\-subject data by the authors\.

## References

- Ahfock and McLachlan, \(2020\)Ahfock, D\. and McLachlan, G\. J\. \(2020\)\.An apparent paradox: a classifier based on a partially classified sample may have smaller expected error rate than that if the sample were completely classified\.Statistics and Computing, 30:1779–1790\.[https://doi\.org/10\.1007/s11222\-020\-09971\-5](https://doi.org/10.1007/s11222-020-09971-5)\.
- Amin et al\., \(2021\)Amin, K\., DeSalvo, G\., and Rostamizadeh, A\. \(2021\)\.Learning with labeling induced abstentions\.InAdvances in Neural Information Processing Systems, volume 34, pages 12576–12586\. Curran Associates, Inc\.
- Bach, \(2006\)Bach, F\. R\. \(2006\)\.Active learning for misspecified generalized linear models\.InAdvances in Neural Information Processing Systems, volume 19, pages 65–72\. MIT Press\.
- Bickford Smith et al\., \(2023\)Bickford Smith, F\., Kirsch, A\., Farquhar, S\., Gal, Y\., Foster, A\., and Rainforth, T\. \(2023\)\.Prediction\-oriented Bayesian active learning\.InProceedings of the 26th International Conference on Artificial Intelligence and Statistics, volume 206 ofProceedings of Machine Learning Research, pages 7331–7348\. PMLR\.Available at:[https://proceedings\.mlr\.press/v206/bickfordsmith23a\.html](https://proceedings.mlr.press/v206/bickfordsmith23a.html)\.
- Chen and Biros, \(2023\)Chen, Y\. and Biros, G\. \(2023\)\.FIRAL: An active learning algorithm for multinomial logistic regression\.InAdvances in Neural Information Processing Systems, volume 36, pages 65894–65905\. Curran Associates, Inc\.[https://doi\.org/10\.52202/075280\-2877](https://doi.org/10.52202/075280-2877)\.
- Fang and Zhu, \(2014\)Fang, M\. and Zhu, X\. \(2014\)\.Active learning with uncertain labeling knowledge\.Pattern Recognition Letters, 43:98–108\.[https://doi\.org/10\.1016/j\.patrec\.2013\.10\.011](https://doi.org/10.1016/j.patrec.2013.10.011)\.
- Filstroff et al\., \(2024\)Filstroff, L\., Sundin, I\., Mikkola, P\., Tiulpin, A\., Kylmäoja, J\., and Kaski, S\. \(2024\)\.Targeted active learning for Bayesian decision\-making\.Transactions on Machine Learning Research\.Available at:[https://openreview\.net/forum?id=KxPjuiMgmm](https://openreview.net/forum?id=KxPjuiMgmm)\.
- Hoi et al\., \(2006\)Hoi, S\. C\. H\., Jin, R\., Zhu, J\., and Lyu, M\. R\. \(2006\)\.Batch mode active learning and its application to medical image classification\.InProceedings of the 23rd International Conference on Machine Learning, pages 417–424\. ACM\.[https://doi\.org/10\.1145/1143844\.1143897](https://doi.org/10.1145/1143844.1143897)\.
- Huang et al\., \(2026\)Huang, Z\., Bickford Smith, F\., and Rainforth, T\. \(2026\)\.Loss\-driven Bayesian active learning\.InProceedings of the 29th International Conference on Artificial Intelligence and Statistics, volume 300 ofProceedings of Machine Learning Research, pages 5140–5148\. PMLR\.Available at:[https://proceedings\.mlr\.press/v300/huang26b\.html](https://proceedings.mlr.press/v300/huang26b.html)\.
- Imberg et al\., \(2020\)Imberg, H\., Jonasson, J\., and Axelson\-Fisk, M\. \(2020\)\.Optimal sampling in unbiased active learning\.InProceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 ofProceedings of Machine Learning Research, pages 559–569\. PMLR\.Available at:[https://proceedings\.mlr\.press/v108/imberg20a\.html](https://proceedings.mlr.press/v108/imberg20a.html)\.
- Lewis and Catlett, \(1994\)Lewis, D\. D\. and Catlett, J\. \(1994\)\.Heterogeneous uncertainty sampling for supervised learning\.InMachine Learning Proceedings 1994, pages 148–156\. Morgan Kaufmann\.[https://doi\.org/10\.1016/B978\-1\-55860\-335\-6\.50026\-X](https://doi.org/10.1016/B978-1-55860-335-6.50026-X)\.
- Lewis and Gale, \(1994\)Lewis, D\. D\. and Gale, W\. A\. \(1994\)\.A sequential algorithm for training text classifiers\.InProceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3–12\. Springer\.[https://doi\.org/10\.1007/978\-1\-4471\-2099\-5\_1](https://doi.org/10.1007/978-1-4471-2099-5_1)\.
- Nguyen et al\., \(2022\)Nguyen, C\. V\., Ho, L\. S\. T\., Xu, H\., Dinh, V\., and Nguyen, B\. T\. \(2022\)\.Bayesian active learning with abstention feedbacks\.Neurocomputing, 471:242–250\.[https://doi\.org/10\.1016/j\.neucom\.2021\.11\.027](https://doi.org/10.1016/j.neucom.2021.11.027)\.
- Pukelsheim, \(2006\)Pukelsheim, F\. \(2006\)\.Optimal Design of Experiments\.Society for Industrial and Applied Mathematics, Philadelphia, PA\.[https://doi\.org/10\.1137/1\.9780898719109](https://doi.org/10.1137/1.9780898719109)\.
- Robinson et al\., \(2024\)Robinson, T\. S\., Tax, N\., Mudd, R\., and Guy, I\. \(2024\)\.Active learning with biased non\-response to label requests\.Data Mining and Knowledge Discovery, 38\(4\):2117–2140\.[https://doi\.org/10\.1007/s10618\-024\-01026\-x](https://doi.org/10.1007/s10618-024-01026-x)\.
- Roy and McCallum, \(2001\)Roy, N\. and McCallum, A\. \(2001\)\.Toward optimal active learning through Monte Carlo estimation of error reduction\.InProceedings of the Eighteenth International Conference on Machine Learning, pages 441–448\. Morgan Kaufmann\.
- Rubin, \(1976\)Rubin, D\. B\. \(1976\)\.Inference and missing data\.Biometrika, 63\(3\):581–592\.[https://doi\.org/10\.1093/biomet/63\.3\.581](https://doi.org/10.1093/biomet/63.3.581)\.
- \(18\)Setoudehtazangi, F\. and McLachlan, G\. J\. \(2026a\)\.Informative label missingness in multiclass classification: Information geometry and excess risk\.arXiv preprint arXiv:2608\.30561\.[https://doi\.org/10\.48550/arXiv\.2608\.30561](https://doi.org/10.48550/arXiv.2608.30561)\.
- \(19\)Setoudehtazangi, F\. and McLachlan, G\. J\. \(2026b\)\.Classification\-risk\-optimal label acquisition\.arXiv preprint arXiv:2609\.06873\.[https://doi\.org/10\.48550/arXiv\.2609\.06873](https://doi.org/10.48550/arXiv.2609.06873)\.
- Settles, \(2009\)Settles, B\. \(2009\)\.Active learning literature survey\.Computer Sciences Technical Report 1648, University of Wisconsin–Madison\.Available at:[https://research\.cs\.wisc\.edu/techreports/2009/TR1648\.pdf](https://research.cs.wisc.edu/techreports/2009/TR1648.pdf)\.
- Sourati et al\., \(2017\)Sourati, J\., Akcakaya, M\., Leen, T\. K\., Erdogmus, D\., and Dy, J\. G\. \(2017\)\.Asymptotic analysis of objectives based on Fisher information in active learning\.Journal of Machine Learning Research, 18\(34\):1–41\.Available at:[https://www\.jmlr\.org/papers/v18/15\-104\.html](https://www.jmlr.org/papers/v18/15-104.html)\.
- Totakura et al\., \(2026\)Totakura, V\., Singh, A\., Dong, Y\., and Chakraborty, S\. \(2026\)\.An analysis of active learning algorithms using real\-world crowd\-sourced text annotations\.InProceedings of the IEEE International Joint Conference on Neural Networks \(IJCNN 2026\)\. Accepted for publication\.Preprint available at:[https://arxiv\.org/abs/2604\.23290](https://arxiv.org/abs/2604.23290)\.
- Wang et al\., \(2018\)Wang, H\., Zhu, R\., and Ma, P\. \(2018\)\.Optimal subsampling for large sample logistic regression\.Journal of the American Statistical Association, 113\(522\):829–844\.[https://doi\.org/10\.1080/01621459\.2017\.1292914](https://doi.org/10.1080/01621459.2017.1292914)\.
- Wang et al\., \(2026\)Wang, Y\.\-G\., Wu, J\., and McLachlan, G\. J\. \(2026\)\.Learning from uncertainty\-dependent missing labels for semi\-supervised classification\.arXiv preprint arXiv:2608\.23960\.[https://doi\.org/10\.48550/arXiv\.2608\.23960](https://doi.org/10.48550/arXiv.2608.23960)\.
- Yan et al\., \(2016\)Yan, S\., Chaudhuri, K\., and Javidi, T\. \(2016\)\.Active learning from imperfect labelers\.InAdvances in Neural Information Processing Systems, volume 29, pages 2128–2136\. Curran Associates, Inc\.
- Yao and Wang, \(2019\)Yao, Y\. and Wang, H\. \(2019\)\.Optimal subsampling for softmax regression\.Statistical Papers, 60\(2\):585–599\.[https://doi\.org/10\.1007/s00362\-018\-01068\-6](https://doi.org/10.1007/s00362-018-01068-6)\.

Supplementary Material for Risk\-optimal active learning with informative abstention

## S1Proofs

### S1\.1Proof of Proposition[2\.1](https://arxiv.org/html/2611.24986#S2.Thmtheorem1)

For one observation, write

r⁡\(Y\)=r⁡\(Y,θ,η\)\.r\(Y\)=r\(Y;\\theta,\\eta\)\.From \([2\.6](https://arxiv.org/html/2611.24986#S2.E6)\)–\([2\.7](https://arxiv.org/html/2611.24986#S2.E7)\), the score forϕ=\(θ⊤,η⊤\)⊤\\phi=\(\\theta^\{\\top\},\\eta^\{\\top\}\)^\{\\top\}can be written as

Sϕ=\(SY0\)\+A⁡\[\(R−r\)​q\+R​S~Z\|Y\],S\_\{\\phi\}=\\begin\{pmatrix\}S\_\{Y\}\\\\ 0\\end\{pmatrix\}\+A\\left\[\(R\-r\)q\+R\\widetilde\{S\}\_\{Z\\mid Y\}\\right\],\(S1\.1\)where

q=\(qθqη\),S~Z\|Y=\(SZ\|Y0\)\.q=\\begin\{pmatrix\}q\_\{\\theta\}\\\\ q\_\{\\eta\}\\end\{pmatrix\},\\qquad\\widetilde\{S\}\_\{Z\\mid Y\}=\\begin\{pmatrix\}S\_\{Z\\mid Y\}\\\\ 0\\end\{pmatrix\}\.
Conditional onY=yY=y, the acquisition indicator satisfies

E⁡\(A∣Y=y\)=a⁡\(y\)\.E\(A\\mid Y=y\)=a\(y\)\.Moreover, by \([2\.2](https://arxiv.org/html/2611.24986#S2.E2)\),

E⁡\(R−r∣Y=y,A=1\)=0E\(R\-r\\mid Y=y,A=1\)=0and

E\{\(R−r\)2∣Y=y,A=1\}=r\(y\)\{1−r\(y\)\}=w\(y\)\.E\\\{\(R\-r\)^\{2\}\\mid Y=y,A=1\\\}=r\(y\)\\\{1\-r\(y\)\\\}=w\(y\)\.SinceSZ\|YS\_\{Z\\mid Y\}is a conditional likelihood score,

E⁡\(SZ\|Y∣Y=y\)=0\.E\(S\_\{Z\\mid Y\}\\mid Y=y\)=0\.Assumption \([2\.3](https://arxiv.org/html/2611.24986#S2.E3)\) further implies that, conditional onY=yY=yandA=1A=1, the response indicatorRRand the class labelZZare independent\. It follows that

E\[\(R−r\)RS~Z\|Y∣Y=y,A=1\]\\displaystyle E\\left\[\(R\-r\)R\\widetilde\{S\}\_\{Z\\mid Y\}\\mid Y=y,A=1\\right\]=E\{\(R−r\)R∣Y=y,A=1\}E\(S~Z\|Y∣Y=y\)=0\.\\displaystyle\\qquad=E\\\{\(R\-r\)R\\mid Y=y,A=1\\\}E\\left\(\\widetilde\{S\}\_\{Z\\mid Y\}\\mid Y=y\\right\)=0\.\(S1\.2\)Thus the score contribution from the response indicator is orthogonal, conditional onYY, to the score contribution from the returned class label\.

The conditional second moment of the returned\-label term is

E\[RS~Z\|YS~Z\|Y⊤∣Y=y,A=1\]\\displaystyle E\\left\[R\\widetilde\{S\}\_\{Z\\mid Y\}\\widetilde\{S\}\_\{Z\\mid Y\}^\{\\top\}\\mid Y=y,A=1\\right\]=r⁡\(y\)​E​\[S~Z\|Y​S~Z\|Y⊤∣Y=y\]\\displaystyle=r\(y\)E\\left\[\\widetilde\{S\}\_\{Z\\mid Y\}\\widetilde\{S\}\_\{Z\\mid Y\}^\{\\top\}\\mid Y=y\\right\]=r⁡\(y\)​J~​\(y\),\\displaystyle=r\(y\)\\widetilde\{J\}\(y\),\(S1\.3\)where

J~​\(y\)=\(J⁡\(y\)000\)\.\\widetilde\{J\}\(y\)=\\begin\{pmatrix\}J\(y\)&0\\\\ 0&0\\end\{pmatrix\}\.Similarly,

E\[\(R−r\)2qq⊤∣Y=y,A=1\]=w\(y\)q\(y\)q\(y\)⊤\.E\\left\[\(R\-r\)^\{2\}qq^\{\\top\}\\mid Y=y,A=1\\right\]=w\(y\)q\(y\)q\(y\)^\{\\top\}\.\(S1\.4\)Combining \([S1\.2](https://arxiv.org/html/2611.24986#S1.E2)\)–\([S1\.4](https://arxiv.org/html/2611.24986#S1.E4)\) gives the conditional Fisher information generated by a query:

K⁡\(y\)=r⁡\(y\)​J~​\(y\)\+w⁡\(y\)​q​\(y\)​q​\(y\)⊤\.K\(y\)=r\(y\)\\widetilde\{J\}\(y\)\+w\(y\)q\(y\)q\(y\)^\{\\top\}\.\(S1\.5\)
It remains to combine the query contribution with the information in the feature vector\. The feature score satisfies

Conditional onYY, the mean of the query\-score contribution in \([S1\.1](https://arxiv.org/html/2611.24986#S1.E1)\) is zero:

E\[\(R−r\)q\+RS~Z\|Y∣Y,A=1\]\\displaystyle E\\left\[\(R\-r\)q\+R\\widetilde\{S\}\_\{Z\\mid Y\}\\mid Y,A=1\\right\]=q​E​\(R−r∣Y,A=1\)\+E⁡\(R∣Y,A=1\)​E​\(S~Z\|Y∣Y\)=0\.\\displaystyle\\qquad=q\\,E\(R\-r\\mid Y,A=1\)\+E\(R\\mid Y,A=1\)E\(\\widetilde\{S\}\_\{Z\\mid Y\}\\mid Y\)=0\.Hence the feature score is orthogonal to the query\-score contribution\. SinceAAis generated according to the known parameter\-free design \([2\.1](https://arxiv.org/html/2611.24986#S2.E1)\),

E⁡\[A​K​\(Y\)∣Y\]=a⁡\(Y\)​K​\(Y\)\.E\\left\[AK\(Y\)\\mid Y\\right\]=a\(Y\)K\(Y\)\.Therefore

ℐ⁡\(a\)\\displaystyle\\mathcal\{I\}\(a\)=E⁡\(Sϕ​Sϕ⊤\)\\displaystyle=E\(S\_\{\\phi\}S\_\{\\phi\}^\{\\top\}\)=\(IY​\(θ\)000\)\+Eθ​\{a⁡\(Y\)​K​\(Y\)\}\\displaystyle=\\begin\{pmatrix\}I\_\{Y\}\(\\theta\)&0\\\\ 0&0\\end\{pmatrix\}\+E\_\{\\theta\}\\\{a\(Y\)K\(Y\)\\\}=ℐY\+Eθ​\{a⁡\(Y\)​K​\(Y\)\},\\displaystyle=\\mathcal\{I\}\_\{Y\}\+E\_\{\\theta\}\\\{a\(Y\)K\(Y\)\\\},which is \([2\.9](https://arxiv.org/html/2611.24986#S2.E9)\)\. Substitution of \([S1\.5](https://arxiv.org/html/2611.24986#S1.E5)\) yields \([2\.10](https://arxiv.org/html/2611.24986#S2.E10)\), completing the proof\.□\\square

### S1\.2Proof of Proposition[3\.1](https://arxiv.org/html/2611.24986#S3.Thmtheorem1)

For notational simplicity, write

a=a⁡\(Y\),w=w⁡\(Y\),qθ=qθ​\(Y\),qη=qη​\(Y\)\.a=a\(Y\),\\qquad w=w\(Y\),\\qquad q\_\{\\theta\}=q\_\{\\theta\}\(Y\),\\qquad q\_\{\\eta\}=q\_\{\\eta\}\(Y\)\.By \([3\.9](https://arxiv.org/html/2611.24986#S3.E9)\),

IR,eff​\(a\)=Cθ​θ−Cθ​η​Cη​η−1​Cη​θ\.I\_\{R,\\mathrm\{eff\}\}\(a\)=C\_\{\\theta\\theta\}\-C\_\{\\theta\\eta\}C\_\{\\eta\\eta\}^\{\-1\}C\_\{\\eta\\theta\}\.Let

Ba=Cθ​η​Cη​η−1\.B\_\{a\}=C\_\{\\theta\\eta\}C\_\{\\eta\\eta\}^\{\-1\}\.SinceCη​ηC\_\{\\eta\\eta\}is symmetric and positive definite,

Ba​Cη​η=Cθ​η,Cη​η​Ba⊤=Cη​θ\.B\_\{a\}C\_\{\\eta\\eta\}=C\_\{\\theta\\eta\},\\qquad C\_\{\\eta\\eta\}B\_\{a\}^\{\\top\}=C\_\{\\eta\\theta\}\.Consider

Eθ​\[a​w​\(qθ−Ba​qη\)​\(qθ−Ba​qη\)⊤\]\.E\_\{\\theta\}\\\!\\left\[aw\(q\_\{\\theta\}\-B\_\{a\}q\_\{\\eta\}\)\(q\_\{\\theta\}\-B\_\{a\}q\_\{\\eta\}\)^\{\\top\}\\right\]\.Expanding the outer product and using \([3\.4](https://arxiv.org/html/2611.24986#S3.E4)\)–\([3\.6](https://arxiv.org/html/2611.24986#S3.E6)\) gives

Eθ​\[a​w​\(qθ−Ba​qη\)​\(qθ−Ba​qη\)⊤\]\\displaystyle E\_\{\\theta\}\\\!\\left\[aw\(q\_\{\\theta\}\-B\_\{a\}q\_\{\\eta\}\)\(q\_\{\\theta\}\-B\_\{a\}q\_\{\\eta\}\)^\{\\top\}\\right\]=Cθ​θ−Ba​Cη​θ−Cθ​η​Ba⊤\+Ba​Cη​η​Ba⊤\.\\displaystyle\\quad=C\_\{\\theta\\theta\}\-B\_\{a\}C\_\{\\eta\\theta\}\-C\_\{\\theta\\eta\}B\_\{a\}^\{\\top\}\+B\_\{a\}C\_\{\\eta\\eta\}B\_\{a\}^\{\\top\}\.Because

Ba​Cη​θ=Cθ​η​Cη​η−1​Cη​θ,B\_\{a\}C\_\{\\eta\\theta\}=C\_\{\\theta\\eta\}C\_\{\\eta\\eta\}^\{\-1\}C\_\{\\eta\\theta\},and

Ba​Cη​η​Ba⊤=Cθ​η​Cη​η−1​Cη​θ,B\_\{a\}C\_\{\\eta\\eta\}B\_\{a\}^\{\\top\}=C\_\{\\theta\\eta\}C\_\{\\eta\\eta\}^\{\-1\}C\_\{\\eta\\theta\},the preceding expression reduces to

Cθ​θ−Cθ​η​Cη​η−1​Cη​θ=IR,eff​\(a\),C\_\{\\theta\\theta\}\-C\_\{\\theta\\eta\}C\_\{\\eta\\eta\}^\{\-1\}C\_\{\\eta\\theta\}=I\_\{R,\\mathrm\{eff\}\}\(a\),which proves \([3\.11](https://arxiv.org/html/2611.24986#S3.E11)\)\.

For anyv∈ℝdv\\in\\mathbb\{R\}^\{d\},

v⊤​IR,eff​\(a\)​v\\displaystyle v^\{\\top\}I\_\{R,\\mathrm\{eff\}\}\(a\)v=Eθ​\[a​w​\{v⊤​\(qθ−Ba​qη\)\}2\]≥0\.\\displaystyle=E\_\{\\theta\}\\\!\\left\[aw\\left\\\{v^\{\\top\}\(q\_\{\\theta\}\-B\_\{a\}q\_\{\\eta\}\)\\right\\\}^\{2\}\\right\]\\geq 0\.Hence

IR,eff​\(a\)⪰0\.I\_\{R,\\mathrm\{eff\}\}\(a\)\\succeq 0\.
It remains to characterize when the matrix is zero\. If

qθ​\(Y\)=Ba​qη​\(Y\)q\_\{\\theta\}\(Y\)=B\_\{a\}q\_\{\\eta\}\(Y\)μa\\mu\_\{a\}\-almost surely, then \([3\.11](https://arxiv.org/html/2611.24986#S3.E11)\) immediately givesIR,eff​\(a\)=0I\_\{R,\\mathrm\{eff\}\}\(a\)=0\.

Conversely, suppose thatIR,eff​\(a\)=0I\_\{R,\\mathrm\{eff\}\}\(a\)=0\. For everyv∈ℝdv\\in\\mathbb\{R\}^\{d\},

Eθ​\[a⁡\(Y\)​w​\(Y\)​\{v⊤​\[qθ​\(Y\)−Ba​qη​\(Y\)\]\}2\]=0\.E\_\{\\theta\}\\\!\\left\[a\(Y\)w\(Y\)\\left\\\{v^\{\\top\}\[q\_\{\\theta\}\(Y\)\-B\_\{a\}q\_\{\\eta\}\(Y\)\]\\right\\\}^\{2\}\\right\]=0\.The integrand is nonnegative, and therefore

v⊤​\[qθ​\(Y\)−Ba​qη​\(Y\)\]=0v^\{\\top\}\[q\_\{\\theta\}\(Y\)\-B\_\{a\}q\_\{\\eta\}\(Y\)\]=0μa\\mu\_\{a\}\-almost surely for everyv∈ℝdv\\in\\mathbb\{R\}^\{d\}\. Takingvvsuccessively to be the elements of the canonical basis ofℝd\\mathbb\{R\}^\{d\}yields

qθ​\(Y\)−Ba​qη​\(Y\)=0q\_\{\\theta\}\(Y\)\-B\_\{a\}q\_\{\\eta\}\(Y\)=0μa\\mu\_\{a\}\-almost surely\. This establishes \([3\.12](https://arxiv.org/html/2611.24986#S3.E12)\) and completes the proof\.□\\square

### S1\.3Proof of Proposition[4\.1](https://arxiv.org/html/2611.24986#S4.Thmtheorem1)

Let

at​\(y\)=a⁡\(y\)\+t​h​\(y\)a\_\{t\}\(y\)=a\(y\)\+th\(y\)forttin a neighborhood of zero such thatata\_\{t\}remains admissible\. By Proposition[2\.1](https://arxiv.org/html/2611.24986#S2.Thmtheorem1),

ℐ⁡\(at\)=ℐY\+Eθ0​\{at​\(Y\)​K​\(Y\)\},\\mathcal\{I\}\(a\_\{t\}\)=\\mathcal\{I\}\_\{Y\}\+E\_\{\\theta\_\{0\}\}\\\{a\_\{t\}\(Y\)K\(Y\)\\\},and hence

dd​t​ℐ​\(at\)\|t=0=Eθ0​\{h⁡\(Y\)​K​\(Y\)\}\.\\left\.\\frac\{d\}\{dt\}\\mathcal\{I\}\(a\_\{t\}\)\\right\|\_\{t=0\}=E\_\{\\theta\_\{0\}\}\\\{h\(Y\)K\(Y\)\\\}\.\(S1\.6\)
For any differentiable nonsingular matrix\-valued functionM⁡\(t\)M\(t\),

dd​t​M​\(t\)−1=−M​\(t\)−1​d​M​\(t\)d​t​M​\(t\)−1\.\\frac\{d\}\{dt\}M\(t\)^\{\-1\}=\-M\(t\)^\{\-1\}\\frac\{dM\(t\)\}\{dt\}M\(t\)^\{\-1\}\.Applying this identity toM⁡\(t\)=ℐ⁡\(at\)M\(t\)=\\mathcal\{I\}\(a\_\{t\}\)and using \([S1\.6](https://arxiv.org/html/2611.24986#S1.E6)\) gives

dd​t​Φ​\(at\)\|t=0\\displaystyle\\left\.\\frac\{d\}\{dt\}\\Phi\(a\_\{t\}\)\\right\|\_\{t=0\}=dd​t​tr⁡\{H~R​ℐ​\(at\)−1\}\|t=0\\displaystyle=\\left\.\\frac\{d\}\{dt\}\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\_\{t\}\)^\{\-1\}\\right\\\}\\right\|\_\{t=0\}=−tr⁡\[H~R​ℐ​\(a\)−1​Eθ0​\{h⁡\(Y\)​K​\(Y\)\}​ℐ​\(a\)−1\]\.\\displaystyle=\-\\operatorname\{tr\}\\left\[\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}E\_\{\\theta\_\{0\}\}\\\{h\(Y\)K\(Y\)\\\}\\mathcal\{I\}\(a\)^\{\-1\}\\right\]\.By linearity of expectation and trace,

dd​t​Φ​\(at\)\|t=0\\displaystyle\\left\.\\frac\{d\}\{dt\}\\Phi\(a\_\{t\}\)\\right\|\_\{t=0\}=−Eθ0​\[h⁡\(Y\)​tr⁡\{H~R​ℐ​\(a\)−1​K​\(Y\)​ℐ​\(a\)−1\}\]\\displaystyle=\-E\_\{\\theta\_\{0\}\}\\left\[h\(Y\)\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}K\(Y\)\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\}\\right\]=−Eθ0​\{h⁡\(Y\)​ψa​\(Y\)\},\\displaystyle=\-E\_\{\\theta\_\{0\}\}\\\{h\(Y\)\\psi\_\{a\}\(Y\)\\\},which establishes \([4\.12](https://arxiv.org/html/2611.24986#S4.E12)\)\.

It remains to obtain the decomposition ofψa​\(y\)\\psi\_\{a\}\(y\)\. From \([2\.8](https://arxiv.org/html/2611.24986#S2.E8)\),

K⁡\(y\)=r⁡\(y\)​J~​\(y\)\+w⁡\(y\)​q​\(y\)​q​\(y\)⊤\.K\(y\)=r\(y\)\\widetilde\{J\}\(y\)\+w\(y\)q\(y\)q\(y\)^\{\\top\}\.Substituting this expression into \([4\.8](https://arxiv.org/html/2611.24986#S4.E8)\) and using linearity of the trace yields

ψa​\(y\)\\displaystyle\\psi\_\{a\}\(y\)=r⁡\(y\)​tr⁡\{H~R​ℐ​\(a\)−1​J~​\(y\)​ℐ​\(a\)−1\}\\displaystyle=r\(y\)\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}\\widetilde\{J\}\(y\)\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\}\+w⁡\(y\)​tr⁡\{H~R​ℐ​\(a\)−1​q​\(y\)​q​\(y\)⊤​ℐ​\(a\)−1\}\.\\displaystyle\\quad\+w\(y\)\\operatorname\{tr\}\\left\\\{\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}q\(y\)q\(y\)^\{\\top\}\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\}\.Using

tr⁡\{M​q​q⊤​N\}=q⊤​N​M​q\\operatorname\{tr\}\\\{Mqq^\{\\top\}N\\\}=q^\{\\top\}NMqwith the symmetry ofℐ​\(a\)−1\\mathcal\{I\}\(a\)^\{\-1\}andH~R\\widetilde\{H\}\_\{R\}, the second term becomes

w⁡\(y\)​q​\(y\)⊤​ℐ​\(a\)−1​H~R​ℐ​\(a\)−1​q​\(y\)\.w\(y\)q\(y\)^\{\\top\}\\mathcal\{I\}\(a\)^\{\-1\}\\widetilde\{H\}\_\{R\}\\mathcal\{I\}\(a\)^\{\-1\}q\(y\)\.Therefore

ψa​\(y\)=ψaL​\(y\)\+ψaR​\(y\),\\psi\_\{a\}\(y\)=\\psi\_\{a\}^\{L\}\(y\)\+\\psi\_\{a\}^\{R\}\(y\),withψaL\\psi\_\{a\}^\{L\}andψaR\\psi\_\{a\}^\{R\}given by \([4\.10](https://arxiv.org/html/2611.24986#S4.E10)\) and \([4\.11](https://arxiv.org/html/2611.24986#S4.E11)\), respectively\.

Finally, sincew⁡\(y\)≥0w\(y\)\\geq 0,H~R⪰0\\widetilde\{H\}\_\{R\}\\succeq 0, andℐ​\(a\)−1\\mathcal\{I\}\(a\)^\{\-1\}is symmetric,

ψaR​\(y\)=w⁡\(y\)​‖H~R1/2​ℐ​\(a\)−1​q​\(y\)‖2≥0\.\\psi\_\{a\}^\{R\}\(y\)=w\(y\)\\left\\\|\\widetilde\{H\}\_\{R\}^\{1/2\}\\mathcal\{I\}\(a\)^\{\-1\}q\(y\)\\right\\\|^\{2\}\\geq 0\.This completes the proof\.□\\square

### S1\.4Proof of Proposition[5\.1](https://arxiv.org/html/2611.24986#S5.Thmtheorem1)

Leta0a\_\{0\}anda1a\_\{1\}be two designs for which the corresponding information matrices are positive definite, and let

at=\(1−t\)​a0\+t​a1,0≤t≤1\.a\_\{t\}=\(1\-t\)a\_\{0\}\+ta\_\{1\},\\qquad 0\\leq t\\leq 1\.By Proposition[2\.1](https://arxiv.org/html/2611.24986#S2.Thmtheorem1),

ℐ⁡\(at\)=\(1−t\)​ℐ​\(a0\)\+t​ℐ​\(a1\)\.\\mathcal\{I\}\(a\_\{t\}\)=\(1\-t\)\\mathcal\{I\}\(a\_\{0\}\)\+t\\mathcal\{I\}\(a\_\{1\}\)\.Set

Mt=ℐ⁡\(at\),D=ℐ⁡\(a1\)−ℐ⁡\(a0\)\.M\_\{t\}=\\mathcal\{I\}\(a\_\{t\}\),\\qquad D=\\mathcal\{I\}\(a\_\{1\}\)\-\\mathcal\{I\}\(a\_\{0\}\)\.Then

d​Mtd​t=D\.\\frac\{dM\_\{t\}\}\{dt\}=D\.Using

dd​t​Mt−1=−Mt−1​D​Mt−1,\\frac\{d\}\{dt\}M\_\{t\}^\{\-1\}=\-M\_\{t\}^\{\-1\}DM\_\{t\}^\{\-1\},we obtain

dd​t​Φ​\(at\)=−tr⁡\(H~R​Mt−1​D​Mt−1\)\.\\frac\{d\}\{dt\}\\Phi\(a\_\{t\}\)=\-\\operatorname\{tr\}\\left\(\\widetilde\{H\}\_\{R\}M\_\{t\}^\{\-1\}DM\_\{t\}^\{\-1\}\\right\)\.Differentiating once more gives

d2d​t2​Φ​\(at\)\\displaystyle\\frac\{d^\{2\}\}\{dt^\{2\}\}\\Phi\(a\_\{t\}\)=2​tr⁡\(H~R​Mt−1​D​Mt−1​D​Mt−1\)\.\\displaystyle=2\\operatorname\{tr\}\\left\(\\widetilde\{H\}\_\{R\}M\_\{t\}^\{\-1\}DM\_\{t\}^\{\-1\}DM\_\{t\}^\{\-1\}\\right\)\.Let

G=Mt−1/2DMt−1/2,C=Mt−1/2H~RMt−1/2\.G=M\_\{t\}^\{\-1/2\}DM\_\{t\}^\{\-1/2\},\\qquad C=M\_\{t\}^\{\-1/2\}\\widetilde\{H\}\_\{R\}M\_\{t\}^\{\-1/2\}\.SinceH~R⪰0\\widetilde\{H\}\_\{R\}\\succeq 0, we haveC⪰0C\\succeq 0, whileGGis symmetric\. Cyclic invariance of the trace yields

d2d​t2​Φ​\(at\)=2​tr⁡\(C​G2\)\.\\frac\{d^\{2\}\}\{dt^\{2\}\}\\Phi\(a\_\{t\}\)=2\\operatorname\{tr\}\(CG^\{2\}\)\.BecauseG2⪰0G^\{2\}\\succeq 0andC⪰0C\\succeq 0,

tr⁡\(C​G2\)=tr⁡\(C1/2​G2​C1/2\)≥0\.\\operatorname\{tr\}\(CG^\{2\}\)=\\operatorname\{tr\}\(C^\{1/2\}G^\{2\}C^\{1/2\}\)\\geq 0\.Hence

d2d​t2​Φ​\(at\)≥0,\\frac\{d^\{2\}\}\{dt^\{2\}\}\\Phi\(a\_\{t\}\)\\geq 0,which proves convexity along every line segment in the positive\-definite information domain\. ThereforeΦ\\Phiis convex on any convex subset of that domain\.□\\square

### S1\.5Proof of Theorem[5\.2](https://arxiv.org/html/2611.24986#S5.Thmtheorem2)

Leta∗a^\{\*\}be optimal and writeψ=ψa∗\\psi=\\psi\_\{a^\{\*\}\}\. All probabilities and expectations in this proof are taken underPθ0P\_\{\\theta\_\{0\}\}\. Forϵ\>0\\epsilon\>0, choose measurable setsB0⊆\{a∗≤1−ϵ\}B\_\{0\}\\subseteq\\\{a^\{\*\}\\leq 1\-\\epsilon\\\}andB1⊆\{a∗≥ϵ\}B\_\{1\}\\subseteq\\\{a^\{\*\}\\geq\\epsilon\\\}of positive probability, and put

h⁡\(y\)=𝟏B0​\(y\)Pθ0​\(B0\)−𝟏B1​\(y\)Pθ0​\(B1\)\.h\(y\)=\\frac\{\\mathbf\{1\}\_\{B\_\{0\}\}\(y\)\}\{P\_\{\\theta\_\{0\}\}\(B\_\{0\}\)\}\-\\frac\{\\mathbf\{1\}\_\{B\_\{1\}\}\(y\)\}\{P\_\{\\theta\_\{0\}\}\(B\_\{1\}\)\}\.SinceEθ0​\{h⁡\(Y\)\}=0E\_\{\\theta\_\{0\}\}\\\{h\(Y\)\\\}=0, this exchange preserves the budget\. The slack onB0B\_\{0\}andB1B\_\{1\}ensures thata∗\+t​ha^\{\*\}\+thremains in\[0,1\]\[0,1\]for sufficiently smallt\>0t\>0\. Its information matrix remains positive definite by continuity\. The right derivative, given by the same differentiation as in Proposition[4\.1](https://arxiv.org/html/2611.24986#S4.Thmtheorem1), therefore satisfies

0≤dd​t​Φ​\(a∗\+t​h\)\|t=0\+=Eθ0​\{ψ⁡\(Y\)∣Y∈B1\}−Eθ0​\{ψ⁡\(Y\)∣Y∈B0\}\.0\\leq\\left\.\\frac\{d\}\{dt\}\\Phi\(a^\{\*\}\+th\)\\right\|\_\{t=0\+\}=E\_\{\\theta\_\{0\}\}\\\{\\psi\(Y\)\\mid Y\\in B\_\{1\}\\\}\-E\_\{\\theta\_\{0\}\}\\\{\\psi\(Y\)\\mid Y\\in B\_\{0\}\\\}\.It follows that

ess​sup\{a∗<1\}ψ≤ess​inf\{a∗\>0\}ψ\.\\operatorname\*\{ess\\,sup\}\_\{\\\{a^\{\*\}<1\\\}\}\\psi\\leq\\operatorname\*\{ess\\,inf\}\_\{\\\{a^\{\*\}\>0\\\}\}\\psi\.Indeed, if this inequality failed, there would be positive\-probability subsets of\{a∗<1\}\\\{a^\{\*\}<1\\\}and\{a∗\>0\}\\\{a^\{\*\}\>0\\\}on which the scores were separated in the opposite order\. Intersecting these subsets with\{a∗≤1−ϵ\}\\\{a^\{\*\}\\leq 1\-\\epsilon\\\}and\{a∗≥ϵ\}\\\{a^\{\*\}\\geq\\epsilon\\\}, respectively, preserves positive probability for sufficiently smallϵ\\epsilon, contradicting the exchange inequality\.

Both sets have positive probability because0<ρ<10<\\rho<1\. Since the score is nonnegative and integrable, the displayed interval contains a finiteλ\\lambda\. For such aλ\\lambda,ψ≤λ\\psi\\leq\\lambdaalmost surely on\{a∗<1\}\\\{a^\{\*\}<1\\\}, andψ≥λ\\psi\\geq\\lambdaalmost surely on\{a∗\>0\}\\\{a^\{\*\}\>0\\\}\. These inequalities give the stated threshold conditions, including equality wherever0<a∗<10<a^\{\*\}<1\.

For sufficiency, letaabe any feasible design and interpretD​Φ​\(a∗\)​\[a−a∗\]D\\Phi\(a^\{\*\}\)\[a\-a^\{\*\}\]as the right derivative along the feasible segment froma∗a^\{\*\}toaa\. By convexity,

Φ⁡\(a\)≥Φ⁡\(a∗\)\+D​Φ​\(a∗\)​\[a−a∗\]\.\\Phi\(a\)\\geq\\Phi\(a^\{\*\}\)\+D\\Phi\(a^\{\*\}\)\[a\-a^\{\*\}\]\.The derivative term is

D​Φ​\(a∗\)​\[a−a∗\]=−Eθ0​\[\{a⁡\(Y\)−a∗​\(Y\)\}​ψa∗​\(Y\)\]\.D\\Phi\(a^\{\*\}\)\[a\-a^\{\*\}\]=\-E\_\{\\theta\_\{0\}\}\\left\[\\\{a\(Y\)\-a^\{\*\}\(Y\)\\\}\\psi\_\{a^\{\*\}\}\(Y\)\\right\]\.Since both designs satisfy the same budget,

Eθ0​\{a⁡\(Y\)−a∗​\(Y\)\}=0,E\_\{\\theta\_\{0\}\}\\\{a\(Y\)\-a^\{\*\}\(Y\)\\\}=0,and therefore

D​Φ​\(a∗\)​\[a−a∗\]=Eθ0​\[\{a⁡\(Y\)−a∗​\(Y\)\}​\{λ−ψa∗​\(Y\)\}\]\.D\\Phi\(a^\{\*\}\)\[a\-a^\{\*\}\]=E\_\{\\theta\_\{0\}\}\\left\[\\\{a\(Y\)\-a^\{\*\}\(Y\)\\\}\\\{\\lambda\-\\psi\_\{a^\{\*\}\}\(Y\)\\\}\\right\]\.Ifa∗​\(y\)=0a^\{\*\}\(y\)=0, thena⁡\(y\)−a∗​\(y\)≥0a\(y\)\-a^\{\*\}\(y\)\\geq 0andλ−ψa∗​\(y\)≥0\\lambda\-\\psi\_\{a^\{\*\}\}\(y\)\\geq 0\. Ifa∗​\(y\)=1a^\{\*\}\(y\)=1, thena⁡\(y\)−a∗​\(y\)≤0a\(y\)\-a^\{\*\}\(y\)\\leq 0andλ−ψa∗​\(y\)≤0\\lambda\-\\psi\_\{a^\{\*\}\}\(y\)\\leq 0\. If0<a∗​\(y\)<10<a^\{\*\}\(y\)<1, thenλ−ψa∗​\(y\)=0\\lambda\-\\psi\_\{a^\{\*\}\}\(y\)=0\. Thus the integrand is nonnegative almost surely, so

D​Φ​\(a∗\)​\[a−a∗\]≥0\.D\\Phi\(a^\{\*\}\)\[a\-a^\{\*\}\]\\geq 0\.Consequently,

Φ⁡\(a\)≥Φ⁡\(a∗\)\\Phi\(a\)\\geq\\Phi\(a^\{\*\}\)for every feasibleaa, proving global optimality\.□\\square

### S1\.6Proof of Proposition[5\.3](https://arxiv.org/html/2611.24986#S5.Thmtheorem3)

From \([5\.3](https://arxiv.org/html/2611.24986#S5.E3)\),

Cη​η​\(a\)=Eθ0​\[a⁡\(Y\)​w​\(Y\)​qη​\(Y\)​qη​\(Y\)⊤\]\.C\_\{\\eta\\eta\}\(a\)=E\_\{\\theta\_\{0\}\}\\left\[a\(Y\)w\(Y\)q\_\{\\eta\}\(Y\)q\_\{\\eta\}\(Y\)^\{\\top\}\\right\]\.Therefore, for anyv∈ℝqv\\in\\mathbb\{R\}^\{q\},

v⊤​Cη​η​\(a\)​v\\displaystyle v^\{\\top\}C\_\{\\eta\\eta\}\(a\)v=Eθ0​\[a⁡\(Y\)​w​\(Y\)​v⊤​qη​\(Y\)​qη​\(Y\)⊤​v\]\\displaystyle=E\_\{\\theta\_\{0\}\}\\left\[a\(Y\)w\(Y\)v^\{\\top\}q\_\{\\eta\}\(Y\)q\_\{\\eta\}\(Y\)^\{\\top\}v\\right\]=Eθ0​\[a⁡\(Y\)​w​\(Y\)​\{v⊤​qη​\(Y\)\}2\],\\displaystyle=E\_\{\\theta\_\{0\}\}\\left\[a\(Y\)w\(Y\)\\\{v^\{\\top\}q\_\{\\eta\}\(Y\)\\\}^\{2\}\\right\],which proves \([5\.4](https://arxiv.org/html/2611.24986#S5.E4)\)\.

Suppose first that there existsv≠0v\\neq 0such that

v⊤​qη​\(Y\)=0v^\{\\top\}q\_\{\\eta\}\(Y\)=0μa\\mu\_\{a\}\-almost surely\. Then

v⊤​Cη​η​\(a\)​v=0,v^\{\\top\}C\_\{\\eta\\eta\}\(a\)v=0,soCη​η​\(a\)C\_\{\\eta\\eta\}\(a\)cannot be positive definite\.

Conversely, ifCη​η​\(a\)C\_\{\\eta\\eta\}\(a\)is not positive definite, its positive semidefiniteness implies that there existsv≠0v\\neq 0for which

v⊤​Cη​η​\(a\)​v=0\.v^\{\\top\}C\_\{\\eta\\eta\}\(a\)v=0\.Hence

Eθ0​\[a⁡\(Y\)​w​\(Y\)​\{v⊤​qη​\(Y\)\}2\]=0\.E\_\{\\theta\_\{0\}\}\\left\[a\(Y\)w\(Y\)\\\{v^\{\\top\}q\_\{\\eta\}\(Y\)\\\}^\{2\}\\right\]=0\.The integrand is nonnegative, and therefore

v⊤​qη​\(Y\)=0v^\{\\top\}q\_\{\\eta\}\(Y\)=0μa\\mu\_\{a\}\-almost surely\. This establishes the equivalence\.

For the model

logit⁡r⁡\(Y,θ,η\)=η0\+η1​U​\(Y,θ\),\\operatorname\{logit\}r\(Y;\\theta,\\eta\)=\\eta\_\{0\}\+\\eta\_\{1\}U\(Y;\\theta\),we have

qη​\(Y\)=\(1U⁡\(Y,θ\)\)\.q\_\{\\eta\}\(Y\)=\\begin\{pmatrix\}1\\\\ U\(Y;\\theta\)\\end\{pmatrix\}\.Let expectation under the normalized measureμa\\mu\_\{a\}be denoted byEμaE\_\{\\mu\_\{a\}\}\. Apart from the positive normalizing constantEθ0​\{a⁡\(Y\)​w​\(Y\)\}E\_\{\\theta\_\{0\}\}\\\{a\(Y\)w\(Y\)\\\}, the nuisance\-information matrix is

Eμa​\[\(1U\)​\(1U\)\]=\(1Eμa​\(U\)Eμa​\(U\)Eμa​\(U2\)\)\.E\_\{\\mu\_\{a\}\}\\left\[\\begin\{pmatrix\}1\\\\ U\\end\{pmatrix\}\\begin\{pmatrix\}1&U\\end\{pmatrix\}\\right\]=\\begin\{pmatrix\}1&E\_\{\\mu\_\{a\}\}\(U\)\\\\ E\_\{\\mu\_\{a\}\}\(U\)&E\_\{\\mu\_\{a\}\}\(U^\{2\}\)\\end\{pmatrix\}\.Its determinant is

Eμa​\(U2\)−Eμa​\(U\)2=Varμa⁡\(U\)\.E\_\{\\mu\_\{a\}\}\(U^\{2\}\)\-E\_\{\\mu\_\{a\}\}\(U\)^\{2\}=\\operatorname\{Var\}\_\{\\mu\_\{a\}\}\(U\)\.Thus, whenEθ0​\{a⁡\(Y\)​w​\(Y\)\}\>0E\_\{\\theta\_\{0\}\}\\\{a\(Y\)w\(Y\)\\\}\>0,

Cη​η\(a\)≻0⟺Varμa\{U\(Y;θ0\)\}\>0,C\_\{\\eta\\eta\}\(a\)\\succ 0\\quad\\Longleftrightarrow\\quad\\operatorname\{Var\}\_\{\\mu\_\{a\}\}\\\{U\(Y;\\theta\_\{0\}\)\\\}\>0,which proves \([5\.7](https://arxiv.org/html/2611.24986#S5.E7)\)\.□\\square

### S1\.7Proof of Theorem[6\.1](https://arxiv.org/html/2611.24986#S6.Thmtheorem1)

#### Oracle criterion value\.

The approximate minimization condition and the definition ofΔn\\Delta\_\{n\}give

Φ⁡\(a^n\)≤Φ^n​\(a^n\)\+Δn≤Φ^n​\(a∗\)\+εn\+Δn≤Φ⁡\(a∗\)\+εn\+2​Δn\.\\Phi\(\\widehat\{a\}\_\{n\}\)\\leq\\widehat\{\\Phi\}\_\{n\}\(\\widehat\{a\}\_\{n\}\)\+\\Delta\_\{n\}\\leq\\widehat\{\\Phi\}\_\{n\}\(a^\{\*\}\)\+\\varepsilon\_\{n\}\+\\Delta\_\{n\}\\leq\\Phi\(a^\{\*\}\)\+\\varepsilon\_\{n\}\+2\\Delta\_\{n\}\.Sincea^n∈𝒟\\widehat\{a\}\_\{n\}\\in\\mathcal\{D\}anda∗a^\{\*\}minimizesΦ\\Phithere,Φ⁡\(a^n\)≥Φ⁡\(a∗\)\\Phi\(\\widehat\{a\}\_\{n\}\)\\geq\\Phi\(a^\{\*\}\)\. This proves \([6\.9](https://arxiv.org/html/2611.24986#S6.E9)\)\. This part does not require independence of the observations\.

#### Uniform fixed\-design second moments\.

For deterministica∈𝒟a\\in\\mathcal\{D\}, put

Tn,a=n​\(ϕ^n,a−ϕ0\),Wn,a=ℐ​\(a\)−1​1n​∑i=1nSa​\(Oi,ϕ0\)\.T\_\{n,a\}=\\sqrt\{n\}\(\\widehat\{\\phi\}\_\{n,a\}\-\\phi\_\{0\}\),\\qquad W\_\{n,a\}=\\mathcal\{I\}\(a\)^\{\-1\}\\frac\{1\}\{\\sqrt\{n\}\}\\sum\_\{i=1\}^\{n\}S\_\{a\}\(O\_\{i\};\\phi\_\{0\}\)\.The iid score identities imply exactly

Ea​Wn,a=0,Ea​\(Wn,a​Wn,a⊤\)=ℐ​\(a\)−1\.E\_\{a\}W\_\{n,a\}=0,\\qquad E\_\{a\}\(W\_\{n,a\}W\_\{n,a\}^\{\\top\}\)=\\mathcal\{I\}\(a\)^\{\-1\}\.Writingd\+qd\+qfor the dimension ofϕ\\phi, the information lower bound therefore gives

supa∈𝒟Ea​‖Wn,a‖2=supa∈𝒟tr⁡\{ℐ​\(a\)−1\}≤\(d\+q\)/κ\.\\sup\_\{a\\in\\mathcal\{D\}\}E\_\{a\}\\\|W\_\{n,a\}\\\|^\{2\}=\\sup\_\{a\\in\\mathcal\{D\}\}\\operatorname\{tr\}\\\{\\mathcal\{I\}\(a\)^\{\-1\}\\\}\\leq\(d\+q\)/\\kappa\.By \([6\.10](https://arxiv.org/html/2611.24986#S6.E10)\),Tn,a=Wn,a\+ζn,aT\_\{n,a\}=W\_\{n,a\}\+\\zeta\_\{n,a\}, with the squared remainder tending to zero uniformly in expectation\. Cauchy–Schwarz applied to the two cross terms yields

supa∈𝒟‖Ea​\(Tn,a​Tn,a⊤\)−ℐ​\(a\)−1‖⟶0,\\sup\_\{a\\in\\mathcal\{D\}\}\\left\\\|E\_\{a\}\(T\_\{n,a\}T\_\{n,a\}^\{\\top\}\)\-\\mathcal\{I\}\(a\)^\{\-1\}\\right\\\|\\longrightarrow 0,where∥⋅∥\\\|\\cdot\\\|denotes the operator norm for matrices\. Withen,a=θ^n,a−θ0e\_\{n,a\}=\\widehat\{\\theta\}\_\{n,a\}\-\\theta\_\{0\}, taking classification blocks and using the block\-inverse identity gives

supa∈𝒟‖n​Ea​\(en,a​en,a⊤\)−Ieff​\(a\)−1‖⟶0\.\\sup\_\{a\\in\\mathcal\{D\}\}\\left\\\|nE\_\{a\}\(e\_\{n,a\}e\_\{n,a\}^\{\\top\}\)\-I\_\{\\mathrm\{eff\}\}\(a\)^\{\-1\}\\right\\\|\\longrightarrow 0\.Consequently,

supa∈𝒟\|n​Ea​\(en,a⊤​HR​en,a\)−Φ⁡\(a\)\|⟶0\.\\sup\_\{a\\in\\mathcal\{D\}\}\\left\|nE\_\{a\}\(e\_\{n,a\}^\{\\top\}H\_\{R\}e\_\{n,a\}\)\-\\Phi\(a\)\\right\|\\longrightarrow 0\.

#### Uniform control of the risk remainder\.

Define

b⁡\(h\)=R⁡\(θ0\+h\)−R∗−12​h⊤​HR​hb\(h\)=R\(\\theta\_\{0\}\+h\)\-R^\{\*\}\-\\tfrac\{1\}\{2\}h^\{\\top\}H\_\{R\}hfor parameter perturbations in the domain of the classifier\. By \([4\.1](https://arxiv.org/html/2611.24986#S4.E1)\), for everyϵ\>0\\epsilon\>0there is a radiuss\>0s\>0such that\|b⁡\(h\)\|≤ϵ​‖h‖2\|b\(h\)\|\\leq\\epsilon\\\|h\\\|^\{2\}whenever‖h‖≤s\\\|h\\\|\\leq s\. Condition \([6\.11](https://arxiv.org/html/2611.24986#S6.E11)\) gives a finite constantCCsuch that, for all sufficiently largenn,

supa∈𝒟Ea​‖n​en,a‖2\+δ≤C\.\\sup\_\{a\\in\\mathcal\{D\}\}E\_\{a\}\\\|\\sqrt\{n\}e\_\{n,a\}\\\|^\{2\+\\delta\}\\leq C\.On the complement of the radius\-ssball, boundedness of classification risk implies\|b⁡\(e\)\|≤1\+12​‖HR‖​‖e‖2\|b\(e\)\|\\leq 1\+\\tfrac\{1\}\{2\}\\\|H\_\{R\}\\\|\\\|e\\\|^\{2\}\. Markov’s inequality gives

supan​Pa​\(‖en,a‖\>s\)\\displaystyle\\sup\_\{a\}nP\_\{a\}\(\\\|e\_\{n,a\}\\\|\>s\)≤Cs−\(2\+δ\)n−δ/2,\\displaystyle\\leq Cs^\{\-\(2\+\\delta\)\}n^\{\-\\delta/2\},supan​Ea​\{‖en,a‖2​𝟏​\(‖en,a‖\>s\)\}\\displaystyle\\sup\_\{a\}nE\_\{a\}\\\{\\\|e\_\{n,a\}\\\|^\{2\}\\mathbf\{1\}\(\\\|e\_\{n,a\}\\\|\>s\)\\\}≤Cs−δn−δ/2\.\\displaystyle\\leq Cs^\{\-\\delta\}n^\{\-\\delta/2\}\.Both bounds tend to zero\. Inside the ball,

supan​Ea​\{\|b⁡\(en,a\)\|​𝟏​\(‖en,a‖≤s\)\}≤ϵ​supaEa​‖n​en,a‖2\.\\sup\_\{a\}nE\_\{a\}\\\{\|b\(e\_\{n,a\}\)\|\\mathbf\{1\}\(\\\|e\_\{n,a\}\\\|\\leq s\)\\\}\\leq\\epsilon\\sup\_\{a\}E\_\{a\}\\\|\\sqrt\{n\}e\_\{n,a\}\\\|^\{2\}\.The last supremum is bounded, by either the moment condition or the preceding second\-moment calculation\. Taking the limit superior and then lettingϵ↓0\\epsilon\\downarrow 0proves

supa∈𝒟n​Ea​\|b⁡\(en,a\)\|⟶0\.\\sup\_\{a\\in\\mathcal\{D\}\}nE\_\{a\}\|b\(e\_\{n,a\}\)\|\\longrightarrow 0\.Together with the quadratic term, this establishes the uniform expansion

supa∈𝒟\|Ea​\{R⁡\(θ^n,a\)−R∗\}−Φ⁡\(a\)2​n\|=o⁡\(n−1\)\.\\sup\_\{a\\in\\mathcal\{D\}\}\\left\|E\_\{a\}\\\{R\(\\widehat\{\\theta\}\_\{n,a\}\)\-R^\{\*\}\\\}\-\\frac\{\\Phi\(a\)\}\{2n\}\\right\|=o\(n^\{\-1\}\)\.\(S1\.7\)

#### Conditioning on the pilot\.

For deterministicaa, letgn​\(a\)=Ea​\{R⁡\(θ^n,a\)−R∗\}g\_\{n\}\(a\)=E\_\{a\}\\\{R\(\\widehat\{\\theta\}\_\{n,a\}\)\-R^\{\*\}\\\}\. The selected design isℱ0,n\\mathcal\{F\}\_\{0,n\}\-measurable, and the estimator uses the pilot only through this design\. The conditional product law \([6\.3](https://arxiv.org/html/2611.24986#S6.E3)\) therefore gives the exact identity

E⁡\{R⁡\(θ^n,a^n\)−R∗∣ℱ0,n\}=gn​\(a^n\)\.E\\\{R\(\\widehat\{\\theta\}\_\{n,\\widehat\{a\}\_\{n\}\}\)\-R^\{\*\}\\mid\\mathcal\{F\}\_\{0,n\}\\\}=g\_\{n\}\(\\widehat\{a\}\_\{n\}\)\.Setrn=gn​\(a^n\)−Φ⁡\(a^n\)/\(2​n\)r\_\{n\}=g\_\{n\}\(\\widehat\{a\}\_\{n\}\)\-\\Phi\(\\widehat\{a\}\_\{n\}\)/\(2n\)\. Equation \([S1\.7](https://arxiv.org/html/2611.24986#S1.E7)\) implies

E​\|rn\|≤supa∈𝒟\|gn​\(a\)−Φ⁡\(a\)2​n\|=o⁡\(n−1\),E\|r\_\{n\}\|\\leq\\sup\_\{a\\in\\mathcal\{D\}\}\\left\|g\_\{n\}\(a\)\-\\frac\{\\Phi\(a\)\}\{2n\}\\right\|=o\(n^\{\-1\}\),which proves \([6\.12](https://arxiv.org/html/2611.24986#S6.E12)\)\. This step is why uniform fixed\-design control is needed instead of an expansion only for each individual deterministic design\.

#### Averaging over the pilot\.

Taking expectations now gives

E⁡\{R⁡\(θ^n,a^n\)\}−R∗=12​n​E​\{Φ⁡\(a^n\)\}\+o⁡\(n−1\)\.E\\\{R\(\\widehat\{\\theta\}\_\{n,\\widehat\{a\}\_\{n\}\}\)\\\}\-R^\{\*\}=\\frac\{1\}\{2n\}E\\\{\\Phi\(\\widehat\{a\}\_\{n\}\)\\\}\+o\(n^\{\-1\}\)\.On𝒟\\mathcal\{D\},

0≤Φ⁡\(a\)≤κ−1​tr⁡\(H~R\)\.0\\leq\\Phi\(a\)\\leq\\kappa^\{\-1\}\\operatorname\{tr\}\(\\widetilde\{H\}\_\{R\}\)\.The convergence in probability in \([6\.9](https://arxiv.org/html/2611.24986#S6.E9)\), together with this deterministic bound, impliesE​Φ​\(a^n\)→Φ⁡\(a∗\)E\\Phi\(\\widehat\{a\}\_\{n\}\)\\to\\Phi\(a^\{\*\}\)\. This proves \([6\.13](https://arxiv.org/html/2611.24986#S6.E13)\)\. Finally,Nn=n\+mnN\_\{n\}=n\+m\_\{n\}satisfiesNn/n→1N\_\{n\}/n\\to 1, so replacingnnbyNnN\_\{n\}in the leading denominator changes the expression byo⁡\(Nn−1\)o\(N\_\{n\}^\{\-1\}\)\. This change of denominator concerns sample\-size accounting only; the proof has not treated the pooled two\-stage observations as iid\.□\\square

### S1\.8Derivation of the binary logistic risk expansion

For the binary logistic model,

τθ​\(y\)=exp⁡\(α\+β​y\)1\+exp⁡\(α\+β​y\),β\>0,\\tau\_\{\\theta\}\(y\)=\\frac\{\\exp\(\\alpha\+\\beta y\)\}\{1\+\\exp\(\\alpha\+\\beta y\)\},\\qquad\\beta\>0,the Bayes classifier under parameterθ\\thetachanges class at

b⁡\(θ\)=−αβ\.b\(\\theta\)=\-\\frac\{\\alpha\}\{\\beta\}\.Let

b0=b⁡\(θ0\)=−α0β0\.b\_\{0\}=b\(\\theta\_\{0\}\)=\-\\frac\{\\alpha\_\{0\}\}\{\\beta\_\{0\}\}\.
Forθ\\thetasufficiently close toθ0\\theta\_\{0\}, the classifier based onθ\\thetadiffers from the Bayes classifier only on the interval betweenb⁡\(θ\)b\(\\theta\)andb0b\_\{0\}\. Hence

R⁡\(θ\)−R∗=\|∫b0b⁡\(θ\)\{2​τθ0​\(y\)−1\}​f​\(y\)​𝑑y\|\.R\(\\theta\)\-R^\{\*\}=\\left\|\\int\_\{b\_\{0\}\}^\{b\(\\theta\)\}\\\{2\\tau\_\{\\theta\_\{0\}\}\(y\)\-1\\\}f\(y\)\\,dy\\right\|\.\(S1\.8\)Since

τθ0​\(b0\)=12,\\tau\_\{\\theta\_\{0\}\}\(b\_\{0\}\)=\\frac\{1\}\{2\},the integrand vanishes atb0b\_\{0\}\. Moreover,

dd​y​τθ0​\(y\)=β0​τθ0​\(y\)​\{1−τθ0​\(y\)\},\\frac\{d\}\{dy\}\\tau\_\{\\theta\_\{0\}\}\(y\)=\\beta\_\{0\}\\tau\_\{\\theta\_\{0\}\}\(y\)\\\{1\-\\tau\_\{\\theta\_\{0\}\}\(y\)\\\},so

dd​y​\{2​τθ0​\(y\)−1\}\|y=b0=β02\.\\left\.\\frac\{d\}\{dy\}\\\{2\\tau\_\{\\theta\_\{0\}\}\(y\)\-1\\\}\\right\|\_\{y=b\_\{0\}\}=\\frac\{\\beta\_\{0\}\}\{2\}\.Therefore,

\{2​τθ0​\(y\)−1\}​f​\(y\)=β0​f​\(b0\)2​\(y−b0\)\+o⁡\(\|y−b0\|\)\.\\\{2\\tau\_\{\\theta\_\{0\}\}\(y\)\-1\\\}f\(y\)=\\frac\{\\beta\_\{0\}f\(b\_\{0\}\)\}\{2\}\(y\-b\_\{0\}\)\+o\(\|y\-b\_\{0\}\|\)\.Substitution into \([S1\.8](https://arxiv.org/html/2611.24986#S1.E8)\) gives

R⁡\(θ\)−R∗=β0​f​\(b0\)4​\{b⁡\(θ\)−b0\}2\+o⁡\(\{b⁡\(θ\)−b0\}2\)\.R\(\\theta\)\-R^\{\*\}=\\frac\{\\beta\_\{0\}f\(b\_\{0\}\)\}\{4\}\\\{b\(\\theta\)\-b\_\{0\}\\\}^\{2\}\+o\\left\(\\\{b\(\\theta\)\-b\_\{0\}\\\}^\{2\}\\right\)\.\(S1\.9\)
Now let

θ=θ0\+h,h=\(hαhβ\)\.\\theta=\\theta\_\{0\}\+h,\\qquad h=\\begin\{pmatrix\}h\_\{\\alpha\}\\\\ h\_\{\\beta\}\\end\{pmatrix\}\.The gradient of the boundary function is

∇θb​\(θ\)=\(−1/βα/β2\)\.\\nabla\_\{\\theta\}b\(\\theta\)=\\begin\{pmatrix\}\-1/\\beta\\\\ \\alpha/\\beta^\{2\}\\end\{pmatrix\}\.Atθ0\\theta\_\{0\}, usingα0=−β0​b0\\alpha\_\{0\}=\-\\beta\_\{0\}b\_\{0\},

∇θb​\(θ0\)=−1β0​\(1b0\)\.\\nabla\_\{\\theta\}b\(\\theta\_\{0\}\)=\-\\frac\{1\}\{\\beta\_\{0\}\}\\begin\{pmatrix\}1\\\\ b\_\{0\}\\end\{pmatrix\}\.Thus

b⁡\(θ0\+h\)−b0=−1β0​\(hα\+b0​hβ\)\+o⁡\(‖h‖\)\.b\(\\theta\_\{0\}\+h\)\-b\_\{0\}=\-\\frac\{1\}\{\\beta\_\{0\}\}\(h\_\{\\alpha\}\+b\_\{0\}h\_\{\\beta\}\)\+o\(\\\|h\\\|\)\.Substituting this expansion into \([S1\.9](https://arxiv.org/html/2611.24986#S1.E9)\) yields

R⁡\(θ0\+h\)−R∗=f⁡\(b0\)4​β0​\(hα\+b0​hβ\)2\+o⁡\(‖h‖2\)\.R\(\\theta\_\{0\}\+h\)\-R^\{\*\}=\\frac\{f\(b\_\{0\}\)\}\{4\\beta\_\{0\}\}\(h\_\{\\alpha\}\+b\_\{0\}h\_\{\\beta\}\)^\{2\}\+o\(\\\|h\\\|^\{2\}\)\.Hence

R⁡\(θ0\+h\)−R∗=12​h⊤​HR​h\+o⁡\(‖h‖2\),R\(\\theta\_\{0\}\+h\)\-R^\{\*\}=\\frac\{1\}\{2\}h^\{\\top\}H\_\{R\}h\+o\(\\\|h\\\|^\{2\}\),with

HR=f⁡\(b0\)2​β0​\(1b0\)​\(1b0\)\.H\_\{R\}=\\frac\{f\(b\_\{0\}\)\}\{2\\beta\_\{0\}\}\\begin\{pmatrix\}1\\\\ b\_\{0\}\\end\{pmatrix\}\\begin\{pmatrix\}1&b\_\{0\}\\end\{pmatrix\}\.This proves \([7\.2](https://arxiv.org/html/2611.24986#S7.E2)\) and \([7\.3](https://arxiv.org/html/2611.24986#S7.E3)\)\.□\\square

## S2Response\-term ablation

The ablation uses the full query–response information matrix and the same joint likelihood as response\-aware acquisition, but omitsψaR\\psi\_\{a\}^\{R\}from candidate scores\. All comparisons use 500 paired replications and total query budgets including the common pilot\.

For each replication, the paired difference is the excess risk of the full criterion minus that of the ablation\. Intervals are pointwise 95% Monte Carlottintervals\. Two\-sided pairedtt\-testpp\-values are adjusted by Holm’s method over the nine regime–budget comparisons separately within the oracle and adaptive studies\. The oracle null comparisons are exactly zero; their intervals are degenerate and theirpp\-values are recorded as one by convention\.

Table S1:Paired excess\-risk differences for response\-aware acquisition versus the label\-only, full\-information ablation\. Differences and interval endpoints are multiplied by10310^\{3\}\. Negative values favor response\-aware acquisition\.StudyResponseBBDifference95% intervalpHolmp\_\{\\mathrm\{Holm\}\}OracleNoninformative3000\.0000\[0\.0000, 0\.0000\]1\.00006000\.0000\[0\.0000, 0\.0000\]1\.00009000\.0000\[0\.0000, 0\.0000\]1\.0000Moderate300\-0\.0874\[\-0\.3741, 0\.1994\]1\.0000600\-0\.0960\[\-0\.2255, 0\.0335\]1\.0000900\-0\.1107\[\-0\.1862, \-0\.0352\]0\.0373Strong300\-0\.0372\[\-0\.1182, 0\.0437\]1\.00006000\.0013\[\-0\.0162, 0\.0188\]1\.0000900\-0\.0071\[\-0\.0198, 0\.0056\]1\.0000AdaptiveNoninformative3000\.2192\[\-0\.1308, 0\.5691\]1\.00006000\.0707\[\-0\.0577, 0\.1990\]1\.00009000\.0354\[\-0\.0181, 0\.0889\]1\.0000Moderate300\-0\.2205\[\-0\.4623, 0\.0214\]0\.5912600\-0\.0832\[\-0\.2036, 0\.0372\]1\.0000900\-0\.1077\[\-0\.1726, \-0\.0428\]0\.0107Strong300\-0\.0305\[\-0\.1247, 0\.0638\]1\.0000600\-0\.0259\[\-0\.0602, 0\.0083\]0\.9648900\-0\.0071\[\-0\.0247, 0\.0104\]1\.0000

Figure[S1](https://arxiv.org/html/2611.24986#S2.F1)displays the paired differences and pointwise intervals\. The intervals are not simultaneous confidence intervals\.

Figure S1:Excess\-risk differences between the full response\-aware criterion and the label\-only, full\-information ablation\. Negative differences favor the full criterion\. Bars are pointwise 95% Monte Carlottintervals over 500 paired replications\. Query budgets include the pilot\.

相似文章

Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning

arXiv cs.LG

This paper proves that using error-penalized scoring rules with abstention as a discrete action can kill both the reward gradient and the KL anchor, causing models to collapse toward refusing everything. It proposes a structural repair — training a mandatory confidence report — and validates the mechanism with simulations and language model experiments.

情境线性优化的主动学习:基于间隔的方法

arXiv cs.LG

本文提出了首个用于上下文线性优化的主动学习方法,采用由智能预测后优化(Smart Predict-then-Optimize,SPO)损失引导的基于间隔的标签获取规则,与监督学习相比,只需更少的标签即可达到相同的超额风险,并在个性化定价和最短路径问题中进行了验证。

跨问题可靠强化学习

arXiv cs.LG

该论文提出了 CARE(跨问题弃答奖励估计),这是一种强化学习方法,在不同风险容忍度下,根据大语言模型以语言表达的置信度与其他问题置信度的相对关系给予奖励,而非依据二元正确性标签,从而显著提高了数学和分布外问答基准测试中风险为 5% 时的覆盖率。

预算协调:少标签联邦主动学习

arXiv cs.LG

本文研究低预算环境下的联邦主动学习,揭示了由于异质性反转,同质数据需要更强的协调。它提出了一种利用联邦表示学习的框架,以实现全局协调的主动选择,性能优于现有方法。