PUe: Biased Positive-Unlabeled Learning Enhancement by Causal Inference

# PUe: Biased Positive-Unlabeled Learning Enhancement by Causal Inference
Source: [https://arxiv.org/html/2607.13428](https://arxiv.org/html/2607.13428)
Xutao Wang Hanting Chen Tianyu Guo Yunhe Wang Huawei Noah’s Ark Lab \{xutao\.wang,chenhanting,tianyu\.guo,yunhe\.wang\}@huawei\.com

###### Abstract

Positive\-Unlabeled \(PU\) learning aims to achieve high\-accuracy binary classification with limited labeled positive examples and numerous unlabeled ones\. Existing cost\-sensitive\-based methods often rely on strong assumptions that examples with an observed positive label were selected entirely at random\. In fact, the uneven distribution of labels is prevalent in real\-world PU problems, indicating that most actual positive and unlabeled data are subject to selection bias\. Building on the SAR\-PU propensity\-weighted framework of Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\], we study a PU learning enhancement \(PUe\) framework using normalized propensity scores and normalized inverse probability weighting \(NIPW\)\. PUe’s main contributions are a normalized inverse\-probability\-weighted PU risk formulation; additional theoretical analyses of normalized sample\-weight error and common PU estimators under biased labeling; regularized deep propensity\-score estimation; integration with modern cost\-sensitive PU methods; and support for selectively labeled negative classes\. Experiments on MNIST, CIFAR\-10, and ADNI demonstrate improvements over several PU baselines under non\-uniform label distributions\. Codes are available at[GitHub](https://github.com/huawei-noah/Noah-research/tree/master/PUe)and[Gitee](https://gitee.com/mindspore/models/tree/master/research/cv/PUe)\.

## 1Introduction

In the era of big data, deep neural networks have achieved outstanding performance across various tasks, even surpassing human performance in many instances, particularly in traditional binary classification problems\. The success of these deep neural networks often hinges on supervised learning using large quantities of labeled data\. However, in reality, acquiring even binary labels can be challenging\. For instance, in recommendation systems, users’ multiple clicks on films may be considered as positive samples\. Nonetheless, all other films cannot be assumed uninteresting and thus should not be treated as negative examples; instead, they should be regarded as unlabeled ones\.

The same issue emerges in text classification, where it is typically more straightforward to define a partial set of positive samples\. However, due to the vast diversity of negative samples, it becomes difficult or even impossible to describe a comprehensive set of negative samples that represent all content not included in the positive samples\. Similar situations occur in medical diagnostics, malicious URL detection, and spam detection, where only a few labeled positives are available amidst a plethora of unlabeled data\. This scenario is a variant of the classical binary classification setup, known as PU\. In recent years, there has been a growing interest in this setting\. Positive\-Unlabeled \(PU\) learning primarily addresses the challenge of learning binary classifiers solely from positive samples and unlabeled data\.

In previous research, numerous PU algorithms have been developed, with cost\-sensitive PU learning emerging as a popular research direction\. Methods such as\[[2](https://arxiv.org/html/2607.13428#bib.bib2),[3](https://arxiv.org/html/2607.13428#bib.bib3),[4](https://arxiv.org/html/2607.13428#bib.bib4)\]reweight positive and negative risks by hyper\-parameters and minimize it\. In addition, Self\-PU\[[5](https://arxiv.org/html/2607.13428#bib.bib5)\]introduces self\-supervision to nnPU through auxiliary tasks, including model calibration and distillation; ImbPU\[[6](https://arxiv.org/html/2607.13428#bib.bib6)\]oversamples and modifies sample weights to address unbalanced data\. Dist\-PU\[[7](https://arxiv.org/html/2607.13428#bib.bib7)\]corrects the negative preference of the classification model through prior information\. PUSB\[[8](https://arxiv.org/html/2607.13428#bib.bib8)\]maintains the order\-preserving assumption\. However, these methods necessitate the assumption that their checked set is uniformly sampled from the population, or else the PU learning risk estimator ceases to be an unbiased or consistent estimator, otherwise resulting in reduced model accuracy\.

In reality, the labeled set is often biased and does not conform to the Selected Completely At Random \(SCAR\) assumption\[[9](https://arxiv.org/html/2607.13428#bib.bib9),[1](https://arxiv.org/html/2607.13428#bib.bib1)\], which posits that the observed labeled examples are a random subset of the complete set of positive examples\. Consequently, it is essential to relax the assumption of the labeled set and replace it with a more general assumption about the labeling mechanism: the probability of selecting positive examples to be labeled depends on their attribute values, known as the Selected At Random \(SAR\) assumption\[[9](https://arxiv.org/html/2607.13428#bib.bib9)\]\.

The most directly related prior work is Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\], which studies positive\-unlabeled learning beyond SCAR under the Selected At Random \(SAR\) labeling mechanism\. PUe adapts this SAR\-PU foundation to a normalized PU risk formulation, develops additional theoretical analyses for normalized weighting and common PU estimators under biased labeling, estimates propensity scores with regularized deep models, and integrates the resulting correction with modern cost\-sensitive PU methods\. A detailed component\-level comparison between Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]and PUe is provided in Appendix D\.

Gerych et al\.\[[10](https://arxiv.org/html/2607.13428#bib.bib10)\]is also related, with a different emphasis on conditions for recovering identifiable propensity scores\. That work discusses Local Certainty and Probabilistic Gap assumptions\. Under the Local Certainty hypothesis, there is no overlap between the positive and negative hypotheses,p​\(xP^∣y=0\)=0p\(x^\{\\widehat\{P\}\}\\mid y=0\)=0\. Under the Probabilistic Gap assumption, it is assumed thate=k​p​\(y=1∣x\)e=kp\(y=1\\mid x\)is linear and there is an anchor point\. We use this work primarily to motivate the challenges of propensity\-score estimation\.

However, the above assumptions are too strict\. We show in the appendix that the estimated propensity scores cannot be completely unbiased\. To address the aforementioned problem, we propose a causal\-inference\-inspired PU learning framework, termed PUe\. Our approach uses propensity weighting to correct the original PU learning risk estimator under biased conditions; PUe formulates this correction using normalized inverse probability weights\. Given that propensity scores for samples are typically unknown, we apply regularization techniques to a deep learning classifier to estimate the propensity score of each sample in the labeled set\. An illustration of the proposed method is shown in Figure[1](https://arxiv.org/html/2607.13428#S1.F1)\. PUe further analyzes normalized sample\-weight error and common PU estimators under biased labeling, and integrates normalized propensity correction with modern cost\-sensitive PU algorithms\.

∙\\bulletLabeled Positive Samples∘\\circUnlabeled Negative Samples∙\\bulletUnlabeled Positive SamplesClassification Boundary
![Refer to caption](https://arxiv.org/html/2607.13428v1/figures/figure1_traditional_pu.jpg)\(a\)Traditional PU
![Refer to caption](https://arxiv.org/html/2607.13428v1/figures/figure1_proposed_pue.jpg)\(b\)Proposed PUe

Figure 1:Our method and traditional PU method classification diagram\. PUe uses reweighting to make the classification plane more accurate\.Our main contributions are summarized as follows:

- •We formulate PUe with normalized inverse probability weighting in the PU risk formulation for biased positive\-unlabeled learning\.
- •We develop additional theoretical analyses of normalized sample\-weight error and common PU estimators under biased labeling\.
- •We extend propensity\-score estimation to deep neural networks and introduce regularization to reduce overfitting and degenerate propensity estimates\.
- •We integrate the normalized propensity\-weighting mechanism with cost\-sensitive PU methods, including uPU, nnPU, PUbN, and Dist\-PU\.
- •We extend the framework to selectively labeled negative classes, resulting in PUbNe\.
- •We evaluate the resulting framework on MNIST, CIFAR\-10, and the Alzheimer’s Disease Neuroimaging Initiative \(ADNI\) database\.

## 2Methodology

In this section, we review existing PU algorithms and their limitations in biased labeling scenarios, then introduce PUe to improve PU learning under biased labeling\.

### 2\.1Review of PU classification

In standard PN classification, letx∈ℝdx\\in\\mathbb\{R\}^\{d\}andy∈\{\+1,−1\}y\\in\\\{\+1,\-1\\\}be the input sample and its corresponding label\. We are given positive data and negative data sampled independently frompP​\(x\)=p​\(x∣y=\+1\)p\_\{P\}\(x\)=p\(x\\mid y=\+1\)andpN​\(x\)=p​\(x∣y=−1\)p\_\{N\}\(x\)=p\(x\\mid y=\-1\)asχP=\{xiP\}i=1nP\\chi\_\{P\}=\\\{x\_\{i\}^\{P\}\\\}\_\{i=1\}^\{n\_\{P\}\}andχN=\{xiN\}i=1nN\\chi\_\{N\}=\\\{x\_\{i\}^\{N\}\\\}\_\{i=1\}^\{n\_\{N\}\}\. We denote the class prior probability asπ=p​\(y=1\)\\pi=p\(y=1\)and follow the convention thatπ\\piis known throughout the paper\[[11](https://arxiv.org/html/2607.13428#bib.bib11)\]\. Class prior can also be estimated under biased positives and unlabeled examples\[[12](https://arxiv.org/html/2607.13428#bib.bib12)\]\. Letg:ℝd→ℝg:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}be the binary classifier,θ\\thetabe its parameter, andL:ℝ×\{\+1,−1\}→ℝ\+L:\\mathbb\{R\}\\times\\\{\+1,\-1\\\}\\to\\mathbb\{R\}\_\{\+\}be a loss function\.

LetRP​\(g,\+1\)=𝔼x∼pP​\(x\)​\[L​\(g​\(x\),\+1\)\]R\_\{P\}\(g,\+1\)=\\mathbb\{E\}\_\{x\\sim p\_\{P\}\(x\)\}\[L\(g\(x\),\+1\)\]andRN​\(g,−1\)=𝔼x∼pN​\(x\)​\[L​\(g​\(x\),−1\)\]R\_\{N\}\(g,\-1\)=\\mathbb\{E\}\_\{x\\sim p\_\{N\}\(x\)\}\[L\(g\(x\),\-1\)\]\. The empirical PN risk is

R^P​N​\(g\)=π​R^P​\(g,\+1\)\+\(1−π\)​R^N​\(g,−1\),\\widehat\{R\}\_\{PN\}\(g\)=\\pi\\widehat\{R\}\_\{P\}\(g,\+1\)\+\(1\-\\pi\)\\widehat\{R\}\_\{N\}\(g,\-1\),\(1\)where

R^P​\(g,\+1\)=1nP​∑i=1nPL​\(g​\(xiP\),\+1\),R^N​\(g,−1\)=1nN​∑i=1nNL​\(g​\(xiN\),−1\)\.\\widehat\{R\}\_\{P\}\(g,\+1\)=\\frac\{1\}\{n\_\{P\}\}\\sum\_\{i=1\}^\{n\_\{P\}\}L\(g\(x\_\{i\}^\{P\}\),\+1\),\\quad\\widehat\{R\}\_\{N\}\(g,\-1\)=\\frac\{1\}\{n\_\{N\}\}\\sum\_\{i=1\}^\{n\_\{N\}\}L\(g\(x\_\{i\}^\{N\}\),\-1\)\.
In standard PU classification, instead of negative dataχN\\chi\_\{N\}, we have unlabeled samplesχU=\{xiU\}i=1nU∼p​\(x\)\\chi\_\{U\}=\\\{x\_\{i\}^\{U\}\\\}\_\{i=1\}^\{n\_\{U\}\}\\sim p\(x\)\. Sincep​\(x\)=π​pP​\(x\)\+\(1−π\)​pN​\(x\)p\(x\)=\\pi p\_\{P\}\(x\)\+\(1\-\\pi\)p\_\{N\}\(x\), we have

\(1−π\)​RN​\(g,−1\)=RU​\(g,−1\)−π​RP​\(g,−1\)\.\(1\-\\pi\)R\_\{N\}\(g,\-1\)=R\_\{U\}\(g,\-1\)\-\\pi R\_\{P\}\(g,\-1\)\.\(2\)The unbiased PU empirical risk is

R^u​P​U​\(g\)=π​R^P​\(g,\+1\)\+R^U​\(g,−1\)−π​R^P​\(g,−1\),\\widehat\{R\}\_\{uPU\}\(g\)=\\pi\\widehat\{R\}\_\{P\}\(g,\+1\)\+\\widehat\{R\}\_\{U\}\(g,\-1\)\-\\pi\\widehat\{R\}\_\{P\}\(g,\-1\),\(3\)where

R^P​\(g,−1\)=1nP​∑i=1nPL​\(g​\(xiP\),−1\),R^U​\(g,−1\)=1nU​∑i=1nUL​\(g​\(xiU\),−1\)\.\\widehat\{R\}\_\{P\}\(g,\-1\)=\\frac\{1\}\{n\_\{P\}\}\\sum\_\{i=1\}^\{n\_\{P\}\}L\(g\(x\_\{i\}^\{P\}\),\-1\),\\quad\\widehat\{R\}\_\{U\}\(g,\-1\)=\\frac\{1\}\{n\_\{U\}\}\\sum\_\{i=1\}^\{n\_\{U\}\}L\(g\(x\_\{i\}^\{U\}\),\-1\)\.
In theory, the risk\(1−π\)​RN​\(g,−1\)=RU​\(g,−1\)−π​RP​\(g,−1\)\(1\-\\pi\)R\_\{N\}\(g,\-1\)=R\_\{U\}\(g,\-1\)\-\\pi R\_\{P\}\(g,\-1\)is non\-negative\. However, ifggis too flexible,R^P​U​\(g^P​U\)\\widehat\{R\}\_\{PU\}\(\\hat\{g\}\_\{PU\}\)can become negative and the model can severely overfit the training data\. The non\-negative risk estimator for PU learning alleviates overfitting\[[11](https://arxiv.org/html/2607.13428#bib.bib11)\]\.

### 2\.2PU classification in biased scenarios

PU learning mostly assumes that all labeled samples are selected completely at random from all positive samples, which is called Selected Completely At Random \(SCAR\)\.

###### Definition 1\(SCAR, selected completely at random\[[4](https://arxiv.org/html/2607.13428#bib.bib4)\]\)\.

Labeled examples are selected completely at random, independent from their attributes, from the positive distribution\. The propensity scoree​\(x\)e\(x\), which is the probability of selecting a positive example, is constant and equal to the label frequency:

e​\(x\)=p​\(s=1∣x,y=1\)=p​\(s=1∣y=1\)=c\.e\(x\)=p\(s=1\\mid x,y=1\)=p\(s=1\\mid y=1\)=c\.\(4\)

However, many PU learning applications suffer from labeling bias\. The SCAR assumption does not conform to reality\. For example, whether someone clicks on a sponsored search ad is influenced by the position in which it is placed\. Similarly, whether a patient with a disease will see a doctor depends on socio\-economic status and symptom severity\. The Selected At Random \(SAR\) assumption is a more general assumption about the labeling mechanism: the probability of selecting positive examples to be labeled depends on attribute values\[[13](https://arxiv.org/html/2607.13428#bib.bib13)\]\.

###### Definition 2\(SAR, selected at random\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]\)\.

The labeling mechanism depends on the values of the attributes of the example\. However, given the attribute values, it does not depend further on the probability of the example being positive\. Instead of assuming a constant probability for all positive examples to be labeled, SAR assumes that the probability is a function of a subset of the example’s attributes:

e​\(x\)=p​\(s=1∣x,y=1\)\.e\(x\)=p\(s=1\\mid x,y=1\)\.

Motivated by causal\-inference methods for correcting selection bias,\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]introduced using propensity scores and inverse\-probability weighting \(IPW\) to construct a normalized correction for biased PU learning\[[14](https://arxiv.org/html/2607.13428#bib.bib14),[15](https://arxiv.org/html/2607.13428#bib.bib15)\]\. A crucial difference from the propensity score in standard causal inference is that the PU propensity score is conditioned on the class being positive\. Since negative examples have zero probability of being labeled as positive, IPW is applied to the labeled positive component\. For each labeled example\(xi,si=1\)\(x\_\{i\},s\_\{i\}=1\)with propensity scoreeie\_\{i\}, there are expected to be1/ei1/e\_\{i\}positive examples\. We propose to enhance this framework by using normalized inverse propensity weighting \(NIPW\)\.

The positive risk can be decomposed by propensity level:

RP​\(g,\+1\)=𝔼x∼pP​\(x\)​\[L​\(g​\(x\),\+1\)\]=𝔼c∈\(0,1\]​\{𝔼x∼pP​\(x\)​\[L​\(g​\(x\),\+1\)∣e​\(x\)=c\]\}\.R\_\{P\}\(g,\+1\)=\\mathbb\{E\}\_\{x\\sim p\_\{P\}\(x\)\}\[L\(g\(x\),\+1\)\]=\\mathbb\{E\}\_\{c\\in\(0,1\]\}\\left\\\{\\mathbb\{E\}\_\{x\\sim p\_\{P\}\(x\)\}\[L\(g\(x\),\+1\)\\mid e\(x\)=c\]\\right\\\}\.\(5\)The PUe positive\-risk estimator with normalized inverse\-propensity weights is

R^Pe​\(g,\+1\)=∑i=1nPωiP​L​\(g​\(xiP\),\+1\)\.\\widehat\{R\}\_\{P\}^\{e\}\(g,\+1\)=\\sum\_\{i=1\}^\{n\_\{P\}\}\\omega\_\{i\}^\{P\}L\(g\(x\_\{i\}^\{P\}\),\+1\)\.\(6\)Similarly,

R^Pe​\(g,−1\)=∑i=1nPωiP​L​\(g​\(xiP\),−1\)\.\\widehat\{R\}\_\{P\}^\{e\}\(g,\-1\)=\\sum\_\{i=1\}^\{n\_\{P\}\}\\omega\_\{i\}^\{P\}L\(g\(x\_\{i\}^\{P\}\),\-1\)\.\(7\)Heree​\(xiP\)e\(x\_\{i\}^\{P\}\)is the true propensity score ande^​\(xiP\)\\widehat\{e\}\(x\_\{i\}^\{P\}\)is its estimate\. The normalized inverse weights are

ωiP=1/e​\(xiP\)∑j=1nP1/e​\(xjP\),ω^iP=1/e^​\(xiP\)∑j=1nP1/e^​\(xjP\)\.\\omega\_\{i\}^\{P\}=\\frac\{1/e\(x\_\{i\}^\{P\}\)\}\{\\sum\_\{j=1\}^\{n\_\{P\}\}1/e\(x\_\{j\}^\{P\}\)\},\\qquad\\widehat\{\\omega\}\_\{i\}^\{P\}=\\frac\{1/\\widehat\{e\}\(x\_\{i\}^\{P\}\)\}\{\\sum\_\{j=1\}^\{n\_\{P\}\}1/\\widehat\{e\}\(x\_\{j\}^\{P\}\)\}\.Thus

∑i=1nPωiP=1,∑i=1nPω^iP=1\.\\sum\_\{i=1\}^\{n\_\{P\}\}\\omega\_\{i\}^\{P\}=1,\\qquad\\sum\_\{i=1\}^\{n\_\{P\}\}\\widehat\{\\omega\}\_\{i\}^\{P\}=1\.
Existing propensity\-score estimation methods such as SAR\-EM in Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]can underestimate propensity scores of positive samples, producing co\-directional bias or degenerate estimates in which labeled samples have propensity score close to 1 and unlabeled samples have propensity score close to 0\. PUe uses normalization to reduce co\-directional deviation and regularization to alleviate overfitting\. To calculate the corrected PU loss in biased scenarios, we consider two cases: known true propensity scores and estimated propensity scores\.

#### Case 1: known true propensity scores\.

The true PN risk with real class labels is

RP​N​\(g∣y\)\\displaystyle R\_\{PN\}\(g\\mid y\)=π​R^P​\(g,\+1\)\+\(1−π\)​R^N​\(g,−1\)\\displaystyle=\\pi\\widehat\{R\}\_\{P\}\(g,\+1\)\+\(1\-\\pi\)\\widehat\{R\}\_\{N\}\(g,\-1\)\(8\)=1n​∑i=1n\[yi​L​\(g​\(xi\),\+1\)\+\(1−yi\)​L​\(g​\(xi\),−1\)\]\.\\displaystyle=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\left\[y\_\{i\}L\(g\(x\_\{i\}\),\+1\)\+\(1\-y\_\{i\}\)L\(g\(x\_\{i\}\),\-1\)\\right\]\.
The following estimator adapts the propensity\-weighted SAR\-PU estimator of Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]using normalized weights and PUe risk notation\.

###### Definition 3\(PUe normalized IPW estimator\)\.

Given propensity scoreseeand PU labels, the inverse probability weighting estimator ofR^P​U​e​\(g\)\\widehat\{R\}\_\{PUe\}\(g\)is

R^P​U​e​\(g\)=π​R^Pe​\(g,\+1\)\+R^U​\(g,−1\)−π​R^Pe​\(g,−1\)\.\\widehat\{R\}\_\{PUe\}\(g\)=\\pi\\widehat\{R\}\_\{P\}^\{e\}\(g,\+1\)\+\\widehat\{R\}\_\{U\}\(g,\-1\)\-\\pi\\widehat\{R\}\_\{P\}^\{e\}\(g,\-1\)\.\(9\)

![Refer to caption](https://arxiv.org/html/2607.13428v1/figures/figure2_pue_framework.jpg)Figure 2:Overview of the PUe framework\. The goal is to estimate the propensity score of labeled samples and modify sample weights using normalized inverse probability weighting to obtain a loss\-function estimator for biased PU learning\.The corresponding unnormalized known\-propensity IPW estimator is unbiased under the SAR\-PU analysis of Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]\. PUe uses normalized inverse\-propensity weights rather than unnormalized reciprocal weights\. Therefore, for the normalized PUe estimator, we state the effect of normalization as a controlled deviation\. Lethg​\(x\)=L​\(g​\(x\),\+1\)−L​\(g​\(x\),−1\)h\_\{g\}\(x\)=L\(g\(x\),\+1\)\-L\(g\(x\),\-1\),ri=1/e​\(xi\)r\_\{i\}=1/e\(x\_\{i\}\), andWe=∑i=1nsi​riW\_\{e\}=\\sum\_\{i=1\}^\{n\}s\_\{i\}r\_\{i\}\. The normalized positive correction and the oracle\-normalized IPW correction satisfy

μe​\(hg\)=∑i=1nsi​ri​hg​\(xi\)We,μN​\(hg\)=1NP​∑i=1nsi​ri​hg​\(xi\),\\mu\_\{e\}\(h\_\{g\}\)=\\frac\{\\sum\_\{i=1\}^\{n\}s\_\{i\}r\_\{i\}h\_\{g\}\(x\_\{i\}\)\}\{W\_\{e\}\},\\qquad\\mu\_\{N\}\(h\_\{g\}\)=\\frac\{1\}\{N\_\{P\}\}\\sum\_\{i=1\}^\{n\}s\_\{i\}r\_\{i\}h\_\{g\}\(x\_\{i\}\),and therefore

μe​\(hg\)−μN​\(hg\)=\(NP−WeWe\)​μN​\(hg\)\.\\mu\_\{e\}\(h\_\{g\}\)\-\\mu\_\{N\}\(h\_\{g\}\)=\\left\(\\frac\{N\_\{P\}\-W\_\{e\}\}\{W\_\{e\}\}\\right\)\\mu\_\{N\}\(h\_\{g\}\)\.Thus, when\|L​\(g​\(x\),y\)\|≤Lmax\|L\(g\(x\),y\)\|\\leq L\_\{\\max\}andWe\>0W\_\{e\}\>0,

\|μe​\(hg\)−μN​\(hg\)\|≤2​Lmax​\|We−NP\|We\.\\left\|\\mu\_\{e\}\(h\_\{g\}\)\-\\mu\_\{N\}\(h\_\{g\}\)\\right\|\\leq 2L\_\{\\max\}\\frac\{\|W\_\{e\}\-N\_\{P\}\|\}\{W\_\{e\}\}\.This decomposition shows that the additional effect introduced by normalization is controlled by the deviation between the sample normalizerWeW\_\{e\}and the positive\-population normalizerNPN\_\{P\}\. When this normalizer gap vanishes, the normalization\-induced deviation also vanishes\. Appendix[E\.6](https://arxiv.org/html/2607.13428#A5.SS6)extends this calculation to estimated propensity scores\.

###### Theorem 1\(Known\-propensity IPW error bound,\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]111This theorem was originally published by\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]as Proposition 1 \(Propensity\-Weighted Estimator Bound\), but was adapted for the notation used in this paper\.\)\.

LetR^I​P​W​\(g\)\\widehat\{R\}\_\{IPW\}\(g\)denote the corresponding unnormalized known\-propensity IPW estimator\. For any predicted classesy^\\widehat\{y\}and real labelsyy, with probability at least1−η1\-\\eta,R^I​P​W​\(g\)\\widehat\{R\}\_\{IPW\}\(g\)does not differ from the true PN lossRP​N​\(g∣y\)R\_\{PN\}\(g\\mid y\)by more than

\|RP​N\(g∣y\)−R^I​P​W\(g\)\|≤Lmax2​ln⁡\(2/η\)2​n\.\\left\|R\_\{PN\}\(g\\mid y\)\-\\widehat\{R\}\_\{IPW\}\(g\)\\right\|\\leq\\sqrt\{\\frac\{L\_\{\\max\}^\{2\}\\ln\(2/\\eta\)\}\{2n\}\}\.\(10\)HereLmaxL\_\{\\max\}is the maximum absolute value of the cost functionL​\(g,y\)L\(g,y\)\.

Using the Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]\-adapted known\-propensity IPW bound above as a reference point, the next result compares the IPW correction used by PUe with the ordinary PU estimator under biased labeling\.

###### Theorem 2\(Error bounds of common PU algorithm in biased scenarios\)\.

For any predicted classesy^\\widehat\{y\}and real labelsyy, with probability at least1−η1\-\\eta, the common PU estimatorR^P​U​\(g\)\\widehat\{R\}\_\{PU\}\(g\)in biased scenarios satisfies

\|RP​N\(g∣y\)−R^P​U\(g\)\|≤2πLmaxnUnU\+nP\+Lmax2​ln⁡\(2/η\)2​n\.\\left\|R\_\{PN\}\(g\\mid y\)\-\\widehat\{R\}\_\{PU\}\(g\)\\right\|\\leq 2\\pi L\_\{\\max\}\\frac\{n\_\{U\}\}\{n\_\{U\}\+n\_\{P\}\}\+\\sqrt\{\\frac\{L\_\{\\max\}^\{2\}\\ln\(2/\\eta\)\}\{2n\}\}\.\(11\)

In biased scenarios, this bound is larger than the known\-propensity IPW reference bound, apart from the controlled normalization term above\.

#### Case 2: estimated propensity scores\.

In practice, the probability of a sample being labeled is usually unknown\. We therefore estimate the propensity score:

R^P​U​e^​\(g\)=π​R^Pe^​\(g,\+1\)\+R^U​\(g,−1\)−π​R^Pe^​\(g,−1\)\.\\widehat\{R\}\_\{PU\\widehat\{e\}\}\(g\)=\\pi\\widehat\{R\}\_\{P\}^\{\\widehat\{e\}\}\(g,\+1\)\+\\widehat\{R\}\_\{U\}\(g,\-1\)\-\\pi\\widehat\{R\}\_\{P\}^\{\\widehat\{e\}\}\(g,\-1\)\.\(12\)The following bias expression is adapted from the estimated\-propensity\-score analysis of Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\], while here we study it under normalized weights and deep propensity\-score estimation\. Letωi\\omega\_\{i\}andω^i\\widehat\{\\omega\}\_\{i\}denote the corresponding normalized inverse weights computed frome​\(xi\)e\(x\_\{i\}\)ande^​\(xi\)\\widehat\{e\}\(x\_\{i\}\):

bias⁡\(R^P​U​e^​\(g\)\)=π​∑i=1nyi​\[\(1NP−ω^iNP​ωi\)​\(L​\(g​\(xi\),\+1\)−L​\(g​\(xi\),−1\)\)\]\.\\operatorname\{bias\}\\left\(\\widehat\{R\}\_\{PU\\widehat\{e\}\}\(g\)\\right\)=\\pi\\sum\_\{i=1\}^\{n\}y\_\{i\}\\left\[\\left\(\\frac\{1\}\{N\_\{P\}\}\-\\frac\{\\widehat\{\\omega\}\_\{i\}\}\{N\_\{P\}\\omega\_\{i\}\}\\right\)\\left\(L\(g\(x\_\{i\}\),\+1\)\-L\(g\(x\_\{i\}\),\-1\)\\right\)\\right\]\.\(13\)From the bias, propensity scores need to be accurate for positive examples\. When an incorrect propensity score is close to 0 or 1, especially close to 0, the bias can become large\. Underestimated propensity scores are expected to result in a model with higher bias\.

We estimate propensity scores by training a binary network and adding regularization:

e^​\(x\)=arg⁡mine⁡π1nP​∑i=1nPL​\(e​\(xiP\),\+1\)\+1−π1nU​∑i=1nUL​\(e​\(xiU\),−1\)\+αe​\|∑xi∈χP∪χUe​\(xi\)−nP\|,\\widehat\{e\}\(x\)=\\arg\\min\_\{e\}\\frac\{\\pi\_\{1\}\}\{n\_\{P\}\}\\sum\_\{i=1\}^\{n\_\{P\}\}L\(e\(x\_\{i\}^\{P\}\),\+1\)\+\\frac\{1\-\\pi\_\{1\}\}\{n\_\{U\}\}\\sum\_\{i=1\}^\{n\_\{U\}\}L\(e\(x\_\{i\}^\{U\}\),\-1\)\+\\alpha\_\{e\}\\left\|\\sum\_\{x\_\{i\}\\in\\chi\_\{P\}\\cup\\chi\_\{U\}\}e\(x\_\{i\}\)\-n\_\{P\}\\right\|,\(14\)whereπ1=nP/\(nP\+nU\)\\pi\_\{1\}=n\_\{P\}/\(n\_\{P\}\+n\_\{U\}\)\. The regularization prevents degenerate estimates in which labeled samples are close to 1 and unlabeled samples are close to 0\.

According to\[[10](https://arxiv.org/html/2607.13428#bib.bib10),[1](https://arxiv.org/html/2607.13428#bib.bib1)\], one can use propensity scores for classification through

P​\(Y^∣E,L\)=1n​∑i=1nsi​\(1ei​δ1\+\(1−1ei\)​δ0\)\+\(1−si\)​δ0\.P\(\\widehat\{Y\}\\mid E,L\)=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}s\_\{i\}\\left\(\\frac\{1\}\{e\_\{i\}\}\\delta\_\{1\}\+\\left\(1\-\\frac\{1\}\{e\_\{i\}\}\\right\)\\delta\_\{0\}\\right\)\+\(1\-s\_\{i\}\)\\delta\_\{0\}\.However, underestimated propensity scores reduce classification performance\. PUe combines normalized inverse\-propensity weights with cost\-sensitive PU learning\. The basic PUe loss is

R^P​U​e^​\(g\)=π​R^Pe^​\(g,\+1\)\+R^U​\(g,−1\)−π​R^Pe^​\(g,−1\)\.\\widehat\{R\}\_\{PU\\widehat\{e\}\}\(g\)=\\pi\\widehat\{R\}\_\{P\}^\{\\widehat\{e\}\}\(g,\+1\)\+\\widehat\{R\}\_\{U\}\(g,\-1\)\-\\pi\\widehat\{R\}\_\{P\}^\{\\widehat\{e\}\}\(g,\-1\)\.\(15\)The algorithm first estimates positive\-sample propensity scores, then modifies positive\-sample weights with normalized propensity scores, and finally applies a PU algorithm with the modified weights\. We also consider selectively labeled negative classes through PUbNe\.

## 3Experiment

We experiment on several common PU methods and compare PUe against the corresponding baselines\.

### 3\.1Experimental settings

Datasets\.We use MNIST for parity classification, CIFAR\-10\[[16](https://arxiv.org/html/2607.13428#bib.bib16)\]for vehicle class recognition, and the Alzheimer’s dataset from Kaggle for real\-world evaluation\. On simulated MNIST and CIFAR\-10 settings, the true propensity score is known a priori, allowing comparison with oracle\-propensity variants\.

Baselines\.We mainly consider uPU\[[17](https://arxiv.org/html/2607.13428#bib.bib17)\], nnPU\[[11](https://arxiv.org/html/2607.13428#bib.bib11)\], PUbN\[[18](https://arxiv.org/html/2607.13428#bib.bib18)\], and Dist\-PU\[[7](https://arxiv.org/html/2607.13428#bib.bib7)\]\.

Evaluation metrics\.We report accuracy \(ACC\), precision \(Prec\.\), recall \(Rec\.\), F1, area under the ROC curve \(AUC\), and average precision \(AP\)\. Experiments are repeated with six random seeds; means and standard deviations are reported\.

Implementation details\.All experiments are run in PyTorch\. The batch size is 256 for MNIST and CIFAR\-10, and 128 for Alzheimer\. We use Adam with cosine annealing, initial learning rate5×10−35\\times 10^\{\-3\}, and weight decay5×10−35\\times 10^\{\-3\}\. PU methods first use a 60\-epoch warm\-up phase and then train another 60 epochs;α\\alphais searched in\[0,20\]\[0,20\]\.

Table 1:Comparative results on MNIST, CIFAR\-10, and Alzheimer\.
### 3\.2Comparison with state\-of\-the\-art methods

The results on all datasets are recorded in Table[1](https://arxiv.org/html/2607.13428#S3.T1)\. In most metrics, the proposed PUe variants outperform the corresponding competitors on biased datasets, improving the original PU method by about 1% to 5%\. A model using known propensity scores is not necessarily the best; a model with estimated propensity scores can perform better in many cases, which is consistent with observations in causal inference\[[15](https://arxiv.org/html/2607.13428#bib.bib15)\]\. In the ablation study, the performance of PUe remains comparable to that of the advanced PU algorithm even when labels are evenly distributed\.

### 3\.3Ablation studies

Effectiveness of hyper\-parameters\.Ablation experiments verify the validity of hyperparameters, as shown in Figure[3](https://arxiv.org/html/2607.13428#S3.F3)\. PUe is sensitive toαe\\alpha\_\{e\}and does not change monotonically\. In the original experiment,αe=15\\alpha\_\{e\}=15gives the best performance\.

![Refer to caption](https://arxiv.org/html/2607.13428v1/figures/figure3_alpha_cifar10.jpg)Figure 3:Influence of differentαe\\alpha\_\{e\}values on CIFAR\-10 with Dist\-PUe\.Effectiveness of labeled\-sample distribution\.Table[2](https://arxiv.org/html/2607.13428#S3.T2)shows that label\-distribution deviation significantly affects the improvement from PUe\. In general, when labeled samples are more biased, PUe provides larger improvements\. When deviation is extremely large, the improvement weakens because some classes have too few labeled samples, making propensity scores close to 0 and affecting model performance\.

Table 2:Ablation results on CIFAR\-10 with checkmark indicating the enabling of the corresponding regularization loss term and different labeled distributions\.

## 4Conclusion

This paper proposes PUe, a PU learning method from the perspective of propensity scores for PU learning with biased labels in deep learning\. PUe enhances original cost\-sensitive PU algorithms, improves prediction precision under biased sample labeling, and has the degenerate ability that prediction precision under unbiased labeling is not lower than that of the original PU algorithm\. PUe consistently outperforms state\-of\-the\-art methods on most metrics on biased labeled datasets, including MNIST, CIFAR\-10, and Alzheimer’s\. We hope that the proposed propensity\-score estimation scheme for deep learning can also provide inspiration for other weakly supervised scenarios, especially those where label distribution is unknown\.

## Acknowledgement

This work was supported by Huawei Noah’s Ark Lab\.

## References

- \[1\]Jessa Bekker, Pieter Robberechts, and Jesse Davis\.Beyond the selected completely at random assumption for learning from positive and unlabeled data\.InMachine Learning and Knowledge Discovery in Databases, volume 11907 ofLecture Notes in Computer Science, pages 71–85\. Springer, 2020\.ECML PKDD 2019\.
- \[2\]Bing Liu, Wee Sun Lee, Philip S\. Yu, and Xiaoli Li\.Partially supervised classification of text documents\.InProceedings of the 19th International Conference on Machine Learning, pages 387–394\. Morgan Kaufmann, 2002\.
- \[3\]Bing Liu, Yang Dai, Xiaoli Li, Wee Sun Lee, and Philip S\. Yu\.Building text classifiers using positive and unlabeled examples\.InProceedings of the Third IEEE International Conference on Data Mining, pages 179–186\. IEEE, 2003\.
- \[4\]Charles Elkan and Keith Noto\.Learning classifiers from only positive and unlabeled data\.InProceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 213–220\. ACM, 2008\.
- \[5\]Xuxi Chen, Wuyang Chen, Tianlong Chen, Ye Yuan, Chen Gong, Kewei Chen, and Zhangyang Wang\.Self\-PU: Self boosted and calibrated positive\-unlabeled training\.InProceedings of the 37th International Conference on Machine Learning, volume 119 ofProceedings of Machine Learning Research, pages 1510–1519\. PMLR, 2020\.
- \[6\]Guangxin Su, Weitong Chen, and Miao Xu\.Positive\-unlabeled learning from imbalanced data\.InProceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pages 2995–3001\. International Joint Conferences on Artificial Intelligence Organization, 2021\.
- \[7\]Yunrui Zhao, Qianqian Xu, Yangbangyan Jiang, Peisong Wen, and Qingming Huang\.Dist\-PU: Positive\-unlabeled learning from a label distribution perspective\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14461–14470, 2022\.
- \[8\]Masahiro Kato, Takeshi Teshima, and Junya Honda\.Learning from positive and unlabeled data with a selection bias\.InInternational Conference on Learning Representations, 2019\.
- \[9\]Antti Hyttinen, Frederick Eberhardt, and Patrik O\. Hoyer\.Experiment selection for causal discovery\.Journal of Machine Learning Research, 14\(1\):3041–3071, 2013\.
- \[10\]Walter Gerych, Thomas Hartvigsen, Luke Buquicchio, Emmanuel Agu, and Elke A\. Rundensteiner\.Recovering the propensity score from biased positive unlabeled data\.InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6694–6702, 2022\.
- \[11\]Ryuichi Kiryo, Gang Niu, Marthinus C\. du Plessis, and Masashi Sugiyama\.Positive\-unlabeled learning with non\-negative risk estimator\.InAdvances in Neural Information Processing Systems, volume 30, pages 1675–1685, 2017\.
- \[12\]Shantanu Jain, Justin Delano, Himanshu Sharma, and Predrag Radivojac\.Class prior estimation with biased positives and unlabeled examples\.InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4255–4263, 2020\.
- \[13\]Jessa Bekker and Jesse Davis\.Positive and unlabeled relational classification through label frequency estimation\.InInductive Logic Programming, volume 11105 ofLecture Notes in Computer Science, pages 16–30\. Springer, 2018\.
- \[14\]Paul R\. Rosenbaum and Donald B\. Rubin\.The central role of the propensity score in observational studies for causal effects\.Biometrika, 70\(1\):41–55, 1983\.
- \[15\]Keisuke Hirano, Guido W\. Imbens, and Geert Ridder\.Efficient estimation of average treatment effects using the estimated propensity score\.Econometrica, 71\(4\):1161–1189, 2003\.
- \[16\]Alex Krizhevsky and Geoffrey Hinton\.Learning multiple layers of features from tiny images\.Technical report, University of Toronto, 2009\.
- \[17\]Marthinus C\. du Plessis, Gang Niu, and Masashi Sugiyama\.Analysis of learning from positive and unlabeled data\.InAdvances in Neural Information Processing Systems, volume 27, pages 703–711, 2014\.
- \[18\]Yu\-Guan Hsieh, Gang Niu, and Masashi Sugiyama\.Classification from positive, unlabeled and biased negative data\.InProceedings of the 36th International Conference on Machine Learning, volume 97 ofProceedings of Machine Learning Research, pages 2820–2829\. PMLR, 2019\.

## Appendix AAlgorithm

Table 3:PUe algorithm\.Require:dataχP,χU\\chi\_\{P\},\\chi\_\{U\}, sizesn,nP,nUn,n\_\{P\},n\_\{U\}, hyperparametersαe,π\\alpha\_\{e\},\\pi\.1\. Estimatee^​\(x\)\\widehat\{e\}\(x\)by minimizing Eq\. \([14](https://arxiv.org/html/2607.13428#S2.E14)\)\.2\. Compute the weight of labeled samples:wiP=π​ω^iPw\_\{i\}^\{P\}=\\pi\\widehat\{\\omega\}\_\{i\}^\{P\}\.3\. For each training iteration, shuffle\(χP,χU\)\(\\chi\_\{P\},\\chi\_\{U\}\)into mini\-batches\.4\. For each mini\-batch\(χjP,χjU\)\(\\chi\_\{j\}^\{P\},\\chi\_\{j\}^\{U\}\), compute the correspondingR^P​U​e​\(g\)\\widehat\{R\}\_\{PUe\}\(g\)\.5\. Updateθ\\thetawith gradient information∇θR^P​U​e​\(g\)\\nabla\_\{\\theta\}\\widehat\{R\}\_\{PUe\}\(g\)\.
## Appendix BExperiment details

Table 4:Summary of used datasets and their corresponding models\.
## Appendix CComplementary experiment

Table 5:Supplemental experiments on MNIST\.LRe denotes logistic\-regression estimation of propensity scores for PU learning\. According to Gerych et al\.\[[10](https://arxiv.org/html/2607.13428#bib.bib10)\], identifiable propensity\-score estimation requires assumptions about the data\. The results above show that PUe improves over self\-supervised and linear\-estimation baselines in the biased\-label setting\.

## Appendix DAdditional Discussion of Related SAR\-PU Work

This appendix summarizes the relationship between PUe and the SAR\-PU framework of Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]\. The main text identifies Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]as the closest prior work; Table[6](https://arxiv.org/html/2607.13428#A4.T6)provides a compact component\-level comparison\.

Table 6:Component\-level comparison with related SAR\-PU work\.
## Appendix EProofs

### E\.1Error bound of bias

Assume that the propensity\-score estimate has a maximum error ratioβ\\beta, with

β​e​\(xiL\)≤e^​\(xiL\)≤e​\(xiL\)\.\\beta e\(x\_\{i\}^\{L\}\)\\leq\\widehat\{e\}\(x\_\{i\}^\{L\}\)\\leq e\(x\_\{i\}^\{L\}\)\.The samplexiLx\_\{i\}^\{L\}has weight1/\(n​e^​\(xiL\)\)1/\(n\\widehat\{e\}\(x\_\{i\}^\{L\}\)\), whose error is bounded by

bias⁡\(1n​e^​\(xiL\)\)≤1n​e​\(xiL\)​\(1β−1\)\.\\operatorname\{bias\}\\left\(\\frac\{1\}\{n\\widehat\{e\}\(x\_\{i\}^\{L\}\)\}\\right\)\\leq\\frac\{1\}\{ne\(x\_\{i\}^\{L\}\)\}\\left\(\\frac\{1\}\{\\beta\}\-1\\right\)\.

### E\.2Error ratio

In PUe, samplexiLx\_\{i\}^\{L\}has normalized weight

π​1/e^​\(xiL\)∑j1/e^​\(xjL\)\.\\pi\\frac\{1/\\widehat\{e\}\(x\_\{i\}^\{L\}\)\}\{\\sum\_\{j\}1/\\widehat\{e\}\(x\_\{j\}^\{L\}\)\}\.LetP​\(γ​e​\(xiL\)<e^​\(xiL\)≤e​\(xiL\)\)=αP\(\\gamma e\(x\_\{i\}^\{L\}\)<\\widehat\{e\}\(x\_\{i\}^\{L\}\)\\leq e\(x\_\{i\}^\{L\}\)\)=\\alphaforS1S\_\{1\}andP​\(β​e​\(xiL\)<e^​\(xiL\)≤γ​e​\(xiL\)\)=1−αP\(\\beta e\(x\_\{i\}^\{L\}\)<\\widehat\{e\}\(x\_\{i\}^\{L\}\)\\leq\\gamma e\(x\_\{i\}^\{L\}\)\)=1\-\\alphaforS2S\_\{2\}, whereβ<γ<1\\beta<\\gamma<1\. If

∑i∈S11e​\(xiL\)=∑i∈S21e​\(xiL\)=B,\\sum\_\{i\\in S\_\{1\}\}\\frac\{1\}\{e\(x\_\{i\}^\{L\}\)\}=\\sum\_\{i\\in S\_\{2\}\}\\frac\{1\}\{e\(x\_\{i\}^\{L\}\)\}=B,then∑j1/e​\(xjL\)=2​B=NP\\sum\_\{j\}1/e\(x\_\{j\}^\{L\}\)=2B=N\_\{P\}\. ForxiL∈S1x\_\{i\}^\{L\}\\in S\_\{1\},

1e​\(xiL\)≤1e^​\(xiL\)<1γ​e​\(xiL\),\\frac\{1\}\{e\(x\_\{i\}^\{L\}\)\}\\leq\\frac\{1\}\{\\widehat\{e\}\(x\_\{i\}^\{L\}\)\}<\\frac\{1\}\{\\gamma e\(x\_\{i\}^\{L\}\)\},and forxiL∈S2x\_\{i\}^\{L\}\\in S\_\{2\},

1γ​e​\(xiL\)≤1e^​\(xiL\)<1β​e​\(xiL\)\.\\frac\{1\}\{\\gamma e\(x\_\{i\}^\{L\}\)\}\\leq\\frac\{1\}\{\\widehat\{e\}\(x\_\{i\}^\{L\}\)\}<\\frac\{1\}\{\\beta e\(x\_\{i\}^\{L\}\)\}\.Thus

B​\(1\+1γ\)≤∑j1e^​\(xjL\)<B​\(1γ\+1β\),B\\left\(1\+\\frac\{1\}\{\\gamma\}\\right\)\\leq\\sum\_\{j\}\\frac\{1\}\{\\widehat\{e\}\(x\_\{j\}^\{L\}\)\}<B\\left\(\\frac\{1\}\{\\gamma\}\+\\frac\{1\}\{\\beta\}\\right\),which bounds the normalized sample\-weight error and shows that normalization reduces the error ratio relative to unnormalized reciprocal weights\.

### E\.3Expectation

According to the propensity\-score definition, each labeled positive samplexjPx\_\{j\}^\{P\}corresponds in expectation to1/e​\(xjP\)1/e\(x\_\{j\}^\{P\}\)positive samples\. BecauseP​\(x∣s=1\)=P​\(x,y=1∣s=1\)P\(x\\mid s=1\)=P\(x,y=1\\mid s=1\),

𝔼P​\(x∣s=1\)​1P​\(s=1∣x,y=1\)\\displaystyle\\mathbb\{E\}\_\{P\(x\\mid s=1\)\}\\frac\{1\}\{P\(s=1\\mid x,y=1\)\}=∑xP​\(x,y=1∣s=1\)​1P​\(s=1∣x,y=1\)\\displaystyle=\\sum\_\{x\}P\(x,y=1\\mid s=1\)\\frac\{1\}\{P\(s=1\\mid x,y=1\)\}=∑xP​\(s=1∣x,y=1\)​P​\(x,y=1\)P​\(s=1\)​1P​\(s=1∣x,y=1\)\\displaystyle=\\sum\_\{x\}\\frac\{P\(s=1\\mid x,y=1\)P\(x,y=1\)\}\{P\(s=1\)\}\\frac\{1\}\{P\(s=1\\mid x,y=1\)\}=∑xP​\(x,y=1\)P​\(s=1\)=nnP​∑xP​\(x,y=1\)=NPnP\.\\displaystyle=\\sum\_\{x\}\\frac\{P\(x,y=1\)\}\{P\(s=1\)\}=\\frac\{n\}\{n\_\{P\}\}\\sum\_\{x\}P\(x,y=1\)=\\frac\{N\_\{P\}\}\{n\_\{P\}\}\.This indicates that∑j=1nP1/e​\(xjP\)=NP\\sum\_\{j=1\}^\{n\_\{P\}\}1/e\(x\_\{j\}^\{P\}\)=N\_\{P\}\.

### E\.4PUbN

Letσ​\(x\)=p​\(s=\+1∣x\)\\sigma\(x\)=p\(s=\+1\\mid x\), which is unknown and replaced byσ^​\(x\)\\widehat\{\\sigma\}\(x\)\. The PUbN risk is

RP​U​b​N​\(g\)=π​RP​\(g,\+1\)\+ρ​Rb​N​\(g,−1\)\+R¯s=−1,η,σ^​\(g\),R\_\{PUbN\}\(g\)=\\pi R\_\{P\}\(g,\+1\)\+\\rho R\_\{bN\}\(g,\-1\)\+\\overline\{R\}\_\{s=\-1,\\eta,\\widehat\{\\sigma\}\}\(g\),where

R¯s=−1,η,σ^​\(g\)\\displaystyle\\overline\{R\}\_\{s=\-1,\\eta,\\widehat\{\\sigma\}\}\(g\)=𝔼x∼p​\(x\)​\[𝟙σ^​\(x\)≤η​L​\(−g​\(x\)\)​\(1−σ^​\(x\)\)\]\\displaystyle=\\mathbb\{E\}\_\{x\\sim p\(x\)\}\\left\[\\mathbb\{1\}\_\{\\widehat\{\\sigma\}\(x\)\\leq\\eta\}L\(\-g\(x\)\)\(1\-\\widehat\{\\sigma\}\(x\)\)\\right\]\+π​𝔼x∼pP​\(x\)​\[𝟙σ^​\(x\)\>η​L​\(−g​\(x\)\)​1−σ^​\(x\)σ^​\(x\)\]\\displaystyle\\quad\+\\pi\\mathbb\{E\}\_\{x\\sim p\_\{P\}\(x\)\}\\left\[\\mathbb\{1\}\_\{\\widehat\{\\sigma\}\(x\)\>\\eta\}L\(\-g\(x\)\)\\frac\{1\-\\widehat\{\\sigma\}\(x\)\}\{\\widehat\{\\sigma\}\(x\)\}\\right\]\+ρ​𝔼x∼pb​N​\(x\)​\[𝟙σ^​\(x\)\>η​L​\(−g​\(x\)\)​1−σ^​\(x\)σ^​\(x\)\]\.\\displaystyle\\quad\+\\rho\\mathbb\{E\}\_\{x\\sim p\_\{bN\}\(x\)\}\\left\[\\mathbb\{1\}\_\{\\widehat\{\\sigma\}\(x\)\>\\eta\}L\(\-g\(x\)\)\\frac\{1\-\\widehat\{\\sigma\}\(x\)\}\{\\widehat\{\\sigma\}\(x\)\}\\right\]\.The corresponding empirical risk is

R^P​U​b​N,η,σ^​\(g\)=π​R^P​\(g,\+1\)\+ρ​R^b​N​\(g,−1\)\+R¯^s=−1,η,σ^​\(g\)\.\\widehat\{R\}\_\{PUbN,\\eta,\\widehat\{\\sigma\}\}\(g\)=\\pi\\widehat\{R\}\_\{P\}\(g,\+1\)\+\\rho\\widehat\{R\}\_\{bN\}\(g,\-1\)\+\\widehat\{\\overline\{R\}\}\_\{s=\-1,\\eta,\\widehat\{\\sigma\}\}\(g\)\.

### E\.5PUbNe

The PUbNe risk is

R^P​U​b​N​e^,η,σ^​\(g\)=π​R^Pe^​\(g,\+1\)\+ρ​R^b​Ne^​\(g,−1\)\+R¯^s=−1,η,σ^e^​\(g\),\\widehat\{R\}\_\{PUbN\\widehat\{e\},\\eta,\\widehat\{\\sigma\}\}\(g\)=\\pi\\widehat\{R\}\_\{P\}^\{\\widehat\{e\}\}\(g,\+1\)\+\\rho\\widehat\{R\}\_\{bN\}^\{\\widehat\{e\}\}\(g,\-1\)\+\\widehat\{\\overline\{R\}\}\_\{s=\-1,\\eta,\\widehat\{\\sigma\}\}^\{\\widehat\{e\}\}\(g\),where

R^b​Ne^​\(g,−1\)=∑i=1nb​Nω^ib​N​L​\(g​\(xib​N\),−1\)\.\\widehat\{R\}\_\{bN\}^\{\\widehat\{e\}\}\(g,\-1\)=\\sum\_\{i=1\}^\{n\_\{bN\}\}\\widehat\{\\omega\}\_\{i\}^\{bN\}L\(g\(x\_\{i\}^\{bN\}\),\-1\)\.

### E\.6Normalized\-weight decomposition

Lethg​\(x\)=L​\(g​\(x\),\+1\)−L​\(g​\(x\),−1\)h\_\{g\}\(x\)=L\(g\(x\),\+1\)\-L\(g\(x\),\-1\),ri=1/e​\(xi\)r\_\{i\}=1/e\(x\_\{i\}\), andr^i=1/e^​\(xi\)\\widehat\{r\}\_\{i\}=1/\\widehat\{e\}\(x\_\{i\}\)\. For the known\-propensity normalizerWe=∑i=1nsi​riW\_\{e\}=\\sum\_\{i=1\}^\{n\}s\_\{i\}r\_\{i\}, define

μe​\(hg\)=∑i=1nsi​ri​hg​\(xi\)We,μN​\(hg\)=1NP​∑i=1nsi​ri​hg​\(xi\)\.\\mu\_\{e\}\(h\_\{g\}\)=\\frac\{\\sum\_\{i=1\}^\{n\}s\_\{i\}r\_\{i\}h\_\{g\}\(x\_\{i\}\)\}\{W\_\{e\}\},\\qquad\\mu\_\{N\}\(h\_\{g\}\)=\\frac\{1\}\{N\_\{P\}\}\\sum\_\{i=1\}^\{n\}s\_\{i\}r\_\{i\}h\_\{g\}\(x\_\{i\}\)\.Then

μe​\(hg\)−μN​\(hg\)\\displaystyle\\mu\_\{e\}\(h\_\{g\}\)\-\\mu\_\{N\}\(h\_\{g\}\)=\(1We−1NP\)​∑i=1nsi​ri​hg​\(xi\)\\displaystyle=\\left\(\\frac\{1\}\{W\_\{e\}\}\-\\frac\{1\}\{N\_\{P\}\}\\right\)\\sum\_\{i=1\}^\{n\}s\_\{i\}r\_\{i\}h\_\{g\}\(x\_\{i\}\)=\(NP−WeWe\)​μN​\(hg\)\.\\displaystyle=\\left\(\\frac\{N\_\{P\}\-W\_\{e\}\}\{W\_\{e\}\}\\right\)\\mu\_\{N\}\(h\_\{g\}\)\.If\|L​\(g​\(x\),y\)\|≤Lmax\|L\(g\(x\),y\)\|\\leq L\_\{\\max\}, then\|hg​\(x\)\|≤2​Lmax\|h\_\{g\}\(x\)\|\\leq 2L\_\{\\max\}and

\|μe​\(hg\)−μN​\(hg\)\|≤2​Lmax​\|We−NP\|We\.\\left\|\\mu\_\{e\}\(h\_\{g\}\)\-\\mu\_\{N\}\(h\_\{g\}\)\\right\|\\leq 2L\_\{\\max\}\\frac\{\|W\_\{e\}\-N\_\{P\}\|\}\{W\_\{e\}\}\.
For estimated propensity scores, letΔi=r^i−ri\\Delta\_\{i\}=\\widehat\{r\}\_\{i\}\-r\_\{i\}andW^=We\+∑i=1nsi​Δi\\widehat\{W\}=W\_\{e\}\+\\sum\_\{i=1\}^\{n\}s\_\{i\}\\Delta\_\{i\}\. The estimated normalized correction

μ^​\(hg\)=∑i=1nsi​r^i​hg​\(xi\)W^\\widehat\{\\mu\}\(h\_\{g\}\)=\\frac\{\\sum\_\{i=1\}^\{n\}s\_\{i\}\\widehat\{r\}\_\{i\}h\_\{g\}\(x\_\{i\}\)\}\{\\widehat\{W\}\}satisfies the exact identity

μ^​\(hg\)−μe​\(hg\)=∑i=1nsi​Δi​\(hg​\(xi\)−μe​\(hg\)\)W^\.\\widehat\{\\mu\}\(h\_\{g\}\)\-\\mu\_\{e\}\(h\_\{g\}\)=\\frac\{\\sum\_\{i=1\}^\{n\}s\_\{i\}\\Delta\_\{i\}\\left\(h\_\{g\}\(x\_\{i\}\)\-\\mu\_\{e\}\(h\_\{g\}\)\\right\)\}\{\\widehat\{W\}\}\.Consequently, ifW^\>0\\widehat\{W\}\>0,

\|μ^​\(hg\)−μe​\(hg\)\|≤4​Lmax​∑i=1nsi​\|Δi\|W^\.\\left\|\\widehat\{\\mu\}\(h\_\{g\}\)\-\\mu\_\{e\}\(h\_\{g\}\)\\right\|\\leq\\frac\{4L\_\{\\max\}\\sum\_\{i=1\}^\{n\}s\_\{i\}\|\\Delta\_\{i\}\|\}\{\\widehat\{W\}\}\.This gives a PUe\-specific normalized\-weight error term: with normalized weights, the estimated\-propensity effect depends on reciprocal\-propensity errors relative to the estimated normalizer, rather than only on pointwise propensity errors\.

### E\.7Known\-propensity IPW identity and bounds

The following proof is for the corresponding unnormalized known\-propensity IPW estimator and follows the propensity\-weighted estimator analysis of Bekker et al\.\[[1](https://arxiv.org/html/2607.13428#bib.bib1)\]\. PUe then uses normalized weights, whose additional effect is controlled in Appendix[E\.6](https://arxiv.org/html/2607.13428#A5.SS6):

𝔼​\[R^I​P​W​\(g\)\]\\displaystyle\\mathbb\{E\}\[\\widehat\{R\}\_\{IPW\}\(g\)\]=𝔼​\[π​R^P,I​P​We​\(g,\+1\)\+R^U​\(g,−1\)−π​R^P,I​P​We​\(g,−1\)\]\\displaystyle=\\mathbb\{E\}\\left\[\\pi\\widehat\{R\}\_\{P,IPW\}^\{e\}\(g,\+1\)\+\\widehat\{R\}\_\{U\}\(g,\-1\)\-\\pi\\widehat\{R\}\_\{P,IPW\}^\{e\}\(g,\-1\)\\right\]=𝔼​\[1n​∑i=1nP1e​\(xiP\)​\(L​\(g​\(xiP\),\+1\)−L​\(g​\(xiP\),−1\)\)\+1n​∑i=1nL​\(g​\(xi\),−1\)\]\\displaystyle=\\mathbb\{E\}\\left\[\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\_\{P\}\}\\frac\{1\}\{e\(x\_\{i\}^\{P\}\)\}\\left\(L\(g\(x\_\{i\}^\{P\}\),\+1\)\-L\(g\(x\_\{i\}^\{P\}\),\-1\)\\right\)\+\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}L\(g\(x\_\{i\}\),\-1\)\\right\]=1n∑i=1n\[yiei1e​\(xi\)L\(g\(xi\),\+1\)\\displaystyle=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\Bigg\[y\_\{i\}e\_\{i\}\\frac\{1\}\{e\(x\_\{i\}\)\}L\(g\(x\_\{i\}\),\+1\)\+yi​ei​\(1−1e​\(xi\)\)​L​\(g​\(xi\),−1\)\\displaystyle\\qquad\+y\_\{i\}e\_\{i\}\\left\(1\-\\frac\{1\}\{e\(x\_\{i\}\)\}\\right\)L\(g\(x\_\{i\}\),\-1\)\+\(1−yiei\)L\(g\(xi\),−1\)\]\\displaystyle\\qquad\+\(1\-y\_\{i\}e\_\{i\}\)L\(g\(x\_\{i\}\),\-1\)\\Bigg\]=1n​∑i=1n\[yi​L​\(g​\(xi\),\+1\)\+\(1−yi\)​L​\(g​\(xi\),−1\)\]\\displaystyle=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\left\[y\_\{i\}L\(g\(x\_\{i\}\),\+1\)\+\(1\-y\_\{i\}\)L\(g\(x\_\{i\}\),\-1\)\\right\]=RP​N​\(g∣y\)\.\\displaystyle=R\_\{PN\}\(g\\mid y\)\.The change ofR^I​P​W​\(g\)\\widehat\{R\}\_\{IPW\}\(g\)is no more thanLmax/nL\_\{\\max\}/nif somexi∈χP∪χUx\_\{i\}\\in\\chi\_\{P\}\\cup\\chi\_\{U\}is replaced\. McDiarmid’s inequality gives

Pr\{\|R^I​P​W\(g\)−RP​N\(g∣y\)\|≥ϵ\}≤2exp\(−2​ϵ2n​\(Lmax/n\)2\)\.\\Pr\\left\\\{\|\\widehat\{R\}\_\{IPW\}\(g\)\-R\_\{PN\}\(g\\mid y\)\|\\geq\\epsilon\\right\\\}\\leq 2\\exp\\left\(\-\\frac\{2\\epsilon^\{2\}\}\{n\(L\_\{\\max\}/n\)^\{2\}\}\\right\)\.Setting the right side equal toη\\etayields

ϵ=Lmax2​ln⁡\(2/η\)2​n,\\epsilon=\\sqrt\{\\frac\{L\_\{\\max\}^\{2\}\\ln\(2/\\eta\)\}\{2n\}\},and hence, with probability at least1−η1\-\\eta,

\|R^I​P​W\(g\)−RP​N\(g∣y\)\|≤Lmax2​ln⁡\(2/η\)2​n\.\\left\|\\widehat\{R\}\_\{IPW\}\(g\)\-R\_\{PN\}\(g\\mid y\)\\right\|\\leq\\sqrt\{\\frac\{L\_\{\\max\}^\{2\}\\ln\(2/\\eta\)\}\{2n\}\}\.
Because

R^I​P​W​\(g\)=π​R^P,I​P​We​\(g,\+1\)\+R^U​\(g,−1\)−π​R^P,I​P​We​\(g,−1\)\\widehat\{R\}\_\{IPW\}\(g\)=\\pi\\widehat\{R\}\_\{P,IPW\}^\{e\}\(g,\+1\)\+\\widehat\{R\}\_\{U\}\(g,\-1\)\-\\pi\\widehat\{R\}\_\{P,IPW\}^\{e\}\(g,\-1\)and

R^P​U​\(g\)=π​R^P​\(g,\+1\)\+R^U​\(g,−1\)−π​R^P​\(g,−1\),\\widehat\{R\}\_\{PU\}\(g\)=\\pi\\widehat\{R\}\_\{P\}\(g,\+1\)\+\\widehat\{R\}\_\{U\}\(g,\-1\)\-\\pi\\widehat\{R\}\_\{P\}\(g,\-1\),we obtain the biased\-labeling comparison term

\|R^I​P​W​\(g\)−R^P​U​\(g\)\|≤2​π​Lmax​nUnU\+nP\.\|\\widehat\{R\}\_\{IPW\}\(g\)\-\\widehat\{R\}\_\{PU\}\(g\)\|\\leq 2\\pi L\_\{\\max\}\\frac\{n\_\{U\}\}\{n\_\{U\}\+n\_\{P\}\}\.Therefore,

\|RP​N\(g∣y\)−R^P​U\(g\)\|≤2πLmaxnUnU\+nP\+Lmax2​ln⁡\(2/η\)2​n\.\\left\|R\_\{PN\}\(g\\mid y\)\-\\widehat\{R\}\_\{PU\}\(g\)\\right\|\\leq 2\\pi L\_\{\\max\}\\frac\{n\_\{U\}\}\{n\_\{U\}\+n\_\{P\}\}\+\\sqrt\{\\frac\{L\_\{\\max\}^\{2\}\\ln\(2/\\eta\)\}\{2n\}\}\.