Importance Weighting for Unlabeled-unlabeled Learning under Distribution Shift

# Importance Weighting for Unlabeled-unlabeled Learning under Distribution Shift
Source: [https://arxiv.org/html/2609.10994](https://arxiv.org/html/2609.10994)
Tomoharu IwataHiroshi TakahashiAffiliation:Taishi Nishiyama,Kazuki Adachi,Yasuhiro FujiwaraAffiliation:NTT, Inc\.Email:[atsutoshi\.kumagai@ntt\.com](mailto:)

###### Abstract

Unlabeled\-unlabeled \(UU\) learning allows us to learn a binary classifier from two sets of unlabeled data with different class\-priors\. It is a general framework because it includes a wide variety of supervised learning such as positive\-unlabeled \(PU\) learning, noisy label learning, and similarity\-based learning\. Existing UU learning assumes that the test and training distributions have the same class\-conditional densities\. However, this assumption rarely holds in practice due to distribution shifts\. This paper proposes a distribution shift adaptation method for UU learning that uses UU data in the training distribution and a few UU data in the test distribution\. The proposed method is based on the importance weighting, which minimizes the test risk by using training data with estimated importance weights\. Although existing importance weighting methods cannot handle UU data, we show that it can be done in a principled manner\. Thanks to the generality of UU learning, our method can handle various learning problems such as PU and noisy label learning under distribution shift within a single framework while existing methods are usually tailored to a specific problem\. Moreover, it does not require any assumption of the shift types such as covariate shift\. We experimentally demonstrate the effectiveness of the proposed method with real\-world datasets\.

## 1Introduction

In supervised learning, a large amount of labeled data is required for learning accurate classifiers\. However, in practice, labeled data are often very costly or even infeasible to collect\. This fact has motivated us to investigate learning algorithms that work well from data with weak supervision \(e\.g\., noisy labels\), which are often easier to collect than clean and complete labeled data\[[50](https://arxiv.org/html/2609.10994#bib.bib50)\]\. In this paper, we consider unlabeled\-unlabeled \(UU\) learning where the aim is to learn a binary classifier from two unlabeled datasets that share the class\-conditional densities but have different class\-priors \(i\.e\., the proportion of positives in each unlabeled data\)\[[33](https://arxiv.org/html/2609.10994#bib.bib33),[35](https://arxiv.org/html/2609.10994#bib.bib35),[32](https://arxiv.org/html/2609.10994#bib.bib32)\]\. In this setting, class\-priors can be regarded as weak supervision\. We focus on this challenging setting because of its high generality: it can represent a wide variety of learning problems, such as positive\-negative \(PN\) learning\[[54](https://arxiv.org/html/2609.10994#bib.bib54)\], positive\-unlabeled \(PU\) learning\[[6](https://arxiv.org/html/2609.10994#bib.bib6),[20](https://arxiv.org/html/2609.10994#bib.bib20),[2](https://arxiv.org/html/2609.10994#bib.bib2)\], noisy label learning\[[58](https://arxiv.org/html/2609.10994#bib.bib58),[4](https://arxiv.org/html/2609.10994#bib.bib4)\], pairwise\-comparison learning\[[10](https://arxiv.org/html/2609.10994#bib.bib10)\], and similarity\-based learning\[[1](https://arxiv.org/html/2609.10994#bib.bib1),[45](https://arxiv.org/html/2609.10994#bib.bib45)\], by specifying values of the class\-priors\[[5](https://arxiv.org/html/2609.10994#bib.bib5)\]\.

Although several UU learning methods have recently been proposed\[[33](https://arxiv.org/html/2609.10994#bib.bib33),[35](https://arxiv.org/html/2609.10994#bib.bib35),[32](https://arxiv.org/html/2609.10994#bib.bib32),[56](https://arxiv.org/html/2609.10994#bib.bib56)\], they assume that the test and training distributions have the same class\-conditional densities\. However, in practice, this assumption is often violated by various factors such as differences in data collection periods\[[27](https://arxiv.org/html/2609.10994#bib.bib27)\]and environments\[[24](https://arxiv.org/html/2609.10994#bib.bib24)\]\. For example, in cyber security, \(noisy\) labeled malicious and normal data can be regarded as UU data, and the data distribution can vary over time by the emergence of new types of attacks\[[26](https://arxiv.org/html/2609.10994#bib.bib26)\]\. When such a distribution shift occurs, the performances of existing methods drastically deteriorate\.

![Refer to caption](https://arxiv.org/html/2609.10994v1/uujoint_fig_ps4.png)Figure 1:Our problem setting\. We are given UU data in the training distribution and a few UU data in the test distribution\. UU data in each distribution shares the class\-conditional densities but have different class\-priors \(i\.e\.,θtra≠θtrb\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\\neq\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}andθtea≠θteb\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\neq\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\)\. The class\-conditional densities can vary between the training and test distributions\. The aim is to learn a binary classifier that can accurately classify test data drawn from the test distribution with true class\-priorπte\\pi\_\{\{\\rm te\}\}by using these UU datasets\. More details of our problem setting are provided in subsection[4\.1](https://arxiv.org/html/2609.10994#S4.SS1)\.In this paper, we propose a distribution shift adaptation method for UU learning that uses UU data in the training distribution and a few UU data in the test distribution111Although the test \(training\) distributions are sometimes referred to as the target \(source\) distributions, this paper uses the former terminology following previous distribution shift adaptation studies\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\.\. A few UU data are relatively easy to collect even in the test distribution\. Figure[1](https://arxiv.org/html/2609.10994#S1.F1)shows the overview of our problem setting\. Our method is based on the importance weighting, which is a well\-established and commonly used framework for distribution shift adaptation on ordinary supervised learning\[[51](https://arxiv.org/html/2609.10994#bib.bib51),[8](https://arxiv.org/html/2609.10994#bib.bib8),[34](https://arxiv.org/html/2609.10994#bib.bib34)\]\. This framework estimates importance weights, i\.e\., the ratio between training and test densities, and learns classifiers by minimizing the importance\-weighted empirical training risk that is approximately equivalent to the test risk\. Although many distribution shift adaptation methods have been proposed for settings such as PN learning\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[8](https://arxiv.org/html/2609.10994#bib.bib8)\]and PU learning\[[27](https://arxiv.org/html/2609.10994#bib.bib27)\], no existing methods can treat UU data\. To the best of our knowledge, we are the first to show that importance weighting can be performed from UU data in a principled manner\. By minimizing a weighted sum of the importance\-weighted empirical risk with UU data in the training distribution and the empirical risk with UU data in the test distribution, the proposed method can train classifiers that fit on the test distribution\.

Thanks to the generality of UU learning, the proposed method can handle various types of learning problems under distribution shift within a single framework, including PN, PU, noisy label, pairwise\-comparison, and similarity\-based learning\. This contrasts with existing distribution shift adaptation methods, which are typically tailored to a specific problem\[[39](https://arxiv.org/html/2609.10994#bib.bib39),[48](https://arxiv.org/html/2609.10994#bib.bib48)\]\. Note that there are no existing distribution shift adaptation methods that assume noisy labels in both distributions nor methods for similarity\-based learning or paired\-comparison learning\. In addition, the proposed method can flexibly handle different types of supervision between the training and test phases, e\.g\., noisy labeled data in the training distribution and a few PN data in the test distribution\. Furthermore, by using UU data in the test distribution, the proposed method does not require assumptions about shift types such as covariate shift\[[46](https://arxiv.org/html/2609.10994#bib.bib46)\]and concept shift\[[37](https://arxiv.org/html/2609.10994#bib.bib37)\], which are usually required in existing distribution shift adaptation methods that use unlabeled data in the test distribution\[[46](https://arxiv.org/html/2609.10994#bib.bib46)\]\. Since shift types are generally difficult or impossible to identify from unlabeled data\[[27](https://arxiv.org/html/2609.10994#bib.bib27)\], this property is preferable in practice\.

## 2Related Work

Discriminative clustering or unsupervised classification methods attempt to learn a classifier from a set of unlabeled data\[[59](https://arxiv.org/html/2609.10994#bib.bib59),[21](https://arxiv.org/html/2609.10994#bib.bib21)\]\. However, they are often suboptimal since they rely on a clustering assumption that one cluster corresponds to one class, which is often violated in practice\[[33](https://arxiv.org/html/2609.10994#bib.bib33),[35](https://arxiv.org/html/2609.10994#bib.bib35)\]\. To overcome this issue, UU learning methods are based on the empirical risk minimization \(ERM\) that rewrites the risk using two unlabeled datasets with different class\-priors\[[33](https://arxiv.org/html/2609.10994#bib.bib33),[35](https://arxiv.org/html/2609.10994#bib.bib35),[36](https://arxiv.org/html/2609.10994#bib.bib36)\]\. It can learn a binary classifier without the clustering assumption\[[33](https://arxiv.org/html/2609.10994#bib.bib33)\]\. Some methods can usemm\(m≥2\)\(m\\geq 2\)sets of unlabeled data with different class\-priors to learn a binary classifier\[[32](https://arxiv.org/html/2609.10994#bib.bib32),[56](https://arxiv.org/html/2609.10994#bib.bib56)\]\. However, all these methods assume that the test and training distributions have the same class\-conditional densities, and thus they are inappropriate for our problem settings where the class\-conditional densities can vary\.

Distribution shift or domain adaptation methods attempt to mitigate the gap between the training and test distributions\[[39](https://arxiv.org/html/2609.10994#bib.bib39),[55](https://arxiv.org/html/2609.10994#bib.bib55),[53](https://arxiv.org/html/2609.10994#bib.bib53),[48](https://arxiv.org/html/2609.10994#bib.bib48)\]\. Although many methods have been proposed, there are no methods for UU learning\. For the adaptation, one representative approach is to learn invariant features between the training and test distributions\[[43](https://arxiv.org/html/2609.10994#bib.bib43),[38](https://arxiv.org/html/2609.10994#bib.bib38),[11](https://arxiv.org/html/2609.10994#bib.bib11),[23](https://arxiv.org/html/2609.10994#bib.bib23),[29](https://arxiv.org/html/2609.10994#bib.bib29)\]\. Although this approach is promising in some applications, the invariant features often do not improve the performance since they do not explicitly minimize the test risk\([63](https://arxiv.org/html/2609.10994#bib.bib63),[27](https://arxiv.org/html/2609.10994#bib.bib27)\)\. Another representative approach is the importance weighting, which can adapt to the shift by explicitly minimizing the test risk, and it has solid theoretical properties such as consistency\([34](https://arxiv.org/html/2609.10994#bib.bib34),[52](https://arxiv.org/html/2609.10994#bib.bib52)\)\. Accordingly, we extend it to the UU setting\.

As for data used for the adaptation, existing methods often assume unlabeled data in the test distribution\[[43](https://arxiv.org/html/2609.10994#bib.bib43),[11](https://arxiv.org/html/2609.10994#bib.bib11),[23](https://arxiv.org/html/2609.10994#bib.bib23),[46](https://arxiv.org/html/2609.10994#bib.bib46)\]\. Since there is no supervision in the test distribution, they typically assume a specific shift type such as covariate shift\[[46](https://arxiv.org/html/2609.10994#bib.bib46)\]\. However, the shift type is generally difficult or impossible to identify from unlabeled data\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\. Some recent importance weighting methods can adapt to the shift without shift type assumptions by using some supervision in the test distribution, such as a few PN data\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[8](https://arxiv.org/html/2609.10994#bib.bib8)\]and a few PU data\[[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\. The proposed method also does not require shift type assumptions because it uses a few UU data in the test distribution\. By appropriately setting the class\-priors, the proposed method can recover these methods\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]as special cases\. In general, since the proposed method must perform importance weighting solely from unlabeled data, it addresses a more challenging problem than these methods that can exploit labeled \(positive\) data\. Although several methods assume noisy labeled data in the training distribution and unlabeled data in the test distribution\[[47](https://arxiv.org/html/2609.10994#bib.bib47),[62](https://arxiv.org/html/2609.10994#bib.bib62),[61](https://arxiv.org/html/2609.10994#bib.bib61),[14](https://arxiv.org/html/2609.10994#bib.bib14)\], they cannot treat noisy labeled data in both the training and test distributions, but the proposed method can\.

## 3Preliminary

We explain UU learning based on the ERM\[[33](https://arxiv.org/html/2609.10994#bib.bib33),[35](https://arxiv.org/html/2609.10994#bib.bib35)\]\. Let𝐱∈𝒳\{\\bf x\}\\in\{\\cal X\}andy∈\{±1\}y\\in\\\{\\pm\{1\}\\\}be the input and output random variables, where𝒳\{\\cal X\}is the input space,y=\+1y=\+1and−1\-1represent positive and negative classes, respectively\. Letp⁡\(𝐱,y\)p\(\{\\bf x\},y\)be the joint density,pp​\(𝐱\):=p⁡\(𝐱\|y=\+1\)p^\{\{\\rm p\}\}\(\{\\bf x\}\):=p\(\{\\bf x\}\|y=\+1\)andpn​\(𝐱\):=p⁡\(𝐱\|y=−1\)p^\{\{\\rm n\}\}\(\{\\bf x\}\):=p\(\{\\bf x\}\|y=\-1\)be the positive\- and negative\-conditional densities,p⁡\(𝐱\)=π​pp​\(𝐱\)\+\(1−π\)​pn​\(𝐱\)p\(\{\\bf x\}\)=\\pi p^\{\{\\rm p\}\}\(\{\\bf x\}\)\+\(1\-\\pi\)p^\{\{\\rm n\}\}\(\{\\bf x\}\)be the marginal density, andπ:=p⁡\(y=\+1\)\\pi:=p\(y=\+1\)be the class\-prior\.f:𝒳→ℝf:\{\\cal X\}\\to\\mathbb\{R\}is a decision function and the predicted label is obtained byt=sign⁡\(f⁡\(𝐱\)\)t=\{\\rm sign\}\(f\(\{\\bf x\}\)\), wheresign⁡\(⋅\)\{\\rm sign\}\(\\cdot\)is a sign function\. Letℓ:ℝ×\{±1\}→ℝ≥0\\ell:\\mathbb\{R\}\\times\\\{\\pm\{1\}\\\}\\to\\mathbb\{R\}\_\{\\geq 0\}be the loss function, such that valueℓ⁡\(t,y\)\\ell\(t,y\)means the loss for ground truth labelyybytt\. The binary classification aims to learnffthat minimizes the expected test error, called the risk,

R⁡\(f\):=𝔼p⁡\(𝐱,y\)​\[ℓ⁡\(f⁡\(𝐱\),y\)\]=π​𝔼pp​\(𝐱\)​\[ℓ⁡\(f⁡\(𝐱\),\+1\)\]\+\(−π\)​𝔼pn​\(𝐱\)​\[ℓ⁡\(f⁡\(𝐱\),−1\)\],\\displaystyle R\(f\):=\\mathbb\{E\}\_\{p\(\{\\bf x\},y\)\}\[\\ell\(f\(\{\\bf x\}\),y\)\]=\\pi\\mathbb\{E\}\_\{p^\{\{\\rm p\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\}\),\+1\)\]\+\(1\\\!\-\\\!\\pi\)\\mathbb\{E\}\_\{p^\{\{\\rm n\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\}\),\-1\)\],\(1\)where𝔼p⁡\(z\)\\mathbb\{E\}\_\{p\(z\)\}is an expectation overp⁡\(z\)p\(z\)\. In standard supervised learning, we are given labeled positive and negative data drawn fromp⁡\(𝐱,y\)p\(\{\\bf x\},y\)\. Thus, we can directly train the classifier by minimizing the empirical risk that is calculated by replacing the expectation in Eq\. \([1](https://arxiv.org/html/2609.10994#S3.E1)\) with the sample average\.

However, in UU learning, we are given two sets of unlabeled dataXaX^\{\{\\rm a\}\}andXbX^\{\{\\rm b\}\}drawn from the following marginal distributions:

Xa=\{𝐱na\}n=1Na∼pa​\(𝐱\):=θa​pp​\(𝐱\)\+\(1−θa\)​pn​\(𝐱\),\\displaystyle X^\{\{\\rm a\}\}\\\!=\\\!\\\{\{\\bf x\}^\{\{\\rm a\}\}\_\{n\}\\\}\_\{n=1\}^\{N^\{\{\\rm a\}\}\}\\\!\\sim\\\!p^\{\{\\rm a\}\}\(\{\\bf x\}\)\\\!:=\\\!\\theta^\{\{\\rm a\}\}p^\{\{\\rm p\}\}\(\{\\bf x\}\)\\\!\+\\\!\(1\-\\theta^\{\{\\rm a\}\}\)p^\{\{\\rm n\}\}\(\{\\bf x\}\),Xb=\{𝐱nb\}n=1Nb∼pb​\(𝐱\):=θb​pp​\(𝐱\)\+\(1−θb\)​pn​\(𝐱\),\\displaystyle X^\{\{\\rm b\}\}\\\!=\\\!\\\{\{\\bf x\}^\{\{\\rm b\}\}\_\{n\}\\\}\_\{n=1\}^\{N^\{\{\\rm b\}\}\}\\\!\\sim\\\!p^\{\{\\rm b\}\}\(\{\\bf x\}\)\\\!:=\\\!\\theta^\{\{\\rm b\}\}p^\{\{\\rm p\}\}\(\{\\bf x\}\)\\\!\+\\\!\(1\-\\theta^\{\{\\rm b\}\}\)p^\{\{\\rm n\}\}\(\{\\bf x\}\),\(2\)whereθa\\theta^\{\{\\rm a\}\}andθb\\theta^\{\{\\rm b\}\}are two class\-priors such thatθa≠θb\\theta^\{\{\\rm a\}\}\\neq\\theta^\{\{\\rm b\}\}\. Marginal densitiesp⁡\(𝐱\)p\(\{\\bf x\}\),pa​\(𝐱\)p^\{\{\\rm a\}\}\(\{\\bf x\}\), andpb​\(𝐱\)p^\{\{\\rm b\}\}\(\{\\bf x\}\)share the class\-conditional densitiespp​\(𝐱\)p^\{\{\\rm p\}\}\(\{\\bf x\}\)andpn​\(𝐱\)p^\{\{\\rm n\}\}\(\{\\bf x\}\)\. To train the classifier from the two unlabeled datasets, UU learning rewrites the risk in Eq\. \([1](https://arxiv.org/html/2609.10994#S3.E1)\) with two marginal densitiespa​\(𝐱\)p^\{\{\\rm a\}\}\(\{\\bf x\}\)andpb​\(𝐱\)p^\{\{\\rm b\}\}\(\{\\bf x\}\)\. Specifically, by solving Eq\. \([2](https://arxiv.org/html/2609.10994#S3.E2)\) forpp​\(𝐱\)p^\{\{\\rm p\}\}\(\{\\bf x\}\)andpn​\(𝐱\)p^\{\{\\rm n\}\}\(\{\\bf x\}\), we can obtain the following equations:

π​pp​\(𝐱\)=a​pa​\(𝐱\)−c​pb​\(𝐱\),\(1−π\)​pn​\(𝐱\)=−b​pa​\(𝐱\)\+d​pb​\(𝐱\),\\displaystyle\\pi p^\{\{\\rm p\}\}\(\{\\bf x\}\)=ap^\{\{\\rm a\}\}\(\{\\bf x\}\)\-cp^\{\{\\rm b\}\}\(\{\\bf x\}\),\\ \\ \(1\-\\pi\)p^\{\{\\rm n\}\}\(\{\\bf x\}\)=\-bp^\{\{\\rm a\}\}\(\{\\bf x\}\)\+dp^\{\{\\rm b\}\}\(\{\\bf x\}\),\(3\)wherea:=\(1−θb\)​πθa−θba:=\\frac\{\(1\-\\theta^\{\{\\rm b\}\}\)\\pi\}\{\\theta^\{\{\\rm a\}\}\-\\theta^\{\{\\rm b\}\}\},b:=θb​\(1−π\)θa−θbb:=\\frac\{\\theta^\{\{\\rm b\}\}\(1\-\\pi\)\}\{\\theta^\{\{\\rm a\}\}\-\\theta^\{\{\\rm b\}\}\},c:=\(1−θa\)​πθa−θbc:=\\frac\{\(1\-\\theta^\{\{\\rm a\}\}\)\\pi\}\{\\theta^\{\{\\rm a\}\}\-\\theta^\{\{\\rm b\}\}\}, andd:=θa​\(1−π\)θa−θbd:=\\frac\{\\theta^\{\{\\rm a\}\}\(1\-\\pi\)\}\{\\theta^\{\{\\rm a\}\}\-\\theta^\{\{\\rm b\}\}\}\. By using Eq\. \([3](https://arxiv.org/html/2609.10994#S3.E3)\), we can obtain

π​𝔼pp​\(𝐱\)​\[ℓ⁡\(f⁡\(𝐱\),\+1\)\]\\displaystyle\\pi\\mathbb\{E\}\_\{p^\{\{\\rm p\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\}\),\+1\)\]=a𝔼pa​\(𝐱\)\[ℓ\(f\(𝐱,\+1\)\]−c𝔼pb​\(𝐱\)\[ℓ\(f\(𝐱,\+1\)\],\\displaystyle=a\\mathbb\{E\}\_\{p^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\},\+1\)\]\-c\\mathbb\{E\}\_\{p^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\},\+1\)\],\(1−π\)​𝔼pn​\(𝐱\)​\[ℓ⁡\(f⁡\(𝐱\),−1\)\]\\displaystyle\(1\-\\pi\)\\mathbb\{E\}\_\{p^\{\{\\rm n\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\}\),\-1\)\]=−b𝔼pa​\(𝐱\)\[ℓ\(f\(𝐱,−1\)\]\+d𝔼pb​\(𝐱\)\[ℓ\(f\(𝐱,−1\)\]\.\\displaystyle=\-b\\mathbb\{E\}\_\{p^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\},\-1\)\]\+d\\mathbb\{E\}\_\{p^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\},\-1\)\]\.\(4\)Plugging them into Eq\. \([1](https://arxiv.org/html/2609.10994#S3.E1)\), we obtain

R⁡\(f\)=a​𝔼pa​\(𝐱\)​\[ℓ⁡\(f⁡\(𝐱\),\+1\)\]−c​𝔼pb​\(𝐱\)​\[ℓ⁡\(f⁡\(𝐱\),\+1\)\]−b​𝔼pa​\(𝐱\)​\[ℓ⁡\(f⁡\(𝐱\),−1\)\]\+d​𝔼pb​\(𝐱\)​\[ℓ⁡\(f⁡\(𝐱\),−1\)\]\.\\displaystyle R\(f\)\\\!=\\\!a\\mathbb\{E\}\_\{p^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\}\),\+1\)\]\\\!\-\\\!c\\mathbb\{E\}\_\{p^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\}\),\+1\)\]\\\!\-\\\!b\\mathbb\{E\}\_\{p^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\}\),\-1\)\]\\\!\+\\\!d\\mathbb\{E\}\_\{p^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\[\\ell\(f\(\{\\bf x\}\),\-1\)\]\.\(5\)Since Eq\. \([5](https://arxiv.org/html/2609.10994#S3.E5)\) is represented withpa​\(𝐱\)p^\{\{\\rm a\}\}\(\{\\bf x\}\)andpb​\(𝐱\)p^\{\{\\rm b\}\}\(\{\\bf x\}\), we can approximate it withXaX^\{\{\\rm a\}\}andXbX^\{\{\\rm b\}\}\. We call this rewritten riskthe UU risk\. Although both equations in \([4](https://arxiv.org/html/2609.10994#S3.E4)\) are non\-negative by definition, when highly expressive models such as neural networks are used forff, their empirical estimates can take negative values leading to serious overfitting\[[35](https://arxiv.org/html/2609.10994#bib.bib35)\]\. To alleviate this issue, non\-negative correction such as the absolute value correction is commonly used\[[35](https://arxiv.org/html/2609.10994#bib.bib35),[32](https://arxiv.org/html/2609.10994#bib.bib32)\]\. Specifically, the empirical estimate of Eq\. \([5](https://arxiv.org/html/2609.10994#S3.E5)\) with the absolute value correction is given by

R^​\(f\)=\|aNa​∑n=1Naℓ⁡\(f⁡\(𝐱na\),\+\)−cNb​∑n=1Nbℓ⁡\(f⁡\(𝐱nb\),\+\)\|\+\|dNb​∑n=1Nbℓ⁡\(f⁡\(𝐱nb\),−\)−bNa​∑n=1Naℓ⁡\(f⁡\(𝐱na\),−\)\|,\\displaystyle\{\\hat\{R\}\}\(f\)\\\!=\\\!\\left\|\\frac\{a\}\{N^\{\{\\rm a\}\}\}\\\!\\sum\_\{n=1\}^\{N^\{\{\\rm a\}\}\}\\\!\\ell\(f\(\{\\bf x\}\_\{n\}^\{\{\\rm a\}\}\),\\\!\+1\\\!\)\\\!\-\\\!\\\!\\frac\{c\}\{N^\{\{\\rm b\}\}\}\\\!\\sum\_\{n=1\}^\{N^\{\{\\rm b\}\}\}\\ell\(f\(\{\\bf x\}\_\{n\}^\{\{\\rm b\}\}\),\\\!\+1\\\!\)\\right\|\\\!\+\\\!\\left\|\\frac\{d\}\{N^\{\{\\rm b\}\}\}\\\!\\sum\_\{n=1\}^\{N^\{\{\\rm b\}\}\}\\\!\\ell\(f\(\{\\bf x\}\_\{n\}^\{\{\\rm b\}\}\),\\\!\-1\\\!\)\\\!\-\\\!\\frac\{b\}\{N^\{\{\\rm a\}\}\}\\\!\\sum\_\{n=1\}^\{N^\{\{\\rm a\}\}\}\\\!\\ell\(f\(\{\\bf x\}\_\{n\}^\{\{\\rm a\}\}\),\\\!\-1\\\!\)\\right\|,\(6\)where\|⋅\|\|\\cdot\|is the absolute value function to penalize the risk for being negative\. When there are many UU data, we can obtain an accurate classifierffby minimizing Eq\. \([6](https://arxiv.org/html/2609.10994#S3.E6)\) w\.r\.t\. parameters offf\. UU learning includes a wide variety of learning problems as special cases\. For example, whenθa=1\\theta^\{\{\\rm a\}\}=1andθb=0\\theta^\{\{\\rm b\}\}=0,XaX^\{\{\\rm a\}\}andXbX^\{\{\\rm b\}\}are positive and negative data, respectively; Eq\. \([6](https://arxiv.org/html/2609.10994#S3.E6)\) becomes the empirical risk on ordinary PN learning with\(a,b,c,d\)=\(π,0,0,1−π\)\(a,b,c,d\)=\(\\pi,0,0,1\-\\pi\)\[[54](https://arxiv.org/html/2609.10994#bib.bib54)\]\. Whenθa=1\\theta^\{\{\\rm a\}\}=1andθb=π\\theta^\{\{\\rm b\}\}=\\pi,XaX^\{\{\\rm a\}\}andXbX^\{\{\\rm b\}\}are positive and unlabeled data, respectively; Eq\. \([6](https://arxiv.org/html/2609.10994#S3.E6)\) becomes the non\-negative empirical risk on PU learning with\(a,b,c,d\)=\(π,π,0,1\)\(a,b,c,d\)=\(\\pi,\\pi,0,1\)\[[13](https://arxiv.org/html/2609.10994#bib.bib13)\]\. When1\>θa\>θb\>01\>\\theta^\{\{\\rm a\}\}\>\\theta^\{\{\\rm b\}\}\>0,XaX^\{\{\\rm a\}\}andXbX^\{\{\\rm b\}\}can be regarded as noisy positive and negative data, respectively; Eq\. \([6](https://arxiv.org/html/2609.10994#S3.E6)\) becomes the non\-negative empirical risk on noisy label learning\.

## 4Proposed Method

We first define our problem setting \(subsection[4\.1](https://arxiv.org/html/2609.10994#S4.SS1)\)\. Then, we derive the importance\-weighted UU risk \(subsection[4\.2](https://arxiv.org/html/2609.10994#S4.SS2)\) and the estimation method for importance weights from UU data \(subsection[4\.3](https://arxiv.org/html/2609.10994#S4.SS3)\)\. After describing our classifier loss function with the importance\-weighted UU risk \(subsection[4\.4](https://arxiv.org/html/2609.10994#S4.SS4)\), we explain our training procedure \(subsection[4\.5](https://arxiv.org/html/2609.10994#S4.SS5)\)\.

### 4\.1Problem Setting

Letptr​\(𝐱,y\)p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)be the joint density of the training distribution,ptrp​\(𝐱\):=ptr​\(𝐱\|y=\+1\)p^\{\{\\rm p\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\):=p\_\{\{\\rm tr\}\}\(\{\\bf x\}\|y=\+1\)andptrn​\(𝐱\):=ptr​\(𝐱\|y=−1\)p\_\{\{\\rm tr\}\}^\{\{\\rm n\}\}\(\{\\bf x\}\):=p\_\{\{\\rm tr\}\}\(\{\\bf x\}\|y=\-1\)be the positive and negative\-conditional training densities, respectively, andπtr:=ptr​\(y=\+1\)\\pi\_\{\{\\rm tr\}\}:=p\_\{\{\\rm tr\}\}\(y=\+1\)be the positive training class\-prior\. Similarly, letpte​\(𝐱,y\)p\_\{\{\\rm te\}\}\(\{\\bf x\},y\)be the joint density of the test distribution,ptep​\(𝐱\):=pte​\(𝐱\|y=\+1\)p^\{\{\\rm p\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\):=p\_\{\{\\rm te\}\}\(\{\\bf x\}\|y=\+1\)andpten​\(𝐱\):=pte​\(𝐱\|y=−1\)p\_\{\{\\rm te\}\}^\{\{\\rm n\}\}\(\{\\bf x\}\):=p\_\{\{\\rm te\}\}\(\{\\bf x\}\|y=\-1\)be the positive and negative\-conditional test densities, respectively, andπte:=pte​\(y=\+1\)\\pi\_\{\{\\rm te\}\}:=p\_\{\{\\rm te\}\}\(y=\+1\)be the positive test class\-prior\. We assume that the training and test distributions are related but different,ptr​\(𝐱,y\)≠pte​\(𝐱,y\)p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)\\neq p\_\{\{\\rm te\}\}\(\{\\bf x\},y\), which is the most general shift form\. Note that we do not know the specific shift type such as covariate shift\.

Suppose that we are given two sets of unlabeled dataXtraX^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}andXtrbX^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}drawn from different marginal densitiesptra​\(𝐱\)p^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\)andptrb​\(𝐱\)p^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\), respectively, in the training phase:

Xtra=\{𝐱tr,na\}n=1Ntra∼ptra​\(𝐱\):=θtra​ptrp​\(𝐱\)\+\(−θtra\)​ptrn​\(𝐱\),\\displaystyle X^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\\\!=\\\!\\\{\{\\bf x\}^\{\{\\rm a\}\}\_\{\{\\rm tr\},n\}\\\}\_\{n=1\}^\{N^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\}\\\!\\sim\\\!p^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\)\\\!:=\\\!\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}p^\{\{\\rm p\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\)\\\!\+\\\!\(1\\\!\-\\\!\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\)p^\{\{\\rm n\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\),Xtrb=\{𝐱tr,nb\}n=1Ntrb∼ptrb​\(𝐱\):=θtrb​ptrp​\(𝐱\)\+\(−θtrb\)​ptrn​\(𝐱\),\\displaystyle X^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\\\!=\\\!\\\{\{\\bf x\}^\{\{\\rm b\}\}\_\{\{\\rm tr\},n\}\\\}\_\{n=1\}^\{N^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\}\\\!\\sim\\\!p^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\)\\\!:=\\\!\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}p^\{\{\\rm p\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\)\\\!\+\\\!\(1\\\!\-\\\!\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\)p^\{\{\\rm n\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\),\(7\)whereθtra\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}andθtrb\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}are two class\-priors such thatθtra≠θtrb\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\\neq\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\. Similarly, suppose that we are also given two sets of unlabeled dataXteaX^\{\{\\rm a\}\}\_\{\{\\rm te\}\}andXtebX^\{\{\\rm b\}\}\_\{\{\\rm te\}\}drawn from different marginal densitiesptea​\(𝐱\)p^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\)andpteb​\(𝐱\)p^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\), respectively, in the test phase:

Xtea=\{𝐱te,na\}n=1Ntea∼ptea​\(𝐱\):=θtea​ptep​\(𝐱\)\+\(−θtea\)​pten​\(𝐱\),\\displaystyle X^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\\!=\\\!\\\{\{\\bf x\}^\{\{\\rm a\}\}\_\{\{\\rm te\},n\}\\\}\_\{n=1\}^\{N^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\}\\\!\\sim\\\!p^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\)\\\!:=\\\!\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}p^\{\{\\rm p\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\)\\\!\+\\\!\(1\\\!\-\\\!\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\)p^\{\{\\rm n\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\),Xteb=\{𝐱te,nb\}n=1Nteb∼pteb​\(𝐱\):=θteb​ptep​\(𝐱\)\+\(−θteb\)​pten​\(𝐱\),\\displaystyle X^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\\\!=\\\!\\\{\{\\bf x\}^\{\{\\rm b\}\}\_\{\{\\rm te\},n\}\\\}\_\{n=1\}^\{N^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\}\\\!\\sim\\\!p^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\)\\\!:=\\\!\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}p^\{\{\\rm p\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\)\\\!\+\\\!\(1\\\!\-\\\!\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\)p^\{\{\\rm n\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\),\(8\)whereθtea\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}andθteb\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}are two class\-priors such thatθtea≠θteb\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\neq\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\. We assume that UU data in the test distribution are much smaller than those in the training distribution, i\.e\.,Ntea\+Nteb≪Ntra\+NtrbN\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\+N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\\ll N\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\+N\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\. We assume that class\-priorsπte\\pi\_\{\{\\rm te\}\},θtra\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\},θtrb\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\},θtea\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}, andθteb\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}are known as in previous UU learning studies\([33](https://arxiv.org/html/2609.10994#bib.bib33),[35](https://arxiv.org/html/2609.10994#bib.bib35),[32](https://arxiv.org/html/2609.10994#bib.bib32)\)\. They can be estimated in some cases\([36](https://arxiv.org/html/2609.10994#bib.bib36),[31](https://arxiv.org/html/2609.10994#bib.bib31),[16](https://arxiv.org/html/2609.10994#bib.bib16)\)\. Our goal is to learn a classifierffthat accurately classifies test instance𝐱\{\\bf x\}drawn frompte​\(𝐱\)=πte​ptep​\(𝐱\)\+\(1−πte\)​pten​\(𝐱\)p\_\{\{\\rm te\}\}\(\{\\bf x\}\)=\\pi\_\{\{\\rm te\}\}p^\{\{\\rm p\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\)\+\(1\-\\pi\_\{\{\\rm te\}\}\)p^\{\{\\rm n\}\}\_\{\{\\rm te\}\}\(\{\\bf x\}\)by using unlabeled datasetsXtra∪Xtrb∪Xtea∪XtebX^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\\cup X^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\. The proposed method can handle various learning problems, such as PN, PU, and noisy labeled learning, by specifying values of the class\-priors as described in Section[3](https://arxiv.org/html/2609.10994#S3)\. It can be naturally applied even when types of supervision differ between the training and test phases, e\.g\., noisy labeled data in the training distribution and a few PN data in the test distribution\.

### 4\.2Importance\-weighted UU Risk

In this subsection, we derive the importance\-weighted empirical risk using UU training data by rewriting the test risk\. First, we consider the risk on the test distribution:

Rte​\(f\):=𝔼pte​\(𝐱,y\)​\[ℓ⁡\(f⁡\(𝐱\),y\)\],\\displaystyle R\_\{\{\\rm te\}\}\(f\):=\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}\(\{\\bf x\},y\)\}\\left\[\\ell\(f\(\{\\bf x\}\),y\)\\right\],\(9\)which is the objective to be minimized for learningffthat fits on the test distribution\. By introducing importance weightw⁡\(𝐱,y\):=pte​\(𝐱,y\)/ptr​\(𝐱,y\)w\(\{\\bf x\},y\):=p\_\{\{\\rm te\}\}\(\{\\bf x\},y\)/p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)222By this definition, the rewritten riskRtrw​\(f\)R\_\{\{\\rm tr\}\}^\{\{\\rm w\}\}\(f\)depends on true class\-priorπtr\\pi\_\{\{\\rm tr\}\}as shown in Eq\. \([10](https://arxiv.org/html/2609.10994#S4.Ex5)\)\. However, since the data generation process in subsection[4\.1](https://arxiv.org/html/2609.10994#S4.SS1)does not depend onπtr\\pi\_\{\{\\rm tr\}\}, the value ofπtr\\pi\_\{\{\\rm tr\}\}can be freely determined in our setting \(we setπtr=0\.5\\pi\_\{\{\\rm tr\}\}=0\.5in our experiments\)\. We use this definition to ensure consistency with existing importance weighting methods\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[8](https://arxiv.org/html/2609.10994#bib.bib8),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\., as shown in previous studies\([7](https://arxiv.org/html/2609.10994#bib.bib7),[8](https://arxiv.org/html/2609.10994#bib.bib8)\), this risk can be rewritten as follows,

Rte​\(f\)\\displaystyle R\_\{\{\\rm te\}\}\(f\)=𝔼ptr​\(𝐱,y\)​\[w⁡\(𝐱,y\)​ℓ​\(f⁡\(𝐱\),y\)\]\\displaystyle\\\!=\\\!\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)\}\\left\[w\(\{\\bf x\},y\)\\ell\(f\(\{\\bf x\}\),y\)\\right\]=πtr​𝔼ptrp​\(𝐱\)​\[ℓw​\(f⁡\(𝐱\),\+1\)\]\+\(−πtr\)​𝔼ptrn​\(𝐱\)​\[ℓw​\(f⁡\(𝐱\),−1\)\]=:Rtrw​\(f\),\\displaystyle\\\!=\\\!\\pi\_\{\{\\rm tr\}\}\{\\mathbb\{E\}\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm p\}\}\(\{\\bf x\}\)\}\\left\[\\ell\_\{w\}\(f\(\{\\bf x\}\),\+1\)\\right\]\\\!\+\\\!\(1\\\!\-\\\!\\pi\_\{\{\\rm tr\}\}\)\{\\mathbb\{E\}\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm n\}\}\(\{\\bf x\}\)\}\\left\[\\ell\_\{w\}\(f\(\{\\bf x\}\),\-1\)\\right\]\\\!=:\\\!R\_\{\{\\rm tr\}\}^\{\{\\rm w\}\}\(f\),\(10\)where we setℓw​\(𝐱,y\):=w⁡\(𝐱,y\)​ℓ​\(f⁡\(𝐱\),y\)\\ell\_\{w\}\(\{\\bf x\},y\):=w\(\{\\bf x\},y\)\\ell\(f\(\{\\bf x\}\),y\)\. This reformulation shows that the test risk can be expressed as the importance\-weighted risk computed on the training distribution\. While this risk depends on class\-conditional densitiesptrp​\(𝐱\)p^\{\{\\rm p\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\)andptrn​\(𝐱\)p\_\{\{\\rm tr\}\}^\{\{\\rm n\}\}\(\{\\bf x\}\), by using the same procedure described in Section[3](https://arxiv.org/html/2609.10994#S3), it can be rewritten with UU training densitiesptra​\(𝐱\)p^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\)andptrb​\(𝐱\)p^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\(\{\\bf x\}\):

Rtrw​\(f\)\\displaystyle R\_\{\{\\rm tr\}\}^\{\{\\rm w\}\}\(f\)=atr​𝔼ptra​\(𝐱\)​\[ℓw​\(f⁡\(𝐱\),\+1\)\]−ctr​𝔼ptrb​\(𝐱\)​\[ℓw​\(f⁡\(𝐱\),\+1\)\]\\displaystyle=a\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\[\\ell\_\{w\}\(f\(\{\\bf x\}\),\+1\)\]\\\!\-\\\!c\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\[\\ell\_\{w\}\(f\(\{\\bf x\}\),\+1\)\]−btr​𝔼ptra​\(𝐱\)​\[ℓw​\(f⁡\(𝐱\),−1\)\]\+dtr​𝔼ptrb​\(𝐱\)​\[ℓw​\(f⁡\(𝐱\),−1\)\],\\displaystyle\\\!\-\\\!b\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\[\\ell\_\{w\}\(f\(\{\\bf x\}\),\-1\)\]\\\!\+\\\!d\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\[\\ell\_\{w\}\(f\(\{\\bf x\}\),\-1\)\],\(11\)whereatr:=\(1−θtrb\)​πtrθtra−θtrba\_\{\{\\rm tr\}\}:=\\frac\{\(1\-\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\)\\pi\_\{\{\\rm tr\}\}\}\{\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\-\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\},btr:=θtrb​\(1−πtr\)θtra−θtrbb\_\{\{\\rm tr\}\}:=\\frac\{\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\(1\-\\pi\_\{\{\\rm tr\}\}\)\}\{\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\-\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\},ctr:=\(1−θtra\)​πtrθtra−θtrbc\_\{\{\\rm tr\}\}:=\\frac\{\(1\-\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\)\\pi\_\{\{\\rm tr\}\}\}\{\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\-\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\}, anddtr:=θtra​\(1−πtr\)θtra−θtrbd\_\{\{\\rm tr\}\}:=\\frac\{\\theta\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\(1\-\\pi\_\{\{\\rm tr\}\}\)\}\{\\theta\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\-\\theta\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\}\. Thus, the empirical estimate of Eq\. \([11](https://arxiv.org/html/2609.10994#S4.Ex6)\) with UU training dataXtra∪XtrbX^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}is represented as

R^trw​\(f\)\\displaystyle\{\\hat\{R\}\}\_\{\{\\rm tr\}\}^\{\{\\rm w\}\}\(f\)=\|atrNtra​∑n=1Ntraℓw​\(f⁡\(𝐱tr,na\),\+1\)−ctrNtrb​∑n=1Ntrbℓw​\(f⁡\(𝐱tr,nb\),\+1\)\|\\displaystyle=\\left\|\\frac\{a\_\{\{\\rm tr\}\}\}\{N\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\}\\sum\_\{n=1\}^\{N\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\}\\ell\_\{w\}\(f\(\{\\bf x\}\_\{\{\\rm tr\},n\}^\{\{\\rm a\}\}\),\+1\)\\\!\-\\\!\\frac\{c\_\{\{\\rm tr\}\}\}\{N\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\}\\sum\_\{n=1\}^\{N\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\}\\ell\_\{w\}\(f\(\{\\bf x\}\_\{\{\\rm tr\},n\}^\{\{\\rm b\}\}\),\+1\)\\right\|\+\|dtrNtrb​∑n=1Ntrbℓw​\(f⁡\(𝐱tr,nb\),−1\)−btrNtra​∑n=1Ntraℓw​\(f⁡\(𝐱tr,na\),−1\)\|,\\displaystyle\\\!\+\\\!\\left\|\\frac\{d\_\{\{\\rm tr\}\}\}\{N\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\}\\sum\_\{n=1\}^\{N\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\}\\ell\_\{w\}\(f\(\{\\bf x\}\_\{\{\\rm tr\},n\}^\{\{\\rm b\}\}\),\-1\)\\\!\-\\\!\\frac\{b\_\{\{\\rm tr\}\}\}\{N\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\}\\sum\_\{n=1\}^\{N\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\}\\ell\_\{w\}\(f\(\{\\bf x\}\_\{\{\\rm tr\},n\}^\{\{\\rm a\}\}\),\-1\)\\right\|,\(12\)where we also used the absolute value correction to prevent overfitting\.

### 4\.3Importance Weight Estimation from UU Data

Because importance weightw⁡\(𝐱,y\)=pte​\(𝐱,y\)/ptr​\(𝐱,y\)w\(\{\\bf x\},y\)=p\_\{\{\\rm te\}\}\(\{\\bf x\},y\)/p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)in Eq \([12](https://arxiv.org/html/2609.10994#S4.Ex7)\) is not observable, we need to estimate it with UU data\. A standard approach for this task is to use density\-ratio estimation methods that directly estimate the ratio between training and test densities from data without density estimation\([52](https://arxiv.org/html/2609.10994#bib.bib52),[17](https://arxiv.org/html/2609.10994#bib.bib17),[3](https://arxiv.org/html/2609.10994#bib.bib3)\)\. However, they are often numerically unstable becausew⁡\(𝐱,y\)w\(\{\\bf x\},y\)can diverge in regions whereptr​\(𝐱,y\)p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)is small\([60](https://arxiv.org/html/2609.10994#bib.bib60),[25](https://arxiv.org/html/2609.10994#bib.bib25)\)\. To mitigate this issue, we instead use the relative density\-ratio as the importance weight:

wα​\(𝐱,y\):=pte​\(𝐱,y\)α​pte​\(𝐱,y\)\+\(1−α\)​ptr​\(𝐱,y\),\\displaystyle w\_\{\\alpha\}\(\{\\bf x\},y\):=\\frac\{p\_\{\{\\rm te\}\}\(\{\\bf x\},y\)\}\{\\alpha p\_\{\{\\rm te\}\}\(\{\\bf x\},y\)\+\(1\-\\alpha\)p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)\},\(13\)whereα∈\[0,1\]\\alpha\\in\[0,1\]is a hyperparameter\([60](https://arxiv.org/html/2609.10994#bib.bib60)\)\.wα​\(𝐱,y\)w\_\{\\alpha\}\(\{\\bf x\},y\)is always bounded above by1/α1/\\alpha, and recoversw⁡\(𝐱,y\)w\(\{\\bf x\},y\)whenα=0\\alpha=0\. Thus,wα​\(𝐱,y\)w\_\{\\alpha\}\(\{\\bf x\},y\)is a bounded generalization ofw⁡\(𝐱,y\)w\(\{\\bf x\},y\)\. The importance weighting with the relative density\-ratio has shown to improve stability and performance\([60](https://arxiv.org/html/2609.10994#bib.bib60),[44](https://arxiv.org/html/2609.10994#bib.bib44),[27](https://arxiv.org/html/2609.10994#bib.bib27)\)\.

To estimatewα​\(𝐱,y\)w\_\{\\alpha\}\(\{\\bf x\},y\), we introduce modelm⁡\(𝐱,y\)∈\[0,1/α\]m\(\{\\bf x\},y\)\\in\[0,1/\\alpha\], such as a neural network\. Following previous studies\[[60](https://arxiv.org/html/2609.10994#bib.bib60),[17](https://arxiv.org/html/2609.10994#bib.bib17),[27](https://arxiv.org/html/2609.10994#bib.bib27)\], we determine parameters ofm⁡\(𝐱,y\)m\(\{\\bf x\},y\)so that the expected squared error between true importance weightwα​\(𝐱,y\)w\_\{\\alpha\}\(\{\\bf x\},y\)andm⁡\(𝐱,y\)m\(\{\\bf x\},y\)is minimized:

J⁡\(m\)\\displaystyle J\(m\):=𝔼pα​\(𝐱,y\)​\[\(wα​\(𝐱,y\)−m⁡\(𝐱,y\)\)2\]\\displaystyle:=\\mathbb\{E\}\_\{p\_\{\\alpha\}\(\{\\bf x\},y\)\}\\left\[\\left\(w\_\{\\alpha\}\(\{\\bf x\},y\)\-m\(\{\\bf x\},y\)\\right\)^\{2\}\\right\]=𝔼pte​\(𝐱,y\)​\[α​m​\(𝐱,y\)2−2​m​\(𝐱,y\)\]\+𝔼ptr​\(𝐱,y\)​\[\(1−α\)​m​\(𝐱,y\)2\]\+C,\\displaystyle=\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}\(\{\\bf x\},y\)\}\\left\[\\alpha m\(\{\\bf x\},y\)^\{2\}\-2m\(\{\\bf x\},y\)\\right\]\+\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)\}\\left\[\(1\-\\alpha\)m\(\{\\bf x\},y\)^\{2\}\\right\]\+C,\(14\)wherepα​\(𝐱,y\):=α​pte​\(𝐱,y\)\+\(1−α\)​ptr​\(𝐱,y\)p\_\{\\alpha\}\(\{\\bf x\},y\):=\\alpha p\_\{\{\\rm te\}\}\(\{\\bf x\},y\)\+\(1\-\\alpha\)p\_\{\{\\rm tr\}\}\(\{\\bf x\},y\)andCCis a constant term that does not depend onmm\. LettingM1​\(𝐱,y\):=α​m​\(𝐱,y\)2−2​m​\(𝐱,y\)M\_\{1\}\(\{\\bf x\},y\):=\\alpha m\(\{\\bf x\},y\)^\{2\}\-2m\(\{\\bf x\},y\)andM2​\(𝐱,y\):=\(1−α\)​m​\(𝐱,y\)2M\_\{2\}\(\{\\bf x\},y\):=\(1\-\\alpha\)m\(\{\\bf x\},y\)^\{2\}, we have

J⁡\(m\)=πte​𝔼ptep​\(𝐱\)​\[M1​\(𝐱,\+1\)\]\+\(−πte\)​𝔼pten​\(𝐱\)​\[M1​\(𝐱,−1\)\]\\displaystyle J\(m\)\\\!=\\\!\\pi\_\{\{\\rm te\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}^\{\{\\rm p\}\}\(\{\\bf x\}\)\}\\left\[M\_\{1\}\(\{\\bf x\},\+1\)\\right\]\\\!\+\\\!\(1\\\!\-\\\!\\pi\_\{\{\\rm te\}\}\)\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}^\{\{\\rm n\}\}\(\{\\bf x\}\)\}\\left\[M\_\{1\}\(\{\\bf x\},\-1\)\\right\]\+πtr​𝔼ptrp​\(𝐱\)​\[M2​\(𝐱,\+1\)\]\+\(−πtr\)​𝔼ptrn​\(𝐱\)​\[M2​\(𝐱,−1\)\]\+C\.\\displaystyle\\\!\+\\\!\\pi\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm p\}\}\(\{\\bf x\}\)\}\\left\[M\_\{2\}\(\{\\bf x\},\+1\)\\right\]\\\!\+\\\!\(1\\\!\-\\\!\\pi\_\{\{\\rm tr\}\}\)\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm n\}\}\(\{\\bf x\}\)\}\\left\[M\_\{2\}\(\{\\bf x\},\-1\)\\right\]\+C\.\(15\)By using Eq\. \([3](https://arxiv.org/html/2609.10994#S3.E3)\), each expectation term in Eq\. \([15](https://arxiv.org/html/2609.10994#S4.Ex9)\) can be rewritten as

πte​𝔼ptep​\(𝐱\)​\[M1​\(𝐱,\+1\)\]\\displaystyle\\pi\_\{\{\\rm te\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}^\{\{\\rm p\}\}\(\{\\bf x\}\)\}\\left\[M\_\{1\}\(\{\\bf x\},\+1\)\\right\]=ate​𝔼ptea​\(𝐱\)​\[M1​\(𝐱,\+1\)\]−cte​𝔼pteb​\(𝐱\)​\[M1​\(𝐱,\+1\)\],\\displaystyle=a\_\{\{\\rm te\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\\left\[M\_\{1\}\(\{\\bf x\},\+1\)\\right\]\-c\_\{\{\\rm te\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\\left\[M\_\{1\}\(\{\\bf x\},\+1\)\\right\],\(16\)\(1−πte\)​𝔼pten​\(𝐱\)​\[M1​\(𝐱,−1\)\]\\displaystyle\(1\-\\pi\_\{\{\\rm te\}\}\)\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}^\{\{\\rm n\}\}\(\{\\bf x\}\)\}\\left\[M\_\{1\}\(\{\\bf x\},\-1\)\\right\]=−bte​𝔼ptea​\(𝐱\)​\[M1​\(𝐱,−1\)\]\+dte​𝔼pteb​\(𝐱\)​\[M1​\(𝐱,−1\)\],\\displaystyle=\-b\_\{\{\\rm te\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\\left\[M\_\{1\}\(\{\\bf x\},\-1\)\\right\]\\\!\+\\\!d\_\{\{\\rm te\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\\left\[M\_\{1\}\(\{\\bf x\},\-1\)\\right\],\(17\)πtr​𝔼ptrp​\(𝐱\)​\[M2​\(𝐱,\+1\)\]\\displaystyle\\pi\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm p\}\}\(\{\\bf x\}\)\}\\left\[M\_\{2\}\(\{\\bf x\},\+1\)\\right\]=atr​𝔼ptra​\(𝐱\)​\[M2​\(𝐱,\+1\)\]−ctr​𝔼ptrb​\(𝐱\)​\[M2​\(𝐱,\+1\)\],\\displaystyle=a\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\\left\[M\_\{2\}\(\{\\bf x\},\+1\)\\right\]\-c\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\\left\[M\_\{2\}\(\{\\bf x\},\+1\)\\right\],\(18\)\(1−πtr\)​𝔼ptrn​\(𝐱\)​\[M2​\(𝐱,−1\)\]\\displaystyle\(1\-\\pi\_\{\{\\rm tr\}\}\)\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm n\}\}\(\{\\bf x\}\)\}\\left\[M\_\{2\}\(\{\\bf x\},\-1\)\\right\]=−btr​𝔼ptra​\(𝐱\)​\[M2​\(𝐱,−1\)\]\+dtr​𝔼ptrb​\(𝐱\)​\[M2​\(𝐱,−1\)\],\\displaystyle=\-b\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm a\}\}\(\{\\bf x\}\)\}\\left\[M\_\{2\}\(\{\\bf x\},\-1\)\\right\]\\\!\+\\\!d\_\{\{\\rm tr\}\}\\mathbb\{E\}\_\{p\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\(\{\\bf x\}\)\}\\left\[M\_\{2\}\(\{\\bf x\},\-1\)\\right\],\(19\)whereate:=\(1−θteb\)​πteθtea−θteba\_\{\{\\rm te\}\}:=\\frac\{\(1\-\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\)\\pi\_\{\{\\rm te\}\}\}\{\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\-\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\},bte:=θteb​\(1−πte\)θtea−θtebb\_\{\{\\rm te\}\}:=\\frac\{\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\(1\-\\pi\_\{\{\\rm te\}\}\)\}\{\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\-\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\},cte:=\(1−θtea\)​πteθtea−θtebc\_\{\{\\rm te\}\}:=\\frac\{\(1\-\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\)\\pi\_\{\{\\rm te\}\}\}\{\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\-\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\}, anddte:=θtea​\(1−πte\)θtea−θtebd\_\{\{\\rm te\}\}:=\\frac\{\\theta\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\(1\-\\pi\_\{\{\\rm te\}\}\)\}\{\\theta\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\-\\theta\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\}\.

SinceM1\(𝐱,y\)=αm\(𝐱,y\)2−2m\(𝐱,y\)≥−1/αM\_\{1\}\(\{\\bf x\},y\)=\\alpha m\(\{\\bf x\},y\)^\{2\}\-2m\(\{\\bf x\},y\)\\geq\-1/\{\\alpha\}for all𝐱\{\\bf x\}andyy, Eqs\. \([16](https://arxiv.org/html/2609.10994#S4.E16)\) and \([17](https://arxiv.org/html/2609.10994#S4.E17)\) are bounded below by−πte/α\-\\pi\_\{\{\\rm te\}\}/\\alphaand−\(1−πte\)/α\-\(1\-\\pi\_\{\{\\rm te\}\}\)/\\alpha, respectively\. However, as in the risk, their empirical estimates can take smaller values and it may cause overfitting\. To prevent this, we apply correction functionF⁡\(z\):=\|z−k\|\+k⁡\(k,z∈ℝ\)F\(z\):=\|z\-k\|\+k\\ \(k,z\\in\\mathbb\{R\}\)wherekkis the lower bound of the loss\([15](https://arxiv.org/html/2609.10994#bib.bib15)\)\. Whenz≥kz\\geq k,F⁡\(z\)F\(z\)returnszz\. Whenz<kz<k,F⁡\(z\)=2​k−z\>kF\(z\)=2k\-z\>k\. Thus, this function can penalize the loss for being smaller thankk\. Additionally, while Eqs\. \([18](https://arxiv.org/html/2609.10994#S4.E18)\) and \([19](https://arxiv.org/html/2609.10994#S4.E19)\) are non\-negative by definition, their empirical estimates can be negative\. We correct this by applying the absolute value functionF⁡\(z\)=\|z\|F\(z\)=\|z\|as in the risk\. Consequently, the empirical estimate of Eq\. \([15](https://arxiv.org/html/2609.10994#S4.Ex9)\) with the loss corrections becomes

J^​\(m\)\\displaystyle\{\\hat\{J\}\}\(m\)=\|ateNtea​∑n=1NteaM1​\(𝐱te,na,\+1\)−cteNteb​∑n=1NtebM1​\(𝐱te,nb,\+1\)−k1\|\\displaystyle=\\left\|\\frac\{a\_\{\{\\rm te\}\}\}\{N^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\}\\sum\_\{n=1\}^\{N^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\}M\_\{1\}\(\{\\bf x\}^\{\{\\rm a\}\}\_\{\{\\rm te\},n\},\+1\)\-\\frac\{c\_\{\{\\rm te\}\}\}\{N^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\}\\sum\_\{n=1\}^\{N^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\}M\_\{1\}\(\{\\bf x\}^\{\{\\rm b\}\}\_\{\{\\rm te\},n\},\+1\)\-k\_\{1\}\\right\|\+\|dteNteb​∑n=1NtebM1​\(𝐱te,nb,−1\)−bteNtea​∑n=1NteaM1​\(𝐱te,na,−1\)−k2\|\\displaystyle\+\\left\|\\frac\{d\_\{\{\\rm te\}\}\}\{N^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\}\\sum\_\{n=1\}^\{N^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\}M\_\{1\}\(\{\\bf x\}^\{\{\\rm b\}\}\_\{\{\\rm te\},n\},\-1\)\-\\frac\{b\_\{\{\\rm te\}\}\}\{N^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\}\\sum\_\{n=1\}^\{N^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\}M\_\{1\}\(\{\\bf x\}^\{\{\\rm a\}\}\_\{\{\\rm te\},n\},\-1\)\-k\_\{2\}\\right\|\+\|atrNtra​∑n=1NtraM2​\(𝐱tr,na,\+1\)−ctrNtrb​∑n=1NtrbM2​\(𝐱tr,nb,\+1\)\|\\displaystyle\+\\left\|\\frac\{a\_\{\{\\rm tr\}\}\}\{N^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\}\\sum\_\{n=1\}^\{N^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\}M\_\{2\}\(\{\\bf x\}^\{\{\\rm a\}\}\_\{\{\\rm tr\},n\},\+1\)\-\\frac\{c\_\{\{\\rm tr\}\}\}\{N^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\}\\sum\_\{n=1\}^\{N^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\}M\_\{2\}\(\{\\bf x\}^\{\{\\rm b\}\}\_\{\{\\rm tr\},n\},\+1\)\\right\|\+\|dtrNtrb​∑n=1NtrbM2​\(𝐱tr,nb,−1\)−btrNtra​∑n=1NtraM2​\(𝐱tr,na,−1\)\|,\\displaystyle\+\\left\|\\frac\{d\_\{\{\\rm tr\}\}\}\{N^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\}\\sum\_\{n=1\}^\{N^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\}M\_\{2\}\(\{\\bf x\}^\{\{\\rm b\}\}\_\{\{\\rm tr\},n\},\-1\)\-\\frac\{b\_\{\{\\rm tr\}\}\}\{N^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\}\\sum\_\{n=1\}^\{N^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\}M\_\{2\}\(\{\\bf x\}^\{\{\\rm a\}\}\_\{\{\\rm tr\},n\},\-1\)\\right\|,\(20\)wherek1:=−πte/αk\_\{1\}:=\-\\pi\_\{\{\\rm te\}\}/\\alpha,k2:=−\(1−πte\)/αk\_\{2\}:=\-\(1\-\\pi\_\{\{\\rm te\}\}\)/\\alpha, and we omit constant terms that do not depend onmm\.

### 4\.4Classifier Loss Function

Our loss function for learning classifierffis a weighted sum of the empirical UU risk on the test distribution and the importance\-weighted empirical UU risk on the training distribution in Eq\. \([12](https://arxiv.org/html/2609.10994#S4.Ex7)\):

L^​\(f,m\):=β​R^te​\(f\)\+\(1−β\)​R^trw​\(f,m\),\\displaystyle\{\\hat\{L\}\}\(f,m\):=\\beta\{\\hat\{R\}\}\_\{\{\\rm te\}\}\(f\)\+\(1\-\\beta\)\{\\hat\{R\}\}\_\{\{\\rm tr\}\}^\{\{\\rm w\}\}\(f,m\),\(21\)whereβ∈\[0,1\)\\beta\\in\[0,1\)is a weighting hyperparameter, andR^te​\(f\)\{\\hat\{R\}\}\_\{\{\\rm te\}\}\(f\)is the empirical test UU risk calculated withXtea∪XtebX^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm te\}\},

R^te​\(f\)\\displaystyle\{\\hat\{R\}\}\_\{\{\\rm te\}\}\(f\)=\|ateNtea​∑n=1Nteaℓ⁡\(f⁡\(𝐱te,na\),\+1\)−cteNteb​∑n=1Ntebℓ⁡\(f⁡\(𝐱te,nb\),\+1\)\|\\displaystyle=\\left\|\\frac\{a\_\{\{\\rm te\}\}\}\{N\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\}\\sum\_\{n=1\}^\{N\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\}\\ell\(f\(\{\\bf x\}\_\{\{\\rm te\},n\}^\{\{\\rm a\}\}\),\+1\)\-\\frac\{c\_\{\{\\rm te\}\}\}\{N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\}\\sum\_\{n=1\}^\{N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\}\\ell\(f\(\{\\bf x\}\_\{\{\\rm te\},n\}^\{\{\\rm b\}\}\),\+1\)\\right\|\+\|dteNteb​∑n=1Ntebℓ⁡\(f⁡\(𝐱te,nb\),−1\)−bteNtea​∑n=1Nteaℓ⁡\(f⁡\(𝐱te,na\),−1\)\|\.\\displaystyle\+\\left\|\\frac\{d\_\{\{\\rm te\}\}\}\{N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\}\\sum\_\{n=1\}^\{N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\}\\ell\(f\(\{\\bf x\}\_\{\{\\rm te\},n\}^\{\{\\rm b\}\}\),\-1\)\-\\frac\{b\_\{\{\\rm te\}\}\}\{N\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\}\\sum\_\{n=1\}^\{N\_\{\{\\rm te\}\}^\{\{\\rm a\}\}\}\\ell\(f\(\{\\bf x\}\_\{\{\\rm te\},n\}^\{\{\\rm a\}\}\),\-1\)\\right\|\.\(22\)Here, we use Eq\. \([6](https://arxiv.org/html/2609.10994#S3.E6)\) to derive Eq\. \([22](https://arxiv.org/html/2609.10994#S4.Ex13)\) and explicitly describe the dependency of the importance weight modelmminR^trw\{\\hat\{R\}\}\_\{\{\\rm tr\}\}^\{\{\\rm w\}\}for clarity\. Although unlabeled dataXtea∪XtebX^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm te\}\}are used for estimating the importance weights, they are not directly used for learningffinR^trw\{\\hat\{R\}\}\_\{\{\\rm tr\}\}^\{\{\\rm w\}\}\. By using both empirical risks, we can use all available dataXtra∪Xtrb∪Xtea∪XtebX^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\\cup X^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm te\}\}for learningffdirectly\.

### 4\.5Training

To computeR^trw​\(f,m\)\{\\hat\{R\}\}\_\{\{\\rm tr\}\}^\{\{\\rm w\}\}\(f,m\), the conventional importance weighting approach first estimates the importance weights and subsequently uses them to calculate the importance\-weighted risk\([60](https://arxiv.org/html/2609.10994#bib.bib60),[52](https://arxiv.org/html/2609.10994#bib.bib52),[17](https://arxiv.org/html/2609.10994#bib.bib17),[3](https://arxiv.org/html/2609.10994#bib.bib3)\)\. However, the importance weight is difficult to estimate when using complex models such as neural networks or complex data such as image data\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[27](https://arxiv.org/html/2609.10994#bib.bib27),[18](https://arxiv.org/html/2609.10994#bib.bib18),[42](https://arxiv.org/html/2609.10994#bib.bib42)\]\. Because the estimation error directly affects the subsequent classifier learning, this two\-step approach often does not work well\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\. To alleviate this, recent methods use a dynamic approach that iterates between the importance weight estimation and classifier learning while sharing a neural network for feature extraction\[[7](https://arxiv.org/html/2609.10994#bib.bib7),[8](https://arxiv.org/html/2609.10994#bib.bib8),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\. By sharing the feature extractor, they can effectively perform the importance weighting\.

Algorithm 1Training procedure of the proposed method0:UU data in the training and test distributions

Xtra∪Xtrb∪Xtea∪XtebX^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\\cup X^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm te\}\}, mini\-batch sizes for the training and test distributions

\(Btr,Bte\)\(B\_\{\{\\rm tr\}\},B\_\{\{\\rm te\}\}\), positive class\-priors

\(πtr,πte,θtra,θtrb,θtea,θteb\)\(\\pi\_\{\{\\rm tr\}\},\\pi\_\{\{\\rm te\}\},\\theta^\{a\}\_\{\{\\rm tr\}\},\\theta^\{b\}\_\{\{\\rm tr\}\},\\theta^\{a\}\_\{\{\\rm te\}\},\\theta^\{b\}\_\{\{\\rm te\}\}\), relative parameter

α\\alpha, and weighting parameter

β\\beta\.

0:Parameters of neural networks

hh,

uu, and

vv\.

1:repeat

2:Sample UU data with size

BtrB\_\{\{\\rm tr\}\}from

Xtra∪XtrbX^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}
3:Sample UU data with size

BteB\_\{\{\\rm te\}\}from

Xtea∪XtebX^\{\{\\rm a\}\}\_\{\{\\rm te\}\}\\cup X^\{\{\\rm b\}\}\_\{\{\\rm te\}\}
4:\# Importance weight estimation

5:Calculate loss in Eq\. \([20](https://arxiv.org/html/2609.10994#S4.Ex10)\) on the sampled data

6:Update parameters of

vvwith the gradient of the loss fixing feature extractor

hh
7:\# Classifier learning

8:Calculate the loss in Eq\. \([21](https://arxiv.org/html/2609.10994#S4.E21)\) on the sampled data with current importance weights

9:Update classifier parameters of

uuand

hhwith the gradient of the loss fixing the importance weights

10:untilEnd condition is satisfied;

The proposed method also follows this dynamic approach\. Specifically, we use the following neural networks for modeling classifiers and importance weights,

f⁡\(𝐱\):=u⁡\(h⁡\(𝐱\)\),m⁡\(𝐱,y\):=v⁡\(\[h⁡\(𝐱\),y\]\),\\displaystyle f\(\{\\bf x\}\):=u\(h\(\{\\bf x\}\)\),\\ m\(\{\\bf x\},y\):=v\(\[h\(\{\\bf x\}\),y\]\),\(23\)whereh:𝒳→ℝKh:\{\\cal X\}\\to\\mathbb\{R\}^\{K\},u:ℝK→ℝu:\\mathbb\{R\}^\{K\}\\to\\mathbb\{R\}, andv:ℝK\+2→ℝv:\\mathbb\{R\}^\{K\+2\}\\to\\mathbb\{R\}are neural networks for feature extraction, classifiers, and importance weights, respectively\. Here, we assume that labely∈\{−1,\+1\}y\\in\\\{\-1,\+1\\\}is represented as one\-hot encoding and\[⋅,⋅\]\[\\cdot,\\cdot\]is a concatenation\. Algorithm[1](https://arxiv.org/html/2609.10994#alg1)shows the training procedure of our method with stochastic gradient descent methods\. We first randomly sample the mini\-batch data \(UU data\) from the training and test distributions \(Lines 2–3\)\. Then, we calculate the loss in Eq\. \([20](https://arxiv.org/html/2609.10994#S4.Ex10)\) for the importance weight estimation \(Line 5\) and update parameters ofvvwith the gradient of the loss fixing feature extractorhh\(Line 6\)\. We fixedhhto avoid overfitting as in\([7](https://arxiv.org/html/2609.10994#bib.bib7),[27](https://arxiv.org/html/2609.10994#bib.bib27)\)\. Then, by using the estimated importance weights, we calculate the loss in Eq\. \([21](https://arxiv.org/html/2609.10994#S4.E21)\) for classifier learning \(Line 8\)\. We update parameters of classifieruuandhhby using the gradient of the loss \(Line 9\)\. In this step, we fixed the importance weights to avoid learning a meaningless model,m⁡\(𝐱,y\)=0m\(\{\\bf x\},y\)=0for all𝐱\{\\bf x\}andyy\.

## 5Experiments

### 5\.1Data

We used four real\-world datasets in the main paper: MNIST\([28](https://arxiv.org/html/2609.10994#bib.bib28)\), FashionMNIST \(FMNIST\)\([57](https://arxiv.org/html/2609.10994#bib.bib57)\), CIFAR10\([22](https://arxiv.org/html/2609.10994#bib.bib22)\), and DIABETES\([12](https://arxiv.org/html/2609.10994#bib.bib12)\)\. They have been commonly used in recent distribution shift adaptation studies\[[27](https://arxiv.org/html/2609.10994#bib.bib27),[26](https://arxiv.org/html/2609.10994#bib.bib26),[7](https://arxiv.org/html/2609.10994#bib.bib7),[41](https://arxiv.org/html/2609.10994#bib.bib41)\]\. MNIST and FMNIST are28×2828\\times 28grayscale image datasets, while CIFAR10 consists of32×3232\\times 32RGB images\. DIABETES is a tabular dataset with distribution shift\([12](https://arxiv.org/html/2609.10994#bib.bib12)\), where the task is to predict whether a respondent has diabetes from142142dimensional features\. Data from the training and test distributions are “white non\-hispanic” and other racial groups, respectively\.

For MNIST, FMNIST, and CIFAR10, following the previous studies\[[27](https://arxiv.org/html/2609.10994#bib.bib27),[8](https://arxiv.org/html/2609.10994#bib.bib8)\], we constructed binary classification tasks by partitioning the original classes of each dataset into positive and negative classes, and created two types of distribution shifts: the first one is the support shift, where the support of input𝐱\{\\bf x\}changes between the training and testing phases\. This shift is a particularly challenging type of covariate shift\[[8](https://arxiv.org/html/2609.10994#bib.bib8),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\. The second one is the input\-output relation shift \(concept shift\), where the inputs’ support does not change butp⁡\(y\|𝐱\)p\(y\|\{\\bf x\}\)varies between the training and testing phases\. Note that no methods use the knowledge that such shifts have occurred in our experiments\. The detailed construction procedure is provided in Section[A](https://arxiv.org/html/2609.10994#A1)\.

For UU data in the training distributionXtraX^\{\{\\rm a\}\}\_\{\{\\rm tr\}\}andXtrbX^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}, we set the numbers of the instances\(Ntra,Ntrb\)\(N\_\{\{\\rm tr\}\}^\{a\},N\_\{\{\\rm tr\}\}^\{b\}\)to\(2500,2500\)\(2500,2500\)\. For UU data in the test distributionXteaX^\{\{\\rm a\}\}\_\{\{\\rm te\}\}andXtebX^\{\{\\rm b\}\}\_\{\{\\rm te\}\}, we changed the numbers of the instances\(Ntea,Nteb\)\(N\_\{\{\\rm te\}\}^\{a\},N\_\{\{\\rm te\}\}^\{b\}\)within\{\(50,50\),\(100,100\),\(150,150\)\}\\\{\(50,50\),\(100,100\),\(150,150\)\\\}\. For these UU datasets, we changed the class\-priors\(θa,θb\):=\(θtra,θtrb\)=\(θtea,θteb\)\(\\theta^\{\{\\rm a\}\},\\theta^\{\{\\rm b\}\}\):=\(\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\},\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\)=\(\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\},\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\)within\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\}\\\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\\\}as in the previous studies\[[35](https://arxiv.org/html/2609.10994#bib.bib35)\]\. We also used different UU data in the test distribution where each size is100100as validation data\. We used20002000positive and20002000negative data in the test distribution as test data for evaluation \(i\.e\.,πte=0\.5\\pi\_\{\{\\rm te\}\}=0\.5\)\. Training, validation, and test datasets did not overlap\. We conducted 10 experiments for each pair of the number of UU data in the test distribution and the class\-priors while changing the random seeds and evaluated the mean test accuracy\.

### 5\.2Comparison Methods

We compared our method with five UU learning\-based methods: UU learning method on the test distribution \(teUU\)\[[35](https://arxiv.org/html/2609.10994#bib.bib35)\], UU learning method on the training distribution \(trUU\), multi\-task learning method of UU learning \(mtUU\), multi\-task learning method of UU learning with a single neural network \(mtsUU\), and domain adaptation method for UU learning \(daUU\)\. All methods including our method used neural networks for classifiers and the absolute value correction to prevent overfitting\.

teUU and trUU learn the classifier by minimizing the non\-negative empirical risk \(in Eq\. \([6](https://arxiv.org/html/2609.10994#S3.E6)\)\) with UU data in the test and training distributions, respectively\. Note that teUU and trUU are state\-of\-the\-art UU learning methods\. Since no distribution adaptation methods for UU learning exist, we created several adaptation methods \(mtUU, mtsUU, and daUU\)\. mtUU and mtsUU learn the classifier by minimizing the weighted sum of the non\-negative empirical UU risks on the training and test distributions\. mtUU used a two\-head neural network for handling the difference between the training and test distributions\. mtsUU used the single neural network as in the proposed method\. The difference between the proposed method and mtsUU is whether or not the importance weights are used in Eq\. \([21](https://arxiv.org/html/2609.10994#S4.E21)\) \(i\.e\., mtsUU usedm⁡\(𝐱,y\)=1m\(\{\\bf x\},y\)=1for all𝐱,y\{\\bf x\},y\)\. daUU minimizes the weighted sum of the non\-negative empirical UU risks on both distributions \(i\.e\., the loss of mtsUU\) while minimizing the feature discrepancy to obtain invariant features as in\([49](https://arxiv.org/html/2609.10994#bib.bib49)\)\. We used the maximum mean discrepancy \(MMD\) loss\([30](https://arxiv.org/html/2609.10994#bib.bib30)\)to minimize the feature discrepancy, which is commonly used for distribution shift adaptation\([9](https://arxiv.org/html/2609.10994#bib.bib9),[27](https://arxiv.org/html/2609.10994#bib.bib27)\)\. The details of network architectures and hyperparameters are described in Sections[B](https://arxiv.org/html/2609.10994#A2)and[C](https://arxiv.org/html/2609.10994#A3)\.

### 5\.3Results

Table 1:Average test accuracies over different UU test data sizes\(Ntea,Nteb\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)within\{\(50,50\),\(100,100\),\(150,150\)\}\\\{\(50,50\),\(100,100\),\(150,150\)\\\}and class\-priors\(θa,θb\)\(\\theta^\{\{\\rm a\}\},\\theta^\{\{\\rm b\}\}\)within\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\}\\\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\\\}\. In the Shift column, ‘S’ and ‘IO’ represent the support shift and input\-output relation shift, respectively\. Values in bold are not statistically different at the5%5\\%level from the best performing method in each row according to a paired t\-test\.Table 2:Results with a few PN data in the test distribution: Average test accuracies over different UU test data sizes\(Ntea,Nteb\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)within\{\(10,10\),\(50,50\),\(100,100\)\}\\\{\(10,10\),\(50,50\),\(100,100\)\\\}and training class\-priors\(θtra,θtrb\)\(\\theta\_\{\{\\rm tr\}\}^\{\{\\rm a\}\},\\theta\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\)within\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\}\\\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\\\}\. Class\-priors of UU test data\(θtea,θteb\)\(\\theta\_\{\{\\rm te\}\}^\{\{\\rm a\}\},\\theta\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)were set to\(1,0\)\(1,0\)\.Table[1](https://arxiv.org/html/2609.10994#S5.T1)shows the average test accuracies for each dataset\. The full results with the standard deviations are described in subsection[D\.11](https://arxiv.org/html/2609.10994#A4.SS11)\. The proposed method performed the best or comparably to it in almost all cases \(6 out of 7 cases\)\. teUU and trUU tended to perform worse than the proposed method because teUU used only a small amount of UU data in the test distribution and trUU used only UU data in the training distribution\. Although mtsUU and mtUU used UU data in both distributions as in the proposed method, their performance was limited since they do not have an explicit mechanism for transfer\. As mtsUU is the proposed method without importance weights, the result highlights the effectiveness of the importance weights\. Although daUU aims to transfer by learning invariant features, it does not explicitly target the test risk\. In contrast, the proposed method achieved better performance in more cases by directly minimizing the test risk within the importance\-weighting framework\. The results with each number of UU data in the test distribution and with each class\-priors were described in subsections[D\.1](https://arxiv.org/html/2609.10994#A4.SS1)and[D\.2](https://arxiv.org/html/2609.10994#A4.SS2), respectively\. The proposed method worked well on each case\.

Figure 2:Importance weight distribution of the proposed method on MNIST with the support shift\.Figure[2](https://arxiv.org/html/2609.10994#S5.F2)shows the distribution of estimated importance weights of the proposed method on MNIST dataset with the support shift when\(Ntea,Nteb\)=\(50,50\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)=\(50,50\)and\(θa,θb\)=\(0\.7,0\.3\)\(\\theta^\{\{\\rm a\}\},\\theta^\{\{\\rm b\}\}\)=\(0\.7,0\.3\)\. ‘High’ and ‘Low’ represent data in the test distribution and data that are not in the test distribution, respectively\. The proposed method was able to correctly estimate importance weights, i\.e\., the weights of ‘High’ and ‘Low’ are large and small, respectively\.

Table[2](https://arxiv.org/html/2609.10994#S5.T2)shows the results when types of supervision were different between the training and testing phases: a few PN data in the test distribution and noisy labeled data in the training distribution\. We evaluated this setting because a few data might be accurately labeled by domain experts\. Considering the cost of precise labeling, we used fewer instances than in previous setting\. The proposed method also performed the best or comparably to it in 6 out of 7 cases\. This result shows that the proposed method works well even with different types of supervision\.

## 6Conclusion

In this paper, we proposed a distribution shift adaptation method for UU learning with the importance weighting\. Our method can handle various types of learning problems under distribution shift such as PN, PU, noisy label, and similarity\-based learning\. Moreover, it can perform adaptation without assumptions of the shift types\. The experiments showed the effectiveness of our method\.

## References

- \[1\]H\. Bao, G\. Niu, and M\. Sugiyama\.Classification from pairwise similarity and unlabeled data\.InICML, 2018\.
- \[2\]J\. Bekker and J\. Davis\.Learning from positive and unlabeled data: A survey\.Machine Learning, 109\(4\):719–760, 2020\.
- \[3\]S\. Bickel, M\. Brückner, and T\. Scheffer\.Discriminative learning under covariate shift\.Journal of Machine Learning Research, 10\(9\), 2009\.
- \[4\]N\. Charoenphakdee, J\. Lee, and M\. Sugiyama\.On symmetric losses for learning from corrupted labels\.InICML, 2019\.
- \[5\]C\.\-K\. Chiang and M\. Sugiyama\.Unified risk analysis for weakly supervised learning\.Transactions on Machine Learning Research, 2025\.
- \[6\]M\. Du Plessis, G\. Niu, and M\. Sugiyama\.Convex formulation for learning from positive and unlabeled data\.InICML, 2015\.
- \[7\]T\. Fang, N\. Lu, G\. Niu, and M\. Sugiyama\.Rethinking importance weighting for deep learning under distribution shift\.NeurIPS, 2020\.
- \[8\]T\. Fang, N\. Lu, G\. Niu, and M\. Sugiyama\.Generalizing importance weighting to a universal solver for distribution shift problems\.NeurIPS, 2023\.
- \[9\]A\. Farahani, S\. Voghoei, K\. Rasheed, and H\. R\. Arabnia\.A brief review of domain adaptation\.ICDATA, 2021\.
- \[10\]L\. Feng, S\. Shu, N\. Lu, B\. Han, M\. Xu, G\. Niu, B\. An, and M\. Sugiyama\.Pointwise binary classification with pairwise confidence comparisons\.InICML, 2021\.
- \[11\]Y\. Ganin and V\. Lempitsky\.Unsupervised domain adaptation by backpropagation\.InICML, 2015\.
- \[12\]J\. Gardner, Z\. Popovic, and L\. Schmidt\.Benchmarking distribution shift in tabular data with tableshift\.NeurIPS, 2023\.
- \[13\]Z\. Hammoudeh and D\. Lowd\.Learning from positive and unlabeled data with arbitrary positive shift\.NeurIPS, 2020\.
- \[14\]Z\. Han, X\.\-J\. Gui, H\. Sun, Y\. Yin, and S\. Li\.Towards accurate and robust domain adaptation under multiple noisy environments\.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45\(5\):6460–6479, 2022\.
- \[15\]T\. Ishida, I\. Yamane, T\. Sakai, G\. Niu, and M\. Sugiyama\.Do we need zero training loss after achieving zero training error?InICML, 2020\.
- \[16\]S\. Jain, M\. White, and P\. Radivojac\.Estimating the class prior and posterior from noisy positives and unlabeled data\.NeurIPS, 2016\.
- \[17\]T\. Kanamori, S\. Hido, and M\. Sugiyama\.A least\-squares approach to direct importance estimation\.The Journal of Machine Learning Research, 10:1391–1445, 2009\.
- \[18\]M\. Kato and T\. Teshima\.Non\-negative bregman divergence minimization for deep direct density ratio estimation\.InICML, 2021\.
- \[19\]D\. P\. Kingma and J\. Ba\.Adam: a method for stochastic optimization\.arXiv preprint arXiv:1412\.6980, 2014\.
- \[20\]R\. Kiryo, G\. Niu, M\. C\. Du Plessis, and M\. Sugiyama\.Positive\-unlabeled learning with non\-negative risk estimator\.NeurIPS, 2017\.
- \[21\]A\. Krause, P\. Perona, and R\. Gomes\.Discriminative clustering by regularized information maximization\.NeurIPS, 2010\.
- \[22\]A\. Krizhevsky, G\. Hinton, et al\.Learning multiple layers of features from tiny images\.2009\.
- \[23\]A\. Kumagai and T\. Iwata\.Unsupervised domain adaptation by matching distributions based on the maximum mean discrepancy via unilateral transformations\.InAAAI, 2019\.
- \[24\]A\. Kumagai, T\. Iwata, and Y\. Fujiwara\.Transfer anomaly detection by inferring latent domain representations\.NeurIPS, 2019\.
- \[25\]A\. Kumagai, T\. Iwata, and Y\. Fujiwara\.Meta\-learning for relative density\-ratio estimation\.InNeurIPS, 2021\.
- \[26\]A\. Kumagai, T\. Iwata, H\. Takahashi, T\. Nishiyama, and Y\. Fujiwara\.Auc maximization under positive distribution shift\.NeurIPS, 2024\.
- \[27\]A\. Kumagai, T\. Iwata, H\. Takahashi, T\. Nishiyama, and Y\. Fujiwara\.Importance\-weighted positive\-unlabeled learning for distribution shift adaptation\.InAISTATS, 2025\.
- \[28\]Y\. LeCun, L\. Bottou, Y\. Bengio, and P\. Haffner\.Gradient\-based learning applied to document recognition\.Proceedings of the IEEE, 86\(11\):2278–2324, 1998\.
- \[29\]C\.\-Y\. Lee, T\. Batra, M\. H\. Baig, and D\. Ulbricht\.Sliced wasserstein discrepancy for unsupervised domain adaptation\.InCVPR, 2019\.
- \[30\]Y\. Li, K\. Swersky, and R\. Zemel\.Generative moment matching networks\.InICML, 2015\.
- \[31\]T\. Liu and D\. Tao\.Classification with noisy labels by importance reweighting\.IEEE Transactions on pattern analysis and machine intelligence, 38\(3\):447–461, 2015\.
- \[32\]N\. Lu, S\. Lei, G\. Niu, I\. Sato, and M\. Sugiyama\.Binary classification from multiple unlabeled datasets via surrogate set classification\.InICML, 2021\.
- \[33\]N\. Lu, G\. Niu, A\. K\. Menon, and M\. Sugiyama\.On the minimal supervision for training any binary classifier from only unlabeled data\.InICLR, 2019\.
- \[34\]N\. Lu, T\. Zhang, T\. Fang, T\. Teshima, and M\. Sugiyama\.Rethinking importance weighting for transfer learning\.InFederated and Transfer Learning, pages 185–231\. Springer, 2022\.
- \[35\]N\. Lu, T\. Zhang, G\. Niu, and M\. Sugiyama\.Mitigating overfitting in supervised classification from two unlabeled datasets: a consistent risk correction approach\.InAISTATS, 2020\.
- \[36\]A\. Menon, B\. Van Rooyen, C\. S\. Ong, and B\. Williamson\.Learning from corrupted binary labels via class\-probability estimation\.InICML, 2015\.
- \[37\]J\. G\. Moreno\-Torres, T\. Raeder, R\. Alaiz\-Rodríguez, N\. V\. Chawla, and F\. Herrera\.A unifying view on dataset shift in classification\.Pattern recognition, 45\(1\):521–530, 2012\.
- \[38\]S\. Motiian, M\. Piccirilli, D\. A\. Adjeroh, and G\. Doretto\.Unified deep supervised domain adaptation and generalization\.InICCV, 2017\.
- \[39\]S\. J\. Pan and Q\. Yang\.A survey on transfer learning\.IEEE Transactions on knowledge and data engineering, 22\(10\):1345–1359, 2009\.
- \[40\]A\. Paszke, S\. Gross, S\. Chintala, G\. Chanan, E\. Yang, Z\. DeVito, Z\. Lin, A\. Desmaison, L\. Antiga, and A\. Lerer\.Automatic differentiation in pytorch\.2017\.
- \[41\]Y\.\-Y\. Qian, P\. Zhao, Y\.\-J\. Zhang, M\. Sugiyama, and Z\.\-H\. Zhou\.Efficient non\-stationary online learning by wavelets with applications to online distribution shift adaptation\.InICML, 2024\.
- \[42\]B\. Rhodes, K\. Xu, and M\. U\. Gutmann\.Telescoping density\-ratio estimation\.NeurIPS, 2020\.
- \[43\]K\. Saito, K\. Watanabe, Y\. Ushiku, and T\. Harada\.Maximum classifier discrepancy for unsupervised domain adaptation\.InCVPR, 2018\.
- \[44\]T\. Sakai and N\. Shimizu\.Covariate shift adaptation on learning from positive and unlabeled data\.InAAAI, 2019\.
- \[45\]T\. Shimada, H\. Bao, I\. Sato, and M\. Sugiyama\.Classification from pairwise similarities/dissimilarities and unlabeled data via empirical risk minimization\.Neural Computation, 33\(5\):1234–1268, 2021\.
- \[46\]H\. Shimodaira\.Improving predictive inference under covariate shift by weighting the log\-likelihood function\.Journal of statistical planning and inference, 90\(2\):227–244, 2000\.
- \[47\]Y\. Shu, Z\. Cao, M\. Long, and J\. Wang\.Transferable curriculum for weakly\-supervised domain adaptation\.InAAAI, 2019\.
- \[48\]P\. Singhal, R\. Walambe, S\. Ramanna, and K\. Kotecha\.Domain adaptation: challenges, methods, datasets, and applications\.IEEE access, 11:6973–7020, 2023\.
- \[49\]J\. Sonntag, G\. Behrens, and L\. Schmidt\-Thieme\.Positive\-unlabeled domain adaptation\.arXiv preprint arXiv:2202\.05695, 2022\.
- \[50\]M\. Sugiyama, H\. Bao, T\. Ishida, N\. Lu, and T\. Sakai\.Machine learning from weak supervision: an empirical risk minimization approach\.MIT Press, 2022\.
- \[51\]M\. Sugiyama, M\. Krauledat, and K\.\-R\. Müller\.Covariate shift adaptation by importance weighted cross validation\.Journal of Machine Learning Research, 8\(5\), 2007\.
- \[52\]M\. Sugiyama, T\. Suzuki, and T\. Kanamori\.Density ratio estimation in machine learning\.Cambridge University Press, 2012\.
- \[53\]C\. Tan, F\. Sun, T\. Kong, W\. Zhang, C\. Yang, and C\. Liu\.A survey on deep transfer learning\.InICANN, 2018\.
- \[54\]V\. Vapnik\.Statistical learning theory\.John Wiley & Sons google schola, 2:831–842, 1998\.
- \[55\]G\. Wilson and D\. J\. Cook\.A survey of unsupervised deep domain adaptation\.ACM Transactions on Intelligent Systems and Technology, 11\(5\):1–46, 2020\.
- \[56\]Y\. Wu, X\. Xia, J\. Yu, B\. Han, G\. Niu, M\. Sugiyama, and T\. Liu\.Making binary classification from multiple unlabeled datasets almost free of supervision\.arXiv preprint arXiv:2306\.07036, 2023\.
- \[57\]H\. Xiao, K\. Rasul, and R\. Vollgraf\.Fashion\-mnist: a novel image dataset for benchmarking machine learning algorithms\.arXiv preprint arXiv:1708\.07747, 2017\.
- \[58\]Z\. Xie, Y\. Liu, H\.\-Y\. He, M\. Li, and Z\.\-H\. Zhou\.Weakly supervised auc optimization: a unified partial auc approach\.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024\.
- \[59\]L\. Xu, J\. Neufeld, B\. Larson, and D\. Schuurmans\.Maximum margin clustering\.NeurIPS, 2004\.
- \[60\]M\. Yamada, T\. Suzuki, T\. Kanamori, H\. Hachiya, and M\. Sugiyama\.Relative density\-ratio estimation for robust distribution comparison\.Neural computation, 25\(5\):1324–1370, 2013\.
- \[61\]S\. Yu, Z\. Zhu, B\. Liu, A\. K\. Jain, and J\. Zhou\.Robust unsupervised domain adaptation from a corrupted source\.InICDM, 2022\.
- \[62\]X\. Yu, T\. Liu, M\. Gong, K\. Zhang, K\. Batmanghelich, and D\. Tao\.Label\-noise robust domain adaptation\.InICML, 2020\.
- \[63\]H\. Zhao, R\. T\. Des Combes, K\. Zhang, and G\. Gordon\.On learning invariant representations for domain adaptation\.InICML, 2019\.

## Appendix AConstruction Procedure of Support and Input\-output Relation Shifts

In this section, we describe the construction procedure for the distribution shifts, which is the same as that used in the previous study\[[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\.

We first describe the procedure to construct the support shift\. For MNIST, even and odd digits were treated as negative and positive, respectively\. We used digits 0, 2, and 4 \(4, 6, and 8\) for negative\- and digits 1, 3, and 5 \(5, 7, and 9\) for positive\-conditional densities of the training \(test\) distribution\. For FMNIST, the upper garments \(0, 2, 3, 4, and 6\) and the others were treated as negative and positive, respectively, where numbers in parentheses represent class labels\. We used class labels 0, 2, and 3 \(3, 4, and 6\) for negative\- and class labels 1, 5, and 7 \(7, 8, and 9\) for positive\-conditional densities of the training \(test\) distribution\. For CIFAR10, the animal categories \(2, 3, 4, 5, 6, and 7\) and the others were treated as negative and positive, respectively\. We used class labels 2, 3, 4, and 5 \(4, 5, 6, and 7\) as negative and class labels 0, 1, and 8 \(1, 8, and 9\) as positive in the training \(test\) distribution\.

We next describe the procedure to construct the input\-output shift\. For MNIST, we used even digits for negative\- and odd digits for positive\-conditional densities of the test distribution\. For the training distribution, we swapped the digits 0 and 2 with 1 and 3 \(e\.g\., the digits 0, 2, 5, 7, and 9 were used for the positive\-conditional density of the training distribution\)\. For FMNIST, we used upper garments \(0, 2, 3, 4, and 6\) for negative\- and the others for positive\-conditional densities of the test distribution\. We swapped the class labels 0 and 2 with 1 and 5 in the training distribution\. For CIFAR10, we used the animal categories \(2, 3, 4, 5, 6, and 7\) for negative\- and the vehicles for positive\-conditional densities of the test distribution\. We swapped the class labels 2 and 3 with 0 and 1 in the training distribution\.

## Appendix BNeural Network Architectures

For MNIST, FMNIST, and DIABETES, a three\-layered feed\-forward neural network with ReLU activation was used for feature extractorhh\. The number of hidden and output nodes was128128\. For CIFAR10, a convolutional neural network, which consisted of two convolutional blocks followed by a two\-layered feed\-forward neural network, was used for feature extractorhh\. The first \(second\) convolutional block comprised a 6 \(16\) filter5×55\\times 5convolution, the ReLU activation, and a2×22\\times 2max\-pooling layer\. The numbers of the hidden and output nodes were 120 and 84, respectively\. One\-layered and two\-layered feed\-forward neural networks were used foruuandvv, respectively\. For the output activation ofvv, we used1α​σ​\(⋅\)\\frac\{1\}\{\\alpha\}\\sigma\(\\cdot\), whereσ\\sigmais a sigmoid function, to match the value range of the relative density\-ratio \(importance weights\)\. For all comparison methods, the same neural network architecture \(i\.e\.,u⁡\(h⁡\(𝐱\)\)u\(h\(\{\\bf x\}\)\)\) was used for the classifier\. For mtUU, two different classifier heads \(i\.e\.,uu\) were used for the training and test distributions\.

## Appendix CHyperparameters

For all methods, the empirical UU risk with validation data on the test distribution was used to select hyperparameters and early\-stopping to mitigate overfitting\. For all methods, the sigmoid loss was used for lossℓ\\ellas in the previous study\[[20](https://arxiv.org/html/2609.10994#bib.bib20),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\. For the proposed method, relative parameterα\\alphawas selected from\{0\.1,0\.5,0\.9\}\\\{0\.1,0\.5,0\.9\\\}and training class\-priorπtr\\pi\_\{\{\\rm tr\}\}was set to0\.50\.5\. For the proposed method, mtUU, mtsUU, and daUU, weighting parameterβ\\betawas selected from\{0,0\.01,0\.1,0\.3,0\.5,0\.7,0\.9\}\\\{0,0\.01,0\.1,0\.3,0\.5,0\.7,0\.9\\\}\. Note that the proposed method, mtUU, and mtsUU are equivalent to teUU whenβ=1\\beta=1\. For daUU, the MMD loss was applied toh⁡\(𝐱\)h\(\{\\bf x\}\)\. The number of RBF kernel mixtures was set to55as in the previous study\[[30](https://arxiv.org/html/2609.10994#bib.bib30)\]\. The weighting parameter of the MMD loss was chosen from\{10−1,10−2,10−3\}\\\{10^\{\-1\},10^\{\-2\},10^\{\-3\}\\\}\. The mini\-batch sizes for the training and test distributions,BtrB\_\{\{\\rm tr\}\}andBteB\_\{\{\\rm te\}\}, were set to512512\. For all methods, we used the Adam optimizer\[[19](https://arxiv.org/html/2609.10994#bib.bib19)\]\. We set the learning rate to10−410^\{\-4\}and10−310^\{\-3\}for classifier learning and importance weight estimation, respectively\. The maximum number of epochs was200200\. All methods were implemented using Pytorch\[[40](https://arxiv.org/html/2609.10994#bib.bib40)\], and all experiments were conducted on a Linux server with an Intel Xeon CPU and A100 GPU\.

\(a\)MNIST\(S\)\(b\)MNIST\(IO\)\(c\)FMNIST\(S\)\(d\)FMNIST\(IO\)\(e\)CIFAR10\(S\)\(f\)CIFAR10\(IO\)\(g\)DIABETES
Figure 3:The average test accuracies with the standard errors when changing the number of UU data in the test distribution\.
## Appendix DAdditional Experimental Results

### D\.1Results with Different Numbers of UU Data in the Test Distribution

Figure[3](https://arxiv.org/html/2609.10994#A3.F3)shows the average test accuracies with the standard errors when changing the numbers of UU data in the test distribution in the case where data of both the training and test distributions are noisy\. As the number of UU data increased, the performance of the proposed method improved as expected\. The proposed method tended to work well in each UU data size\.

\(a\)MNIST\(S\)\(b\)MNIST\(IO\)\(c\)FMNIST\(S\)\(d\)FMNIST\(IO\)\(e\)CIFAR10\(S\)\(f\)CIFAR10\(IO\)\(g\)DIABETES
Figure 4:The average test accuracies with the standard errors when changing the class\-priors of UU data\.
### D\.2Results with Different Class\-priors of Given UU Datasets

Figure[4](https://arxiv.org/html/2609.10994#A4.F4)shows the average test accuracies with the standard errors when changing the class\-priors of UU data in the test distribution in the case where data of both the training and test distributions are noisy\. As the class\-priors of UU data became closer, the performance of each method decreased\. This is because when the class\-priors are close, two sets of unlabeled data become similar and thus are less informative to learn a binary classifier\. The proposed method tended to work well in each setting of the class\-priors\.

\(a\)MNIST\(S\)\(b\)MNIST\(IO\)\(c\)FMNIST\(S\)\(d\)FMNIST\(IO\)\(e\)CIFAR10\(S\)\(f\)CIFAR10\(IO\)\(g\)DIABETES
Figure 5:The average test accuracies with the standard errors of our method when changingβ\\beta\.\(a\)MNIST\(S\)\(b\)MNIST\(IO\)\(c\)FMNIST\(S\)\(d\)FMNIST\(IO\)\(e\)CIFAR10\(S\)\(f\)CIFAR10\(IO\)\(g\)DIABETES
Figure 6:The average test accuracies with the standard errors of our method when changingα\\alpha\.
### D\.3Dependency of Weighting Parameterβ\\beta

Figure[5](https://arxiv.org/html/2609.10994#A4.F5)shows the average test accuracies with the standard errors when changing weighting parameterβ\\betain the case where data of both the training and test distributions are noisy\. The value ofβ\\betais used in Eq\. \([21](https://arxiv.org/html/2609.10994#S4.E21)\) for controlling the effects of two empirical risks\. Note that the proposed method withβ=1\\beta=1is equivalent to teUU that uses only UU data in the test distribution\. Although the trend of the results varied across datasets, the proposed method was able to select goodβ\\betaby using validation UU data\.

### D\.4Dependency of Relative Parameterα\\alpha

Figure[6](https://arxiv.org/html/2609.10994#A4.F6)shows the average test accuracies with the standard errors when changing relative parameterα\\alphaof Eq\. \([13](https://arxiv.org/html/2609.10994#S4.E13)\) in the case where data of both the training and test distributions are noisy\. Although the trend varied across datasets, the performance tended not to change significantly even whenα\\alphawas changed\. This result shows that the proposed method is relatively robust against the value ofα\\alpha\.

### D\.5Comparison with Our Method that Sequentially Performs Label Estimation and Importance Weighting

The proposed method directly performs the importance weighting from given UU datasets\. However, one possible approach is to first estimate \(pseudo\) PN data from the given UU data using existing UU learning methods and then apply the importance weighting method for PN data\[[8](https://arxiv.org/html/2609.10994#bib.bib8)\]\. We compared this approach \(Ours w/ seq\) with the proposed method in Table[3](https://arxiv.org/html/2609.10994#A4.T3)\. Note that Ours w/ seq is also considered within our proposal since importance weighting methods for UU data do not exist\. The proposed method performed comparably to or even better than Ours w/ seq\. Since Ours w/ seq sequentially performs the label estimation and the importance weighting, the error on the label estimation propagates the subsequent importance weighting, which may lead to the performance degrading in some cases\. In contrast, since the proposed method performs both simultaneously, it might be more robust\.

Table 3:Comparison with the proposed method that sequentially performs the label estimation and the importance weighting \(Ours w/ seq\): average test accuracies over different UU data sizes in the test distribution and class\-priors\. Values in bold are not statistically different at the5%5\\%level from the best performing method in each row according to a paired t\-test\.
### D\.6Results with Noisy Class\-priors

Hitherto, we have assumed that true class\-priors of given UU datasetsθtra\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\},θtrb\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\},θtea\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}, andθteb\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}are known to train classifiers\. However, they might be unavailable in practice\. Therefore, we investigated the performance of the proposed method with noisy class\-priors\.

We first considered independent noise in the class\-priors following the previous work\[[32](https://arxiv.org/html/2609.10994#bib.bib32)\]\. Specifically, we first set true class\-priors\(θa,θb\):=\(θtra,θtrb\)=\(θtea,θteb\)\(\\theta^\{\\rm a\},\\theta^\{\\rm b\}\):=\(\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\},\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\}\)=\(\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\},\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\)for the data generation to\(0\.7,0\.3\)\(0\.7,0\.3\)\. Then, we replaced eachθ∈\{θtra,θtrb,θtea,θteb\}\\theta\\in\\\{\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\},\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\},\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\},\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}\\\}byθ′=θ\+γ​σ\\theta^\{\\prime\}=\\theta\+\\gamma\\sigmawhereγ\\gammauniform randomly takes a value in\{−1,\+1\}\\\{\-1,\+1\\\}andσ∈\{0\.05,0\.1,0\.15,0\.195\}\\sigma\\in\\\{0\.05,0\.1,0\.15,0\.195\\\}\. We used each noisyθ′\\theta^\{\\prime\}as the class\-prior for training classifiers\. Table[4](https://arxiv.org/html/2609.10994#A4.T4)shows the average test accuracies when changing values ofσ\\sigma\. Here,σ=0\\sigma=0means that there is no noise\. As the value ofσ\\sigma\(magnitude of noise\) increased, the performance of the proposed method tended to decrease\. This result is reasonable since information on class\-priors is only supervision to train a classifier\.

We next considered a simple correlated/systematic noise setting, in which all class\-priors of given UU datasets,θtra\\theta^\{\{\\rm a\}\}\_\{\{\\rm tr\}\},θtrb\\theta^\{\{\\rm b\}\}\_\{\{\\rm tr\}\},θtea\\theta^\{\{\\rm a\}\}\_\{\{\\rm te\}\}, andθteb\\theta^\{\{\\rm b\}\}\_\{\{\\rm te\}\}, are jointly biased in the same direction rather than perturbed independently\. Specifically, we replaced eachθ\\thetabyθ′=θ\+σ\\theta^\{\\prime\}=\\theta\+\\sigma\. We chose this setting because, in practice, estimates or prior knowledge may tend to be consistently overestimated or underestimated, rather than perturbed independently\. Table[5](https://arxiv.org/html/2609.10994#A4.T5)shows the average test accuracies when changing values ofσ\\sigma\. As the noise magnitude\|σ\|\|\\sigma\|increased, the performance of the proposed method also tended to decrease\. However, compared with the independent\-noise result in Table[4](https://arxiv.org/html/2609.10994#A4.T4), the degradation was overall more moderate \(e\.g\., on CIFAR10 \(S\) withσ=0\.195\\sigma=0\.195, the test accuracy is0\.82290\.8229in Table[5](https://arxiv.org/html/2609.10994#A4.T5), compared with0\.76110\.7611in Table[4](https://arxiv.org/html/2609.10994#A4.T4)\)\. A possible reason is that this noise shifts all class\-priors in the same direction and thus tends to preserve their relative relationships, leading to a milder degradation\.

Table 4:Results of the proposed method with independent noise in the class\-priors: average test accuracies over different UU data sizes in the test distribution\.σ\\sigmarepresents the magnitude of noise\.Table 5:Results of the proposed method with correlated noise in the class\-priors: average test accuracies over different UU data sizes in the test distribution\.\|σ\|\|\\sigma\|represents the magnitude of noise\.
### D\.7Results on Larger\-scale Image Datasets

Here, we evaluated the proposed method on larger versions of the image datasets used in the main paper \(i\.e\., MNIST, FMNIST, and CIFAR10\)\. For each image dataset, we set the training UU data size to\(Ntra,Ntrb\)=\(6,000,6,000\)\(N\_\{\{\\rm tr\}\}^\{\{\\rm a\}\},N\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\)=\(6,000,6,000\)and the test UU data size to\(Ntea,Nteb\)=\(100,100\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)=\(100,100\)\. Table[6](https://arxiv.org/html/2609.10994#A4.T6)shows the results\. The proposed method performed the best or comparably to it in almost all cases \(5 out of 6 cases\)\. The results show the effectiveness of the proposed method on the image datasets with increased data sizes\.

Table 6:Results on larger\-scale image data: average test accuracies over different class\-priors\(θa,θb\)\(\\theta^\{\{\\rm a\}\},\\theta^\{\{\\rm b\}\}\)within\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\}\\\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\\\}\. Values in bold are not statistically different at the5%5\\%level from the best performing method in each row according to a paired t\-test\.Table 7:Results on larger\-scale tabular data with distribution shift: average test accuracies over different class\-priors\(θa,θb\)\(\\theta^\{\{\\rm a\}\},\\theta^\{\{\\rm b\}\}\)within\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\}\\\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\\\}\. Values in bold are not statistically different at the5%5\\%level from the best performing method in each row according to a paired t\-test\.Table 8:Training time \[s\] of the proposed method on MNIST\(S\)\.Table 9:Average F1 scores over different UU test data sizes\(Ntea,Nteb\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)within\{\(50,50\),\(100,100\),\(150,150\)\}\\\{\(50,50\),\(100,100\),\(150,150\)\\\}and class\-priors\(θa,θb\)\(\\theta^\{\{\\rm a\}\},\\theta^\{\{\\rm b\}\}\)within\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\}\\\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\\\}\. In the Shift column, ‘S’ and ‘IO’ represent the support shift and input\-output relation shift, respectively\. Values in bold are not statistically different at the5%5\\%level from the best performing method in each row according to a paired t\-test\.
### D\.8Results on Larger\-scale Tabular Datasets with Real Distribution Shift

We evaluated the proposed method with larger\-scale tabular datasets with real distribution shift\. In this experiments, we newly used a FOODSTAMP, a real\-world tabular dataset used for recent distribution adaptation studies\[[12](https://arxiv.org/html/2609.10994#bib.bib12),[27](https://arxiv.org/html/2609.10994#bib.bib27)\]\. In this dataset, the task is to predict whether an individual is receiving food stamps\. The distribution shift occurs due to the difference of the geographic regions in which individuals live\. For FOODSTAMP and DIABETES datasets, we set the training UU data size\(Ntra,Ntrb\)=\(6,000,6,000\)\(N\_\{\{\\rm tr\}\}^\{\{\\rm a\}\},N\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\)=\(6,000,6,000\)and the test UU data size\(Ntea,Nteb\)=\(150,150\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)=\(150,150\)to evaluate the proposed method with a larger\-scale regime\. Table[7](https://arxiv.org/html/2609.10994#A4.T7)shows the results\. The proposed method remains competitive in such larger\-scale cases\.

### D\.9Computation Costs

We evaluated the training time of the proposed method on MNIST\(S\) with\(Ntea,Nteb\)=\(150,150\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)=\(150,150\)\. We used a Linux server with a 2\.20Hz CPU\. For comparison, we also evaluated the methods that use both UU data in the training and the test distributions as in the proposed method\. Table[8](https://arxiv.org/html/2609.10994#A4.T8)shows the results\. Since the proposed method estimated both importance weights and classifiers, it had slightly longer training time than mtsUU and mtUU\. However, the differences were not significant\. daUU had longer training time than the proposed method due to the calculation of the MMD loss to mitigate the feature discrepancy\. This result indicates that the proposed method is practical in terms of computation costs\.

### D\.10F1 Scores

Although we used test accuracies as the primary evaluation metric in the main paper, alternative metrics, such as F1 scores, can also be informative\. Table[9](https://arxiv.org/html/2609.10994#A4.T9)shows the average F1 scores for each dataset under the setting where data of both the training and test distributions are noisy\. The proposed method also performed well in terms of the F1 scores\.

### D\.11Full Results with Standard Deviations

Tables[10](https://arxiv.org/html/2609.10994#A4.T10)and[11](https://arxiv.org/html/2609.10994#A4.T11)show the average test accuracies with their standard deviations of each method\.

Table 10:Average test accuracies with standard deviations over different UU test data sizes\(Ntea,Nteb\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)within\{\(50,50\),\(100,100\),\(150,150\)\}\\\{\(50,50\),\(100,100\),\(150,150\)\\\}and class\-priors\(θa,θb\)\(\\theta^\{\{\\rm a\}\},\\theta^\{\{\\rm b\}\}\)within\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\}\\\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\\\}\. In the Shift column, ‘S’ and ‘IO’ represent the support shift and input\-output relation shift, respectively\. Values in bold are not statistically different at the5%5\\%level from the best performing method in each row according to a paired t\-test\.Table 11:Results with a few PN data in the test distribution: Average test accuracies with standard deviations over different UU test data sizes\(Ntea,Nteb\)\(N\_\{\{\\rm te\}\}^\{\{\\rm a\}\},N\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)within\{\(10,10\),\(50,50\),\(100,100\)\}\\\{\(10,10\),\(50,50\),\(100,100\)\\\}and training class\-priors\(θtra,θtrb\)\(\\theta\_\{\{\\rm tr\}\}^\{\{\\rm a\}\},\\theta\_\{\{\\rm tr\}\}^\{\{\\rm b\}\}\)within\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\}\\\{\(0\.8,0\.2\),\(0\.7,0\.3\),\(0\.6,0\.4\)\\\}\. Class\-priors of UU test data\(θtea,θteb\)\(\\theta\_\{\{\\rm te\}\}^\{\{\\rm a\}\},\\theta\_\{\{\\rm te\}\}^\{\{\\rm b\}\}\)were set to\(1,0\)\(1,0\)\.